Intel VROC (VMD NVMe RAID) unter Linux

In diesem Artikel zeigen wir, wie Informationen zu einem konfigurierten RAID 1 eines Intel VROC (VMD NVMe RAID)s unter Ubuntu 20.04 mittels mdadm abgefragt werden können. Das RAID wurde in diesem Beispiel im BIOS eines Servers mit Intel Ice Lake CPU eingerichtet.

lspci

Die folgenden Ausgaben zeigen Informationen von lspci:

admin@ubuntu-20-04:~$ lspci -nn | grep -i raid
0000:97:00.5 RAID bus controller [0104]: Intel Corporation Volume Management Device NVMe RAID Controller [8086:28c0] (rev 04)
0000:e2:00.5 RAID bus controller [0104]: Intel Corporation Volume Management Device NVMe RAID Controller [8086:28c0] (rev 04)

admin@ubuntu-20-04:~$ lspci -nn | grep Non-Volatile
10000:01:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 [144d:a808]
10000:03:00.0 Non-Volatile memory controller [0108]: Device [1db2:2302] (rev 03)
10000:04:00.0 Non-Volatile memory controller [0108]: Device [1db2:2302] (rev 03)

admin@ubuntu-20-04:~$ sudo lspci -s 0000:97:00.5 -vvv -nn
0000:97:00.5 RAID bus controller [0104]: Intel Corporation Volume Management Device NVMe RAID Controller [8086:28c0] (rev 04)
        Subsystem: Intel Corporation Volume Management Device NVMe RAID Controller [8086:0000]
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        NUMA node: 1
        Region 0: Memory at 206ffc000000 (64-bit, prefetchable) [size=32M]
        Region 2: Memory at d8000000 (32-bit, non-prefetchable) [size=32M]
        Region 4: Memory at 206ffff00000 (64-bit, non-prefetchable) [size=1M]
        Capabilities: [80] MSI-X: Enable+ Count=64 Masked-
                Vector table: BAR=4 offset=00000000
                PBA: BAR=4 offset=00001000
        Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0
                        ExtTag- RBE- FLReset-
                DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis-, NROPrPrP-, LTR-
                         10BitTagComp-, 10BitTagReq-, OBFF Not Supported, ExtFmt-, EETLPPrefix-
                         EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
                         FRS-
                         AtomicOpsCap: 32bit- 64bit- 128bitCAS-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                         AtomicOpsCtl: ReqEn-
        Capabilities: [e0] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Kernel driver in use: vmd
        Kernel modules: vmd

admin@ubuntu-20-04:~$ sudo lspci -s 0000:e2:00.5 -vvv -nn
0000:e2:00.5 RAID bus controller [0104]: Intel Corporation Volume Management Device NVMe RAID Controller [8086:28c0] (rev 04)
        Subsystem: Intel Corporation Volume Management Device NVMe RAID Controller [8086:0000]
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        NUMA node: 1
        Region 0: Memory at 209ffc000000 (64-bit, prefetchable) [size=32M]
        Region 2: Memory at f8000000 (32-bit, non-prefetchable) [size=32M]
        Region 4: Memory at 209ffff00000 (64-bit, non-prefetchable) [size=1M]
        Capabilities: [80] MSI-X: Enable+ Count=64 Masked-
                Vector table: BAR=4 offset=00000000
                PBA: BAR=4 offset=00001000
        Capabilities: [90] Express (v2) Root Complex Integrated Endpoint, MSI 00
                DevCap: MaxPayload 128 bytes, PhantFunc 0
                        ExtTag- RBE- FLReset-
                DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
                        RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
                        MaxPayload 128 bytes, MaxReadReq 128 bytes
                DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
                DevCap2: Completion Timeout: Not Supported, TimeoutDis-, NROPrPrP-, LTR-
                         10BitTagComp-, 10BitTagReq-, OBFF Not Supported, ExtFmt-, EETLPPrefix-
                         EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
                         FRS-
                         AtomicOpsCap: 32bit- 64bit- 128bitCAS-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                         AtomicOpsCtl: ReqEn-
        Capabilities: [e0] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Kernel driver in use: vmd
        Kernel modules: vmd

md

Die folgenden Ausgaben zeigen mdadm und /proc/mdstat:

mdadm --detail-platform

admin@ubuntu-20-04:~$ sudo mdadm --detail-platform
mdadm: imsm capabilities not found for controller: /sys/devices/pci0000:00/0000:00:17.0 (type SATA)
mdadm: imsm capabilities not found for controller: /sys/devices/pci0000:00/0000:00:11.5 (type SATA)
       Platform : Intel(R) Virtual RAID on CPU
        Version : 7.6.0.1012
    RAID Levels : raid0 raid1 raid10 raid5
    Chunk Sizes : 4k 8k 16k 32k 64k 128k
    2TB volumes : supported
      2TB disks : supported
      Max Disks : 48
    Max Volumes : 2 per array, 24 per controller
 3rd party NVMe : supported
 I/O Controller : /sys/devices/pci0000:97/0000:97:00.5 (VMD)
 NVMe under VMD : /sys/devices/pci0000:97/0000:97:00.5/pci10000:00/10000:00:04.0/10000:03:00.0
 NVMe under VMD : /sys/devices/pci0000:97/0000:97:00.5/pci10000:00/10000:00:02.0/10000:01:00.0
 NVMe under VMD : /sys/devices/pci0000:97/0000:97:00.5/pci10000:00/10000:00:05.0/10000:04:00.0
 I/O Controller : /sys/devices/pci0000:e2/0000:e2:00.5 (VMD)

mdadm -D

admin@ubuntu-20-04:~$ sudo mdadm -D /dev/md/Volume0_0
/dev/md/Volume0_0:
         Container : /dev/md/imsm0, member 0
        Raid Level : raid1
        Array Size : 111357952 (106.20 GiB 114.03 GB)
     Used Dev Size : 111357952 (106.20 GiB 114.03 GB)
      Raid Devices : 2
     Total Devices : 2

             State : clean
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : resync


              UUID : 32026e61:d472715e:b654d740:30e648f4
    Number   Major   Minor   RaidDevice State
       1     259        1        0      active sync   /dev/nvme1n1
       0     259        0        1      active sync   /dev/nvme2n1

admin@ubuntu-20-04:~$ sudo mdadm -D /dev/md/imsm0
/dev/md/imsm0:
           Version : imsm
        Raid Level : container
     Total Devices : 2

   Working Devices : 2


              UUID : c0ea71b0:9c041a79:c15f4eb8:dfc49403
     Member Arrays : /dev/md/Volume0_0

    Number   Major   Minor   RaidDevice

       -     259        0        -        /dev/nvme2n1
       -     259        1        -        /dev/nvme1n1

cat /proc/mdstat

admin@ubuntu-20-04:~$ cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md126 : active raid1 nvme1n1[1] nvme2n1[0]
      111357952 blocks super external:/md127/0 [2/2] [UU]

md127 : inactive nvme2n1[1](S) nvme1n1[0](S)
      10402 blocks super external:imsm

unused devices: <none>

Weitere Informationen

• Intel Virtual RAID oc CPU (Intel VROC) - Linux Software User Guide (www.intel.com, June 2020)

Autor: Werner Fischer Werner Fischer arbeitet im Product Management Team von Thomas-Krenn. Er evaluiert dabei neueste Technologien und teilt sein Wissen in Fachartikeln, bei Konferenzen und im Thomas-Krenn Wiki. Bereits 2005 - ein Jahr nach seinem Abschluss des Studiums zu Computer- und Mediensicherheit an der FH Hagenberg - heuerte er beim bayerischen Server-Hersteller an. Als Öffi-Fan nutzt er gerne Bus & Bahn und genießt seinen morgendlichen Spaziergang ins Büro.