wikiVMware Performance Comparison SCSI Controller and NVMe Controller
Today, more and more workloads are running in virtual machines (VMs), including workloads that require significantly more IO in the guest operating system. In a VM on VMware vSphere, all virtual disks (VMDKs) are attached to the LSI Logical SAS SCSI Adapter in the default configuration. This adapter is recognized by all operating systems without installing additional drivers, but does not always provide the best performance, especially when an SSD RAID or NVMe Storage is used. In this article we have compared the virtual storage controllers LSI Logical SAS, VMware Paravirtual and the NVMe Controller.
Controller models
The standard controller in almost every VM is the LSI Logical SAS SCSI controller. This controller is recognized and supported by every guest operating system without additional drivers. It is suitable for almost any workload that does not have large I/O requirements. It is also necessary for the configuration of Microsoft Server Cluster Service (MSCS).
Starting with ESXi 4.0 and virtual hardware version 7, the VMware Paravirtual controller is available. This controller was developed for high performance storage systems, because it can handle much higher I/O and reduces the CPU load. In order for the controller to be used by the guest operating system, the VMware Tools must be installed.
Starting with ESXi 6.5 and virtual hardware version 13, an NVMe controller can also be added to the VM. This controller further optimizes the performance of SSD RAIDs, NVMe and PMEM storage. This Controller is the default Controller for Windows VMs in vSphere 7.0.
The choice of the right controller depends on the applications within the VM. For example, if it is an office VM, relatively little performance is required and the standard LSI Logical SAS SCSI controller can be used. If more storage performance is required within the VM and the storage system behind it also offers more performance, the VMware Paravirtual Controller is usually more suitable. For absolute high end performance when using an SSD RAID, NVMe or PMEM storage and very high performance requirements within the VM, the NVMe controller is the best choice.
Performance test
We have conducted various performance tests for different scenarios. The test scenarios are only examples, the individual values should be adjusted individually to the own workload to achieve realistic results. Details of the test system used:
Hardware / Software:
- Supermicro Mainboard X11DPi-NT
- 2x Intel Xeon Gold 5222 (3.80GHz, 4-Core, 16.5MB)
- 256GB ECC Registered (RDIMM) DDR4 2666 RAM 4 Rank
- 3.2 TB Samsung SSD NVMe PCI-E 3.0 (PM1725b)
- ESXi 6.7.0 Update 2 (Build 13981272)
Test VM
- Windows 10 Pro (18362)
- 2 CPU sockets
- 8 vCPUs
- 8GB RAM
- VMware Paravirtual
- LSI Logical SAS
- NVMe Controller
- Thick-Provisioned eager-zeroed VMDK
LSI Logical SAS
VMware Paravirtual
NVMe Controller
Performance Comparison
Database Server
| Database Server (8K Random; 70% Read; 8 Threads; 16 Outstanding IO) | |||||||
| IOPS | MByte/s | Latency (ms) | CPU (%) | ||||
|---|---|---|---|---|---|---|---|
| LSI Logical SAS | 78210.16 | 611.02 | 1.633 | 24.81 | |||
| VMware Paravirtual | 153723.45 | 1200.96 | 0,832 | 31.27 | |||
| NVMe Controller | 203612.54 | 1590.72 | 0,628 | 48.03 | |||
E-Mail-Server
| E-Mail-Server (4K Random; 60% Read; 8 Threads; 16 Outstanding IO) | |||||||
| IOPS | MByte/s | Latency (ms) | CPU (%) | ||||
|---|---|---|---|---|---|---|---|
| LSI Logical SAS | 83403.47 | 325,79 | 1.506 | 23.52 | |||
| VMware Paravirtual | 157624.97 | 615.72 | 0,811 | 31.46 | |||
| NVMe Controller | 236622.59 | 924.31 | 0,540 | 52.11 | |||
File-Server
| File-Server (64K Sequential; 90% Read; 8 Threads; 16 Outstanding IO) | |||||||
| IOPS | MByte/s | Latency (ms) | CPU (%) | ||||
|---|---|---|---|---|---|---|---|
| LSI Logical SAS | 44739.43 | 2796.21 | 2.860 | 12.29 | |||
| VMware Paravirtual | 53717.26 | 3357.33 | 2.382 | 16.87 | |||
| NVMe Controller | 48929.05 | 3058.07 | 2.615 | 14.14 | |||
Streaming-Server
| Streaming Server (5120K Random; 80% Read; 8 Threads; 16 Outstanding IO) | |||||||
| IOPS | MByte/s | Latency (ms) | CPU (%) | ||||
|---|---|---|---|---|---|---|---|
| LSI Logical SAS | 458.16 | 2290.81 | 279.607 | 2.18 | |||
| VMware Paravirtual | 504.22 | 2521.10 | 253.949 | 12.26 | |||
| NVMe Controller | 505.14 | 2525.68 | 253.659 | 1.56 | |||
VDI-Workload
| VDI-Workload (4K Random; 20% Read; 8 Threads; 8 Outstanding IO) | |||||||
| IOPS | MByte/s | Latency (ms) | CPU (%) | ||||
|---|---|---|---|---|---|---|---|
| LSI Logical SAS | 140155.89 | 547.48 | 0,456 | 35.69 | |||
| VMware Paravirtual | 163073.26 | 637.00 | 0,392 | 37,98 | |||
| NVMe Controller | 203464.89 | 794.78 | 0.314 | 49.55 | |||
Author: Sebastian Köbke
