I/O data throughput limits

Numerous performance optimizations raise the question of the maximum possible data throughput. In this short article we show you how to localize the bottleneck by analyzing all connections in the I/O path using a diagram based on a Supermicro X11DPi-NT motherboard (for Intel Xeon Scalable Processors).

Example Server with SAS expander backplane

The following example shows an example server with SAS expander backplane:

• In this example, the SAS expander backplane is connected to a Broadcom 9300-8i via 2 SAS 4-port connections.
• The HBA is connected to the mainboard via a PCIe 3.0 x8 interface. The PCIe 3.0 x8 interface offers up to 7,880 MB/s. In this application it represents the limit for the maximum possible data throughput.
• The SAS expander backplane offers a high degree of flexibility with regard to the ongoing expansion of storage devices - in this example up to 24 SSDs or HDDs. With such a full assembly, the data throughput is limited by the shared PCI 3.0 x8 connection:

SAS expander backplane effects

The SAS expander chip of a corresponding backplane is located in the I/O path between CPU and SSDs/HDDs. It thus causes a slightly increased latency and also influences the possible data throughput per storage medium. Details can be found in the article SAS expander backplane performance.

Example Adaptec RAID controller

When using an Adaptec RAID controller, the Adaptec arcconf CLI shows the available host bus speed (7,880 Mbps):

root@:/usr/home/dg # ./arcconf getconfig 1 | grep -i Host\ Bus
   Host bus type                            : PCIe 3.0
   Host bus speed                           : 7880 MBps
   Host bus link width                      : 8 bit(s)/link(s)