Data Locality

StarWind NVMe over Fabrics
(NVMe-oF)

In attempt to maximize VM performance, admins and business decision makers sometimes forget about smart hardware utilization. Especially, this question is critical for SMBs and ROBOs that often just cannot afford to build all-NVMe storages. In order to ensure that users get the required performance with minimum hardware footprint, StarWind added NVMe over Fabrics (NVMe-oF) support to StarWind Virtual SAN.

Problem

Even though the world is going crazy about NVMe, it is still challenging to present PCIe SSDs to the entire Hyper-V cluster effectively. Their latency shoots up, and applications are likely to access only half of flash performance when such disks are presented over the network. As a result, the true potential of PCIe SSDs cannot be unleashed. That’s, actually, why NVMe is not yet that popular among SMBs and ROBOs.

The problem is that Hyper-V VMs cannot talk effectively to PCIe SSDs. It is impossible to enjoy the NVMe drives’ performance if they are presented over iSCSI, iSER, SMB3, or NFS. These protocols were designed to connect slow spindle drives, not flash! They have single short command queue and entail significant I/O overhead once being used to connect to NVMe drives.

StarWind V2V Converter - Allows easily converting VM formats - pic 1
SAS: Performance is bottlenecked by single short command queue

All that being said, it seems obvious why your IOPS-hungry applications do not get the expected performance while talking to storage over the legacy protocols. Because of the protocols, adding flash to your setup usually grants you like 20% more IOPS than spindle drives can provide. But, obviously, it’s nothing compared to the true PCIe SSD performance! NVMe drives need a whole other tech to be presented over network…

Solution

NVMe-oF is the protocol that enables to achieve the peak NVMe performance. The single short command queue is replaced with 64 thousand queues, 64 thousand commands each. Such design enables NVMe devices connected over NVMe-oF to achieve consistent microsecond-scale latency and extract all the performance that flash can potentially provide. In our lab, we managed to get over 2M IOPS out of 4 Intel Optane SSD 900P drives in a bare metal environment, and we keep on going!

StarWind V2V Converter - Allows easily converting VM formats - pic 1
NVMe-oF: Multiple long command queues. Network is not a bottleneck anymore

StarWind brings NVMe-oF to Hyper-V environments. We have added this protocol support to StarWind Virtual SAN. Now, you can get all the IOPS that are claimed in your PCIe SSD vendor’s datasheet. With smart hardware utilization, you do not need to go all-flash to make VMs to run at full throttle.

Conclusion

When it comes to storage performance, NVMe drives are the true king of the hill. However, it is still challenging to get all PCIe SSD IOPS as traditional SCSI-derived protocols do not work that well for flash. With NVMe-oF, you can get all the performance that PCIe SSDs can provide. StarWind added NVMe-oF support to StarWind Virtual SAN so that, from now on, applications run on Hyper-V VMs can run at full throttle on flash storage.