A side effect of the massive server workload consolidation enabled by hypervisor-based virtualization has been the addition of 7 to 16 I/O ports per machine to handle the I/O requirements of the hosted apps. But, before the data can be sent to and from storage devices, virtualization administrators often confront a more challenging problem that some have taken to calling the I/O Blender Effect. The I/O Blender effect is seen when multiple virtual machines send their I/O streams at the same time to a hypervisor for processing, increasing random accesses and increasing latency. Some software-defined storage (SDS) architectures are actually making the problem worse by caching raw small logical block writes using flash memory devices, leading to accelerated wear of the device and contributing virtually no improvement in storage I/O performance.
The solution isn’t bigger or faster flash memories, but rather a log-structured file system that better organizes, writes and enables a return to sequential accesses. Don’t look for log structuring technology (yet) in the SDS offerings of leading hypervisor vendors – it is still on their product roadmaps. To get it today, you need to look at SDS innovators like StarWind Software whose approach to virtual SAN enables small writes to coalesce before writing them to flash-based caches and that enables most read requests to be handled from the cache directly.