StarWind NVMe over Fabrics (NVMe-oF) Initiator

Introduction to StarWind NVMe-oF Initiator

NVMe over Fabrics is a protocol that is set to be the standard for shared NVMe storage, which is much faster and more efficient than usual SAS. Moving things forward, StarWind Software designed NVMe over Fabrics  which can be connected on the host on demand and provide really fast NVMe storage for Windows Server. Any other NMVeoF target, like Linux Kernel or SPDK, can be connected with the help of StarWind NVMe-oF Initiator also.

This guide is intended for IT professionals and enthusiasts who would like to configure the StarWind Virtual SAN solution and NVMe over Fabrics. It provides detailed instructions on how to configure the Initiator and connect shared storage on Windows Server 2016.

A full set of up-to-date technical documentation can always be found here, or by pressing the Help button in StarWind Management Console.

For any technical inquiries, please, visit our online community, Frequently Asked Questions page, or use the support form to contact our technical support department.

VSAN from StarWind eliminates any need for physical shared storage just by mirroring internal flash and storage resources between hypervisor servers. Furthermore, the solution can be run on the off-the-shelf hardware. Such design allows VSAN from StarWind to not only achieve high performance and efficient hardware utilization but also reduce operational and capital expenses.
Find out more about ➡ VSAN from StarWind

Solution Diagram

Here is the network diagram for the configuration described in this guide.

StarWind NVMe-oF Initiator

 

Preconfiguring the Servers

In this document, the first host with Mellanox ConnectX-5 adapter and NVMe drive is running Windows Server 2016. The second host is running Windows Server 2016, too, and has Mellanox ConnectX-5 adapter installed accordingly. StarWind NVMe over Fabrics Initiator is deployed on the second Windows Server 2016 and connected to the first Windows Server 2106 via 172.16.77.x subnet.

Windows Servers 2016 should have the latest Mellanox driver installed, which is available here:

https://www.mellanox.com/products/adapter-software/ethernet/windows/winof-2

Network Connections

To check the RDMA connectivity and bandwidth between Windows Servers 2016, use the StarWind rPerf utility, which can be downloaded here: https://www.starwindsoftware.com/starwind-rperf

 

Installing StarWind NVMe over Fabrics Initiator

NOTE: To create the NVMe-oF target, please, follow the guide in the link below:

https://www.starwindsoftware.com/resource-library/starwind-virtual-san-creating-nvme-of

1. Download StarWind NVMeoF in thlink here:  https://www.starwindsoftware.com/starwind-nvme-of-initiator

2. Execute the starwind-nvmeof.exe  to install StarWind NVMe-oF Initiator and follow the steps in the wizard.

StarWind License
3. Open Device Manager to check that StarWind NVMe-oF Initiator is installed on the system.

NVMe-oF Starwind

Working with StarWind NVMe over Fabrics Initiator

The StarNVMeoF_Ctrl.exe command-line utility is used to work with the NVMe-oF initiator:

1. Find available NVMe-oF controllers and subsystems (targets). The command will show a list of names and properties of controllers for the address:

StarNVMeoF_Ctrl.exe discovery <target_ip_addr[:port]> <local_ip_addr>

2. Connect the NVMe-oF controller. The command will create a connection to the target.

StarNVMeoF_Ctrl.exe insert <target_ip_addr[:port]> <local_ip_addr> <SubNQN> <HostNQN> [<num_io_queues> <io_queue_depth>]

3. Show the list of connected NVMe-oF controllers. The command will show the list of connected controllers:

StarNVMeoF_Ctrl.exe list

4. Disconnect the NVMe-oF controller that is connected to the system. The command will disconnect the controller. The LUNs will be removed from the system:

StarNVMeoF_Ctrl.exe remove <controllerId>

Discovering and Connecting NVMe-over-Fabrics Targets on Windows Server 2016

1. To discover the target, run the discovery command: StarNVMeoF_Ctrl.exe discovery <target_ip_addr:[port]> <local_ip_addr> by specifying the protocol, target host IP address and port number:

Example: StarNVMeoF_Ctrl.exe discovery 172.16.77.1:4420 172.16.77.2

Where:

<172.16.77.1:4420> — NVMe-oF target host IP and port;
<172.16.77.2> — local host IP.

NVMe-oF Starwind

2. To connect the target, run the command: StarNVMeoF_Ctrl.exe insert <target_ip_addr[:port]> <local_ip_addr> <SubNQN> <HostNQN> [<num_io_queues> <io_queue_depth> <first_core>]

Example: StarNVMeoF_Ctrl.exe insert 172.16.77.1:4420 172.16.77.2 nqn.2008-08.com.starwindsoftware:NVMeofTarget SW_1 512 6 0

Where:

  • <172.16.77.1:4420 > — target host IP and port;
  • <172.16.77.2> — local host IP;
  • <nqn.2008-08.com.starwindsoftware:NVMeofTarget> — SubNQN of the target (may be copied from the discovery results screen);
  • <SW_1> — local HostNQN;
  • <512> — quantity of connections to the target;
  • <6> — queue depth;
  • <0> — number of the initial core.

NVMe-oF StarWind

3. After the “insert” command is executed, disk LUNs for the connected controller namespaces should appear in the system.

NVMe-oF StarWind

4. To show the list of connected NVMe-oF controllers, run the “StarNVMeoF_Ctrl.exe list” command.

NVMe-oF StarWind

5. To disconnect LUNs from the system, run the the controller disconnection command StarNVMeoF_Ctrl.exe remove <controllerId>
Example: StarNVMeoF_Ctrl.exe remove 1
Where:

  • <1> — controller Id

NVMe-oF StarWind

NOTE: Make sure that LUNs are not used by other applications at the moment of disconnection, as removing LUNs with active file operations may lead to data corruption.

   
Save your time finding all the answers to your questions in one place!
Have a question? Doubt something? Or just want to know an independent opinion?
StarWind Forum exists for you to “trust and verify” any issue already discussed and solved