Building stretched cluster enables to achieve higher applications and services availability run in it. Such setup also promotes uptime and allows implementing better disaster recovery strategies. This provides the detailed instructions on how to set up a 2-node Hyper-V stretched cluster on Windows Server 2016 with StarWind Virtual SAN as a storage provider.
StarWind Virtual SAN® is a native Windows hypervisor-centric hardware-agnostic VM storage solution. By mirroring the existing servers’ storage and RAM between the participating cluster nodes, it allows creating a fault-tolerant high-performing storage purpose-built for intensive virtualization workloads. All I/O is processed by local RAM, SSD cache, and disks. Thus, it never gets bottlenecked by the storage fabric. The mirrored storage is utilized by all cluster nodes and is treated by all hypervisors and clustered applications just as one big local storage. Running locally on the hypervisor, StarWind Virtual SAN delivers higher performance compared to any dedicated SAN solution. Also, the solution delivers High Availability thanks to providing multipath access to all storage nodes.
The diagram below illustrates the connection scheme of the StarWind stretched cluster configuration described in this guide.
Make sure that the prerequisites for deploying StarWind stretched Hyper-V cluster are met:
- Design L2/L3 multisite network according to the appropriate StarWind failover strategy.
- Each iSCSI and Synchronization network channel throughput should be at least 1Gbps
- Up to 25ms Round-trip time (RTT) network latency between locations.
- Deployed on-premises Active Directory structure and DNS.
- Windows Server 2016 installed on the server that is going to be clustered.
StarWind Failover Strategies
Before discussing StarWind failover strategies, it should be noted that the failover strategy can be chosen only during the device creation. It cannot be changed afterward. StarWind Virtual SAN provides two options: Heartbeat failover strategy and Node Majority failover strategy.
Heartbeat Failover Strategy
Heartbeat is a technology that allows avoiding the so-called “split-brain” scenario when the HA cluster nodes are unable to synchronize but continue to accept write commands from the initiators independently. It can occur when all synchronization and heartbeat channels disconnect simultaneously, and the other partner nodes do not respond to the node’s requests. As a result, StarWind service assumes the partner nodes to be offline and continues operations in a single-node mode using data written to it.
If at least one heartbeat link is online, StarWind services can communicate each other via this link. The device with the lowest priority will be marked as not synchronized one and gets subsequently blocked for the further read and write operations until the synchronization channel resumption. Then, the partner device on the synchronized node flushes data from the cache to the disk to preserve data integrity in case the node goes down unexpectedly. It is recommended to assign more independent heartbeat channels during replica creation to improve system stability and avoid the “split-brain” issue. With heartbeat failover strategy, the storage cluster will continue working with only one StarWind node available.
Heartbeat Failover Strategy Network Design
- Management / Heartbeat – 100Mbps network or higher.
- iSCSI / Heartbeat – 1Gbps network or higher. The 10Gbps or higher bandwidth link is highly recommended.
- Synchronization – 1Gbps network or higher. The 10Gbps or higher bandwidth link is highly recommended.
Node Majority Failover Strategy
This strategy ensures synchronization connection without any additional heartbeat links. The failure-handling process occurs when the node has detected the absence of connection with the partner. The main requirement for keeping the node operational is an active connection with more than a half of the HA device’s nodes. Calculation of the available partners is based on their “votes”. In case of a two-node HA storage, all nodes will be disconnected if there is a problem with the node itself, or with communication within the cluster. Therefore, the Node Majority failover strategy does not work in case of only two synchronous nodes. To resolve this issue, it is necessary to add the third entity. It can be a Witness node which participates the nodes count for the majority, but neither contains data on it nor is involved in processing clients’ requests.
With Node Majority failover strategy, failure of only one node can be tolerated. If two nodes fail, the third one will also become unavailable to clients’ requests. The Witness node should be additionally configured for an HA device that uses Node Majority failover strategy if it is replicated between 2 nodes. In case an HA device is replicated between 3 nodes, no Witness node is required.
Node Majority Failover Strategy Network Design
- Management / Heartbeat /Synchronization – 1Gbps network or higher. The 10Gbps or higher bandwidth link is highly recommended.
Preconfiguring the Servers
1. Make sure that a domain controller is present, and servers are added to the domain.
2. Install Failover Clustering and Multipath I/O features, as well as the Hyper-V role on both servers. This can be done through Server Manager (Add Roles and Features menu item).
3. Configure network interfaces on each node to make sure that Management/Live Migration and StarWind Synchronization interfaces are in different subnets and connected according to the network diagram above. In this document, 10.212.0.x subnet is used for Management/Live Migration while 10.212.1.x subnet is used for the Synchronization traffic.
4. In order to allow iSCSI Initiators discover all StarWind Virtual SAN interfaces, the StarWind configuration file (StarWind.cfg) should be changed after stopping the StarWind service on the node where it will be edited. Locate the StarWind Virtual SAN configuration file (the default path is “C:\Program Files\StarWind Software\StarWind\StarWind.cfg”) and open it with WordPad as Administrator. Find the <iScsiDiscoveryListInterfaces value=”0”/> string and change the value from 0 to 1 (should look as follows: <iScsiDiscoveryListInterfaces value=”1”/>). Save the changes and exit WordPad. Once StarWind.cfg is changed and saved, the StarWind service can be restarted.
NOTE: Additional network connections may be necessary, depending on the cluster setup and applications requirements. For any technical help with configuring the additional networks, please, do not hesitate to contact StarWind Support Department via online community forum, or via the support form (depends on the support plan).
Enabling Multipath Support
5. Open the MPIO Properties manager: Start -> Windows Administrative Tools -> MPIO. Alternatively, run the following PowerShell command:
6. In the Discover Multi-Paths tab, tick the Add support for iSCSI devices checkbox and click on Add.
7. When prompted to restart the server, click Yes to proceed.
NOTE: Repeat the procedure on the other server.
Downloading, Installing, and Registering the Software
8. Download the StarWind setup executable file from the official StarWind website:
NOTE: The setup file is the same for x86 and x64 systems, as well as for all Virtual SAN deployment scenarios.
9. Launch the downloaded setup file on the server to install StarWind Virtual SAN or one of its components. The Setup wizard will appear. Read and accept the License Agreement. Click Next to continue.
10. Carefully read the information about the new features and improvements. Red text indicates warnings for users that are updating the existing software installations. Click Next to continue.
11. Select Browse to modify the installation path if necessary. Click Next to continue.
12. Select the following components for the minimum setup:
- StarWind Virtual SAN Service. StarWind service is the “core” of the software. It allows creating iSCSI targets as well as share virtual and physical devices. The service can be managed from StarWind Management Console on any Windows computer or VSA that is on the same network. Alternatively, the service can be managed from StarWind Web Console deployed separately.
- StarWind Management Console. Management Console is the Graphic User Interface (GUI) part of the software that controls and monitors all storage-related operations (e.g., allows users to create targets and devices on StarWind Virtual SAN servers connected to the network).
Click Next to continue.
13. Specify Start Menu Folder. Click Next to continue.
14. Enable the checkbox if a desktop icon needs to be created. Click Next to continue.
15. When the license key prompt appears, choose the appropriate option:
- Request time-limited fully functional evaluation key
- Request FREE version key.
- Thank you, I do have a key already.
Click Next to continue.
16. Click on Browse to locate the license file. Press Next to continue.
17. Review the licensing information. Click Next to continue.
18. Verify the installation settings. Click Back to make any changes. Press Install to proceed with the installation.
19. Enable the appropriate checkbox to launch StarWind Management Console right after the setup wizard is closed. Click Finish to close the wizard.
20. Repeat the installation steps on the partner node.
NOTE: To manage StarWind Virtual SAN installed on a Windows Server Core edition without GUI, StarWind Management Console should be installed on a different computer running a GUI-enabled Windows edition.
Configuring Shared Storage
21. Double-click the StarWind tray icon to launch StarWind Management Console.
NOTE: StarWind Management Console cannot be installed on an operating system without GUI. It can be installed on any GUI-enabled Windows edition including Windows desktop versions.
If the StarWind VSAN service and StarWind Management Console are installed on the same server, Management Console will automatically add the local StarWind VSAN instance to the console tree after the first launch. Then, StarWind Management Console automatically connects to it using the default credentials. To add remote StarWind VSAN servers to the console, use the Add Server button on the control panel.
22. The StarWind Management Console will ask to specify the default storage pool on the server it connects to for the first time. Configure the default storage pool to use one of the volumes that have been prepared previously. All devices created through the Add Device wizard will be stored on it. In case an alternative storage path is required for StarWind virtual disks, use the Add Device (advanced) menu item.
Press the Yes button to configure the storage pool. If the storage pool destination needs to be changed, press Choose path… and point the browser to the necessary disk.
NOTE: Any of the arrays which will be used by StarWind Virtual SAN to store virtual disk images should meet the following requirements:
- Initialized as GPT.
- Have a single NTFS-formatted partition.
- Have a drive letter assigned.
The steps below cover the procedure of an HA device creation for the Witness drive. Other devices should be created in the same way.
23. Right-click on the Servers field and click on the Add Server button. Add a new StarWind Server which will be used as the second StarWind VSAN node.
24. Select the StarWind server where the device needs to be created and press the Add Device (advanced) button on the toolbar.
25. Add Device Wizard will appear. Select Hard Disk Device and click Next.
26. Select Virtual Disk and click Next.
27. Specify the virtual disk location, name, and size. Click Next.
28. Specify the virtual disk options and click Next.
29. Define the caching policy and specify the cache size (in GB). Click Next to continue.
NOTE: The basic recommendation is to assign 1 GB of L1 cache in Write-Back or Write-Through mode per 1 TB of storage capacity. Yet, the cache size should correspond with the storage working set of the servers.
30. Define Flash Cache Parameters and size if necessary. Choose SSD location in the wizard. Press Next.
NOTE: The recommended size of the L2 cache is 10% of the initial StarWind device capacity.
31. Specify the target parameters. Enable the Target Name checkbox to customize the target name. Otherwise, the name will be generated automatically based on the target alias. Click Next.
32. Click Create to add a new device and attach it to the target. Then, click Close.
33. Right-click on the newly created device and select Replication Manager. The Replication Manager window will appear. Press the Add Replica button.
34. Select Synchronous two-way replication. Click Next to proceed.
35. Specify the partner server IP address. The default StarWind management port is 3261. If a different port has been configured, type it in the Port Number field. Click Next to continue.
Heartbeat Failover Strategy
36. Check Heartbeat failover strategy according to the network design. Click Next to continue.
NOTE: For setting the Node Majority failover strategy, jump to the step #44.
37. Choose Create new Partner Device. Click Next.
38. Specify the partner device location if necessary and/or modify the target name of the device. Click Next.
39. Select the synchronization and heartbeat networks for the HA device by clicking Change Network Settings.
40. Specify the interfaces for synchronization and heartbeat. Press OK. Then, click Next.
NOTE: It is recommended to configure the Heartbeat and iSCSI channels on the same interfaces to avoid the split-brain issue. If the Synchronization and Heartbeat interfaces are located on the same network adapter, it is recommended to assign one more Heartbeat interface to a separate adapter.
41. Select Synchronize from existing Device for the partner device initialization mode.
42. Press the Create Replica button. Then, click Close.
43. The added device will appear in StarWind Management Console
Repeat the HA device creation steps for any virtual disks that will be further used as a Cluster Shared Volumes.
Once all devices are created, Management Console should look similar to the screenshot below.
Node Majority Failover Strategy
44. Check Node Majority Failover Strategy according to the network design. Click Next to continue.
45. Choose Create new Partner Device. Click Next.
46. Specify the partner device location if necessary and/or modify the target name of the device. Click Next.
47. Select the synchronization and heartbeat networks for the HA device by clicking Change Network Settings.
48. Specify the interfaces for synchronization and heartbeat. Press OK. Then, click Next.
49. Select Synchronize from existing Device for the partner device initialization mode. Click Next.
50. Press the Create Replica button. Then click Close.
51. The added device will appear in StarWind Management Console.
Repeat HA device creation steps for any virtual disks that will be further used as a Cluster Shared Volumes.
Once all devices are created, Management Console should look similar to the screenshot below.
Adding Witness Node
This section covers on adding a Witness node, the node participating nodes count for the majority, but neither containing data nor involved in clients requests processing.
Witness node should be configured in a separate location. There are two options to do so: it can either be a virtual machine run in the cloud or a host at another site. Witness node should have StarWind Virtual SAN service installed on it.
52. Open the StarWind Management Console, right-click on the Servers field and press the Add Server button. Add a new StarWind Server which will be used as the Witness node and click OK.
53. Right-click on the HA device with the configured Node Majority failover policy and select Replication Manager. The Replication Manager window will appear. Press the Add Replica button.
54. Select Witness Node and click Next
55. Specify the Witness node name or its IP address.
56. Specify the Witness device location and its target name if necessary. Click Next.
57. For the HA device, select the synchronization channel with the Witness node by clicking on the Change network settings button.
58. Specify the interface for synchronization, confirm, and click Next.
59. Click Create Replica, then close the Wizard by pressing the Close button.
60. Repeat the steps above to create other virtual disks.
61. Once all devices are created, StarWind Management Console should look as follows:
Discovering Target Portals
This part describes how to discover Target Portals on each StarWind node.
62. Launch Microsoft iSCSI Initiator: Start -> Windows Administrative Tools -> iSCSI Initiator. Alternatively, launch it typing the command below in the command line interface:
63. Navigate to the Discovery tab.
64. Click on the Discover Portal button. The Discover Target Portal dialog appears. Type 127.0.0.1.
65. Click on the Advanced button. Select Microsoft iSCSI Initiator as a Local adapter and select Initiator IP (leave default for 127.0.0.1). Confirm the actions to complete the Target Portal discovery.
66. Click the Discover Portal… button again.
67. In Discover Target Portal dialog, type in the iSCSI interface IP address of the partner node that will be used to connect the StarWind provisioned targets. Click Advanced.
68. Select Microsoft iSCSI Initiator as the Local adapter, select the Initiator IP in the same subnet as the IP address of the partner server from the previous step. Confirm the actions to complete the Target Portal discovery.
69. Now all target portals are added to the first node.
70. Go through Target Portals Discovery steps on the partner node.
71. The resulting Discovery tab on the partner node will look similar to the one shown above.
72. Click on the Targets tab. The previously created targets are listed in the Discovered Targets section.
NOTE: If the created targets are not listed, check the firewall settings of the StarWind Server as well as the list of networks served by the StarWind Server (go to StarWind Management Console -> Configuration -> Network). Alternatively, check the Access Rights tab on the corresponding StarWind VSAN server in StarWind Management Console for any restrictions implemented.
73. Select the Witness target from the local server and click on Connect.
74. Enable checkboxes as in the image below. Click Advanced.
75. Select Microsoft iSCSI Initiator in the Local adapter dropdown menu. In Target portal IP, select 127.0.0.1. Confirm the actions.
NOTE: It is recommended to connect the Witness device only by loopback (127.0.0.1) address. Do not connect the target to the Witness device from the partner StarWind node.
76. Select the CSV1 target discovered from the local server and click Connect.
77. Enable checkboxes as in the image below. Click Advanced.
78. Select Microsoft iSCSI Initiator in the Local adapter dropdown menu. In Target portal IP, select 127.0.0.1. Confirm the actions.
79. Select the partner target from the other StarWind node and click Connect.
80. Enable checkboxes as in the image below. Click Advanced.
81. Select Microsoft iSCSI Initiator in the Local adapter dropdown menu. In the Initiator IP field, select the IP address for the iSCSI channel. In the Target portal IP, select the corresponding portal IP from the same subnet. Confirm the actions.
82. Repeat the steps above for all remaining HA device targets. The result should look like in the screenshot below.
83. Repeat the steps described in this part on the other StarWind node, specifying corresponding local and data channel IP addresses. The result should look like in the screenshot below.
NOTE: It is recommended to configure the different MPIO policies depending on iSCSI channel throughput. For 1 Gbps iSCSI channel throughput, it is recommended to set Failover Only or Least Queue Depth MPIO load balancing policy. For 10 Gbps iSCSI channel throughput, it is recommended to set Round Robin or Least Queue Depth MPIO load balancing policy.
84. Configure the MPIO policy for each target except Witness with the load balance policy of choice. Select the target located on the local server and click Devices.
85. In the Devices dialog, click MPIO.
86. Select the appropriate load balancing policy.
87. For the Witness target, set the load balance policy to Failover Only.
88. Repeat the steps above for configuring the MPIO policy for each remaining device on the current node and on the partner node.
NOTE: In case the Failover Only MPIO policy is used, be sure to check that the local path (127.0.0.1) is set to Active, while the partner connection is set to Standby.
Connecting Disks to Servers
89. Open the Disk Management snap-in. The StarWind disks will appear as unallocated and will be offline.
90. Bring the disks online by right-clicking on them and selecting the Online menu option.
91. Select a disk other than Witness (check the disk size to be sure) and right-click on it to initialize.
92. By default, the system will offer to initialize all non-initialized disks. Use the Select Disks area to choose the disks. Select GPT (GUID Partition Style) for the partition style to be applied to the disks. Press OK to confirm.
93. Right-click on the selected disk and choose New Simple Volume.
94. In New Simple Volume Wizard, indicate the volume size. Click on the Next button.
95. Assign a drive letter to the disk. Click Next.
96. Select NTFS in the File System dropdown menu. Keep the Allocation unit size as Default. Set the Volume Label of choice. Click Next.
97. Press Finish to complete the operation.
98. Complete the steps above for the Witness disk. Do not assign any drive letter or drive path for it.
99. On the partner node, open the Disk Management snap-in. All StarWind disks will appear offline. If the status is different from the one shown in the screenshot below, click Refresh under Action in the top menu to update the information about the disks.
100. Bring all StarWind disks online as described above.
Creating a Cluster
NOTE: To avoid issues during the cluster validation configuration, it is recommended to install the latest Microsoft updates on each node.
101. Open Server Manager. Select the Failover Cluster Manager item from the Tools menu.
102. Click the Create Cluster link in the Actions section of the Failover Cluster Manager.
103. Specify the servers to be added to the cluster. Click Next to continue.
104. Validate the configuration by passing the cluster validation tests: select Yes… and click Next to continue.
105. Specify the cluster name.
NOTE: If the cluster servers get IP addresses over DHCP, the cluster also gets its IP address over DHCP, though this scenario is not recommended. If the IP addresses are set statically, setting the cluster IP address manually is required. Click Next to continue.
106. Make sure that all settings are correct. Click Previous to make any changes. Click Next to proceed.
NOTE: If the Add all eligible storage to the cluster checkbox is enabled, the wizard will add all disks to the cluster automatically. The smallest device will be assigned as the cluster witness disk. It is recommended to uncheck this checkbox before clicking Next and assign the cluster witness disk later manually.
107. Upon the cluster creation completion, the system displays a report with the detailed information that can be view by clicking on the View Report button. Alternatively, click Finish to close the wizard.
Adding Storage to the Cluster
Follow the steps below to add the Cluster Shared Volumes (CSV) that are necessary for working with Hyper-V virtual machines.
108. In the Failover Cluster Manager, navigate to Cluster -> Storage -> Disks. Click Add Disk in the Actions panel, choose StarWind disks from the list and confirm the selection.
109. To configure the cluster witness disk, right-click on Cluster and proceed to More Actions -> Configure Cluster Quorum Settings.
110. Follow the wizard and use the Select the quorum witness option. Click Next.
111. Select Configure a disk witness. Click Next.
112. Select the Witness disk to be assigned as the cluster witness disk. Click Next and press Finish to complete the operation.
113. In Failover Cluster Manager, select a disk. Right-click on it and select Add to Cluster Shared Volumes.
114. If changing the name of the cluster shared volume is required, right-click on the disk and select Properties. Type the new name for the disk and click Apply followed by OK.
115. Perform the steps above for any other disk in Failover Cluster Manager. The resulting list of disks will look similar to the screenshot below.
116. Once the disks are added to the cluster shared volumes and the network preferences are set, highly available virtual machines can be created on the cluster shared volumes. Select Roles and in the Action tab, click Virtual Machines -> New Virtual Machine. Complete the wizard.
NOTE: to avoid unnecessary CSV overhead, configure each CSV to be owned by the different cluster node. This node should also be the preferred owner of the VMs running on that CSV.
Building a stretched failover cluster increases the availability of services or applications running in it and excludes the downtime during maintenance. The process of setting up the cluster with StarWind Virtual SAN is similar to configuring any traditional HyperConverged scenario. This document also discusses failover strategies used by StarWind Virtual SAN to deliver the highest possible uptime and data security.