Starwind VSAN 2 node cluster using ESXi

Software-based VM-centric and flash-friendly VM storage + free version

Moderators: art (staff), anton (staff), Anatoly (staff), Max (staff)

Post Reply
jmchristy
Posts: 37
Joined: Thu Mar 15, 2012 2:55 pm

Wed Oct 22, 2014 12:17 pm

I'm trying to setup and configure the Starwind vSAN on an ESXi cluster following this technical whitepaper.
http://www.starwindsoftware.com/starwin ... re-vsphere

My goal is to utilize the local storage that is on my ESXi hosts in a 2 node cluster, along with my physical 2 node Starwind HA SANs. I am, for the most part, setting up the environment exactly as described in the document with the exception of a few IP addresses.

I had an existing HA infrastructure that I was taking offline, so I figured it would be a good testing bed to give this a try. I attached a quick rough diagram of the setup (allot is going on). I also attached the networking setup of one of the ESXi hosts.

Everything appears to function as it should, until I get to page 33 of the PDF with adding my dynamic IPs and scanning for the storage. I add the first Starwind node (SW1) from the virtual local network of 172.16.127.11. I add another IP address of the second Starwind node (SW2) from networks SYNC1 and SYNC2 (172.16.210.125, 172.16.211.125). I click OK, and rescan but my Starwind devices on my virtual starwind adapters do not appear.

Any clue as to what I'm missing? Or is this an unsupported config?

Another thing I noticed as I was troubleshooting - is that in Starwind v8 under Tools there is a Deploy VSAN option. Should I be using that instead?
Attachments
vSAN config on ESXI host
vSAN config on ESXI host
esxi vsan network config.PNG (18.38 KiB) Viewed 6199 times
vSAN topology w/Starwind SAN
vSAN topology w/Starwind SAN
vSAN setup.jpg (81.99 KiB) Viewed 6199 times
thefinkster
Posts: 46
Joined: Thu Sep 18, 2014 7:15 pm

Wed Oct 22, 2014 7:15 pm

I assume you can ping all the interfaces on all the systems for the vSAN and virtual Sync networks?

Does the StarWind console show you the iSCSI sessions (that are sessions for the other node(s)) for the Target between both Virtual SW nodes?

Can you use Windows iSCSI initiator to see the targets (just to make sure it's not a VMWare configuration issue)?

Are you having any issues accessing your physical StarWind nodes (again to make sure VMWare is working)?
jmchristy
Posts: 37
Joined: Thu Mar 15, 2012 2:55 pm

Thu Oct 23, 2014 11:33 am

thefinkster wrote:I assume you can ping all the interfaces on all the systems for the vSAN and virtual Sync networks?
Here is my list of IP addresses related to the vSAN targets. From SW1 which is hosted on ESXi 1, I can ping every IP address except for the iSCSI vSAN on ESXi 2 or the iSCSI vSAN on SW2. The same thing happens on SW2, I can ping any IP address except for the iSCSI vSAN on ESXi 1 or the iSCSI vSAN on SW1. I imagine that's by design since the virtual adapter of each virtual SW node resides on a separate physical host with no physical adapter bound to it, it would have no way of communicating with it...correct? I also did verify (following pages 33, 34 from the PDF attached below) that from the ESXi Host I could ping the iSCSI vSAN of SW1 and also ping the 2 iSCSI SYNC channels of SW2.

ESXi 1 vmkernels
SYNC 1 - 172.16.210.111
SYNC 2 - 172.16.211.111
iSCSI VSAN - 172.16.127.111

ESXi 2 vmkernels
SYNC 1 - 172.16.210.122
SYNC 2 - 172.16.211.122
iSCSI vSAN - 172.16.127.122

SW1
SYNC channel 1 - 172.16.210.11
SYNC channel 2 - 172.16.211.11
iSCSI vSAN - 172.16.127.11

SW2
SYNC channel 1 - 172.16.210.125
SYNC channel 2 - 172.16.211.125
iSCSI vSAN - 172.16.127.22
thefinkster wrote:Does the StarWind console show you the iSCSI sessions (that are sessions for the other node(s)) for the Target between both Virtual SW nodes?
Yes, I have iSCSI sessions on either virtual SW node for my virtual SAN. Each SW node can see each other, and they synchronize the HA target when I created it through the replication manager.
thefinkster wrote:Can you use Windows iSCSI initiator to see the targets (just to make sure it's not a VMWare configuration issue)?
When you say to use Windows iSCSI initiator, do you mean build up a seperate Hyper-V host and try and access it from there or just another virtual 2012 r2 virtual box?
thefinkster wrote:Are you having any issues accessing your physical StarWind nodes (again to make sure VMWare is working)?
I'm having no issues accessing my physical StarWind nodes, they are operational through VMware and currently running 6 virtual machines
jmchristy
Posts: 37
Joined: Thu Mar 15, 2012 2:55 pm

Thu Oct 23, 2014 2:54 pm

Ok - Here's some additional information that I have overlooked. It's not very clearly defined either in the guide that I put a link to in my original post.

Page 3 shows in the diagram that the 2 physical cables between ESXi hosts have 2 IP addresses in different subnets on them. When I originally created this setup, I set it up with one IP address at either host for the SYNC channel - thinking the 4th NIC that doesn't have any physical adapters bound to it would be what is used for the iSCSI Data traffic. I completely overlooked the 2 IP addresses in that diagram.

I'm only assuming, that you are supposed to create 2 VMKernels per physical adapter on your ESXi host - one for the SYNC Channel and one for the iSCSI data. So that is what I did, but after I do an iSCSI Software Adapter rescan it finds no additional devices. I'm relatively new to ESXi 5.5 and it's latest capabilities, but I thought the only way that you could have a device show up under the iSCSI Software Adapter was to bind a VMKernel port to the iSCSI software adapter. So that is what I did next, I added the VMKernel iSCSI VMKernels to the iSCSI Software Adapter and once I did that and performed a rescan - boom there's my device on the SW virtual nodes.

I'm not understanding what the point of the adapter that is added with no physical adapter bound to it - how does it locate the storage on the virtual SW nodes if it's not bound to the iSCSI Software Adapter? Also - one thing I noticed...when I do a ping from my virtual SW node to one of those adapters that has no physical adapter, it times out from time to time. I ping 172.16.127.111 from SW1 node, and it times out from time to time....responds...then times out.

I attached an updated diagram of my setup, the network configuration on one of my ESXi hosts, and the ping response time from my SW1 node that is timing out during a continuous ping.
Attachments
Ping response from SW1 node to iSCSI vSAN vmkernel on ESXi host
Ping response from SW1 node to iSCSI vSAN vmkernel on ESXi host
SW1 ping iSCSI vSAN vmkernel.PNG (56.26 KiB) Viewed 6175 times
ESXi setup v2
ESXi setup v2
vSAN esxi setup v2.PNG (18.78 KiB) Viewed 6175 times
vSAN setup v2
vSAN setup v2
vSAN setup v2.jpg (48.33 KiB) Viewed 6175 times
jmchristy
Posts: 37
Joined: Thu Mar 15, 2012 2:55 pm

Thu Oct 23, 2014 4:16 pm

I resolved the ping timeout issue in my last post by removing the virtual VMkernal iSCSI network adapter from the virtual machine, and re-adding it as a VMXNET3 adapter. I no longer have any ping timeouts on the 172.16.127.* subnet.

I still cannot add a device to the iSCSI target without first binding the 2 VMKernel ports w/physical adapters to the iSCSI Software Adapter first.
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Thu Oct 30, 2014 11:07 am

I’m afraid that you configuration is not fully clear for me.

Let me suggest you minimal configuration that should work like charm and it will be understandable to all of us:

ESX1
vmKernel1 - iSCSI 172.16.210.11
vmKernel2 - SYNC 172.16.110.11

VM1
iSCSI - 172.16.210.100
SYNC - 172.16.110.100


ESX2
vmKernel1 - iSCSI 172.16.210.22
vmKernel2 - SYNC 172.16.110.22

VM2
iSCSI - 172.16.210.200
SYNC - 172.16.110.200

Try to add iSCSI disk.

If everything works, then proceed with adding another one SYNC channel or whatever you want.
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
jmchristy
Posts: 37
Joined: Thu Mar 15, 2012 2:55 pm

Mon Nov 03, 2014 3:34 pm

Got it - I thought the same thing. So I scratched my entire setup and rebuilt it with your suggested setup below and it works now. I think I know why it wasn't working for me so my first question..

On the guide, page 5, it shows a virtual NIC with no physical adapters bound to it - what is that for? That's what was throwing me off, I thought I had to use that to access my virtual storage.

Also with the vSAN, is there a size limitation on your HA device? I thought I read that there wasn't, but my license indicates a size limit of 128GB. I may be using the wrong license file. I got my software/license file from this page.
http://www.starwindsoftware.com/registr ... irtual-san and selected "free" and "vmware" for my download.

Lastly related to performance...
Do the performance best practices for vSAN still apply as they did for the physical Starwind SAN servers? (i.e. setting the storage MPIO to Round Robin, iops equal to 1, Jumbo Frames 9000, etc.)

Appreciate your help!
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Wed Nov 12, 2014 9:04 pm

Also with the vSAN, is there a size limitation on your HA device?
Mostly we are limited by the maximum file size that vmfs allows. Here is what internet says about it:
VMFS3 is used by ESX Server v3.x and vSphere 4.x. Notably, it introduces directory structure in the filesystem. VMFS5 is used by vSphere 5.x. Notably, it raises the extent limit to 64 TB and the file size limit to 62 TB, though vSphere versions earlier than 5.5 are limited to VMDKs smaller than 2 TB.
I thought I read that there wasn't, but my license indicates a size limit of 128GB. I may be using the wrong license file.
Wrong license. Email to sales@ and say that you want the free license for vSphere deployments. That should do the job.
Do the performance best practices for vSAN still apply as they did for the physical Starwind SAN servers? (i.e. setting the storage MPIO to Round Robin, iops equal to 1, Jumbo Frames 9000, etc.)
Mostly. Usually I recommend to benchmark the system with default settings and see if it will work.
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
Post Reply