VSAN cluster with single network backed by multiple physical NICs
Posted: Fri Feb 16, 2024 5:00 pm
Hello,
I'm pretty sure our scenario is not supported for VSAN (Free, as a start) but I'm just trying to be sure.
We have several Hyper-V hosts (WS 2022), some are running as standalone hosts and several S2D clusters. Our hosts, for technical reasons, usual have 4x10+Gbps NICs, two of them connected to what is DC public network while 2 of them connected to the private network so they are redundant. Again, for technical reasons, such NICs are usually teamed (we're aware about SET, we cannot use that). Our VMs are usually connected to what is our private network for their public traffic, again for technical reasons. So each network is backed by two redundant physical NICs connected to different uplinks. We can easily aggregate bandwidth that way but, again, we teamed - instead of using SET - because of specific restrictions.
We run several guest clusters and so far we had no real issues with this setup. We have several S2D guest clusters that basically had been running this setup for years but we also have different kind of clusters (GlusterFS, Galera etc.). When appropriate, we also run 2-nodes clusters (for ex. S2D), when we want to ensure better resiliency we run 3+ nodes clusters. So far so good. We usually use the other physical network for heartbeats, replication (for ex. Hyper-V replica), backups and so on. However, from a technical point of view, even if we usually configure 2 networks for clusterized VMs, such network mostly run onto the same physical (and teamed) network which can grow up to 50Gbps. In facts, our NIC support 50Gbps uplink though we have allocated usually 2x10Gbps.
While generally speaking we have no issues, there are a couple of scenarios where we run specific PHP applications on Windows Server and such applications - since 2-3 years - don't seem to deal with Windows Server clusters in a good way. Not sure why, because they were very fast up until 2-3 years ago while now PHP on Windows has very bad performances. However, we need to run such applications on Windows (so please no "switch to Linux" suggestions. Thanks!
We both tried to setup a S2D hyper-converged guest cluster for each of them or connect them (actually, the guest OS) to HA-SMB storage clusters. While clusters run fast, data transfers are very good especially when using multi-path, they exhibit different issues. In case of plain S2D hyper-converged setup, PHP seems to be unable to deal with its temp files written onto the cluster (hence, we lose sessions). When connected to HA-SMB clusters, they have very poor performances while the cluster and the guest OS are very fast.
Congratulations if you are still reading! Our goal would be getting rid of S2D or SMB on such guests because there is clearly some kind of issue between PHP and such network paths/disks so we are exploring the chance to use VSan that we already used in the pre-S2D era and we were very happy with that. At that time, we were using them to build a WS failover cluster for our VMs.
So my question is: I guess our scenario is absolutely not supported for VSan. Is that confirmed? Basically, we would need to create 2 logical networks in the guest OS to run sync and traffic (and heartbeats) but such networks would be backed by that aforementioned single physical network, or better by 2 teamed NICs on our hosts which provide failover in case of issues. I seem to remember, though, that VSan needs at least 2 physical networks, 1xtraffic/hb + 1sync and other scenarios are not supported at all.
Is this still the case or would our double uplink be enough to run both networks in a supported - or at least resilient - way?
Thanks a lot!
Thanks,
Andrea
I'm pretty sure our scenario is not supported for VSAN (Free, as a start) but I'm just trying to be sure.
We have several Hyper-V hosts (WS 2022), some are running as standalone hosts and several S2D clusters. Our hosts, for technical reasons, usual have 4x10+Gbps NICs, two of them connected to what is DC public network while 2 of them connected to the private network so they are redundant. Again, for technical reasons, such NICs are usually teamed (we're aware about SET, we cannot use that). Our VMs are usually connected to what is our private network for their public traffic, again for technical reasons. So each network is backed by two redundant physical NICs connected to different uplinks. We can easily aggregate bandwidth that way but, again, we teamed - instead of using SET - because of specific restrictions.
We run several guest clusters and so far we had no real issues with this setup. We have several S2D guest clusters that basically had been running this setup for years but we also have different kind of clusters (GlusterFS, Galera etc.). When appropriate, we also run 2-nodes clusters (for ex. S2D), when we want to ensure better resiliency we run 3+ nodes clusters. So far so good. We usually use the other physical network for heartbeats, replication (for ex. Hyper-V replica), backups and so on. However, from a technical point of view, even if we usually configure 2 networks for clusterized VMs, such network mostly run onto the same physical (and teamed) network which can grow up to 50Gbps. In facts, our NIC support 50Gbps uplink though we have allocated usually 2x10Gbps.
While generally speaking we have no issues, there are a couple of scenarios where we run specific PHP applications on Windows Server and such applications - since 2-3 years - don't seem to deal with Windows Server clusters in a good way. Not sure why, because they were very fast up until 2-3 years ago while now PHP on Windows has very bad performances. However, we need to run such applications on Windows (so please no "switch to Linux" suggestions. Thanks!
We both tried to setup a S2D hyper-converged guest cluster for each of them or connect them (actually, the guest OS) to HA-SMB storage clusters. While clusters run fast, data transfers are very good especially when using multi-path, they exhibit different issues. In case of plain S2D hyper-converged setup, PHP seems to be unable to deal with its temp files written onto the cluster (hence, we lose sessions). When connected to HA-SMB clusters, they have very poor performances while the cluster and the guest OS are very fast.
Congratulations if you are still reading! Our goal would be getting rid of S2D or SMB on such guests because there is clearly some kind of issue between PHP and such network paths/disks so we are exploring the chance to use VSan that we already used in the pre-S2D era and we were very happy with that. At that time, we were using them to build a WS failover cluster for our VMs.
So my question is: I guess our scenario is absolutely not supported for VSan. Is that confirmed? Basically, we would need to create 2 logical networks in the guest OS to run sync and traffic (and heartbeats) but such networks would be backed by that aforementioned single physical network, or better by 2 teamed NICs on our hosts which provide failover in case of issues. I seem to remember, though, that VSan needs at least 2 physical networks, 1xtraffic/hb + 1sync and other scenarios are not supported at all.
Is this still the case or would our double uplink be enough to run both networks in a supported - or at least resilient - way?
Thanks a lot!
Thanks,
Andrea