Share on Facebook0Share on Google+0Share on LinkedIn0Share on Reddit6Tweet about this on Twitter0

Introduction

If you read my blog on Switch Embedded Teaming with RDMA (for SMB Direct) you’ll notice that I set the -IeeePriorityTag to “On” on the vNICs that use DCB for QoS. This requires some explanation.

When you configure a Switch Embedded Teaming (SET) vSwitch and define one or more management OS vNICs on which you enable RDMA you will see that the SMB Direct traffic gets it priority tag set correctly. This always happens no matter what you set the -IeeePriorityTag option to. On or Off, it doesn’t make a difference. It works out of the box.

When you have other traffic that needs to be tagged, let’s say backup traffic over TCP/IP for example, not because you want to make it lossless via PFC but because you need the priority tag set to configure ETS (minimum QoS) you’ll notice different behavior. No matter how correctly you have configured your Windows DCB setting that traffic doesn’t seem to get tagged.

The lab setup

To explain what’s going on we’ll use a little lab setup to demonstrate a couple of things. This is our 2 node Windows Server 2016 lab clusters with a converged SET vSwitch. You can read more about such a setup in my blog post.

mapped RDMA vNIC to their respective RDMA pNIC

The DCB configuration is as follows on Node-A:

Don’t forget this needs to be done on Node-B as well, I hope that is evident.

StarWind Virtual SAN eliminates any need for physical shared storage just by mirroring internal flash and storage resources between hypervisor servers. Furthermore, the solution can be run on the off-the-shelf hardware. Such design allows StarWind Virtual SAN to not only achieve high performance and efficient hardware utilization but also reduce operational and capital expenses.
Learn more about ➡ StarWind Virtual SAN.

What’s going on?

So, what’s going on here? Well, the traffic on the host does get tagged as configured by DCB actually. It’s when we get to the vSwitch it becomes interesting.

Let’s start out with the RDMA vNICs were we have not set the -IeeePriorityTag to “On”. SMB Direct traffic on the vNIC bypasses the vSwitch by design and it works as you would expect with or without the -IeeePriorityTag set to “On”.

Things are different for non-SMB Direct traffic. If you do not set the -IeeePriorityTag to “On” the priority value is reset to 0. Note that this is not untagged traffic, the VLAN tag is still there, but the priority is set to 0. Below you can see TCP/IP traffic over a tagged vNIC that gets its priority set to 1 but when it hits the vSwitch it’s reset to 0.

TCP IP traffic over a tagged vNIC

If you do set -IeeePriorityTag to “On” the priority tagged packets will go by unchanged.

To do so we run:

set -IeeePriorityTag to “On” the priority tagged packets

Do remember to do this on all hosts for this to work in all directions. Below you can see me sending traffic from Node-A now with its priority tag 1 intact instead of being reset to 0.

sending traffic from Node-A now with its priority tag 1 intact

When we now run priority tagged TCP/IP traffic and SMB Direct traffic from node A to node B we see they both get their minimum bandwidth as defined by the DCB/ETS settings. 9% for TCP/IP, 90% for SMB Direct (1% is the remainder, the default).

run priority tagged TCP IP traffic and SMB Direct traffic from node A to node B

Conclusion

Unless you really don’t care about anything else than SMB Direct traffic to be tagged with a QoS priority you’ll need to enable IeeeTagging on your vNIC. If you do not do so, you cannot avoid the priority tag for non-SMB Direct traffic being set to 0.  Which means that you cannot configure PFC and ETS for that traffic. This is something to keep in mind. The fact that the RDMA traffic on a vNIC actually bypasses the switch by default and as such won’t have its priority tag set to 0 confused me initially as I figured the priority tag for DCB would not be removed for any type of traffic. That’s before I found out that I indeed still needed to turn on the IeeePriorityTag on those vNICs. In reality, it’s not different from how we deal with classification and tagging for QoS in the management OS and trusted virtual machine without hardware-based QoS via hardware (NICs & switches via DCB with PFC/ETS) being involved.

Views All Time
1
Views Today
9
Appreciate how useful this article was to you?
1 vote, average: 5.00 out of 51 vote, average: 5.00 out of 51 vote, average: 5.00 out of 51 vote, average: 5.00 out of 51 vote, average: 5.00 out of 5
5 out of 5, based on 1 review
Loading...
Back to blog
The following two tabs change content below.
Didier Van Hoye
Didier Van Hoye
Didier Van Hoye is an IT veteran with over 17 years of expertise in Microsoft technologies, storage, virtualization and networking. He works mainly as a subject matter expert advisor and infrastructure architect in Wintel environments leveraging DELL hardware to build the best possible high performance solutions with great value for money. He contributes his experience and knowledge to the global community as Microsoft MVP in Hyper-V, a Veeam Vanguard, a member of the Microsoft Extended Experts Team in Belgium and a DELL TechCenter Rockstar. He does so as a blogger, author, presenter and public speaker