The importance of IeeePriorityTag with converged RDMA Switch Embedded Teaming

Posted by Didier Van Hoye on September 27, 2017
Share on Facebook0Share on Google+0Share on LinkedIn1Share on Reddit5Tweet about this on Twitter0
No ratings yet.

Introduction

If you read my blog on Switch Embedded Teaming with RDMA (for SMB Direct) you’ll notice that I set the -IeeePriorityTag to “On” on the vNICs that use DCB for QoS. This requires some explanation.

When you configure a Switch Embedded Teaming (SET) vSwitch and define one or more management OS vNICs on which you enable RDMA you will see that the SMB Direct traffic gets it priority tag set correctly. This always happens no matter what you set the -IeeePriorityTag option to. On or Off, it doesn’t make a difference. It works out of the box.

When you have other traffic that needs to be tagged, let’s say backup traffic over TCP/IP for example, not because you want to make it lossless via PFC but because you need the priority tag set to configure ETS (minimum QoS) you’ll notice different behavior. No matter how correctly you have configured your Windows DCB setting that traffic doesn’t seem to get tagged.

The lab setup

To explain what’s going on we’ll use a little lab setup to demonstrate a couple of things. This is our 2 node Windows Server 2016 lab clusters with a converged SET vSwitch. You can read more about such a setup in my blog post.

mapped RDMA vNIC to their respective RDMA pNIC

The DCB configuration is as follows on Node-A:

Don’t forget this needs to be done on Node-B as well, I hope that is evident.

What’s going on?

So, what’s going on here? Well, the traffic on the host does get tagged as configured by DCB actually. It’s when we get to the vSwitch it becomes interesting.

Let’s start out with the RDMA vNICs were we have not set the -IeeePriorityTag to “On”. SMB Direct traffic on the vNIC bypasses the vSwitch by design and it works as you would expect with or without the -IeeePriorityTag set to “On”.

Things are different for non-SMB Direct traffic. If you do not set the -IeeePriorityTag to “On” the priority value is reset to 0. Note that this is not untagged traffic, the VLAN tag is still there, but the priority is set to 0. Below you can see TCP/IP traffic over a tagged vNIC that gets its priority set to 1 but when it hits the vSwitch it’s reset to 0.

TCP IP traffic over a tagged vNIC

If you do set -IeeePriorityTag to “On” the priority tagged packets will go by unchanged.

To do so we run:

set -IeeePriorityTag to “On” the priority tagged packets

Do remember to do this on all hosts for this to work in all directions. Below you can see me sending traffic from Node-A now with its priority tag 1 intact instead of being reset to 0.

sending traffic from Node-A now with its priority tag 1 intact

When we now run priority tagged TCP/IP traffic and SMB Direct traffic from node A to node B we see they both get their minimum bandwidth as defined by the DCB/ETS settings. 9% for TCP/IP, 90% for SMB Direct (1% is the remainder, the default).

run priority tagged TCP IP traffic and SMB Direct traffic from node A to node B

Conclusion

Unless you really don’t care about anything else than SMB Direct traffic to be tagged with a QoS priority you’ll need to enable IeeeTagging on your vNIC. If you do not do so, you cannot avoid the priority tag for non-SMB Direct traffic being set to 0.  Which means that you cannot configure PFC and ETS for that traffic. This is something to keep in mind. The fact that the RDMA traffic on a vNIC actually bypasses the switch by default and as such won’t have its priority tag set to 0 confused me initially as I figured the priority tag for DCB would not be removed for any type of traffic. That’s before I found out that I indeed still needed to turn on the IeeePriorityTag on those vNICs. In reality, it’s not different from how we deal with classification and tagging for QoS in the management OS and trusted virtual machine without hardware-based QoS via hardware (NICs & switches via DCB with PFC/ETS) being involved.

Related materials:

Views All Time
19
Views Today
22

Please rate this

To download the software products, please, make your choice below. An installer link and a license key will be sent to the e-mail address you’ve specified. If you consider StarWind Virtual SAN but are uncertain of the version, please check the following document Free vs. Paid. The recent build of Release Notes. A totally unrestricted NFR (Not For Resale) version of StarWind Virtual SAN is available for certain use cases. Learn more details here.



Return to all posts

What is Veeam Powered Network (VeeamPN) and why you need it?
Hyper-V VMs on an NFS share on Windows Server 2016 – is that real?
The following two tabs change content below.
Didier Van Hoye
Didier Van Hoye
Didier Van Hoye is an IT veteran with over 17 years of expertise in Microsoft technologies, storage, virtualization and networking. He works mainly as a subject matter expert advisor and infrastructure architect in Wintel environments leveraging DELL hardware to build the best possible high performance solutions with great value for money. He contributes his experience and knowledge to the global community as Microsoft MVP in Hyper-V, a Veeam Vanguard, a member of the Microsoft Extended Experts Team in Belgium and a DELL TechCenter Rockstar. He does so as a blogger, author, presenter and public speaker