This article is part two of a two-part series on configuring a highly available, on-premises SMTP relay solution. In this part, we walk through setting up the actual SMTP relay services. To learn more about the design and all the preparation work, please read part I.

Setting up anonymous SMTP

Log on to your virtual machine where you installed the SMTP Server feature in part one of this article, open the IIS 6 management console, and navigate to the default SMTP virtual server. Rename it from [SMTP Virtual Server #1] to Anonymous SMTP Relay.

Anonymous SMTP Relay

Now right-click on Anonymous SMTP Relay and select “Properties” to configure the virtual server. Below we will walk through the tabs and the different settings.


Select the IP address you want the anonymous SMTP relay service to listen on.

Anonymous SMTP Relay Properties

Click on the Properties button and configure your logging preference. I roll over logs daily and use local time. I also change the logging directory to the one we prepared earlier.

Configure your logging preference

Click the “Advanced” tab to configure logging properties. I select them all to have everything when needed.

Advanced Properties


Navigate to the “Access” tab and click on the “Authentication” button.

"Authentication" button

Check that only Anonymous access is enabled.

Anonymous access is enabled

Move on to the Connection button. Select “Only the list below” and add the IP addresses you want to allow to connect. You must include the real IP addresses of both LoadMaster nodes for health checking and the IP address of the SMTP relay VIP so that they can connect. Do not forget to add the actual IP address of the clients if you use transparency on the LoadMaster as then you will see those as the connecting IP addresses. I added and (VIP). You might also want to (temporarily) add the IP address from where you are testing SMTP if that is your workstation. In my case, this is


Click on the Relay button. Select “Only the list below” and add the IP addresses you want to allow to relay. Add the VIP address of the LoadMaster anonymous SMTP relay service so it can connect. Do not forget you must add the actual IP address of the clients if you use transparency on the LoadMaster as then you will see those as the connecting IP addresses. You can see I added the IP of my LoadMaster anonymous SMTP relay virtual service and that of my test workstation.

Relay Restrictions


Select the “Messages” tab and fill out the email address where you want the NDR sent. You must also change the Badmail directory to the custom path that you set up.

Fill out the email address where you want the NDR sent

Right now, you might think, great; we have changed the Badmail directory path, but what about the Drop, Pickup, and Queue directory paths? Well, we can change that in the Metabase.xml file or use PowerShell.

To edit the Metabase.xml, we need to check “Enable Direct Metabase Edit” under the properties of the server itself in IIS 6.0.

Enable Direct Metabase Edit

Then stop the Anonymous SMTP Relay site.

Open the MetaBase file located at C:\Windows\System32\inetsrv\MetaBase.xml with elevated credentials. Search for and update the paths to the “Badmail,” “Drop,” “Pickup,” and “Queue” directories. Save the Metabase.xml file.

"Queue" directories

Start the Anonymous SMTP Relay site. When done, you can uncheck “Enable Direct Metabase Edit” again.

But PowerShell is the winner here. A lot easier.

Now some things in IIS SMTP are very complicated to do with PowerShell. The Connection Settings are easily automated. Editing the Relay settings, for example, that comes down to a byte array structure with byte mapping/counting for IP addresses, network ranges, and FQDNs. Manipulating that byte array is not documented. So this route requires a lot of reverse engineering via empirical testing. Trust me, I tried. In the end, I resorted to VBScript for that part, Ouch!


On the “Delivery” tab, you have three buttons. For outbound security, select Basic Authentication. Use “apikey” for the username. For the password paste the API key you got when you created the API key for SMTP Relay in SendGrid. Remember that this key was only visible during the creation. You should have saved it somewhere in a secure way. Alternatively, you can create a separate one for each use case. Just give them a proper name in SendGrid. You must know what their use case is to keep your SendGrid environment tidy. You do not want it filled with phantom or unknown API keys, you no longer remember the use for, let alone where you used them.

Use "apikey" for the username

Make sure you select TLS encryption!

Under delivery, outbound connections, change the TCP port to 587.

TCP port to 587

Under delivery, advanced, you can set the FQDN to what you want to use. I leave it at the servers DNS name for anonymous SMTP relay. You must configure the smart host for SendGrid, which is

Test & verify

When you have done all the above, you should be able to test the anonymous SMTP relay via a script. You’ll need to change the variables to your specific values.

Finally, rinse and repeat for the second server in your load balanced, highly available design.

Setting up authenticated SMTP

Setting up an authenticated SMTP relay is, in many aspects, the same as setting up anonymous SMTP relay. So here we will focus on the differences. Do not forget the rest of the configuration that is the same as for anonymous SMTP, but remember to change values (paths to directories) where applicable.

Username and password

The first difference is that we need to create at least one local user account for basic authentication over TLS. This user requires no special privileges. Just create one with the same name and password on each server for the authenticated SMTP relay service. You do not need to do anything else. In this example, the user is called “MySmtpAuthUser” with “MySmtpAuthPassword” as a password.

You can create multiple accounts for different use cases so that you don’t have to share passwords. Do make sure you create the same combinations of usernames and passwords on both virtual machines.

General – Change the SMTP port for incoming mail to 587 (TLS)

On the “General” tab of the Authenticated SMTP Relay Properties, we click on Advanced and edit the TCP port to be 587, the default port for mail submission.

Change the SMTP port for incoming mail to 587 (TLS)

Access – Use TLS for secure communication and authentication

Navigate to the “Access” tab and click on the “Authentication” button. Uncheck Anonymous access. Check Basic authentication. Also, check Requires TLS encryption and fill out the FQDN for authenticated SMTP Relay. The one you used to register that virtual service on the LoadMaster in DNS.

Requires TLS encryption

When you have installed a proper TLS certificate, it shows up under “Secure communication”. You must select Require TLS encryption.

Test your configuration

The rest of the settings are precisely like for anonymous SMTP relay. So you should look there for guidance. When done, test your setup to verify it works.

Don’t forget, do all the above on both virtual machines in your load balanced, highly available design.

Tips from the field

Test your failovers

When you have configured both servers completely, test failover scenarios. Stop a virtual machine, failover a LoadMaster node. Get a feel for its behavior and adapt your LoadMaster configuration to fit your needs. If something does not work, test your setup and verify it works.

Certificate Validation

For basic authentication with TLS, you need a valid certificate. The certificate can be a commercial one, one from your own PKI, or a self-signed one. Many appliances and software don’t check validity or, perhaps, more accurately put, they ignore certificate validation errors. That means that a self-signed certificate might work in one device or software but not with another.

The solution is to make sure the certificate has a valid certificate chain where ever it is consumed. For Windows, this means we add the required intermediate and root certificates to their respective local machine certificate stores. In the case of a self-signed certificate, we have to add it to the Trusted Root Certification Authorities store for the local machine. The ability to upload and configure intermediate and root certificates will differ with the devices and software.

Let’s look at an example. Our backup software, running on Windows, successfully sends emails to our authenticated SMTP relay service with the FQDN or the IP address. The fact that it cannot validate the certificate chain does not matter. With a PowerShell test script on the same Windows host, however, we do run into an issue with this.

It throws an error: “The remote certificate is invalid according to the validation procedure.”

The remote certificate is invalid - img

For this to work, with any certificate (self-signed, own PKI, or commercial), we have a couple of options. The first is to make sure the root certificate and intermediate certificates are available in their respective local machine certificate stores. Once that is done, our PowerShell script runs fine as well. Another approach is to ignore the certificate validation errors. In PowerShell for windows, you can do this via

It also means that you can now use an IP address instead of an FQDN matching what is in the certificate. For testing where you do not have name resolution and cannot use a host file, this comes in handy. Use to

to undo this setting.

Remember that when you use whitelisting, the IP off the host where you run the test script must be in there.

Let’s Encrypt

While “Let’s Encrypt” is popular, automated certificate management can be a challenge. Some devices, appliances, and software do not provide much in terms of automation options, or this is not a readily available skill. So, for this reason, I would not opt for them here. Not unless you have automated certificate renewal entirely worked out for all clients and devices, which potentially goes way beyond the two SMTP relay services on your virtual machines. If you have all that under control, by all means, use “Let’s Encrypt”.

Forgot the SendGrid API key used in IIS SMTP?

Forgot the SMTP password (the API key you got from SendGrid), but had it configured in IIS SMTP already? Grab it with PowerShell!

VSAN from StarWind is software-defined storage (SDS) solution created with restricted budgets and maximum output in mind. It pulls close to 100% of IOPS from existing hardware, ensures high uptime and fault tolerance starting with just two nodes. StarWind VSAN is hypervisor and hardware agnostic, allowing you to forget about hardware restrictions and crazy expensive physical shared storage.

Build your infrastructure with off-the-shelf hardware, scale however you like, increase return on investment (ROI) and enjoy Enterprise-grade virtualization features and benefits at SMB price today!


You have seen how to set up a highly available SMTP relay solution that offers both an anonymous and an authenticated service. It can be done both cheap and efficiently by leveraging the built-in SMTP feature in Windows Server. The actual SMTP service lives in “the cloud” (SendGrid). We added a highly available load balancer to complete the solution. I hope this helps some of you out there.

We could also take a look at a container-based solution. When doing this on-premises, we do not always have the option to do so, but if you can do it, a container-based approach is undoubtedly an option. A drawback is that whitelisting for connections and relays might not be the most straightforward exercise in operations. The Windows SMTP feature provides a GUI and the ability to delegate operational rights inbox. It all depends on your needs and skills.


Views All Time
Views Today
Back to blog
The following two tabs change content below.
Didier Van Hoye
Didier Van Hoye
Didier Van Hoye is an IT veteran with over 17 years of expertise in Microsoft technologies, storage, virtualization and networking. He works mainly as a subject matter expert advisor and infrastructure architect in Wintel environments leveraging DELL hardware to build the best possible high performance solutions with great value for money. He contributes his experience and knowledge to the global community as Microsoft MVP in Hyper-V, a Veeam Vanguard, a member of the Microsoft Extended Experts Team in Belgium and a DELL TechCenter Rockstar. He does so as a blogger, author, presenter and public speaker