Exchange Online Transport Encryption: What You Need to Know from a Security Viewpoint

The Simple Mail Transfer Protocol (SMTP) is originally not secured by design, but there are many additional protocols added to improve security. One such protocol is Transport Layer Security (TLS) that encrypts the connections between mail servers aiming to prevent mail interception. This post discusses the different options with Exchange Online Transport Encryption. I warn you that some of the terms I use don’t always have the same definition other vendors use, take that in mind.

Opportunistic TLS

This is the default setting within Exchange Online. I also call this best effort transport as both sending and receiving mail servers negotiate on which encryption standards they will use. This proces starts with the STARTTLS command. Both servers start at the safest TLS versions & ciphers each of them support. They negotiate to find a combination that both parties support. It is obviously ideal when both servers support the most modern and safest ways to encrypt. Microsoft published their Technical reference details about encryption used by Exchange Online, including a list of supported ciphers.

The receiving server is the only one that does require a certificate. It depends on the configuration on the sending side what requirements it should have; whether it’s from a trusted certificate authority or self-signed for instance, or that it requires a specific common/certificate name. But it is common practice to accept any certificate, even invalid ones. The Microsoft Learn article How Exchange Online uses TLS to secure email connections states that Exchange Online uses a trusted certificate from Digicert.

It can happen that no match is possible. Sometimes one of the servers is a legacy device that won’t or can’t get any security updates. Or there is a company policy that explicitly disables older protocols & ciphers. Opportunistic TLS then falls back to no transport encryption at all, prioritizing mail transport above security.

Forced TLS

Forced TLS is the name I use for TLS connections that require encryption but are not domain specific. They apply to every connection, such as the default, unless a more specific connection is present or a protocol like MTA-STS or DANE is used (more on that later). This is the same proces with Opportunistic TLS, but with no fallback to unencrypted transport. I wrote a post about why we still accept Opportunistic TLS as a default and how you can configure this in Exchange Online.

While writing this post, I did find RFC3207 (from 2002 😊) that states in section 4:

A publicly-referenced SMTP server MUST NOT require use of the STARTTLS extension in order to deliver mail locally. This rule prevents the STARTTLS extension from damaging the interoperability of the Internet’s SMTP infrastructure. A publicly-referenced SMTP server is an SMTP server which runs on port 25 of an Internet host listed in the MX record (or A record if an MX record is not present)for the domain name on the right hand side of an Internet mail address.

In my interpretation this means that you cannot force other servers to use TLS. I did not find anything on sending, so you could still force using TLS to all your remote domains. Meaning that you can change your outbound but not your inbound connector if you want to remain RFC3207 compliant. That might not be a big problem if your organization adopts the principle that the sending organization is responsible for security and compliancy.

But there is RFC8689 proposal with the title “SMTP Require TLS Option” which explicitly addresses my issues I have. The sender will have the option to express the expectation whether mail is sent encrypted or explicitly unencrypted. This is an RFC to keep an eye on within the context of this blog. I need to read up on my RFCs and particularly this one.

Partner or Mandatory TLS

This is similar to Forced TLS but it uses specific remote mail FQDNs or domains. You can add specific requirements to the receiver certificate. Sometimes this definition is the same as Forced TLS. I wanted to differentiate the technical option not to have specific certificate requirements. The defining feature of Mandatory TLS is that its configuration is specific to a partner organization. A common alternate name is Partner TLS. It can have certain requirements, which can differ on outbound or inbound connections.

You can configure both the Inbound and Outbound connectors to accept or send mail only when all requirements are met. This could mean that there are specific certificate requirements. Such as matching the subject name with the domain. Or being a valid and trusted certificate and in addition set specific IP addresses as a requirement.

Theses settings give a great deal of flexibility in the level of security you want or require. It increases the probability the connection with the other side is actually the correct organization you need to communicate with. The downside however is the management and maintenance (certificates that expire!) of those connections sometimes per mail domain. It often requires communication with an admin from the partner organization. In my opinion this method does not scale that great.

MTA-STS

Now, the previous paragraphs already showed some of the issues and limitations of traditional SMTP connections. One big remaining risk is that of interception, but not by breaking TLS. Certificates and domain registrations are trivial to acquire and set up. Furthermore DNS in itself isn’t always that safe. We require a more robust way to ensure the endpoint is the correct one.

MTA-STS (Message Transfer Agent Strict Transport Security) and DANE (DNS-based Authentication of Named Entities) are designed to mitigating Advisary-in-the-Middle (AitM) attacks. They do this while removing need for per domain configurations.

Both of them still use TLS connections, but the proces before setting up the transport is more elaborate. Servers with MTA-STS support still use an MX lookup, but also check for MTA-STS support on the recipient domain. It will check for the existence of MTA-STS records and a policy file. This file contains the FQDN names of all SMTP endpoints. These should correspond with the values found in the MX record(s). It still mitigates AitM attacks, but only after first connection. Any discrepancies on found MX record values and that in the policy file should fail subsequent connections. Therefore the first connection is precarious, but still provides more security than traditional connections.

DANE

DANE has a less elaborate process, where as it checks for support and specific DNS records that publish information about the certificate the TLS connection should use. The sending server knows with a high degree of certainty what certificate it should expect. DNSSEC protects DNS queries against manipulation. If there is a mismatch, not TLS transport will occur. The beauty of this system is that it provides the highest form of security, even more robust than that of most securely configured Partner TLS connection. But it does not come with the relatively high admin cost as this does not require specific configurations per partner organization. Both organizations use DANE when they support it.

Which perhaps begs the question: what if our organization supports all of these protocols, what about organizations that haven’t adopted them all? Generally, the principle that Opportunistic TLS uses is still valid: a priority to transporting mail, before the most secure connection. Exchange Online will try DANE first, then MTA-STS, and if all else fails, Opportunistic TLS. However, Exchange Online prioritizes custom Partner connections over the use of DANE and MTA-STS.

It is prudent to regularly check with your partner organizations what their adoption plans are regarding MTA-STS and DANE, especially if you currently have Partner TLS configured with them. Partner TLS is sometimes preferred as it prioritizes security over transporting. Microsoft announced Mandatory DANE, which can be configured per tenant/domain. When enabled, it will only send mail when a connection using DANE can be made. Immensely increasing your Exchange Online transport encryption posture.

Concluding

There are multiple ways to achieve certainties of a secure mail transport. However, as SMTP is not secure by design and RFC definitions it is problematic to reach for 100% encrypted transport on the receiving end. Sending is different as you can decide to only send mail when Exchange Online Transport Encryption is possible. Monitor your outgoing queues if you decide to do that.

Some options such as DANE are highly secure and very manageable but unfortunately lack adoption. Not always due to a lack of support of the DANE protocol, but rather a lack of DNSSEC support on their whole DNS infrastructure. Adopting DANE could mean switching DNS providers to one that does support DNSSEC. Luckily this becomes more common (Azure DNS supports DNSSEC!!!!). Others such as Partner TLS are common practice but have a high admin cost or still have security gaps compared to other solutions.

There have been enough new developments in recent years that require you to reconsider your security stance on sharing information via mail. It is almost unavoidable to not consider adding message encryption and/or other safer ways to share information with other organizations. It is also a good practice to check your current custom configurations to see whether you can adopt MTA-STS and especially DANE. I hope this helps you to make a risk analysis for your organization and increase your Exchange Online Transport Encryption policies.