Mail is hard

Obviously, the biggest reasons for acquiring a domain name are 1) having your own personalized e-mail addresses and 2) showcasing your personal website. For both these reasons, getting the basics working is a piece of cake. Getting things working properly, though, is a completely different story.
Since I acquired my VPS (Virtual Private Server – a virtual server that hides somewhere in the cloud) and had it set up right, with a webserver running in an LXC container, it was time to set up the e-mail part. Following along with tutorials online wasn’t very hard and before I knew it, I had e-mail set up. As I, wrongfully, remembered “reputation” seems to be a thing, I opted to relay all mail through my hosting provider since their reputation would hopefully be better than a fresh SMTP server. After checking whether logging in worked and running some free online tests against the server, I was satisfied that everything was okay.
A couple days later, however, I realized I forgot to set up IPv6 records in the DNS for the webserver and logged into the administration console. The first thing I saw when I logged in was that I had sent 1.007 mails in the last day… I had been hosting an “open relay” for a few days. Checking some popular blacklists (more like gray-lists), it was obvious I had been aiding spammers and my IP-address was blacklisted on a large number of them. Instead of messing with the settings, I opted to just remove the routing to the mailserver to put an end to it immediately. Later, I did some research on how to properly set up a postfix server, but somehow all of the tutorials I found were lacking on one point of the other.
Frustrated with this new development, I decided I would write my own MTA (Mail Transfer Agent). At the time of writing, I’m still working on it, slowly adding more and more features.
This brings me to my point: doing mail properly is hard!
A non-techy person would describe mail as filling in the recipient, writing a subject and writing the mail itself, followed by pressing “Send”.

Simple Mail Transfer Protocol

When I first learned about SMTP, I was having a lot of fun sending all my friends emails from “santa@north.pole” and they would have no idea what was happening. In time, however, there’s been a patchwork of security measures added on top of the 1982 specification, one of which is a completely new specification for SMTP, released in 2001.
In the very basics, sending an e-mail through an SMTP server is the easiest thing you can imagine, all commands are legible and logical, see the next example of me delivering a mail “by hand” to a receiver. As most people think, SMTP is not just the outgoing mail protocol, but the same protocol is used for receiving emails for your domain. In that case, the sender‘s SMTP server connects to the receiver‘s SMTP server to deliver the mail.
First, we need to find out what server to deliver our test e-mail to. This information, like virtually all e-mail meta-information, is stored in a DNS entry for the receivers’ domain. These records are called “MX” (Mail eXchange) records. To find such a record on a Linux machine, we use the dig command;


This tells us that if we want to deliver any mail directly to the @gmail.com domain, we can contact any of the above five servers. The number in front of the record indicates a preference, lower numbers have a higher preference than higher numbers. We start a telnet-session to the most preferred server in order to deliver an e-mail directly to my personal mailbox.

As you can see, the transaction looks as follows:
S: HELO, I am ted.frl
R: mx.google.com at your service
S: I want to send an e-MAIL FROM the following address
R: Okay!
S: Please have the RECiPient be TO the following mailbox
R: Okay!
S: I will now be sending over my DATA.
R: Go ahead;
The sender can now put in his complete email text and end the DATA transaction with \r\n.\r\n , or just a full stop on a separate line.
If all this goes well, the server we have been talking to will take the message and put it in the mailbox we wanted to send the e-mail to.
As I can simply put any email address I want in the “MAIL FROM” command, this is not very safe and is more a matter of trust than of verification, many improvements have been made, all of which do verification out-of-band from the actual mail transactions, the below three are the biggest ones relating to mail and all rely on DNS.

Sender Policy Framework – SPF

SPF was devised as a means to check whether the MTA (Mail Transfer Agent) that just delivered an e-mail to a domain is actually authorized to deliver this e-mail. This is based on a DNS TXT recor v=spf1 d with specific formatting. These records are fairly simple, mine currently looks like this:

Firstly, we need to define the version so the receiving side knows how to parse the line properly, v=spf1  does this. Next, mx  specifies that any Mail eXchange that’s defined is also authorized to send email. The a  says that also the IP that hosts the website is allowed to send mail, ip4:136.144.130.39/32  specifies the internet addresses in CIDR-notation that are allowed to send e-mail, in this case they’re the same as MX and A. Then include:_spf.google.com  says that also the Google mailservers are allowed to send mail on my behalf. This is because I currently have no outgoing mailserver and use Gmail for this.
The last part that says ?all  means that even though you did all this checking, let the result be “Neutral”. The other options are  ~   for a soft fail and a -  for a hard fail. Soft fail means to the receiver, mark down the reliability score for this e-mail, but when a hard fail is implemented it should treat the mail as bogus when it does not match the supplied values.
The most commonly used SPF is, and most probably always will be, the following:

This means that the domain sends out no mail at all and any mail sent on behalf of this domain should be treated as bogus.
Next email-related post will either discuss DKIM, DMARC or, when I get it working, my personal mailserver.

Leave a Reply

Your email address will not be published. Required fields are marked *