One of the most powerful ways of stopping spam from entering your mailbox is to stop it from being received in the first place!
The way email is transmitted on the internet is via the SMTP protocol. We apply a number of checks at the SMTP stage to stop spammers.
RBLs (real-time blackhole lists) and RHSBLs (right hand side blackhole lists) are a very common way to detect and stop known spam sending machines. The main problem with RBLs is that they often have "false positives" and block legitimate systems, incorrectly causing email to be bounced back to senders. To stop that happening, FastMail minimises the use of RBLs and RHSBLs for outright blocking.
We use only one RBL for blocking, the SpamHaus XBL. The XBL is a highly accurate block list that lists the IPs of machines with known trojans and proxies. Independent testing shows that the ZEN RBL (of which XBL is a part) has a high block rate, with basically no false positives.
We use a few RHSBLs for blocking the hostname or helo string of connecting servers. RHSBLs usually have lower false positives because the domains tend to have to be explicitly added or require multiple collected reports, unlike RBLs which can easily auto-collect IPs based even on single user reports in some cases.
A number of other RBLs and RHSBLs/URIBLs are used during the spam checking phase that occurs after email is received, but these RBLs never outright block email; they're used as part of an overall scoring heuristic along with many other factors to filter spam into your Spam folder.
Address enumeration detection is designed to stop other people trying to find what email addresses are valid at FastMail. It does this by detecting attempts to send to many different, but similar, email addresses in a short period of time from a non-email server host.
Address enumeration detection provides a defence against spammers trying to discover valid email addresses simply by trying many different addresses over and over.
When a server is transferring email from one system to another, there's a series of conventions they have to follow defined in the SMTP RFC 2821 specification. Spam bots are often poorly implemented, or try and use tricks to work around these conventions. Many spam bots can be found and blocked by detecting some of the common violations.
Through our extensive, long-term experience running an email service, we've been able to closely analyse how spam bots try to deliver email to our servers. By using this data, we're able to pre-emptively detect spam bots soon after they connect, and block them from delivering their spam email at all.
Using a number of heuristics, we limit the rate that servers can send email to our servers. Again through our extensive, long-term experience running an email service, we've been able to tune these heuristics to minimise any effect on legitimate sending services, while significantly limiting the ability of spammers to deliver spam emails.
Greylisting is a process designed to detect and stop spam being accepted from poorly written sending email servers, which almost all spam bots are. One of the main concerns users have with greylisting is that poor implementations will often delay all email. The FastMail implementation uses a number of tweaks to ensure that legitimate email is delivered with no delay, while spam email is delayed or stopped from delivery entirely.
Greylisting works by returning a temporary failure response (
4xx code) to the first attempt to deliver an email, but accepting it on the second attempt. Every proper email server will attempt to redeliver a message after a temporary failure response, while currently almost all spam software does not, thus effectively blocking the emails from the spam software.
To avoid delaying proper email servers, our greylisting implementation performs a number of checks on a connection before applying the greylisting policy:
If a host opens an SMTP session with a HELO that is not an IP address and is not the same reverse DNS as its connecting IP, but the forward DNS of the name does resolve to the connecting IP, then that host is not subject to greylisting.
For example: The machine at IP
184.108.40.206 connects to us. The reverse DNS for
206-223-169-73.beanfield.net, which looks like a common dialup/dsl IP name, and would be a candidate for greylisting. However, the machine advertises itself to us with a
HELO mx3.hub.org line. Doing a forward lookup of
mx3.hub.org gives the IP
220.127.116.11, which is the same as the connecting IP, so we exclude it from greylisting
When combined, these features provide an excellent balance of greylisting hosts which should not be sending email, and allowing those hosts which should be sending email to get their email straight through.
Additionally, to help with the tracking of any problems, once a message passes greylisting and is accepted, a new header is added called
X-Spam-greylist. This header tells you how many seconds the email was delayed and whether that host has been whitelisted for 24 hours. (Technically, the delay figure actually indicates the length of the last delay for the ip/sender/recipient combination, so in the case of multiple emails from the same person, to the same person, from the same machine in a short time period, the figure will be a bit messy and hard to calculate.)
If you feel that email is being delayed by greylisting, please check the headers of your email for the
X-Spam-greylist header first. If it's not present, then the email was not delayed by greylisting, and it was probably just held up on the sender's side, something beyond our control.
The result of all these tests is that most spam is blocked before it even enters our systems. However, we can only run these tests when the email is delivered directly to us, not forwarded on from another mail server. If you use your own domain, we strongly encourage you to host your email directly with us rather than having it forwarded on from your registrar, as this will enable us to be much more effective at filtering out spam.