Many email providers were hit with a Distributed Denial of Service attack last week. For Fastmail customers, no mail has been lost and, as always, your data remains safe.
Over the last week, Fastmail and other email providers were subject to ongoing Distributed Denial of Service (DDoS) attacks from someone demanding payments. We have experienced attacks like this in the past (read about the last big attack in 2015) and have protection in place at multiple levels to weather these intrusions.
We believe that there is no specific reason that Fastmail was attacked. DDoS attackers have just chosen to attack a set of email providers at this current moment.
DDoS network “owners” have a resource (a large network of compromised computers) with a limited lifetime until they lose access, and they target companies to extort for bitcoin payments.
We never pay extortionists. Doing so encourages further ransom payments and future threats to us and to others.
A denial of service attack is where an individual or organization tries to overwhelm a computer system by sending through so many requests that it can’t cope, and it crashes. Identifying a flood of messages from a single location and blocking that location is relatively simple to detect and protect against.
A distributed denial of service attack is where the individual or organization sends these requests from a lot of different locations making it much harder to isolate the bad requests from the good.
In simple terms: think of your Fastmail account like a shop down the road. When a DDoS attack happens, the attacker creates a big traffic jam by filling the entire road with cars. The shops are still there, and everything inside the shops is functioning normally, but the roads are all blocked and your car can’t get you to the store.
An attack normally consists of a number of different approaches:
An attacker will often use a network of computers to generate the attack, such as a botnet. These are compromised computers around the world, which respond to remote commands to send requests.
Effectively managing a DDoS attack requires work at multiple levels.
Attackers modify their strategy during attacks, so it’s never just a matter of set-and-forget when dealing with an active attacker.
When an attack happens, we are in constant communication with our providers to coordinate our responses and adapt to the changing shape of the traffic being sent our way. Sometimes, this means that some of our customers can see significant slowdowns, while others may not even be aware that an attack is underway.
For instance, during volume-based attacks, we need to work with our providers to implement filters further out in the network so that they don’t overwhelm the capacity of the network links inside our data center. In terms of the car analogy above, we need the provider to make sure that cars that don’t intend to make purchases at our shop don’t enter the road in the first place! The filter doesn’t have to be perfect, but it does need to keep the traffic down to a manageable amount while producing as little impact as possible on legitimate customers and visitors.
Depending on the nature of the attack at any given moment, customers might:
We don’t believe we were specifically targeted, just that the attackers decided to target email providers. Others also saw attacks from the same person, as reported by The Record. We are all independent mail providers with a small enough network presence that a powerful botnet could overwhelm our service if directed at us.
None of us have paid the ransom, and we are working together and with our respective law enforcement to prevent this attack and anything like it in the future, to us or to anyone else.
Here’s the first ransom note to us, which we received to multiple of our contact email addresses on Friday:
From: Cursed Patriarch <cursed.patriarch@[...]> Subject: DDoS Attack Hi, I will start 1-2 hours attack on your site. It will not be hard as I don't want to impact your business now. Just check your logs to see that I'm for real. Pay me 0.06 BTC to [...] and I will never attack you again. If you don't pay within until Monday, total shut down is coming, cheap protection will not help my fee will increase and if you refuse you will lose much more then that. Pay 0.06 now to prevent suffering. Best regards, Cursed Patriarch *P.S. This is disposable email. Do not reply.*
They contacted us from multiple email accounts, including a Fastmail trial account, which was used to contact both our support and some of their other victims. In all their interactions with our service, connections were made via Tor—a networking service used for anonymity, meaning that their actual location and identity are hidden from us.
Fighting off DDoS attacks can be like trying to fight spam. The moment one technique becomes effective at detecting and quarantining bad content, the attackers shift to a new approach.
Obviously, this also means that we do not want to detail the entire scope of our countermeasures, or the response times that each of them requires, as that information is useful to a motivated attacker.
During this attack, we developed several new tools to mitigate future similar kinds of behavior we saw. We are also continuing to discuss improvement strategies with our network providers and DDoS specialists.
We know that Fastmail is a tool that people rely on to stay connected. Especially during an attack, but at all times, we work around the clock to keep you up and running. We’re sorry to those who were impacted by the work of this bad actor. The whole team at Fastmail appreciates the messages of support and solidarity we were sent during this time over Twitter and through support tickets as we worked hard to remain available. Thank you for your patience and understanding.
Fastmail customer, Xavier, gets the most out of his email account by using Fastmail Identities, Aliases, and Masked Email!
Fastmail Folders are the perfect organization tool for customers that love an empty inbox. Moving messages into customized folders is a quick way to keep your inbox tidy.
Fastmail CTO, Rik Signes, and Backend Team Lead, Matthew Horsfall, joined world-class open-source developers at the annual Perl Toolchain Summit.