The security community was just taking a breather because we hadn’t seen a massive DDoS attack since the Mirai thingbot took down Dyn in October 2016 with a 1.2 terabit per second DDoS attack. Yesterday, that world record attack was broken when GitHub was hit with a 1.3 terabit per second DDoS attack.1 This attack was launched from memcached systems mistakenly open to the big bad Internet, compromised by attackers, and then used to launch amplification attacks coming from UDP port 11211.
Like the cleverness of Mirai’s initial DDoS attacks where the attackers used GRE to tunnel packets over the network (that DDoS mitigators weren’t expecting, so they weren’t explicitly denied and therefore let through), this latest DDoS attack is also clever in how the attackers were able to amplify their attack tremendously.2 The amplification factor was up to 51,000, meaning that for each byte sent by the attacker, up to 51KB was sent toward the target. There have been numerous attacks and exploits against MongoDB and Elastic Search systems in the past couple of years from which we didn’t see amplification attacks. We call out the “cleverness” of these attacks because it speaks to the skills of the threat actors. Technical chops is one thing (which often just means you know how to run someone else’s exploit code), but now we have insidious innovation coming up with new ways to launch the same old attacks or making them more effective.
All organizations should manage their security programs around the fact that any vulnerable system on the big bad Internet, especially if it’s open to the entire Internet, will be discovered, explored, and exploited by attackers.
Reflection attacks (leveraging your apps to attack yourself and others) like we saw in the latest DDoS attack are nothing new. Open mail relay systems to the Internet, compromised and used for spam relays, were probably the most common example of reflection attacks that have been happening for decades. DNS reflection attacks are now just part of the threat landscape. It should not come as a surprise to anyone in the security community that attackers are now looking for memcached ports open to the entire Internet as opportunities to exploit for DDoS attacks. Attackers are getting smarter by leveraging these kinds of application services to weaponize. They’re looking at application infrastructure and what can be spoofed or subverted. Let this be a warning call to all to get our networks and applications in order!
No application infrastructure, especially databases or database caching system should ever touch the Internet without hardening and strong access control. This is true whether your apps are on-premises or in the cloud. If you have these in your environment, lock them down now. The memcached port should be your number one priority, given we are seeing this attack in the wild right now. But memcached isn’t the only database caching system in the world. There are numerous others, like Redis, where the same attack would apply if your databases were misconfigured and exposed to the Internet. If your databases need Internet connectivity for legitimate business reasons, lock them down to a set of allowlisted IP addresses.
In addition, expect any other application infrastructure systems to be leveraged for attack. Consider content distribution network (CDN) devices. CDN servers hold cached images and files that help speed up web sites and applications. An attacker can use spoofed hash requests appended to a legitimate request to call for a non-existent file. This can cause a CDN to try to pull data from the main servers for non-existent items, draining CPU resources from the site. This causes the CDN to apply additional load on the main web site instead of reducing it. Insidious attacks like these where your own application infrastructure is used to knock you or others down are trending new attack methods.
Scan your network for ports and services open on the Internet as frequently as you can. All it takes is one configuration change in a code push to make your work insecure again, so scan everything and often.
In addition to calling out the need to assess the security posture of your networked systems, this attack highlights the importance of threat intelligence feeds, and specifically ones that give you real-time updates on bad IP addresses to block. Reflection attacks allow for the collection of the actual source IP addresses, which you can assume are weak or compromised servers. Either way, you don’t want them communicating with your network and can use them as a blocklist.
