Last Week’s Record DDoS Attack Shows How Much CDNs Are Investing In Security
For anyone who follows me on social media, or is following the latest technology news, you’ve seen that there has been a significant amount of news generated by a new DDoS attack vector. The memcached UDP-reflection attack, that some are referring to as memcrashed, spoofs origin IP addresses to take control of memcached instances exposed to the public internet. One result of this type of attack was the largest DDoS attack ever recorded last week, levied against GitHub at 1.35 Tbps, or roughly twice the volume that the Mirai botnet levied against the Krebs site (a prior record).
From a streaming media perspective this is significant, since memcached instances are a common element in technology stacks powering streaming media infrastructure. Just because a business has implemented memcached does not mean that they are susceptible to having their machines taken over, however at the time of this writing there are nearly 60,000 memcached instances that are openly accessible. Since UDP is easy to spoof, these instances are not only vulnerable, but can amplify traffic by a factor of over 50,000, meaning a 203 byte request results in a 100 megabyte response. So if an attacker spoofs the IP address with that of their target, they have the means to generate a potentially massive DDoS attack.
The important thing to consider is that because of the way this attack vector works, and how relatively easy it is to initiate, the potential exists for even larger attacks to come. In fact, if you look at reports from Akamai and others, in a 24 hour period there were reports of attacks from 190 Gbps, 500 Gbps, and subsequently the largest reported at 1.35 Tbps. This means that any media company or digital business needs to have a strategy in place and take action. Most are blocking the default port used by memcached: UDP port 11211, but beyond that, you should be reviewing the SLA provided by your current DDoS protection provider to understand their capacity limits and behaviors if you come under attack. You don’t want one of your high value or critical streams to be disrupted or taken offline because your provider started to black hole your traffic.
All CDNs inherently protect you from most volumetric attacks, but more sophisticated stuff will require active filtering (or for these ports to be default blocked). Recent attacks are bigger than most any non-institutional DDoS provider to absorb themselves. If you were monitoring these events last week, you might have seen this tweet from Thousand Eyes highlighting their capture of GitHub withdrawing their routes from the telcos (and a subsequent writeup they performed on the entire event) providing their service and moving them to Akamai’s Prolexic platform. Because GitHub had already configured their platform for Prolexic, once they routed on to the platform they were able to restore service.
But more importantly, you need to know what the actual mitigation plan is with your DDoS provider. If there is going to be a hard cut into scrubbing mode, you need to plan for down time. What’s that going to do to your failover? What systems will be impacted? Are you sure you can recover gracefully? Github engineered for this, so their downtime was “only” 6mins. For most web facing applications, it should just come right up, but if you’re cutting over an API which you AND your customers rely on, how’s that going to behave? Akamai, Amazon, Fastly, Google and others all offer edge cloud based WAF and DDoS services and customers should look for solutions that are inline all the time and fully distributed across the entire network/platform.
Across the industry, CDNs continue to evolve their solution set and cloud-based WAF and DDoS solutions have become the new products CDNs are investing heavily in. While video was the killer app for a long time, security is now the new CDN.
The live stream of Super Bowl 52 is looking great so far, across all the devices and platforms I’ve tried. I’ve tested the stream on Amazon Fire TV (NBC Sports App, SlingTV), MacBook (Firefox, Chrome, Safari), iPad Mini (NBC Sports App), Xbox One (NBC Sports App, Sling TV), Apple TV 3/4, (NBC Sports App) and on mobile using the Yahoo Sports App and NFL Mobile App on iPhone, both celluar and WiFi. (Updated 8:14pm ET: Sony’s live streaming service PlayStation Vue is having major problems, in multiple cities, keeping many from being able to watch the Super Bowl. Seems to be fixed in some cities but not others.) (Updated 10:40pm ET: Hulu’s live stream of the Super Bowl has