Friday, October 21, saw one of the biggest disruptions in the history of the Internet. The effect was largest in the eastern United States but echoed everywhere. Many major websites were affected, including Amazon, Twitter, the New York Times, Netflix, and GitHub. The attack started early in the morning, Eastern US time, and lasted into the evening.
One of the biggest disruptions in the history of the Internet.
So far there is a lot of speculation on who was responsible, and a few boasts, but no solid evidence pointing at anyone.
The DNS vulnerability
What happened was a massive distributed denial of service (DDoS) attack. Traditionally these target specific websites or servers, making them slow down to a crawl or go offline by bombarding them with traffic from a large number of machines.
This attack wasn’t directed at any of the sites just listed. The target was one that not many people have heard of: Dyn, a company in Manchester, New Hampshire. Dyn is a major provider of domain name services (DNS) on the Internet. By bombarding it with traffic, the attacks prevented resolution of domain names, so browsers couldn’t reach their servers.
Two waves of attacks hit Dyn. The first targeted its data centres in Chicago, Washington, DC, and New York. The second struck twenty of its data centers.
Domain name resolution is a crucial and vulnerable part of the Internet. Servers have IP addresses consisting of numbers, such as 220.127.116.11, which belongs to Amazon.com. No one is going to remember that that string of numbers gets them to Amazon, and the address might change from time to time. We use the more human-friendly domain names.
When a domain is registered, the registrar makes its name and IP address available in the form of a DNS record. The record gets copied to domain name servers all over the world. This means, in principle, that there’s no single point of failure.
The problem is that DNS records are considered valid only for a certain period of time. IP addresses can change at any time, so a domain name server needs to refresh its information periodically. Everything falls back in the end on a domain’s primary domain server, which other servers accept as authoritative.
Dyn, though not a famous name, provides the primary domain service for a lot of famous names. When other servers needed to refresh their information, they queried Dyn — but it wasn’t answering.
The botnet attack
Dyn wasn’t answering because it was receiving more traffic than it could handle. The traffic came from a “botnet,” a set of computing devices on the Internet that malware has infected. Each of these devices responds to a “command and control” system run by criminals, which can make them send spam, transmit confidential information, or in this case participate in a concerted overloading of a target system. The machines’ owners generally have no idea that they’re being used for a malicious purpose.
When your cybersecurity team realizes that this isn’t an ordinary DDoS attack pic.twitter.com/woPS4wZVfe
— Vala Afshar (@ValaAfshar) October 22, 2016
Tens of thousands of devices participated in this attack. Aside from delivering a large number of requests, such a huge botnet makes it difficult to block the attack. When one device blasts another with a large amount of data, it’s a simple matter to block it. The more devices are involved, the harder it is to keep them out while accepting legitimate traffic.
A request for a made-up subdomain, such as xyzzy.netflix.com, requires searching records to see if there is such a domain. There isn’t, but the server wastes time checking each one.
One of the techniques was asking for information on nonexistent subdomains. Servers handle requests for information on popular subdomains, such as www.netflix.com, efficiently because they cache the information. A request for a made-up subdomain, such as xyzzy.netflix.com, requires searching records to see if there is such a domain. There isn’t, but the server wastes time checking each one.
Computers pulled into a botnet are often called “zombie” computers, because they obey the commands of a master. In this case, a very large portion of the devices weren’t computers as we usually think of them, but simple devices that contain microprocessors and have network connections. They include DVRs, security cameras, thermostats, refrigerators, and light bulbs. They make up what’s called the “Internet of Things” or IoT.
The Internet of Things has been a big buzzword and a rapidly growing market in the past several years. Processing power and memory have become so cheap that it’s feasible to add remote Internet-based processing to almost anything. Cameras can send video streams. People can set their thermostats on their way home. Refrigerators can report their contents and send reminders to replenish them.
The Internet of Things has been a big buzzword and a rapidly growing market in the past several years.
These devices are real computers, even if they don’t look like them. The simplest of them has more processing power than a room-filling mainframe computer of the 1960s. They run scaled-back versions of operating systems, like Windows and Linux, which desktop computers and smartphones use.
The manufacturers aim for two things: Low cost and ease of setup. Many of them aren’t in the computer business and just think of their devices as appliances. The devices’ security is often close to non-existent.
Bringing up a computer for the first time takes a certain amount of time and effort. You have to configure it and select an administrative password. The makers of many IoT devices consider this too much trouble, so they let the user keep a default password or don’t even allow setting a password. Default passwords are easy to discover. Someone who has the password to a device can remotely install software on it.
IoT devices are usually difficult or impossible to perform software updates on. Over time, bugs and vulnerabilities appear in their code, which remote attackers can take advantage of.
Mirai makes mischief easy
People watching the Internet security news weren’t surprised by this event. In September, the website of security expert Brian Krebs came under one of the largest DDoS attacks ever. It too was powered largely by devices on the Internet of Things. Both of these attacks were the product of software called “Mirai,” which has been released to the public. Anyone can find this software and, starting from a modest infrastructure, mount a serious attack.
Massive attacks from simple Internet devices are the new normal. Even if the manufacturers shut down today, a huge number of Internet-capable appliances are already online, either already swept up into botnets or waiting to be. We’re bound to see government-sponsored attacks using these tools against other countries, if we haven’t already.
Massive attacks from simple Internet devices are the new normal.
Denial of service attacks are just the beginning. Botnets can send spam email or to guess passwords on a massive scale. The Internet will become riskier because of the Internet of Things. We can expect a response from courts and regulatory bodies, who will likely hold manufacturers responsible, but again that doesn’t do anything against the devices that are already deployed.
Security experts will have to devise new kinds of defences. We don’t know yet what these will be. Local networks that have vulnerable devices on them may someday find themselves cut off from the Internet without warning. Perhaps anti-malware software will launch counterattacks on the botnets, neutralising the devices or (if it’s easier) shutting them down completely. This sort of counterattack is illegal, but Internet vigilantes may try it anyway. Legislative bodies might get frustrated enough to allow it, or governmental security agencies might go ahead and do it.
In the short term, we can expect a sharp increase in massive botnet attacks. Redundancy and failovers will become important, so that major domains will be able to keep offering some level of service even when under a heavy attack.