Mehdi Daoudi, CEO and co-founder of internet performance monitoring platform Catchpoint, believes that the Cloudflare outage , which took down numerous sites (from dev.ua to DownDetector), should be a wake-up call for companies.
Mehdi Daoudi, CEO and co-founder of internet performance monitoring platform Catchpoint, believes that the Cloudflare outage , which took down numerous sites (from dev.ua to DownDetector), should be a wake-up call for companies.
"Everyone puts all their eggs in one basket and then they're surprised when there's a problem. Companies need to take care of redundancy and fault tolerance," Dowdy stated.
Cloudflare powers a large portion of the internet — the company said last year that about 20% of websites run on its network. It also serves 35% of the Fortune 500 companies, as well as “millions” of other customers, The Verge reports .
Cloudflare's speed and high level of security make it a popular choice for websites around the world, but the November 18 outage draws attention to how concentrated the web infrastructure industry has become.
After a recent AWS outage took down secure messaging app Signal, the service’s president, Meredith Whittaker, said the company had no choice but to use a large cloud provider. “Virtually the entire stack is owned by 3-4 players,” she wrote.
But even if companies rely on just a few web infrastructure providers, they should have a backup plan. “Outages are going to happen, and they’re going to happen more often. The radius of impact is going to continue to grow. The question is what do you do about it,” Dowdy said.
While Microsoft and AWS attributed their outages to issues with DNS — the system that translates website domain names into IP addresses — Cloudflare found that a single file was the cause of the outage.
“The root cause of the outage was a configuration file that is automatically generated to manage threat traffic. The file exceeded the expected record size and caused a crash in the software system that handles traffic for a number of Cloudflare services,” said Cloudflare spokeswoman Jackie Dutton.
It seems amazing that such a file problem could bring down entire segments of the internet, but for large companies like Cloudflare, it can happen.
Serhiy Kolesnichenko, COO at Ukrainian cloud provider Ucloud, noted that Cloudflare’s powerful infrastructure is “difficult to collapse with an attack,” but did not rule out the possibility of a “human factor.” He told dev.ua that if this factor affects 1–2% of the entire infrastructure, it will not be much on the company’s scale, but for users it will seem like they are witnessing a disaster.
According to Dowdy, similar mistakes will be repeated.
“Are you going to complain about this every time Cloudflare sneezes? Or are you going to build around it?” the Catchpoint CEO asked.
Красава!