Varto
Хочете будувати кар’єру в Data Science та AI? Vodafone Big Data Lab відкриває новий набір —>

The zombie apocalypse on the Internet has reached Ukraine. Reports of armies of bots coming to our sites and why they already owe us money

The terrifying truth was revealed to us by Cloudflare, a California-based company that helps deliver and protect content around the world, in a recent 2024 report on the latest trends in the Internet.

Leave a comment
The zombie apocalypse on the Internet has reached Ukraine. Reports of armies of bots coming to our sites and why they already owe us money

The terrifying truth was revealed to us by Cloudflare, a California-based company that helps deliver and protect content around the world, in a recent 2024 report on the latest trends in the Internet.

The largest generator of bot (i.e. non-human) traffic in the world is the USA. This country accounts for 34.6% of such Internet requests. The top five countries that «supply» «dead» traffic to the Internet also include such countries as Germany (6.8%), Iran (5.2%), China (4.1%), and Singapore (3.7%).

A strange mix, isn’t it? Curious to know more?

Let’s figure out why it happened that the USA is the main «supplier» of bots and why this is bad (and for us too).

First, you need to understand that not all bots are necessarily bad. To put it simply, they can generate not only dirty traffic to cheat sites with a dubious reputation, but also be service functions of various platforms or «spiders» (crawlers) or agents of projects that train artificial intelligence.

The last category of bots became particularly interesting in 2024. So this type of bots will be the one we will mostly talk about.

AI bots have come for digital casts of websites

Here’s what Cloudflare itself writes in its report on this type: «AI bots and crawlers were in the news throughout 2024 as they voraciously consumed content to train ever-evolving models. This has sparked controversy as not all bots and crawlers follow content owner guidelines to limit crawling activity.»

Now let’s explain in simple terms what is happening.

The fact is that American companies such as Google, Meta, Apple and others massively train their AI models. In order to train them, they need a large amount of content generated by people. Where to get it? That’s right — from sites where it is available in large quantities. This could be, for example, a news site that stores a huge archive of news and stories about what has happened to humanity over the past decades. And this is actually a very valuable knowledge base for algorithms that are quickly getting smarter.

In fact, Cloudflare data supports this theory.

The largest share of bot traffic (almost 13%) falls on two autonomous systems (this is a specific permanent designation of a group of IP addresses) of Amazon. This can be explained by the fact that a large number of AI projects (both their own and third-party) are «spinning» in Amazon clouds, which train their models. And apparently, the main load falls on OpenAI models (ChatGPT), because this company uses the AWS cloud.

The next figure is even more impressive (well, for the author of this column). Note: only 6% of the traffic from Amazon’s largest autonomous system, which is shown in this study, is human traffic. The rest is bot traffic.

Traffic from one of Amazon’s autonomous systems

Agree, this is powerful! Although I assume that there is still more human traffic on other autonomous Amazon systems.

Next comes Google and Google Cloud with their two autonomous systems. They account for more than 8% of the world’s bot traffic. And only 3% of Google’s traffic (one of its autonomous systems) is from live people.

In this list, you can also see IP address pools from Microsoft (4% — people), Apple, and Facebook (7% — people).

You can’t even imagine the number of bot requests coming from these giants. In fact, AI agents are now simply eating up the data that we (people) have accumulated — in forums, articles, posts, photos, and so on. We can talk about billions of bot requests.

And now the most important thing: what’s in it for me?

The fact is that the uncontrolled use of human content for training AI, in fact, without the consent of the authors, is, so to speak, gentle, not very polite. Because someone paid for the production of this content with their time or money. And what is this action called when you take something from someone for free and without their knowledge? That’s right — it’s theft!

In fact, some Western content owners have already taken notice and started fighting back.

«If you don’t compensate creators in some way, they stop creating, and that’s a detail that needs to be addressed,» Cloudflare CEO Matthew Prince told TechCrunch.

He even announced the creation of a special marketplace primarily for small site owners, where they will be able to sell AI providers access to collect content from their resources.

Cloudflare has also made a button that ensures that customers can block all AI bots with a single click.

«This came about because of the frustration we heard when people felt like their content was stolen,» Prince said.

In fact, there is a noticeable amount of bot traffic coming from Cloudflare addresses as well. But at least the company is trying to be honest with content creators.

Last year, Wiredreported that nearly 90% of top American media outlets, such as The New York Times, had already blocked access to AI bots on their websites.

Last year, we saw major content platforms try to monetize access to their content for training AI models. For example, Reddit started allowing only Google agents to access its pages, as the latter had a special deal with the platform to train bots on Reddit content, which costs Google $60 million per year.

What is happening in Ukraine?

Analyzing Cloudflare data, we can see that Ukraine is not a source of bot traffic for the world. Only 0,4% of the world’s non-human traffic comes from our country. This is due to the fact that large AI projects are not hosted on our facilities (and we don’t seem to have any significant ones of our own).

The only more or less noticeable source of bot traffic in Ukraine is the Kyivstar autonomous system, which accounts for 8% of total bot traffic. Other providers have much less.

Sources of bot traffic in Ukraine

It should be said right away that the traffic coming from Kyivstar is mostly human. Cloudflare shows that it accounts for 90%. The rest are bots. At other large Internet providers, the share of bots is even lower: Vodafone — 8%, lifecell — 6%.

Interestingly, a fairly large share of bot traffic is recorded in the autonomous system of a provider such as Triolan — 17% (with peaks at night up to 45%).

Traffic in the Triolan autonomous system

There is also a lot of bot traffic in the autonomous system of the Crimean internet provider Miranda Media, which is subordinate to the occupation authorities. What kind of bot traffic could this be? It’s hard to say.

Are American giants using our sites to train their AI bots? They are probably already starting. For example, Facebook’s autonomous system generates bot queries in our country. It’s on the map. We are also in Google .

Traffic from Facebook AI agents in Ukraine

After all, even on dev.ua, traffic from OpenAI’s spiders is starting to be a little noticeable. It increased by 641% in November, according to Similarweb.

What are the enemies up to?

It is much more interesting to consider Iranian providers. Remember, Iran was in the top 3 «suppliers» of bot traffic? Well, the largest source is the national provider IranCell, whose 59% of traffic is bots. One can only guess why this share is so large. It is quite possible that the Iranians are also training their own AI models, which we do not know about yet. And in the Russian Federation, the largest hotbed of bot traffic is the autonomous Yandex system (I think it is one of them). There are almost 100% bots. And it is interesting that they train on Russians and Serbs.

Where do Yandex’s AI agents go?

Conclusions

The issue with traffic from AI agents is not somewhere out there in the West. It is starting to be noticeable here too. After all, these agents, whether harmless or, on the contrary, dangerous, will definitely come (already here) according to our sites. And most likely, we will give them away for free, it is unclear to whom.

Bots are starting to make website statistics unclear and greatly blur them. They often imitate human behavior to avoid being noticed.

Website owners see random waves of traffic to their projects on the Google Analytics graph. They rejoice. They think that their content has finally become better. But in reality, it’s just another algorithm that made a copy of a popular site in the country and moved on.

At least he paid the money.

Also Read
Що робити, щоб ваші твіти не блокували кремлеботи. Інструкція
Що робити, щоб ваші твіти не блокували кремлеботи. Інструкція
Що робити, щоб ваші твіти не блокували кремлеботи. Інструкція
Соціальна мережа Twitter — один з головних майданчиків для поширення інформації про війну в Україні західній спільноті. Але де правда про війну, там відразу з’являються боти Кремля, які кидають страйки. Розповідаємо що робити, аби ваші твіти не блокували.
1 comment
Яка кількість російських ботів воює проти України. Підрахунок експерта
Яка кількість російських ботів воює проти України. Підрахунок експерта
Яка кількість російських ботів воює проти України. Підрахунок експерта
ЗСУ та кіберполіція в пошуку програмістів для створення ботів, а в Франківську потрібні спецілізовані кадри для створення дронів
ЗСУ та кіберполіція в пошуку програмістів для створення ботів, а в Франківську потрібні спецілізовані кадри для створення дронів
ЗСУ та кіберполіція в пошуку програмістів для створення ботів, а в Франківську потрібні спецілізовані кадри для створення дронів
1 comment
Запрацював бот єВорог від Дія, якому можна повідомити про ворожу техніку та військових
Запрацював бот єВорог від Дія, якому можна повідомити про ворожу техніку та військових
Запрацював бот єВорог від Дія, якому можна повідомити про ворожу техніку та військових

Have important news to share? Message our Telegram bot

Key events and useful links in our Telegram channel

Discussion
No comments yet.