Cybersecurity researchers discovered that spammers were using OpenAI’s API to create unique messages, allowing them to bypass spam filters and send fraudulent emails to over 80,000 websites in four months. They convinced the AI that it was helping them with their marketing.
Cybersecurity researchers discovered that spammers were using OpenAI’s API to create unique messages, allowing them to bypass spam filters and send fraudulent emails to over 80,000 websites in four months. They convinced the AI that it was helping them with their marketing.
This is stated in a report by cybersecurity company SentinelLabs. Their research shows that the capabilities of AI for large-scale content generation can easily be used by attackers, Ars Technica reports.
OpenAI blocked the spammers' account after receiving SentinelLabs' exposure, but it had been active for about 4 months before that.
The attackers used AkiraBot, a framework that automates sending bulk messages to promote «gray» search engine optimization services for small and medium-sized websites. AkiraBot used Python-based scripts to rotate the domain names advertised in the messages.
The framework also used OpenAI’s chat API, tied to the GPT-4o-mini model, to generate unique messages tailored to each site it spammed. This likely helped it bypass filters that look for and block identical content sent to numerous sites. The messages were sent via contact forms and live chat widgets embedded on the targeted sites.
«AkiraBot’s use of LLM-generated spam content demonstrates the new challenges that artificial intelligence creates for protecting websites from spam attacks. The easiest indicators to block are the changing set of domains used to sell Akira and ServiceWrap SEO offerings, as there is no longer a consistent approach to the spam content, as was the case in previous campaigns to sell these firms’ services,» SentinelLabs researchers note.
AkiraBot gave GPT-4o-mini the following instruction: «You are a helpful assistant who generates marketing messages.» The prompt instructed the AI to replace the variables with the site name provided at runtime. As a result, the body of each message from the receiving site included its name and a brief description of the services it provides.
«The resulting message contains a short description of the target website, making it look like the author’s. The advantage of creating each message using LLM is that the message content is unique and spam filtering becomes more difficult compared to using a consistent message template that can be trivially filtered,» the researchers write.
SentinelLabs obtained log files that AkiraBot left on its server to measure success and failure rates. One file showed that unique messages were successfully delivered to over 80,000 websites from September 2024 to January of this year. Messages aimed at approximately 11,000 domains failed.
OpenAI thanked the researchers and reiterated that such use of its chatbots violates its terms of service. The company previously published a report describing trends in the use of its platform by attackers. The company removed dozens of accounts that were using ChatGPT for fraud, such as generating fake content or job postings.
This is not the first time cybercriminals have implemented artificial intelligence to scale attacks and generate revenue. Malware generation, automated phishing, and self-learning AI worms are just some tools used in modern cybercrime. dev.ua reported on how malicious AI models are changing the cyberthreat landscape.
