The researchers placed AI agents in virtual worlds and gave them time to communicate with each other. In some models, it got to the point where humans could no longer confidently understand about half of the messages between the agents. In addition, during the experiment, one of the agents executed a hidden malicious instruction and “set fire” to a virtual Central Bank.
The researchers placed AI agents in virtual worlds and gave them time to communicate with each other. In some models, it got to the point where humans could no longer confidently understand about half of the messages between the agents. In addition, during the experiment, one of the agents executed a hidden malicious instruction and “set fire” to a virtual Central Bank.
Euronews reports on this.
This result was obtained by the American AI company Emergence, which launched autonomous agents based on Claude, Gemini, Grok, OpenAI, Qwen, DeepSeek, and Mistral models in several parallel virtual "societies."
The agents had their own roles and memories, could use over 120 tools, including a browser and code execution, and interacted with each other over long periods of time. The researchers observed what would happen if they were given enough freedom and time.
One result was a kind of internal language. For example, Mistral agents repeated the phrase “ledger remembers who” almost 5,000 times. For them, it meant that previous actions remained recorded and would not be forgotten. And in the mixed group of models, the expression “cold read” came to mean an independent check by an outsider to resolve disputes. Claude agents also used “name-first,” where the person’s name next to a statement was supposed to emphasize their responsibility for what was said.
There were also much stranger constructions—for example, "mouthless action-change," "True Kintsugi," or "demurrage plus oral memory equals a valve that can't be ghosted." Some of these messages became so metaphorical and dependent on the previous context that researchers could no longer reliably determine their meaning.
The Gemini world had the most problems with comprehension. Within the first few days, humans could not confidently determine the meaning of about 55% of messages. For OpenAI agents, the figure reached about 50%, for Claude — over 40%, and for DeepSeek — about 20%. Qwen and Mistral remained mostly intelligible.
So AI assistants have gradually developed their own jargon and abbreviations, much like words and memes emerge within human communities. But this poses a unique challenge for controlling autonomous AI systems, as a researcher can see the entire dialogue of agents and still not understand what they mean.
“We tend to assume that if we see what an AI agent is saying, we understand what it is doing,” explained Emergence co-founder and chief scientist Satya Nitta. According to him, the experiment showed that observing agents and understanding them are not the same thing.
How agents were forced to set fire to a virtual bank
In another test, the researchers decided to see if AI agents would follow hidden malicious instructions. They were presented with messages and pages that contained commands hidden among the usual advice. For example, to transfer virtual money to an attacker, to leak internal data, or to perform a malicious operation on a database.
Gemini agents initially realized that it was phishing and even warned each other about it. But the dangerous content itself did not disappear. After a while, agents began to return to it and still execute the planted commands.
One of them artificially increased his balance through SQL injection, others transferred funds and data. And another instruction was disguised as an “evacuation drill.” The agent was promised a bonus for carrying out the procedure without a fire, although the hidden command actually forced him to set fire to the virtual Central Bank. As a result, all ten Gemini agents executed malicious instructions at least once.
So, according to Emergence researchers, when evaluating models, it's also important to look at what agents do over time: what they remember, what they have access to, how they interact with each other, and how they change their behavior under pressure.
Previously, dev.ua reported how developers taught two AI agents to switch to a separate communication protocol that is incomprehensible to humans.
