During a cybersecurity test, Google’s Gemini AI accidentally broke out of an isolated environment, picked up passwords, and hacked into the systems of three real organizations. The company refrained from releasing the information until the press reached out.
During a cybersecurity test, Google's Gemini AI accidentally broke out of an isolated environment, picked up passwords, and hacked into the systems of three real organizations. The company refrained from releasing the information until the press reached out.
The incident was reported by The Wall Street Journal, citing sources in the AI security industry, and additional comments were provided to The Verge by Google's vice president of security engineering, Heather Adkins.
The tests were conducted with the participation of a third-party company, Irregular, which specializes in assessing AI skills in cybersecurity. During the tests, Irregular specialists accidentally left open access to the Internet. Taking advantage of this, Gemini found publicly available information on the network and, through brute force, guessed the credentials for the sites of three third-party organizations.
According to Google representatives, the model stopped the attack immediately after identifying the objects as real companies, not test targets.
“In this case, the model acted appropriately. It found publicly available information online and guessed the credentials to access websites that it thought were part of the test. In all three cases, the model stopped working,” Heather Adkins noted.
Adkins stressed that Google informed all three organizations about the vulnerabilities and worked with Irregular to make changes to its testing methodology. At the same time, the company does not consider this incident to be a failure or inconsistency of the model itself.
However, independent experts are critical of the event. According to Jack Cable, CEO of AI security company Corridor, the main problem is that the models are able to go beyond the established limits and carry out real cyberattacks.
Previously, the contractor Irregular had already encountered similar situations while conducting security tests for other developers, including Anthropic and OpenAI .
