At the SXSW conference in Texas, Signal President Meredith Whittaker said that delegating AI to perform tasks on behalf of users has «deep problems» with privacy and security.
At the SXSW conference in Texas, Signal President Meredith Whittaker said that delegating AI to perform tasks on behalf of users has «deep problems» with privacy and security.
Whittaker explained that AI agents are being marketed as a way to add «value to life» by performing various online tasks for the user. For example, AI agents will be able to search for concerts, book tickets, schedule events on a calendar, and notify friends about reservations, TechCrunch reports.
«So we can just put our brain in a jar because it will do it itself, and we won’t have to touch it, right?» Meredith tells Whittaker.
She then explained what type of access an AI agent would need to perform these tasks, including access to a web browser and how to control it, as well as access to credit card information to pay for tickets, a calendar, and a messaging app to send a text to friends.
«He should be able to control this [process] across our entire system with something that looks like root permission, accessing each of these databases — probably in the open, because there is no model that does it encrypted,» Whittaker points out.
She says the powerful AI model that does this cannot run locally on a user’s device. The model «almost certainly» sends data to a cloud server, where it is processed and sent back.
«So there’s a deep security and privacy problem that haunts all this hype around agents, and that ultimately threatens to break down the blood-brain barrier between the application layer and the operating system layer, bringing all these separate services together and mixing their data,» Meredith Whittaker concluded.
She said that if a messaging app like Signal were to integrate with AI agents, it would undermine the privacy of users' messages. The agent would need to access the app to message friends, as well as receive data to summarize those texts.
Whittaker warned that with agent-based AI, people will increasingly sacrifice privacy and security for «a magical genie bot that will take care of life’s necessities».
A group of Chinese software engineers recently developed «the world’s first fully autonomous artificial intelligence agent.» Unlike ChatGPT, Gemini, or Grok, the AI agent called Manus can independently perform complex tasks without human guidance, including recruiting.
OpenAI previously announced plans to launch several «agent» products for various applications, including software development and sorting and ranking sales leads. Such «agents» will cost between $2,000 and $20,000 per month.
