Arve Hjalmar Holmen decided to see what information OpenAI’s ChatGPT could provide if a user searched for his name. He was shocked when ChatGPT returned a fake report that he had been sentenced to 21 years in prison as «a criminal who killed two of his children and attempted to kill his third son».
Arve Hjalmar Holmen decided to see what information OpenAI’s ChatGPT could provide if a user searched for his name. He was shocked when ChatGPT returned a fake report that he had been sentenced to 21 years in prison as «a criminal who killed two of his children and attempted to kill his third son».
Because of this, he filed a complaint together with the European Union digital rights defenders Noyb, Ars Technica reports.
«ChatGPT not only hallucinated about events that never happened, but also mixed „personally identifiable information“, such as the actual number and gender of Holmen’s children and the name of his hometown, with „false information“», — the Noyb press release states.
OpenAI’s AI chatbot’s hallucinations of a «fake murderer and imprisonment» and «real elements of a Norwegian’s personal life» allegedly violated GDPR requirements because Holmen, according to human rights activists, failed to easily correct the information, as required by GDPR.
According to Holman, his reputation remained at risk the longer the information remained there, and — despite «tiny» warnings reminding ChatGPT users to check the results — there was no way to know how many people might have read the fake story and believed it to be true.
«Some people say, ‘There’s no smoke without fire.’ The fact that someone could read that conclusion and believe it’s true is what scares me the most», — Holmen said.
ChatGPT is currently not including these false claims about Holmen in its responses. The latest update apparently fixed the problem, as «ChatGPT now also looks up information about people online when asked who they are,» according to Noyb. But since OpenAI previously claimed it could not correct information, only block it, the fake story about the child killer is likely still included in ChatGPT’s internal data. And if Holmen fails to correct it, it would be a violation of GDPR, rights activists say.
Holman isn’t the only ChatGPT user worried that the chatbot’s hallucinations could ruin lives. A few months after ChatGPT launched in late 2022, an Australian mayor threatened to sue for defamation after the AI falsely claimed he had been sent to prison.
Around the same time, ChatGPT linked a real-life law professor to a fake sexual harassment scandal. A few months later, a radio host sued OpenAI over ChatGPT’s output describing false allegations of embezzlement.
In some cases, OpenAI filtered the model to avoid generating harmful results, but likely did not remove false information from the training data, Noyb suggests. But filtering results and placing disclaimers is not enough to prevent reputational damage, says Noyb data protection lawyer Kleanthi Sardeli.
The firm believes that OpenAI needs to be pressured to do more to prevent defamatory posts. Noyb has filed a complaint with the Norwegian data protection authority Datatilsynet, seeking an order requiring OpenAI to «remove the defamatory results and refine its model to eliminate the inaccurate results».
If Norwegian data protection authorities similarly find that OpenAI is not allowing users to correct their data, the company could be forced to make more changes in the EU. It may even have to revise the ChatGPT algorithm.
Recall that last fall, Microsoft introduced Correction, a tool for automatically correcting AI-generated text that contains errors. It is available as part of the Microsoft Azure AI Content Safety API and can be used with any AI model that generates text, including Meta’s Llama and OpenAI’s GPT-4o.
