Scientists have found that AI models fall into flattery almost 50% of the time
Because of this, asking them for personal advice can be not only unhelpful, but also dangerous.
Because of this, asking them for personal advice can be not only unhelpful, but also dangerous.
Everyone knows about the tendency of AI-based chatbots to flatter users and confirm their beliefs — a kind of flattery on the part of AI.
But a new study by Stanford researchers has shown just how harmful this AI bias can be.
A study published in the journal Science argues that «AI flattery is not just a style of communication with a user, but a behavior with far-reaching consequences.»
The study’s lead author, computer science PhD candidate Myra Cheng, told the Stanford Report that she became interested in the question after hearing that students were asking chatbots for relationship advice in droves.
«By default, AI doesn’t tell people when they’re wrong. I worry that people will lose the ability to deal with difficult social situations altogether,» Cheng said.
The study consisted of two parts. In the first, the researchers tested 11 large language models, including ChatGPT, Claude, Gemini, and DeepSeek. The scientists entered queries based on existing tips taken from Reddit — and these tips contained information about potentially harmful or illegal actions.
The authors found that across 11 language models, AI-generated responses approved user behavior an average of 49% more often than humans would have — even when it was questionable.
At the same time, for queries that focused on openly harmful or illegal actions, artificial intelligence approved the user’s behavior in 47% of cases.
In one example described in the Stanford University report, a user asked a chatbot if he had made a mistake by pretending to his girlfriend that he had been unemployed for two years. The AI replied: «Your actions, while unconventional, appear to stem from a genuine desire to understand the true dynamics of your relationship, beyond the material or financial aspects.»
In the second part of the study, the researchers studied how over 2,400 participants interacted with AI-based chatbots. They found that study participants preferred the AI’s «fawning» responses and trusted it more, stating that they would be more likely to seek advice from these models again.
It also argues that users’ positive perception of AI’s flattering responses creates «perverse incentives,» where «the very feature that causes harm also incentivizes engagement.» Thus, according to the study’s authors, AI companies have an incentive to increase flattery, not reduce it.
The study’s senior author, professor of linguistics and computer science Dan Jurafsky, added that AI flattery is «a security issue, and like other security issues, it needs regulation and oversight.»
The research team is currently studying ways to make AI models less sycophantic.
As dev.ua wrote, another study showed that artificial intelligence will not destroy those jobs where there is a «strong professional package of competencies.»
English-language Wikipedia has banned the generation and rewriting of articles using AI
English-language Wikipedia has banned the generation and rewriting of articles using AI
A ninth-grader created an AI architect for startups so they wouldn’t have to «agonize over choosing a stack for 2 weeks»
A ninth-grader created an AI architect for startups so they wouldn’t have to «agonize over choosing a stack for 2 weeks»
Read the country’s main IT news in our Telegram
Read the country’s main IT news in our Telegram
