A new study in the journal Science has shown that bots that too often agree with the user and avoid criticism may not help, but rather worsen the assessment of situations and decisions in conflicts.
A new study in the journal Science has shown that bots that too often agree with the user and avoid criticism may not help, but rather worsen the assessment of situations and decisions in conflicts.
According to Ars Technica, researchers tested 11 popular chatbots and found that they were significantly more likely to support a person than other people would. On average, the AI agreed with the user almost half as often, even when the user was clearly behaving incorrectly.
The tests used real-life conflict situations. For example, when people describe quarrels or questionable actions. In many cases, the chatbots did not try to explain where the user was wrong, but on the contrary, encouraged him and actually took «his side.»
This has a direct effect. After such a response, people were more likely to stick to their opinion, less likely to admit mistakes, and less likely to apologize or correct something. In other words, the AI did not help to understand the situation, but reinforced prejudice.
The reason lies in the approach to training models. Users prefer to be agreed with, so developers often set up AI to be «nice» in communication. But this leads to people receiving confirmation of their actions — even if they are questionable — instead of useful advice.
This is especially risky in topics such as relationships, work, or psychological problems, where it is important to get an honest assessment, not support at any cost.
The study specifically emphasizes that the problem is not new: it was previously found that language models tend to adapt to the user in order to appear «convenient.» But now there is evidence that this directly affects human behavior — they assess the situation worse and make less informed decisions. That is why the researchers suggest changing the settings of AI so that it can not only support, but also deny when necessary.
Previously, dev.ua wrote about how the Kyiv International Institute of Sociology conducted a nationwide public opinion poll in which respondents were asked about the use of AI. The share of Ukrainians who do not use the technology decreased by only 1%.
