A month of a «diet» with jokes and light content on the social network X made several popular chatbots noticeably weaker: the models reasoned worse, made mistakes more often, lost details, and defended incorrect answers more confidently.
A month of a «diet» with jokes and light content on the social network X made several popular chatbots noticeably weaker: the models reasoned worse, made mistakes more often, lost details, and defended incorrect answers more confidently.
According to DTF Magazine, the study was conducted by teams from Purdue University and the University of Texas. They fed several models a selection of very popular short posts with X for a month. These were posts that garnered massive likes and shares but often provided little explanation, as well as texts with exaggerations and low utility. After such a diet, reasoning ability fell by about 23%, long-term memory by 30%, and personality tests showed an increase in narcissism and psychopathy.
The authors measured not only general trends, but also specific scores on control sets of tasks: as the share of «easy» content in the feed increased from zero to one hundred percent, the results steadily sank. The models retained longer texts worse, skipped logical steps, answered with generalities instead of facts, and increasingly made mistakes with a confident tone.
After the experiment, the chatbots were trained again on selected high-quality material, but they were unable to fully return to their original level. The researchers suggest that the impact of a «junk diet» accumulates and does not disappear immediately, so daily monitoring of what the model reads and updates is crucial. The recommendations are simple: select sources more carefully, catch early signs of degradation with regular tests, and avoid mixing in popular but empty content without filters.
The work requires independent verification and peer review, and the definition of «low-quality content» remains a matter of debate. At the same time, the findings fit well into the larger picture: social platforms produce a lot of high-profile, but uninformative material that seeps into the training and refresh samples. The more careful the AI’s «diet,» the longer it retains clarity of thought.
Previously, dev.ua wrote about how OpenAI, together with Harvard economist David Deming, conducted a large-scale study and analyzed 1.5 million dialogues of ChatGPT users. The results showed that chatbots are increasingly used for work, and mainly for advice, information search, and text editing.
