The US-based Tow Center for Digital Journalism examined eight AI search engines, including ChatGPT Search, Perplexity, Perplexity Pro, Gemini, DeepSeek Search, Grok-2 Search, Grok-3 Search, and Copilot. The study tested each for accuracy and recorded how often the tools refused to respond.
The US-based Tow Center for Digital Journalism examined eight AI search engines, including ChatGPT Search, Perplexity, Perplexity Pro, Gemini, DeepSeek Search, Grok-2 Search, Grok-3 Search, and Copilot. The study tested each for accuracy and recorded how often the tools refused to respond.
The researchers randomly selected 200 news articles from 20 news outlets (10 from each), TechSpot reports . They made sure that each article ranked in the top three results in a Google search when a quoted excerpt from the article was used.
Then, they ran the same query on each AI search engine and scored accuracy based on whether the search correctly cited A) the article, B) the news organization, and C) the URL.
The researchers also rated each search by its degree of accuracy, from "absolutely correct" to "absolutely incorrect."
As you can see from the chart below, with the exception of both versions of Perplexity, AI did not perform well. Overall, AI search engines get it wrong 60% of the time.
ChatGPT seems to be programmed to respond to every user input at all costs. ChatGPT Search was the only AI tool that answered all 200 article queries. However, it only showed 28% accuracy and was completely inaccurate 57% of the time.
Both versions of Grok AI from X performed poorly, with Grok-3 Search proving to be 94% inaccurate.
Microsoft's Copilot wasn't much better, considering it refused to answer 104 of the 200 queries. Of the remaining 96, only 16 were "completely correct," 14 were "partially correct," and 66 were "completely incorrect," representing a roughly 70% inaccuracy.
Perhaps the craziest thing about this situation is that the companies that create these tools don't advertise this lack of accuracy, yet charge users anywhere from $20 to $200 per month for access to their latest AI models.
Moreover, Perplexity Pro ($20/month) and Grok-3 Search ($40/month) gave slightly more correct answers to queries than their free versions (Perplexity and Grok-2 Search), but had a significantly higher error rate.
The researchers also noted that 5 of the 8 chatbots tested in this study (ChatGPT, Perplexity and Perplexity Pro, Copilot, and Gemini) made public the names of their scanners, giving publishers the opportunity to block them, while the scanners used by the other three (DeepSeek, Grok 2, and Grok 3) are unknown.
You can read more about the research in the Tow Center, published in the Columbia Journalism Review.
