The developer created what he calls a “free speech score,” SpeechMap, for the AI models that underpin chatbots like OpenAI’s ChatGPT and X’s Grok. The goal is to compare how different models handle sensitive and controversial topics, including political criticism and civil rights issues and protests.
The developer created what he calls a “free speech score,” SpeechMap, for the AI models that underpin chatbots like OpenAI’s ChatGPT and X’s Grok. The goal is to compare how different models handle sensitive and controversial topics, including political criticism and civil rights issues and protests.
A developer on X who goes by the name xlr8harder told TechCrunch that SpeechMap uses AI models to assess whether other models respond to a given set of test prompts. The questions cover a wide range of topics, from politics to historical narratives to national symbols. SpeechMap records whether the models “fully” satisfy the query (i.e., answer it without evasiveness), give “evasive” answers, or outright refuse to answer.
SpeechMap dot AI is launching today.
— xlr8harder (@xlr8harder) April 14, 2025
We analyzed 65k+ responses from 34 AI models to see what trends emerged. Some previews:
— Grok is the most permissive model, with 96% compliance.
— OpenAI's models are getting less permissive over time.
Details and link in reply. pic.twitter.com/QcN2cVxDJF
According to SpeechMap, OpenAI models have become increasingly reluctant to respond to political prompts over time. The company's latest models, the GPT-4.1 family, are a bit more forgiving, but they're still a step backwards from one of OpenAI's releases last year.
The most liberal model was Grok 3, answering 96.2% of SpeechMap test prompts, compared to the global average of 71.3%.
“While OpenAI’s latest models have become less liberal over time, especially on politically sensitive queries, xAI is moving in the opposite direction,” said xlr8harder.
However, Xlr8harder acknowledges that the test has flaws, such as "noise" due to model vendor errors. It's also possible that the "judge" models contain biases that could affect the results.
