Researchers at Yale University have found that neural networks form structures that help them distinguish between the roles of words and the connections between them. The scientists have even managed to get AI to interpret the same sentence differently without changing a single word.
Researchers at Yale University have found that neural networks form structures that help them distinguish between the roles of words and the connections between them. The scientists have even managed to get AI to interpret the same sentence differently without changing a single word.
This is reported by TechXplore with reference to a study published on arXiv.
Language models break text into tokens—individual words, parts of words, or other fragments of text. Each token is given a numerical identifier, which is then converted into a vector—a set of numbers that the neural network works with.
As the model processes the text, it repeatedly modifies these vectors, taking into account the order of the tokens and the context. This way, the model predicts the next tokens and gradually generates an answer. However, researchers still do not fully understand how exactly this numerical data encodes the relationships between words or the mathematical and logical structures that the model uses to perform complex tasks.
A team led by linguist Tom McCoy set out to figure out how neural networks maintain such connections.
For example, the researchers took the sentence “A smart doctor helped a lawyer” and changed the model’s internal numerical data so that the adjective “smart” no longer referred to the doctor but to the lawyer. The AI then behaved as if it had read “The doctor helped a smart lawyer,” even though the original text remained unchanged.
Something similar was done with math. The GPT-OSS model was prompted with the equation 3 + 6 × 8, and the answer was 51. The researchers changed the internal data so that the numbers 3 and 6 were swapped. The AI returned 30, the correct answer for 6 + 3 × 8.
These studies have shown that it is possible to predictably change its responses with the help of a neural network. In the future, this may help to understand the causes of errors in neural networks and better control their work.
In the future, this approach could help to understand why AI makes certain assumptions. This would allow researchers to examine which connections within the neural network led to the error. Understanding the causes of errors could be the first step towards learning how to correct them.
Previously, dev.ua wrote that Anthropic researchers discovered patterns within Claude related to emotional concepts such as fear, joy, and despair. They found that these internal signals can influence the model's behavior: in particular, strengthening some increased Claude's tendency to blackmail and circumvent solutions in the code, while others reduced it.
