A team of researchers from Chinese AI company DeepSeek recently published a paper describing a method called Manifold-Constrained Hyper-Connections, or mHC for short. This will allow developers to build powerful language models while significantly saving on computational resources that were previously considered indispensable for such scales.
A team of researchers from Chinese AI company DeepSeek recently published a paper describing a method called Manifold-Constrained Hyper-Connections, or mHC for short. This will allow developers to build powerful language models while significantly saving on computational resources that were previously considered indispensable for such scales.
DeepSeek came to the fore a year ago with the release of the R1 , a direct competitor to OpenAI's o1, but at a much lower cost. The release came as a shock to American developers , as it proved that you don't need billion-dollar budgets and thousands of servers to create top-notch AI.
The new paper on mHC could become the technological foundation for DeepSeek's upcoming model, the R2, which was scheduled to be released in the middle of last year but was delayed due to China's limited access to advanced AI chips and concerns from company CEO Liang Wenfeng about the model's performance.
This paper, published on the arXiv preprint server — a popular online resource where researchers share the results of work that has not yet been peer-reviewed — is an attempt to bridge a complex and important technical gap that hinders the scalability of AI models, writes ZDNET.
LLMs are built on neural networks, which in turn are designed to store signals across multiple layers. The problem is that the more layers you add, the more the signal can become attenuated or degraded, and the greater the risk of it turning into noise. It’s a bit like playing telephone: the more people you add, the higher the chance that the original message will be obfuscated and altered.
So the main challenge is to create models that can preserve signal strength in as many layers as possible — or, as the DeepSeek researchers put it in their new paper, “better optimize the trade-off between plasticity and stability.”
The authors of the new paper — including DeepSeek CEO Liang Wenfeng — relied on the concept of “hyperlinks” (HCs). This structure was proposed in 2024 by researchers at ByteDance to diversify the channels through which layers of a neural network exchange information with each other. However, hyperlinks create the risk of losing the original signal. In addition, they require significant memory costs, which makes them difficult to implement on a large scale.
The mHC architecture aims to address this issue by limiting the number of hyperlinks in the model, thereby preserving the information complexity provided by HC while avoiding the memory problem. This, in turn, can allow training of very complex models in a way that is practical and scalable even for developers with fewer resources.
As with the R1 release in January 2025, the debut of the mHC framework could hint at a new direction in AI evolution.
Until now, the AI race has been dominated by the notion that only the biggest and richest companies can afford to build cutting-edge models. But DeepSeek consistently demonstrates that workarounds are possible, and that breakthroughs can only be achieved through smart engineering.
The fact that the company has published its new research on the mHC method means that it could be widely adopted by smaller developers, especially if it is used in the long-awaited R2 model (the release date of which has not been officially announced).
