mHC: Manifold-Constrained Hyper-Connections | alphaXiv

View 3 comments: Evidence document: As I mentioned in the public comment, HC paper has serious logical confusion, experimental error and many other problems. Each paper has its own source of inspiration, and the...

Overview Manifold-Constrained Hyper-Connections (mHC) represents a significant advancement in deep learning architecture design that addresses critical stability issues plaguing modern large language models. The work, developed by researchers at DeepSeek-AI, tackles fundamental problems with expanded residual connection architectures that have limited their practical application in large-scale model training. Figure 1: Comparison of (a) traditional residual connections, (b) Hyper-Connections (HC), and (c) the proposed Manifold-Constrained HC (mHC). The green boxes indicate the manifold…

saved by

• Ali Ahmad

related reading

• arxiv.org/pdf/2512.24880#page=3.56arxiv.org
• [2512.24880] mHC: Manifold-Constrained Hyper-Connectionsarxiv.org
• The Practitioner’s Guide to the Maximal Update Parameterization - Cerebrascerebras.ai
• Hyper-Connectionsarxiv.org
• DeepSeek and the Day Before New Year'swheremachinesthink.substack.com
• Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
• Modular Manifolds - Thinking Machines Labthinkingmachines.ai
• The Decade of Deep Learning | Leo Gaobmk.sh
• frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
• bachlechner21a.pdfproceedings.mlr.press
• Scaling Laws, Carefully | Lil'Loglilianweng.github.io
• Residual neural network - Wikipediaen.wikipedia.org