mHC: Manifold-Constrained Hyper-Connections | alphaXiv
View 3 comments: Evidence document: As I mentioned in the public comment, HC paper has serious logical confusion, experimental error and many other problems. Each paper has its own source of inspiration, and the...
Overview Manifold-Constrained Hyper-Connections (mHC) represents a significant advancement in deep learning architecture design that addresses critical stability issues plaguing modern large language models. The work, developed by researchers at DeepSeek-AI, tackles fundamental problems with expanded residual connection architectures that have limited their practical application in large-scale model training. Figure 1: Comparison of (a) traditional residual connections, (b) Hyper-Connections (HC), and (c) the proposed Manifold-Constrained HC (mHC). The green boxes indicate the manifold…
saved by
related reading
- [2512.24880] mHC: Manifold-Constrained Hyper-Connectionsarxiv.org
- arxiv.org/pdf/2512.24880#page=3.56arxiv.org
- NL.pdfabehrouz.github.io
- Modular Manifolds - Thinking Machines Labthinkingmachines.ai
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- Hyper-Connectionsarxiv.org
- Neural Networks, Manifolds, and Topology -- colah's blogcolah.github.io
- DeepSeek and the Day Before New Year'swheremachinesthink.substack.com
- Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
- H-Nets - the Past | Goomba Labgoombalab.github.io
- In-context Learning and Induction Headstransformer-circuits.pub
- Dynamic Chunking for End-to-End Hierarchical Sequence Modelingarxiv.org