H-Nets - the Future | Goomba Lab

Homepage of the Goomba AI Lab @ CMU MLD.

H-Nets - the Future | Goomba Lab H-Nets - the Future This post is part of a two-part series. H-Nets: the Past H-Nets: the Future [ Paper ] [ Code ] In this post, I’m going to try to convince you why H-Nets are fundamental and important. There was only so much content that could make it to the paper, and I think there are a lot of downstream consequences and interesting technical connections that we didn’t cover. Much of this will be based on deeper (but mostly unvalidated) intuitions I have and speculative implications about H-Nets. For fun, I’ll formulate several concrete hypotheses and predi

related reading

• H-Nets - the Past | Goomba Labgoombalab.github.io
• Aman's AI Journal • Primers • Ilya Sutskever's Top 30aman.ai
• On neural scaling and the quanta hypothesisericjmichaud.com
• The Scaling Hypothesis · Gwern.netgwern.net
• On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io
• frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
• My picture of the present in AI — LessWronglesswrong.com
• H3: Language Modeling with State Space Models and (Almost) No Attention · Hazy Researchhazyresearch.stanford.edu
• I am worried about near-term non-LLM AI developments — LessWronglesswrong.com
• Hyena Hierarchy: Towards Larger Convolutional Language Models · Hazy Researchhazyresearch.stanford.edu
• Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
• The Short Case for Nvidia Stock | YouTube Transcript Optimizeryoutubetranscriptoptimizer.com