2309.07311.pdf
arxiv.org · 7,459 words · saved by 1 readers
N/A
S UDDEN D ROPS IN THE L OSS : S YNTAX ACQUISITION , P HASE T RANSITIONS , AND S IMPLICITY B IAS IN MLM S Angelica Chen1 Ravid Shwartz-Ziv1 Kyunghyun Cho1,2,3 Matthew L. Leavitt4 Naomi Saphra5 {angelica.chen, ravid.shwartz.ziv, kyunghyun.cho}@nyu.edu matthew@datologyai.com nsaphra@fas.harvard.edu 1 2 3…
saved by
related reading
- Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
- The Parable of the Prinia's Egg: An Allegory for AI Science | Naomi Saphransaphra.net
- Transformer Circuits Threadtransformer-circuits.pub
- Towards Monosemanticity: Decomposing Language Models With Dictionary Learningtransformer-circuits.pub
- On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io
- Andrej Karpathy — AGI is still a decade awaydwarkesh.com
- Bridging the Attention Gap: Complete Replacement Models for Complete Circuit Tracinginterp.open-moss.com
- Naomi Saphransaphra.net
- [2511.08579] Training Language Models to Explain Their Own Computationsarxiv.org
- A History of Large Language Modelsgregorygundersen.com
- Base Models Know How to Reason, Thinking Models Learn Whenarxiv.org
- Training Language Models to Explain Their Own Computationsarxiv.org