Sanjay Amirthraj
1 followers · 6 following · 290 views
on the atlas · 4
- AI: A Comprehensive Guide to 'Attention Is All You Need' in Transformers - Deep AI — Leading Generative AI-powered Solutions for Business1 savers
- Towards Monosemanticity: Decomposing Language Models With Dictionary Learning23 savers
- Toy Models of Superposition30 savers
- A Mathematical Framework for Transformer Circuits40 savers
highlights · 3
Key and query composition are very different from value composition
A Mathematical Framework for Transformer CircuitsInceptionV1 Circuits work
A Mathematical Framework for Transformer CircuitsZero layer transformers model bigram statistics. The bigram table can be accessed directly from the weights.
A Mathematical Framework for Transformer Circuits