Your Transformer is Secretly an EOT Solver | Elements of a Vector Space

How the search for a simple approximation revealed a beautiful, exact solution.

Nine months ago, or was it ten? I've learned not to trust my sense of time when deep in a problem. Anyone faced with a major deadline or enveloped in the mire of one particular question for an extended period intuitively understands that Time becomes something else then. Not the measured progression of hours, but rather a kind of thickness, a medium through which one moves with increasing difficulty, like a swimmer who has ventured too far from shore and finds the water has become viscous, resistant. I was doing what I had seen others do, what perhaps I had been doing all along without quite a

saved by

• Kushal Thaman
• Vihan Tiwari
• Ishaan Panigrahi

related reading

• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• An Intuition for Attention | Jay Modyjaykmody.com
• Everything About Transformerskrupadave.com
• transformer_attention.pdfarxiv.org
• Attention Is Off By One – Evan Millerevanmiller.org
• Transformers from Scratche2eml.school
• Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• The Annotated Transformernlp.seas.harvard.edu
• Quick start guide - POT Python Optimal Transport 0.9.5 documentationpythonot.github.io
• 1706.03762arxiv.org