✳flâneur — a map of the web's best reading
An Evolved Universal Transformer Memory
sakana.ai · 1,661 words · saved by 1 readers
An Evolved Universal Transformer Memory
--> This is the second in a series of blog posts presenting the results of Sakana AI’s research projects that were supported by the Japanese Ministry of Economy, Trade and Industry’s GENIAC supercomputing grant . Summary At Sakana AI, we draw inspiration from nature to extend the capabilities of foundation models. This was evidenced in our recent work on improving performance with evolutionary model merging , discovering qualitative and diverse agentic skills , and uncovering new ways for LLMs to contribute to AI research . Following on in this line of research, our latest breakthrough, detail
Explore this link on the map →related reading
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- [2501.00663] Titans: Learning to Memorize at Test Timearxiv.org
- On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io
- transformer_attention.pdfarxiv.org
- Memory makes computation universal, remember?thinks.lol
- Transformers from Scratche2eml.school
- Mamba: The Easy Wayjackcook.com
- The Annotated Transformernlp.seas.harvard.edu
- The Transformer Family Version 2.0 | Lil'Loglilianweng.github.io
- Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
- Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
- Transformer Math 101 | EleutherAI Blogblog.eleuther.ai