kitft/natural_language_autoencoders ·

No description, website, or topics provided.

Natural Language Autoencoders (NLA) Open-source library accompanying the Anthropic Transformer Circuits post Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations . 📄 Blog post · ▶ Video walkthrough · 🔬 Try the released NLAs on Neuronpedia A Natural Language Autoencoder is a pair of fine-tuned LMs that map residual-stream activation vectors to natural language and back: direction mechanism AV (activation verbalizer) vector → text inject the vector as a single token embedding into a fixed prompt, autoregress a description AR (activation reconstructor) text → vecto

related reading

• Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activationstransformer-circuits.pub
• Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations — LessWronglesswrong.com
• GitHub - asherps/EasyNLA: Minimal codebase for efficiently training Natural Language Autoencoders (NLAs). Built on Celeste's nanoNLA: https://github.com/ceselder/nanoNLAgithub.com
• Natural Language Autoencoders \ Anthropicanthropic.com
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• The Annotated Transformernlp.seas.harvard.edu
• GitHub - NVIDIA-NeMo/Automodel: 🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face supportgithub.com
• GitHub - PaulPauls/llama3_interpretability_sae: A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and fully reproducible.github.com
• Llama 2 · Hugging Facehuggingface.co
• Let's build GPT: from scratch, in code, spelled out.youtube.com
• GPT in 60 Lines of NumPy | Jay Modyjaykmody.com
• GitHub - rasbt/LLMs-from-scratch: Implement a ChatGPT-like LLM in PyTorch from scratch, step by stepgithub.com