asherps/EasyNLA: Minimal codebase for efficiently training Natural Language Autoencoders (NLAs). Built on Celeste's nanoNLA: https://github.com/ceselder/nanoNLA ·
Minimal codebase for efficiently training Natural Language Autoencoders (NLAs). Built on Celeste's nanoNLA: https://github.com/ceselder/nanoNLA - asherps/EasyNLA
EasyNLA — fast, distributed training of Natural Language Autoencoders EasyNLA trains Natural Language Autoencoders (NLAs) — models that read a residual-stream activation and write a natural-language explanation of it, then reconstruct the activation back from that text. The focus here is fast, distributed training. It scales with data parallelism: launch under torchrun with one rank per GPU, each running its own single-GPU vLLM rollout engine + trainer on a slice of the batch, gradients all-reduced every step. Weights resync to the sampler each step over fast GPU→GPU IPC, and the whole NLA…
saved by
related reading
- Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activationstransformer-circuits.pub
- GitHub - kitft/natural_language_autoencoders · GitHubgithub.com
- Natural Language Autoencoders \ Anthropicanthropic.com
- Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations — LessWronglesswrong.com
- Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
- An Intuitive Explanation of Sparse Autoencoders for LLM Interpretability | Adam Karvonenadamkarvonen.github.io
- Parsed | Custom, interpretable AI systems that continuously learnparsed.com
- Neuronpedianeuronpedia.org
- Together AI | The AI Native Cloudtogether.ai
- GitHub - NVIDIA-NeMo/Automodel: 🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face supportgithub.com
- GitHub - PaulPauls/llama3_interpretability_sae: A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and fully reproducible.github.com
- GitHub - karpathy/autoresearch: AI agents running research on single-GPU nanochat training automaticallygithub.com