nnsight — nnsight 0.0.7 documentation
Site Navigation Site Navigation nnsight interpretable neural networks nnsight (/ɛn.saɪt/) is a package for interpreting and manipulating the internals of large models Get Started Tutorials Docs Key Features Integration Pass in a ID from any HuggingFace Transformer repo and access its weights with nnsight. Interpretability Access the internal gradients and activations at any point or module in a model. Intuitive Set up a context block and manipulate model internals with only a couple lines of code. Interoperable Enable grad and train interventions like LORA or probes on any point in a model. Next Getting Started © Copyright 2023, NDIF. Created using Sphinx 7.2.6. Built with the PyData Sphinx Theme 0.14.2.
What is NNsight?¶ NNsight is a Python library that enables interpreting and intervening on the internals of deep learning models. It provides a clean, Pythonic interface for: Accessing activations at any layer during forward passes Modifying activations to study causal effects Computing gradients with respect to intermediate values Batching interventions across multiple inputs efficiently Originally developed by the NDIF team at Northeastern University, NNsight supports local execution on any PyTorch model and remote execution on large models via the NDIF infrastructure. What does…
saved by
related reading
- Neuronpedianeuronpedia.org
- Walkthrough - nnsightnnsight.net
- Language models can explain neurons in language modelsopenaipublic.blob.core.windows.net
- Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
- Transformer Circuits Threadtransformer-circuits.pub
- An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
- Interpretability Infrastructure at Frontier Scale: Harvesting Activations from a Trillion-Parameter Modelgoodfire.ai
- An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2 — AI Alignment Forumalignmentforum.org
- Goodfire AIgoodfire.ai
- Towards Automated Circuit Discovery for Mechanistic Interpretabilityarxiv.org
- Softmax Linear Unitstransformer-circuits.pub
- [2607.14111] Introspection Fine-Tuning (IFT): Training Small LLMs to Introspectarxiv.org