[2606.18667] Can neurons speak? Semantic narration of vision at single-cell resolution

Abstract:Identifying what individual neurons encode in higher-order visual cortex is an open problem. Responses resist intuitive parameterization, and the deep-network embeddings used in their place are black boxes. Here, we introduce NEURRATOR, a framework that decodes spiking activity into free-form natural-language narration of the viewed scene at single-neuron resolution. A learned encoder maps spike trains from arbitrary subsets of simultaneously-recorded neurons into the patch-embedding space of a frozen CLIP, from which a multimodal language model and sparse autoencoder generates and validates a description with no language-side training. Applied to Neuropixel recordings of mouse visual cortex during natural-movie viewing, NEURRATOR narrates from thousands of neurons, singular cortical regions, local populations, or from a molecularly-defined cell-types. We use this property to (i) quantify how decoding fidelity scales with population size and cortical region, and (ii) "neurrate", in plain language, what individual neurons and genetically-tagged inhibitory cell-types contribute to visual representation. This recasts cell identity from a classification target into a functional probe of the visual system, providing a new unit of biological insights in neural systems.

Can neurons speak? Semantic narration of vision at single-cell resolution Arnau Marin-Llobet Richard Hakim Sara Matias Harvard University Kempner Institute Center for Brain Science Harvard University Harvard University arXiv:2606.18667v1 [q-bio.NC] 17 Jun 2026…

saved by

• Yoyo Yuan

related reading

• Towards a “universal translator” for neural dynamics at single-cell, single-spike resolutionarxiv.org
• The worminator project postpreprintarxiv.org
• Verbalizable Representations Form a Global Workspace in Language Modelstransformer-circuits.pub
• Semantic reconstruction of continuous language from non-invasive brain recordings | Nature Neurosciencenature.com
• [2409.06843] Universal scale-free representations in human visual cortexarxiv.org
• Language models can explain neurons in language modelsopenaipublic.blob.core.windows.net
• Semantic encoding during language comprehension at single-cell resolution | Naturenature.com
• [2305.18274] Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priorsarxiv.org
• Language Model Circuits Are Sparse in the Neuron Basis | Transluce AItransluce.org
• Multimodal Neurons in Artificial Neural Networksdistill.pub
• High-level visual representations in the human brain are aligned with large language models | Nature Machine Intelligencenature.com
• Dimensionality reduction in neural data analysisxcorr.net