Semantic reconstruction of continuous language from non-invasive brain recordings | Nature Neuroscience
Tang et al. show that continuous language can be decoded from functional MRI recordings to recover the meaning of perceived and imagined speech stimuli and silent videos and that this language decoding requires subject cooperation.
Subjects Functional magnetic resonance imaging Language Neural decoding Abstract A brain–computer interface that decodes continuous language from non-invasive recordings would have many scientific and practical applications. Currently, however, non-invasive language decoders can only identify stimuli from among a small set of words or phrases. Here we introduce a non-invasive decoder that reconstructs continuous language from cortical semantic representations recorded using functional magnetic resonance imaging (fMRI). Given novel brain recordings, this decoder generates intelligible word sequ
saved by
related reading
- Verbalizable Representations Form a Global Workspace in Language Modelstransformer-circuits.pub
- Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priorsmedarc-ai.github.io
- AI to Interpret Your Mindjonathanxu.com
- Brain2Qwerty — Decoding typed sentences from non-invasive brain activityfacebookresearch.github.io
- Toward accessible, real-time brain decoding: Introducing ENIGMA | Alljoined Blogalljoined.com
- How we collected 10,000 hours of neuro-language data in our basement - Conduitcondu.it
- Retrieving and reconstructing conceptually similar images from fMRI with latent diffusion models and a neuro-inspired brain decoding model - IOPscienceiopscience.iop.org
- Evidence that recurrent circuits are critical to the ventral stream’s execution of core object recognition behaviornature.com
- Semantic encoding during language comprehension at single-cell resolution | Naturenature.com
- [2606.18667] Can neurons speak? Semantic narration of vision at single-cell resolutionarxiv.org
- High-level visual representations in the human brain are aligned with large language models | Nature Machine Intelligencenature.com
- [2305.18274] Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priorsarxiv.org