Stable Diffusion with Brain Activity

Accepted at CVPR 2023 Yu Takagi* 1,2 , Shinji Nishimoto 1,2 1. Graduate School of Frontier Biosciences, Osaka University, Japan 2. CiNet, NICT, Japan

High-resolution image reconstruction with latent diffusion models from human brain activity Accepted at CVPR 2023 1. Graduate School of Frontier Biosciences, Osaka University, Japan 2. CiNet, NICT, Japan Abstract Reconstructing visual experiences from human brain activity offers a unique way to understand how the brain represents the world, and to interpret the connection between computer vision models and our visual system. While deep generative models have recently been employed for this task, reconstructing realistic images with high semantic fidelity is still a challenging problem.…

saved by

• Sara Jafroudi

related reading

• Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priorsmedarc-ai.github.io
• AI to Interpret Your Mindjonathanxu.com
• Retrieving and reconstructing conceptually similar images from fMRI with latent diffusion models and a neuro-inspired brain decoding model - IOPscienceiopscience.iop.org
• [2305.18274] Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priorsarxiv.org
• MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Datamedarc-ai.github.io
• GitHub - nkmjm/mental_img_recon: Mental image reconstruction from human brain activity · GitHubgithub.com
• 2306.16934.pdfarxiv.org
• What are Diffusion Models? | Lil'Loglilianweng.github.io
• MinD-Vismind-vis.github.io
• The Illustrated Stable Diffusion – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Frontiers | End-to-End Deep Image Reconstruction From Human Brain Activityfrontiersin.org
• What are Diffusion Models?lilianweng.github.io