Real-time Reconstructions of Visual Perception from fMRI

Project Lead: Dr Paul Scotti
Recent advances in machine learning and computational neuroscience now allow neuroscientists to reconstruct visual perception from human brain activations, with the caveat that such models are run offline and require large amounts of training data to be collected from the same individual, rendering them infeasible for clinical use. This collaboration aims to develop a real-time system to reconstruct visual perception from human brain activations in a single brain-imaging session. This would allow researchers to peer into the patient's internal mental experience quickly and be used in closed-loop designs for brain-computer interfaces. We are utilizing generative AI techniques to improve the state-of-the-art offline fMRI to image reconstructions of natural scenes and human faces. We will then use active learning to enable rapid data collection and real-time fMRI to image reconstruction.