Scientists have developed a method to reconstruct videos from the brain activity of mice, showing exactly what the animals were viewing. This work, led by researchers at University College London, focuses on the visual cortex, which is the brain area that handles sight. Previous studies used functional magnetic resonance imaging, or fMRI - a scan that tracks brain activity through blood flow changes - in humans to decode visual signals at a pixel level, but these were less precise. The new approach relies on single-cell recordings, which measure the activity of individual neurons, or brain cells, offering finer detail.
The researchers adapted a dynamic neural encoding model, a computer program that predicts how neurons react to videos. This model considers factors like the mouse's movements and pupil diameter, which indicates eye focus. They trained it using data from mice watching movies, where neuron firing was detected via calcium levels - chemical boosts that signal when cells activate. By starting with a blank screen prediction and comparing it to real brain responses, an algorithm adjusted pixels, the tiny dots making up images, to match the original video.
Advancing visual brain research
Once refined, the model recreated 10-second video clips solely from neural data of unseen footage. Accuracy increased with more neuron inputs, and reliability was checked using pixel correlation, a measure of how closely pixels in the original and rebuilt videos match. Timing between the two was nearly identical, though resolution and scene coverage need improvement.
This technique highlights how brain representations of sight can differ from reality, not as errors but as useful adaptations that shape interpretation. Future plans include exploring these differences to understand visual processing better, such as how the brain enhances or alters sensory input. The findings could extend to studying perception in other animals and humans, providing a tool to investigate deviations in neural responses to visual cues.
The scientists hjave described the methods and results of this study in a paper published in eLife.