Understanding why neurons respond to particular features of visual stimuli
Sensory inputs are used to guide future actions. For example, when returning your opponent’s serve in a game of tennis, it is essential to be able to predict the ball’s position at the future time when you aim to hit it. A paper just published in eLife by Andrew King’s group suggests that the brain may represent the sensory world in a way that best predicts future input. This work builds on their earlier computational study showing that an artificial network of neurons trained to predict the next few video frames from their immediate past resulted in stimulus preferences – in particular, a preference for moving edges – that resembled those of real neurons recorded in the primary visual cortex.
The stimulus features favoured by neurons in the visual system change from flashing spots of light in the retina, to moving edges in the primary visual cortex, to more complex moving textures in higher cortical areas. The new study by Yosef Singer, Luke Taylor, Ben Willmore, Andrew King and Nicol Harper developed a hierarchical version of their earlier network model by iteratively applying the temporal prediction principle to the activity of successive stacks in the model. They found that this replicated the emergence of progressively more complex receptive field properties that takes place along the visual pathway, from the retina to the visual cortical areas in the brain. The temporal prediction model was found to outperform most other current computational models of the visual system in its capacity to predict the responses of real neurons to movies.
Nicol Harper comments, 'As ChatGPT is trained to predict the next word in text given past words, the artificial brain cells of our model were trained to predict the next frame in movies of natural scenes from past frames, or the next pattern of artificial brain activity from past activity. Remarkably, when we compared our model to the real brain, we saw that it reproduced to a notable degree the response properties of neurons along the visual pathway from the eye to visual cortical areas. This includes how brain cells respond to moving lines, patterns and even the visual motion in snippets from movies like Goldfinger and Shakespeare in Love. Our findings highlight the power of this simple prediction principle to explain many aspects of the sensory brain.'
The group has also shown that the temporal prediction principle can account for the preferences of neurons in the auditory system for particular sound features. This work therefore provides further evidence that temporal prediction may be a general principle underlying sensory coding in the brain.