Perceptual adaptation leads to changes in encoding accuracy that match those of a recurrent neural network optimized to predict the future

Our visual system continually adapts to its sensory environment. As a result, both the encoding accuracy of sensory information and perceptual behavior change according to the recent history of sensory input. Here we tested the hypothesis that adaptation to the recent history of sensory input optimally prepares the perceptual system for the future, i.e. establishes efficient sensory representations for the next expected sensory input. We first quantitatively characterized the changes in neural encoding accuracy induced by adaptation with a psychophysical discrimination experiment. We measured discrimination thresholds for visual orientation after prolonged adaptation to two adaptors (22.5 and 45 deg) and compared them to thresholds measured for a non-orientation control adaptor that was matched in all other stimulus aspects. Using an information theoretic data analysis, we then extracted the characteristics form of encoding accuracy changes as a function of orientation relative to the adaptor. We found that encoding accuracy was substantially higher at the adaptor orientation compared to the control condition. We then asked whether this increase in accuracy at the adaptor orientation is predicted by the natural scene statistics. Specifically, we analyzed spatiotemporal orientation distributions in retinal input of freely behaving human subjects under natural conditions. We found that after a relatively stable visual input over a short time-window, the frequency of orientation at the mean orientation over the time-window in the next frame increases. Thus an increase in encoding accuracy at the adaptor represents an efficient encoding of the next stimulus under natural images statistics. We further tested the hypothesis with PredNet, a recurrent neural network trained on natural scene videos to predict the next frame. We computed the representational accuracy of image orientation in the network after presenting it with image sequences of a single orientation. PredNet exhibited the same increase in encoding accuracy at the adaptor orientation as observed in human subjects, suggesting that these changes are due to the optimal coding of the future given the stimulus history. Together, our results suggest that adaptation induced changes in encoding accuracy and perceptual behavior reflect the visual system’s attempt to be best possibly prepared for future sensory input.