Temporal trajectories

…representations that follow straighter temporal trajectories than their initial photoreceptor encoding, facilitating prediction by linear extrapolation (Hénaff et al 2019; 2021). Can the brain use this principle to learn visual representations? Here, we develop an objective that quantifies straightening, augment it with a regularizer to prevent collapse to trivial solutions, and use it to train deep feedforward neural networks on video sequences. Decoding of the learned representation with a separately trained readout

All organisms make temporal predictions, and their evolutionary fitness level generally scales with the accuracy of these predictions. In visual perception, observer motion and continuous deformations of objects and textures imbue our visual input with distinct temporal structures, enabling partial prediction of future inputs from past ones. Inspired by recent hypotheses that primate visual representations support prediction by ``straightening'' the temporal trajectories of naturally-occurring input (Henaff et. al., 2019), we formulate an architecture for image representation that facilitates

In vivo, single neurons have high trial-to-trial variability in their information transmission, which presents a challenge for networks whose outputs must be consistent for high task performance. Reliability becomes increasingly difficult to attain for working memory tasks where information must be preserved for extended periods of time. This implies that relative to systems without noise, the brain must have mechanisms dedicated to compensating for unreliability in neuron activity. How trained networks mitigate the effects of noise while completing working memory tasks remains largely unexplo