Learning predictive neural representations by straightening natural videos
Xueyan Niu, Cristina Savin, Eero Simoncelli · COSYNE 2023
Fri, Mar 10, 2023
Recent experiments demonstrate that the brain transforms visual inputs into representations that follow straighter temporal trajectories than their initial photoreceptor encoding, facilitating prediction by linear extrapolation (Hénaff et al 2019; 2021). Can the brain use this principle to learn visual representations? Here, we develop an objective that quantifies straightening, augment it with a regularizer to prevent collapse to trivial solutions, and use it to train deep feedforward neural networks on video sequences. Decoding of the learned representation with a separately trained readout