Topic: implicit regularization

Recent studies show that, even in constant environments, the tuning of single neurons to outside world variables gradually changes over time in a variety of brain regions [1, 2, 3, 4, 5]. This phenomenon, termed representational drift, has changed the way we think about the stability of memory and perception, but its driving forces and properties are still mostly unknown. We combine data analysis and modeling to show that implicit regularization can partially explain drift. We train an artificial neural network on a predictive coding task. We continue training in the presence of noise after go

In vivo, single neurons have high trial-to-trial variability in their information transmission, which presents a challenge for networks whose outputs must be consistent for high task performance. Reliability becomes increasingly difficult to attain for working memory tasks where information must be preserved for extended periods of time. This implies that relative to systems without noise, the brain must have mechanisms dedicated to compensating for unreliability in neuron activity. How trained networks mitigate the effects of noise while completing working memory tasks remains largely unexplo