Topic: negative samples

Living organisms receive an immense amount of information per unit time, but much of it is of little relevance for behavior. A simple approach to process this high-dimensional input is to focus on a lower-dimensional subspace and ignore the directions that are less informative. Principal component analysis (PCA) achieves this by discarding the directions with low variance. Such an approach is, however, inefficient in cases where the irrelevant directions are very noisy, thus having greater variance than the relevant ones. If we have access to representative samples of variability in irrelevant

Learning in the brain is thought to be largely unsupervised since the bulk of an animal’s natural experience arrives neither with associated reinforcement signals nor explicit supervisory signals. A compelling account of unsupervised learning in animals is provided by the notion that the brain learns by trying to predict the future, thereby identifying statistical redundancies in sensory inputs over time. Based on this principle, self-supervised learning (SSL) techniques have enabled deep artificial neural networks (ANNs) to achieve competitive results on challenging perceptual tasks. SSL atte