Learning in the brain is thought to be largely unsupervised since the bulk of an animal’s natural experience arrives neither with associated reinforcement signals nor explicit supervisory signals. A compelling account of unsupervised learning in animals is provided by the notion that the brain learns by trying to predict the future, thereby identifying statistical redundancies in sensory inputs over time. Based on this principle, self-supervised learning (SSL) techniques have enabled deep artificial neural networks (ANNs) to achieve competitive results on challenging perceptual tasks. SSL attempts to minimize the representational distance between inputs that are semantically related or temporally adjacent, while maximizing the distance between inputs that are not (negative samples). Importantly, this objective enables learning even when applied in a layer-local manner, suggesting that local learning rules derived from SSL objectives could provide a novel approach for understanding unsupervised learning in the brain. However, a central problem with this interpretation is the difficulty of envisioning how biological systems could access negative samples, without which current SSL methods suffer from representational collapse, wherein ANNs learn a trivial mapping from every input to a single representation vector. Here, we address this issue by developing negative-sample-free SSL objectives that remain amenable to optimization through local learning rules. Moreover, we show that the resulting learning rules are closely related to classical Hebbian plasticity, and primarily differ in the additional predictive term. We demonstrate through a series of tasks that the predictive term steers learning towards prioritizing slow, predictive features whereas the Hebbian term prevents collapse. Furthermore, we show that this mechanism is crucial for learning complex features in deep ANNs. In summary, we develop learning rules for non-contrastive SSL that are largely consistent with Hebbian plasticity, with key differences which indicate novel mechanisms that might be crucial for functional representation learning in the brain.