Self-timed self-supervised learning

Life can be easier if one knows the structure of the world, for instance, that a distant roar, a whiff of a heavy musky smell, and black stripes on orange background are caused by a single physical entity. Indeed, the question how such structures can be discovered by the neural networks of the brain has challenged neuroscientists for many decades. A key constraint is that central nervous systems must learn about the structure of the world from observing correlations within continuous streams of spikes that arrive from their sensory peripheries. Recently, a novel family of unsupervised spiking neural network models, self-supervised neural networks, have been shown highly potent in discovering ensembles of recurring spike patterns even when their individual occurrences within background noise were rare and temporally asynchronous. In these models an internal supervisory circuit drives learning within a layer of processing neurons by providing teaching signals computed from the processing layer's past activity. A central limitation of these models is their reliance on the presence of an externally given trial structure: The given end of a sensory episode prompts the supervisory circuit to compute its teaching signals and initiate a learning step within the processing layer. Here we develop a self-timed version of self-supervised networks whose teaching circuit requires neither external clock nor trial-end signals, but rather uses the processing layer's activity to also decide the timing of learning steps. We demonstrate that self-timed self-supervised networks match the performance (convergence times) of the original trial-based learning model, substantially broadening the range of settings to which this approach is applicable. We explore the stability of the learning dynamics arising from the interactions between synaptic plasticity and homeostatic mechanisms. Our work provides a biologically plausible unsupervised neural network model for multi-modal learning of recurring spike patterns within parallel continuous sensory streams.