Topic: memorization

Lenka Zdeborová introduces the Rules-and-Facts model, in which some observations follow a shared rule while others are isolated exceptions requiring memorization. The framework studies when a learner can acquire the rule and retain those exceptions simultaneously. Its results emphasize how capacity is organized and deployed, rather than capacity alone. The talk examines how regularization and the geometry of kernels or learned feature maps can reserve resources for memorization without undermining rule learning. It connects the balance between abstraction and memory to architectural and algori

In recent years, my lab and others have demonstrated the value of vector symbolic algebras (VSAs) for capturing a wide variety of neural and behavioural results. In this talk I discuss the surprising and compelling variety of tasks and styles of reasoning that are well-suited to descriptions using a specific VSA. These tasks include path integration, navigation, Bayesian reasoning, sampling, memorization, and logical inference. The resulting spiking neural network models capture various hippocampal cell types (grid, place, border, etc.), behavioural errors, and a variety of observed neural dyn

Our recent work presents a stochastic process model of the activations within an ANN and shows a promising indicator to distinguish memorizing from generalizing ANNs. The average λ, or mean firing rate (MFR), of a hidden layer, shows stable differences between memorizing and generalizing networks, comparatively independent of the underlying data used for evaluation. We first show the performance of this indicator during training on benchmark computer vision datasets such as MNIST and CIFAR-10. In a second step, we extend the work to the real-life use case of calibrating a pre-trained model to