The Rules-and-Facts Model for Simultaneous Generalization and Memorization in Neural Networks
Lenka Zdeborová · École Polytechnique Fédérale de Lausanne (EPFL)
Wed, Sep 30, 2026 · 15:00 UTC
Lenka Zdeborová introduces the Rules-and-Facts model, in which some observations follow a shared rule while others are isolated exceptions requiring memorization. The framework studies when a learner can acquire the rule and retain those exceptions simultaneously. Its results emphasize how capacity is organized and deployed, rather than capacity alone. The talk examines how regularization and the geometry of kernels or learned feature maps can reserve resources for memorization without undermining rule learning. It connects the balance between abstraction and memory to architectural and algori