Scaling laws

Matthias Vigl examines whether the computational scaling strategies behind modern machine learning can benefit particle physics. Empirical scaling laws relate model performance to computing resources and help balance model size against training-data volume. For a fixed dataset, increasing model capacity beyond the interpolation threshold can improve generalization through double descent, but gains eventually encounter limits imposed by the data. The talk then considers scaling data and model size together, deriving compute-optimal relations for transformer-based jet taggers trained on as many

…The seminar examines scaling laws, the removal of non-learnable field modes, benign overfitting and open questions about feature learning. The confirmed Quantum Matter seminar takes place in the Bob Room at Perimeter Institute on 8 September 2026, from 15:30 to 17:00 Toronto time.

…walks make it possible to investigate scaling laws, sample complexity and unexpected temperature effects in controlled settings. The presentation also considers how parameterisation changes hyperparameter scaling, including the importance of scaling the embedding-layer learning rate.

…understanding generalization or “scaling laws” in learned neural networks as a function of basic control variables. I’ll discuss a taxonomy we develop that classifies different regimes of scaling behavior. We identify regimes where generalization exhibits universal scaling behavior and others where it can be

It is sometimes said that there are two reasons why physics is so successful as a science. One is that it deals with very simple problems. The other is that it attempts to account only for universal aspects of systems at a desired level of description, with lower level phenomena subsumed into a small number of adjustable parameters. It is a widespread belief that this approach seems unlikely to be useful in biology, which is intimidatingly complex, where “everything has an exception”, and where there are a huge number of undetermined parameters. I will try to argue, nonetheless, that there are