Linear Algebra

David Persson presents Polar Express, a method for polar decomposition and the matrix sign function motivated by the Muon neural-network optimizer. It uses matrix multiplications suited to GPUs and adapts each polynomial update through a minimax problem to reduce worst-case error. The talk covers convergence, finite-precision implementation in bfloat16 and validation-loss improvements when training GPT-2 on FineWeb data. This is an in-person PACM IDeAS seminar at Princeton.

The Society for Industrial and Applied Mathematics convenes its triennial conference on applied and numerical linear algebra. The programme covers matrix computations, machine learning, quantum simulation, inverse problems, high-performance computing, dynamical systems, model reduction, optimal transport, network science, tensor methods and related applications. Proposal submissions are currently open, with minitutorial and minisymposium proposals due 26 October 2026 and contributed presentation abstracts due 23 November 2026.

Networks of excitatory and inhibitory (EI) neurons form a canonical circuit in the brain. Classical theoretical analyses of dynamics in EI networks have revealed key principles such as EI balance or paradoxical responses to external inputs. These seminal results assume that synaptic strengths depend on the type of neurons they connect but are otherwise statistically independent. However, recent synaptic physiology datasets have uncovered connectivity patterns that deviate significantly from independent connection models. Simultaneously, studies of task-trained recurrent networks have emphasized the role of connectivity structure in implementing neural computations. Despite these findings, integrating detailed connectivity structures into mean-field theories of EI networks remains a substantial challenge. In this talk, I will outline a theoretical approach to understanding dynamics in structured EI networks by employing a low-rank approximation based on an analytical computation of the dominant eigenvalues of the full connectivity matrix. I will illustrate this approach by investigating the effects of pair-wise connectivity motifs on linear dynamics in EI networks. Specifically, I will present recent results demonstrating that an over-representation of chain motifs induces a strong positive eigenvalue in inhibition-dominated networks, generating a potential instability that challenges classical EI balance criteria. Furthermore, by examining the effects of external input, we found that chain motifs can, on their own, induce paradoxical responses, wherein an increased input to inhibitory neurons leads to a counterintuitive decrease in their activity through recurrent feedback mechanisms. Altogether, our theoretical approach opens new avenues for relating recorded connectivity structures with dynamics and computations in biological networks. Presented in the van Vreeswijk Theoretical Neuroscience Seminar series (formerly WWTNS) on 2025-01-22. Recording duration: 00:47:27.

David Persson presents Polar Express, a method for polar decomposition and the matrix sign function motivated by the Muon neural-network optimizer. It uses matrix multiplications suited to GPUs and adapts each polynomial update through a minimax problem to reduce worst-case error. The talk covers convergence, finite-precision implementation in bfloat16 and validation-loss improvements when training GPT-2 on FineWeb data. This is an in-person PACM IDeAS seminar at Princeton.

The Society for Industrial and Applied Mathematics convenes its triennial conference on applied and numerical linear algebra. The programme covers matrix computations, machine learning, quantum simulation, inverse problems, high-performance computing, dynamical systems, model reduction, optimal transport, network science, tensor methods and related applications. Proposal submissions are currently open, with minitutorial and minisymposium proposals due 26 October 2026 and contributed presentation abstracts due 23 November 2026.