Topic: Response functions

Deep neural network learning dynamics are very complex with large numbers of learnable weights and many sources of disorder. In this talk, I will discuss mean field approaches to analyze the learning dynamics of neural networks in large system size limits when starting from random initial conditions. The result of this analysis is a dynamical mean field theory (DMFT) where all neurons obey independent stochastic single site dynamics. Correlation functions (kernels) and response functions for the features and gradients at each layer can be computed self-consistently from these stochastic proces

Sensory neural circuits rely on a common set of motifs to process inputs, including convergence of multiple inputs to a single neuron, divergence of inputs into parallel pathways, and nonlinearities that are selective for some inputs over others. Past work has detailed how optimized response nonlinearities and synaptic weights can maximize encoded information, but these solutions depend on tightly tuned response functions and connectivities. Our study found that incorporating generic, non-invertible, selectivity-inducing nonlinearities into a circuit with divergent and convergent structure can

The critical brain hypothesis proposes that neural networks operate near criticality to reap the computational benefits of accessing a wide range of timescales [1]. Proponents of this viewpoint highlight the presence of heavy-tailed (power-law) spatiotemporal correlations as markers of criticality in the brain [2]. Critics of this hypothesis argue that such correlations could be inherited from upstream sources, such as sensory input [3]. Similarly, Ref. [4] constructed a model of independent neurons driven by shared noise input that exhibited neural activity with power-law tails, which could b