Topic: inductive biases

Stimuli trigger a pattern of activity across neurons in cortex, whose firing rates define a stimulus's representation in a high-dimensional vector space. Learning a visuomotor task can affect the responses of visual cortical neurons, but how and why training modifies population-level representations is unclear. One hypothesis is that representational plasticity in visual cortex facilitates visuomotor associations by downstream motor systems. Learning systems exhibit "inductive biases", meaning they form some stimulus-motor associations more easily than others. An animal's inductive biases pr

Artificial Intelligence (AI) research has provided key insights into the mechanics of learning complex tasks. However, AI models have long overlooked innateness: how strong pressures for survival lead to the encoding of complex behaviors in the nascent wiring of a brain. Although innate neural solutions have inspired AI approaches from layered architectures to ConvNets, the underlying neuroevolutionary search for novel heuristics has not been successfully systematized. In this manuscript, we examine how neurodevelopmental principles can inform the discovery of useful inductive biases. We begin

Artificial neural networks (ANNs) for simulated motor control and robotics often adopt generic architectures like fully connected multi-layer perceptrons (MLPs) and randomly connected recurrent neural networks (RNNs). While general, these tabula rasa architectures rely on large amounts of experience to learn, and their internal dynamics are difficult to interpret. In nature, animals are born with highly structured connectivity in their nervous systems shaped by evolution; this innate circuitry acts synergistically with learning mechanisms to provide inductive biases that enable most animals to

The power of deep neural networks for pattern recognition tasks lies in their universal approximation abilities, and their theoretical expressivity is well-studied. Practical expressivity, however, is known to lag behind theoretical expressivity. Network performance achieved in practice is determined by a set of inductive biases such as architectural constraints (e.g. residual connections). In this work, we introduce weakly coupled residual recurrent networks (WCRNNs) in which the residual connections result in well-defined Lyapunov exponents. This allows for studying properties of their dynam