Meta-Learning the Inductive Biases of Simple Neural Circuits

Animals are presented with noisy and incomplete information from which we learn how to react in novel situations. For example, a given training set could have many explanations, each implying a different behavioural response; acting appropriately requires inferring the most plausible explanation. The way animals choose to wield Occam's razor, selecting the most parsimonious explanation, is called their inductive bias, and it is implicitly built into the operation of animals' neural circuits. This relationship between an observed sensory circuit and its inductive bias is a useful explanatory window for neuroscience, allowing design choices to be understood normatively. However, it is generally very difficult to map circuit structure to inductive bias. In this work we present a neural network tool to bridge this gap by meta-learning the inductive bias of neural circuits. We show that in systems where the inductive bias is known analytically, i.e. linear and kernel regression, it recovers them. Then, we show it is able to flexibly extract inductive biases from differentiable circuits, including spiking neural networks. This illustrates the intended use case of our tool: understanding the role of otherwise opaque pieces of neural functionality, such as non-linearities, learning rules, or connectomic data, through the inductive bias they induce.