Trained neural networks

Damian Borth describes learning from trained neural networks themselves as a new source of training data. The conversation explores representations of model weights, transferring knowledge across architectures and tasks, and the possibility of specializing models with less computation by reusing what existing networks have learned.

Recently, the field of computational neuroscience has seen an explosion of the use of trained neural networks (NNs) to model patterns of neural activity. These NN models are typically characterized by tuned interactions between rate 'units' whose dynamics are governed by smooth, continuous differential equations. However, the response of biological single neurons is better described by all-or-none events - spikes - that are triggered in response to the processing of their synaptic input by the complex dynamics of their membrane. One line of research has attempted to resolve this discrepancy by