ePoster, COSYNE 2022
Bayesian synaptic plasticity is energy efficient
- Lisbon, Portugal
- Computational Neuroscience, Bayesian Statistics and
Abstract
Recent work (Aitchison et al. 2021) suggests that synaptic plasticity performs Bayesian inference and that, moreover, variability in synaptic efficacy might reflect uncertainty in the synaptic weight. Here, we consider whether similar phenomena emerge merely from minimizing synapse energetic costs during learning. Following established biophysical principles, we assume that noisy synaptic transmission is energetically cheaper than precise transmission. However, stochastic synapses may be detrimental to performance because noisy transmission adds randomness to a neural network’s activity, and so may push the network’s input-output function away from its optimum. We explored this dilemma using a combination of analysis of simple models and experiments in artificial neural networks. We found that optimizing a performance-energy trade-off results in low noise for “important” synapses, those for which reliable transmission is crucial to task performance; in contrast greater noise is afforded for less critical synapses. Interestingly, the resulting synaptic noise reflects parameter uncertainty. This implies that energy-efficient synapses should approximate Bayesian inference; we make this connection explicit. In addition, Aitchison et al. (2021) proposed that the learning rate should depend on the Bayesian posterior uncertainty, with more uncertainty leading to faster updates in light of new information. Remarkably, exactly the same phenomenon can be obtained without consideration of Bayesian inference, simply by maximizing performance under assumptions about how quickly the task changes. Jointly maximizing performance and energy efficiency leads to adaptive learning rates and synaptic noise levels, that are dependent on inferred uncertainty.
Topics
- synaptic plasticity, search World Wide
- Bayesian inference, search World Wide
- synaptic efficacy, search World Wide
- noisy synaptic transmission, search World Wide
- synapse energetic costs, search World Wide
- performance-energy trade-off, search World Wide
- artificial neural networks, search World Wide
- adaptive learning rates, search World Wide
- synaptic noise, search World Wide