Parallel synapses with transmission nonlinearities increase the neuronal classification capacity

Recent experimental results have found that a cortical neuron typically has multiple synaptic contacts with a given postsynaptic neuron. Here, we refer to such synaptic connections with the same presynaptic axon and the same postsynaptic neuron as parallel synapses. Whether they provide a functional benefit is currently under investigation. To avoid functional redundancy, these parallel synapses must have different computational properties. To model this, we assume that the amount of neurotransmitter released by each synapse can be described by a sigmoidal transmission function of its presynaptic input. These transmission functions' size, slope, and threshold can be learned via gradient descent. We determine the classification capacity of a neuron with such nonlinear parallel synapses, defined as the maximum number of random patterns it can learn in a typical binary classification task. Given a small number of parallel synapses on each axon, this quantity exceeds the perceptron capacity of a neuron with only linear synaptic connections, which is twice the number of presynaptic axons.
Furthermore, if the number of parallel synapses is unconstrained during training, the model neuron can effectively implement almost arbitrary aggregate transmission functions for each axon, as long as they are monotonically increasing. In this case, we show that the capacity achievable by our learning algorithm is even larger than with a number of parallel synapses that’s limited during the learning process. We also found that this capacity appears to grow somewhat faster than linearly with the number of presynaptic axons. Interestingly, however, the number of parallel synapses needed to implement the solutions resulting from successful learning in this model is typically still relatively small. Altogether, our study shows that multiple parallel and nonlinear synapses connecting each input axon to a postsynaptic neuron can enhance its capacity for learning and memory.