Reward prediction error neurons implement an efficient code for value

Dopaminergic reward prediction error neurons in the midbrain are the most prominent reward representation in the brain. To understand why these neurons represent rewards the way they do when stochastic rewards are encountered, we derive the optimal population of neurons to encode rewards drawn from a given distribution. To do so, we extend an existing framework for efficient coding to decouple the density of neurons from their slopes. We compare the optimally efficient population to measurements of reward prediction error neurons. We find three aspects of the optimized population confirmed: First, RPEN thresholds cover the range of the distribution with an emphasis on higher thresholds. Second, RPENs with higher thresholds have higher gains. And third, the asymmetry of RPENs' responses around their thresholds depends on their threshold switching from concave to convex with growing threshold. The shift of the thresholds compared to the reward distribution and the increase of gain with threshold were not explained so far. The change in curvature around threshold was recently used as evidence for distributional reinforcement learning, but is explained by efficient coding as well. Thus, reward prediction error neurons may broadcast an efficient reward signal.