Topic: distributional reinforcement learning

Dopaminergic reward prediction error neurons in the midbrain are the most prominent reward representation in the brain. To understand why these neurons represent rewards the way they do when stochastic rewards are encountered, we derive the optimal population of neurons to encode rewards drawn from a given distribution. To do so, we extend an existing framework for efficient coding to decouple the density of neurons from their slopes. We compare the optimally efficient population to measurements of reward prediction error neurons. We find three aspects of the optimized population confirmed: Fi

Various models and investigations over the years have attempted to pin down a mechanistic explanation for how dopamine (DA) neurons in the brain can exhibit reward prediction error (RPE), usually through direct analogy to temporal difference (TD) learning. However, there are two key hurdles in imagining how TD could plausibly be implemented in the brain. First, TD models of DA learning frequently require arbitrarily constructed and unrealistic components, such as a temporal chain of feature-specific neurons that uniformly tile the time from stimulus onset to reward arrival. Secondly, various p