Dopamine release in the nucleus accumbens during backward conditioning

Temporal-difference reinforcement learning proposed that dopamine signals reward prediction errors, which backpropagate scalar values inherent in rewards to reward-predictive cues. This theory is supported by studies of dopamine activity in cue-reward learning, where a reward-paired cue acquires the dopamine response previously evoked by the reward. However, dopamine has recently been implicated in backward reward-cue learning. Specifically, a recent optogenetic study used a backward conditioning paradigm, where sensory cues are presented following the delivery of rewards, and showed that inhibition of dopaminergic neurons at the onset of backward cues during learning disrupted the use of these cues to guide actions. This showed the necessity of dopamine neurons in backward reward-cue learning. However, the dynamics of dopamine release during backward conditioning are unclear. Here, we used fiber photometry recordings to examine dopamine release while rats performed a backward conditioning task. We measured dopamine release in the nucleus accumbens using GRABDA, a dopamine biosensor. In cue-reward learning (i.e., forward-conditioning task), dopamine release progressively decreased at reward delivery and increased at the onset of forward cues, consistent with well-established findings. However, in the backward-conditioning task, the response to reward increased across time. Further, dopamine release at the onset of backward cues was initially high and diminished with additional training. In order to further examine dopamine response to backward cues, we performed a summation test where a forward cue was put in compound with a backward cue predicting the same or different outcome. We found stronger dopamine responses to the compound of forward and backward cues, regardless of whether they predicted the same outcome, revealing a general excitatory response to the backward cues when delivered unexpectedly. These results suggest dopamine release contributes to backward reward-cue associations, demonstrating dopamine acts as a universal teaching signal for learning regardless of motivational significance of the predicted outcome.