How does the activity of midbrain dopamine (DA) neurons influence behavior? A prominent, but incompletely tested, hypothesis is that the activity of ventral tegmental area (VTA) DA neurons generates downstream representations of state values in ventral striatum (NAc), while substantia nigra pars compacta DA activity generates action values in the dorsal striatum. To test this model, we performed Neuropixels recordings in a trial-and-error probabilistic learning task where mice continuously adapted their choices to obtain a reward of optogenetic stimulation of VTA DA neurons (paired with an auditory cue). Among several reinforcement learning models, we found that a variation of Q-learning best described choice trajectories. We then assessed neural representations of model-estimated state values (reward expectations) and action values (decision variables), revealing for the first time that VTA DA stimulation is sufficient to generate neural value correlates. Surprisingly, both state and action value representations were much stronger in dorsolateral striatum (DLS) than NAc, contradicting the classic dorsal/ventral organization hypothesis. The dorsal localization of value representations could not be explained by motor correlates in DLS. Moreover, a mismatch in timescales for value computation does not account for these results, as neural correlates of value in both regions were best explained by a learning rate well-matched to the behavioral learning rate. Instead, we found that (1) VTA DA stimulation was sufficient to produce learned cue responses in NAc in a Pavlovian task (in the absence of an action requirement), and (2) that animals work for this cue, rather than VTA DA stimulation itself, in our main probabilistic learning task. Overall, this suggests that DA supports trial-and-error learning by endowing stimuli with value via the NAc, which in turn contributes to the generation of state and action values in DLS.