In natural environments, animals must efficiently allocate their choices across multiple concurrently available resources when foraging, a complex decision-making process not fully captured by existing models. To understand how animals learn to navigate this challenge we developed a novel paradigm in which untrained, water-restricted mice were free to sample from six options rewarded at a range of deterministic intervals and positioned around the walls of a large arena. Combining core features of quantitative models of foraging and decision-making, we developed a novel reinforcement learning (RL) model, AQUA, that accurately reproduced the behavior of ~20 mice initially exposed to the multi-option foraging paradigm. Fiber photometry recordings revealed that dopamine in the nucleus accumbens core (NAcC) more closely reflected a global, dynamic learning rate component of AQUA than local error-based updating. Altogether, our results provide insight into the neural substrate of a learning algorithm that allows mice to rapidly acquire near-optimal foraging strategies across many options in a large naturalistic spatial environment.