Emotions significantly shape behavior, with mood---a prolonged emotional state---affecting decision-making processes by biasing perceptions of future outcomes. Recent studies suggest that mood functions as an extrapolation mechanism, tracking deviations between recent experiences and expectations to predict and inform future behavior. Understanding this mechanism has important implications for both biological systems and artificial agents. In this study, we incorporate a mood-like mechanism, modeled as momentum, into the policy dynamics of a Reinforcement Learning (RL) agent. Specifically, we focus on Neural Actor-Critics and propose a momentum-based update within the Policy Mirror Descent (PMD) framework---a general family of algorithms encompassing many fundamental RL methods. Our theoretical analysis shows that introducing momentum enables faster convergence, reducing the iterations needed to reach the optimal policy. Complementing this, our numerical experiments demonstrate that adding momentum enhances policy learning efficiency, particularly in environments with ill-conditioned optimization landscapes characterized by sparse connectivity and long horizons. We also investigate the effect of inexact value function approximation and find that momentum retains acceleration benefits up to a certain level of approximation error. Our findings suggest that mood-like mechanisms can accelerate policy learning in RL agents. By incorporating recent experiences, agents better navigate the optimization landscape, avoiding oscillations and re-exploration of suboptimal regions, leading to more efficient policy updates and faster convergence. These results underscore the significance of integrating emotion-inspired mechanisms into artificial agents, offering valuable insights for advancing RL algorithms. Our algorithm provides computational benefits and insights on how mood-like processes facilitate learning and adaptation in uncertain environments. It offers a computational framework linking mood dynamics to behavioral adaptation and provides testable predictions about how mood could influence learning speed and decision biases in biological systems. Understanding the interaction between mood, reward processing, and learning is crucial for tackling mental health challenges like anxiety and depression.