Serotonin (5-HT) is an essential neuromodulator affecting behavioral, affective, and cognitive functions. Recent studies by tryptophan depletion in humans and devaluation paradigm in mice suggest that 5-HT promotes model-based decision making. However, the influence of 5-HT on such decision making remains poorly understood. Recent development of rodent two-step decision making tasks enables us to examine the effect of 5-HT manipulations in situations where rewards do not simply reinforce previous choices but rather interact with task structure to influence policy. Here we took a two-step decision making task and tested the effect of optogenetic inhibition of 5-HT neurons using Tph2-ArchT transgenic mice. We implanted an optic probe above the dorsal raphe nucleus (DRN) and applied photoinhibition by yellow light from outcome delivery to first-step choice at the next trial. In control trials, blue light stimulation was applied. Mice showed choice behavior using model-based controls. The photoinhibition shortened the time to make first-step choices, possibly due to disrupted deliberative model-based decision making process. Furthermore, fitting model-free/model-based hybrid reinforcement learning model suggested that photoinhibition decreased the reliance on model-based decision making. These results revealed the computational role of DRN 5-HT neurons in model-based decision making in dynamic environments.