Mental fatigue alters behavior in diverse and sometimes counter-intuitive ways. Could this be explained by the optimal allocation of limited cognitive resources? We consider that cognitive control allows to obtain more rewards from the environment, and that it gives rise to fatigue, which impedes future ability for the agent to gain rewards. Under those hypotheses, we formulated an optimal computational agent with a control allocation policy that maximizes future rewards, with two scenarios : limited resources that do not allow for more control exertion after a limit has been reached, or accumulation of toxic byproducts of neural activity that decrease the efficiency of future control. This gives rise to a normative formulation for the cost of control, as the amount of future reward foregone by investing a given amount of control. It also allows to derive nontrivial predictions for human behavior, such as a relative stability of performance over time in a fatiguing task. Indeed we have been able to replicate results from Blain et al. (2016) in which fatigue in human participants led to more impulsive choices, but not to a deterioration of performance in the depleting task. Crucially, the behavior of this agent relies on accurate representations of its efficacy, its fatigue dynamics and the contingencies of the environment. We show that under biased representations, the agent can exhibit patterns of over-investment of control leading to suboptimal outcomes. Future work will build on that framework to investigate control over-investment in persons at risk for burnout syndrome.