While simple value-based learning efficiently predicts an outcome in a stable environment, goal-directed learning can deal with dynamic and uncertain environments1. It is this complex nature that poses a challenge for experimental design to investigate such learning processes. For example, it is hard to predict how task manipulations affect the latent process of goal-directed learning. Here we present a computational framework of goal-directed task control to guide goal-directed learning. The proposed framework is based on an asymmetric two-player game setting: while a computational model of human RL (called a cognitive model) performs a goal-conditioned two-stage Markov decision task, an RL algorithm (called a task controller) learns a behavioral policy to drive the key variable (i.e., state prediction error) of the cognitive model to the arbitrarily chosen state, by manipulating the task parameters (i.e., state-action-state transition uncertainty and goal conditions) on a trial-by-trial basis. We fitted the cognitive models individually to 82 human subjects’ data, and subsequently used them to train the task controller in two different scenarios, minimizing and maximizing state prediction error, each intended to improve and reduce the motivation for goal-directed learning, respectively. The model permutation analysis revealed a subject-independent task control policy, suggesting that the task controller pre-trained with cognitive models could generalize to actual human subjects without further training. To directly test the efficacy of our framework, we ran fMRI experiments on another 21 human subjects. We confirmed the task controller successfully manipulates human goal-directed behavior. Notably, we found neural effects of the task control on the insular and lateral prefrontal cortex, the cortical regions known to encode state prediction error during goal-directed learning2,3. Our work not only advances recent task optimization confined to simple decision-making tasks4,5 but also demonstrates the control effect at the behavioral and neural levels.