Higher-order brain activity can be quite complex, switching between fast, stimulus- or motor-driven dynamics, and slower, rising and falling persistent activity. Many of these activity motifs can be successfully modelled by recurrent neural networks (RNNs) that are trained on specific tasks. However, the training of RNNs is usually an ill-posed problem so that, at least in principle, multiple solutions exist for any particular task. Accordingly, a specific match of a trained RNN to data can be serendipitous, providing only limited insight into the reasons underlying the similarity. Here we take a normative approach by first stating the goal of an agent's internal task representation. We distinguish two goals: The goal of a ‘habitual agent’ (HA) is to take correct actions, while the goal of a ‘model-based agent' (MBA) is to predict all ethologically relevant observations. We define these two behavioural strategies within the framework of partially observable reinforcement learning. Each strategy imposes different constraints on the representation of task variables. Our main contribution here is to show how to find, among all representations consistent with the agent's goals, the one that eliminates all irrelevant information, thereby following the efficient coding hypothesis. We showcase this approach on a classical working memory task. Formally, we parameterize HA and MBA representations with switching linear dynamical systems regularized by an information bottleneck, which squeezes out all the information in the representation that is not needed to achieve the behavioural goal. In both agents, we find that efficient representations reproduce the key features of population activities recorded from the prefrontal cortex (PFC). However, only the MBA closely reproduces the persistent delay dynamics. In either case, the representational motifs are directly interpretable in terms of the goal they serve, thus yielding potential insights into the goals underlying higher-order brain representations.