Flexible computation is a hallmark of intelligent behavior. Yet, little is known about how recurrently connected circuits contextually reconfigure for different computations. Neuronal activity is primarily studied during the performance of a single task due to difficulty of animal training. To investigate neural network flexibility for multiple computations, we trained recurrent neural networks (RNNs) to perform a set of sensori-motor and cognitive tasks. We examined networks through the lens of dynamical systems, where the evolution of population activity is dependent on activity in the previous timestep and on inputs to the system. Population activity was organized such that similar task computations operated in nearby parts of state space, sharing the same dynamical landscape. For example, a ring attractor was reused across tasks that required memory of the same circular variable input. We refer to shared point attractors, ring attractors and decision boundaries as dynamical motifs. Using fixed point analysis and analysis of population variance, we found that individual dynamical motifs were implemented by clusters of units. Cluster lesions resulted in modular effects on network performance: lesioning one dynamical motif had little to no effect on other dynamical motifs. These results are due to the orthogonal organization of motifs, which allows for compositionality. Finally, modular dynamical motifs could be reconfigured for fast transfer learning without catastrophic forgetting. After slow initial learning of dynamical motifs, a subsequent faster stage of learning reconfigures motifs to perform novel tasks. Overall, our work provides a framework to understand flexible computation in recurrently connected circuits. Our lesion studies make direct hypotheses that could be tested experimentally, and many of our analyses could be applied to neural population recordings. As more neuronal data is collected from multiple brain regions simultaneously and chronically, we hypothesize that modular dynamical motifs will provide a theoretical framework to understand these data.