The ability to combine simple learned elements (primitives) and create novel complex behaviors is a fundamental aspect of cognition in biological brains. However, composing computational primitives in artificial neural networks remains a significant challenge, suggesting that biological systems may possess structural priors which enhance this capability. In this study, we examine the mechanisms by which network structure facilitates flexible compositions of computational primitives. To investigate this question, we analyze recurrent neural networks (RNNs) that perform a temporal vector addition task inspired by experiments with non-human primates. In this task, durations of two stimuli represent the x and y-coordinates of a vector that needs to be reported by a movement on the 2d plane. Our goal is to understand how the processing of individual x and y inputs is combined to process combinations of inputs. In a first step, we train networks to generate primitives that process inputs along a single input direction. We use low-rank RNNs to identify primitives of minimal dimensionality, and show that they have different generalization properties than full-rank networks due to different underlying dynamics. In a second step, we examine how two primitives can be composed to process combinations of inputs. Using minimal-rank primitives, we create new networks by adding connectivities of individual primitives together with a global inhibition term. Employing the theory of low-rank networks, we analyze the dynamics and computations that emerge. Our key finding is that the strength of the global inhibition can modulate the combined dynamics, leading to a flexible transition between two types of dynamics and computations. At low inhibition, the network processes input dimensions independently, preserving the individual computations of each primitive but without combining them in a vector addition. As inhibition increases, the network transitions to a factorized computation, combining primitives to output the vector sum of inputs.