Targeted photostimulation optogenetics offers unprecedented precision in perturbing neural circuits in action. However, due to the complexity of neural responses, perturbing neurons chosen randomly or based on their task selectivity often produces weak or counter-intuitive effects. Recurrent neural networks (RNNs) trained to replicate neural activity and task behavior hold promise for guiding the design of targeted perturbations. Yet, it remains uncertain whether low-dimensional neural responses during cognitive tasks sufficiently constrain high-dimensional models to predict outcomes of patterned perturbations reliably. We investigated this question in task-optimized RNNs, which provide full access to the ground-truth dynamics and connectivity. We trained a 200-unit RNN to perform a delayed decision-making task, which we designated as the ground-truth RNN. We then separately trained an ensemble of 200-unit RNNs, referred to as "twins", to replicate the neural responses and behavior of the ground-truth RNN, matching the activity of their units one-to-one. Although all twins nearly perfectly reproduced the activity and behavior of the ground-truth RNN under normal task conditions, they had varying connectivity that significantly deviated from the ground truth. As a result, the trajectories of the ground-truth RNN and its twin diverged drastically when we delivered the same patterned perturbations to their units. We found that, despite differences in their full high-dimensional connectivity, all twins and the ground-truth RNN shared the same latent low-rank connectivity driving task-relevant dynamics, which we revealed by fitting RNN responses with low-dimensional latent circuit models. Patterned perturbations aligned with this latent low-rank connectivity produced highly consistent effects between the twins and ground-truth RNN. Our results show that low-dimensional neural responses do not sufficiently constrain high-dimensional connectivity in RNNs fitted to reproduce data, undermining their causal predictive power. Nevertheless, low-dimensional data uniquely constrain latent low-rank connectivity in RNNs, which provides the maximal predictive power for guiding targeted perturbations.