Recent technological advances have enabled the measurement of the ‘connectome,’ or synaptic wiring diagram, of neurons across large brain regions or entire brains. Theoretical work has long sought to link connectivity features of large neural networks to their emergent dynamics. Nevertheless, how to integrate connectomics data and neural recordings to determine the function of a neural system remains challenging, given the physiological properties of individual neurons that remain unconstrained. To address this, we developed a theory that describes the space of solutions when inferring activity in connectivity-constrained neural networks with unconstrained single-neuron parameters. We studied recurrent neural networks (RNNs) in a teacher-student set-up, where the teacher RNN represents a biological neural network for which a connectome is available. The student has access to this connectome, but not the teacher’s single-neuron parameters (e.g. bias, gains, etc). We then optimized the student to match the activity of a subsample of the teacher activity. We applied this framework to multiple teacher RNNs, and in particular, two networks constrained by connectomes: the motor-premotor circuit in larval Drosophila and the head-direction system in adult Drosophila. We found that the activity of unrecorded neurons can be inferred well above chance given a small number of recorded neurons. The minimum number of recorded neurons depends on the dimensionality of the network dynamics, but not the number of units in the network. In contrast, when the connectome is not known, inference of unrecorded activity is unsuccessful. The theory estimates which neurons should be prioritized for neural recordings to maximally improve future predictions of network activity. Altogether, we provide a guide for when connectome data can, or cannot, predict the function of neural circuits and draw attention to a qualitative difference in the solution spaces of connectome-constrained and unconstrained networks.