Summary: Modulation of neural activity by hormones, peptides, and other small molecules is known to be required for learning, motor control, and homeostasis, and it can induce similarly structured networks to exhibit different behaviors. Yet, relatively few theoretic studies have attempted to detangle the role and function the various types of modulation play throughout the brain. In particular, it is important to understand how changing neuromodulator concentrations can move a network through its space of possible behaviors. A better understanding of neuromodulatory movement through behavioral space could be applied to pharmacological interventions, allowing a clinician to perturb diseased networks away from pathological activity regimes. In this work we use techniques from information geometry to define and study the model manifolds of spiking networks. Points on these manifolds correspond to distributions of possible activities of a network as some of its parameters—such as the membrane and synaptic time constants—vary. Neuromodulators can alter physiological parameters like the timescales of a network, so a change in neuromodulator concentration can be viewed as a trajectory along these model manifolds. For exponential families, the model manifold can be characterized by a finite number of coordinates, which are the eigenmodes of a centered distance metric and are physically interpretable relative to the model parameters. We first apply a Gaussian approximation to the membrane potential dynamics of networks of stochastically spiking neurons. This approximation—expected to be valid away from bifurcations in network activity—permits explicit identification of manifold coordinates, estimation of its effective dimensionality, and building of intuition regarding neuromodulatory trajectories across these manifolds. This information geometric perspective provides a framework for understanding how different activity regimes a network can inhabit are connected by modulation. Further work with this methodology will address neuromodulation in motor network models and extend our approach to networks with non-equilibrium steady states.