Latent dynamical system (LDS) models have been widely used to characterize the dynamics of high-dimensional neural population activity. However, these models typically ignore the fact that the brain contains multiple cell types. This limits their ability to capture the functional roles of distinct cell classes, and thus to capture the effects of cell-specific optogenetic perturbations on neural activity and behavior. To address this gap, we introduce the "cell-type dynamical systems" (CTDS) model, which extends latent linear dynamical systems to contain distinct latent variables for each cell class, with biologically inspired constraints on both dynamics and emissions parameters. To illustrate our approach, we considered simultaneous neural recordings from excitatory (E) and inhibitory (I) populations in rat frontal orienting fields (FOF) and anterior dorsal striatum (ADS) during an auditory decision-making task. We fit the CTDS model to electrophysiological recordings of unperturbed activity in the two brain regions, and found that---remarkably---the model accurately predicted the effects of cell-type-specific optogenetic perturbations. Specifically, we examined early and late perturbations in the FOF and ADS, as well as silencing of the projection from FOF to ADS, and found that the model precisely captured the effects of these different perturbations on behavior. By contrast, standard LDS models, which do not differentiate between cell types, failed to account for these effects. The CTDS model provided insight into the effects of different perturbations by revealing the dynamics of different cell-specific latents, leading us to conclude that the recurrent connections between FOF and ADS are crucial during evidence accumulation. These results illustrate the power of CTDS to provide accurate and interpretable descriptions of neural dynamics that arise from interactions between distinct cell classes.