Flexible behaviors are critical for survival in diverse and dynamic environments. However, the synaptic mechanisms that underlie flexible behaviors are largely unknown. Neuronal responses during behavior are diverse, ranging from reliable ‘classical’ responses (e.g. pure tone tuning in auditory cortex) to irregular ‘non-classically responsive’ firing. Previous work has shown that both classically and non-classically responsive cortical neurons contain significant information about sensory stimuli and behavioral decisions and are necessary for task performance suggesting that non-classically responsive cells may play an underappreciated role in perception and behavior. To explore the synaptic origin and functional contribution of diverse response profiles to flexible task performance and dynamics, we adapted a recently developed, spiking recurrent neural network (RNN) model that incorporated excitatory and inhibitory spike-timing-dependent plasticity (STDP). We demonstrate that this model can be successfully trained to perform a reversal learning task similar to that of behaving animals where the rewarded target tone is spontaneously changed (i.e. reversed) partway through the task. This model recapitulates the distribution of classically and non-classically responsive neurons measured from the cortex of behaving rodents. While the network can be trained to perform the initial, pre-reversal task over a wide range of model parameters, we find that the network can only successfully learn the full reversal task when network parameters result in a balance of excitatory and inhibitory input to individual units. Interestingly, these balanced networks also demonstrate the greatest diversity of response types and the emergence of distinct functional roles for classically and non-classically responsive units as demonstrated by detailed network perturbations. We identify a specific parameter regime for these flexible, balanced networks where tonic inhibition lies between baseline and evoked excitation in a “see-saw” mechanism that facilitates stimulus-choice remapping. These results elucidate the relationship between synaptic dynamics, response type diversity, and behavioral flexibility.