Nonlinear rate-based models, such as stabilized supralinear networks (SSNs), have been crucial in explaining key aspects of cortical dynamics, including the nonlinear integration of visual inputs and the modulation of response variability. Yet, the absence of spike-based signals in these frameworks has hindered the study of millisecond-scale interactions between pairs of neurons, which have been shown to drive synaptic plasticity. For instance, spike-timing dependent plasticity (STDP), a phenomenological description of long-term synaptic plasticity, can shape neural circuits based on spike-based interactions on much faster timescales than firing rate correlations. To investigate how STDP shapes spiking networks with SSN properties, we developed a theoretical framework using linear response theory that incorporates SSN nonlinearities while generating spiking activity with Poisson-like statistics. Our spiking model replicates several core characteristics of SSNs, including loose input balancing, modulation of response variability, and input normalization mechanisms. By integrating STDP into this model, we examined how various plasticity rules shape the structure and function of these circuits. We analytically characterized the firing rates and pairwise correlations across different network operating regimes, and found that plasticity is modulated non-monotonically, as the SSN moves from the supralinear, weakly inhibited regime to the sublinear, strongly inhibited one. Overall, this analysis sheds light on how variations in input or connectivity structure influence plasticity, thereby shaping the emergent dynamics of the network. Our study contributes to the growing research on nonlinear properties of circuit dynamics and their implications for neuronal computation and synaptic plasticity.