Stable neural activity plays a pivotal role in various cognitive, behavioral, and physiological functions, including information processing, memory consolidation, behavioral regulation, and attentional mechanisms. The orchestration of such stability is governed by the intricate interplay of synaptic connections within neural populations. Extensive theoretical and empirical investigations have underscored the influential role of synaptic weights in shaping both the collective dynamics and the fluctuations of neural activity. A salient inquiry arising from this understanding pertains to the mechanism responsible for fine-tuning synaptic weights to values conducive to neural stability. In our conjecture, we posit asymmetrical Spike-Timing-Dependent Plasticity (STDP) as a plausible candidate underlying this tuning process. STDP claims that the modification in synaptic efficacies depends on the time interval between pre and post synaptic spikes. In our theoretical study, we employed a firing rate model to describe the dynamics of a recurrent neural network comprising excitatory and inhibitory neuronal populations. The synaptic plasticity dynamics were governed by the temporal overlap between neural activity correlations and the STDP learning rule. Our investigation revealed that STDP originates a robust emergence of critical rhythmogenesis—a phenomenon characterized by intermittent rhythmic neural activity—while concurrently fostering neural stability. Furthermore, STDP facilitates synaptic fluctuations that preserve this stable behavior.