Inhibitory neurons (I), akin to their excitatory (E) counterparts, adjust synaptic efficacies in response to neural activity. Contrary to the traditional view of inhibition as a purely global activity modulator, recent studies highlighted the importance of specificity in (E/I) wiring and motif structures. These motifs, encompassing mutual and lateral inhibition, take essential functional roles in tuning of receptive fields, learning of representations, predictive coding, and in the regulation of the degrees of freedom of the network’s dynamics (i.e. the network dimensionality). Despite numerous characterizations of inhibitory plasticity rules, integrating them into a cohesive understanding of inhibition’s various aspects remains challenging. Here we tackle this question, revealing how rules based solely on local interactions and spike-timing can effectively mediate both excitatory stabilization and structure formation. We distinguish the effects of spike-timing dependent plasticity (STDP) into two components: one dependent on firing rates, which tunes inhibitory synapses broadly and hinders motif formation, and another exclusively regulated by higher order statistics, which instead fosters sparser, more structured connectivity (Fig. 1A-C). Selecting STDP rules that are either rate- or covariance- dominated, we compare the effect of the two regimes in simple circuits. Rate-dominated rules show low sensitivity to structure (Fig. 1A). By contrast, covariance-dominated regimes promote either mutual E/I connections, or later inhibition, contingent on the shape of the pairwise STDP interaction (Fig. 1B,C, respectively). Applying our theory to large recurrent circuit simulations with random connectivity, we further reveal how covariance-dominated rules can create sparse connectivity with specific E/I motifs, and stabilize overall circuit dynamics, thus bridging the gap between local synaptic plasticity and network-level organization (Fig. 1D,E,F). Finally, to investigate the effects of covariance-dominated rules on networks with a specific macro-structure, we simulated a ring-like network, where the excitatory connections depend on the angular distance between neurons. The network has two inhibitory subpopulations with two different STDP rules, free to regulate their weights. We found that the inhibitory populations take distinct connectivity profiles, resulting in functional properties such as surround suppression (Fig. 1G-H).