Over the years, excitatory neurons received much attention for their role in brain dynamics, while the inhibitory neurons were understood to play more of a support, stabilizing role. However, recent advances have revealed that memories are encoded both by excitatory and inhibitory neurons (EI engrams), adding to the evidence of the functional importance of inhibitory neurons beyond mere stability. Still, the computations enabled by such EI engrams are unclear, and the synaptic plasticity rules that could create and sustain them are unknown. Here, we explore in silico a mechanism for EI engram storage and recall, as well as the computational benefits of such engrams over the classical, excitatory-only engrams. To this end, we consider recurrent spiking networks with Hebbian synaptic plasticity both at inhibitory-to-inhibitory (I-I) and inhibitory-to-excitatory (I-E) synapses. We show that the conjunction of these rules can robustly stabilize embedded EI engrams while keeping the activity of the network asynchronous and irregular, in the absence of specific cues. Such engrams can then be recalled by two distinct mechanisms: stimulation or disinhibition, as seen in the experimental data. Both of these mechanisms lead to reliable pattern completion and separation. Crucially, we show that similar networks that lack I-to-I plasticity are unable to recall the overlapping engrams via the disinhibitory route, limiting the functionality of those networks. Our work suggests that inhibitory plasticity facilitates recall of the overlapping engrams allowing inhibitory neurons to control multiple excitatory engrams. Overall, our study proposes that inhibitory neurons play a central role in memory, especially during disinhibition. These added capabilities are unlocked by synapse-specific plasticity rules.