Neuroligin-2 (Nlgn2) is a postsynaptic adhesion protein that regulates the organization of GABAergic inhibitory synapses. NLGN2 gene variants have been linked to several psychiatric disorders, and deletion of Nlgn2 in mice results in a pronounced increase in innate anxiety-related avoidance behaviors. It is unknown, however, whether Nlgn2 also regulates other forms of defensive behaviors, and through which neuronal circuits these effects are mediated. Here, we therefore investigated the consequences of Nlgn2 deletion on the neuronal circuits that control associative fear learning in mice. We found that Nlgn2 knock-out (KO) mice displayed robust impairments in auditory, but not contextual fear learning, despite the concurrent increase in innate avoidance behaviors. These impairments were accompanied by abnormal activation of neurons in the basal amygdala (BA), lateral amygdala (LA), auditory cortex (AuC) and prelimbic cortex (PreL). Local AAV-mediated deletion of Nlgn2 selectively in these brain regions revealed a role for Nlgn2 in the BA and PreL, but not the LA or AuC. To determine which neuronal subpopulations were involved in the learning deficit, we conditionally deleted Nlgn2 from parvalbumin (PV)+, somatostatin (SOM)+, vasoactive intestinal peptide (VIP)+ and CaMKII+ neurons. Strikingly, Nlgn2 deletion from VIP+ or SOM+ neurons partially recapitulated the fear learning impairment observed in constitutive Nlgn2 KO mice, but not the increased avoidance behaviors, whereas Nlgn2 deletion from CaMKII+ neurons reproduced the enhanced avoidance behaviors without altering associative fear learning. Our findings indicate that Nlgn2 may contribute to psychiatrically relevant behaviors through differential effects at different subtypes of GABAergic synapses.