Hierarchical working memory refers to the human ability to group information on-the-fly into larger chunks, thereby overcoming the limited capacity of Working Memory (WM), as first suggested by George A. Miller in his 1956 paper. Although chunking has been explored experimentally in numerous studies since then, its neuronal underpinnings remain unknown to this day. In our work, we address this issue by proposing a novel Recurrent Neural Network (RNN) model that enables the spontaneous organization of memory into chunks within WM. Our model leverages specialized “chunking neurons” and long-term synaptic plasticity, leading to a significant increase in working memory capacity. Moreover, we show that this improved capacity is still constrained by a universal limit, given by the elegant formula M = 2^{C-1}, where C represents the basic capacity (currently estimated to be around 4 in literature). Our theoretical predictions were further validated through the analysis of two sets of experimental results: We found evidence supporting our hypothesized chunking neurons in single-neuron recordings from epileptic patients; The universal limit M closely matched the average maximal number of words that could be fully recalled in a classical memory experiment with verbal material. By establishing a putative link between higher cognitive functions and synaptic transmission dynamics, our results offer a fruitful testing ground for new experiments that bridge the gap between these two extremes of neuroscience.