High-dimensional neural representations are observed in many brain areas and are believed to be a powerful substrate for flexible computation. Theories have suggested that the vertebrate cerebellum and other cerebellum-like structures produce such high-dimensional representations by expanding their input into a vast layer of granule-like cells. Signals are typically routed to such expanded representations by anatomical “bottlenecks”: in the mammalian cortico-cerebellar pathway, input from across the cortex converges in the pontine nuclei, from which roughly half of the mossy fiber input to granule cells originates. Similarly, in the insect olfactory system, responses of odor receptor neurons in the antenna are compressed in the antennal lobe glomeruli, before being routed to Kenyon cells in the mushroom body (a cerebellum-like structure). This bottleneck motif has been largely ignored in models and is at first sight at odds with the goal of maximizing dimensionality. Here, we use a combination of simulations, analytical calculations, and analysis of neural data from flies and mice to develop a normative theory of cerebellum-like structures in conjunction with their afferent pathways. We propose that the bottleneck architecture of regions presynaptic to granule-like layers reformats the input representation to maximize the efficacy of the subsequent expansion. When applied to the insect olfactory system, our theory explains its glomerular organization and inter-glomerular interactions. The same objective, when applied to distributed input from motor cortex, implies that the pontine nuclei select the task-relevant subspace within cortical activity. Our theory predicts that cerebellar granule cells expand the task-relevant dimensionality, reconciling theories of dimensionality expansion with recent observations of high correlations among granule cells (Wagner et al., 2019). Our conclusions are not limited to cerebellum-like structures and relate the statistical properties of a neural representation to the architectures that optimally transform it to facilitate learning downstream.