Models of recurrent neural networks provide fundamental insights about the interplay between structure and dynamics in biological neural networks. Most previous work focused on networks with random or simple connectivity structures, yet experiments show that overrepresented motifs shape the patterns of network activity. A theory relating such complex connectivity structures to network dynamics and function is lacking. Here, we show that third and higher-order correlations in synaptic connectivities have a large impact on neuronal dynamics. Strong triangular cyclic correlations in a network suppress irregular neural dynamics, and result in oscillatory or fixed activity. Further, we show that when irregular activity emerges, strong higher-order correlations reduce its dimensionality, while intermediate correlations can increase it. We also show that an increase in the order of correlations further reduces the dimensionality of attractors. We use recent developments in random matrix theory to explain these results and show how complex synaptic structures shape neural dynamics. We demonstrate that specific higher-order network configurations drive distinct features of neural activity. Our findings provide empirical predictions by linking specific high-order connectivity patterns to distinct dynamical states.