Hierarchical cortical organization unifies the brain’s structural and functional organization, yet its relationship to task-evoked cognitive processes remains unclear. How might intrinsic hierarchical properties shape and constrain the cognitive processes required to perform the wide variety of tasks encountered in daily life? By analyzing a human functional magnetic resonance imaging (fMRI) data set with 26 unique cognitive tasks collected per participant, we characterized the geometry and topography of multi-task representations across the cortical hierarchy using representational similarity analysis. Empirically, we found that task representations in unimodal (sensorimotor) areas were high dimensional and segregated from other functional networks, while task representations in transmodal (association) areas were low dimensional but integrated across networks. Further analysis of whole-cortex representational organization revealed a sensory-association-motor axis that first compressed, then expanded multi-task representations from sensory to motor cortices. To identify the computational mechanisms underlying the compression-then-expansion of task representations, we trained a multi-layer artificial neural network modeling (ANN) to model the transformation of empirical task activations. We found that the compression-then-expansion of task representations exclusively emerged in a “rich” training regime, when ANNs were initialized with low-norm weights. In this rich training regime, the ANNs’ internal representations had greater similarity to empirical fMRI representations across the cortical hierarchy. Further analysis of the ANN’s organization revealed that richly trained ANNs learned low-dimensional connectivity weights with heavy-tailed distributions, resulting in hierarchically structured internal representations. In contrast, ANNs trained in the so-called “lazy” regime, where ANNs were initialized with large-norm weights, ANNs failed to learn hierarchically structured representations. Together, these results provide a characterization of multi-task representations across the cortical hierarchy, while establishing computational mechanisms for building brain-like, hierarchical representations in ANN models.