Center-surround processing (CSP) is a ubiquitous feature of the brain, starting from early sensory processing (e.g., context-dependent retinal encoding) to representing abstract concepts (e.g., context-dependent object representation), manifesting itself in both behavior and neural responses. While CSP in neural responses is typically modeled using divisive normalization or in purely descriptive terms, none of them can account for the bewildering range of effects documented for CSP of motion alone. Furthermore, a normative model that links these neural effects to function and behavior is still lacking. Here, we build on a recently developed normative model for motion perception in which CSP emerges as the result of representing motion relative to a reference frame which is inferred by causal inference. We derive neural predictions for mean responses (tuning) and neural variability (Fano Factor) of single neurons for the entire (4D) stimulus space of center and surround (CS) directions and CS speeds. These neural predictions show both complex structures and variability across participants in the interaction of CS directions and speeds – unexplainable by classic divisive normalization models. Importantly, we show that our predictions qualitatively match a wide range of prior empirical findings from neural recordings in the middle temporal visual area (MT) of monkeys. Our results pave the way for new theory-driven neurophysiology experiments to test the normative predictions, and to probe the circuit-level implementation of the associated computations.