To decipher the algorithm of perception, it is important to characterize neural tuning functions and identify the directions of maximal sensitivity and invariance to stimulus features. To this end, parametric stimuli are typically used, but they make strong assumptions about the stimulus features to which neurons are selective and invariant. Recently, the “inception loop” paradigm was developed, which combines large-scale neuronal recordings with deep learning system identification models which are trained to predict the responses of neurons to arbitrary stimuli. Inception loops have been used to find and verify the most exciting inputs of neurons (MEIs) in mouse primary visual cortex (V1). The MEIs exhibited complex spatial features deviating strikingly from the textbook Gabor-like optimal stimuli for V1, challenging decades-old dogma of V1 representations. Here, we extend these deep learning imaging synthesis methods to study neural tuning invariance. The progressive increase in invariance of neuronal responses to nuisance transformations of visual features is a hallmark of hierarchical visual processing. However, a systematic characterization of neural tuning invariance across sensory systems is currently missing. We introduce “diverse exciting inputs” (DEIs), a synthesized set of diverse images that strongly excite neurons, and verify the high activation of these diverse images in vivo. We found that the DEIs of many neurons in mouse V1 exhibit novel types of invariances. Empirically, these tuning invariance can be mostly characterized by phase shifts within a spatially localized region, a property that cannot be explained by the canonical Hubel & Wiesel simple-complex cell model. Thus, we propose a localized-phase-invariant model to explain these observed single-neuron invariances in mouse V1. Taken together, we introduce a novel framework to study sensory processing using deep learning and discover a novel type of single-neuron invariance in mouse primary visual cortex.