In systems neuroscience, animals are often trained until they saturate the psychometric function, producing stereotypical behaviors. This may obscure important system properties, such as generalization and neural correlates. In this study, we trained and tested mice on visual texture classification problems, where they needed to generalize at test time based on mid-level visual features. After learning, mice consistently discriminated the training stimuli at a high accuracy, while their performance on test stimuli varied systematically across categories. To determine a possible neural basis for this variation, we simultaneously recorded over 50,000 neurons from primary (V1) and higher-order visual areas (HVA). We found that medial HVAs, especially superficial populations, encoded invariant visual features that could be used for generalization. Medial HVAs had much less invariance in dark-reared mice, suggesting that the invariance requires visual experience. Finally, the image-by-image neural representation similarity correlated with behavioral choices on test stimuli but not on training stimuli. Thus, animal behavior on test stimuli is directly related to the neural representation of those stimuli. Our findings establish mice as a viable model for studying higher-level visual invariance, and suggest that train/test behavior cross-validation is required to understand neural computations in sensory brain areas.