The internal activations of particular deep neural networks (DNNs) are remarkably similar to the neuronal population responses along the ventral visual cortex in primates. Nevertheless, the similarities between the two are often investigated through stimulus sets consisting of everyday objects under naturalistic settings. Recent work has revealed a gap in generalization ability of these models in predicting neuronal responses to out-of-distribution (OOD) samples (i.e. samples that are not regarded as natural photos) . Here, we investigated how the recent progress in improving DNNs’ object recognition generalization have impacted the generalization gap in neural predictivity. We quantified each model's neural prediction generalization in terms of its OOD neural prediction accuracy and generalization gap (difference between In-Distribution (ID) and OOD neural prediction accuracy). To study what factors contribute to such generalization capacity, we performed experiments on a wide range of DNNs and investigated how various DNN design choices that were shown to improve object-recognition generalization behavior in these models affect their neural prediction generalization capacity. We found that: 1) Increasing the network depth or width does not consistently improve the generalization gap or the OOD prediction accuracy ; 2) Comparing between supervised and unsupervised learning algorithms, we found that a particular unsupervised learning algorithm (Momentum Contrast) can significantly improve the neural prediction generalization ; 3) While adversarially robust models show consistently lower neural prediction accuracy on both ID and OOD samples, compared to other baselines, they achieve a smaller generalization gap compared to regular models; 4) The neural prediction generalization gap is significantly correlated with adversarial robustness gap while surprisingly, it is not significantly correlated with OOD object recognition generalization gap. Together, our results suggest that unsupervised and robust DNNs may lead to more general models of neuronal responses in the visual cortex.