Modeling neural responses to naturalistic stimuli has been instrumental in advancing our understanding of the visual system. Dominant computational modeling efforts in this direction have been deeply rooted in preconceived hypotheses. Here, we develop a hypothesis-neutral computational methodology which brings neuroscience data directly to bear on the model development process. We demonstrate the effectiveness of this technique in modeling as well as systematically characterizing voxel tuning properties. We leverage the unprecedented scale of the Natural Scenes Dataset to constrain parametrized neural models of higher-order visual systems with brain response measurements and achieve novel predictive precision, outperforming the predictive success of state-of-the-art models. Next, we ask what kinds of functional properties emerge spontaneously in these response-optimized models? We examine trained networks through structural and functional analysis by running `virtual' fMRI experiments on large-scale datasets. Strikingly, despite no category-level supervision, since the models are optimized for brain response prediction from scratch, the units in the networks after optimization act strongly as detectors for semantic concepts like `faces' or `words', thereby providing one of the strongest evidences for categorical selectivity in these areas. The observed selectivity raises another question: are these units simply functioning as detectors for their preferred category or are they a by-product of a non-category-specific processing mechanism? To investigate this, we create selective deprivations in the visual diet of these models and study semantic selectivity in the `deprived' networks, thereby also elucidating the role of specific visual experiences in shaping neuronal tuning. Beyond characterizing tuning properties, we study the transferability of representations in response-optimized networks on different perceptual tasks. We find that the sole objective of reproducing neural targets, without any task-specific supervision, grants these networks intriguing functionalities. Together, this new class of response-optimized models combined with novel interpretability techniques reveal themselves as a powerful framework for probing the brain.