Understanding the brain-behavior relationship is a central goal in neuroscience. Recent neuroimaging studies have demonstrated that individual differences in both brain anatomy and the connectivity between regions are predictive of behavior. We hypothesize that behavior can also be predicted by how the same brain region, in different individuals, encodes information. As an analogy, athletic ability is not only related to the size of the components of the cardiovascular and musculoskeletal systems or the strength of the connections between components, it is also related to the proficiency of the individual components. Here we propose a framework, built on encoding-models, to evaluate our hypothesis that individual differences in how information is encoded in the brain can predict behavior measures. We evaluate our framework on fMRI data collected when 90 participants from the Human Connectome Project (HCP) watched naturalistic video clips, and when they performed a tightly controlled motor task. We find that individual differences exist in where and what stimulus information is encoded in the brain, and that these differences are predictive of variability in cognitive behavior. Given what is expected in predicting behavior from fMRI data, our results argue that encoding-models are a powerful tool for studying the brain-behavior relationship. Crucially, our results also reveal that the ability to predict different behavior measures depends on the choice of task and encoding-model. These findings open the door for neuroimaging studies to increase our understanding of the brain-behavior relationship by investigating the relationship between individual differences in brain encoding and behavior. Further, the task/encoding-model specificity suggests that experimenters interested in predicting behavior should tailor their choice of task and encoding-model to the behavior of interest.