It has generally been presumed that sensory information encoded by a nervous system should be as accurate as its biological limitations allow. However, perhaps counter intuitively, accurate representations of sensory signals do not necessarily maximize the organism’s chances of survival. We show that neural codes that maximize reward expectation—and not accurate sensory representations—account for retinal responses in insects, and retinotopically-specific adaptive codes in humans. Thus, our results provide evidence that fitness-maximizing rules imposed by the environment are applied at the earliest stages of sensory processing.