The accuracy of behavior is constrained by the encoding of sensory information. However little is known about how reliable encoding emerges as animals develop during early life. A better understanding of these developmental processes has the potential to aid in the design of artificial learning systems, and help reveal how neurodevelopmental disorders alter information processing. Here we examine the developmental changes in signal and noise correlation structure of neural activity by analyzing neural recordings of the zebrafish optic tectum evoked by prey-like visual stimuli. We found that signal dimensionality remained stable, but coding reliability increased due to an increase in the distance between neural representations of adjacent stimuli and a reduction in trial-to-trial noise. However, despite a decrease in noise dimensionality, noise became increasingly aligned with the signal direction, reducing the information available for decoding. We then showed that a self-supervised learning model replicated these experimental findings, showing stable signal dimensionality, reduced noise dimensionality, and increasing signal-noise correlation over training, even as visual acuity improved. Together these results suggest that during development, animals optimize visual circuitry for better signal discrimination through noise reduction rather than modifying the orientation of noise, potentially due to developmental constraints on neural wiring.