For visual perception to be unaffected by viewing conditions, animals must adapt their sensitivity to changing visual statistics, such as mean illumination. Consistently, photoreceptors in many species control their luminance gain and encode relative changes in luminance [1,2], termed contrast. However, this early gain control is insufficient for luminance invariant contrast estimation in dynamic conditions. Sudden transitions to dim or bright environment would lead to an erroneous, reduced or enhanced perception of contrasts, thus imposing a two-way challenge on contrast estimation. Yet, visual behaviors are luminance invariant [3,4], and it is not understood how the brains achieve such invariance. In fruit flies, a distinct visual pathway preserves luminance information past photoreceptors and enhances contrast estimation in sudden dim light [4]. Here, we show that the pathway implements a generalized gain correction to tackle the two-way contrast coding deficits in dynamic conditions. When blocking the output of the luminance-sensitive interneurons L3, the flies underestimate large contrasts in sudden dim light and overestimate smaller contrasts in sudden bright light. Furthermore, the gain correction improves visibility of very dim stimuli at all contrasts. To explain how the gain correction is implemented in these widely differing scenarios, we formulated an algorithmic model. Here, the luminance channel plays a dichotomous role: when correcting contrast deficits in dynamic conditions, it interacts with contrast signals in a multiplicative way; in absolute dim light, it acts as an independent contrast channel to improve signal detection. Together, our work combines experimental and computational modelling approaches to demonstrate how post-receptor gain correction is key to perceptually relevant vision. Given that visual systems of all behaving animals face similar challenges, and that luminance information is preserved in the vertebrate retina too [5], the corrective gain control might be a universal strategy of visual systems.