The vast majority of natural animal behavior involves far more than passive perception and reflexive responses. Indeed, animals actively move themselves in the world and systematically collect sensory observations over time; such active sensing allows them to infer environmental states that are not directly measurable, which ultimately guide ethologically important behaviors. One clear example of such a behavior is odor plume localization in flying insects. By changing their speed and turning, insects produce variations in their multisensory inputs that, once integrated over time, can resolve key properties of the environment, such as the true direction of the wind from which the odor may originate. However, an algorithmic understanding of how this crucial behavior can be performed with only biologically plausible sensory percepts remains lacking. In this project, we used a deep reinforcement learning (DRL) approach to train agent-based recurrent neural network (RNN) models to localize odor plumes in an arena with a simulated plume, providing the agent with only biologically realistic sensory inputs. Remarkably, we found our agents were able to localize the odor source by integrating their egocentric perception of wind direction with optical flow angle to compute the true wind direction. Compared to agents trained with true wind direction as an input, which mostly traveled at their maximum speed, our agents traveled more slowly and adjusted their speed more often. Further, recent changes in wind direction strongly drove our agents to adjust their speed and turning velocity. These observations are consistent with the agents using active sensing strategies to improve their wind direction estimates. We suggest training DRL agents complements experimental approaches to understanding active sensing and offer unique insights into the neuroethology of complex behaviors.