Central place foraging insects like the honeybee (Apis mellifera) are the masters of visual navigation of the insect world: They are able to reliably return to their nest under a wide range of visual conditions after only a few learning flights, (a feat so far unrivalled by any artificial system). Furthermore, they perform complex navigational tasks, such as ‘computing’ novel shortcuts between previously known but unrelated food sources, supporting claims about the existence of a ‘cognitive map’ in honeybees. This raises the question which internal representations of the world (and the animal’s relation to it) are necessary and sufficient to explain these observed behaviors. In this project, we take a normative computational approach based on Deep Reinforcement Learning (DRL) to address this question. We use one deep neural network to model the entire processing pipeline from ommatidium-level visual input to an abstract, high-level action space in a naturalistic virtual environment. Learning is implemented as a modified SARSA algorithm, which maximises the cumulative discounted reward of any state-action pair based on an extrinsic reward landscape. Differential learning rates for the network layers allow for a functional and anatomical separation of processing stages: We interpret the output layer as the locus of fast, behavioral learning, operating on complex latent representations of the visual input (including e.g. self-motion estimation and memory traces), passed on from the preceding layers. Anatomically, this maps roughly to dopamine-mediated learning of steering signals to the central complex, based on ‘visual’ Kenyon cell input into the mushroom bodies. Conversely, shallow and hidden layers represent the ‘slow’ learning (on evolutionary time scales) of visual features and robust, generalisable latent representations which support reliable and rapid learning of navigational tasks. Within anatomical constraints, we keep the network architecture as general as possible, to maximise the space of possible emerging latent representations: For example, can path integration and vector memories - essential mechanisms for insect navigation – emerge as a complex representation of visual input from our model? Can Kenyon cells therefore be considered ‘place cells’?