Modelling ecological constraints on visual processing with deep reinforcement learning

Normative principles describe how to efficiently encode visual scenes, and supervised learning models show how formal tasks shape visual processing, yet animal vision ultimately adapts to encode survival-relevant features. To study how ecology constrains visual processing, we present a reinforcement learning (RL) framework in which an agent aims to survive in a 3-d environment that it perceives through a vision model. In particular, we consider a visually-guided foraging task: the environment is populated with randomly placed objects, and the agent must learn to identify those that increase or decrease health, so that it may gather or avoid them, respectively. We modulate task difficulty by varying the diversity of the images/textures that we map onto objects, ranging from simple pairs of images, up to the entire CIFAR-10 dataset.

We first demonstrate that a linear vision model is sufficient for agent survival when only two visually distinct objects are task-relevant, even in the presence of task-irrelevant distractors, yet increasing the visual diversity of relevant objects eventually necessitates a nonlinear visual system. We next show that the structure of model receptive fields (RFs) is determined by the statistics of task-relevant images, and is unaffected by task-irrelevant images. Lastly, we show that sparsity and efficiency constraints can silence unnecessary channels in the vision model, and allow the agent to survive with orders of magnitude less neural activity.

Our framework allows researchers to explore in silico how ecology shapes visual processing, and how behavioral goals and object relevance emerges from an agent's drive for survival. In future work we aim to study the effects of incorporating functionally different objects, such as obstacles, predators, and mates into the environment of the agent.