3d Vision
3D vision
N/A
PAVIS is looking to strengthen its activities on 3D multi-modal scene understanding. The research will focus on novel ML and CV methods that efficiently incorporate priors and constraints from world physical models and semantic priors, derived from vision, language models, or other modalities. The project will explore the interplay between vision and large language models to address tasks in 3D reasoning, visual (re-)localization, active vision, and neural/geometrical novel view rendering. The aim is to develop models applicable to interdisciplinary research, including drug discovery and robotics, utilizing in-house robotics platforms and HPC computational facilities.
Seeing the world through moving photoreceptors - binocular photomechanical microsaccades give fruit fly hyperacute 3D-vision
To move efficiently, animals must continuously work out their x,y,z positions with respect to real-world objects, and many animals have a pair of eyes to achieve this. How photoreceptors actively sample the eyes’ optical image disparity is not understood because this fundamental information-limiting step has not been investigated in vivo over the eyes’ whole sampling matrix. This integrative multiscale study will advance our current understanding of stereopsis from static image disparity comparison to a morphodynamic active sampling theory. It shows how photomechanical photoreceptor microsaccades enable Drosophila superresolution three-dimensional vision and proposes neural computations for accurately predicting these flies’ depth-perception dynamics, limits, and visual behaviors.