TALK DETAILS VIDEO RECORDING DOI QR CODE RELATED TALKS SEARCH POSTERS

TALK DETAILS

Saccade Mechanisms for Image Classification, Object Detection and Tracking

Zachary Daniels — Saurabh Farkya, Zachary Daniels, Aswin Nadamuni Raghavan, David Zhang, Michael Piacentino

Show Affils
First Author
► Zachary Daniels

Contributors
► Saurabh Farkya — SRI International
► Zachary Daniels — SRI International
► Aswin Nadamuni Raghavan — SRI International
► David Zhang — SRI International
► Michael Piacentino — SRI International
28 September 2022
We examine how the saccade mechanism from biological vision can be used to make deep neural networks more efficient for classification and object detection problems. Our proposed approach is based on the ideas of attention-driven visual processing and saccades, miniature eye movements influenced by attention. We conduct experiments by analyzing: i) the robustness of different deep neural network (DNN) feature extractors to partially-sensed images for image classification and object detection, and ii) the utility of saccades in masking image patches for image classification and object tracking. Experiments with convolutional nets (ResNet-18) and transformer-based models (ViT, DETR, TransTrack) are conducted on several datasets (CIFAR-10, DAVSOD, MSCOCO, and MOT17). Our experiments show intelligent data reduction via learning to mimic human saccades when used in conjunction with state-of-the-art DNNs for classification, detection, and tracking tasks. We observed minimal drop in performance for the classification and detection tasks while only using about 30% of the original sensor data. We discuss how the saccade mechanism can inform hardware design via "in-pixel" processing.
doi.org/10.57736/nmc-05d8-da14📋

VIDEO RECORDING

QR CODE

TALKS YOU MIGHT BE INTERESTED IN