Incorporation of avian neurophysiology offers new methods to enhance object tracking models

Abstract

The most successful Computer Vision (CV) models have been inspired by biological systems, most frequently mammalian neurophysiology. While mammalian vision is exceptional at object recognition and classification, we hypothesize that the emphasis on mammalian bioinspiration underserves some computer vision application domains increasingly critical for intelligent systems, like single-object tracking (SOT) in a non-stationary frame. We propose a framework to identify unique mechanisms of the avian visual system that can then be incorporated into conventional SOT models to test how mimicking avian vision modifies current tracking capability. Avian neurophysiology has evolved mechanisms to ensure successful tracking while in motion. From this framework, three experiments were selected to explore mechanisms tied to tracking and unique to the avian visual system: image filtering (retinal structure), motion discrimination and saliency (Tectofugal Pathway), and image unification (Centrifugal Pathway). For each experiment, we also utilize different bioinspiration approaches. For the retina, an existing eagle-eye-based adaptation mechanism was applied and specific scenarios were observed where this prevents tracking loss. For the Tectofugal Pathway, a spiking neural network was developed based on the pigeon nucleus rotundus that was able to identify salient objects better than a metric-based method. For the Centrifugal Pathway, a recurrent modification to a convolutional neural network based Event+RGB tracker was proposed that had better initial accuracy. These results show improvements in specific cases and pave the way for more detailed analysis. The resulting framework has the potential to evaluate bio-inspired modifications to computer vision pipelines.