Efficient feature extraction is a pivotal requirement for Deep Neural Network (DNN) models, particularly in the realm of visual tasks where effective feature extraction relies on well-designed receptive fields. Nevertheless, the evaluation and fulfillment of the criteria for “effective receptive fields”, along with their essential conditions, continue to pose numerous unresolved mysteries. In this Analysis we amalgamated principles from the field of biologically inspired vision to reevaluate the role of receptive fields within neural networks, and have proposed a comprehensive design theory aimed at characterizing and guiding the research of efficient DNNs. Our design theory draws from both the principles of biological vision and the latest advances in DNN design theory. It encompasses five fundamental axes of receptive field design research: the Global Artificial Neuron Workspace (GANW), Layered Processing Theory (LPT), Multi-Receptive Field Integration Design (M-RFID), Heterogeneous Receptive Field (HRF), and Receptive Field Plasticity Theory (RFPT). We have applied these design theories to validate traditional and cutting-edge neural networks, conducting in-depth analyses of the outcomes to provide a summary of the current landscape of research on visual receptive fields. Furthermore, we have classified current visual receptive fields into three categories: sample-specific local receptive fields, global spatial receptive fields, and adaptive dynamic receptive fields. From the perspective of perceptual fields and network lightweight design, we have introduced some classic and state-of-the-art DNN models, with a particular emphasis on those employing innovative algorithms that have yielded the most recent achievements. We have concluded by highlighting the key areas of interest in the field of visual receptive fields for future research endeavors.
Read full abstract