Abstract
SummaryAchievement of human-level image recognition by deep neural networks (DNNs) has spurred interest in whether and how DNNs are brain-like. Both DNNs and the visual cortex perform hierarchical processing, and correspondence has been shown between hierarchical visual areas and DNN layers in representing visual features. Here, we propose the brain hierarchy (BH) score as a metric to quantify the degree of hierarchical correspondence based on neural decoding and encoding analyses where DNN unit activations and human brain activity are predicted from each other. We find that BH scores for 29 pre-trained DNNs with various architectures are negatively correlated with image recognition performance, thus indicating that recently developed high-performance DNNs are not necessarily brain-like. Experimental manipulations of DNN models suggest that single-path sequential feedforward architecture with broad spatial integration is critical to brain-like hierarchy. Our method may provide new ways to design DNNs in light of their representational homology to the brain.
Highlights
The design of deep neural networks (DNNs) is typically based on brain-like multi-stage hierarchical structures
Using the brain hierarchy (BH) score and the distributions of the top ROIs for DNN units in each layer, we examine the degree of hierarchical similarity to the brain in 29 representative DNNs that were pre-trained on an object classification task, including AlexNet, the VGG family, the ResNet family, the DenseNet family, and the Inception family
We investigate the correlations between BH scores and ImageNet top-1 accuracy to determine whether high-performance DNNs are hierarchically more brain-like, while testing the robustness of the BH score and the relationship with other measures of brain–DNN similarity
Summary
The design of deep neural networks (DNNs) is typically based on brain-like multi-stage hierarchical structures. The brain areas that best predict unit activations in a DNN layer are reported to have gradually shifted from lower (e.g., V1, V2, and V3) to middle and higher visual areas (e.g., V4, lateral occipital complex, fusiform face area, and parahippocampal place area) as the target DNN layer shifts from lower to higher. This finding indicates the functional similarity of the hierarchical representations between DNNs and biological brains
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
