Computational models of category-selective brain regions enable high-throughput tests of selectivity

N Apurva Ratan Murty,Nancy Kanwisher,Pouya Bashivan,James J Dicarlo,Alex Abate

doi:10.1038/s41467-021-25409-6

Abstract

Cortical regions apparently selective to faces, places, and bodies have provided important evidence for domain-specific theories of human cognition, development, and evolution. But claims of category selectivity are not quantitatively precise and remain vulnerable to empirical refutation. Here we develop artificial neural network-based encoding models that accurately predict the response to novel images in the fusiform face area, parahippocampal place area, and extrastriate body area, outperforming descriptive models and experts. We use these models to subject claims of category selectivity to strong tests, by screening for and synthesizing images predicted to produce high responses. We find that these high-response-predicted images are all unambiguous members of the hypothesized preferred category for each region. These results provide accurate, image-computable encoding models of each category-selective region, strengthen evidence for domain specificity in the brain, and point the way for future research characterizing the functional organization of the brain with unprecedented computational precision.

Highlights

Cortical regions apparently selective to faces, places, and bodies have provided important evidence for domain-specific theories of human cognition, development, and evolution
Specific artificial neural networks (ANNs) are considered our most quantitatively accurate computational models of visual processing in the primate ventral visual stream[21]. It remains unclear whether or how the understanding provided by these models engages with previous theories of visual processing in the brain[24], or whether they even represent any significant advance in our understanding beyond what is already known from decades of published work on these regions. We addressed these questions by collecting highquality event-related functional MRI responses in the fusiform face area (FFA), parahippocampal place area (PPA), and EBA and screening a large number of ANN-based models of the ventral stream for their ability to predict observed responses in each region
How well do computational models of the ventral stream predict the observed response to natural stimuli in the FFA, PPA, and EBA? To find out, we modeled the average response across participants of each of six functionally-defined regions of interest (fROIs) to 185 natural images using a regression-based model-to-brain alignment approach[20,21,23,26,31,32] (Fig. 1)

Summary

Introduction

Cortical regions apparently selective to faces, places, and bodies have provided important evidence for domain-specific theories of human cognition, development, and evolution. We develop artificial neural network-based encoding models that accurately predict the response to novel images in the fusiform face area, parahippocampal place area, and extrastriate body area, outperforming descriptive models and experts We use these models to subject claims of category selectivity to strong tests, by screening for and synthesizing images predicted to produce high responses. The internal representations developed at different processing stages within these ANNs mirror the hierarchical organization of the visual cortex[17,18,19], and activations in these networks can be linearly combined to accurately predict the observed response to previously unseen images at different stages of the visual processing hierarchy[20,21,22,23] For these reasons, specific ANNs are considered our most quantitatively accurate computational models of visual processing in the primate ventral visual stream[21]. This method enables us to turbo-charge the search for counterevidence to the claimed selectivity of the FFA, PPA, and EBA, thereby conducting strong tests of longstanding hypotheses about the category selectivity of each region

Objectives

Methods

Results

Conclusion