Using fMRI to examine nonlinear mixed selectivity tuning to task and category in the human brain

Abstract

Abstract Recent experimental and theoretical work has shown that nonlinear mixed selectivity, where neurons exhibit interaction effects in their tuning to multiple variables (e.g., stimulus and task) plays a key role in enabling the primate brain to form representations that can adapt to changing task contexts. Thus far all such studies have relied on invasive neural recording techniques. In this study, we demonstrate the feasibility of measuring nonlinear mixed selectivity tuning in the human brain noninvasively using fMRI pattern decoding. To do so, we examined the joint representation of object category and task information across human early, ventral stream, and dorsal stream areas while participants performed either an oddball detection task or a one-back repetition detection task on the same stimuli. These tasks were chosen to equate spatial, object-based, and feature-based attention, in order to test whether task modulations of visual representations still occur when the inputs to visual processing are kept constant between the two tasks, with only the subsequent cognitive operations varying. We found moderate but significant evidence for nonlinear mixed selectivity tuning to object category and task in fMRI response patterns in both human ventral and dorsal areas, suggesting that neurons exhibiting nonlinear mixed selectivity for category and task not only exist in these regions, but cluster at a scale visible to fMRI. Importantly, while such coding in ventral areas corresponds to a rotation or shift in the object representational geometry without changing the representational content (i.e., with the relative similarity among the categories preserved), nonlinear mixed selectivity coding in dorsal areas corresponds to a reshaping of representational geometry, indicative of a change in representational content.