Learning fMRI-guided predictor of video shot changes

Abstract

In natural stimulus fMRI during video watching, it is natural to postulate that a human participant's attention system would respond to shot changes of the video stream. However, quantitative assessment of the relationship between the functional activities of the attention system and the dynamics of video shot changes has been rarely explored yet. This paper presents a novel framework for modeling the functional interactions and dynamics within the human attention system via natural stimulus fMRI and learning fMRI-based brain response predictors of video shot changes. The basic idea is to derive sub-networks from the attention system and correlate the functional synchronization measurements of these sub-networks with video shot changes. Then, the most relevant sub-networks are identified from training samples and a regression model is constructed as the predictor of video shot changes. In the application stage, the learned predictive models demonstrated good accuracy of estimating video shot changes in independent testing datasets. This study suggests that the fMRI-guided predictive models of functional attention network activities can potentially serve as the brain decoders of video shot changes.