The impact of reward value on early sensory processing and its interaction with selective attention

Abstract

Reward value affects the earliest stages of sensory perception. Whereas a host of previous studies have investigated the underlying mechanisms of reward-driven modulation of visual perception, reward effects in other sensory modalities have remained underexplored. Specifically, it has remained unknown how reward signals should be coordinated and communicated across sensory modalities. The current PhD thesis aimed to gain insight into the underlying mechanisms of reward-driven modulation of perception and its interaction with attention across sensory modalities. To this end, three experiments were conducted to identify the behavioral and electrophysiological correlates of reward effects. In Study 1, we found that high reward, task-irrelevant visual cues (intra-modal) elicited an early suppression of visual event-related potentials (ERPs). High reward auditory cues (cross-modal), on the other hand, led to a later modulation of visual ERPs and facilitated behavioral performance. Study 2 tested the dependence of reward effects on the spatial and temporal arrangement of intra-modal and cross-modal cues relative to the target, and showed that each reward cue maximally exerts its effect under a specific size of attentional focus. Study 3 explicitly manipulated the spatial attention and tested how reward associations of an audiovisual stimulus influence the allocation of attention. We found that auditory rewards enhanced the attentional modulation of both visual and auditory ERPs. Interestingly, although visual rewards did not lead to a distinguishable ERP modulation, they led to strong modulations when they were combined with auditory rewards, suggesting that integration across modalities boosts the reward effects. Taken together, the current PhD thesis identified the behavioral and neural signatures of reward-driven modulation of perception under different modes of reward signaling and different degrees of attentional engagement. Our findings inspire a two-stage model of reward processing, with local, intra-modal reward effects occurring at an early stage and long-range, multimodal reward effects arising at a later stage. Cross-modal reward signals have important ramifications for clinical applications where the impaired function of one sense can be rehabilitated by motivational signals conveyed through another sensory modality.