Perceptual Aftereffects Research Articles

How Video Passthrough Headsets Influence Perception of Self and Others.

With the increasing adoption of mixed reality (MR) headsets with video passthrough functionality, concerns over perceptual and social effects have surfaced. Building on prior qualitative findings,1 this study quantitatively investigates the impact of video passthrough on users. Forty participants completed a body transfer task twice, once while wearing a headset in video passthrough and once without a headset. Using video passthrough induced simulator sickness, created social absence (another person in the physical room feels less present), altered self-reported body schema, and distorted distance perception. On the other hand, compared with past research that showed perceptual aftereffects from video passthrough, the current study found none. We discuss the broader implications for the widespread adoption of MR headsets and their impact on theories surrounding presence and body transfer.

Inattentional aftereffects: The role of attention on the strength of the motion aftereffect.

The way that attention affects the processing of visual information is one of the most intriguing fields in the study of visual perception. One way to examine this interaction is by studying the way perceptual aftereffects are modulated by attention. In the present study, we have manipulated attention during adaptation to translational motion generated by coherently moving random dots, in order to investigate the effect of the distraction of attention on the strength of the peripheral dynamic motion aftereffect (MAE). A foveal rapid serial visual presentation task (RSVP) of varying difficulty was introduced during the adaptation period while the adaptation and test stimuli were presented peripherally. Furthermore, to examine the interaction between the physical characteristics of the stimulus and attention, we have manipulated the motion coherence level of the adaptation stimuli. Our results suggested that the removal of attention through an irrelevant task modulated the MAE's magnitude moderately and that such an effect depends on the stimulus strength. We also showed that the MAE still persists with subthreshold and unattended stimuli, suggesting that perhaps attention is not required for the complete development of the MAE.

Perceptual aftereffects of adiposity transfer from hands to whole bodies.

Adaptation aftereffects for features such as identity and gender have been shown to transfer between faces and bodies, and faces and body parts, i.e. hands. However, no studies have investigated transfer of adaptation aftereffects between whole bodies and body parts. The present study investigated whether visual adaptation aftereffects transfer between hands and whole bodies in the context of adiposity judgements (i.e. how thin or fat a body is). On each trial, participants had to decide whether the body they saw was thinner or fatter than average. Participants performed the task before and after exposure to a thin/fat hand. Consistent with body adaptation studies, after exposure to a slim hand participants judged subsequently presented bodies to be fatter than after adaptation to a fat hand. These results suggest that there may be links between visual representations of body adiposity for whole bodies and body parts.

Adaptation to walking direction in biological motion.

The direction that we see another person walking provides us with an important cue to their intentions, but little is known about how the brain encodes walking direction across a neuronal population. The current study used an adaptation technique to investigate the sensory coding of perceived walking direction. We measured perceived walking direction of point-light stimuli before and after adaptation, and found that adaptation to a specific walking direction resulted in repulsive perceptual aftereffects. The magnitude of these aftereffects was tuned to the walking direction of the adaptor relative to the test, with local repulsion of perceived walking direction for test stimuli oriented on either side of the adapted walking direction. The specific tuning profiles that we observed are well explained by a population-coding model, in which perceived walking direction is coded in terms of the relative activity across a bank of sensory channels with peak tuning distributed across the full 360° range of walking directions. Further experiments showed specificity in how horizontal (azimuth) walking direction is coded when moving away from the observer compared to when moving toward the observer. Moreover, there was clear specificity in these perceptual aftereffects for walking direction compared to a nonbiological form of 3D motion (a rotating sphere). These results indicate the existence of neural mechanisms in the human visual system tuned to specific walking directions, provide insight into the number of sensory channels and how their responses are combined to encode walking direction, and demonstrate the specificity of adaptation to biological motion. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Perceptual Doping: A Hypothesis on How Early Audiovisual Speech Stimulation Enhances Subsequent Auditory Speech Processing.

Face-to-face communication is one of the most common means of communication in daily life. We benefit from both auditory and visual speech signals that lead to better language understanding. People prefer face-to-face communication when access to auditory speech cues is limited because of background noise in the surrounding environment or in the case of hearing impairment. We demonstrated that an early, short period of exposure to audiovisual speech stimuli facilitates subsequent auditory processing of speech stimuli for correct identification, but early auditory exposure does not. We called this effect "perceptual doping" as an early audiovisual speech stimulation dopes or recalibrates auditory phonological and lexical maps in the mental lexicon in a way that results in better processing of auditory speech signals for correct identification. This short opinion paper provides an overview of perceptual doping and how it differs from similar auditory perceptual aftereffects following exposure to audiovisual speech materials, its underlying cognitive mechanism, and its potential usefulness in the aural rehabilitation of people with hearing difficulties.

Spatial selectivity in adaptation to gaze direction.

A person's focus of attention is conveyed by the direction of their eyes and face, providing a simple visual cue fundamental to social interaction. A growing body of research examines the visual mechanisms that encode the direction of another person's gaze as we observe them. Here we investigate the spatial receptive field properties of these mechanisms, by testing the spatial selectivity of sensory adaptation to gaze direction. Human observers were adapted to faces with averted gaze presented in one visual hemifield, then tested in their perception of gaze direction for faces presented in the same or opposite hemifield. Adaptation caused strong, repulsive perceptual aftereffects, but only for faces presented in the same hemifield as the adapter. This occurred even though adapting and test stimuli were in the same external location across saccades. Hence, there was clear evidence for retinotopic adaptation and a relative lack of either spatiotopic or spatially invariant adaptation. These results indicate that adaptable representations of gaze direction in the human visual system have retinotopic spatial receptive fields. This strategy of coding others' direction of gaze with positional specificity relative to one's own eye position may facilitate key functions of gaze perception, such as socially cued shifts in visual attention.

The role of eye-specific attention in ocular dominance plasticity.

It is well known how selective attention biases information processing in real time, but few work investigates the aftereffects of prolonged attention, letalone the underlying neural mechanisms. To examine perceptual aftereffect after prolonged attention to a monocular pathway, movie images played normally were presented to normal adult's one eye (attended eye), while movie images of the same episode but played backwards were presented to the opposite eye (unattended eye). One hour of watching this dichoptic movie caused a shift of perceptual ocular dominance towards the unattended eye. Interestingly, the aftereffect positively correlated with the advantage of neural activity for the attended-eye over unattended-eye signals at the frontal electrodes measured with steady-state visual evoked potentials. Moreover, the aftereffect disappeared when interocular competition was minimized during adaptation. These results suggest that top-down eye-specific attention can induce ocular dominance plasticity through binocular rivalry mechanisms. The present study opens the route to explain at least part of short-term ocular dominance plasticity with the ocular-opponency-neuron model, which may be an interesting complement to the homeostatic compensation theory.

Vocal emotion adaptation aftereffects within and across speaker genders: Roles of timbre and fundamental frequency

While the human perceptual system constantly adapts to the environment, some of the underlying mechanisms are still poorly understood. For instance, although previous research demonstrated perceptual aftereffects in emotional voice adaptation, the contribution of different vocal cues to these effects is unclear. In two experiments, we used parameter-specific morphing of adaptor voices to investigate the relative roles of fundamental frequency (F0) and timbre in vocal emotion adaptation, using angry and fearful utterances. Participants adapted to voices containing emotion-specific information in either F0 or timbre, with all other parameters kept constant at an intermediate 50% morph level. Full emotional voices and ambiguous voices were used as reference conditions. All adaptor stimuli were either of the same (Experiment 1) or opposite speaker gender (Experiment 2) of subsequently presented target voices. In Experiment 1, we found consistent aftereffects in all adaptation conditions. Crucially, aftereffects following timbre adaptation were much larger than following F0 adaptation and were only marginally smaller than those following full adaptation. In Experiment 2, adaptation aftereffects appeared massively and proportionally reduced, with differences between morph types being no longer significant. These results suggest that timbre plays a larger role than F0 in vocal emotion adaptation, and that vocal emotion adaptation is compromised by eliminating gender-correspondence between adaptor and target stimuli. Our findings also add to mounting evidence suggesting a major role of timbre in auditory adaptation.

A Sensorimotor Numerosity System.

Incoming sensory input provides information for the planning and execution of actions, which yield motor outcomes that are themselves sensory inputs. One dimension where action and perception strongly interact is numerosity perception. Many non-human animals can estimate approximately the number of external elements as well as their own actions, and neurons have been identified that respond to both. Recent psychophysical adaptation studies on humans also provide evidence for neural mechanisms responding to both the number of externally generated events and self-produced actions. Here we advance the idea that these strong connections may arise from dedicated sensorimotor mechanisms in the brain, part of a more generalized system interfacing action with the processing of other quantitative magnitudes such as space and time.

Preventing action slows down performance in perceptual judgment

Action and vision are known to be tightly coupled with each other. In a previous study, we found that repeatedly grasping an object without any visual feedback might result in a perceptual aftereffect when the object was visually presented in the context of a perceptual judgement task. In this study, we explored whether and how such an effect could be modulated by presenting the object behind a transparent barrier. Our conjecture was that if perceptual judgment relies, in part at least, on the same processes and representations as those involved in action, then one should expect to find a slowdown in judgment performance when the target object looks to be out of reach. And this was what we actually found. This indicates that not only acting upon an object but also being prevented from acting upon it can affect how the object is perceptually judged.

A crowd of emotional voices influences the perception of emotional faces: Using adaptation, stimulus salience, and attention to probe audio-visual interactions for emotional stimuli.

Correctly assessing the emotional state of others is a crucial part of social interaction. While facial expressions provide much information, faces are often not viewed in isolation, but occur with concurrent sounds, usually voices, which also provide information about the emotion being portrayed. Many studies have examined the crossmodal processing of faces and sounds, but results have been mixed, with different paradigms yielding different results. Using a psychophysical adaptation paradigm, we carried out a series of four experiments to determine whether there is a perceptual advantage when faces and voices match in emotion (congruent), versus when they do not match (incongruent). We presented a single face and a crowd of voices, a crowd of faces and a crowd of voices, a single face of reduced salience and a crowd of voices, and tested this last condition with and without attention directed to the emotion in the face. While we observed aftereffects in the hypothesized direction (adaptation to faces conveying positive emotion yielded negative, contrastive, perceptual aftereffects), we only found a congruent advantage (stronger adaptation effects) when faces were attended and of reduced salience, in line with the theory of inverse effectiveness.

Optical, Perceptual and Adaptational Implications of different Presbyopic corrections

This publication represents the summary of results of the doctoral thesis Presbyopia Corrections: Optical, Perceptual and Adaptational Implications presented at the Universidad Complutense de Madrid, Spain. Presbyopia, the physiological inability of the aging crystalline lens to accommodate for objects at near distance, can be treated using a multitude of optical corrections. Systematic evaluation of the visual system’s ability to interact with these corrections will help in optimization of their performance. In this thesis we studied optical and perceptual performance of different presbyopic corrections such as alternating vision, monovision and simultaneous vision and the effect of adaptation on perceptual performances. We used custom-developed adaptive optics setup to measure and correct ocular aberrations and projected manipulated images simulating pure simultaneous vision corrections of different far/near energy profiles and near additions. We also developed and validated an on-bench and a hand-held simultaneous vision simulator to optically simulate pure or segmented simultaneous vision corrections. Psychophysical methods were employed to study the changes in perceptual quality and after-effects of adaptation. We performed numerical simulations to predict perceptual performance from the ocular aberrations of the subjects. We demonstrated that mechanism of adaptation to simultaneous vision is similar to that of blur adaptation, influenced mostly by retinal image contrast and that systematic changes in visual and perceptual performance influenced by multifocal design and testing distance. The ocular optics of the subject’s eye correlated significantly with visual performance and was associated with the intersubject variability in performance. We found that a cyclopean locus for perception and adaptation, in subjects with different blur magnitude between eyes, influenced by the eyes with better optical quality. The internal code for blur was also influenced by the eye with better optical quality, both in orientation and magnitude. Our results confirm that the existing optical solutions should be chosen based on the subjective needs and the ocular optics would be an ideal starting point to customize optical solutions of presbyopia for optimal performance. We also demonstrated the usefulness of rapid assessment of perception to multifocal designs using the handheld, seethrough simultaneous vision simulator.

Gender and the Body Size Aftereffect: Implications for Neural Processing.

Prolonged exposure to wide (thin) bodies causes a perceptual aftereffect such that subsequently viewed bodies appear thinner (wider) than they actually are. This phenomenon is known as visual adaptation. We used the adaptation paradigm to examine the gender selectivity of the neural mechanisms encoding body size and shape. Observers adjusted female and male test bodies to appear normal-sized both before and after adaptation to bodies digitally altered to appear heavier or lighter. In Experiment 1, observers adapted simultaneously to bodies of each gender distorted in opposite directions, e.g., thin females and wide males. The direction of resultant aftereffects was contingent on the gender of the test stimulus, such that in this example female test bodies appeared wider while male test bodies appeared thinner. This indicates at least some separation of the neural mechanisms processing body size and shape for the two genders. In Experiment 2, adaptation involved either wide females, thin females, wide males or thin males. Aftereffects were present in all conditions, but were stronger when test and adaptation genders were congruent, suggesting some overlap in the tuning of gender-selective neural mechanisms. Given that visual adaptation has been implicated in real-world examples of body size and shape misperception (e.g., in anorexia nervosa or obesity), these results may have implications for the development of body image therapies based on the adaptation model.

Rapid recalibration of temporal order judgements: Response bias accounts for contradictory results.

Recent history influences subsequent perception, decision-making and motor behaviours. In this article, we address a discrepancy in the effects of recent sensory history on the perceived timing of auditory and visual stimuli. In the synchrony judgement (SJ) task, similar timing relationships in consecutive trials seem more synchronous (i.e. less like the repeated temporal order). This effect is known as rapid recalibration and is consistent with a negative perceptual aftereffect. Interestingly, the opposite is found in the temporal order judgement (TOJ) task (positive rapid recalibration). We aimed to determine whether a simple bias to repeat judgements on consecutive trials (choice-repetition bias) could account for the discrepant results in these tasks. Preliminary simulations and analyses indicated that a choice-repetition bias could produce apparently positive rapid recalibration in the TOJ and not the SJ task. Our first experiment revealed no evidence of rapid recalibration of TOJs, but negative rapid recalibration of associated confidence. This suggests that timing perception was rapidly recalibrated, but that the negative recalibration effect was obfuscated by a positive bias effect. In our second experiment, we experimentally mitigated the choice-repetition bias effect and found negative rapid recalibration of TOJs. We therefore conclude that timing perception is negatively rapidly recalibrated, and this is observed consistently across timing tasks. These results contribute to a growing body of evidence that indicates multisensory perception is constantly undergoing recalibration, such that perceptual synchrony is maintained. This work also demonstrates that participants' task responses reflect judgements that are contaminated by independent biases of perception and decision-making.

Seeing a Face in a Crowd of Emotional Voices: Changes in Perception and Cortisol in Response to Emotional Information across the Senses.

One source of information we glean from everyday experience, which guides social interaction, is assessing the emotional state of others. Emotional state can be expressed through several modalities: body posture or movements, body odor, touch, facial expression, or the intonation in a voice. Much research has examined emotional processing within one sensory modality or the transfer of emotional processing from one modality to another. Yet, less is known regarding interactions across different modalities when perceiving emotions, despite our common experience of seeing emotion in a face while hearing the corresponding emotion in a voice. Our study examined if visual and auditory emotions of matched valence (congruent) conferred stronger perceptual and physiological effects compared to visual and auditory emotions of unmatched valence (incongruent). We quantified how exposure to emotional faces and/or voices altered perception using psychophysics and how it altered a physiological proxy for stress or arousal using salivary cortisol. While we found no significant advantage of congruent over incongruent emotions, we found that changes in cortisol were associated with perceptual changes. Following exposure to negative emotional content, larger decreases in cortisol, indicative of less stress, correlated with more positive perceptual after-effects, indicative of stronger biases to see neutral faces as happier.

Confidence as a diagnostic tool for perceptual aftereffects

Perceptual judgements are, by nature, a product both of sensation and the cognitive processes responsible for interpreting and reporting subjective experiences. Changed perceptual judgements may thus result from changes in how the world appears (perception), or subsequent interpretation (judgement). This ambiguity has led to persistent debates about how to interpret changes in decision-making, and if higher-order cognitions can change how the world looks, or sounds, or feels. Here we introduce an approach that can help resolve these ambiguities. In three motion-direction experiments, we measured perceptual judgements and subjective confidence. We show that each measure is sensitive to sensory information and can index sensory adaptation. Each measure is also sensitive to decision biases, but response bias impacts the central tendency of decision and confidence distributions differently. Our findings show that subjective confidence, when measured in addition to perceptual decisions, can supply important diagnostic information about the cause of aftereffects.

Adaptive sensory coding of gaze direction in schizophrenia.

Schizophrenia has been associated with differences in how the visual system processes sensory input. A fundamental mechanism that regulates sensory processing in the brain is gain control, whereby the responses of sensory neurons to a given stimulus are modulated in accordance with the spatial and temporal context. Some studies indicate an impairment of certain cortical gain control mechanisms in schizophrenia in low-level vision, reflected, for instance, in how the visual appearance of a stimulus is affected by the presence of other stimuli around it. In the present study, we investigated higher-level, social vision in schizophrenia, namely the perception of other people's direction of gaze (i.e. a type of face processing). Recent computational modelling work indicates that perceptual aftereffects—changes in perception that occur following repeated exposure to faces that display a specific direction of gaze—are indicative of two distinct forms of gain control involved in the coding of gaze direction across sensory neurons. We find that individuals with schizophrenia display strong perceptual aftereffects following repeated exposure to faces with averted gaze, and a modelling analysis indicates similarly robust gain control in the form of (i) short-term adjustment of channel sensitivities in response to the recent sensory history and (ii) divisive normalization of the encoded gaze direction. Together, this speaks to the typical coding of other people's direction of gaze in the visual system in schizophrenia, including flexible gain control, despite the social–cognitive impairments that can occur in this condition.

Rapid Adaptation to the Timbre of Natural Sounds

Timbre, the unique quality of a sound that points to its source, allows us to quickly identify a loved one’s voice in a crowd and distinguish a buzzy, bright trumpet from a warm cello. Despite its importance for perceiving the richness of auditory objects, timbre is a relatively poorly understood feature of sounds. Here we demonstrate for the first time that listeners adapt to the timbre of a wide variety of natural sounds. For each of several sound classes, participants were repeatedly exposed to two sounds (e.g., clarinet and oboe, male and female voice) that formed the endpoints of a morphed continuum. Adaptation to timbre resulted in consistent perceptual aftereffects, such that hearing sound A significantly altered perception of a neutral morph between A and B, making it sound more like B. Furthermore, these aftereffects were robust to moderate pitch changes, suggesting that adaptation to timbral features used for object identification drives these effects, analogous to face adaptation in vision.

The high-level basis of body adaptation.

Prolonged visual exposure, or ‘adaptation’, to thin (wide) bodies causes a perceptual aftereffect such that subsequently seen bodies appear wider (thinner) than they actually are. Here, we conducted two experiments investigating the effect of rotating the orientation of the test stimuli by 90° from that of the adaptor. Aftereffects were maximal when adapting and test bodies had the same orientation. When they differed, the axis of the perceived distortion changed with the orientation of the body. Experiment 1 demonstrated a 58% transfer of the aftereffect across orientations. Experiment 2 demonstrated an even greater degree of aftereffect transfer when the influence of low-level mechanisms was reduced further by using adaptation and test stimuli with different sizes. These results indicate that the body aftereffect is mediated primarily by high-level object-based processes, with low-level retinotopic mechanisms playing only a minor role. The influence of these low-level processes is further reduced when test stimuli differ in size from adaptation stimuli.

知觉负载对注视知觉适应后效的影响 <sup>*</sup>

知觉负载对注视知觉适应后效的影响 <sup>*</sup>