Coupling Of Perception Research Articles

Perceptual coupling in human dyads: Kinematics does not affect interpersonal synchronization

The minimal, essential condition for individuals to interact is that they exchange information via at least one sensory channel. Once informational coupling is established, it enables basic forms of coordinated behavior to spontaneously emerge from the interaction. Our previous study revealed different coordination dynamics in dyads engaged in a joint finger-tapping task based on visual versus auditory coupling. This observation led us to propose the ‘modality-dependent hypothesis', which posits that coordination dynamics are influenced by the sensory modality mediating informational coupling. However, recognizing that different modalities have inherent differences in accessing spatiotemporal features of perceived movement, we formulated the alternative ‘kinematic hypothesis'. This hypothesis posits that differences in dynamics would vanish given equivalent kinematic information across modalities. The study involved forty (N = 40) participants, grouped into twenty (N = 20) dyads, who engaged in a joint finger-tapping task. This task was conducted under varying conditions of visual and auditory coupling, with manipulations in the access to kinematic information, categorized as discrete and continuous. Contrary to our initial predictions, the results strongly supported the ‘modality-dependent hypothesis'. We observed that visual and auditory coupling consistently yielded distinct attractor dynamics, regardless of the access to kinematic information. Furthermore, all conditions of auditory coupling resulted in higher levels of synchronization than their visual counterparts. These findings suggest that the differences in interpersonal synchronization are predominantly influenced by the sensory modality, rather than the continuity of kinematic information. Our study highlights the significance of sensorimotor interactions in interpersonal synchronization and addresses the potential of sonification strategies in supporting motor training and rehabilitation.

Bimodal moment-by-moment coupling in perceptual multistability.

Multistable perception occurs in all sensory modalities, and there is ongoing theoretical debate about whether there are overarching mechanisms driving multistability across modalities. Here we study whether multistable percepts are coupled across vision and audition on a moment-by-moment basis. To assess perception simultaneously for both modalities without provoking a dual-task situation, we query auditory perception by direct report, while measuring visual perception indirectly via eye movements. A support-vector-machine (SVM)-based classifier allows us to decode visual perception from the eye-tracking data on a moment-by-moment basis. For each timepoint, we compare visual percept (SVM output) and auditory percept (report) and quantify the co-occurrence of integrated (one-object) or segregated (two-objects) interpretations in the two modalities. Our results show an above-chance coupling of auditory and visual perceptual interpretations. By titrating stimulus parameters toward an approximately symmetric distribution of integrated and segregated percepts for each modality and individual, we minimize the amount of coupling expected by chance. Because of the nature of our task, we can rule out that the coupling stems from postperceptual levels (i.e., decision or response interference). Our results thus indicate moment-by-moment perceptual coupling in the resolution of visual and auditory multistability, lending support to theories that postulate joint mechanisms for multistable perception across the senses.

Distinctive and Complementary Roles of Default Mode Network Subsystems in Semantic Cognition.

The default mode network (DMN) typically deactivates to external tasks, yet supports semantic cognition. It comprises medial temporal (MT), core, and frontotemporal (FT) subsystems, but its functional organization is unclear: the requirement for perceptual coupling versus decoupling, input modality (visual/verbal), type of information (social/spatial), and control demands all potentially affect its recruitment. We examined the effect of these factors on activation and deactivation of DMN subsystems during semantic cognition, across four task-based human functional magnetic resonance imaging (fMRI) datasets, and localized these responses in whole-brain state space defined by gradients of intrinsic connectivity. FT showed activation consistent with a central role across domains, tasks, and modalities, although it was most responsive to abstract, verbal tasks; this subsystem uniquely showed more "tuned" states characterized by increases in both activation and deactivation when semantic retrieval demands were higher. MT also activated to both perceptually coupled (scenes) and decoupled (autobiographical memory) tasks and showed stronger responses to picture associations, consistent with a role in scene construction. Core DMN consistently showed deactivation, especially to externally oriented tasks. These diverse contributions of DMN subsystems to semantic cognition were related to their location on intrinsic connectivity gradients: activation was closer to the sensory-motor cortex than deactivation, particularly for FT and MT, while activation for core DMN was distant from both visual cortex and cognitive control. These results reveal distinctive yet complementary DMN responses: MT and FT support different memory-based representations that are accessed externally and internally, while deactivation in core DMN is associated with demanding, external semantic tasks.

Autonomous Learning of Page Flipping Movements via Tactile Feedback

Robotic manipulation is challenging when both the objects being manipulated and the tactile sensors are deformable. In this article, we addressed the interplay between the manipulation of deformable objects, tactile sensing, and model-free reinforcement learning on a real robot. We showed how a real robot can learn to manipulate a deformable, thin-shell object via feedback from deformable, multimodal tactile sensors. We addressed the learning of a page flipping task using a two-stage approach. For the first stage, we learned nominal page flipping trajectories for two page sizes by constructing a reward function that quantifies functional task performance from the perspective of tactile sensing. For the second stage, we learned adapted trajectories using tactile-driven perceptual coupling, with an intuitive assumption that, while the page flipping trajectories for different task contexts (page sizes) might differ, similar tactile feedback should be expected from functional trajectories for each context. We also investigated the quality of information encoded by two different representations of tactile sensing data: one based on the artificial apical tuft of bio-inspired tactile sensors, and another based on principal component analysis eigenvalues. The results and effectiveness of our learning framework were demonstrated on a real seven-degree of freedom robot arm and gripper outfitted with tactile sensors.

Perceptual coupling and decoupling of the default mode network during mind-wandering and reading.

While reading, our mind can wander to unrelated autobiographical information, creating a perceptually decoupled state detrimental to narrative comprehension. To understand how this mind-wandering state emerges, we asked whether retrieving autobiographical content necessitates functional disengagement from visual input. In Experiment 1, brain activity was recorded using functional magnetic resonance imaging (fMRI) in an experimental situation mimicking naturally occurring mind-wandering, allowing us to precisely delineate neural regions involved in memory and reading. Individuals read expository texts and ignored personally relevant autobiographical memories, as well as the opposite situation. Medial regions of the default mode network (DMN) were recruited during memory retrieval. In contrast, left temporal and lateral prefrontal regions of the DMN, as well as ventral visual cortex, were recruited when reading for comprehension. Experiment two used functional connectivity both at rest and during tasks to establish that (i) DMN regions linked to memory are more functionally decoupled from regions of ventral visual cortex than regions in the same network engaged when reading; and (ii) individuals with more self-generated mental contents and poorer comprehension, while reading in the lab, showed more decoupling between visually connected DMN sites important for reading and primary visual cortex. A similar pattern of connectivity was found in Experiment 1, with greater coupling between this DMN site and visual cortex when participants reported greater focus on reading in the face of conflict from autobiographical memory cues; moreover, the retrieval of personally relevant memories increased the decoupling of these sites. These converging data suggest we lose track of the narrative when our minds wander because generating autobiographical mental content relies on cortical regions within the DMN which are functionally decoupled from ventral visual regions engaged during reading.

Perceptual coupling and decoupling are associated with individual differences in working memory encoding and maintenance.

Working memory (WM) allows goal-relevant information to be encoded and maintained in mind, even when the contents of WM are incongruent with the immediate environment. While regions of heteromodal cortex are important for WM, the neural mechanisms that relate to individual differences in the encoding and maintenance of goal-relevant information remain unclear. Here, we used behavioral correlates of two large-scale heteromodal networks at rest, the default mode (DMN) and frontoparietal (FPN) networks, to understand their contributions to distinct features of WM. We assessed each individual’s ability to resist distracting information during the encoding and maintenance phases of a visuospatial WM task. Individuals with stronger connectivity of DMN with medial visual and retrosplenial cortex were less affected by encoding distraction. Conversely, weaker connectivity of both DMN and FPN with visual regions was associated with better WM performance when target information was no longer in the environment and distractors were presented in the maintenance phase. Our study suggests that stronger coupling between heteromodal cortex and visual–spatial regions supports WM encoding by reducing the influence of concurrently presented distractors, while weaker visual coupling is associated with better maintenance of goal-relevant information because it relates to the capacity to ignore task-irrelevant changes in the environment.

Change not State: Perceptual coupling in multistable displays reflects transient bias induced by perceptual change

We investigated how changes in dynamic spatial context influence visual perception. Specifically, we reexamined the perceptual coupling phenomenon when two multistable displays viewed simultaneously tend to be in the same dominant state and switch in accord. Current models assume this interaction reflecting mutual bias produced by a dominant perceptual state. In contrast, we demonstrate that influence of spatial context is strongest when perception changes. First, we replicated earlier work using bistable kinetic-depth effect displays, then extended it by employing asynchronous presentation to show that perceptual coupling cannot be accounted for by the static context provided by perceptually dominant states. Next, we demonstrated that perceptual coupling reflects transient bias induced by perceptual change, both in ambiguous and disambiguated displays. We used a hierarchical Bayesian model to characterize its timing, demonstrating that the transient bias is induced 50–70 ms after the exogenous trigger event and decays within ~200–300 ms. Both endogenous and exogenous switches led to quantitatively and qualitatively similar perceptual consequences, activating similar perceptual reevaluation mechanisms within a spatial surround. We explain how they can be understood within a transient selective visual attention framework or using local lateral connections within sensory representations. We suggest that observed perceptual effects reflect general mechanisms of perceptual inference for dynamic visual scene perception.

Perceptual Coupling Based on Depth and Motion Cues in Stereovision-Impaired Subjects.

When an object is partially occluded, the different parts of the object have to be perceptually coupled. Cues that can be used for perceptual coupling are, for instance, depth ordering and visual motion information. In subjects with impaired stereovision, the brain is less able to use stereoscopic depth cues, making them more reliant on other cues. Therefore, our hypothesis is that stereovision-impaired subjects have stronger motion coupling than stereoscopic subjects. We compared perceptual coupling in 8 stereoscopic and 10 stereovision-impaired subjects, using random moving dot patterns that defined an ambiguous rotating cylinder and a coaxially presented nonambiguous half cylinder. Our results show that, whereas stereoscopic subjects exhibit significant coupling in the far plane, stereovision-impaired subjects show no coupling and under our conditions also no stronger motion coupling than stereoscopic subjects.

Interpersonal visual interaction induces local and global stabilisation of rhythmic coordination

Perceptual coupling between people can lead to the spontaneous synchronisation of their rhythmic movements. In the current study, we hypothesised that the sight of a co-actor generates anchoring (local stabilisation around specific spatiotemporal points within movement cycles), and that such anchoring supports the occurrence and stability of spontaneous interpersonal synchronisation (global stabilisation across cycles). To test these hypotheses, we re-examined previously published data from a study where participants were required to perform auditory-motor coordination of whole-body movements with versus without visual contact. Paired participants performed two kinds of coordination task – either knee flexion or extension repeatedly with auditory metronome beats (Flexion-on-the-beat and Extension-on-the-beat conditions) while standing face-to-face or back-to-back to manipulate visual interaction. The analysis of individual movement trajectories showed that visual interaction led to decreased variability along the entire trajectory, except the maximum extension position. The results also indicated that the strength of this anchoring was correlated with the degree to which the variability of interpersonal phase relations decreased with visual coupling, suggesting that local stabilisation supported global interpersonal stabilisation. Therefore, the sight of a co-actor generates anchoring effects that may play a crucial role in the stabilisation of spontaneous interpersonal synchronisation.

Perceptual coupling induces co-rotation and speeds up alternations in adjacent bi-stable structure-from-motion objects.

When two bi-stable structure-from-motion (SFM) spheres are presented simultaneously, they tend to rotate in the same direction. This effect reflects a common state bias that is present for various multistable displays. However, it was also reported that when two spheres are positioned so that they touch each other, they tend to counterrotate instead. The latter effect is interpreted as a frictional interaction, indicating the influence of the embedded physics on our visual perception. Here, we examined the interplay between these two biases in two experiments using a wide range of conditions. Those included two SFM shapes, two types of disambiguation cues, the presence or absence of the disambiguation cues, different layout options, and two samples of observers from two different universities (in sum 26 participants). Contrary to the prior report, we observed a robust common state bias for all conditions, including those that were optimized for frictional and "gear meshing" interactions. We found that stronger coupling of perceptual states is accompanied by more frequent synchronous perceptual reversals of the two objects. However, we found that the simultaneity of the individual switches does not predict the duration of the following dominance phase. Finally, we report that stronger perceptual coupling speeds up perceptual alternations.

Language experience outweighs perceptual enhancement in category learning

Some theorists claim that cues that contribute to the same acoustic percept are naturally coupled in category learning. In contrast, others claim that no pairs of cues are privileged in this way, and listeners simply learn cue correlations from language input. We compared the cue weighting and cue-shifting between pitch and breathiness, which are enhancing but do not signal a phonemic contrast in English (Experiment I), and pitch and closure duration, which are not enhancing but are correlated in signaling the English voicing contrast (Experiment II). When the cues were enhancing but not contrastive, English listeners successfully learned to weight the more informative cue higher. In contrast, preliminary results show that listeners have difficulty weighting the informative cue higher when the two cues are not enhancing but only contrastive. Results suggest that language experience outweighs auditory enhancement relationships in the perceptual coupling of cues.

Transfer of learning between unimanual and bimanual rhythmic movement coordination: transfer is a function of the task dynamic.

Under certain conditions, learning can transfer from a trained task to an untrained version of that same task. However, it is as yet unclear what those certain conditions are or why learning transfers when it does. Coordinated rhythmic movement is a valuable model system for investigating transfer because we have a model of the underlying task dynamic that includes perceptual coupling between the limbs being coordinated. The model predicts that (1) coordinated rhythmic movements, both bimanual and unimanual, are organised with respect to relative motion information for relative phase in the coupling function, (2) unimanual is less stable than bimanual coordination because the coupling is unidirectional rather than bidirectional, and (3) learning a new coordination is primarily about learning to perceive and use the relevant information which, with equal perceptual improvement due to training, yields equal transfer of learning from bimanual to unimanual coordination and vice versa [but, given prediction (2), the resulting performance is also conditioned by the intrinsic stability of each task]. In the present study, two groups were trained to produce 90° either unimanually or bimanually, respectively, and tested in respect to learning (namely improved performance in the trained 90° coordination task and improved visual discrimination of 90°) and transfer of learning (to the other, untrained 90° coordination task). Both groups improved in the task condition in which they were trained and in their ability to visually discriminate 90°, and this learning transferred to the untrained condition. When scaled by the relative intrinsic stability of each task, transfer levels were found to be equal. The results are discussed in the context of the perception-action approach to learning and performance.

The effects of warning cues and attention-capturing stimuli on the sustained attention to response task.

Performance on the sustained attention to response task (SART) is often characterized by a speed-accuracy trade-off, and SART performance may be influenced by strategic factors (Head and Helton Conscious Cogn 22: 913-919, 2013). Previous research indicates a significant difference between reliable and unreliable warning cues on response times and errors (commission and omission), suggesting that SART tasks are influenced by strategic factors (Helton et al. Conscious Cogn 20: 1732-1737, 2011; Exp Brain Res 209: 401-407, 2011). With regards to warning stimuli, we chose to use cute images (exhibiting infantile features) during a SART, as previous literature indicates cute images cause participants to engage attention. If viewing cute things makes the viewer exert more attention than normal, then exposure to cute stimuli during the SART should improve performance if SART performance is a measure of perceptual coupling. Reliable warning cues were shown to reduce both response time and errors of commission, and increase errors of omission, relative to unreliable warning cues. Cuteness of the warning stimuli, however, had no significant effect on SART performance. These results suggest the importance of strategic factors in SART performance, not increased attention, and add to the growing literature which suggests the SART is not a good measure of sustained attention, vigilance or perceptual coupling.

Learning to tie well with others: bimanual versus intermanual performance of a highly practised skill

Studies indicate that novices are faster in manual tasks when performing with a partner (‘intermanual’) than with their own two hands (‘bimanual’). The generality of this ‘mode effect’ was examined using a highly practised bimanual task, shoe tying, at which participants were experts. Speed–variability correlations confirmed participants were bimanually skilled but not intermanually skilled. Contrary to results using novices, intermanual was slower, such that prior skill reverses the effect. Analyses incorporating the similarity of shoe-tying strategies across dyads implicated a perceptual rather than shared knowledge/representation basis for intermanual performance. Practice effects indicated that intermanual performance built upon prior bimanual skill, such that novel relative timings between dyads' hands must be acquired. Motor transfer effects provided support for this conclusion. During shoe tying, hands were tightly coupled in the intermanual mode due to the perceptual coupling constraints of intermanual performance. Increased coupling was correlated with slower performance. Implications for real-world tasks (e.g. surgical knot tying) are described.

Asymmetric interpersonal coupling in a cyclic sports-related movement task

In interactive sports, teammates and/or opponents mutually tune their behavior. Expert performance thus implies certain interactive abilities, which critically depend on perceptual coupling. To illustrate this assertion, we examined the coordination dynamics with asymmetric interaction of dyads performing a sports-related cyclical movement task. In pairs, basketball players performed lateral defensive slides in in-phase, until a cue prompted them to switch to antiphase coordination. We assessed how these switches were mediated by phase adaptations of each agent under bidirectional (i.e., agents facing one another) and unidirectional (i.e., one agent facing the back of the other) visual interaction conditions. This imposed asymmetry in visual coupling exemplified an imbalance in the interaction (or ‘interact-ability’) between two agents. The results concurred the asymmetric coupling: during the switch the agent facing the other adapted his phasing more than the other agent. Furthermore, also in the bidirectional condition the coupling revealed dyad-intrinsic asymmetries (e.g., related to implicit follower-leader strategies). Together, this illustrates that interpersonal coordination is characterized by asymmetric coupling between the agents, and highlights how mutual perception of pertinent information mediates interpersonal coordination. This study offered a first step towards analyzing interpersonal coordination dynamics in relation to ‘interact-ability’.

The Influence of Ground Contact and Visible Horizon on Perception of Distance and Size under Severely Degraded Vision

For low vision navigation, misperceiving the locations of hazards can have serious consequences. Potential sources of such misperceptions are hazards that are not visually associated with the ground plane, thus, depriving the viewer of important perspective cues for egocentric distance. In Experiment 1, we assessed absolute distance and size judgments to targets on stands under degraded vision conditions. Normally sighted observers wore blur goggles that severely reduced acuity and contrast, and viewed targets placed on either detectable or undetectable stands. Participants in the detectable stand condition demonstrated accurate distance judgments, whereas participants in the undetectable stand condition overestimated target distances. Similarly, the perceived size of targets in the undetectable stand condition was judged to be significantly larger than in the detectable stand condition, suggesting a perceptual coupling of size and distance in conditions of degraded vision. In Experiment 2, we investigated size and implied distance perception of targets elevated above a visible horizon for individuals in an induced state of degraded vision. When participants' size judgments are inserted into the size-distance invariance hypothesis (SDIH) formula, distance to above-horizon objects increased compared to those below the horizon. Together, our results emphasize the importance of salient visible ground-contact information for accurate distance perception. The absence of this ground-contact information could be the source of perceptual errors leading to potential hazards for low vision individuals with severely degraded acuity and contrast sensitivity.

Feedback resolves ambiguous stimuli and mediates perceptual coupling

Feedback resolves ambiguous stimuli and mediates perceptual coupling

Occlusion-related lateral connections stabilize kinetic depth stimuli through perceptual coupling

Local sensory information is often ambiguous forcing the brain to integrate spatiotemporally separated information for stable conscious perception. Lateral connections between clusters of similarly tuned neurons in the visual cortex are a potential neural substrate for the coupling of spatially separated visual information. Ecological optics suggests that perceptual coupling of visual information is particularly beneficial in occlusion situations. Here we present a novel neural network model and a series of human psychophysical experiments that can together explain the perceptual coupling of kinetic depth stimuli with activity-driven lateral information sharing in the far depth plane. Our most striking finding is the perceptual coupling of an ambiguous kinetic depth cylinder with a coaxially presented and disparity defined cylinder backside, while a similar frontside fails to evoke coupling. Altogether, our findings are consistent with the idea that clusters of similarly tuned far depth neurons share spatially separated motion information in order to resolve local perceptual ambiguities. The classification of far depth in the facilitation mechanism results from a combination of absolute and relative depth that suggests a functional role of these lateral connections in the perception of partially occluded objects.

Perceptual coupling of multiple point-light figures

When confronted with multiple bistable stimuli at the same time, the visual system tends to generate a common interpretation for all stimuli. We exploit this perceptual-coupling phenomenon to investigate the perception of depth in bistable point-light figures. Observers indicate the global depth orientation of simultaneously presented point-light figures while the similarity between the stimuli is manipulated. In a first experiment, a higher occurrence of coupled percepts is found for identical figures, but coupling breaks down when either the movement pattern or both the viewpoint and the phase-relation are changed. A second experiment confirms these results, but also demonstrates that different point-light actions can be subject to perceptual coupling as long as they share the same viewpoint and exhibit equivalent degrees of perceptual ambiguity. The data are consistent with an explanation in terms of differential contributions of stored, view-dependent object and action representations and of an interaction between stages processing local stimulus features. The consequences of these results are discussed in the framework of an explicit model for the perception of depth in biological motion.

Developmental study of visual perception of handwriting movement: Influence of motor competencies?

This paper investigates the influence of motor competencies for the visual perception of human movements in 6-10 years old children. To this end, we compared the kinematics of actual performed and perceptual preferred handwriting movements. The two children's tasks were (1) to write the letter e on a digitizer (handwriting task) and (2) to adjust the velocity of an e displayed on a screen so that it would correspond to "their preferred velocity" (perceptive task). In both tasks, the size of the letter (from 3.4 to 54.02 cm) was different on each trial. Results showed that irrespective of age and task, total movement time conforms to the isochrony principle, i.e., the tendency to maintain constant the duration of movement across changes of amplitude. However, concerning movement speed, there is no developmental correspondence between results obtained in the motor and the perceptive tasks. In handwriting task, movement time decreased with age but no effect of age was observed in the perceptive task. Therefore, perceptual preference of handwriting movement in children could not be strictly interpreted in terms of motor-perceptual coupling.