Multidimensional Recurrence Quantification Analysis Research Articles

Exploring the role of gaze behaviour in socially shared regulation of collaborative learning in a group task

AbstractBackgroundThe necessity of supporting socially shared regulation of learning (SSRL) for learners is widely acknowledged as a crucial element for successful collaborative learning (CL). However, inherent challenges are presented for the observation, study, and support of SSRL. Non‐verbal behaviours have been identified as a potentially rich resource for understanding learner interactions within CL. Yet, the exploration of its impact on SSRL remains largely unexplored.ObjectiveThe present study aims to explore how gaze behaviour, as a non‐verbal behaviour, relates to verbal interactions that characterise SSRL in CL.MethodsTwenty‐seven Finnish secondary school students were given a 30‐min group task of creating a breakfast smoothie, in which trigger events were integrated to examine possible interactions that characterise SSRL. In this experimental setting, data from multiple modalities were collected. For analysis, process mining was used to explore patterns of interaction, and multidimensional recurrence quantification analysis (MdRQA) was used, alongside with statistical analysis.Results and ConclusionsOur study revealed that gaze behaviour is intricately linked with SSRL interactions, suggesting a complex interplay in collaborative learning contexts. When temporally investigated, SSRL interactions varied in frequency while gaze behaviour remained constant. Moreover, the recurrence of gaze and SSRL interactions did not vary due to trigger events, confirming an established collaborative pattern.ImplicationsThis study advances the field by extending research on SSRL by introducing a ‘non‐verbal perspective’. This perspective enriches our understanding of collaborative learning environments by revealing hidden interaction patterns and guiding the development of more responsive and adaptive digital learning tools.

Interpersonal heart rate synchrony predicts effective information processing in a naturalistic group decision-making task

Groups often outperform individuals in problem-solving. Nevertheless, failure to critically evaluate ideas risks suboptimal outcomes through so-called groupthink. Prior studies have shown that people who hold shared goals, perspectives, or understanding of the environment show similar patterns of brain activity, which itself can be enhanced by consensus-building discussions. Whether shared arousal alone can predict collective decision-making outcomes, however, remains unknown. To address this gap, we computed interpersonal heart rate synchrony, a peripheral index of shared arousal associated with joint attention, empathic accuracy, and group cohesion, in 44 groups (n = 204) performing a collective decision-making task. The task required critical examination of all available information to override inferior, default options and make the right choice. Using multidimensional recurrence quantification analysis (MdRQA) and machine learning, we found that heart rate synchrony predicted the probability of groups reaching the correct consensus decision with >70% cross-validation accuracy-significantly higher than that predicted by the duration of discussions, subjective assessment of team function or baseline heart rates alone. We propose that heart rate synchrony during group discussion provides a biomarker of interpersonal engagement that facilitates adaptive learning and effective information sharing during collective decision-making.

The State of the Art of Diagnostic Multiparty Eye Tracking in Synchronous Computer-Mediated Collaboration.

In recent years, innovative multiparty eye tracking setups have been introduced to synchronously capture eye movements of multiple individuals engaged in computer-mediated collaboration. Despite its great potential for studying cognitive processes within groups, the method was primarily used as an interactive tool to enable and evaluate shared gaze visualizations in remote interaction. We conducted a systematic literature review to provide a comprehensive overview of what to consider when using multiparty eye tracking as a diagnostic method in experiments and how to process the collected data to compute and analyze group-level metrics. By synthesizing our findings in an integrative conceptual framework, we identified fundamental requirements for a meaningful implementation. In addition, we derived several implications for future research, as multiparty eye tracking was mainly used to study the correlation between joint attention and task performance in dyadic interaction. We found multidimensional recurrence quantification analysis, a novel method to quantify group-level dynamics in physiological data, to be a promising procedure for addressing some of the highlighted research gaps. In particular, the computation method enables scholars to investigate more complex cognitive processes within larger groups, as it scales up to multiple data streams.

Multidimensional recurrence quantification analysis of human-metronome phasing.

Perception-action coordination (also known as sensorimotor synchronization, SMS) is often studied by analyzing motor coordination with auditory rhythms. The current study assesses phasing-a compositional technique in which two people tap the same rhythm at varying phases by adjusting tempi-to explore how SMS is impacted by individual and situational factors. After practice trials, participants engaged in the experimental phasing task with a metronome at tempi ranging from 80-140 beats per minute (bpm). Multidimensional recurrence quantification analysis (MdRQA) was used to compare nonlinear dynamics of phasing performance. Varying coupling patterns emerged and were significantly predicted by tempo and linguistic experience. Participants who successfully phased replicated findings from an original case study, demonstrating stable tapping patterns near in-phase and antiphase, while those unsuccessful at phasing showed weaker attraction to in-phase and antiphase.

A Computational Approach to Examining Team Coordination Breakdowns During Crisis Situations

During crisis situations, teams are more prone to coordination breakdowns that are characterized by a temporary, diminished ability to function effectively as a team. However, team research currently lacks robust approaches for identifying transitions from effective team functioning to coordination breakdowns. With the current study, we aimed to develop such robust approaches, and to deepen our understanding of how team coordination dynamics across various physiological signals reflect coordination breakdowns. Consequently, we used audiovisual data from four-person teams involved in a stressful collaborative game task to manually identify coordination breakdowns. Next, we set out to computationally identify coordination breakdowns by applying continuous measures of team coordination (windowed synchronization coefficient and multidimensional recurrence quantification analysis) to photoplethysmogram and electrodermal activity data obtained during the task, and identifying transitions therein with change point and nonlinear prediction algorithms. We found that our computational coordination breakdown identification approaches can identify up to 96% of the manually identified coordination breakdowns although our results also show that the precision of our approaches falls far behind. Our findings contribute theoretically and methodologically to the systematic investigation of coordination breakdowns, which may ultimately facilitate the support of teams in responding to and mitigating negative consequences of crisis situations.

Physiological Synchrony Predict Task Performance and Negative Emotional State during a Three-Member Collaborative Task.

Evaluation of team performance in naturalistic contexts has gained popularity during the last two decades. Among other human factors, physiological synchrony has been adopted to investigate team performance and emotional state when engaged in collaborative team tasks. A variety of methods have been reported to quantify physiological synchrony with a varying degree of correlation with the collaborative team task performance and emotional state, reflected in the inconclusive nature of findings. Little is known about the effect of the choice of synchrony calculation methods and the level of analysis on these findings. In this research work, we investigate the relationship between outcomes of different methods to quantify physiological synchrony, emotional state, and team performance of three-member teams performing a collaborative team task. The proposed research work employs dyadic-level linear (cross-correlation) and team-level non-linear (multidimensional recurrence quantification analysis) synchrony calculation measures to quantify task performance and the emotional state of the team. Our investigation indicates that the physiological synchrony estimated using multidimensional recurrence quantification analysis revealed a significant negative relationship between the subjectively reported frustration levels and overall task performance. However, no relationship was found between cross-correlation-based physiological synchrony and task performance. The proposed research highlights that the method of choice for physiological synchrony calculation has direct impact on the derived relationship of team task performance and emotional states.

Into the Hive-Mind: Shared Absorption and Cardiac Interrelations in Expert and Student String Quartets

Expert musicians portray awe-inspiring precision, timing, and phrasing and may be thought to partake in a “hive-mind.” Such a shared musical absorption is characterized by a heightened empathic relation, mutual trust, and a sense that the music “takes over,” thus uniting the performers’ musical intentions. Previous studies have found correlations between empathic concern or shared experience and cardiac synchrony (CS). We aimed to investigate shared musical absorption in terms of CS by analyzing CS in two quartets: a student quartet, the Borealis String Quartet (BSQ), and an expert quartet, the Danish String Quartet (DSQ), world-renowned for their interpretations and cohesion. These two quartets performed the same Haydn excerpt in seven conditions, some of which were designed to disrupt their absorption. Using multidimensional recurrence quantification analysis (MdRQA), we found that: (1) performing resulted in significantly increased CS in both quartets compared with resting; (2) across all conditions, the DSQ had a significantly higher CS than the BSQ; (3) the BSQ's CS was inversely correlated with the degree of disruption; 4) for the DSQ, the CS remained constant across all levels of disruption, besides one added extreme disruption—a sight-reading condition. These findings tentatively support the claim that a sense of shared musical absorption, as well as group expertise, is correlated with CS.

Flexibility Versus Routineness in Multimodal Health Indicators: A Sensor-based Longitudinal in Situ Study of Information Workers

Although some research highlights the benefits of behavioral routines for individual functioning, other research indicates that routines can reflect an individual's inflexibility and lower well-being. Given conflicting accounts on the benefits of routine, research is needed to examine how routineness versus flexibility in health-related behaviors correspond to personality traits, health, and occupational outcomes. We adopt a nonlinear dynamical systems approach to understanding routine using automatically sensed health-related behaviors collected from 483 information workers over a roughly two-month period. We utilized multidimensional recurrence quantification analysis to derive a measure of health regularity (routineness) from measures of daily step count, sleep duration, and heart rate variability (which relates to stress). Participants also completed measures of personality, health, and job performance at the start of the study and for two months via Ecological Momentary Assessments. Greater regularity was associated with higher neuroticism, lower agreeableness, and greater interpersonal and organizational deviance. Importantly, these results were independent of overall levels of each health indicator in addition to demographics. It is often believed that routine is desirable, but the results suggest that associations with routineness are more nuanced, and wearable sensors can provide insights into beneficial health behaviors.

MultiSyncPy: A Python package for assessing multivariate coordination dynamics

In order to support the burgeoning field of research into intra- and interpersonal synchrony, we present an open-source software package: multiSyncPy. Multivariate synchrony goes beyond the bivariate case and can be useful for quantifying how groups, teams, and families coordinate their behaviors, or estimating the degree to which multiple modalities from an individual become synchronized. Our package includes state-of-the-art multivariate methods including symbolic entropy, multidimensional recurrence quantification analysis, coherence (with an additional sum-normalized modification), the cluster-phase ‘Rho’ metric, and a statistical test based on the Kuramoto order parameter. We also include functions for two surrogation techniques to compare the observed coordination dynamics with chance levels and a windowing function to examine time-varying coordination for most of the measures. Taken together, our collation and presentation of these methods make the study of interpersonal synchronization and coordination dynamics applicable to larger, more complex and often more ecologically valid study designs. In this work, we summarize the relevant theoretical background and present illustrative practical examples, lessons learned, as well as guidance for the usage of our package – using synthetic as well as empirical data. Furthermore, we provide a discussion of our work and software and outline interesting further directions and perspectives. multiSyncPy is freely available under the LGPL license at: https://github.com/cslab-hub/multiSyncPy, and also available at the Python package index.

Changes in the Complexity of Limb Movements during the First Year of Life across Different Tasks.

Infants’ limb movements evolve from disorganized to more selectively coordinated during the first year of life as they learn to navigate and interact with an ever-changing environment more efficiently. However, how these coordination patterns change during the first year of life and across different contexts is unknown. Here, we used wearable motion trackers to study the developmental changes in the complexity of limb movements (arms and legs) at 4, 6, 9 and 12 months of age in two different tasks: rhythmic rattle-shaking and free play. We applied Multidimensional Recurrence Quantification Analysis (MdRQA) to capture the nonlinear changes in infants’ limb complexity. We show that the MdRQA parameters (entropy, recurrence rate and mean line) are task-dependent only at 9 and 12 months of age, with higher values in rattle-shaking than free play. Since rattle-shaking elicits more stable and repetitive limb movements than the free exploration of multiple objects, we interpret our data as reflecting an increase in infants’ motor control that allows for stable body positioning and easier execution of limb movements. Infants’ motor system becomes more stable and flexible with age, allowing for flexible adaptation of behaviors to task demands.

Structural classification of proteins based on the computationally efficient recurrence quantification analysis and horizontal visibility graphs.

Protein structural class prediction is one of the most significant problems in bioinformatics, as it has a prominent role in understanding the function and evolution of proteins. Designing a computationally efficient but at the same time accurate prediction method remains a pressing issue, especially for sequences that we cannot obtain a sufficient amount of homologous information from existing protein sequence databases. Several studies demonstrate the potential of utilizing chaos game representation along with time series analysis tools such as recurrence quantification analysis, complex networks, horizontal visibility graphs (HVG) and others. However, the majority of existing works involve a large amount of features and they require an exhaustive, time consuming search of the optimal parameters. To address the aforementioned problems, this work adopts the generalized multidimensional recurrence quantification analysis (GmdRQA) as an efficient tool that enables to process concurrently a multidimensional time series and reduce the number of features. In addition, two data-driven algorithms, namely average mutual information and false nearest neighbors, are utilized to define in a fast yet precise manner the optimal GmdRQA parameters. The classification accuracy is improved by the combination of GmdRQA with the HVG. Experimental evaluation on a real benchmark dataset demonstrates that our methods achieve similar performance with the state-of-the-art but with a smaller computational cost. The code to reproduce all the results is available at https://github.com/aretiz/protein_structure_classification/tree/main. Supplementary data are available at Bioinformatics online.

Looking for a Deal?

Whereas social visual attention has been examined in computer-mediated (e.g., shared screen) or video-mediated (e.g., FaceTime) interaction, it has yet to be studied in mixed-media interfaces that combine video of the conversant along with other UI elements. We analyzed eye gaze of 37 dyads (74 participants) who were tasked with negotiating the price of a new car (as a buyer and seller) using mixed-media video conferencing under competitive or cooperative negotiation instructions (experimental manipulation). We used multidimensional recurrence quantification analysis to extract spatio-temporal patterns corresponding to mutual gaze (individuals look at each other), joint attention (individuals focus on the same elements of the interface), and gaze aversion (an individual looks at their partner, who is looking elsewhere). Our results indicated that joint attention predicted the sum of points attained by the buyer and seller (i.e., the joint score). In contrast, gaze aversion was associated with faster time to complete the negotiation, but with a lower joint score. Unexpectedly, mutual gaze was highly infrequent and unrelated to the negotiation outcomes and none of the gaze patterns predicted subjective perceptions of the negotiation. There were also no effects of gender composition or negotiation condition on the gaze patterns or negotiation outcomes. Our results suggest that social visual attention may operate differently in mixed-media collaborative interfaces than in face-to-face interaction. As mixed-media collaborative interfaces gain prominence, our work can be leveraged to inform the design of gaze-sensitive user interfaces that support remote negotiations among other tasks.

Multiwavelet scale multidimensional recurrence quantification analysis.

The multiwavelet scale multidimensional recurrence quantification analysis (MWMRQA) method is proposed in this paper, which is a combination of multidimensional recurrence quantification analysis and wavelet packet decomposition. It allows us to quantify the recurrence properties of a single multidimensional time series under different wavelet scales. We apply the MWMRQA method to the Lorenz system and the Chinese stock market, respectively, and show the feasibility of this method as well as the dynamic variation of the Lorenz system and the Chinese stock market under different wavelet scales. This provides another perspective for other disciplines that need to study the recurrence properties of different scales in the future.

What does physiological synchrony reveal about metacognitive experiences and group performance?

AbstractThere is a growing body of research on physiological synchrony (PS) in Collaborative Problem Solving (CPS). However, the current literature presents inconclusive findings about the way in which PS is reflected in cognitive and affective group processes and performance. In light of this, this study investigates the relationship between PS and metacognitive experiences (ie, judgement of confidence, task interest, task difficulty, mental effort and emotional valence) that are manifested during CPS. In addition, the study explores the association between PS and group performance. The participants were 77 university students who worked together on a computer‐based CPS simulation in groups of three. Participants’ electrodermal activity (EDA) was recorded as they worked on the simulation and metacognitive experiences were measured with situated self‐reports. A Multidimensional Recurrence Quantification Analysis was used to calculate the PS among the collaborators. The results show a positive relationship between continuous PS episodes and groups’ collective mental effort. No relationship was found between PS and judgement of confidence, task interest, task difficulty or emotional valence. The relationship between PS and group performance was also non‐significant. The current work addresses several challenges in utilising multimodal data analytics in CPS research and discusses future research directions.

Beyond Dyadic Coordination: Multimodal Behavioral Irregularity in Triads Predicts Facets of Collaborative Problem Solving.

We hypothesize that effective collaboration is facilitated when individuals and environmental components form a synergy where they work together and regulate one another to produce stable patterns of behavior, or regularity, as well as adaptively reorganize to form new behaviors, or irregularity. We tested this hypothesis in a study with 32 triads who collaboratively solved a challenging visual computer programming task for 20min following an introductory warm-up phase. Multidimensional recurrence quantification analysis was used to examine fine-grained (i.e., every 10s) collective patterns of regularity across team members' speech rate, body movement, and team interaction with the shared user interface. We found that teams exhibited significant patterns of regularity as compared to shuffled baselines, but there were no systematic trends in regularity across time. We also found that periods of regularity were associated with a reduction in overall behavior. Notably, the production of irregular behavior predicted expert-coded metrics of collaborative activity, such as teams' ability to construct shared knowledge and effectively negotiate and coordinate execution of solutions, net of overall behavioral production and behavioral self-similarity. Our findings support the theory that groups can interact to form interpersonal synergies and indicate that information about system-level dynamics is a viable way to understand and predict effective collaborative processes.

Complexity in Science Learning: Measuring the Underlying Dynamics of Persistent Mistakes

Mistaken beliefs pose a barrier to science learning. For this reason, it is important to understand the circumstances in which they emerge and change. In the current paper, we apply complexity theory to shed light on the nature of mistaken beliefs. The strength of this approach lies in conceptualizing beliefs as dynamic stabilities, a well-defined construct that can be indexed precisely. For example, Recurrence Quantification Analysis (RQA) can determine the presence of dynamic stabilities by analyzing variability in time-series data. We applied this analytical tool to probe for mistaken beliefs in a beam-balancing task, a task that is known to elicit mistaken beliefs in preschoolers. Using a case-study design with four preschoolers, we tracked children’s hand position with motion sensors as they balanced various beams. The resulting time series of hand position was submitted to RQA, yielding two important results: First, we found that consistent mistakes in trying to balance the beams were not always accompanied by dynamic stability. This undermines the common assumption that overt consistency in task performance is sufficient to conclude the presence of beliefs. Second, we found strong individual differences over time, as children explored the balance beams. Applications to science education are discussed.HighlightsA classical task of beam balancing was used to explore the underlying dynamics of children’s mistaken beliefs.Moment-to-moment hand movements were tracked and subjected to a multi-dimensional recurrence quantification analysis (RQA).Dynamic stability was captured through percent laminarity (%LAM), a measure of rigidity in children’s explorations.The RQA measure of %LAM shed light on patterns of stability that were not available from the analysis of overt behavior.In line with complexity theory, a model of persistent mistakes is offered that has important implications for science education.

Moving and Reflective Functioning Under Stress.

According to mentalization theory, reflective functioning is a core feature of healthy affect regulation which involves interactions among implicit and explicit processes across multiple systems of the individual in relation with others. Mother-infant interactions point to the role of whole body movement as a feature of developing affect regulation, promoting self-organization. Using behavioral imaging technology, we examined the legacy of whole body movement in adults undergoing an interpersonal stress task (Trier Social Stress Test; TSST). Movement was assessed as a multidimensional system over time, allowing for examination of rigid recurrence and self-organized determinism in movement dynamics (Multidimensional Recurrence Quantification Analysis). Reflective functioning was assessed using an automated analysis of transcripts. We found flexible yet self-organized movement uniquely predicted reflective function. Self-reported personality organization, assessed at least one week prior, showed some bivariate relationship with indices of movement dynamics, while self-reported attachment styles did not. Using novel methodology, this study demonstrated the cooccurrence of reflective functioning and specific movement dynamics. The authors suggest theoretical approaches from phenomenology to understand these findings and call for further research.

Examining shared monitoring in collaborative learning: A case of a recurrence quantification analysis approach

The aim of this study is to investigate the relationship between shared monitoring of collaborative learning processes and physiological synchrony between the collaborating group members. Shared monitoring fuels collaborative learning in groups. Video and electrodermal activity data were collected from a group of high school students (two male, one female) during two sessions of collaborative learning. Shared monitoring of learning progress among the group members, in terms of frequency and duration, were coded and calculated in video data. Physiological synchrony in electrodermal activity among the collaborating students was calculated with Multidimensional Recurrence Quantification Analysis (MdRQA). Results revealed that the relationship between physiological synchrony and shared monitoring might be dependent on task type. That is, a significant relationship was observed between MdRQA indices and shared monitoring in one session, whereas no significant relationship was observed in the other session. The current study contributes to the literature on computer-supported collaborative learning through demonstrating the utility of MdRQA to investigate the temporal dynamicity of collaborative learning processes. In conclusion, the chosen methods contribute to research on collaborative learning, as capturing invisible physiological signals and matching them with visible instances of monitoring processes facilitates identification of critical moments in collaboration.

Multidimensional Cross-Recurrence Quantification Analysis (MdCRQA) – A Method for Quantifying Correlation between Multivariate Time-Series

In this paper, Multidimensional Cross-Recurrence Quantification Analysis (MdCRQA) is introduced. It is an extension of Multidimensional Recurrence Quantification Analysis (MdRQA), which allows to quantify the (auto-)recurrence properties of a single multidimensional time-series. MdCRQA extends MdRQA to bi-variate cases to allow for the quantification of the co-evolution of two multidimensional time-series. Moreover, it is shown how a Diagonal Cross-Recurrence Profile (DCRP) can be computed from the MdCRQA output that allows to capture time-lagged coupling between two multidimensional time-series. The core concepts of these analyses are described, as well as practical aspects of their application. In the supplementary materials to this paper, implementations of MdCRQA and the DCRP as MatLab- and R-functions are provided.

Analyzing Multivariate Dynamics Using Cross-Recurrence Quantification Analysis (CRQA), Diagonal-Cross-Recurrence Profiles (DCRP), and Multidimensional Recurrence Quantification Analysis (MdRQA) - A Tutorial in R.

This paper provides a practical, hands-on introduction to cross-recurrence quantification analysis (CRQA), diagonal cross-recurrence profiles (DCRP), and multidimensional recurrence quantification analysis (MdRQA) in R. These methods have enjoyed increasing popularity in the cognitive and social sciences since a recognition that many behavioral and neurophysiological processes are intrinsically time dependent and reliant on environmental and social context has emerged. Recurrence-based methods are particularly suited for time-series that are non-stationary or have complicated dynamics, such as longer recordings of continuous physiological or movement data, but are also useful in the case of time-series of symbolic data, as in the case of text/verbal transcriptions or categorically coded behaviors. In the past, they have been used to assess changes in the dynamics of, or coupling between physiological and behavioral measures, for example in joint action research to determine the co-evolution of the behavior between individuals in dyads or groups, or for assessing the strength of coupling/correlation between two or more time-series. In this paper, we provide readers with a conceptual introduction, followed by a step-by-step explanation on how the analyses are performed in R with a summary of the current best practices of their application.