Performance In Collaborative Problem Solving Research Articles

Individuals in a group: exploring engagement patterns via within-group configurations of role profiles and their impact on performance in collaborative problem solving

ABSTRACT Collaborative problem-solving (CPS) has been widely used in K-12, higher education, and informal learning to enhance the quality of student learning. Understanding the relationship between learning engagement and group performance is crucial for CPS pedagogy and analytics. However, few empirical studies investigated individual engagement role profiles in CPS and the impact of individual engagement, as reflected in within-group configurations of role profiles, on learning performance. In this study, deep learning was employed to evaluate learner engagement, cluster analysis was performed to identify engagement role profiles, and the qualitative cases were used to in-depth compare within-group configurations of role profiles in higher- and lower-achieving groups. The results showed that six distinct role profiles with varying levels of cognitive and socio-emotional engagement have been identified: free-rider, normal-constructer, immersed-engager, balanced-engager, questioning-refuter, and active-constructer. In addition, high-achieving groups were generally characterized by dominant role profiles of active-constructers and balanced-engagers, while lower-achieving groups were characterized by dominant role profiles of free-riders. Based on these findings, some implications for promoting high-quality collaborative learning through optimal design and structure of CPS activities are discussed.

A multimodal analysis of college students' collaborative problem solving in virtual experimentation activities: a perspective of cognitive load.

Distance education programs have become the preferred option for most higher education institutions to continue teaching during the COVID-19 pandemic, but the effectiveness of some online courses, especially those engineering courses with experimentation activities, remains disputed. The main challenge is fostering collaborative problem solving skills for novice students as online collaboration increases their cognitive load. This research thus tapped into novice engineering students’ cognitive load to develop a more granular, multimodal view of how cognitive load influences student performance in collaborative problem solving during virtual experimentation activities. The findings of this research provided significant implications for the future design and implementation of virtual laboratories in computer science engineering education.

The relationship between collaborative problem solving behaviors and solution outcomes in a game-based learning environment

Collaborative problem solving (CPS) is an essential skill for the 21st century workforce but remains difficult to assess. Understanding how CPS skills affect CPS performance outcomes can inform CPS training, task design, feedback design, and automated assessment. We investigated CPS behaviors (individually and in co-occurring patterns) in 101 (N = 303) remote triads who collaboratively played an educational game called Physics Playground for 45-min. Team interactions consisted of open-ended speech occurring over videoconferencing with screen sharing. We coded participant's utterances relative to a CPS framework consisting of three facets (i.e., competencies such as constructing shared knowledge) manifested in 19 specific indicators (e.g., responds to others' questions/ideas). A matching technique was used to isolate the effect of CPS behaviors on CPS outcomes (quality of solution of a game level) controlling for pertinent covariates. Mixed-effects ordinal regression models indicated that proposing solution ideas and discussing results were the major predictors of CPS performance, and that team-member activities surrounding idea generation mattered. These findings highlighted the importance of both individual and collective contributions and social and cognitive skills in successful CPS outcomes.

The pivotal role of monitoring for collaborative problem solving seen in interaction, performance, and interpersonal physiology

Being aware of the progress towards one’s goals is considered one of the main characteristics of the self-regulation process. This is also the case for collaborative problem solving, which invites group members to metacognitively monitor the progress with their goals and externalize it in social interactions while solving a problem. Monitoring challenges can activate group members to control the situation together, which can be seen as adjustments on different systemic levels (physiological, psychological, and interpersonal) of a collaborative group. This study examines how the pivotal role of monitoring for collaborative problem solving is reflected in interactions, performance, and interpersonal physiology. The study has foci in two central characteristics of monitoring interactions that facilitate groups’ regulation in reaching their goals. First is valence of monitoring, indicating whether the group members think they are progressing towards their goal or not. Second is equality of participation in monitoring interactions between group members. Participants of the study were volunteering higher education students (N = 57), randomly assigned to groups of three members whose collaborative task was to learn to run a business simulation. The collaborative task was video recorded, and the physiological arousal of each participant was recorded from their electrodermal activity. The results of the study suggest that both the valence and equality of participation are identifiable in monitoring interactions and they both positively predict groups’ performance in the task. Equality of participation to monitoring was not related to the interpersonal physiology. However, valence of monitoring was related to interpersonal physiology in terms of physiological synchrony and arousal. The findings support the view that characteristics of monitoring interactions make a difference to task performance in collaborative problem solving and that interpersonal physiology relates to these characteristics.

Multilevel Effects of Student and School Factors on Collaborative Problem-Solving Performance in Taiwan

Multilevel Effects of Student and School Factors on Collaborative Problem-Solving Performance in Taiwan

Comparative analysis of student performance in collaborative problem solving: What does it tell us?

Collaboration is a complex skill, comprised of multiple subskills, that is of growing interest to policy makers, educators and researchers. Several definitions and frameworks have been described in the literature to support assessment of collaboration; however, the inherent structure of the construct still needs better definition. In 2015, the Organisation for Economic Cooperation and Development, in their Programme for International Student Assessment assessed 15-year-old students’ collaborative problem solving achievement, with the use of computer-simulated agents, aiming to address the lack of internationally comparable data in this field. This paper explores what the data from this assessment tell us about the skill, and how these data compare with data from two other assessments of collaboration. Analyses enable comment on the extent to which the three assessments are measuring the same construct, and the extent to which the construct can be covered using computer-based assessments. These investigations generate better understanding of this complex and innovative domain.

Interplay of metacognitive experiences and performance in collaborative problem solving

Metacognitive experiences are the feelings and judgments that emerge in relation to an ongoing learning task. Much of the work on metacognitive experiences has studied these constructs piecemeal and mostly in individual learning settings. Thus, little is known about how metacognitive experiences co-occur in social learning settings. In light of this, we investigated the relationships between metacognitive experiences and their impact on perceived and objective task performance in a collaborative problem solving (CPS) simulation. Seventy-seven higher education students participated in the study. Metacognitive experiences (judgment of confidence, mental effort, task difficulty, task interest, and emotional valence) were measured with self-reports at multiple time points during CPS. A path analysis was conducted to investigate the relationship between metacognitive experiences and perceived performance. A generalized estimating equation was used to observe the relationships between metacognitive experiences and objective group CPS performance. Overall, the findings indicate complex relationships among the metacognitive experiences and performance in CPS and further highlight the social characteristics of metacognition.

Technology-Enhanced Learning: An Optimal CPS Learning Application

Collaborative problem-solving (CPS) is highly valued in the sustainability of learning to foster the key soft power of talent for the future. In this study, a CPS learning application was built to train and assess individuals with the aim of increasing CPS skills. For effective learning to take place, several issues need to be carefully considered, and these were investigated while testing the proposed application. This study examined the impact of collaborative interactions (CIs) (human–computer agent (HCA) and human–human (HH) interactions) on the CPS performance of students. Gender and learning styles, which may have interaction effects with CIs on CPS performance, were also explored. The results show that the students’ CPS performance in HCA was significantly greater than that in HH. The interaction effect between gender and CI was not significant. The impact of learning style on CPS performance in HH was not significant. In contrast, in HCA, students with verbal, global, and reflective learning styles performed significantly better on CPS tasks than did students with visual, sequential, and active learning styles. Finally, we discussed the optimal ways to teach CPS and the practical effects of a CPS learning application.

Computer-Based Collaborative Problem Solving in PISA 2015 and the Role of Personality

Collaborative problem solving (CPS) is an essential 21st century skill at the intersection of social collaboration and cognitive problem solving, and is increasingly integrated in educational programs, such as the influential Programme for International Student Assessment (PISA). As research has identified the impact of the Big Five personality traits either on cognitive ability or social collaboration skills in groups, this study firstly identified their impact on the conjoint construct of CPS. Results from structural equation modelling (N = 483) found openness to experience and agreeableness as predictors for CPS performance. The results are embedded in the lifelong learning and investment model by Ackermann and provide implications for PISA 2015, as original PISA 2015 CPS tasks were used.

The role of achievement emotions in the collaborative problem-solving performance of adolescents

We explored the relationship between adolescents' activity-based achievement emotions and their performance during collaborative problem solving (CPS) tasks, which was operationalized as having objective social and cognitive performance dimensions. Participants were 100 adolescent dyads (n = 200) who completed a series of five computer-based CPS tasks while their activity emotions of enjoyment, boredom, and anger were recorded. It was hypothesized, using a partially mediated structural regression model, that individual differences in students' activity emotions would be linked to effort regulation, which in turn, would be associated with both CPS social and cognitive performance. On the basis that more effective collaboration efforts enable better cognitive performance, we also expected CPS social performance to influence CPS cognitive performance. Our hypothesized model fit the data well. All emotions were associated with effort regulation, and effort regulation influenced cognitive performance but showed a weak association with social performance. Instead, anger and enjoyment affected CPS social performance directly. Our findings provide valuable insight into the role of affective experiences in the growing area of measuring 21st century skills in educational settings.

Advancing the Science of Collaborative Problem Solving

Collaborative problem solving (CPS) has been receiving increasing international attention because much of the complex work in the modern world is performed by teams. However, systematic education and training on CPS is lacking for those entering and participating in the workforce. In 2015, the Programme for International Student Assessment (PISA), a global test of educational progress, documented the low levels of proficiency in CPS. This result not only underscores a significant societal need but also presents an important opportunity for psychological scientists to develop, adopt, and implement theory and empirical research on CPS and to work with educators and policy experts to improve training in CPS. This article offers some directions for psychological science to participate in the growing attention to CPS throughout the world. First, it identifies the existing theoretical frameworks and empirical research that focus on CPS. Second, it provides examples of how recent technologies can automate analyses of CPS processes and assessments so that substantially larger data sets can be analyzed and so students can receive immediate feedback on their CPS performance. Third, it identifies some challenges, debates, and uncertainties in creating an infrastructure for research, education, and training in CPS. CPS education and assessment are expected to improve when supported by larger data sets and theoretical frameworks that are informed by psychological science. This will require interdisciplinary efforts that include expertise in psychological science, education, assessment, intelligent digital technologies, and policy.

Multiscale movement coordination dynamics in collaborative team problem solving

During collaborative problem solving (CPS), coordination occurs at different spatial and temporal scales. This multiscale coordination should play a functional role in facilitating effective collaboration. To evaluate this, we conducted a study of computer-based CPS with 42 dyadic teams. We used cross-wavelet coherence to examine movement coordination, extracted from videos, at several scales, and tested whether the observed coordination was greater than expected due to chance and due to task demands. We found that coordination at scales less than 2s was greater than chance and at most scales (except 16s, 1m, and 2m), was greater than expected due to task demands. Lastly, we observed that coherence at .25s and 1s scales was predictive of performance. However, when including relative phase, our results suggest that higher in-phase movement coordination at the 1s scale was the strongest predictor of CPS performance. Further, we used growth curve modeling to examine how movement coordination changes across the duration of the task and whether this is moderated by CPS performance. We found that coordination over the duration of the CPS task is quadratic (a U shape) and that better performing teams have higher coordination with a shallower curve. We discuss these findings and their relevance to understanding how low-level movement coordination facilitates CPS.

Pair interactions in online assessments of collaborative problem solving: case-based portraits

This exploratory case study focuses on how pairs of students can build a shared understanding and acquire collaborative problem-solving (CPS) practices during an online assessment of CPS skills, which is seen in the context of the CPS construct, in a symmetrical and asymmetrical task type. Even though CPS is widely recognised as a core twenty-first-century competency, its nature is not yet well understood. Also, until recently, most of studies have focused on the individual’s solution to a problem or on the skills individuals bring into a problem-solving space. This study extends from an individual- to group-level focus in CPS, emphasising the role and quality of the social aspects in CPS processes and outcomes. Focusing on the group level because it mediates multiple levels of learning, including individual cognition and socio-cultural practices, may provide us with a better understanding of how pairs establish CPS practices. Because of the complexity of CPS and the general challenges of remote collaboration in an online context, the study relies on the triangulation of multiple data sources and phases of analysis. In this paper, the aim is to explore and visualise through contrasting case-based portraits of two pairs how micro-interaction processes evolve at the pair level. The results show that despite students’ similar CPS performance outcome scores and task designs aimed to facilitate collaboration, variations in micro-interactions occur across pairs, for example as individual and joint solution endeavours and as balanced and unbalanced dynamics of group interactions. Studying these patterns at the pair level may provide new insights into CPS and support strategies for acquiring these practices.

The assessment of collaborative problem solving in PISA 2015: Can computer agents replace humans?

Despite the relevance of collaborative problem solving (CPS), there are limited empirical results on the assessment of CPS. In 2015, the large-scale Programme for International Student Assessment (PISA) first assessed CPS with virtual tasks requiring participants to collaborate with computer-simulated agents (human-to-agent; H-A). The approach created dynamic CPS situations while standardizing assessment conditions across participating countries. However, H-A approaches are sometimes regarded as poor substitutes for natural collaboration, and only a few studies have identified if the collaborations with agents capture real dynamics of human interactions. To address this, we validated the original PISA 2015 CPS assessment by investigating the effects of replacing computer agents with real students in classroom tests (human-to-human; H-H). We obtained the original PISA 2015 CPS tasks from the OECD and replaced agents with real students to provide more real-life collaboration environments with less control over conversations; the H-H was less constrained than the H-A but still limited by predefined sets of possible answers from which the humans’ would make selections. The interface remained nearly identical to the original PISA 2015 CPS assessment. Students were told the types of collaboration partners, namely humans versus agents. We applied structural equation modeling and multivariate analyses of variance to a sample of 386 students to identify the dimensionality of the CPS construct and compared the effects in CPS performance accuracy and number of behavioral actions. Results indicated no significant differences between type of collaboration partner. However, students performed a larger number of actions when collaborating with a human agent.

Problem-Solving Phase Transitions During Team Collaboration.

Multiple theories of problem-solving hypothesize that there are distinct qualitative phases exhibited during effective problem-solving. However, limited research has attempted to identify when transitions between phases occur. We integrate theory on collaborative problem-solving (CPS) with dynamical systems theory suggesting that when a system is undergoing a phase transition it should exhibit a peak in entropy and that entropy levels should also relate to team performance. Communications from 40 teams that collaborated on a complex problem were coded for occurrence of problem-solving processes. We applied a sliding window entropy technique to each team's communications and specified criteria for (a) identifying data points that qualify as peaks and (b) determining which peaks were robust. We used multilevel modeling, and provide a qualitative example, to evaluate whether phases exhibit distinct distributions of communication processes. We also tested whether there was a relationship between entropy values at transition points and CPS performance. We found that a proportion of entropy peaks was robust and that the relative occurrence of communication codes varied significantly across phases. Peaks in entropy thus corresponded to qualitative shifts in teams' CPS communications, providing empirical evidence that teams exhibit phase transitions during CPS. Also, lower average levels of entropy at the phase transition points predicted better CPS performance. We specify future directions to improve understanding of phase transitions during CPS, and collaborative cognition, more broadly.

Preventing undesirable effects of mutual trust and the development of skepticism in virtual groups by applying the knowledge and information awareness approach

Empirical studies have proven the effectiveness of the knowledge and information awareness approach of Engelmann and colleagues for improving collaboration and collaborative problem-solving performance of spatially distributed group members. This approach informs group members about both their collaborators’ knowledge structures and their collaborators’ information. In the current study, we investigated whether this implicit approach reduces undesirable effects of mutual trust and mutual skepticism. Trust is an important influencing factor with regard to behavior and performance of groups. High mutual trust can have a negative impact on group effectiveness because it reduces mutual control and, as a result, the detection of the others’ mistakes. In an empirical study, 20 triads collaborating with the knowledge and information awareness approach were compared with 20 triads collaborating without this approach. The members of a triad were spatially distributed and participated in a computer-supported collaboration. The results demonstrated that the availability of the knowledge and information awareness approach overrides the negative impact of too much mutual trust and counteracts the development of mutual skepticism. This study contributes to further clarifying the impact of trust on effectiveness and efficiency of virtual groups depending upon different situational contexts.