Complex Problem-solving Tasks Research Articles

Polytomous Effectiveness Indicators in Complex Problem-Solving Tasks and Their Applications in Developing Measurement Model.

Recent years have witnessed the emergence of measurement models for analyzing action sequences in computer-based problem-solving interactive tasks. The cutting-edge psychometrics process models require pre-specification of the effectiveness of state transitions often simplifying them into dichotomous indicators. However, the dichotomous effectiveness becomes impractical when dealing with complex tasks that involve multiple optimal paths and numerous state transitions. Building on the concept of problem-solving, we introduce polytomous indicators to assess the effectiveness of problem states and state-to-state transitions . The three-step evaluation method for these two types of indicators is proposed and illustrated across two real problem-solving tasks. We further present a novel psychometrics process model, the sequential response model with polytomous effectiveness indicators (SRM-PEI), which is tailored to encompass a broader range of problem-solving tasks. Monte Carlo simulations indicated that SRM-PEI performed well in the estimation of latent ability and transition tendency parameters across different conditions. Empirical studies conducted on two real tasks supported the better fit of SRM-PEI over previous models such as SRM and SRMM, providing rational and interpretable estimates of latent abilities and transition tendencies through effectiveness indicators. The paper concludes by outlining potential avenues for the further application and enhancement of polytomous effectiveness indicators and SRM-PEI.

Learning enhancement through deploying personal learning environment, network and problem-based learning assessment

Intended to foster students’ self-directed learning and research ability at the postgraduate level, this educational research evaluated the implications of personal learning environment & network (PLE&N) and problem-based learning (PBL) assessment as student-centric and student-driven pedagogic approaches. Compared to conventional instructional paradigms, the PLE&N platform presents an agile and interactive platform that empowers students to partake in collaborative discourse while orchestrating their learning journey through the judicious utilization of communal resources. The vitality of the PLE&N model is substantiated by the enduring engagement of peers, teachers, professionals, and external experts in the field, fostering a milieu of enriched knowledge exchange, synergistic collaboration, and personalized curation of learning approaches. In contrast, the PBL assessment was designed by incorporating intricate scenarios similar to the real world, thereby affording an avenue for cultivating students’ cogitation and adroit problem-solving skills within a guided learning environment. By integrating constructivist and connectivist pedagogies, the PBL assessment facilitated teamwork while engaging students in independent research and empowered their knowledge construction on selected topics. Students’ self-evaluation through the survey indicated their enhancement in self-directed learning behavior and competence building for complex problem-solving tasks. Moreover, peer feedback and suggestions received through both PLE&N and PBL were highly beneficial for students to analyze information and learning resources from diverse perspectives, which further helped to expand their learning horizon. This study underscored the significance and advantages of modern student-centric approaches such as PLE&N and PBL. These pedagogical paradigms stand as seminal strategies for nurturing nascent post-graduate students, thereby endowing them with indispensable proficiencies in self-directed learning and robust competencies in scholarly inquiry. Keywords: PLE&N, PBL, Constructivism, Connectivism, Pedagogy, Self-directed learning.

Parenting Influences on Frontal Lobe Gray Matter and Preterm Toddlers' Problem-Solving Skills.

Children born preterm often face challenges with self-regulation during toddlerhood. This study examined the relationship between prematurity, supportive parent behaviors, frontal lobe gray matter volume (GMV), and emotion regulation (ER) among toddlers during a parent-assisted, increasingly complex problem-solving task, validated for this age range. Data were collected from preterm toddlers (n = 57) ages 15-30 months corrected for prematurity and their primary caregivers. MRI data were collected during toddlers' natural sleep. The sample contained three gestational groups: 22-27 weeks (extremely preterm; EPT), 28-33 weeks (very preterm; VPT), and 34-36 weeks (late preterm; LPT). Older toddlers became more compliant as the Tool Task increased in difficulty, but this pattern varied by gestational group. Engagement was highest for LPT toddlers, for older toddlers, and for the easiest task condition. Parents did not differentiate their support depending on task difficulty or their child's age or gestational group. Older children had greater frontal lobe GMV, and for EPT toddlers only, more parent support was related to larger right frontal lobe GMV. We found that parent support had the greatest impact on high birth risk (≤27 gestational weeks) toddler brain development, thus early parent interventions may normalize preterm child neurodevelopment and have lasting impacts.

Unsuccessful and successful complex problem solvers – A log file analysis of complex problem solving strategies across multiple tasks

Complex problem solving (CPS) is a key competence in educational contexts with strong conceptual links to students' overall intelligence. However, the mechanisms underlying successful CPS are not fully understood. Therefore, this study investigated several factors presumed to be relevant to CPS success using log file data to code each individual student action during six CPS tasks with different characteristics (N = 1276). We coded individual strategy combinations per student for each item for different strategy combinations of vary-one-thing-at-a-time (VOTAT), hold-one-thing-at-a-time (HOTAT), vary-no-thing-at-a-time (NOTAT), and change-all (CA). Results from generalized linear mixed models showed that CPS success was likely to be achieved by using VOTAT. However, there was an increased chance of solving an item when additional strategies, such as NOTAT or NOTAT plus HOTAT were used. This result was moderated by the presence/absence of eigendynamics as an important determinant of item difficulty. Strategy combinations of VOTAT together with other strategies (all including NOTAT) showed higher chances of CPS success when eigendynamics were present. Additionally, higher chances of solving an item when using VOTAT with additional strategies was demonstrated for items without eigendynamics. Overall, our results suggest that flexibility in strategy application is the driving force for successful CPS performance. Implications are discussed in light of the presumed benefit of pedagogically relevant metacognitive skills, such as planning, monitoring and reflecting, for CPS success. Based on our findings, we provide specific recommendations for the development of computer-based learning simulations to train CPS and related competencies, ultimately enhancing students' skills in educational contexts.

Promoting Complex Problem Solving by Introducing Schema-Governed Categories of Key Causal Models.

The ability to recognize key causal models across situations is associated with expertise. The acquisition of schema-governed category knowledge of key causal models may underlie this ability. In an experimental study (n = 183), we investigated the effects of promoting the construction of schema-governed categories and how an enhanced ability to recognize the key causal models relates to performance in complex problem-solving tasks that are based on the key causal models. In a 2 × 2 design, we tested the effects of an adapted version of an intervention designed to build abstract mental representations of the key causal models and a tutorial designed to convey conceptual understanding of the key causal models and procedural knowledge. Participants who were enabled to recognize the underlying key causal models across situations as a result of the intervention and the tutorial (i.e., causal sorters) outperformed non-causal sorters in the subsequent complex problem-solving task. Causal sorters outperformed the control group, except for the subtask knowledge application in the experimental group that did not receive the tutorial and, hence, did not have the opportunity to elaborate their conceptual understanding of the key causal models. The findings highlight that being able to categorize novel situations according to their underlying key causal model alone is insufficient for enhancing the transfer of the according concept. Instead, for successful application, conceptual and procedural knowledge also seem to be necessary. By using a complex problem-solving task as the dependent variable for transfer, we extended the scope of the results to dynamic tasks that reflect some of the typical challenges of the 21st century.

Cycles of goal silencing and reactivation underlie complex problem-solving in primate frontal and parietal cortex

While classic views proposed that working memory (WM) is mediated by sustained firing, recent evidence suggests a contribution of activity-silent states. Within WM, human neuroimaging studies suggest a switch between attentional foreground and background, with only the foregrounded item represented in active neural firing. To address this process at the cellular level, we recorded prefrontal (PFC) and posterior parietal (PPC) neurons in a complex problem-solving task, with monkeys searching for one or two target locations in a first cycle of trials, and retaining them for memory-guided revisits on subsequent cycles. When target locations were discovered, neither frontal nor parietal neurons showed sustained goal-location codes continuing into subsequent trials and cycles. Instead there were sequences of timely goal silencing and reactivation, and following reactivation, sustained states until behavioral response. With two target locations, goal representations in both regions showed evidence of transitions between foreground and background, but the PFC representation was more complete, extending beyond the current trial to include both past and future selections. In the absence of unbroken sustained codes, different neuronal states interact to support maintenance and retrieval of WM representations across successive trials.

The role of learning in complex problem solving using MicroDYN

It is still an open question which cognitive and non-cognitive personality traits are useful for describing and explaining behaviour and performance in complex problems. During complex problem solving (CPS), problem solvers have to interact with the task in a way in which learning ability might be beneficial for successful task completion. By investigating the relationship between learning ability and CPS, while accounting for interactions between complex system characteristics and person characteristics, this paper aims to understand the role of learning processes in CPS more closely. In a sample of N = 241 participants, we performed a preregistered analysis to investigate the relationship between knowledge acquisition performance in a CPS test (MicroDYN) and learning test performance (ADAFI) with a multilevel modeling approach across 10 CPS systems with various characteristics. In line with our expectations, we replicated previous findings on a relationship between learning test and MicroDYN performance and found this relationship to be more pronounced in systems with (vs. without) autonomous changes. Further system and person characteristics also showed effects as expected, with better performance in systems with lower complexity, with more experience with the task, and with more strategic exploration behaviour. Our results provide further evidence for the notion that learning is an important component for the successful completion of CPS tasks.

Microfoundations of Adaptive Search in Complex Tasks: The Role of Cognitive Abilities and Styles

Problem-solving in complex environments requires a cognitively demanding search for task solutions. Managing this search process presents a major challenge in organizations. We contribute to the literature on this topic by providing new evidence on the cognitive antecedents that shape how individuals search when engaged in complex problem-solving tasks. We present results from three laboratory studies, wherein 335 individuals solved a complex task. In doing so, they generated behavioral data coupled with survey-based measurements of the individuals’ cognitive styles and performance-based tests of their cognitive abilities. Our data analysis contributes to the current literature by documenting systematic heterogeneity in the persistence and distance of search that can be explained by the participants’ level of creativity, attention to detail, and executive functions. We extend the research on the microfoundations of adaptive search by linking cognitive antecedents with a complex search task, widening our insight into what search behavior certain cognitive microfoundations lead to, and showing how managers can more effectively shape organizational search. History: This paper has been accepted for the Organization Science Special Issue on Experiments in Organizational Theory. Supplemental Material: The online appendix is available at https://doi.org/10.1287/orsc.2023.1654 .

G's little helpers – VOTAT and NOTAT mediate the relation between intelligence and complex problem solving

Intelligence and complex problem solving (CPS) correlate closely, but little is known about the mechanism that translates intelligence into successful CPS. Therefore, this study considered the strategic exploration behaviors VOTAT (vary-one-thing-at-a-time) and NOTAT (vary no-thing-at-a-time) as possible mediators. A sample of 495 high-school students worked on nine CPS tasks, six of which with solely direct effects and three with direct and eigendynamic effects. We expected substantial mediation effects if the applied strategic behaviors were optimal to identify the particular underlying effect types (i.e., direct effects: VOTAT; direct and eigendynamic effects: VOTAT and NOTAT). The model for tasks with only direct effects revealed VOTAT and NOTAT to be substantial mediators: Whereas VOTAT showed substantial positive relations to intelligence and CPS performance, NOTAT unexpectedly showed substantial negative relations. Both VOTAT and NOTAT resulted in significant indirect mediation effects. The model for tasks with direct and eigendynamic effects showed substantial positive relations of VOTAT and NOTAT to intelligence and CPS-performance and resulted in significant and positive indirect mediation effects. Moreover, the indirect effects differed between VOTAT and NOTAT and across the two facets of CPS performance. Overall, strategic exploration behaviors are relevant for explaining the g-CPS-relation.

Design and Formative Evaluation of a Virtual Voice-Based Coach for Problem-solving Treatment: Observational Study.

BackgroundArtificial intelligence has provided new opportunities for human interactions with technology for the practice of medicine. Among the recent artificial intelligence innovations, personal voice assistants have been broadly adopted. This highlights their potential for health care–related applications such as behavioral counseling to promote healthy lifestyle habits and emotional well-being. However, the use of voice-based applications for behavioral therapy has not been previously evaluated.ObjectiveThis study aimed to conduct a formative user evaluation of Lumen, a virtual voice-based coach developed as an Alexa skill that delivers evidence-based, problem-solving treatment for patients with mild to moderate depression and/or anxiety.MethodsA total of 26 participants completed 2 therapy sessions—an introductory (session 1) and a problem-solving (session 2)—with Lumen. Following each session with Lumen, participants completed user experience, task-related workload, and work alliance surveys. They also participated in semistructured interviews addressing the benefits, challenges and barriers to Lumen use, and design recommendations. We evaluated the differences in user experience, task load, and work alliance between sessions using 2-tailed paired t tests. Interview transcripts were coded using an inductive thematic analysis to characterize the participants’ perspectives regarding Lumen use.ResultsParticipants found Lumen to provide high pragmatic usability and favorable user experience, with marginal task load during interactions for both Lumen sessions. However, participants experienced a higher temporal workload during the problem-solving session, suggesting a feeling of being rushed during their communicative interactions. On the basis of the qualitative analysis, the following themes were identified: Lumen’s on-demand accessibility and the delivery of a complex problem-solving treatment task with a simplistic structure for achieving therapy goals; themes related to Lumen improvements included streamlining and improved personalization of conversations, slower pacing of conversations, and providing additional context during therapy sessions.ConclusionsOn the basis of an in-depth formative evaluation, we found that Lumen supported the ability to conduct cognitively plausible interactions for the delivery of behavioral therapy. Several design suggestions identified from the study including reducing temporal and cognitive load during conversational interactions, developing more natural conversations, and expanding privacy and security features were incorporated in the revised version of Lumen. Although further research is needed, the promising findings from this study highlight the potential for using Lumen to deliver personalized and accessible mental health care, filling a gap in traditional mental health services.

Focusing on eigendynamic effects promotes students’ performance in complex problem solving: A log-file analysis of strategic behavior

Human beings are frequently confronted with complex systems that change dynamically over time. Correspondingly, so called eigendynamic effects are an important feature in complex problem solving (CPS) tasks. Eigendynamic effects describe increases or decreases in outcome variables over time without the individual's intervention. Identifying the eigendynamic effects from the start while exploring the system is reasonably an effective strategic behavior when systems are comprised of different effect types (i.e., eigendynamic effects alongside other effects). In this study, we hypothesized that the strategic behavior to identify eigendynamic effects early when exploring the system was related to CPS performance and mediated the relationship between intelligence and CPS performance. In a sample of N = 262 German high school students, the investigated strategic behavior predicted CPS performance when exploring the system (knowledge acquisition phase) and when controlling the system (knowledge application phase) in single CPS tasks and in a set of CPS tasks comprising of different effect types. In mediation models the strategic behavior to identify eigendynamic effects early mediated the intelligence-CPS performance relation in the knowledge acquisition phase, but not in the knowledge application phase. We discuss these findings in the light of how students interact with complex and dynamic systems, pointing to the importance of this strategic behavior for the relationship between intelligence and CPS.

Problem-based learning as an assessment: Enhancing students’ connective learning and constructive learning

For the sake of enhancing students’ broad knowledge, problem-solving skills, and professional competences, this educational research project implemented problem-based learning (PBL) as a pedagogic tool in a large-class setting with 154 students involved at The Hong Kong Polytechnic University. The students participated in a problem-solving task that accounted for 25% of their course assessment (summative), which was designed with complex scenarios similar to real-life circumstances. The PBL assessment was structured in a way that required students' efforts to explore and connect relevant ideas supplemented with verifiable study materials, and a guided learning environment was provided by the course instructor and tutors. Learning activities were tailored to foster students’ active engagement and self-directed learning attitude towards resolving the PBL assessment. To evaluate the performance of the PBL approach, pre-and post-test questionnaire surveys were conducted to understand the improvement of students’ confidence regarding their learning ability, knowledge building, critical thinking, skill transfer, etc. The findings of these surveys revealed that the PBL assessment and related learning activities successfully stimulated students’ self-directed learning behaviour, enhanced their collaboration among peers, and expanded their capacity to develop critical skills for complex problem-solving tasks. Despite a challenging online learning environment compared to traditional classroom settings owing to the pandemic situation, the students considered this PBL assessment with guidance by the course instructor and tutors as creative, exciting, and very effective for life-long skill development. This research highlighted great prospects regarding implementing PBL approaches, even as an assessment component, in higher education institutions, which could be highly beneficial for nurturing competent graduates with essential practice-oriented skills and self-directed learning behaviour. Keywords: Higher Education, Scholarship of Teaching and Learning (SoTL), Learning Theories, Teaching Pedagogy, Constructivism, Connectivism.

Neuropsychological profile of Turner Syndrome in relation to deficits in academic and psychosocial areas. A case report

Introduction Previous reviews reported an association between Turner Syndrome (TS) and a profile of deficits in some neurocognitive domains (visual-spatial domains, mathematics, and executive functions: cognitive flexibility, working memory, cognitive inhibition, and problem solving), although pointing out individual variability.ObjectivesTo describe the neuropsychological profile of a patient with diagnosis of TS and psychosocial difficulties attended at the Service of Psychiatry, Clinical Psychology and Mental Health at La Paz University Hospital (Madrid).MethodsA descriptive study is conducted on a single case of a 11-year-old woman with diagnosis of TS attended by a clinical psychologist at a child-adolescent Mental health center for social, family and academic difficulties. Neuropsychological assessment was completed in October, 2021. The Wechsler Intelligence Scale for Children-Five Edition (WISC-V) and Neuropsychological Assessment of Executive Functions in Children (ENFEN) batteries were administered.ResultsThe full-scale intelligence quotient was observed in the normal range, with lower scores in non-verbal tasks. Deficits (range from z = -2.00 to -1.75) were observed in tests of working memory, processing speed and complex problem-solving tasks. The results showed great variability in other executive functioning tasks (selective attention tasks: from z = -1.75 to -0.75; and cognitive flexibility tasks: from z = -2.25 to 0.25).ConclusionsThe neurocognitive profile described in the literature was partially consistent with the results obtained in this study. The neuropsychological assessment can support the elucidation of clinical diagnostic and therapeutic factors in TS patients with relevant psychosocial or cognitive difficulties.DisclosureNo significant relationships.

Design Thinking in the Framework of Visual Thinking and Characterization of Service Design Ideation Methods Using Visual Reasoning Model

Design Thinking is well recognized as a mindset and as a process in design and in business innovation and diverse complex problem-solving tasks. In this conceptual paper, design thinking process is explained using visual thinking composed of seeing – imagining – drawing so that its interactive and iterative nature can be better understood. The visual reasoning model developed earlier to support visual thinking with underlying cognitive activities as well as schema and knowledge is used to characterize four specific imagining methods for service design. Further research would be desirable to justify the potential of the visual reasoning model as a framework in devising methods and tools to support design thinking.

Animation as a dynamic visualization technique for improving process model comprehension

Process models are widely used for various system analysis and design activities, but it is challenging for stakeholders to understand these complex artifacts. In this work, we focus on the use of dynamic visualization techniques, in particular animation, to help reduce users’ cognitive load when making sense of process models. We built on the principles of the cognitive theory of multimedia learning, cognitive load theory, and cognitive dimensions framework to develop an adaptive animation solution. Our experiments suggested that process model comprehension improves when users of process models are provided with animation features; the effect is moderated by process modeling expertise according to a U-shape. Our study contributes to the field of conceptual modeling by making a strong case for the use of animation to support complex problem-solving tasks. Moreover, our animation solution offers ample opportunities for being integrated into industrial modeling tools.

Wisdom of crowds and collective decision-making in a survival situation with complex information integration

BackgroundThe wisdom of crowds and collective decision-making are important tools for integrating information between individuals, which can exceed the capacity of individual judgments. They are based on different forms of information integration. The wisdom of crowds refers to the aggregation of many independent judgments without deliberation and consensus, while collective decision-making is aggregation with deliberation and consensus. Recent research has shown that collective decision-making outperforms the wisdom of crowds. Additionally, many studies have shown that metacognitive knowledge of subjective confidence is useful for improving aggregation performance. However, because most of these studies have employed relatively simple problems; for example, involving general knowledge and estimating values and quantities of objects, it remains unclear whether their findings can be generalized to real-life situations involving complex information integration. This study explores the performance and process of the wisdom of crowds and collective decision-making by applying the wisdom of crowds with weighted confidence to a survival situation task commonly used in studies of collective decision-making.ResultsThe wisdom of crowds and collective decision-making outperformed individual judgment. However, collective decision-making did not outperform the wisdom of crowds. Contrary to previous studies, weighted confidence showed no advantage from comparison between confidence-weighted and non-weighted aggregations; a simulation analysis varying in group size and sensitivity of confidence weighting revealed interaction between group size and sensitivity of confidence weighting. This reveals that it is because of small group size and not the peculiarity of the survival task that results in no advantage of weighted confidence.ConclusionsThe study’s findings suggest that the wisdom of crowds could be applicable to complex problem-solving tasks, and interaction between group size and sensitivity of confidence weighting is important for confidence-weighted aggregation effects.

“Let’s Get Out of Here!”: Cognitive Motivation and Maximizing Help Teams Solving an Escape Room

Contemporary teams often face complex problem-solving tasks. We theorized that two individual differences previously neglected in team research (cognitive motivation and maximizing) would be helpful for teams facing such situations. We tested this assertion on 81 teams participating in an escape-room simulation in which teams were locked into a pre-arranged room and had to solve various complex problems to escape the room as quickly as possible. The findings show that the average of the team members’ cognitive motivation had a positive direct relation to team performance, while maximizing had a positive indirect relation to team performance via cooperation.

Programming experience associated with neural efficiency during figural reasoning

In the present study, we investigated neural processes underlying programming experience. Individuals with high programming experience might develop a form of computational thinking, which they can apply on complex problem-solving tasks such as reasoning tests. Therefore, N = 20 healthy young participants with previous programming experience and N = 21 participants without any programming experience performed three reasoning tests: Figural Inductive Reasoning (FIR), Numerical Inductive Reasoning (NIR), Verbal Deductive Reasoning (VDR). Using multi-channel EEG measurements, task-related changes in alpha and theta power as well as brain connectivity were investigated. Group differences were only observed in the FIR task. Programmers showed an improved performance in the FIR task as compared to non-programmers. Additionally, programmers exhibited a more efficient neural processing when solving FIR tasks, as indicated by lower brain activation and brain connectivity especially in easy tasks. Hence, behavioral and neural measures differed between groups only in tasks that are similar to mental processes required during programming, such as pattern recognition and algorithmic thinking by applying complex rules (FIR), rather than in tasks that require more the application of mathematical operations (NIR) or verbal tasks (VDR). Our results provide new evidence for neural efficiency in individuals with higher programming experience in problem-solving tasks.

Students’ Use of Retrieval in Self-Regulated Learning: Implications for Monitoring and Regulating Effortful Learning Experiences

Retrieval practice has been widely studied as an effective strategy for enhancing memory. In this review article, we discuss how its effects on learning complex problem-solving procedures are less straightforward, however, with repeated studying of worked examples sometimes more effective than problem-solving practice. This worked example benefit has been interpreted within the framework of cognitive load theory. In both memory-based tasks and problem-solving tasks, students rate retrieval as more effortful, and less effective for learning, than repeated study. Self-regulated learning decisions do not align with the evidence about the effectiveness of retrieval, as students often avoid using retrieval in memory-based tasks but frequently use it in more complex problem-solving tasks. Patterns associated with self-report survey data and self-regulated learning decisions suggest that retrieval may be used primarily as a means of checking knowledge, and the effort experienced during retrieval may drive subsequent study decisions (i.e., the choice to engage in retrieval vs. further study) to the extent that the experienced effort is interpreted as a sign that learning has been ineffective. We discuss implications of students’ views about the purpose of retrieval for effective monitoring and regulation of effort during learning, and propose interventions that may improve students’ optimal use of retrieval in their study decisions.

Students' Complex Problem Solving Profiles.

Complex problem solving (CPS) is an up-and-coming twenty-first century skill that requires test-takers to solve dynamically changing problems, often assessed using computer-based tests. The log data that users produce when interacting with a computer-based test provide valuable information about each individual behavioral action they undertake, but such data are rather difficult to handle from a statistical point of view. This paper addresses this issue by building upon recent research focused on decoding log data and aims to identify homogeneous student profiles with regard to their ability to solve CPS tasks. Therefore, we estimated a discrete two-tier item response theory model, which allowed us to profile units (i.e., students) while taking into account the multidimensionality of the data and the explanatory effect of individual characteristics. The results indicate that: (1) CPS can be thought of as a three-dimensional latent variable; (2) there are ten latent classes of students with homogenous profiles regarding the CPS dimensions; (3) students in the higher latent classes generally demonstrate higher cognitive and non-cognitive performances; (4) some of the latent classes seem to profit from learning-by-doing within tasks, whereas others seem to exhibit the reverse behavior; (5) cognitive and non-cognitive skills, as well as gender and to some extent age, contribute to distinguishing among the latent classes.