Cognitive Model Framework Research Articles

Team-centered IDAC: Modeling and simulation of operating crew in complex systems – Part 1: Team model and fundamentals

Operation of highly complex systems such as Nuclear Power Plants (NPPs) generally require highly trained professional operating teams. Factors associated with teamwork, such as ineffective communication and coordination, can be important contributing factors to accidents and unsafe behavior. The impact of crew interactions on team effectiveness and, consequently, on the entire system, has not been fully and quantitatively explored in high-risk environments such as NPPs. Since a team is an interactive social system, team-specific issues must be studied and evaluated from a “team perspective”—based on team dynamics and processes. This paper is part of a two-papers series that presents a simulation-based Team Model for NPP control room operations. The current paper, Part 1, describes the theoretical fundaments of the model and details its elements. The accompanying paper describes the simulation aspects, and a full application of the method to a pipe break accident in a four- four-loop steam generator feedwater system. The proposed model is based on the IDAC (Information, Decision, and Action in Crew context) cognitive model framework. The resulting model, Team-centered IDAC (Tc-IDAC), examines the team activities “Collaborative information collection,” “Shared decision making,” and “Distributed action execution” through specific modules for Team Error Management. These modules include error detection, error indication and error correction, and team performance shaping factors.

New Paradigm of Identifiable General-response Cognitive Diagnostic Models: Beyond Categorical Data.

Cognitive diagnostic models (CDMs) are a popular family of discrete latent variable models that model students' mastery or deficiency of multiple fine-grained skills. CDMs have been most widely used to model categorical item response data such as binary or polytomous responses. With advances in technology and the emergence of varying test formats in modern educational assessments, new response types, including continuous responses such as response times, and count-valued responses from tests with repetitive tasks or eye-tracking sensors, have also become available. Variants of CDMs have been proposed recently for modeling such responses. However, whether these extended CDMs are identifiable and estimable is entirely unknown. We propose a very general cognitive diagnostic modeling framework for arbitrary types of multivariate responses with minimal assumptions, and establish identifiability in this general setting. Surprisingly, we prove that our general-response CDMs are identifiable under -matrix-based conditions similar to those for traditional categorical-response CDMs. Our conclusions set up a new paradigm of identifiable general-response CDMs. We propose an EM algorithm to efficiently estimate a broad class of exponential family-based general-response CDMs. We conduct simulation studies under various response types. The simulation results not only corroborate our identifiability theory, but also demonstrate the superior empirical performance of our estimation algorithms. We illustrate our methodology by applying it to a TIMSS 2019 response time dataset.

베이지언 네트워크를 이용한 인지진단모형의 적용

Bayesian networks(BN) are a probabilistic graphical model that represents a system of variables and
 their conditional interdependencies using directed acyclic graphs. Once the probabilistic models are
 built and embedded in a graphical structure, evidence of variables can be efficiently propagated
 through the network and inference about students can be made. This study is to explore the potential
 of BN as a new framework for cognitive diagnostic modeling by developing and applying BN-based
 cognitive diagnostic models to a real test data set and comparing the results with those from the
 conventional cognitive diagnostic modeling. For this purpose, the current study surveyed theoretical and
 pedagogical aspects of reading in English as a foreign/second language, introduced provisional models,
 and applied the models to a set of reading comprehension data from a TOEIC practice test. The
 results show that BN-based cognitive diagnostic modeling can be an efficient framework that can
 handle diverse aspects of reading comprehension. The study then addresses some issues from the
 analysis and discusses the implications in relation to application of cognitive diagnostic modeling.

The promotional effect of prosocial motivation on time-based prospective memory.

Time-based prospective memory (TBPM) is the ability to remember to do a planned task at the right time. In social interactions, people are often motivated to do things for others, which reflects an important factor that influences prospective memory, namely prosocial motivation. According to the motivational cognitive model, prosocial motivation promotes TBPM by paying more attention or adopting more effective strategies. This study explored the effect of prosocial motivation on TBPM under different time-monitoring conditions within the motivational cognitive model framework. One hundred and thirty-one university students participated in this experiment that adopted a 2 (groups: control, prosocial motivation) × 2 (viewing time conditions: limited, unlimited) between-subjects design. The results revealed that the prosocial motivation group had better TBPM performance than the control group under both limited and unlimited viewing time conditions. At the same time, compared with the control group, the prosocial motivation group consumed more internal attention and utilized more strategies under both viewing time conditions, and their external attention was more effective. In addition, the external attention of the prosocial motivation group was higher only when time-monitoring was unlimited. The results of this study further extend knowledge of the motivational cognitive model and expand its scope of application, which has theoretical significance.

A Mixture Fluency model using responses and response times with cognitive diagnosis model framework.

Random guessing behaviors are frequently observed in low-stakes assessments, often attributed to factors such as test-takers lacking motivation or experiencing time constraints and fatigue. Existing research suggests that responses stemming from random guessing behaviors introduce biases into the constructs and relationships of interest. This is particularly problematic when estimating the relationship between speed and ability. This study introduces a Mixture Fluency model designed to account for random guessing behaviors while utilizing valid response accuracy and response time to uncover students' latent attribute profiles. The model directly addresses a limitation present in the Fluency cognitive diagnostic model (Wang & Chen, Psychometrika, 85, 600-629, (2020), which assumes that test-takers consistently employ solution behaviors when answering questions. To investigate the effectiveness of the proposed Mixture Fluency model, we conducted a simulation study encompassing various simulation conditions. Results from this study not only confirm the model's ability to detect potential random guessing behaviors but also demonstrate its capacity to enhance the inference of targeted latent constructs within the assessment. Additionally, we showcase the practical utility of the proposed model through an application to real data.

The Investigation of The Differential Item Functions of The 2012 High School Entrance Exam Mathematics Test Based On The G-DINA Model

Purpose: This study aims to determine the item parameters and estimations of student attribute profiles related to the 2012 8th Grade HSEE Mathematics subtest item responses using the G-DINA model and examine whether the items show DIF according to the gender variable.
 Design/Methodology/Approach: The study was conducted on the data of 1.063.570 students who took the 2012 HSEE Mathematics subtest. The analysis was carried out on the target population data to avoid sampling error. This study is a descriptive, survey type study.
 Findings: When the model fit indices and comparison results were examined, it was concluded that the model that best explained the behavior of students responding to the items in the 20-item form of the 2012 HSEE Mathematics subtest was G- DINA. When the G-DINA model parameters were examined, it was found that the 𝛼! = [0000] attribute profile (with 61%) in which none of the four defined attributes were found in the student, and the 𝛼!" = [1111] attribute profile (with 17%) in which all four attributes were present in the student (17%) were the most common attribute profiles. As a result of DIF analysis within the scope of CDM, and it was identified that item 4 showed a significant uniform DIF in favor of female students while item 19 showed a uniform DIF in favor of male students at moderate level.
 Highlights: It is thought that DIF analyses within the framework of cognitive diagnostic models can provide a statistical basis for item bias decisions.

A Multistrategy Cognitive Diagnosis Model Incorporating Item Response Times Based on Strategy Selection Theories

Response times (RTs) facilitate the quantification of underlying cognitive processes in problem-solving behavior. To provide more comprehensive diagnostic feedback on strategy selection and attribute profiles with multistrategy cognitive diagnosis model (CDM) and utilize additional information for item RTs, this study develops a multistrategy cognitive diagnosis modeling framework combined with RTs. The proposed model integrates individual response accuracy and RT into a unified framework to define strategy selection and make it closer to the individual’s strategy selection process. Simulation studies demonstrated that the proposed model had reasonable parameter recovery and attribute classification accuracy and outperformed the existing multistrategy CDMs and single-strategy CDMs in terms of performance. Empirical results further illustrated the practical application and the advantages of the proposed model.

Inspiring management of innovative projects

The role and place of inspirational management in the management of projects for the creation of innovative projects for the development of organizations is considered. A conceptual model of inspiring management in the processes of managing innovative projects based on emotional intelligence, inspiring intuition, leadership and motivation is presented. The influence of cognitive models on the development of inspiring management of innovative projects regarding the acquisition of knowledge and management technologies is determined. Within the framework of cognitive models, on the example of a construction project, the qualitative effects of individual competencies on the formation of inspiring management are determined. The model related to the application of the system of knowledge and competencies for the management of innovative projects and P2M programs allows to evaluate the key competencies of managers. Within this model, the priority competencies that form inspiring project management are defined. Based on the assessment of cognitive maps in the usual mode and in the mode of inspiring management, a model of the influence of inspiring management on the competencies for the implementation of innovative projects and programs was built.

A cognitive process modeling framework for the ABCD study stop-signal task

The Adolescent Brain Cognitive Development (ABCD) Study is a longitudinal neuroimaging study of unprecedented scale that is in the process of following over 11,000 youth from middle childhood though age 20. However, a design feature of the study's stop-signal task violates "context independence", an assumption critical to current non-parametric methods for estimating stop-signal reaction time (SSRT), a key measure of inhibitory ability in the study. This has led some experts to call for the task to be changed and for previously collected data to be used with caution. We present a cognitive process modeling framework, the RDEX-ABCD model, that provides a parsimonious explanation for the impact of this design feature on “go” stimulus processing and successfully accounts for key behavioral trends in the ABCD data. Simulation studies using this model suggest that failing to account for the context independence violations in the ABCD design can lead to erroneous inferences in several realistic scenarios. However, we demonstrate that RDEX-ABCD effectively addresses these violations and can be used to accurately measure SSRT along with an array of additional mechanistic parameters of interest (e.g., attention to the stop signal, cognitive efficiency), advancing investigators’ ability to draw valid and nuanced inferences from ABCD data. Availability of data and materialsData from the ABCD Study are available through the NIH Data Archive (NDA): nda.nih.gov/abcd. Code for all analyses featured in this study is openly available on the Open Science Framework (OSF): osf.io/2h8a7/.

Unleashing the Potentials of Quantum Probability Theory for Customer Experience Analytics

In information systems research, the advantages of Customer Experience (CX) and its contribution to organizations are largely recognized. The CX analytics evaluate how customers perceive products, ranging from their functional usage to their cognitive states regarding the product, such as emotions, sentiment, and satisfaction. The most recent research in psychology reveals that cognition analytics research based on Classical Probability Theory (CPT) and statistical learning, which is used to evaluate people’s cognitive states, is limited due to their reliance on rational decision-making. However, the cognitive attitudes of customers are characterized by uncertainty and entanglement, resulting in irrational decision-making bias. What is captured by traditional CPT-based data science in the context of cognition aspects of CX analytics is only a small portion of what should be captured. Current CX analytics efforts fall far short of their full potential. In this paper, we set a novel research direction for CX analytics by Quantum Probability Theory (QPT). QPT-based analytics have been introduced recently in psychology research and reveal better cognition assessment under uncertainty, with a high level of irrational behavior. Adopting recent advances in the psychology domain, this paper develops a vision and sets a research agenda for expanding the application of CX analytics by QPT to overcome CPT shortcomings, identifies research areas that contribute to the vision, and proposes elements of a future research agenda. To stimulate debate and research QPT-CX analytics, we attempt a preliminary characterization of the novel method by introducing a QPT-based rich mathematical framework for CX cognitive modeling based on quantum superposition, Bloch sphere, and Hilbert space. We demonstrate the implementation of the QPT-CX model by the use case of customers’ emotional motivator assessments while implementing quantum vector space with a set of mathematical axioms for CX analytics. Finally, we outline the key advantages of quantum CX over classical by supporting theoretical proof for each key.

Artificial Psychologist: An intelligent virtual/robotic assistant based on a cognitive modeling framework

Applications of socially emotional artificial intelligence are demanded in many practical areas, including psychological counselling and psychotherapy. A person who is potentially in trouble and needs help has to overcome an internal barrier before deciding to see a psychiatrist. On the other hand, obtaining an anonymous online consultation or taking an online test is psychologically easy and acceptable for the many. At the same time, state-of-the-art solutions of this sort are limited. Here a concept of an intelligent system is proposed that should help to alleviate the problem. The system can be implemented as an embodied actor based on a virtual environment, VR/XR, or a robotic platform with social-emotional capabilities. The actor is controlled by a cognitive architecture with a possibility of its replacement by a deep neural network model. Interaction with the user is based on a multimodal human-computer interface. Possibilities that will be provided by this virtual psychologist system will enable conducting an initial screening, the results of which can later be used at the next session with a specialist, if chosen by the user. The key issue of ethics, privacy and anonymity is discussed. Systems of this sort could be used beyond psychological counseling and are expected to have an impact on the healthcare system in general.

Do background characteristics matter in Children's mastery of digital literacy? A cognitive diagnosis model analysis

This study aims to investigate the mastery profiles of digital literacy skills of Hong Kong primary students using a general cognitive diagnosis model (CDM) framework. In particular, the relationship between the mastery of each digital skill and a number of students' background characteristics is explored using a three-step approach. The current study analyzes data collected from 642 Grade 3 students in Hong Kong using a newly developed digital literacy assessment (DLA). CDMs are fitted to the data to determine students' mastery profiles of five digital skills, as well as test properties; subsequently latent logistic regression analyses were implemented to determine the relationship between skill mastery and the covariates. Results indicate that CDM analysis is an appropriate method to analyze the DLA performance data, which exhibited measurement invariance across gender and socioeconomic status (SES). Despite low mastery proportions for all digital skills, students' skill mastery can be accurately classified. Finally, the latent logistic regression results indicate that children's background characteristics (i.e., gender, educational aspiration, home language, SES, and access to digital devices) are differentially related to their mastery of each digital skill.

Mood-Related Changes in Bayesian Reward Learning Processes and Their Relationship to Anhedonia in Trauma-Exposed Veterans

Anhedonia, a reduced drive to pursue healthy rewards, is a common symptom in individuals recently exposed to trauma, and is associated with worse clinical outcomes, including higher rates of suicidality and substance use disorders (SUD). Using a Bayesian cognitive model framework, this study aimed to identify robust neurocognitive markers of anhedonia in combat trauma-exposed Veterans.

Data-driven experimental design and model development using Gaussian process with active learning

Interest in computational modeling of cognition and behavior continues to grow. To be most productive, modelers should be equipped with tools that ensure optimal efficiency in data collection and in the integrity of inference about the phenomenon of interest. Traditionally, models in cognitive science have been parametric, which are particularly susceptible to model misspecification because their strong assumptions (e.g. parameterization, functional form) may introduce unjustified biases in data collection and inference. To address this issue, we propose a data-driven nonparametric framework for model development, one that also includes optimal experimental design as a goal. It combines Gaussian Processes, a stochastic process often used for regression and classification, with active learning, from machine learning, to iteratively fit the model and use it to optimize the design selection throughout the experiment. The approach, dubbed Gaussian process with active learning (GPAL), is an extension of the parametric, adaptive design optimization (ADO) framework (Cavagnaro, Myung, Pitt, & Kujala, 2010). We demonstrate the application and features of GPAL in a delay discounting task and compare its performance to ADO in two experiments. The results show that GPAL is a viable modeling framework that is noteworthy for its high sensitivity to individual differences, identifying novel patterns in the data that were missed by the model-constrained ADO. This investigation represents a first step towards the development of a data-driven cognitive modeling framework that serves as a middle ground between raw data, which can be difficult to interpret, and parametric models, which rely on strong assumptions.

Evaluating the effectiveness of 'MCQ development workshop using cognitive model framework: A pre-post study.

A workshop on MCQ development using cognitive model framework was conducted for health educators from Aga Khan University (AKU) and other academic institutions. The aim was to develop the skill of preparing MCQs for assessing higher cognitive levels. A pre-post study was conducted, participant satisfaction was evaluated and pre-post test scores were used to assess learning capability of the workshop participants. Out of the 19 who attended the workshop, 16 participated in the pre- and post-tests and were included in the study through convenience sampling. The total duration of the study was six months. There was a significant difference in the overall pre-post test scores of the participants with a mean difference of -4.176 ± 4.83 (p-value < 0.05). A significant difference was observed in the mean pre-post test scores of junior faculty (-6.350± 4.5829; p-value = 0.02). The mean pre-test scores of junior faculty were significantly lower 4.950±2.83 as compared to the senior faculty 10.417±1.56 (p-value= 0.001). Active participation in faculty development workshops may lead to enhancing skills for preparing one-best MCQs based on international guidelines.

Using Response Times and Response Accuracy to Measure Fluency Within Cognitive Diagnosis Models.

The recent "Every Student Succeed Act" encourages schools to use an innovative assessment to provide feedback about students' mastery level of grade-level content standards. Mastery of a skill requires the ability to complete the task with not only accuracy but also fluency. This paper offers a new sight on using both response times and response accuracy to measure fluency with cognitive diagnosis model framework. Defining fluency as the highest level of a categorical latent attribute, a polytomous response accuracy model and two forms of response time models are proposed to infer fluency jointly. A Bayesian estimation approach is developed to calibrate the newly proposed models. These models were applied to analyze data collected from a spatial rotation test. Results demonstrate that compared with the traditional CDM that using response accuracy only, the proposed joint models were able to reveal more information regarding test takers' spatial skills. A set of simulation studies were conducted to evaluate the accuracy of model estimation algorithm and illustrate the various degrees of model complexities.

THE USE OF SOLUTION-ORIENTED STRATEGIES TO SUPPORT GRANDPARENTS RAISING GRANDCHILDREN

There is consistent evidence that caregiving for grandchildren is psychologically, emotionally and physically challenging for grandparents. However, many of them may not be able to receive the support they need from their own families or public services. Using the Selective Optimization with Compensation framework and principles of solution-oriented cognitive models, we implemented a six-week session pilot program to increase grandparents’ effective coping strategies, reduce stress, and improve parenting and communication with grandchildren. Preliminary data from 22 grandparents suggest that participants improved their use of effective selection strategies to set priorities and pursue goals related to their wellbeing and the quality of their relationship with grandchildren. Their pre-post test scores also indicated a higher level of parental efficacy, social relations and confidence in organizing their time and priorities. Results indicate that focusing on solution-oriented strategies may empower grandparents to improve their psychological adjustment and take care of their needs.

Development of a New Instrument for Depression With Cognitive Diagnosis Models.

Most existing instruments for depression are developed based on classical test theory, factor analysis, or sometimes, item response theory, and focus on the accurate measurement of the severity of depressive disorder. Nevertheless, they tend to be less useful in supporting the decision based on ICD-10 or DSM-5 because of the lack of detailed information for symptoms. To gain rich and valid information at the symptom level, this article developed a depression test under the framework of cognitive diagnosis models (CDMs), referred to as CDMs-D. A total of 1,181 individuals were finally recruited and their responses were used to examine the psychometric properties of CDMs-D. After excluding poor items for statistical reasons (e.g., low discrimination, poor model-fit or having DIF), 56 items were included in the CDMs-D. The CDMs-D measures all ten symptom criteria for depression defined in ICD-10 and covers five domains of depression defined by Gibbons et al. (2012). Comparing with the existing self-report measures (such as PHQ-9, SDS, CES-D and so on), a distinguishing feature of the CDMs-D is that it can provide both overall information about the severity of depressive disorder and the assessment information about specific symptoms, which could be useful for diagnostic and interventional purposes.

The CDA-BPD: retrofitting a traditional borderline personality questionnaire under the cognitive diagnosis model framework

To obtain rich information about the cognitive diagnosis of borderline personality disorder (BPD), this study attempted to retrofit a traditional borderline personality questionnaire so that the improved assessment (called CDA-BPD) could provide more diagnostic information. The retrofitting processes included the following steps: (1) applied an cognitive diagnosis model to analyze the psychometric characteristics of the traditional questionnaire; (2) under the guidance of cognitive diagnosis assessment (CDA), high-quality items were chosen to develop the CDA-BPD and tested on 1,097 subjects; (3) the quality of the CDA-BPD was evaluated; (4) the structure of the CDA-BPD was analyzed. Results indicated that: (1) the CDA-BPD had acceptable reliability and validity; (2) the CDA-BPD had sensitivity of 0.985 and specificity of 0.853 with area under curve (AUC) = 0.956; (3) the two structural factors of the traditional questionnaire were confirmed in the CDA-BPD; χ2 was 83.01 with df = 26, p &lt; .0001, comparative fit index (CFI) = 0.97, root mean square error of approximation (RMSEA) = 0.045. It was concluded that the practice of retrofitting a traditional borderline personality assessment for cognitive diagnostic purpose was feasible. Most importantly, under the cognitive diagnosis model framework, CDA-BPD could simultaneously provide general-level information and the detailed symptom criteria-level information about the posterior probability of satisfying each symptom criterion in the Diagnostic and Statistical Manual of Mental Disorders (5th edition; DSM-5; American Psychiatric Association, 2013 ) for each individual, which gave further insight into tailoring individual-specific treatments for borderline personality disorder.

Effects of Item Calibration Errors on Computerized Adaptive Testing under Cognitive Diagnosis Models

In a cognitive diagnostic computerized adaptive testing (CD-CAT) exam, an item pool that consists of items with calibrated item parameters is used for item selection and attribute estimation. The parameter estimates for the items in the item pool are often treated as if they were the true population parameters, and therefore, the calibration errors are ignored. The purpose of this study was to investigate the effects of calibration errors on the attribute classification accuracy, the measurement precision of attribute mastery classification, and the test information under the log-linear cognitive diagnosis model (LCDM) framework. The deterministic input, noisy “and” gate (DINA) model and the compensatory re-parameterized unified model (C-RUM) were used in fixed-length CD-CAT simulations. The results showed that high levels of calibration errors were associated with low classification accuracy, low test information, and misleading estimation of measurement precision. The effects of calibration errors decreased as the test length increased, and the DINA model appeared to be more vulnerable in the presence of calibration errors. The C-RUM was less influenced by calibration errors because of its additive characteristics in the LCDM framework. The same conclusions applied when item exposure control was incorporated and when different item selection methods were used. Finally, the use of a larger calibration sample size to calibrate the item pool was found to reduce the magnitudes of error variances and increase the attribute classification accuracy.