Cognitive Diagnostic Testing Research Articles

Cognitive Diagnostic Tests for Dementia in Primary Care

Cognitive Diagnostic Tests for Dementia in Primary Care

Profil Miskonsepsi Siswa pada Materi Bentuk Aljabar di Sekolah Menengah Pertama Negeri Kota Banda Aceh

Learning loss has an impact on various problems, including misconceptions by students when recognizing and learning algebraic operations. This research aims to identify the profile of students' misconceptions in algebraic concepts as a consequence of learning loss due to the Covid-19 pandemic in public junior high school in Banda Aceh. The study adopts a quantitative approach. The population of the research consists of all eighth-grade students in public junior high school in Banda Aceh and the sample comprises 27 students from one of these schools. The sample was obtained through purposive sampling, selecting students from the top-ranked junior high school in Banda Aceh, known for having the highest average scores. Misconceptions were assessed using an online-based cognitive diagnostic test on algebraic concepts. Data analysis was conducted by examining the composition of misconceptions based on their types. The research findings indicate that the majority of students experience misconceptions, with 25.20% experiencing misconceptions in notation, 20.37% in generalization, 8.98% in understanding symbols, 9.97% in applying rules, 1.14% in notation and generalization, 0.57% in notation and understanding symbols, and 2% in generalization and applying rules. This study indicates a necessity to an analysis of the causes of misconceptions and efforts to minimize students' misconceptions in algebraic concepts, such as implementing cognitive conflict learning or remedial teaching.

Analisis Hasil Asesmen Diagnostiksebagai Dasar Pelaksanaan Pembelajaran Berdiferensiasi Di Sekolah Dasar

Students have different characteristics and a diversity of learning needs. Educators need to map students' learning needs according to students' learning abilities, interests and learning talents. Cognitive and non-cognitive diagnostic assessments make it easier for teachers to map students' learning needs through tests or expanded questionnaires. Diagnostic assessments are carried out with the aim of reviewing students' initial abilities in learning readiness. So it can be a solution in applying differentiated learning. The purpose of this observation is to analyze the number of diagnostic assessments as a benchmark for differentiated learning for Pancasila Education subjects in class IV. The research method used is descriptive qualitative. This data was collected using non-participatory observation from carrying out cognitive diagnostic assessment tests. Data analysis techniques use the Milles and Huberman models, including value reduction, data presentation and drawing conclusions. Research in class IV A. The results of the research show that educators have mapped groups based on students' learning readiness using cognitive diagnostic assessment tests. Groupings are made from developing, and advanced groups. Content differentiated learning can be applied to provide learning material based on proficiency level so that students' learning needs are met.

Novel item selection strategies for cognitive diagnostic computerized adaptive testing: A heuristic search framework.

The computerized adaptive form of cognitive diagnostic testing, CD-CAT, has gained increasing attention in the domain of personalized measurements for its ability to categorize individual mastery status of fine-grained attributes more accurately and efficiently through administering items tailored to one's ability progressively. How to select the next item based on previous response(s) is crucial for the success of CD-CAT. Previous item selection strategies for CD-CAT have often followed a greedy or semi-greedy approach, which makes it difficult to strike a balance between diagnostic performance and item bank utilization. To address this issue, this study takes a graph perspective and transforms the item selection problem in CD-CAT into a path-searching problem, in which paths refer to possible test construction and nodes refer to individual items. A heuristic function is defined to predict the prospect of a path, indicating how well the corresponding test can diagnose the current examinee. Two search mechanisms with different biases towards item exposure control are proposed to approximate the optimal path with the best prospect. The first unused item on the resulting path is selected as the next item. The above components compose a novel CD-CAT item selection framework based on heuristic search. Simulation studies are conducted under a variety of conditions regarding bank designs, bank-quality conditions, and testing scenarios. The results are compared with different types of classic item selection strategies in CD-CAT, showing that the proposed framework can enhance bank utilization at a smaller cost of diagnostic performance.

Multisensory supplementary instructional material in earth and space science

This study determined the effect of the developed multisensory supplementary instructional material in enhancing scientific knowledge of grade 9 learners. Moreover, the study assessed significant difference between the pre-and post-assessment scores of the learners who used the instructional material and the significant relationship between the perceived acceptability of the multisensory supplementary instructional material and the students’ level of scientific knowledge. Using descriptive-developmental research, it involved 30 grade 9 students during the school year 2022-2023. A cognitive diagnostic test in terms of remembering, understanding, applying, analyzing, evaluating, and creating was used to identify the least-mastered competency among the topics. Pre-and post-assessment, which measured the scientific knowledge of the students, went through internal and external validation of the panel of examiners and group of teachers. Results revealed a significant difference between the pre-and post-assessment scores of the students as to content and epistemic knowledge whereas there is no significant difference found in the procedural knowledge. It can be inferred that the students enhanced their scientific knowledge after the utilization of the multisensory supplementary instructional material. However, there is no significant relationship obtained between the students’ perception on the acceptability of the material to their level of scientific knowledge.

Strategi Sekolah Menengah Pertama dalam mengimplementasikan Kurikulum Merdeka

This research aims to determine the implementation of the independent curriculum at SMP Negeri 9 Pontianak which is a driving school and the independent curriculum system implemented in grade 7. The learning system approach reflects an educational philosophy that provides space for creativity, independent thinking and greater understanding of students. The P5 chosen were local culture, entrepreneurship and sustainable living. This research method uses descriptive qualitative. The research population is teachers and students of SMP Negeri 9 Pontianak, the research sample consists of teachers, curriculum assistants and students. Research analysis was carried out according to Milles and Huberman's analysis. The research results show that at the beginning of the learning process there must be cognitive diagnostic tests and non-cognitive tests. At the end of the teaching and learning process. This diagnostic planning assessment is made by the teacher. Difficulties in implementing the independent curriculum at SMP Negeri 9 Pontianak were overcome by providing special trainers by the education center, conducting IHT house learning, every Friday the school has a meeting to discuss obstacles related to implementing the independent curriculum and the four SMP Negeri 9 have their own application provided by the government called Merdeka Learning contains information about the independent curriculum, such as providing training topics for teachers, references for teaching tools and teachers can also find out students' ability levels.

Investigating the 8th-grade Afghan students’ mathematics status and skills using the cognitive diagnostic model

Cognitive diagnostic models (CDMs) are multidimensional multivariate verification flow models with complex structure. In this research, these models were used to investigate the status of eighth grade high school students in mathematics using the TIMMS questionnaire. The cognitive diagnostic test based on 13 attributes including 32 questions was conducted on a sample of 274 students who were selected based on the multi-stage cluster sampling method among the students of Firuzkoh city. IRT and RESMA models were used to determine the psychometric properties of the questions. Data analysis using DINA and DINO models in cognitive diagnostic modeling of mathematics showed that 13 attributes explain the mathematical performance of eighth-grade students. The result shows that Afghan students have a weak mastery level in most attributes compared to 45 other countries (the countries that were included in the TIMMS questionnaire) also general results show that the examinees perform better in the field of numbers (0.49), while they perform worse in data and chance (0.12). Moreover, there exists some difference in estimating item parameters under the DINA and DINO models, such as Item 3 and Item 27. One possible explanation is that the DINA model is completely compensatory while the DINO model is fully non-compensatory. Similar to the results under the DINA model, the SEs of guessing parameters are lower than those of slipping parameters under the DINO model.

Investigating the status and skills of Afghan students using the Cognitive Diagnostic Model (CDM) of Afghanistan's 8th grade mathematics.

Cognitive diagnostic models (CDMs) are multidimensional multivariate verification flow models with complex structure. In this research, these models were used to investigate the status of eighth grade high school students in mathematics using the TIMMS questionnaire. The cognitive diagnostic test based on 13 attributes including 32 questions was conducted on a sample of 274 students who were selected based on the multi-stage cluster sampling method among the students of Firuzkoh city. IRT and RESMA models were used to determine the psychometric properties of the questions. Data analysis using DINA and DINO models in cognitive diagnostic mathematics modeling showed that 13 attributes explain the mathematical performance of eighth-grade students. The result shows that Afghan students have a weak mastery level in most features compared to 45 other countries (the countries that were included in the TIMMS questionnaire) also, general results show that the examinees perform better in the field of numbers (0.49), while they perform worse in data and chance (0.12 ). Moreover, some difference exists in estimating item parameters under the DINA and DINO model, such as Item 3 and Item 27. One possible explanation is that the DINA model is completely compensatory while the DINO model is fully non-compensatory. Similar to the results under the DINA model, the SEs of guessing parameters are lower than those of slipping parameters under the DINO model.

FlexCDMs: A Web-based Platform for Cognitive Diagnostic Data Analysis

Cognitive diagnosis is an important component of modern measurement theory and has received widespread attention from researchers in the fields of education and psychological measurement. Existing cognitive diagnosis analysis tools rely on professional software packages (such as R packages), which creates significant challenges for users, especially those who are not familiar with computer programming. To remove this technical barrier, our team has developed a web-based, user-friendly platform, named flexCDMs, for cognitive diagnosis data analysis. This article describes the features of the platform, the functional modules, the implemented cognitive diagnosis models (CDMs) and algorithms, and illustrates the operations of the platform. This platform can be used to analyze data based on various cognitive diagnosis models, carry out Q-matrix theory, model-data fit tests, parameter estimation, quality analysis of cognitive diagnostic tests, differential item functioning (DIF) detection, and Q-matrix modification. It produces various charts and graphs to report results. It is a powerful, yet easy to use cognitive diagnosis data analysis tool. The website for the flexCDMs platform is: http://111.230.233.68:1001/?Id=false&Block

Using a Generalized Logistic Regression Method to Detect Differential Item Functioning With Multiple Groups in Cognitive Diagnostic Tests.

Items with the presence of differential item functioning (DIF) will compromise the validity and fairness of a test. Studies have investigated the DIF effect in the context of cognitive diagnostic assessment (CDA), and some DIF detection methods have been proposed. Most of these methods are mainly designed to perform the presence of DIF between two groups; however, empirical situations may contain more than two groups. To date, only a handful of studies have detected the DIF effect with multiple groups in the CDA context. This study uses the generalized logistic regression (GLR) method to detect DIF items by using the estimated attribute profile as matching criteria. A simulation study is conducted to examine the performance of the two GLR methods, GLR-based Wald test (GLR-Wald) and GLR-based likelihood ratio test (GLR-LRT), in detecting the DIF items, the results based on the ordinary Wald test are also reported. Results show that (1) both GLR-Wald and GLR-LRT have more reasonable performance in controlling Type I error rates than the ordinary Wald test in most conditions; (2) the GLR method also produces higher empirical rejection rates than the ordinary Wald test in most conditions; and (3) using the estimated attribute profile as the matching criteria can produce similar Type I error rates and empirical rejection rates for GLR-Wald and GLR-LRT. A real data example is also analyzed to illustrate the application of these DIF detection methods in multiple groups.

Statistical Applications to Cognitive Diagnostic Testing

Diagnostic classification tests are designed to assess examinees’ discrete mastery status on a set of skills or attributes. Such tests have gained increasing attention in educational and psychological measurement. We review diagnostic classification models and their applications to testing and learning, discuss their statistical and machine learning connections and related challenges, and introduce some contemporary and future extensions.

Generic Identifiability of the DINA Model and Blessing of Latent Dependence.

Cognitive diagnostic models are a powerful family of fine-grained discrete latent variable models in psychometrics. Within this family, the DINA model is a fundamental and parsimonious one that has received significant attention. Similar to other complex latent variable models, identifiability is an important issue for CDMs, including the DINA model. Gu and Xu (Psychometrika 84(2):468-483, 2019) established the necessary and sufficient conditions for strict identifiability of the DINA model. Despite being the strongest possible notion of identifiability, strict identifiability may impose overly stringent requirements on designing the cognitive diagnostic tests. This work studies a slightly weaker yet very useful notion, generic identifiability, which means parameters are identifiable almost everywhere in the parameter space, excluding only a negligible subset of measure zero. We propose transparent generic identifiability conditions for the DINA model, relaxing existing conditions in nontrivial ways. Under generic identifiability, we also explicitly characterize the forms of the measure-zero sets where identifiability breaks down. In addition, we reveal an interesting blessing-of-latent-dependence phenomenon under DINA-that is, dependence between the latent attributes can restore identifiability under some otherwise unidentifiable [Formula: see text]-matrix designs. The blessing of latent dependence provides useful practical implications and reassurance for real-world designs of cognitive diagnostic assessments.

Cognitive Diagnostic Multistage Testing by Partitioning Hierarchically Structured Attributes

AbstractIn cognitive diagnostic models (CDMs), a set of fine‐grained attributes is required to characterize complex problem solving and provide detailed diagnostic information about an examinee. However, it is challenging to ensure reliable estimation and control computational complexity when The test aims to identify the examinee's attribute profile in a large‐scale map of attributes. To address this problem, this study proposes a cognitive diagnostic multistage testing by partitioning hierarchically structured attributes (CD‐MST‐PH) as a multistage testing for CDM. In CD‐MST‐PH, multiple testlets can be constructed based on separate attribute groups before testing occurs, which retains the advantages of multistage testing over fully adaptive testing or the on‐the‐fly approach. Moreover, testlets are offered sequentially and adaptively, thus improving test accuracy and efficiency. An item information measure is proposed to compute the discrimination power of an item for each attribute, and a module assembly method is presented to construct modules anchored at each separate attribute group. Several module selection indices for CD‐MST‐PH are also proposed by modifying the item selection indices used in cognitive diagnostic computerized adaptive testing. The results of simulation study show that CD‐MST‐PH can improve test accuracy and efficiency relative to the conventional test without adaptive stages.

A Multi-level Remedial Teaching Design Based on Cognitive Diagnostic Assessment: Taking the Electromagnetic Induction as an Example

Multi-level teaching has been proven to be more effective than a one-size-fits-all learning approach. This study aimed to develop and implement a multi-level remedial teaching scheme in various high school classes containing students of a wide range of learning levels and to determine its effect of their learning. The deterministic inputs noisy and gate model of cognitive diagnosis theory was used to classify students at multiple levels according to their knowledge and desired learning outcomes. A total of 680 senior high school students from central provinces in China participated in the initial cognitive diagnostic test, and 1,615 high school sophomores from seven high schools in China participated in a formal cognitive diagnosis test. Thirty-six high school students from Southwestern China participated in the think-aloud protocols, and 258 seniors from three high schools in southwest China participated in the remedial teaching experiment. Through an analysis of students’ think-aloud protocols, cognitive errors of students at all levels were determined, and multi-level remedial teaching programs were designed to address these common cognitive errors. The remedial teaching programs were then implemented in three schools and compared with a control group. The results indicated that the students in the experimental group showed a more significant improvement. In this study, the steps of designing multi-level remedial teaching include assessment, classification, and preparing a teaching scheme, which are feasible and can have remarkable teaching effects. This process can be used for reference by teachers of various subjects.

A Simplified Method for Predicting Pattern Match Ratio.

Cognitive diagnostic test design (CDTD) has a direct impact on the pattern match ratio (PMR) of the classification of examinees. It is more helpful to know the quality of a test during the stage of the test design than after the examination is taken. The theoretical construct validity (TCV) is an index of the test quality that can be calculated without testing, and the relationship between the PMR and the TCV will be revealed. The TCV captures the three aspects of the appeal of the test design as follows: (1) the TCV is a measure of test construct validity, and this index will navigate the processes of item construction and test design toward achieving the goal of measuring the intended objectives, (2) it is the upper bound of the PMR of the knowledge states of examinees, so it can predict the PMR, and (3) it can detect the defects of test design, revise the test in time, improve the efficiency of test design, and save the cost of test design. Furthermore, the TCV is related to the distribution of knowledge states and item categories and has nothing to do with the number of items.

Competence to Consent to Oral and Injectable PrEP Trials Among Adolescent Males Who Have Sex with Males.

Adherence to oral pre-exposure prophylaxis (PrEP) is challenging for adolescent males who have sex with males (AMSM). Once adult trials comparing oral to longer lasting injectable PrEP are completed, there will be a need for adolescent studies. However, lack of data on adolescent consent capacity may sustain guardian permission requirements identified as a barrier to AMSM participation in prior PrEP trials. This online study assessed AMSM's (14-17years) consent capacity for these trials, comparing performance to MSM (18-19years) for whom guardian permission is not required. Applying the MacCAT-CR, participants (N = 214) viewed a video and mock consent form followed by open-ended and yes/no items. Cognitive diagnostic models and means testing analyses supported AMSM capacity to consent to these trials: 16-17 and most 14-15year-olds, demonstrated consent understanding, appreciation and reasoning at 18-19year-old levels. Data also identified vulnerabilities requiring attention during informed consent.

Developing a Learning Progression for Probability Based on the GDINA Model in China.

This research focuses on developing a learning progression of probability for middle school students, and it applies the GDINA model in cognitive diagnosis models to data analysis. GDINA model analysis firstly extracted nine cognitive attributes and constructed their attribute hierarchy and the hypothesized learning progression according to previous studies, curriculum standards, and textbooks. Then the cognitive diagnostic test was developed based on Q-matrix theory. Finally, we used the GDINA model to analyze a sample of 1624 Chinese middle school students’ item response patterns to identify their attribute master patterns, verify and modify the hypothesized learning progression. The results show that, first of all, the psychometric quality of the measurement instrument is good. Secondly, the hypothesized learning progression is basically reasonable and modified according to the attribute mastery probability. The results also show that the level of probabilistic thinking of middle school students is improving steadily. However, the students in grade 8 are slightly regressive. These results demonstrate the feasibility and superiority of using cognitive diagnosis models to develop a learning progression.

Assessing kindergarteners’ mathematics problem solving: The development of a cognitive diagnostic test

This study aimed to develop an instrument for assessing kindergarteners’ mathematics problem solving (MPS) by using cognitive diagnostic assessment (CDA). A total of 747 children were recruited to examine the psychometric properties of the cognitive diagnostic test. The results showed that the classification accuracy of 11 cognitive attributes ranged from .68 to .99, with the average being .84. Both the cognitive diagnostic test score and the average mastery probabilities of the 11 cognitive attributes had moderate correlations with the Applied Problem subtest and the Calculation subtest of the Woodcock–Johnson IV Tests of Achievement. Moreover, the correlation between the cognitive diagnostic test and the Applied Problems subtest was higher than that between the cognitive diagnostic test and the Calculation subtest. The results indicated that the formal cognitive diagnostic test was a reliable instrument for assessing kindergarteners’ MPS in the domain of number and operations.

The development and influencing factors of Kindergarteners’ mathematics problem solving based on cognitive diagnosis assessment

In this study we investigated the development of 245 kindergarteners’ mathematics problem solving (MPS) at two time points within 7 months using a cognitive diagnostic test, which measured three cognitive components (mathematical knowledge and skills, semantic understanding, and quantitative reasoning) with eleven cognitive attributes. The effects of five influencing factors [language ability, mathematics application ability, calculation ability, approaches to learning (ATL) and socio-economic status (SES)] on the children’s MPS ability at the two time points were examined. The results indicated that the 245 children’s mastery of mathematical knowledge and skills was much better than their mastery of semantic understanding and quantitative reasoning. Furthermore, they achieved significant progress in set comparison and addition and subtraction within the range of natural numbers to ten (two of the three cognitive attributes of mathematical knowledge and skills), additive composition reasoning and one-to-many correspondence reasoning (two of the three cognitive attributes of quantitative reasoning). Only their mathematics application ability significantly predicted their mastery of the three cognitive components of MPS ability at the two-time points. Their language and calculation abilities predicted only their mastery of mathematical knowledge and skills at the two time points. Their ATL predicted their mastery of the three cognitive components of MPS only at the first time point, and their SES predicted only their mastery of semantic understanding 7 months later.

A Comparison of Differential Item Functioning Detection Methods in Cognitive Diagnostic Models.

As a class of discrete latent variable models, cognitive diagnostic models have been widely researched in education, psychology, and many other disciplines. Detecting and eliminating differential item functioning (DIF) items from cognitive diagnostic tests is of great importance for test fairness and validity. A Monte Carlo study with varying manipulated factors was carried out to investigate the performance of the Mantel-Haenszel (MH), logistic regression (LR), and Wald tests based on item-wise information, cross-product information, observed information, and sandwich-type covariance matrices (denoted by Wd, WXPD, WObs, and WSw, respectively) for DIF detection. The results showed that (1) the WXPD and LR methods had the best performance in controlling Type I error rates among the six methods investigated in this study and (2) under the uniform DIF condition, when the item quality was high or medium, the power of WXPD, WObs, and WSw was comparable with or superior to that of MH and LR, but when the item quality was low, WXPD, WObs, and WSw were less powerful than MH and LR. Under the non-uniform DIF condition, the power of WXPD, WObs, and WSw was comparable with or higher than that of LR.