Cognitive Diagnostic Assessment Research Articles

An exploratory Q-matrix estimation method based on sparse non-negative matrix factorization.

Cognitive diagnostic assessment (CDA) is widely used because it can provide refined diagnostic information. The Q-matrix is the basis of CDA, and can be specified by domain experts or by data-driven estimation methods based on observed response data. The data-driven Q-matrix estimation methods have become a research hotspot because of their objectivity, accuracy, and low calibration cost. However, most of the existing data-driven methods require known prior knowledge, such as initial Q-matrix, partial q-vector, or the number of attributes. Under the G-DINA model, we propose to estimate the number of attributes and Q-matrix elements simultaneously without any prior knowledge by the sparse non-negative matrix factorization (SNMF) method, which has the advantage of high scalability and universality. Simulation studies are carried out to investigate the performance of the SNMF. The results under a wide variety of simulation conditions indicate that the SNMF has good performance in the accuracy of attribute number and Q-matrix elements estimation. In addition, a set of real data is taken as an example to illustrate its application. Finally, we discuss the limitations of the current study and directions for future research.

Asesmen Diagnostik Sebagai Penilaian Pembelajaran Dalam Kurikulum Merdeka di MIN 2 Boyolali

Diagnostic assessment is an assessment of learning assessment in the Merdeka Curriculum, which teachers conduct on students before the teacher designs learning. However, teachers' understanding of diagnostic assessment is still lacking, so it is challenging to compile diagnostic assessments. This study aims to describe diagnostic assessment as a learning assessment in the Merdeka Curriculum in elementary schools. The research method uses a literature study by collecting data from the results of exploring various literature in books and scientific articles from national and internationally reputable journals. The data analysis technique uses content analysis. The results obtained in this study indicate that there are two parts to the diagnostic assessment: the non-cognitive and cognitive diagnostic assessment. The non-cognitive diagnostic assessment aims to obtain information regarding family conditions, learning styles, characteristics, and students' learning interests. The cognitive diagnostic assessment aims to provide an overview of the initial abilities possessed by students in a subject topic. Teachers need to carry out diagnostic assessments of students to identify the abilities, strengths, and weaknesses so that teachers can design learning that is tailored to the abilities and characteristics of students.

Modeling Partial Knowledge in Multiple-Choice Cognitive Diagnostic Assessment

Diagnostic tests are typically administered in a multiple-choice (MC) format due to their advantages of objectivity and time efficiency. The MC-deterministic input, noisy “and” gate (DINA) family of models, a representative class of cognitive diagnostic models for MC items, efficiently and parsimoniously estimates the mastery profiles of examinees. However, the existing models often overestimate the latent traits of examinees when they respond with partial knowledge, which is often observed in educational assessment. Therefore, the novel models of the MC-DINA family that can appropriately handle such responses were developed in this study. Unlike the existing models, the proposed models placed no restrictions on the Q-vector, which represents attribute specifications. Simulation and empirical studies verified that the proposed approach could resolve the overestimation problem.

Cognitive diagnostic assessment: A Q-matrix constraint-based neural network method.

Cognitive diagnosis is a crucial element of intelligent education that aims to assess the proficiency of specific skills or traits in students at a refined level and provide insights into their strengths and weaknesses for personalized learning. Researchers have developed numerous cognitive diagnostic models. However, previous studies indicate that diagnostic accuracy can be significantly influenced by the appropriateness of the model and the sample size. Thus, designing a general model that can adapt to different assumptions and sample sizes remains a considerable challenge. Artificial neural networks have been proposed as a promising approach in some studies. In this paper, we propose a cognitive diagnosis model of a neural network constrained by a Q-matrix and named QNN. Specifically, we employ the Q-matrix to determine the connections between neurons and the width and depth of the neural network. Moreover, to reduce the human effort in the training algorithm, we designed a self-organizing map-based cognitive diagnosis training framework called SOM-NN, which enables the QNN to be trained unsupervised. Extensive experimental results on simulated and real datasets demonstrate that our approaches are effective in both accuracy and interpretability. Notably, under unsupervised conditions, our approach has significant advantages on small sample datasets with high levels of guessing and slipping, especially on the pattern-wise agreement rates. This work bridges the gap between psychometrics and machine learning and provides a realistic and implementable reference solution for classroom instructional assessment and the cold start of personalized and adaptive assessment systems.

Cognitive Diagnostic Assessment Instrument with Individual Feedback for Language Receptive Learning on Multimodal Text

Cognitive Diagnostic Assessment Instrument with Individual Feedback for Language Receptive Learning on Multimodal Text

Analisis Hasil Asesmen Diagnostiksebagai Dasar Pelaksanaan Pembelajaran Berdiferensiasi Di Sekolah Dasar

Students have different characteristics and a diversity of learning needs. Educators need to map students' learning needs according to students' learning abilities, interests and learning talents. Cognitive and non-cognitive diagnostic assessments make it easier for teachers to map students' learning needs through tests or expanded questionnaires. Diagnostic assessments are carried out with the aim of reviewing students' initial abilities in learning readiness. So it can be a solution in applying differentiated learning. The purpose of this observation is to analyze the number of diagnostic assessments as a benchmark for differentiated learning for Pancasila Education subjects in class IV. The research method used is descriptive qualitative. This data was collected using non-participatory observation from carrying out cognitive diagnostic assessment tests. Data analysis techniques use the Milles and Huberman models, including value reduction, data presentation and drawing conclusions. Research in class IV A. The results of the research show that educators have mapped groups based on students' learning readiness using cognitive diagnostic assessment tests. Groupings are made from developing, and advanced groups. Content differentiated learning can be applied to provide learning material based on proficiency level so that students' learning needs are met.

Using Regularized Methods to Validate Q-Matrix in Cognitive Diagnostic Assessment

One of the central components of cognitive diagnostic assessment is the Q-matrix, which is an essential loading indicator matrix and is typically constructed by subject matter experts. Nonetheless, to a large extent, the construction of Q-matrix remains a subjective process and might lead to misspecifications. Many researchers have recognized the importance of estimating or validating the Q-matrix, but most of them focus on the conditions of relatively large sample sizes. This article aims to explore Q-matrix validation possibilities under small sample conditions and uses regularized methods to validate the Q-matrix based on the compensatory reparametrized unified model and generalized deterministic inputs, noisy “and” gate models. Simulation studies were conducted to evaluate the viability of the modified least absolute shrinkage and selection operator (Lasso) and modified smoothly clipped absolute deviation (SCAD) methods, comparing them with existing methods. Results show that the modified Lasso and the modified SCAD methods outperform the stepwise, Hull, and MLR-B methods in general, especially under the conditions of small sample sizes. While good recovery in all small sample size conditions is not guaranteed, the modified methods demonstrate advantages across various item quality conditions. Also, a real data set is analyzed to illustrate the application of the modified methods.

Exploration of polytomous-attribute Q-matrix validation in cognitive diagnostic assessment

Compared with typical binary attributes, polytomous attributes can take three or more values (corresponding to different levels of mastery of a respondent or measurement of an item). They can indicate whether a respondent possesses the attributes of interest and mastery levels. Therefore, the test with polytomous-attribute Q-matrix can become more informative and provide respondents with richer diagnostic information than the test based on the dichotomous-attribute Q-matrix. This paper extends the S-statistic and the residual method applicable for the Q-matrix of binary attributes to validate the polytomous-attribute Q-matrix. Under two common scenarios in real-world applications, two associated validation algorithms: the joint validation (JV) algorithm and the online validation (OV) algorithm, are proposed. Both simulation studies and an empirical data example were employed to assess the robustness and usefulness of these two methods under various conditions. Results indicate that the JV algorithm is suitable for validating a Q-matrix predefined by subject matter experts. Especially when the Q-matrix contains fewer misspecifications, while the OV algorithm can be applied to define the attribute vector of "new items". Based on a certain number of "operational items", the OV algorithm can achieve a promising performance for obtaining the specification of the new items.

Analisis asesmen diagnostik kognitif materi geometri di kelas IV sekolah dasar

The study aims to describe the stages in the cognitive diagnostic assessment of geometry material in grade IV of elementary school which consists of the preparation, implementation and follow up stages. Through a cognitive diagnostic assessment, the teacher can identify the level of student learning readiness regarding geometry material. The research method used is descriptive analysis with a qualitative approach. Data collection techniques obtained through observation, interviews, and documentation. The research was conducted in 2 schools, namely at one of the SDN in Cihideung district and one of the SDN in Tawang district. Based on the research conducted, the results obtained regarding the cognitive diagnostic assessment of geometry material in grade IV of elementary school are as follows: (1) the preparation stage, the class teacher makes a grid of questions and teaching modules as a guide in making questions, (2) the implementation stage, the class teacher delivering geometry material before the test begins, (3) follow up stage, the class teacher provides follow up in the form of remedial and additional lessons. Thus the class teacher can find out the learning needs of students based on cognitive diagnostic assessment of geometry material.

Utilizing Cognitive Diagnostic Assessments to Identify and Address Student Needs in Differentiated Classrooms

Utilizing Cognitive Diagnostic Assessments to Identify and Address Student Needs in Differentiated Classrooms

Preferred Learning styles among Junior High School students Using non Cognitive Diagnostic Assessment

Every student possesses individual learning styles, making it crucial to examine their preferred approaches as a means of designing courses and implementing effective instructional methods. The aim of this research endeavor was to identify the predominant learning style among junior high school students, along with discerning any gender-related associations with specific learning styles. Employing a descriptive quantitative research methodology, data were collected from 97 participants via non-cognitive diagnostic assessments. The results revealed that the visual learning style emerged as the most favored choice, with a prevalence rate of 60.8%. It was followed by the auditory style (12.4%), kinesthetic style (9.3%), visual-auditory style (7.2%), visual-kinesthetic style (7.2%), and auditory-kinesthetic style (3.1%). In terms of gender distribution, a significant majority of male students exhibited a preference for the visual style (42.3%), while the visual style also predominated among female students (18.6%). These findings hold valuable implications for fostering pedagogical excellence. Based on these results, educators are strongly advised to embrace diverse teaching methods and design multifaceted activities that encompass and accommodate the different learning styles exhibited among students.

Research on English Teaching Reading Quality Assessment Based on Cognitive Diagnostic Assessment

Research on English Teaching Reading Quality Assessment Based on Cognitive Diagnostic Assessment

Q-matrix theory and its applications in cognitive diagnostic assessment

Q-matrix theory and its applications in cognitive diagnostic assessment

Cognitive diagnostic assessment based on signal detection theory: Modeling and application

Cognitive diagnostic assessment based on signal detection theory: Modeling and application

Analysis of Cognitive Diagnostic Assessment (CDA) in Differentiated Learning on English Subject Phase D

This article presents a meticulous examination of Cognitive Diagnostic Assessment (CDA) in the context of Phase D differentiated learning, with a specific emphasis on English subjects. The study investigates the practice of diagnostic assessment in differentiated learning in Phase D English diagnostic courses and evaluate the challenges teachers face in implementing cognitive diagnostic assessment in differentiated learning. Through a qualitative inquiry, the research administrated questioner and an interview by involving 3 (three) English teacher at the seventh-grade students as the techniques of collecting data, and further analyzed the data by conducting data coding, interpretation, and triangulation. The study concluded that the practice of CDA was based on their prior knowledge of the assessment, and their believe that cognitive diagnostic assessment needs to be conducted in the differentiated learning on English subject phase D. Further, the assessment is distributed by using rubrics and checklist diagnostic tools, and rely on observation method. The respondent believed that CDA strongly informs their differentiated instructional practice and see moderate impact on their instructional practice in conducting it, and they need support from the institution to strengthen their capability in conducting CDA. Related to the challenging of CDA, the most challenges aspects are in struggling with time limitations in implementing CDA, dealing with student reluctance or opposition, interpreting assessment outcomes, and training on cognitive diagnostic tools.

인지진단평가 영어 시험 적용에서 Q행렬 구축 방법과 후속 학습 효과 연구

In cognitive diagnostic modeling research, one area that has not had enough research interests is remedial learning or instruction based on the information provided by cognitive diagnostic assessment (CDA). The present study tries to address this research gap by exploring how effective subsequent self-directed learning will be after applying cognitive diagnostic assessment to diagnosing English grammar ability, which was not addressed in the existing CDA research. The significance of the present study is that it provides an in-depth self-study guide to investigate its effectiveness. To this end, the study administers a diagnostic test and provides detailed descriptions of their mastery levels of grammar for a small group of students who will follow through with the guided self-regulated learning. The post-test reveals that the guided self-study was notably effective with the improved total score and heightened skill mastery probabilities. The current research also examines the effectiveness of distractor analysis in optimally modifying Q-matrix for CDA. The result indicates the usefulness of distractor analysis for Q-matrix refinement when the response data are available. The article ends with some suggestions for future research based on the implications of the findings and limitations of the study.

베이지안 방식의 데이터 증강을 이용한 지도학습 기반 인지진단평가 연구

Objectives The purposes of this study were to propose and validate a method for applying supervised learning for cognitive diagnostic assessment by augmenting expert-labeled dataset using Bayesian parameter estimation techniques. Methods The method was evaluated with a simulation study and an application to real data. The simulation study was conducted to compare the diagnostic accuracy of neural networks trained with both ‘Bayesian-based aug-mented dataset’ and ‘expert-labeled dataset’ across four factors: ‘item quality’, ‘expert labeling error rate’, ‘size of the expert labeled dataset’, and ‘size of the augmented dataset’. Additionally, the real data consists of the binary responses of 936 middle-school students to 24 arithmetic problems on linear equations and linear inequalities. Results The performance of supervised learning trained on the ‘Bayesian-based augmented dataset’ showed significant improvements in all conditions compared to solely using the ‘expert-labeled dataset’. The effect of the labeling error rate and the size of the expert-labeled dataset was trivial. As the quality of the items increased and the size of the augmented dataset grew, accuracy improved, though beyond a certain size N=3,000 of the aug-mented dataset, the difference was negligible. For the real data on linear equations and inequalities, supervised learning trained on the ‘Bayesian-based augmented dataset’ showed enhanced accuracy over just on the ‘ex-pert-labeled dataset’. It is also evident that students understand solving linear equations better than solving linear inequalities. Conclusions When performing supervised learning based cognitive diagnosis with a small expert-labeled data-set, the diagnostic performance can be enhanced through Bayesian-based data augmentation method. Moreover, students tend to find understanding inequalities more challenging than equations, suggesting the need for opportunities to master cognitive attributes related to inequalities.

Development of learning path map of work and energy for high schoolers by using cognitive diagnostic assessment

The conception of work and energy is fundamental to learning physics and is essential to learning other subjects. However, most students still lack knowledge and understanding of work and energy. This may be due to previous research that aimed to develop students using similar teaching methods without considering the individual knowledge state of each student. We, thus, sought to develop the mastery test on work and energy and the learning path map of work and energy using cognitive diagnostic assessment. Participants were 537 tenth graders in Bangkok, Thailand, which were chosen by the multistage random sampling. The mastery test on work and energy developed is divided into six attributes, i.e., (1) work, (2) power, (3) kinetic energy, (4) gravitational potential energy, (5) elastic potential energy, and (6) law of conservation of energy. The test exhibited good psychometric properties, which were evaluated based on item parameters, content validity, construct validity, concurrent validity, classification consistency index, and classification accuracy index. The significant finding was the development of the learning path map of work and energy. The map illustrates students’ learning progression in different attribute profiles regarding work and energy. It proves to be highly beneficial for teachers in designing personalized learning methods for individual students. Additionally, it allows for tracking the learning progress of students until they have a comprehensive understanding of work and energy in all its attributes.

Another Reason for CDA in Language Assessment: A Critical Synthesis from a Perspective of Validating Test Constructs

Cognitive diagnostic assessment is known for its capacity to provide detailed feedback to test-takers, which can inform instruction and guide learning. Besides this core strength of the assessment approach, the measurement procedure also works as a construct validation method in itself. Previous studies do not call attention to this aspect of cognitive diagnostic assessment and how the CDA validation can be different from existing construct validation methods. Thus, the purpose of this article is to synthesize cognitive diagnostic assessment research from a standpoint of seeing it as a construct validation procedure with a focus on its strengths as such. In doing so, the article closely examines studies that applied two earlier cognitive diagnostic models to learner data because they established the research foundations of CDA in language testing. Before looking into the details of the procedure of cognitive diagnostic assessment, the paper critically reviews non-CDA studies on construct validation in language testing. It then scrutinizes the methods and objects of construct validation in CDA research to find any significant differences. The paper concludes with several avenues of future research in CDA, which are related to the matter of verifying test constructs.

Long short-term attentional neuro-cognitive diagnostic model for skill growth assessment in intelligent tutoring systems

Measuring student growth and providing diagnostic feedback are core components of cognitive diagnostic assessment. However, most current cognitive diagnostic models solely rely on data from a single occasion to diagnose student skill states, overlooking the substantial long-term information encapsulated in the learning history from multiple occasions. In this paper, we propose a long short-term attentional cognitive diagnostic (LS-ENCD) model for skill growth assessment in intelligent tutoring systems. Specifically, we first embed exercise and student features into high-dimensional vectors. Then, we use a measurement module with a bilayer architecture to establish the interaction between students and exercises, considering guessing and slipping factors. To capture long short-term dependencies on historical data, we design the long short-term learning transfer module based on the attention mechanism, which computes state transfer weights by incorporating occasion time and mastery state. Finally, extensive experimental results on four public datasets demonstrate the superiority and good interpretability of our proposed model.