Item Difficulty Research Articles

Fostering Expertise: Designing and Standardizing a Knowledge Test for Agricultural Scientists

The study adheres to a methodology for creating and standardizing test items designed to assess the knowledge of Agricultural scientists. Initially, 75 test items were identified through literature review. Subsequently, through consultation with experts, a refined set of 60 items emerged after screening, enhancement, and editing processes. These items underwent item analysis, including item difficulty index and discrimination index based on ratings from a panel of 60 experts. Thirty-four test items were retained for the standardization phase from the initial 60 items. To assess the validity of the test items, point biserial correlation was conducted, resulting in the final selection of 30 items for the knowledge test. The reliability of the test items was evaluated using the Spearman Brown split-half coefficient, yielding a value of 0.911, indicating high reliability and fairness for the study. Consequently, the standardized items proved the suitability for measuring the knowledge of Agricultural scientists.

Development of a measure for assessing malingered incompetency in criminal proceedings: Denney competency related test (D-CRT)

ABSTRACT Introduction Experts frequently assess competency in criminal settings where the rate of feigning cognitive deficit is demonstrably elevated. We describe the construction and validation of the Denney Competency Related Test (D-CRT) to assess feigned incompetency of defendants in the criminal adjudicative setting. It was expected the D-CRT would prove effective at identifying feigned incompetence based on its two alternative, forced-choice and performance curve characteristics. Method Development and validation of the D-CRT occurred in described phases. Items were developed to measure competency based upon expert review. Item analysis and adjustments were completed with 304 young teenage volunteers to obtain a proper spread of item difficulty in preparation for eventual performance curve analysis (PCA). Test-retest reliability was assessed with 44 adult community volunteers. Validation included an analog simulation design with 101 jail detainees using MacArthur Competency Assessment Test-Criminal Adjudication and Word Memory Test as criterion measures. Effects of racial/ethnic demographic differences were examined in a separate study of 208 undergraduate volunteers. D-CRT specificity was identified with 46 elderly clinic referrals diagnosed with mild cognitive impairment and dementia. Results Item development, adjustment, and repeat analysis resulted in item probabilities evenly spread from .28 to 1.0. Test-retest correlation was good (.83). Internal consistency of items was excellent (KR-20 > .91). D-CRT demonstrated convergent validity in regard to measuring competency related information and as well as malingering. The test successfully differentiated between jail inmates asked to perforfm their best and inmates asked to simulate incompetency (AUC = .945). There were no statistically significant differences found in performance across racial/ethnic backgrounds. D-CRT specificity remained excellent among elderly clinic referrals with significant cognitive compromise at the recommended total score cutoff. Conclusions D-CRT is an effective measure of feigned criminal incompetency in the context of potential cognitive deficiency, and PCA is assistive in the determination. Additional validation using knowns groups designs with various mental health-related conditions are needed.

Psychometric Properties of the Persian Translation of the 12-Item Utah Photophobia Symptom Impact Scale Questionnaire

ABSTRACT To translate the the Utah Photophobia Symptom Impact Scale-12 questionnaire into Persian and assess the psychometric aspects to check its validity and reliability based on the Rasch modelling method. Translation and cultural adjustment of the English language UPSIS-12 questionnaire to Persian was undertaken. A total of 61 patients with complaints of photophobia participated in evaluating validity and reliability aspects. All the participants were asked to complete the Persian translation of the UPSIS-12 questionnaire. Rasch analyses of the survey items were conducted using WINSTEPS. All items fit the Rasch model. Point-measure correlation values varied from 0.41 to 0.77, providing a preliminary indication of adequate construct validity. All factor loadings were found more than 0.4. All items obtained infit and outfit mean square (MnSq) values of < 2.0. All participants except 5 had normal outfit values. Patients’ abilities relative to the items’ difficulty were analysed. Item difficulty was estimated and item characteristic curves were included. Sufficient unidimensionality, hierarchical order, and equal interval scoring were obtained. In conclusion, the Persian UPSIS-10 questionnaire has excellent psychometric properties and it will be valuable in both clinical practice and research. It will help Persian practitioners to assess their patients’ photophobia.

Psychometric evaluation of a novel tool for assessing gestational diabetes and hypertension care: knowledge, attitudes, and practices of midwives and nurses.

While standardized assessment of knowledge, attitudes, and practices (KAP) related to gestational diabetes and hypertension is possible with a valid tool, existing research remains limited. This prospective validation study aimed to develop and validate a novel tool to assess the KAP of midwives and obstetric nurses. We included 125 midwives and obstetric nurses who routinely care for patients with gestational diabetes and hypertension. The tool demonstrated good internal consistency (Cronbach's alpha): knowledge (0.729, 95% CI, 0.654-0.776), attitude (0.756, 95% CI, 0.690-0.814), and practices (0.925, 95% CI, 0.905-0.943). Difficulty indices (d) ranged from 0.38 to 0.99 (knowledge), 0.41 to 0.99 (attitudes), and 0.41 to 0.93 (practices), indicating appropriate item difficulty. Discrimination indices (D) confirmed items could differentiate between respondents with low and high knowledge levels (D range: 0.02-0.77 for knowledge, 0.06-0.64 for attitudes, 0.20-0.84 for practices). The robust psychometric properties of this tool support its use in future research on KAP related to diabetes and gestational hypertension management in midwives and nurses. This instrument has the potential to be valuable in various settings, including baseline assessment before educational programs or evaluation of learning outcomes after interventions.

한국판 정서·행동평가시스템(K·BASC-3), 교사보고형 검사-아동용(TRS-C)의 타당화 연구

The purpose of this study was to evaluate the validity of the Korean Behavior Assessment System for Children 3rd Edition, Teacher Rating Scales-Child Form (K·BASC-3, TRS-C). The 156-item K·BASC-3, TRS-C was administered to 318 teachers of children. Equating using common item non-equivalent group designs, a general norm was made for a total of 954. The generalized partial credit model based on item response theory analysis was used to conduct internal validity. Additionally, as a measure of external validity, concurrent validity was tested using the Teacher’s Report Form for Ages 6-18 (TRF 6-18). Results from the analysis indicated that items included in clinical subscale provided appropriate information at the ability levels higher than the mean. Also, the results revealed that the most item difficulty and discrimination were appropriate. The items that yielded less than desirable discrimination scores correspond to those measuring symptoms related to the DSM diagnostic categories or protective and risk factors for childhood emotional/behavioral disorders. The anxiety, hyperactivity, and attention problems subscales of the K·BASC-3, TRS-C demonstrated strong concurrent validity statistics with the related subscales of the TRF 6-18. Other relevant findings, practical implications regarding the use of the K·BASC-3, TRS-C, strengths and weaknesses of the current study and suggestions for future research are included in the discussion.

Concept and implementation of an adaptive digital hearing training system for cochlear implant aftercare

Within the context of an interdisciplinary research project, we created acutting-edge prototype of an adaptive digital auditory training system designed for cochlear implant (CI) users. By leveraging the evidence-centered design (ECD) framework, we integrated adynamic difficulty adjustment feature that tailors the experience to the unique performance capabilities of each individual user. The ECD provides aconceptual design framework suitable for complex assessments of competence and dynamic performance. In the first phase, the domain of hearing was first defined in the context of CI users. In the development phase the three core models of the ECD, the competence model, the evidence model, and the task model, were developed and implemented. In addition, an asset pool of sound and language files was created, which included comprehensive linguistic feature descriptions for calculating item difficulties. Based on the requirements described, an adaptive exercise generator, an AI service, and other components were implemented. This included the development of agame environment and adashboard for patient data management. The exercises' difficulty levels were determined based on various parameters (e.g., sound, word frequency and number of words, grammatical properties) in combination with defined task types and levels. An adaptive digital auditory training system can help to supervise and train CI patients in acontinuous, interactive process based on their individual needs. We see the ECD as an effective way to build auser-based adaptive system.

Language Assessment at a Thai University: A CEFR-Based Test of English Proficiency Development

The increasing popularity of the Common European Framework of Reference (CEFR) in non-native English-speaking countries has generated a demand for concrete examples in the creation of CEFR-based tests that assess the four main English skills. In response, this research endeavors to provide insight into the development and validation of a CEFR-based test aimed at evaluating undergraduate students’ English proficiency for placement tests and exit exams. The CEFR served as the framework for item development while Classical Test Theory informed the test evaluation process. A sample of 2,248 first-year students participated in Testing 1 and 3,655 first- and second-year students took part in Testing 2. The results of the analysis of the multiple-choice listening and reading tests indicated favorable levels of item difficulty and discrimination indices, as well as high reliability coefficients obtained from Cronbach’s alpha, Kuder-Richardson, and split-half reliability. The correlation and regression analyses revealed close relationships between the subtests and between each subtest and the total score, supporting the test’s criterion validity. The study also demonstrated significant predictive validity on TOEIC scores. The findings of this study offer implications for the development of university-level English proficiency tests that integrate CEFR levels and CTT analysis.

How well does the Index of Race-Related Stress-brief function in a trauma-exposed Black community sample? An item response theory examination of discrimination and difficulty parameters.

Race-related stress negatively impacts the mental health of Black Americans to a greater degree than other racialized groups. Additionally, trauma exposure is associated with more severe levels of posttraumatic stress disorder for individuals who also experience race-related stress. Therefore, an accurate assessment of race-related stress in a trauma-exposed sample of Black Americans is necessary to ensure the validity and reliability of empirical findings regarding race-related stress and intervention efficacy. The Index of Race-Related Stress (IRRS) is one of the most commonly used measures to assess race-related stress among Black Americans. Due to a lack of psychometric support for the abbreviated version of IRRS-brief (IRRS-B) on a trauma-exposed sample of Black Americans, our study aims to address this gap in literature. We used item response theory (IRT) to assess item difficulty, discrimination, and factor structure in a sample of trauma-exposed Black Americans (n = 226). We employed a multidimensional graded response model with corresponding items loaded on to the three previously established factors of the IRRS. The most discriminating items asked about observing harsh treatment of Black individuals, experiencing less courtesy in establishments, and being stared at as though you do not belong. The item with the lowest difficulty described negative media representation of Black individuals while the item with the highest difficulty described lack of positive media portrayals of Black Americans. These results indicate that items varied considerably in the degree to which they adequately captured race-related stress. Future research should use IRT with newly worded questions to further improve the assessment of race-related stress in Black Americans. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Assessing the Performance of a Nutrition Knowledge Questionnaire With a Low Socioeconomic Status Population Using Rasch Analysis.

Parents and preschool teachers play a key role in shaping children's dietary behaviors. Knowledge of nutrition and healthy dietary choices is a key component to improve dietary habits and reduce the prevalence of obesity and associated co-morbidities. Using valid and reliable instruments is necessary for accurate assessment of knowledge to tailor interventions and measure effectiveness specific to the population of interest. The objectives of this paper are to (1) identify potential gaps in the baseline nutrition knowledge among parents and teachers using a previously validated questionnaire prior to a preschool obesity prevention intervention; and (2) assess the instrument's reliability and construct validity for a low socioeconomic status population using a post hoc Rasch analysis. Participants included 177 parents and 75 teachers who participated in a Head Start intervention study. Knowledge scores, instrument reliability, and item fit and difficulty were assessed using a Rasch analysis; t-tests were used to determine differences in scores between parents and teachers. Parents answered 38% of questions correctly while teachers correctly answered 46% of the questions. Adequate item fit and reliability were indicated for Sections 1 and 2 of the Nutrition Knowledge Questionnaire (NKQ). Section 3 demonstrated less adequate reliability. The items were found to adequately and reliably define the unidimensional measures of the three components of knowledge represented in this instrument, providing evidence of construct validity. However, Rasch measures indicated the NKQ overall was difficult for participants. Recommendations for improving the instrument for nutrition education/intervention and research practice areas related to obesity and obesity-related conditions are addressed.

Measurement properties of the Body Awareness Scale Movement Quality (BAS MQ) in persons on the autism spectrum: A preliminary Rasch analysis

BackgroundPersons on the autism spectrum exhibit poorer body awareness than neurotypical persons. Since movement quality may be regarded as an expression of body awareness, assessment of movement quality is important. Sound assessments of measurement properties are essential if reliable decisions about body awareness interventions for persons on the autism spectrum are to be made, but there is insufficient research. ObjectiveTo assess measurement properties of the Body Awareness Scale Movement Quality (BAS MQ) in an autism and a neurotypical reference group. MethodsPersons on the autism spectrum (n=108) and neurotypical references (n=32) were included. All were assessed with BAS MQ. Data were analyzed according to the Rasch model. ResultsBAS MQ was found to have acceptable unidimensionality, supported by the fit statistics. The hierarchical ordering showed that coordination ability was the most difficult, followed by stability and relating. Response category functioning worked as intended for 19 out of 23 items. There were few difficult items, which decreased targeting. Reliability measures were good. BAS MQ discriminated between the autism and the reference groups, with the autism group exhibiting poorer movement quality, reflecting clinical observations and previous research. ConclusionsBAS MQ was found to have acceptable measurement properties, though suffering from problems with targeting item difficulty to person ability for persons on the autism spectrum. The BAS MQ may, along with experienced movement quality, contribute to clinically relevant information of persons on the autism spectrum, although we encourage refinements and further analyses to improve its measurement properties.

Effect of changing multiple choice questions from “all of the above” to “select all that apply”

Introduction: The purpose of this study was to describe the effect of converting multiple choice questions (MCQs) that include an “all of the above” (AOTA) answer option to a “select all that apply” (SATA) question type on question performance.Methods: A summative assessment at the end of the first professional pharmacy year was comprised of approximately 50 multiple choice questions covering material from all courses taught. Eight questions contained AOTA answer options and were converted to SATA items in the subsequent year by eliminating the AOTA option and including the words “select all that apply” in the stem. Majority of the other questions included on the exam remained the same between the two years. Item difficulty, item discrimination, point biserial, and distractor efficiency were used to compare the MCQs on exams in the two years.Results: The AOTA questions were significantly easier and less discriminating than the SATA items. The performance of the remaining questions on the exam did not differ between the years. The distractor efficiency increased significantly when the questions were converted to SATA items.Conclusions: MCQs with AOTA answer options are discouraged due to poor item construction resulting in poor discrimination between high and low performing students. The AOTA questions are easily converted to the SATA format. The result of this conversion is a more difficult and more discriminating question with all answer options chosen, which prevents students from easily guessing the correct answer.

Evaluating Relapse Knowledge in People with Multiple Sclerosis: A Cross-Sectional Study on the Development and Validation of the Relapse Knowledge Questionnaire

BackgroundMultiple sclerosis (MS) knowledge is a prerequisite for active patient engagement in medical decision-making. Treatment of relapses in MS is a clinical field with many uncertainties and each acute relapse requires decisions regarding possible options for action, indicating the need for patient involvement. However, there is no validated instrument assessing relapse knowledge in people with MS. Our study aims to develop a valid MS relapse questionnaire for use as an outcome instrument for educational interventions. MethodsA multidisciplinary panel developed the relapse knowledge questionnaire (RKQ) based on a previously developed questionnaire. We tested the RKQ on MS patients for comprehensibility, usability and acceptance in qualitative think-aloud interviews and conducted a cross-sectional quantitative online survey to validate the questionnaire. People with suspected or confirmed relapsing-remitting MS and a recent relapse experience were eligible for inclusion. We checked normal distribution of the RKQ score and determined the item difficulty. Construct validity was analysed using correlational analysis. ResultsThe final RKQ consists of 10 items. After minor changes of the RKQ during pre-testing (n = 2), pilot testing (n = 10) confirmed the usability and acceptance of the instrument. The subsequent validation study (n = 203) resulted in a mean item difficulty of 0.44, ranging from 0.18 to 0.83. Seven items were particularly difficult and answered incorrectly by more than 50 % of participants. Construct validity of the RKQ was satisfactory. The RKQ score correlated only weakly with participants’ degree of education (|rp|>0.1), years since diagnosis (|rp|>0.1), and the intention to receive corticosteroids (|rp|>0.1). ConclusionThis study indicates the validity of the RKQ and proposes that the RKQ is a suitable instrument to assess relapse knowledge in people with MS participating in educational interventions.

How can organizations measure the integration of environmental, social, and governance (ESG) criteria? Validation of an instrument using item response theory to capture workers' perception

AbstractEvaluation of corporate sustainability parameters has become part of organizational management. Item Response Theory (IRT) serves as a valuable approach for quantifying and exploring observable variables associated with these parameters. Therefore, this article aims to propose, apply, and validate an instrument utilizing IRT to measure environmental, social, and governance (ESG) impacts on corporate sustainability performance, using the perspective of workers. Workers were chosen as key stakeholders for the present study, as they were pointed by previous research as central to corporate sustainability decisions. The present paper addresses a literature gap by introducing, implementing, and validating a research instrument, along with a comparative scale, to assess the effects of integrating ESG criteria on corporate sustainability performance. Workers of two major Brazilian electricity sector companies were respondents for this instrument. Multigroup IRT method was used to assess item discrimination and difficulty levels, and to construct a scale reflecting workers' perceptions of integrating ESG criteria. The questionnaire consisted of 15 items, demonstrating Content Validity Coefficients (CVC) above .97, confirming the instrument's content effectiveness. Cronbach's Alpha and McDonald's Omega were greater than .94 for both companies. Questionnaire items demonstrated satisfactory discrimination ability and varying difficulty levels, resulting in a five‐level scale to gauge the impacts of ESG criteria. This research contributes through the introduction and validation of the research instrument, the latent trait assessment scale, and the evaluation of workers' perceptions regarding ESG criteria impacts on corporate sustainability performance. In conclusion, the proposed instrument showed robust psychometric properties, displaying favorable reliability and validity parameters. It effectively measures the impacts of ESG criteria, providing interpretable levels aligned with the unique characteristics of its constituent items

Item distribution, internal consistency and structural validity of the German language person-centred climate questionnaire - staff version (PCQ-G-S): a cross-sectional study

BackgroundPerson-centredness is considered as best practice for people living with dementia. A frequently used instrument to assess person-centredness of a care environment is the Person-centred Climate Questionnaire (PCQ). The questionnaire comprises of 14 items with the three subscales a climate of safety, a climate of everydayness and a climate of community.AimThe aim of the study is to describe the translation process of the English language Person-centred Climate Questionnaire (Staff version, Patient version, Family version) into German language (PCQ-G) and to evaluate the first psychometric properties of the German language Person-centred Climate Questionnaire– Staff version (PCQ-G-S).MethodsWe conducted a cross-sectional study. The three versions of the 14-item English PCQ were translated into German language (PCQ-G) based on the recommendations for cross-cultural adaption of measures. Item distribution, internal consistency and structural validity of the questionnaire were assessed among nursing home staff (PCQ-G-S). Item distribution was calculated using descriptive statistics. Structural validity was tested using principal component analysis (PCA), and internal consistency was assessed for the resulting subscales using Cronbach’s alpha. Data collection took place from May to September 2021.ResultsA total sample of 120 nurses was included in the data analysis. Nine out of 14 items of the PCQ-G-S demonstrated acceptable item difficulty, while five times showed a ceiling effect. The PCA analysis demonstrated a strong structural validity for a three-factor solution explaining 68.6% of the total variance. The three subscales demonstrated a good internal consistency with Cronbach’s alpha scores of 0.8 for each of the subscales.ConclusionThe analysis of the 14-item German version (PCQ-G-S) showed first evidence for a strong internal consistency and structural validity for evaluating staff perceptions of the person-centredness in German nursing homes. Based on this, further investigations for scale validity of the PCQ-G versions should be carried out.

Structural Validity of the Mini-Balance Evaluation Systems Test in Individuals With Spinocerebellar Ataxia: A Rasch Analysis Study

ObjectiveTo examine the structural validity of the Mini-Balance Evaluation Systems Test (Mini-BESTest) in individuals with spinocerebellar ataxia (SCA). DesignMethodological research on data gathered in a cross-sectional study. A Rasch analysis was conducted (partial credit model). SettingInpatients in a hospital rehabilitation setting. ParticipantsA pooled sample of patients with SCA (N=65 [total 110 data]; 23 women, 42 men; mean±SD age 63.1±9.9y). InterventionsNot applicable. Main Outcome MeasuresWe evaluated the Mini-BESTest's category structure, unidimensionality, and measurement accuracy (0: unable to perform or requiring help to 2: normal performance). ResultsThe Mini-BESTest rating scale fulfilled the category functioning criteria. The analysis of the standardized Rasch residuals showed the scale's unidimensionality, but there were 7 item pairs indicating local dependence. All of the items fit the underlying scale construct (dynamic balance), with the exception of item #1, "Sit to stand," which was an underfit. The Mini-BESTest demonstrated adequate reliability (person separation reliability=.87) and separated the patients into 5 strata. The item-difficulty measures ranged from −4.49 to 2.02 logits, and the person ability–item difficulty matching was very good (the mean of person ability=−.07 logits and the mean of item difficulty=.00). No floor or ceiling effects were detected. The keyform identified items with small (#11, "Walk with head turns, horizontal") and large (#3, "Stand on 1 leg") item thresholds. ConclusionsThe Mini-BESTest has a unidimensional balance assessment scale with good category structure and reliability even for individuals with SCA. However, it also has some inherent shortcomings such as fit statistics, local item dependencies, and item thresholds. The results obtained when the Mini-BESTest is administered to patients with cerebellar ataxia should, thus, be interpreted cautiously.

Design and Validation of a Virtual Reality Mental Rotation Test

Mental rotation, a common measure of spatial ability, has traditionally been assessed through paper-based instruments like the Mental Rotation Test (MRT) or the Purdue Spatial Visualization Test: Rotations (PSVT:R). The fact that these instruments present 3D shapes in a 2D format devoid of natural cues like shading and perspective likely limits their ability to accurately assess the fundamental skill of mentally rotating 3D shapes. In this paper, we describe the Virtual Reality Mental Rotation Assessment (VRMRA), a virtual reality-based mental rotation assessment derived from the Revised PSVT:R and MRT. The VRMRA reimagines traditional mental rotation assessments in a room-scale virtual environment and uses hand-tracking and elements of gamification in attempts to create an intuitive, engaging experience for test-takers. To validate the instrument, we compared response patterns in the VRMRA with patterns observed on the MRT and Revised PSVT:R. For the PSVT:R-type questions, items requiring a rotation around two axes were significantly harder than items requiring rotations around a single axis in the VRMRA, which is not the case in the Revised PSVT:R. For the MRT-type questions in the VRMRA, a moderate negative correlation was found between the degree of rotation in the X direction and item difficulty. While the problem of occlusion was reduced, features of the shapes and distractors accounted for 50.6% of the variance in item difficulty. Results suggest that the VRMRA is likely a more accurate tool to assess mental rotation ability in comparison to traditional instruments which present the stimuli through 2D media. Our findings also point to potential problems with the fundamental designs of the Revised PSVT:R and MRT question formats.

Development and Rasch Analysis of the Prior Knowledge of Chemistry Concepts Test for Pre-medical Students in the Philippines

Chemistry concepts are embedded in several contexts that are related to the future professional roles of pre-medical students. In the current academic landscape of pre-medical programs, an objective measurement of prior knowledge of chemistry concepts is necessary to promote students’ academic success. To address this issue, this study aimed to develop and validate the Prior Knowledge of Chemistry Concepts Test. This multiple-choice research instrument is designed to measure pre-medical students’ knowledge of essential chemistry concepts from pre-requisite chemistry courses in pre-medical programs in the Philippines. The final item pool, consisting of 26 general chemistry concepts and 21 general organic chemistry concepts was administered to 470 college students from four tertiary institutions in the Philippines. Item content validity indices (I-CVI) and kappa statistics guided initial revisions, while dichotomous Rasch analysis was utilized to determine the psychometric properties of the research instrument. The analysis provided evidence of adequate person and item reliability, item fit, local independence, and unidimensionality, although some items may be omitted in subsequent iterations. The application of the research instrument to assessment and evaluation was elaborated using the student ability and item difficulty logits along the person-item map. Suggestions for the improvement of the research instrument were also discussed.

Breaking Ground in Skill Assessment: The Construction and Validation of a Problem-Solving Scale for University Students

Students today is expected to possess 21st-century skills, one of which is Problem Solving Skills (PSS). The importance of mastering this skill needs to be accompanied by the availability of a measuring tool. This research addresses this need by developing and validating the Problem-Solving Skill Scale. The development of this scale is focused on guidance and counseling students. The research and development of this instrument follow the educational design research method according to Plomp and Nieveen. Four hundred students were involved to test the construct validity. Problem-solving skills were measured based on three aspects: (1) Problem-Solving Confidence, (2) Approach-Avoidance Style, (3) Personal Control. Data were analyzed using the Rasch model by the WinSteps software. The research results describe the instrument's quality, the person-item interaction , the item difficulty level (item measures), fit and misfit items, and the distribution of respondents' abilities and the distribution of item levels on the same scale, and the scale’s ability to measure what it intends to measure. The Rasch analysis results show that the PSS Scale is good, precise, and has item fit with the model. The PSS Scale is a valid and reliable measuring tool for accurately measuring students' problem-solving skills. This research discusses implications and recommendations for further research efforts to facilitate the improvement of problem-solving skills in students using this measurement tool.

Item response theory to discriminate COVID-19 knowledge and attitudes among university students

The study sought to compare two-item response theory (IRT) models, the Rasch and 2PL models, and to uncover insights on COVID-19 knowledge and attitude item difficulty and discrimination among university students. We premise this study on ITM to argue that logical flow, degree of difficulty, and discrimination of items for the constructs among respondents contribute to the validity and quality of statistical inferences. The developed Rasch and 2PL models are compared to determine the difficulty and discrimination of knowledge and attitude items, with an application to COVID-19. Our results show that although the Rasch and 2PL models provide rich diagnostic tools to understand multiple traits, the 2PL model provides more robust results for the assessment of knowledge and attitude of students about the COVID-19 epidemic. Moreover, of the two constructs, the items for the attitude construct recieved more reliable responses than the knowledge construct items. Accordingly, under any pandemic, the lack of proper and evolving knowledge could have dire consequences; hence, strict efforts should be made while designing knowledge items.

State-Aware Deep Item Response Theory using student facial features.

This paper introduces a novel approach to Item Response Theory (IRT) by incorporating deep learning to analyze student facial expressions to enhance the prediction and understanding of student responses to test items. This research is based on the assertion that students' facial expressions offer crucial insights into their cognitive and affective states during testing, subsequently influencing their item responses. The proposed State-Aware Deep Item Response Theory (SAD-IRT) model introduces a new parameter, the student state parameter, which can be viewed as a relative subjective difficulty parameter. It is latent-regressed from students' facial features while solving test items using state-of-the-art deep learning techniques. In an experiment with 20 students, SAD-IRT boosted prediction performance in students' responses compared to prior models without the student state parameter, including standard IRT and its deep neural network implementation, while maintaining consistent predictions of student ability and item difficulty parameters. The research further illustrates the model's early prediction ability in predicting the student's response result before the student answered. This study holds substantial implications for educational assessment, laying the groundwork for more personalized and effective learning and assessment strategies that consider students' emotional and cognitive states.