Classical Test Theory Research Articles

Level of Understanding of Pre-School Teachers on Child Rights Fulfillment and Protection at Paud Unit in Gunung Talang Sub-District, Solok Regency, West Sumatra

This research is based on the unknown level of understanding of PAUD teachers about the fulfillment of children's rights and protection in the PAUD Unit of Gunung Talang Sub-district. Quantitative descriptive research with the method used is the survey method. The subjects of this study were PAUD teachers in Gunung Talang Sub-district, totaling people. The instrument in this study used a Yes-No choice test where the data analysis technique used Rasch Model analysis. The Rasch model uses the principle of probability on each available option which in classical test theory is preferred to the total score of the results of the test or questionnaire. The results showed that the level of understanding of PAUD teachers about the fulfillment of children's rights and protection in the PAUD Unit of Gunung Talang Subdistrict was in the "low" category of 51.31%, "medium" of 40.79%, and "high" of 7.9%. Based on the average value, which is 3.28%, the level of understanding of PAUD teachers about the fulfillment of the rights and protection of children in the PAUD Unit of Gunung Talang Sub-district, Solok District, West Sumatra is in the "low" category.

Psychometric properties of the chinese version of multidimensional experiential avoidance questionnaire-30

BackgroundExperiential avoidance represents the tendency to avoid negative internal experiences, which is a key concept in Acceptance and Commitment Therapy. However, existing measures of experiential avoidance (i.e., Acceptance and Action Questionnaire–II, AAQ-II) have some limitations. This study aims to assess the psychometric properties of the Chinese version of Multidimensional Experiential Avoidance Questionnaire-30 (MEAQ-30) and provide evidence for the reliability and validity of this new instrument.MethodsTwo questionnaire surveys were conducted. The first sample (N = 546) was analyzed using classical test theory (CTT), and the second sample (N = 511) was analyzed using multidimensional item response theory (MIRT).ResultsCTT supported the six-factor structure of MEAQ-30, indicating good internal consistency and measurement invariance across genders. Furthermore, the Chinese version of MEAQ-30 showed satisfactory convergent and discriminant validity. The incremental validity test showed that after controlling for the effects of neuroticism and AAQ-II, the Chinese version of MEAQ-30 could still significantly predict depression, anxiety, and stress. MIRT indicated that 30 items had good discrimination and difficulty, and the six subscales were sufficiently reliable across the continuum of experiential avoidance.ConclusionThe Chinese version of MEAQ-30 has good reliability and validity and is suitable for assessing experiential avoidance among Chinese populations.

Study on the applicability of the Brief Job Stress Questionnaire based on classical test theory and item response theory

Objective: To evaluate the quality and item characteristics of the Brief Job Stress Questionnaire (BJSQ) among employees in manufacturing and service industries. Methods: From December 2021 to December 2022, a total of 2077 employees from 4 manufacturing and service enterprises in Beijing were selected by the method of combining purpose sampling and convenience sampling. The Chinese version of BJSQ was used to carry out a survey on occupational stress from 57 items in 4 dimensions including job stress factors, stress response, social support factors and satisfaction. Classical test theory (CTT) was used to analyze the validity and reliability of the questionnaire. The Semejima hierarchical response model in item response theory (IRT) was used to analyze the differentiation a, difficulty coefficient b and information content of each item. Results: Among the 2077 subjects, the age M (Q(1), Q(3)) was 33 (28, 37) years old, and the working age M (Q(1), Q(3)) was 4 (2, 8) years. There were 723 (34.8%) people in manufacturing industry and 1354 (65.2%) people in service industry. Eleven common factors were extracted by exploratory factor analysis, with a cumulative variance contribution rate of 62.823%, the variance of the common factors of each entry ranging from 0.451 to 0.865, and the range of factor loading values from 0.413 to 0.825, with 5 items having cross-loadings. The results of the validation factor analysis showed that the model fit indexes of root means square error of approximation was 0.055, comparative fit index was 0.950, Tucker Lewis index was 0.948, and standardized root mean square was 0.066. Content validity results showed that the total scores of the Chinese version of the BJSQ were positively correlated with the scores of the 4 dimensions (r(s)=0.487-0.936, P<0.05) . The results of the reliability analysis showed that the Cronbach's alpha coefficient for the total questionnaire was 0.945, and the Cronbach's alpha coefficients of job stress factors, stress response and social support factors were 0.775, 0.957, and 0.830, respectively. The Spearman-Brown coefficient for the total questionnaire was 0.866, and for the 3 dimensions, it was 0.572, 0.882 and 0.772. The results of IRT analysis showed that only 1 of the 57 items had a differentiation a-value<0.30, the difficulty coefficient b-value of each item ranged from -12.02 to 11.09, of which 8 items had a difficulty coefficient that was too high, 3 items had a difficulty coefficient that was too low, and 3 items did not meet the requirements, and the average amount of information in each item ranged from 0.022 to 2.566, and there were 47 items with average amount of information>0.088 (5/57) . Conclusion: The Chinese version of BJSQ has good reliability and validity in the typical occupational groups of manufacturing and service industries in China, and most of the items have good performance, but some items still need to be further improved and optimized or deleted.

The Patient Activation Measure-13 (PAM-13) in an oncology patient population: psychometric properties and dimensionality evaluation

BackgroundAccurate assessment and enhancement of health-related skills among oncology patients are pivotal for optimizing cancer care. The Patient Activation Measure (PAM-13), a questionnaire designed to reflect an individual’s knowledge, skills, and confidence in self-healthcare management, has been validated across diverse countries and settings. Concerns have been raised regarding the cross-situational applicability, as patients with specific diseases and cultural backgrounds interpret questionnaire items differently. This study aimed to examine the structural validity and psychometric properties of the PAM-13 in an oncological patient cohort.MethodsBaseline data from a longitudinal non-randomized controlled study involving cancer out-patients (n = 1,125) from Comprehensive Cancer Centres in Southern Germany were analysed. The German version of the PAM-13 was employed. With classical test and item response theory methods data quality, reliability, convergent and structural validity, as well as psychometric properties were assessed. Exploratory (EFA) and confirmatory factor analyses (CFA) were employed to investigate the postulated unidimensionality of the underlying construct. With a partial credit model (PCM) we examined item fit, targeting, local independence and differential item functioning.ResultsParticipants were predominantly female (73.0%) with a breast cancer diagnosis (41.3%). While items were generally well-accepted, ceiling effects were observed and a high mean PAM-13 score (69.7, SD = 14.2) was noted, potentially compromising responsiveness to interventions. Reliability was adequate (Cronbach’s α = 0.81), person and item separation reliability were good to excellent (0.81 and 0.99, respectively). Explorations of the unidimensionality of the construct (EFA, CFA, PCM) yielded inconclusive results, hinting towards a two-factor solution. Item difficulty rankings deviated from the original. No differential item functioning was identified, and local independence was confirmed.ConclusionsWhile the PAM-13 serves as a valuable instrument for comprehending and promoting health-related skills in cancer patients, the identification of ceiling effects, disordered item-difficulty rankings, and inconclusive findings regarding unidimensionality contribute to the expanding body of evidence, emphasizing the dependency of PAM-13’s validity and reliability on distinctive characteristics within the population under investigation. Future research should prioritize refining or adding PAM-13 items to better capture the specific health-related challenges within diverse populations, paving the way for more effective patient engagement strategies in oncology.Trial registration numberDRKS00021779

Digital literacy scale: Validity and reliability study with the rasch model

AbstractThe study aimed to develop a Likert-type measurement tool (Digital Literacy Scale, DLS) to determine the digital literacy levels of secondary school students. The validity and reliability of the developed measurement tool were verified using the Rasch model. The Rasch Model can estimate missing data and allow for small study groups (Rasch, Rasch, Probabilistic models for some intelligence and attainment tests, Danish Institute for Educational Research, 1960). Additionally, this model can verify the expected pattern of the measurement tool by calibrating it across participants and items. The content validity of DLS was ensured by expert opinion, and the construct validity was ensured by using the Rach model. In the content validity study, a 25-item pool was created for the draft DLS, and 5 items were removed from the draft DLS at this stage. Construct validity studies were carried out with the remaining 20 items using the classical test theory and item response theory (Rasch model). Exploratory Factor Analysis (EFA) and first-level Confirmatory Factor Analysis (CFA) were used within the scope of classical test theory. Then some Rasch assumptions such as dimensionlessness, local independence, monotonicity, and bias were tested for DLS. In the validity and reliability analysis of DLS, no items were eliminated, and the quantitative theoretical results were statistically confirmed. The analysis results showed that the minimum statistical values required for a good measurement tool were met. Accordingly, 20 valid and reliable compatible items that can be used to determine the digital literacy status of secondary school students were produced.

Afghanistan National Depression Screening (ANDs) scale: development and psychometric testing

BackgroundDepression is a complex mental health disorder marked by persistent sadness, hopelessness, and reduced interest in daily activities. The present study developed the Afghanistan National Depression Screening (ANDs) scale and examined its psychometric properties within the Afghan population.MethodsThe study enrolled 1245 Afghan individuals as participants. The items within the ANDs scale were developed through a thorough examination of pre-existing depression scales, expert assessments, and participant interviews. Multiple psychometric evaluations were administered to ensure the reliability and validity of the scale.ResultsAfter review and corrected item-total correlation testing, 15-items exhibiting acceptable corrected item-total correlation coefficients (ranging from 0.315 to 0.647) were retained and further validated through significant factor loadings (ranging from 0.358 to 0.725). Additionally, various properties assessed through classical test theory demonstrated satisfactory results on the 15-item scale. Specifically, measures of reliability such as internal consistency (α = 0.846) and Spearman-Brown coefficient (0.975) were deemed acceptable. Convergent validity was evidenced by strong correlations with established scales, including the DASS-21 (depression subscale, r = 0.854), GHQ-28 (depression subscale, r = 0.693), CES-D 20 (r = 0.922), and PHQ-9 (r = 0.758).ConclusionThe 15-item ANDs scale is developed to assess depression in Afghan populations, showing promise in correlation with established measures like PHQ-9, DASS-21, CESD-20, and GHQ-28. Its potential for identifying depression symptoms warrants further investigation in epidemiological studies and clinical settings, though replication across diverse samples is necessary for generalizability.

Is it True That My Students don't Understand the Static Fluid Concepts? Rasch Modeling Perspective

Students' conceptual understanding can be analysed using the Classical Test Theory (CTT) and Item Response Theory (IRT) approaches. CTT can only explain students' conceptual understanding at the group level, while IRT can at the individual level. Therefore, this research evaluates students' conceptual understanding using the Rasch model. This research used a survey research type and was carried out in class XI. Students' conceptual understanding was collected using 10 multiple-choice questions on static fluid material. The level of concept understanding was analyzed in stages using Rasch modelling. Understanding analysis begins by analyzing the difficulty level of the questions and then continues with descriptive analysis. The Wright map is used to see the hierarchy between students' understanding of concepts and the difficulty level of the questions used. Finally, a person diagnostic map will be used to see students' understanding of concepts in detail. The analysis results show that students' conceptual understanding is generally in the high and medium categories. Person diagnostic maps have identified patterns of correct and incorrect answers from students that do not reflect their true abilities. So, it is necessary to analyze the exact location of the weaknesses and strengths of students' conceptual understanding. This has implications for the suitability of the learning plan that the teacher will use.

Technical Adequacy of the Data-Based Instruction Knowledge and Skills Assessment in Writing

Technical Adequacy of the Data-Based Instruction Knowledge and Skills Assessment in Writing

Psychometric properties of the Beliefs About Adults with ID Scale in American physicians: Application of classical test and Rasch measurement theories.

Physicians' erroneous assumptions about individuals with intellectual disability (ID) negatively impact the quality of care provided to this population. This study aimed to investigate the psychometric properties of the Beliefs About Adults with ID (BAID), an instrument we developed for measuring physicians' erroneous assumptions about adults with ID. Two hundred ninety-two American physicians participated. Classical test theory and Rasch measurement theory were used to refine the scale (through item analysis, exploratory factor analysis, infit and outfit mean-squares statistics, and differential item functioning) and investigate its psychometric properties (functioning of the response scale, reliability, and validity). The BAID provided a unidimensional, reliable, valid, and precise measure in assessing high levels of erroneous assumptions. It showed convergent and divergent validity with the different factors of a scale measuring attitudes towards ID. The BAID items were discriminant, non-redundant, unambiguous, and invariant across gender and previous ID training. The BAID response scale was found to be appropriate for measuring physicians' erroneous assumptions about adults with ID. BAID is a brief instrument with good psychometric properties to assess erroneous assumptions about adults with ID in physicians of different genders and who have/have not previously received ID training. Therefore, it might be helpful for research and medical education purposes.

Item analysis on the quality of life scale for anxiety disorders QLICD-AD(V2.0) based on classical test theory and item response theory

BackgroundAnxiety disorders can cause serious physical and psychological damage, so many anxiety scales have been developed internationally to measure anxiety disorders, but due to the cultural differences and cultural dependence of quality of life between Chinese and Western cultures, it is difficult to reflect the main characteristics of Chinese patients. Therefore, we developed a scale suitable for Chinese patients with anxiety disorders: the Anxiety Disorders Scale of the Quality of Life Instruments for Chronic Diseases (QLICD-AD), hoping to achieve satisfactory QOL assessments for anxiety disorders.ObjectivesItems from the Anxiety Disorders Scale of the Quality of Life in Chronic Disease Instrument QLICD-AD system were analyzed using CTT and IRT to lay the groundwork for further refinement of the scale to accurately measure anxiety disorders.Methods120 patients with anxiety disorder were assessed using the QLICD-AD (V2.0). Descriptive statistics, variability method, correlation coefficient method, factor analysis and Cronbach’s coefficient of CTT, and graded response model (GRM) of item response theory were used to analyze the items of the scale.ResultCTT analysis showed that the standard deviation of each item was between 0.928 and 1.466; Pearson correlation coefficients of item-to-domain were generally greater than 0.5 and also greater than that of item-to-other domain; the Cronbach ‘s of the total scale was 0.931, α of each domain was between 0.706 and 0.865. IRT analysis showed that the discrimination was between 1.14 and 1.44. The difficulty parameter of all items increased with the increase of grade. But some items (GPH6,GPH8,GPS3,GSO2-GSO4,AD2,AD5) difficulty parameters were less than 4 or greater than 4. The average of information amount was between 0.022 and 0.910.ConclusionBased on CTT and IRT analysis, most items of the QLICD-AD (V2.0) scale have good performance and good differentiation, but a few items still need further revision. Suggests that the QLICD-AD (V2.0) appears to be a valid measure of anxiety disorders. It may effectively improve the diagnosticity of anxiety disorders, but due to the limitations of the current sample, further validation is needed in a broader population extrapolation trial.

Construct validity of the Braden scale in acute- and long-term care settings in Austria: A structural equation modeling analysis

Objective: The Braden scale is frequently used to assess pressure ulcer risk in health care settings. Selected psychometric properties have been tested using various methods of classical test theory in international studies. However, limited information on construct validity is available. Aim was to determine if the Braden subscale items correlate with the construct pressure ulcer risk and whether the construct validity concerning the factor structure of the Braden scale is adequate in acute and long-term settings.Methods: A quantitative design with secondary analysis of data from one acute (n = 328) and eight long-term care facilities (n = 311) in Austria was used to test construct validity. Data analysis included principal axis factor analysis with Promax rotation and assessment of internal consistency, followed by structural equation modeling.Results: For the acute care setting, a structure equation model with two latent factors and for the long-term care setting with one latent factor was tested according to principal axis factoring results. The Braden subscale items correlated with the construct pressure ulcer risk. Almost all examined model fit indices were within recommended reference values. Thus, the construct validity of the Braden scale was adequate in both settings.Conclusions: The factor structure in the acute care setting did not match that in the investigated long-term care setting. Further research regarding the construct validity of the Braden scale is therefore necessary.

The Implicit Association Test and its difficulty(ies): Introducing the test difficulty concept to increase the true-score variance and, consequently, the predictive power of implicit association tests.

We introduce the test difficulty concept from classical test theory to tackle the issue of low predictive power of implicit association tests (IATs). Following classical test theory, we argue that IATs of moderate difficulty (defined as mean IAT scores of zero) have more predictive power than IATs of extreme difficulties (defined as mean IAT scores deviating strongly from zero). Furthermore, we assume this relationship to be mediated by the true-score variance in IAT scores, with moderate difficulty resulting in more true-score variance. To test our hypotheses, we used nonexperimental (Studies 1 and 2) and experimental designs (Study 3). In Studies 1 and 2, we compared IATs of different test difficulties with regard to their ability to predict direct attitude measures, drawing on the Attitudes, Identities, and Individual Differences study. In Study 1, a subset of 95 attitude IATs (n = 127,259) was analyzed using multilevel structural equation models. As expected, IAT test difficulty strongly moderated the predictive power of IATs, and this effect was mediated by IAT true-score variance. In Study 2, we replicated the results with the same analyses but a different subset of 95 identity IATs (n = 43,745). In Study 3, we experimentally manipulated the IAT test difficulty. In total, three IATs (n = 480) were analyzed using multigroup structural equation models. Again, the IAT closer to moderate difficulty had more true-score variance and predictive power than the IATs of extreme difficulty. Accordingly, for correlational research, we recommend developing moderately difficult IATs to maximize IAT true-score variance and provide suggestions on how to achieve that. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Development and Validation of the Coronary Heart Disease Scale Among the System of Quality of Life Instruments for Chronic Diseases QLICD-CHD (V2.0) Based on Classical Test Theory and Generalizability Theory.

Coronary heart disease (CHD) is a common and frequent disease with a long and incurable course, and the quality of life of patients is severely reduced. This study was to develop and validate a quality of life scale for patients with CHD based on the Chinese context. The scale QLICD-CHD (V2.0) was developed based on the QLICD-CHD (V1.0), using a programmed decision procedures. Based on the data measuring QoL 3 times before and after treatments from 189 patients with CHD, the psychometric properties of the scale were evaluated with respect to validity, reliability and responsiveness employing correlation analysis, multi-trait scaling analysis, structural equation modeling, t-test and also G-study and D-study of generalizability theory analysis. The SF-36 scale was used as the criterion to evaluate the criterion-related validity. Paired t tests were conducted to evaluate the responsiveness on each domain/facet as well as the total of the scale, with Standardized Response Mean (SRM) being calculated. The QLICD-CHD (V2.0) has been developed with 42 items in 4 domains. The Cronbach's α of the general module, the specific module and the total scale were 0.91, 0.92 and 0.91 respectively. The overall score and the test-retest reliability coefficients in all domains are higher than 0.60, except for the specific module. Correlation and factor analysis confirmed good construct validity and criterion-related validity. After treatments, the overall score and score of all domains have statistically significant changes (P<0.01). The SRM of domain-level score ranges from 0.27 to 0.50. Generalizability Theory further confirm the reliability of the scale through more accurate variance component studies. The QLICD-CHD (V2.0) could be used as a useful instrument in assessing QoL for patients with CHD, with good psychometric properties.

Profiling Programming Language Learning

This paper documents a year-long experiment to “profile” the process of learning a programming language: gathering data to understand what makes a language hard to learn, and using that data to improve the learning process. We added interactive quizzes to The Rust Programming Language, the official textbook for learning Rust. Over 13 months, 62,526 readers answered questions 1,140,202 times. First, we analyze the trajectories of readers. We find that many readers drop-out of the book early when faced with difficult language concepts like Rust’s ownership types. Second, we use classical test theory and item response theory to analyze the characteristics of quiz questions. We find that better questions are more conceptual in nature, such as asking why a program does not compile vs. whether a program compiles. Third, we performed 12 interventions into the book to help readers with difficult questions. We find that on average, interventions improved quiz scores on the targeted questions by +20

A Psychometric Evaluation of the Dysphagia Handicap Index Using Rasch Analysis.

Background/Objectives: The Dysphagia Handicap Index (DHI) is commonly used in oropharyngeal dysphagia (OD) research as a self-report measure of functional health status and health-related quality of life. The DHI was developed and validated using classic test theory. The aim of this study was to use item response theory (Rasch analysis) to evaluate the psychometric properties of the DHI. Methods: Prospective, consecutive patient data were collected at dysphagia or otorhinolaryngology clinics. The sample included 256 adults (53.1% male; mean age 65.2) at risk of OD. The measure's response scale, person and item fit characteristics, differential item functioning, and dimensionality were evaluated. Results: The rating scale was ordered but showed a potential gap in the rating category labels for the overall measure. The overall person (0.91) and item (0.97) reliability was excellent. The overall measure reliably separated persons into at least three distinct groups (person separation index = 3.23) based on swallowing abilities, but the subscales showed inadequate separation. All infit mean squares were in the acceptable range except for the underfitting for item 22 (F). More misfitting was evident in the Z-Standard statistics. Differential item functioning results indicated good performance at an item level for the overall measure; however, contrary to expectation, an OD diagnosis presented only with marginal DIF. The dimensionality of the DHI showed two dimensions in contrast to the three dimensions suggested by the original authors. Conclusions: The DHI failed to reproduce the original three subscales. Caution is needed using the DHI subscales; only the DHI total score should be used. A redevelopment of the DHI is needed; however, given the complexities involved in addressing these issues, the development of a new measure that ensures good content validity may be preferred.

Baby Observational Selective Control AppRaisal (BabyOSCAR): Construct validity and test performance.

To investigate the construct validity of the Baby Observational Selective Control AppRaisal (BabyOSCAR), an assessment of independent joint motion in infants with cerebral palsy (CP). BabyOSCAR was scored for 75 infants (45 with CP and 30 without CP). Rasch analysis was used in combination with classical test theory to assess areas of strength or improvement. Overall fit and precision, unidimensionality, local independence, reliability indices, Wright's child-item map, and differential item functioning were examined as part of Rasch analysis to investigate the item properties, internal construct validity, and reliability of BabyOSCAR. Cronbach's α was used to evaluate items' internal consistency. Analysis demonstrated good fit to the Rasch model, with only one erratic item. Unidimensionality results suggest two dimensions, split between arm and leg items. Item calibration reliability was between 0.84 and 0.86, with three distinct item difficulty levels. Infant measure reliability was between 0.82 and 0.91, separating infants into three ability levels. Together, the two subscales covered the full range of skills, with redundancy mostly between the same motion on both sides of the body. Cronbach's α was between 0.90 and 0.95. BabyOSCAR's construct validity was supported. Arm and leg subscales can be translated to a logit scale. Baby Observational Selective Control AppRaisal (BabyOSCAR) has excellent construct validity with good overall fit and precision. Individual BabyOSCAR items contribute and work well together, forming an interval-level assessment. BabyOSCAR has two separate subscales, arms and legs, that complement each other. BabyOSCAR's items represent a continuum of skills with three distinct difficulty levels. BabyOSCAR's continuum of skills reliably separates infants into three ability levels.

Persistence of Conceptual Errors in First-Year University Physics Course and Its Possible Relationship with Learning Styles

In this work, we study the persistence of some fundamental previous ideas in physics in a group of freshmen at the Universidad Politécnica de Madrid (Spain). For this purpose, we analyze the answers to a questionnaire consisted of 24 multiple-choice items, most of them borrowed from the Force Concept Inventory (FCI). Our study is performed in two different ways by using, on the one hand, classical test theory and, on the other hand, the Pearson product–moment correlation. The survivance of some of the previous ideas at the end of the course is assessed by comparing and critically analyzing the answers of the students to the same test at the beginning and the end of the term. A possible connection with Honey–Alonso learning styles (LS) is also discussed. The results yielded by our study demonstrate the persistence of some of the initial and previous ideas, no matter the students’ previous qualifications or their current LS.

Cronbach’s Alpha: Genesis, Issues and Alternatives

The most widely used metric for assessing a scale’s reliability is Cronbach’s alpha. Since its introduction in 1951 by Lee Cronbach, it has evolved into the accepted benchmark for scale reliability. A quick search for ‘Cronbach’s alpha’ in Google Scholar yields more than 8 hundred thousand results. By any measure, this is enormous, and the usage of alpha appears to be present in practically every domain of academics. The fact that alpha has been so successful is surprising, as researchers have consistently criticised and have pointed out a plethora of problems with it. For instance, is that Cronbach never suggested alpha as a reliability metric—rather he proposed alpha as an alternative measure for equivalency in a test-retest context. Another issue Cronbach never suggested the lower bound of 0.70 as a benchmark of reliability. However, the lower bound of 0.70 has become the holy grail of scale reliability. Alpha continues to lead all measures of scale reliability despite its numerous issues. This article examines the history of alpha, its derivations based on classical test theory, and its limitations. It then suggests three alternative measures, along with software procedures to calculate them: alpha with confidence interval, omega and greatest lower bound. The purpose of this article is to enable researchers to have a better idea about the limitations of Cronbach’s alpha, and to make them aware of other measures of reliability which are available. It is hoped that this article will help researchers report better and more accurate reliability measures in their research works alongside the alpha.

Improving the ability to discriminate medical multiple-choice questions through the analysis of the competitive examination to assign residency positions in Spain

IntroductionPsychometrics plays a vital role in evaluating educational research, including the analysis of multiple-choice exams. This study aims to improve the discriminatory ability of the “Médico Interno Residente” (MIR) medical exam in Spain, used to rank candidates for specialized healthcare training, through psychometric analysis.MethodsWe analyzed 2,890 MIR exam questions from 2009 to 2021 (totaling 147,214 exams), categorizing them based on methodology and response type. Evaluation employed classical test theory and item response theory (IRT). Classical test theory determined difficulty and discrimination indices, while IRT assessed the relationship between knowledge levels and question performance.ResultsQuestion distribution varied across categories and years. Frequently addressed knowledge areas included various medical specialties. Non-image-associated clinical cases were the easiest, while case-based clinical questions exhibited the highest discriminatory capacity, differing significantly from image-based case or negative questions. High-quality questions without images had longer stems but shorter answer choices. Adding images reduced discriminatory power and question difficulty, with image-based questions being easier. Clinical cases with images had shorter stems and longer answer choices.ConclusionsFor improved exam performance, we recommend using a clinical case format followed by direct short-answer questions. Questions should be of low difficulty, providing clear and specific answers based on scientific evidence and avoiding ambiguity. Typical clinical cases with key characteristic features should be presented, excluding uncertain boundaries of medical knowledge. Questions should have lengthy stems and concise answer choices, minimizing speculation. If images are used, they should be typical, clear, consistent with the exam, and presented within clinical cases using clinical semiotics and propaedeutics.

Development and validation of the patient-reported outcome for older people living with HIV/AIDS in China (PROHIV-OLD)

BackgroundThe involvement of quality of life as the UNAIDS fourth 90 target to monitor the global HIV response highlighted the development of patient-reported outcome (PRO) measures to help address the holistic needs of people living with HIV/AIDS (PLWHA) beyond viral suppression. This study developed and tested preliminary measurement properties of a new patient-reported outcome (PROHIV-OLD) measure designed specifically to capture influences of HIV on patients aged 50 and older in China.MethodsNinety-three older people living with HIV/AIDS (PLWHA) were interviewed to solicit items and two rounds of patient cognitive interviews were conducted to modify the content and wording of the initial items. A validation study was then conducted to refine the initial instrument and evaluate measurement properties. Patients were recruited between February 2021 and November 2021, and followed six months later after the first investigation. Classical test theory (CTT) and item response theory (IRT) were used to select items using the baseline data. The follow-up data were used to evaluate the measurement properties of the final instrument.ResultsA total of 600 patients were recruited at the baseline. Of the 485 patients who completed the follow-up investigation, 483 were included in the validation sample. The final scale of PROHIV-OLD contained 25 items describing five dimensions (physical symptoms, mental status, illness perception, family relationship, and treatment). All the PROHIV-OLD dimensions had satisfactory reliability with Cronbach’s alpha coefficient, McDonald’s ω, and composite reliability of each dimension being all higher than 0.85. Most dimensions met the test-retest reliability standard except for the physical symptoms dimension (ICC = 0.64). Confirmatory factor analysis supported the structural validity of the final scale, and the model fit index satisfied the criterion. The correlations between dimensions of PROHIV-OLD and MOS-HIV met hypotheses in general. Significant differences on scores of the PROHIV-OLD were found between demographic and clinical subgroups, supporting known-groups validity.ConclusionsThe PROHIV-OLD was found to have good feasibility, reliability and validity for evaluating health outcome of Chinese older PLWHA. Other measurement properties such as responsiveness and interpretability will be further examined.