Classical Test Theory Research Articles

Refinement and Validation of the Empowerment Audiology Questionnaire: Rasch Analysis and Traditional Psychometric Evaluation.

Empowerment is the process through which individuals with hearing-related challenges acquire and use knowledge, skills and strategies, and increase self-efficacy, participation, and control of their hearing health care, hearing solutions, and everyday lives. The aim was to refine and validate the Empowerment Audiology Questionnaire (EmpAQ), a hearing-specific measure of empowerment. This was achieved through (1) refinement via Rasch analysis (study 1), and (2) traditional psychometric analysis of the final survey structure (study 2). In study 1, 307 adult hearing aid owners completed the initial empowerment measure (33 items) online. To inform an intended item reduction, Rasch analysis was used to assess a range of psychometric properties for individual items. The psychometric properties included analysis of individual items (e.g., response dependency, fit to the polytomous Rasch model, threshold ordering) and the whole EmpAQ (e.g., dimensionality). Item reduction resulted in a 15-item version (EmpAQ-15) and a short-form 5-item version (EmpAQ-5), validated using modern (Rasch), and traditional (Classical Test Theory) psychometric analysis (study 2). In study 2, 178 adult hearing aid owners completed the EmpAQ-15 and EmpAQ-5, alongside 5 questionnaires to measure related constructs. These included two hearing-specific questionnaires (Social Participation Restrictions Questionnaire and Self-Assessment of Communication), two general health-related questionnaires (Patient Activation Measure and World Health Organization Disability Assessment Schedule 2.0), and a general empowerment questionnaire (Health Care Empowerment Questionnaire). Modern (Rasch) and traditional psychometric analysis techniques (internal consistency, construct validity, and criterion validity) were used to assess the psychometric properties of the EmpAQ-15 and EmpAQ-5. Rasch analysis of the initial 33-item measure of empowerment identified 18 items with high response dependency, poor fit to the Rasch model, and threshold disordering, which were removed, resulting in a long-form (EmpAQ-15) hearing-specific measure of empowerment. A short-form (EmpAQ-5) version was developed for use in the clinic setting. Validation of the two EmpAQ measures using Rasch analysis showed good item fit to the Rasch model, appropriate threshold targeting, and the existence of unidimensionality. Traditional psychometric evaluation showed that both questionnaires had high internal consistency and positive correlations with the hearing-specific questionnaires. However, in contrast with our hypotheses, correlations with general health questionnaires were stronger than with hearing-specific questionnaires; all questionnaires were correlated with the EmpAQ and in the direction hypothesized. Taken together, these findings support the construct validity of the EmpAQ-15 and EmpAQ-5. The EmpAQ-15 and EmpAQ-5 are the first self-report measures to be developed specifically for the measurement of empowerment. The EmpAQ-15 and EmpAQ-5 were found to meet the Rasch model criteria for interval-level measurements. Traditional psychometric evaluation supports the construct validity of both measures. The EmpAQ measures have the potential to be used in both research and clinical practice to evaluate empowerment along the hearing journey. The next stage of this research will be to further validate these measures by assessing their responsiveness, minimal clinically important difference, and clinical interpretability in a clinical population.

Further refinement of the Patient-Reported Impact of Dermatological Diseases (PRIDD) measure using classical test theory and item response theory.

Existing dermatology-specific Patient-Reported Outcome Measures (PROMs) do not fully capture the substantial physical, psychological and social impact of dermatological conditions on patients' lives and are not recommended for use according to the COSMIN criteria. Most were developed with insufficient patient involvement and relied on classical psychometric methods. We are developing the new Patient-Reported Impact of Dermatological Diseases (PRIDD) measure for use in research and clinical practice in partnership with patients. To examine the factor structure of PRIDD, determine the definitive selection of items for each subscale, and establish structural validity and internal consistency through classical and modern psychometric methods. Two cross-sectional online surveys were conducted. Adults (≥ 18 years) worldwide living with a dermatological condition were recruited through the membership network of the International Alliance of Dermatology Patient Organizations (GlobalSkin). They completed the PRIDD questionnaire and a demographics questionnaire via an online survey. We examined missing data and distribution of scores for each item. The factor structure was assessed using confirmatory and exploratory factor analysis (Survey 1). Internal consistency was examined using Cronbach's α. Rasch measurement theory analyses were conducted, including iterative assessment of rating scale function, fit to the Rasch model, unidimensionality, reliability, local dependence, targeting and differential item functioning (DIF) (Surveys 1 and 2). Participants in Surveys 1 and 2 numbered 483 and 504 people, respectively. All items had ≤ 3% missing scores and all five response options were used. A four-factor model showed the best fit. PRIDD and all four subscales were internally consistent but showed some misfit to the Rasch measurement model. Adjustments were made to rectify disordered thresholds, remove misfitting items, local dependency and DIF, and improve targeting. The resulting 16-item version and subscales fit the Rasch model, showed no local dependency or DIF at the test level, and were well targeted. This field test study produced the final PRIDD measure, consisting of 16 items across four domains. The data triangulated and refined the conceptual framework of impact and provide evidence of PRIDD's structural validity and internal consistency. The final step in the development and validation of the PRIDD measure is to test the remaining measurement properties.

Examining the Reliability of the International English Language Testing System Design

Background: Despite the evolution of the International English Language Testing System (IELTS) to accommodate the evolving needs of test-takers and institutions, concerns have been raised regarding its reliability in assessing English language proficiency. Purpose: This study aimed to investigate the reliability of the International English Language Testing System (IELTS) design as a measure of English language proficiency. Methodology: To explore this issue, the research reviewed relevant literature and employed various methods, including frequency analyses, data analytics, and machine learning algorithms based on mathematical principles. Unlike previous studies, this work analysed a dataset of 33,505 authentic band scores, allowing for a comprehensive examination of IELTS test score patterns. Findings: The findings revealed that the concept of reliability is complex, and the statistical methods used by IELTS test providers, based on classical test theory, may provide approximate rather than precise scores. Relying solely on the reputation and authority of the IELTS test is insufficient to establish its reliability, as it involves logical fallacies and arguments from authority. Research implications: It is necessary to conduct further independent research to ensure the IELTS test's reliability and address the concerns expressed by researchers and test-takers.

Measurement properties of Brief-BESTest scores from children, adolescents, and youth with visual impairments

Purpose: This study aimed to vet the measurement properties of Brief-BESTest scores in children, adolescents, and youth with visual impairment and blindness (YVI). Methods: A cross-sectional sample of YVI (N = 101) completed the Brief-BESTest, a modified version of the Y-Balance Test, the 360-degree turn test, bipedal quiet stance, and Activity-Specific Balance Confidence Scales. Thirty-seven YVI completed test-retest of the Brief-BESTest using a one-year interval. Using classical test theory, various forms of total and item-level Brief-BESTest score reliability and validity were investigated in YVI. Results: All inter-rater reliability coefficients were ≥ .80. When considering the eight items of the Brief-BESTest, 27 of the 28 possible correlations were statistically significant (p<.05). Various internal consistency and item difficulty results were strong. When taking total Brief-BESTest scores and their association with the complementary balance tasks/metrics into account, 11 of 13 associations were statistically significant (p<.05) providing strong convergent validity evidence. Being multimorbid and degree of vision significantly predicted total Brief-BESTest scores (p<.001) suggesting construct (i.e. known groups) validity. Numerous test-retest results (e.g. coefficients, limits of agreement) following the one-year interval were indicative of score stability. Conclusion: Practitioners and researchers should have confidence in, and consider adopting, the Brief-BESTest to examine multidimensional balance in YVI.

Severe Impairment Rating Scale (SIRS) in assessing patients with severe dementia in Chinese Community

AbstractBackgroundLike other advanced countries, Hong Kong is facing the challenges of an aging population, reduced fertility, and increasing longevity. Most older people who diagnosed with moderate to severe dementia would need institutional care. Nevertheless, the current instruments available in the Chinese population are having flooring effects in measuring the functional conditions for this specific group of population.MethodThis study aims to validate the 14‐item Chinese Severe Impairment Rating Scale (C‐SIRS), as an instrument designed to measure functional condition of older people with moderate and severe dementia. The development of this validation study was under classical test theory to examine and test (1) the degree of clarity, understandability, and relevance (i.e. content validity), (2) the test‐retest reliability score of SIRS, (3) the degree of the inter‐relatedness among SIRS items like internal consistency, and (4) the correlation between SIRS score and the functional measures by Hong Kong Montreal Cognitive Assessment (HK‐MoCA) and Modified Barthel Index 100 (MBI‐100).ResultWhen assessing the relevance and sensitivity of the question with the universal agreement calculation method, the scale‐level content validity index (S‐CVI/UA) of C‐SIRS received relevance ratings of “almost agree” or “absolutely agree” by all the experts. The Fleiss' kappa showed that there was moderate agreement between the panellists’ judgements, with at least 8 years of experience, κ = .326 (95% CI, .319 to .343), p &lt; .001. 105 older people with moderate to severe dementia recruited from nursing home were enrolled in the validation, the C‐SIRS had a high level of internal consistency, as determined by a Cronbach’s alpha of 0.99 with F = .005, p = .831. The SIRS shows significant correlation with both cognitive, as measured by MoCA (with r = .337, p &lt; .001), and functional measures as measured by MBI (with r = .64, p &lt; .01).ConclusionC‐SIRS is sensitive in measuring the functional condition in older people with moderate to severe dementia. Moreover, C‐SIRS is easier to administrate with shorter time required. Further study should help by investigating the sensitivity of C‐SIRS to assess the change over a longer period of time.

Conducting a Rasch Analysis in jMetrik

This step-by-step guide to conducting a Rasch analysis using the jMetrik software package describes how to format and import data, conduct a simple classical test theory item analysis, and then conduct a simple Rasch analysis. Some basic graphical outputs are also described. After completing the steps described in this guide, novice users should be able to conduct a basic analysis unaided and be able to explore the more advanced features of jMetrik by referring to the manual. Keywords: jMetrik, Rasch analysis, item analysis

Comparison of Test Equating Methods Based on Classical Test Theory and Item Response Theory in Polytomously Scored Tests

Aynı örtük özelliği ölçen testin benzer zorluklara sahip iki formunun puanlarının birbirine dönüştürülmesini içeren istatistiksel süreç test eşitleme olarak tanımlanır. Bu çalışma* çok kategorili puanlanan maddelerden oluşan test formlarının eşitlenmesi sürecinde Klasik Test Kuramı ve Madde Tepki Kuramı’na dayalı test eşitleme yöntemlerinden elde edilen hataların karşılaştırılmasını amaçlayan bir olasılıksal simülasyon çalışmasıdır. Yürütülen çalışmada örneklem büyüklüğü 50, 100, 300, 1000 ve 3000 kişi ve test uzunluğu 10, 20, 30, 40 ve 50 madde olmak üzere 2 değişimlenen koşul için 25 alt koşul belirlenmiştir. Bu alt koşullara uygun olarak 0 (yanlış), 1(kısmen doğru) ve 2 (tam doğru) şeklinde puanlanan genelleştirilmiş kısmi puan modeline uyumlu simülatif veriler kullanılmıştır. Çalışmada veri setlerini üretmek için WinGen3 programından faydalanılmıştır. Üretilen veri setlerinin eşitlenme süreci R programı ile yürütülmüştür. Bu çalışmada doğrusal, ortalama ve eşit yüzdelikli eşitleme yöntemleri olmak üzere üç Klasik Test Kuramı yöntemi (KTK) ve ortalama-ortalama ve ortalama-standart sapma olmak üzere iki Madde Tepki Kuramı (MTK) yöntemi kullanılmıştır. Üretilen test formları kullanılarak üç KTK yönteminden ve iki MTK yönteminden elde edilen eşitlemenin standart hataları hesaplanmış ve yöntemlerden elde edilen hata miktarları karşılaştırılmıştır. Araştırma sonucunda, farklı örneklem büyüklüğü ve farklı test uzunluğuna göre üretilen test formlarının eşitlenmesi sürecinde en az hata ile eşitleme yapan yöntemin eşit yüzdelikli eşitleme yöntemi ve en fazla hata ile eşitleme yapan yöntemin ortalama-ortalama yöntemi olduğu saptanmıştır. Çalışmanın sonucunun literatürle uyumlu olduğu görülmüştür. Test eşitleme yöntemleri örneklem uzunluğu bağlamında incelendiğinde örneklem büyüklüğü arttıkça test eşitleme hatasının azaldığı belirlenmiştir. Yöntemler, test uzunluğu bağlamında değerlendirildiğinde ise madde sayısı arttıkça hesaplanan hata miktarının da arttığı sonucuna ulaşılmıştır.

Use of Item Response Theory (IRT) in subjective assessment of concert halls

Item response theory (IRT) is considered an improvement of the classical theory of tests to determine the educational level using the learning skills of students. The analysis of these tests together with the knowledge of different elements of the social and familiar context, makes it possible to evaluate the educational level of countries, as well as the effect of contextual variables on learning skills levels. The IRT method considers that there is a latent unidimensional continuous variable whose determination makes it possible to establish quantitatively the abilities of individuals and the difficulties of the questions. In this paper, we present an adaptation of this methodology to the analysis of ratings to questions in subjective rating surveys of concert halls, for experts and general public. The latent dimension (LD), provided by the IRT method, has been determined for the surveys of different concerts in 9 concert halls of different types. For each concert hall, a map of the distribution of LD values and subjective rating responses is shown, and the effect of the different context variables is analyzed.

Analysis of Pre-Olympic Middle School Mathematics Test Instruments Based on Item Response Theory

Junior high school of mathematics Olympiad is one of the routine activities every year in Indonesia. Analysis of the items on the Mathematical Olympiad test instrument has been carried out by several researchers using the concept of classical test theory. The theory has many weaknesses when compared to the concept Item Response theory (IRT). The purpose of this article is to describe the analysis of pre-Olympiad mathematics test instrument for junior high school students in Pacitan East Java based on the IRT. This analysis was carried out using a quantitative descriptive method. IRT-based item analysis was performed using R software with the package irtawsi. The results of the analysis with the package show that the instruments used fit to 2PL model. From the 15 items used, there were two items that did not fit, namely items 4 and 9. Furthermore, from the 13 test items that were fit, there were 3 items that did not meet the quality of the discriminant parameter used. These three items are 1, 6 and 13 items, these three items must be dropped because they can provide biased information when used to estimate students' math olympiad abilities. Based on this information it can be concluded that there are 10 test items that have quality that meets the qualification parameters of difficulty and discriminant with a standard error of 0.25.

Symptom Burden in Idiopathic Multicentric Castleman Disease: Protocol for the Development of a Novel Standardized Patient Reported Outcome Measure

Background and Significance Idiopathic multicentric Castleman disease (iMCD) is a rare lymphoproliferative disorder with a high symptom burden. These symptoms impact many aspects of daily life for people living with iMCD, including work/education, social life, travel, mobility, personal relationships and sexual functioning. Despite this significant impact [Shupo F et al. Hemasphere 2022, Jun 23;6(Suppl):802-803], no validated disease-specific measure exists to assess symptom burden in iMCD. This is a major unmet need, considering the mainstay of iMCD treatment is symptom control and preventing serious complications. A novel symptom burden scale for iMCD is therefore timely and important and would be a beneficial clinical tool, both in routine clinical practice and clinical trials, for the comprehensive assessment of disease burden at diagnosis, during progression, and for treatment efficacy. Study Design and Methods A protocol for the development of a novel, standardized, patient reported outcome measure (PROM) for assessing symptom burden in iMCD has been developed and approved by a multi-stakeholder group (including patients, clinicians, industry representatives, and researchers). A four-stage development process has been proposed, alongside the generation of a novel patient advisory group and wider multi-stakeholder advisory group. In stage one, draft PROM content will be generated from existing literature and expert opinions. In stage two, the content validity of the draft PROM will be assessed in online qualitative interviews with people living with iMCD, with any revisions to the content decided in consultation with patient and expert advisors. In stage three, the revised PROM will be administered alongside existing measures of symptom burden and/or health-related quality of life to evaluate its psychometric performance and inform decisions on content for the final PROM, using classical test theory and/or Rasch psychometric analyses. The PROM will be finalized based on the qualitative and quantitative evidence generated in consultation with project advisors. Finally, in stage four, the PROM will be re-administered to observe change in symptom burden over time. This will be complemented with qualitative interviews in a mixed methods design to estimate a minimally clinically important difference (MCID) for the measure. The project will involve investigators and iMCD patients from the U.S., U.K., Canada, Australia, New Zealand and Brazil. Conclusion This is an international effort to develop and validate a novel iMCD symptom burden PROM through a collaboration of iMCD patients, clinical, health economics, public health, statistical, psychometrics, and industry experts. The development of the measure follows U.S. Food and Drug Administration (FDA) regulatory guidance, with modifications for rare diseases as required. Special Considerations This project has been submitted to clintrials.gov, documentation sent to ASH and Secretary, and awaiting approval of the submission.

Teknologi Calon Guru Sekolah Dasar Dalam Menggunakan Rasch Model

Knowledge of mathematics, didactics, and technology that must be owned by pre-service and in-service teachers, previously measured by the classical test theory (CTT), but has limitations. Study aims to determine that knowledge with a more accurate prediction method using the item response theory (IRT) from the Rasch model. Study used a survey of mathematical and didactic knowledge, and a survey of technological knowledge with a scale. The subjects were 38 second- year students from an elementary school teacher education institution in Pekanbaru. This study indicates that pre-service teachers have relatively sufficient mathematics and technological knowledge, but they were lack of didactic knowledge. Analysis of pre-service teachers’ knowledge using the Rasch model can provide an overview of the difficulty level of the items given, the level of pre-service teachers’ knowledge, and the interaction between respondents and between questions. Institutions need to develop learning instruction for pre-service teachers that can support the strengthening and development of their mathematical, didactic, and technological knowledge.

Initial validation of the powered mobility device autonomy residential screen (PoMoDARS)

Background The Powered Mobility Device Autonomy Residential Screen (PoMoDARS) is a new tool to enable clinicians to screen resident capacity and performance skills for powered mobility device (PMD) use in residential aged care settings. The PoMoDARS is context specific, time efficient and promotes resident autonomy and safety. Aims To (i) undertake initial face and content validation of the PoMoDARS, and (ii) use the research findings to make any modifications. Methods A mixed-methods study design, underpinned by Classical Test Theory. Eight clinicians completed 20 PoMoDARS screens and provided both quantitative and qualitative feedback on item importance and ease of use within a formal interview. Results Initial face and content validity of the PoMoDARS were supported, with small modifications made to item descriptors and instructions. Conclusions The PoMoDARS has been developed for use in residential aged care settings to screen resident PMD use. While initial validation has been undertaken, further studies to determine the reliability of the tool and continue the validation process are required. Significance Older adults in residential aged care facilities benefit greatly from the autonomy gained through PMD use. The PoMoDARS promotes collaboration between occupational therapists, nurses, and the wider team to support residents and safe PMD use.

Evaluation of Validity and Reliability of a German General and Sports Nutrition Knowledge Questionnaire for Athletes and Coaches (GSNKQ-AC).

Despite the availability of nutritional recommendations, studies have reported inadequate nutrition in athletes. The existing literature highlights the importance of the nutritional knowledge of both athletes and coaches in influencing athletes' food choices and behavior, as well as its direct and indirect impact on athletes' performance and health. To adequately assess nutritional knowledge, monitoring via valid and reliable questionnaires is required. As no questionnaire tailored to German-speaking athletes and coaches exists, this study aimed at developing a new General and Sports Nutritional Knowledge Questionnaire for Athletes and Coaches (GSNKQ-AC). The development followed a literature-based, ten-step validation approach. The initial questionnaire (63 items) was revised and reduced to 29 items in the final version after conducting construct verification in the target group (n = 84 athletes and coaches), evaluating content validity by a panel of nutrition experts (n = 8), verifying face validity by think-out-loud interviews in the target group (n = 7), and conducting classical test theory for item reduction analysis (n = 53). For the final GSNKQ-AC, internal consistency, calculated as Cronbach's alpha, was 0.87. Students with a focus on sports nutrition (n = 31) scored significantly higher than athletes and coaches (n = 53), revealing good construct validity (77% vs. 62%, p < 0.001). Test-retest reliability (n = 42, matched pairs) showed a Spearman's correlation coefficient of r = 0.61 (p < 0.01). The brief GSNKQ-AC can be used for status quo or longitudinal assessment of nutritional knowledge among athletes and coaches to reveal gaps and ensure purposeful planning of educational interventions.

The HAIR-QoL measure Part 2: Validation of an instrument to measure of the severity and impact of chemotherapy-induced alopecia (CIA)

ObjectiveChemotherapy-induced alopecia (CIA) is one of the most common and distressing side effects related to cancer treatment. Few patient-reported outcome measures (PROMs) have been developed specifically to measure the patient-experience of hair loss during chemotherapy treatment. In part I of this research, a set of provisional questionnaire items were developed for a new tool. This article focusses on the validation and psychometric testing of the CIA-specific scoring items in the provisional HAIR-QoL measure. MethodsThe provisional HAIR-QoL questionnaire, comprised of 43 CIA-specific items, was administered to a large sample of patients (n = 228) from 8 international centres, who were undergoing chemotherapy. Psychometric testing used a combination of Classical Test Theory (CTT) and Item Response Theory methods; Principal Components Analysis (PCA) and Rasch Analysis, respectively. ResultsFrom the initial item pool of 43 items, 28 items satisfied the criteria for inclusion into four components. The four subscales identified were as follows, 1) Concern about change in appearance, 2) Image of self, 3) Preparedness for hair loss and 4) Impact of hair loss on you and others. The results of the quantitative analyses, PCA and Rasch analysis, suggested modest amendment to the number of items to the provisional HAIR-QoL measure. Psychometric properties were reported to be very good/excellent for all four subscales and were found to be sensitive to detecting group differences. ConclusionThe refined items which comprise the final HAIR-QoL measure were found to tap into several important psycho-social domains of concern for patients who experienced CIA. The subscale scores have the potential to inform future decision-making on patients’ unmet information and support needs and pave the way for personalised psychological care and more robust research on efficacy of scalp cooling.

The development and evaluation of the worker-occupation fit inventory

BackgroundPerson-environment fit (PEF) theory, one of the foundational theories of occupational stress, has primarily found applications in organizational behavior and human resource management. Given the alignment between the definition of occupational stress and the essence of PEF, we introduced the concept of worker-occupation fit (WOF). To validate our theoretical model, the development of an instrument to measure WOF becomes imperative.MethodsThe Worker-Occupation Fit Inventory (WOFI) comprises three dimensions: personal trait fit (PTF), need-supply fit (NSF) and demand-ability fit (DAF). Job-related mental disorders (JRMDs) were assessed using the DASS-21. During the pre-investigation, items of the WOFI underwent screening through classic test theory (CTT) analysis. In the formal investigation, item response theory (IRT) analysis was employed to evaluate the selected items. The relationship between WOF and JRMD was verified by Pearson’s correlation analysis and multiple logistic regression analysis.ResultsThe initial version consisted of 26 items. Three common factors were extracted by exploratory factor analysis (EFA): 6 items were included in the PTF, 6 items were included in the NSF, 4 items were included in the DAF, and 10 items were deleted because of unacceptable factor loadings. The confirmatory factor analysis (CFA) verified the structure of the WOFI with χ2/df = 1.822, CFI = 0.947, and SRMSR = 0.056. The Cronbach's alpha coefficients of the PTF, NSF, and DAF were 0.91, 0.92, and 0.80, respectively. In IRT analysis, the discrimination values of all items ranged from 1.25 to 2.53, and the difficulty values of all items ranged from -6.28 to 1.30 (with no difficulty of reversal). The WOF was negatively related to job-related stress (r = -0.34, p<0.001), anxiety (r = -0.37, p<0.001), and depression (r = -0.41, p<0.001). The multiple logistic regression analysis suggested that a high level of WOF was a protective factor against job-related mental disorders, with ORs all less than 1 (p<0.001), and a low level of WOF was a risk factor for job-related mental disorders, with ORs all more than 1.0 (p<0.001).ConclusionsThe results of CTT and IRT analysis indicated that the WOFI exhibits reliability and validation. The WOF effectively predicted job-related mental disorders. Subsequent studies will delve into the influence of WOFI on diverse professions and various health outcomes.

Psychometric properties of the mock interview rating scale for autistic transition-age youth.

Employment is a major contributor to quality of life. However, autistic people are often unemployed and underemployed. One potential barrier to employment is the job interview. However, the availability of psychometrically-evaluated assessments of job interviewing skills is limited for autism services providers and researchers. We analyzed the psychometric properties of the Mock Interview Rating Scale that was adapted for research with autistic transition-age youth (A-MIRS; a comprehensive assessment of video-recorded job interview role-play scenarios using anchor-based ratings for 14 scripted job scenarios). Eighty-five transition-age youth with autism completed one of two randomized controlled trials to test the effectiveness of two interventions focused on job interview skills. All participants completed a single job interview role-play at pre-test that was scored by raters using the A-MIRS. We analyzed the structure of the A-MIRS using classical test theory, which involved conducting both exploratory and confirmatory factor analyzes, Rasch model analysis and calibration techniques. We then assessed internal consistency, inter-rater reliability, and test-retest reliability. Pearson correlations were used to assess the A-MIRS' construct, convergent, divergent, criterion, and predictive validities by comparing it to demographic, clinical, cognitive, work history measures, and employment outcomes. Results revealed an 11-item unidimensional construct with strong internal consistency, inter-rater reliability, and test-retest reliability. Construct [pragmatic social skills (r = 0.61, p < 0.001), self-reported interview skills (r = 0.34, p = 0.001)], divergent [e.g., age (r = -0.13, p = 0.26), race (r = 0.02, p = 0.87)], and predictive validities [competitive employment (r = 0.31, p = 0.03)] received initial support via study correlations, while convergent [e.g., intrinsic motivation (r = 0.32, p = 0.007), job interview anxiety (r = -0.19, p = 0.08)] and criterion [e.g., prior employment (r = 0.22, p = 0.046), current employment (r = 0.21, p = 0.054)] validities were limited. The psychometric properties of the 11-item A-MIRS ranged from strong-to-acceptable, indicating it may have utility as a reliable and valid method for assessing the job interview skills of autistic transition-age youth.

Psychometric Evaluation of the PNH Symptom Questionnaire (PNH-SQ) Among Patients with Paroxysmal Nocturnal Hemoglobinuria from Three Phase 2 Clinical Trials with Pozelimab Monotherapy or in Combination with Cemdisiran

Background: Paroxysmal nocturnal hemoglobinuria (PNH) is a rare, acquired, life-threatening disease caused by chronic dysregulation of the complement system. PNH has a high symptom burden with significant impacts on the life of a patient. The PNH Symptom Questionnaire (PNH-SQ) is a PNH-specific patient-reported outcome (PRO) questionnaire designed to assess daily symptom severity in patients with PNH. The objective of this current study was to evaluate PNH-SQ psychometric properties among patients with PNH from three phase 2 clinical trials with pozelimab monotherapy or in combination with cemdisiran. Methods: PRO data from three phase 2 PNH clinical trials, study 1852 (NCT03946748; pozelimab monotherapy; n=24), study 2092 (NCT04811716; combination therapy; n=24), and study 20105 (NCT04888507; combination therapy; n=6), were used for analysis. The PNH-SQ is a patient-reported daily diary that assesses 10 core symptoms of PNH using a five-point severity scale (none; very mild; mild; moderate; severe; very severe) for fatigue, shortness of breath, muscle weakness, headaches, abdominal pain, pain in back or legs, and chest discomfort, and a five-point difficulty scale (none, not at all, a little bit, somewhat, quite, very) for difficulty sleeping, difficulty thinking clearly, and difficulty swallowing. Prespecified psychometric analyses were performed after study database lock using blinded data. Analyses included quality of completion (ie ePRO prevented item-level missingness) and item responses over time. Reliability and validity of daily assessments with classical test theory (internal consistency and test-retest reliability, construct validity by observing correlations between pairs of PNH-SQ items and of the PNH-SQ daily score with other PRO scores from the EORTC QLQ-C30, FACIT-Fatigue and EQ-5D) and Rasch measurement theory (RMT) analyses were conducted. Test-retest reliability (within stable patients 4 weeks apart) and construct validity of various exploratory methods for aggregating daily scores over time were assessed. Results: Completion rates of PNH-SQ items were higher in studies 20105 and 2092 (at Day 1: 54.2% in 1852 vs 83.3% in 20105 and 91.7% in 2092). Fatigue was consistently the most experienced symptom at baseline of the three studies, while difficulty swallowing was not experienced by any patient. A high proportion of ‘none’ responses were observed across all assessments. Cronbach's alpha calculated using all daily assessments was 0.75, indicating modest reliability. The correlation between each pair of items at Day 1 ranged between 0.11 and 0.93, with the lowest correlations being observed between pain items and sleep/cognitive items, and between abdominal pain and fatigue/dyspnea. At Day 1, PNH-SQ daily score trended higher according to the Patient Global Impression of Severity (PGIS)-symptoms categories and showed moderate correlations in expected direction with fatigue and pain, as assessed by FACIT-Fatigue and EORTC QLQ-C30, and lower correlations with dyspnea and overall health status (EQ-5D visual analog scale). RMT analyses of the daily PNH-SQ data highlighted that for response options “Very mild” for severity/“Not at all” for difficulty did not provide any useful information. RMT also uncovered a meaningful hierarchy of symptoms ranging from fatigue to difficulty swallowing. A fixed 7-day average led to the best test-retest reliability (intraclass correlation coefficient, 0.76). Conclusions: Our analyses documented the PNH-SQ as an instrument to capture symptoms of PNH. The most frequent symptoms reported in our sample using the PNH-SQ, fatigue, dyspnea, and headaches, were aligned with observational study data. The sample of observations from the three trials was strongly skewed towards absence of symptoms, probably reflecting the low symptomatic severity of patients included in the study. Despite this restriction, early supportive data for the scoring of the PNH-SQ from a sample of patients with PNH were obtained, generating useful insights into the function of the response scales, opportunities for the creation of daily assessments of symptom severity, and for the creation of a measure reflecting severity over a longer period, for which a fixed 7-day average appeared as a potential good scoring approach. Generating further evidence, especially in a sample of more symptomatic patients, is recommended to substantiate these results.

Psychometric Analysis at Item Level of the Waec May/June Mathematics Multiple Choice Questions Using the Classical Test Theory

Psychometric Analysis at Item Level of the Waec May/June Mathematics Multiple Choice Questions Using the Classical Test Theory

Five Golden Rules for Successful Classroom Assessment Based on What We Have Learnt from JD Brown

Classroom language assessment aims to gather various types of information related to student language learning and use and use it to inform a range of formative-summative decisions, including how to provide feedback to help students improve their learning efficacy, what teaching adjustments are needed, and whether students have demonstrated their learning attainment defined by a given syllabus (Turner, 2012). This article presents five golden rules based on the contributions of JD Brown. These golden rules address a need for a practical and accessible assessment approach that helps language teachers effectively assess students’ learning achievement in the language classroom. This article is, therefore, written for language teachers and pre-service TESOL (Teaching English to Speakers of Other Languages) students. The five golden rules presented in this article are: (1) choose the suitable types of language assessment tasks; (2) recognise the two types of assessment interpretations; (3) consider and apply basic classical test theory for analysis of the assessment of learning; (4) develop well-defined assessment criteria and rubrics for judging students’ performance; and (5) use assessment of language learning as a source of feedback to improve assessment tasks and student learning.

ИЗБОР НА ПОДХОДЯЩ МОДЕЛ ЗА ИЗСЛЕДВАНЕ НА ЗАДАЧИТЕ ОТ НВО ПО МАТЕМАТИКА СЛЕД VII КЛАС СПОРЕД МЕТОДИТЕ НА IRT

In the article the methods of Item Response Theory were used for analyzing the measuring qualities of the test tasks with a selectable answer from the National External Evaluation for the Grade 7 conducted in 2021 in the Sofia city and Pleven districts. After agreement with the empirical data, the choice of the two-parameter model was justified using this methodology. A comparison was made between the obtained data and those from the Classical Test Theory.