Item Response Theory Scores Research Articles

Item response theory validation of the Oxford knee score and Activity and Participation Questionnaire: a step toward a common metric

ObjectivesThe Oxford knee score (OKS) and OKS Activity and Participation Questionnaire (OKS-APQ) are patient-reported outcome measures used to assess people undergoing knee replacement surgery. They have not explicitly been tested for unidimensionality (whether they measure one underlying trait such as ‘knee health’). This study applied item response theory (IRT) to improve the validity of the instruments to optimize for ongoing use. Study Design and SettingParticipants undergoing primary total knee replacement (TKR) provided preoperative and postoperative responses for OKS and OKS-APQ. Confirmatory factor analysis (CFA) were performed on the OKS and OKS-APQ separately and then on both when pooled into one. An IRT model was fitted to the data. Results2972 individual response patterns were analyzed. CFA demonstrated that when combining OKS and OKS-APQ as one instrument, they measure one latent health trait. A user-friendly, free-to-use, web app has been developed to allow clinicians to upload raw data and instantly receive IRT scores. ConclusionsThe OKS and OKS-APQ can be combined to use effectively as a single instrument (producing a single score). For the separate OKS and OKS-APQ the original items and response options can continue to be posed to patients, and this study has confirmed the suitability of IRT-weighted scoring. Applying IRT to existing responses converts traditional sum scores into continuous measurements with greater granularity, including individual measurement error.

Exposure to per- and polyfluoroalkyl substances and age-related macular degeneration in U.S. middle-aged and older adults

Despite various health effects of per- and polyfluoroalkyl substances (PFAS) exposure, the association between PFAS exposure and age-related macular degeneration (AMD) has not been investigated. We aimed to assess associations of PFAS exposure with AMD, using data from 1722 U.S. adults aged 40 years or more participating in the National Health and Nutrition Examination Survey 2005–2008 with complete data on PFAS measurement, AMD diagnosis, and covariates. Serum concentrations of PFAS, including perfluorooctanoic acid (PFOA), perfluorononanoic acid (PFNA), perfluorohexane sulfonic acid (PFHxS), and perfluorooctane sulfonic acid (PFOS), were measured. An overall PFAS burden score was calculated using item response theory scoring. Individual PFAS concentration and overall PFAS burden score were categorized into low (reference), medium, and high groups. Diagnosis of AMD was based on retinal image examination. Any AMD was defined as the presence of early or late AMD. Survey-weighted logistic regression adjusted for potential confounders was used to calculate odds ratios (ORs) and 95% confidence intervals (CIs) for presence of AMD according to PFAS exposure. Overall, 132 (6.5%) individuals were diagnosed as any AMD, including 115 (5.7%) individuals with early AMD. A significant dose-response association was observed between serum PFOS concentration and any AMD (p-trend = 0.03), with a significant OR of 1.99 (95% CI: 1.05, 3.79) for the high group compared to the reference. Overall PFAS burden showed a non-monotonic association with any AMD, with a significant OR of 2.18 (95% CI: 1.18, 4.04) for the medium. Inverted U-shaped associations were observed by restricted cubic spline analyses. Also, early AMD showed similar patterns in PFOS and overall PFAS burden and additionally an inverted U-shape association in PFNA. Our findings suggest that exposure to PFAS estimated by serum PFOS and PFNA as well as overall PFAS burden might be a risk factor for AMD in middle-aged and older population.

Circumventing construct-irrelevant variance in international assessments using cognitive diagnostic modeling: A curriculum-sensitive measure

International large-scale assessments such as TIMSS administer achievement tests that are based on an analysis of national curricula to compare student achievement across countries. The organizations that coordinate these studies use Rasch or more generalized item response theory (IRT) models in which all test items are assumed to measure a single latent ability. The test responses are then used to estimate this ability, and the ability scores are used to compare countries.A central but yet-to-be-contested assumption of this approach is that the achievement tests measure an unobserved unidimensional continuous variable that is comparable across countries. One threat to this assumption is the fact that countries and even regions or school tracks within countries have different curricula. When seeking to fairly compare countries, it seems legitimate to account for the fact that applicable curricula differ and that some students may not have been taught the full test content yet. When seeking to fairly compare countries, it seems imperative to account for the fact that national curricula differ and that some countries may not have taught the full test content yet. Nevertheless, existing IRT-based rankings ignore such differences.The present study proposes a direct method to deal with differing curricula and create a fair ranking of educational quality between countries. The new method compares countries solely on test content that has already been taught; it uses information on whether students have mastered skills taught in class or not and does not consider contents that have not been taught yet. Mastery is assessed via the deterministic-input, noisy, “and” gate (DINA) model, an interpretable and tractable cognitive diagnostic model. To illustrate the new method, we use data from TIMSS 1995 and compare it to the IRT-based scores published in the international study report. We find a mismatch between the TIMSS test contents and national curricula in all countries. At the same time, we observe a high correlation between the scores based on the new method and the conventional IRT scores. This finding underscores the robustness of the performance measures reported in TIMSS despite existing differences across national curricula.

Cerebrovascular reactivity MRI as a biomarker for cerebral small vessel disease-related cognitive decline: Multi-site validation in the MarkVCID Consortium.

Vascular contributions to cognitive impairment and dementia (VCID) represent a major factor in cognitive decline in older adults. The present study examined the relationship between cerebrovascular reactivity (CVR) measured by magnetic resonance imaging (MRI) and cognitive function in a multi-site study, using a predefined hypothesis. We conducted the study in a total of three analysis sites and 263 subjects. Each site performed an identical CVR MRI procedure using 5% carbon dioxide inhalation. A global cognitive measure of Montreal Cognitive Assessment (MoCA) and an executive function measure of item response theory (IRT) score were used as outcomes. CVR and MoCA were positively associated, and this relationship was reproduced at all analysis sites. CVR was found to be positively associated with executive function. The predefined hypothesis on the association between CVR and a global cognitive score was validated in three independent analysis sites, providing support for CVR as a biomarker in VCID. This study measured a novel functional index of small arteries referred to as cerebrovascular reactivity (CVR). CVR was positively associated with global cognition in older adults. This finding was observed in three independent cohorts at three sites. Our statistical analysis plan was predefined before beginning data collection.

Psychometric properties of the health literacy instrument in Brazil (HLS-EU-BR47)

BackgroundHealth literacy (HL) is a key component of health promotion and sustainability and contributes to well-being. Despite its global relevance, HL is an under-researched topic in South America but is now debuting its exploration in Brazil. To leverage its benefits for South America, the mere translation of validated tools into Portuguese is insufficient. Rather, it is necessary to examine their validity. This study aims to assess the psychometric properties of the European Health Literacy Questionnaire (HLS-EU-BR47) using the Item Response Theory (IRT) in a population-based sample of adults in Brazil.MethodsA cross-sectional online study was conducted across Brazil and included 1028 participants aged 18 years and above (80% women). Cronbach’s alpha, McDonald’s omega, factor analysis, graded responses model, Item Characteristic Curve, HL levels based on this, HL standard calculation, IRT, and regular score correlation were computed.ResultsThe instrument exhibit high reliability (Cronbach’s alpha 0.95). Factor analysis yielded one factor. IRT was appropriate for data analysis because it allowed quality evaluation of items and constructed a scale to quantify HL. The 47 items and latent features of respondents in the same unit of measurement are positioned in the construction of the HLS-EU-BR47 instrument. The percentages of individuals at each HL level, calculated using IRT, were found to be comparable to those obtained through the standard computation, e.g., 3.2% of people reported very low HL versus 10.8% inadequate HL, 56.2% reported low HL versus 39.5% problematic HL, 31.1% had moderate HL versus 30.1% sufficient HL, and 9.5% had high HL versus 19.7% with excellent HL. The mean HL scores were comparable between women and men (33.9 vs. 33.7, P = 0.36).ConclusionThis study provides new evidence of the validity of a widely used HL instrument for the population of South America (in this case, Brazil). This tool can be utilized by citizens, health professionals, and regional/national policymakers to inform the development of initiatives to assess and improve the HL of individuals, groups, and communities. Further studies are needed to confirm and extend the findings and to explore the influence of local cultures and practices in the vast Brazilian territory on HL.

Measuring risky loot box use: An item response theory analysis of the Risky Loot Box Index among adolescents.

Loot boxes (LBs) are virtual items embedded within video games that contain randomly generated in-game prizes. LB use can become risky, so it is important to have good measurement instruments, especially among adolescents, who are particularly involved in video gaming and LB purchasing. The present study analyses the adequacy of the Risky Loot Box Index (RLI; Brooks & Clark, 2019) by applying item response theory (IRT). Participants were 2,443 (59% males, mean age = 16.48, SD = 1.22) Italian high school students. The 2PL logistic model was applied. Item properties (i.e., severity and discrimination) were consistent with the aim of efficiently measuring risky LB use. The test information function indicated that the instrument was adequately informative. The RLI appeared to be invariant across gender, with male adolescents more at-risk than female adolescents. LB engagement was also found to explain latent trait of risky LB use over and beyond video gaming and gambling frequency and severity when controlling for gender. The RLI is an efficient screening tool that can specifically measure risky LB use among youth. It can profitably be used for research and intervention purposes. The promising usefulness of the IRT score for clinical purposes is also discussed. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Elevated levels of serum per- and poly-fluoroalkyl substances (PFAS) in contact lens users of U.S. young adults

Despite evidence indicating the presence of per- and poly-fluoroalkyl substances (PFAS) in contact lenses (CL), it remains unclear whether CL use increases PFAS exposure in the general population. We aimed to determine whether CL users have higher serum concentrations of PFAS than non-users, using data of 1660 adults aged 20–39 years participating in the National Health and Nutrition Examination Survey 1999–2000 and 2003–2008. We classified the individuals into CL users and non-users using the record of vision correction types during a vision test. Serum concentrations of six individual PFAS were measured, and the overall PFAS burden was calculated by item response theory scoring. Survey-weighted linear models were used to compare serum PFAS levels between CL users and non-users after adjusting for covariates. Distributions of perfluorooctanoic acid (PFOA) and perfluorooctane sulfonic acid (PFOS) concentrations were compared with reference values (e.g., human biomonitoring [HBM]-II values) indicating potential for adverse health effects. Survey-weighted linear models revealed that covariate-adjusted serum PFOA concentration was higher in CL users (geometric mean [GM]: 3.68 ng/mL; 95% CI: 3.00, 4.50) than in non-users (GM: 3.27 ng/mL; 95% CI: 2.81, 3.80; p = 0.02). Similarly, CL users had a significantly higher serum PFHxS concentration (GM: 1.58 ng/mL; 95% CI: 1.13, 2.20) than non-users (GM: 1.30 ng/mL; 95% CI: 1.10, 1.66; p = 0.03). CL users also had a significantly higher overall burden of PFAS than non-users. The differences in PFAS concentrations between CL users and non-users were more pronounced in females than in males. Moreover, a larger proportion of lens users (4.5%), compared to non-users (3.9%), had PFOA concentrations above the HBM-II, where adverse health effects are expected from PFOA exposure. This study suggests that CL use in general U.S. young adults may contribute significantly to PFAS body burden, which can potentially lead to public health problem.

Measuring academic resilience of socioeconomically disadvantaged students in Taiwan 2011-2017: Two-part latent class growth modeling based on IRT scores

Academic resilience is a critical topic that has been around for decades and contains a variety of theoretical models and research approaches. In this current study, we adopted the developmental view of resilience. We combined the person-focused (classification and trajectory) and variable-focused (regression) methods to analyze the longitudinal data of 744 students from the TDCYP 2011-2017. We conducted item response theory (IRT) models and a latent class growth modeling framework. First, we measured each person’s latent trait level by IRT and simultaneously identified the classes by positive outcome (academic self-concept) and risk factors. Eventually, to explain the class membership and growth factors by adding covariates. The finding of this study as follows, (1) the four-class model was selected, and they are the competent group (17%), the resilient group (34%), the vulnerable group (31%), and the maladaptive group (18%). (2) The developmental hypothesis of academic resilience was not confirmed because the slope factors of academic self-concept and two groups of risk factors showed no significance. (3) Moreover, we found gender, mental health (illness), and teacher support play critical roles in this study, and they could explain the growth trends and even the membership classifying. The family support and poverty factors were lack of influence might due to the data properties. Importantly, this study can advise educators that it is worth further studying and practicing that school-based context on how to support a student, especially the socioeconomically disadvantaged. Keywords: Academic resilience, socioeconomically disadvantaged, latent class growth modeling, longitudinal IRT schools, Taiwan Database of Children and Youth in Poverty (TDCYP)

Trajectory of depressive symptoms prior to dementia diagnosis

AbstractBackgroundDepression has been repeatedly linked to increased risk for dementia. While there is some research to suggest that an individual’s trajectory of depression symptoms over time may inform dementia risk above and beyond depression symptoms at a single time point, this has yet to be demonstrated in a dataset specifically designed to capture longitudinal brain aging and dementia outcomes.MethodA cohort study of 4,171 community‐dwelling older adults (65+ years) who were enrolled in the Adult Changes in Thought (ACT) study. Participants were initially non‐demented at baseline, and underwent biannual assessment of cognitive status (Cognitive Abilities Screening Instrument, diagnostic consensus) and self‐reported depression symptoms (via 20‐item Centers for Epidemiological Studies‐Depression [CESD]) for up to 24 years. We used Item Response Theory (CESD‐IRT) scores to address limited sensitivity to lower depressive symptom level in the general population and nonlinear measurement properties of the CESD. Diagnosis of dementia was made at consensus meeting based on history and clinical examination.ResultSample characteristics are summarized in Table 1. Linear mixed effects modeling, adjusted for covariates (age, sex, education, race), revealed that there was greater increase of depressive symptoms prior to diagnosis of dementia compared to those who did not receive diagnosis of dementia at end of study (p<.0001). This finding was consistent when depression symptom trajectories were estimated using CESD‐20 raw vs. CES‐IRT scores (Fig. 1 & 2, respectively). The difference in depression symptom trajectories by dementia outcome did not differ as a function of sex. Significant differences (p<0.001) in mean depression symptom total scores were observed by dementia outcome as early as 10 years prior to diagnosis/censoring for both the CESD‐20 raw score (1.0 ± 0.2) and the CESD‐IRT scores (0.16 ± 0.03).ConclusionThe pattern of increasing depression symptoms many years prior to dementia diagnosis reinforces the notion that the association between depressive symptoms and dementia in older adults may be primarily due to a shared common cause, or that depressive symptoms represent an early sign of dementia in the preclinical phase. Changes in specific depressive symptoms prior to dementia diagnosis will also be examined.

Psychometric sensitivity analyses can identify bias related to measurement properties in trials that use patient-reported outcome measures: a secondary analysis of a clinical trial using the disabilities of the arm, shoulder, and hand questionnaire

ObjectivesDemonstrate psychometric sensitivity analyses for testing the stability of study findings to assumptions made about patient-reported outcome measures. Study Design and SettingWe performed secondary analyses of Disability of Arm, Shoulder, and Hand (DASH) data collected within the Prevention of Shoulder Problems clinical trial, which compared upper limb function scores in women who had undergone breast cancer surgery, randomized to either an exercise program or usual care. We repeated the principal trial analyses after grouping DASH items into subscales suggested by factorial analyses in this dataset and applied item response theory to account for unequal item weighting. We checked for measurement invariance by participant age and response shift bias using established techniques. ResultsOur analyses suggested that the DASH measured two constructs: motor function and sensory symptoms. The majority of the six-month difference in DASH score was driven by motor function. With item response theory scoring, we found differences in both constructs at 12 months (P = 0.019 and P = 0.007), but in neither construct at 6 months, contrary to the original trial results. We found no differential item function by age or between baseline and 12-month measurements. ConclusionsPsychometric sensitivity analyses aid in the interpretation of the Prevention of Shoulder Problems trial’s results.

Item response theory in early phase clinical trials: Utilization of a reference model to analyze the Montgomery-Åsberg Depression Rating Scale.

In clinical trials, Montgomery-Åsberg Depression Rating Scale (MADRS) questionnaire data are added up to total scores before analysis, assuming equal contribution of each separate question. Item Response Theory (IRT)-based analysis avoids this by using individual question responses to determine the latent variable (ψ), which represents a measure of depression severity. However, utilization of IRT in early phase trials remains difficult, because large datasets are needed to develop IRT models. Therefore, we aimed to evaluate the application and assumptions of a reference IRT model for analysis of an early phase trial. A cross-over, placebo-controlled study investigating the effect of intravenous racemic ketamine on MADRS scores in patients with treatment-resistant major depressive disorder was used as a case study. One hundred forty-seven MADRS responses were measured in 17 patients at five timepoints (predose to 2 weeks after dosing). Two reference IRT models based on different patient populations were selected from literature and used to determine ψ, while testing multiple approaches regarding assumed data distribution. Use of ψ versus total score to determine treatment effect was compared through linear mixed model analysis. Results showed that determined ψ values did not differ significantly between assumed distributions, but were significantly different when changing reference IRT model. Estimated treatment effect size was not significantly affected by chosen approach nor reference population. Finally, increased precision to determine treatment effect was achieved by using IRT versus total scores. This demonstrates the usefulness of reference IRT model application for analysis of questionnaire data in early phase clinical trials.

Does Scoring Method Impact Estimation of Significant Individual Changes Assessed by Patient-Reported Outcome Measures? Comparing Classical Test Theory Versus Item Response Theory

ObjectivesThis study aimed to examine the ability of classical test theory (CTT) and item response theory (IRT) scores assessed by Patient-Reported Outcomes Measurement Information System® (PROMIS®) measures to identify significant individual changes in the setting of clinical studies, using both simulated and empirical data. MethodsWe used simulated data to compare the estimation of significant individual changes between CTT and IRT scores across different conditions and a clinical trial data set to verify the simulation results. We calculated reliable change indexes to estimate significant individual changes. ResultsFor small true change, IRT scores showed a slightly higher rate of classifying change groups than CTT scores and were comparable with CTT scores for a shorter test length. Additionally, IRT scores were found to have a prominent advantage in the classification rates of change groups for medium to high true change over CTT scores. Such an advantage became prominent in a longer test length. The empirical data analysis results using an anchor-based approach further supported the above findings that IRT scores can more accurately classify participants into change groups than CTT scores. ConclusionsGiven that IRT scores perform better, or at least comparably, in most conditions, we recommend using IRT scores to estimate significant individual changes and identify responders to treatment. This study provides evidence-based guidance in detecting individual changes based on CTT and IRT scores under various measurement conditions and leads to recommendations for identifying responders to treatment for participants in clinical trials.

Analysis of the 24-h activity cycle: An illustration examining the association with cognitive function in the Adult Changes in Thought study.

The 24-h activity cycle (24HAC) is a new paradigm for studying activity behaviors in relation to health outcomes. This approach inherently captures the interrelatedness of the daily time spent in physical activity (PA), sedentary behavior (SB), and sleep. We describe three popular approaches for modeling outcome associations with the 24HAC exposure. We apply these approaches to assess an association with a cognitive outcome in a cohort of older adults, discuss statistical challenges, and provide guidance on interpretation and selecting an appropriate approach. We compare the use of the isotemporal substitution model (ISM), compositional data analysis (CoDA), and latent profile analysis (LPA) to analyze 24HAC. We illustrate each method by exploring cross-sectional associations with cognition in 1,034 older adults (Mean age = 77; Age range = 65-100; 55.8% female; 90% White) who were part of the Adult Changes in Thought (ACT) Activity Monitoring (ACT-AM) sub-study. PA and SB were assessed with thigh-worn activPAL accelerometers for 7-days. For each method, we fit a multivariable regression model to examine the cross-sectional association between the 24HAC and Cognitive Abilities Screening Instrument item response theory (CASI-IRT) score, adjusting for baseline characteristics. We highlight differences in assumptions and the scientific questions addressable by each approach. ISM is easiest to apply and interpret; however, the typical ISM assumes a linear association. CoDA uses an isometric log-ratio transformation to directly model the compositional exposure but can be more challenging to apply and interpret. LPA can serve as an exploratory analysis tool to classify individuals into groups with similar time-use patterns. Inference on associations of latent profiles with health outcomes need to account for the uncertainty of the LPA classifications, which is often ignored. Analyses using the three methods did not suggest that less time spent on SB and more in PA was associated with better cognitive function. The three standard analytical approaches for 24HAC each have advantages and limitations, and selection of the most appropriate method should be guided by the scientific questions of interest and applicability of each model's assumptions. Further research is needed into the health implications of the distinct 24HAC patterns identified in this cohort.

ASSESSMENT OF THE INSTRUMENTAL ACTIVITIES OF DAILY LIVING IN MILD COGNITIVE IMPAIRMENT AND DEMENTIA DUE TO ALZHEIMER'S DISEASE: DIAGNOSTIC ACCURACY OF THE SERBIAN VERSION OF THE AMSTERDAM IADL QUESTIONNAIRE

To establish a diagnosis of dementia, it is necessary, in addition to cognitive impairment, to prove the existence of a disorder of instrumental activities of daily living (IADL). The Amsterdam IADL Questionnaire is a reliable instrument translated into different languages. This study aims to assess the diagnostic accuracy of the Serbian version of the Amsterdam IADL Questionnaire. The study included 75 patients with mild cognitive impairment (MCI) and dementia due to Alzheimer's disease. The questionnaire was scored using the weighted average (WA) and item response theory (IRT) scoring method. Diagnostic accuracy was examined using receiver–operating characteristic (ROC) curves. The area under the curves (AUC) was calculated with 95% confidence intervals (CI). The correlation between IRT and WA scores was strong and significant (r=-0.980, p<0.001). The AUC for the IRT scores of A-IADL-Q was 0.832 (95% CI: 0.729 to 0.909), while the AUC for the WA scores of A-IADL-Q was 0.848 (95% CI: 0.746 to 0.920). Both were significantly different from the AUC of 0.5 (p<0.001). There was no significant difference between the AUCs of IRT and WA scoring (z=1.157; p=0.247). Cutoffs and the highest combination of sensitivity and specificity for the IRT (sensitivity 0.767; specificity 0. 844) and WA (sensitivity 0.744; specificity 0. 844) scores of A-IADL-Q were calculated. We have shown that A-IADL-Q has moderate diagnostic accuracy in differentiating dementia and MCI. This instrument can be used in combination with cognitive measures to diagnose dementia in its early stages.

A More Efficient Outcome for Alzheimer Disease Research: the Item Response Theory Based Score for the Clinical Dementia Rating (CDR®)

AbstractBackgroundCDR sum of boxes (CDR‐SB) is widely used as a primary outcome in Alzheimer disease (AD) research due to its reliability and accuracy. However, it is an ordinal rank measure instead of a continuous measure and therefore may not be able to capture subtle changes in cognition. Additionally, it weights each domain equally, without accounting for the patterns of the responses scores across CDR items. To generate a more optimal score, we developed a bi‐factor model based on the item response theory (IRT) which can account for different response pattern, item difficulty and discrimination level to provide an improved estimate of the dementia severity. We demonstrated the superiority of the IRT score in two independent cohorts.MethodBaseline item level CDR data from 2949 participants enrolled in the WashU Knight ADRC study were used to develop the IRT model. The best fitting IRT model was then applied to the longitudinal item level CDR data in both the Knight ADRC and DIAN‐TU‐001 (142 participants) studies to generate the IRT scores (a continuous measure) for each participant at each visit.ResultCorrelations between the CDR IRT score and SB were consistent at different visits (0.89 ∼ 0.92 for DIAN‐TU cohort, 0.92 ∼ 0.94 for Knight ADRC cohort). The IRT score had a much larger effect size (> 60% for the DIAN‐TU cohort and > 40% for the Knight ADRC cohort) than that of CDR‐SB when comparing converter to stable cognitively normal participants. For CDR‐SB of 0 group, the IRT score was significantly associated with some of the clinical/cognitive outcomes and biomarkers. For CDR‐SB > 0 group, the IRT score had slightly higher correlations with other outcomes than CDR‐SB. The IRT score correlated better with the estimated age from dementia onset in the DIAN cohort. Using IRT score as the primary outcome in clinical trials could save 10% ∼ 30% of the sample size compared to that of using CDR‐SB.ConclusionThe CDR IRT score could potentially be used as a more efficient outcome in AD research to gain precision in detecting cognitive decline and improve power (or reduce sample size).

Linking Rivermead Post Concussion Symptoms Questionnaire (RPQ) and Sport Concussion Assessment Tool (SCAT) scores with item response theory.

Despite the public health burden of traumatic brain injury (TBI) across broader society, most TBI studies have been isolated to a distinct subpopulation. The TBI research literature is fragmented further because often studies of distinct populations have used different assessment procedures and instruments. Addressing calls to harmonize the literature will require tools to link data collected from different instruments that measure the same construct, such as civilian mild traumatic brain injury (mTBI) and sports concussion symptom inventories. We used item response theory (IRT) to link scores from the Rivermead Post Concussion Symptoms Questionnaire (RPQ) and the Sport Concussion Assessment Tool (SCAT) symptom checklist, widely used instruments for assessing civilian and sport-related mTBI symptoms, respectively. The sample included data from n = 397 patients who suffered a sports-related concussion, civilian mTBI, orthopedic injury control, or non-athlete control and completed the SCAT and/or RPQ. The results of several analyses supported sufficient unidimensionality to treat the RPQ + SCAT combined item set as measuring a single construct. Fixed-parameter IRT was used to create a cross-walk table that maps RPQ total scores to SCAT symptom severity scores. Linked and observed scores were highly correlated (r = .92). Standard errors of the IRT scores were slightly higher for civilian mTBI patients and orthopedic controls, particularly for RPQ scores linked from the SCAT. By linking the RPQ to the SCAT we facilitated efforts to effectively combine samples and harmonize data relating to mTBI.

Comparison of two rating scales with the orofacial esthetic scale and practical recommendations for its application

PurposeWe compared measurement properties of 5-point and 11-point response formats for the orofacial esthetic scale (OES) items to determine whether collapsing the format would degrade OES score precision.MethodsData were collected from a consecutive sample of adult dental patients from HealthPartners dental clinics in Minnesota (N = 2,078). We fitted an Item Response Theory (IRT) model to the 11-point response format and the six derived 5-point response formats. We compared all response formats using test (or scale) information, correlation between the IRT scores, Cronbach’s alpha estimates for each scaling format, correlations based on the observed scores for the seven OES items and the eighth global item, and the relationship of observed and IRT scores to an external criterion using orofacial appearance (OA) indicators from the Oral Health Impact Profile (OHIP).ResultsThe correlations among scores based on the different response formats were uniformly high for observed (0.97–0.99) and IRT scores (0.96–0.99); as were correlations of both observed and IRT scores and the OHIP measure of OA (0.66–0.68). Cronbach’s alpha based on any of the 5-point formats (α = 0.95) was nearly the same as that based on the 11-point format (α = 0.96). The weighted total information area for five of six derived 5-point response formats was 98% of that for the 11-point response format.ConclusionsOur results support the use of scores based on a 5-point response format for the OES items. The measurement properties of scores based on a 5-point response format are comparable to those of scores based on the 11-point response format.

Improving the Precision of the Glasgow Outcome Scale-Extended Using Item Response Theory: A TRACK-TBI Study.

The Glasgow Outcome Scale-Extended (GOSE) is a functional outcome measure intended to place individuals with traumatic brain injury (TBI) into one of eight broad levels of injury-related disability. This simplicity is not always optimal, particularly when more granular assessment of individuals' injury recovery is desired. The GOSE, however, is customarily assessed using a multi-question interview that contains richer information than is reflected in the GOSE score. Using data from the multi-center Transforming Research and Clinical Knowledge in TBI (TRACK-TBI) study (N = 1544), we used item response theory (IRT) to evaluate whether rescoring the GOSE using IRT, which posits that a continuous latent variable (disability) underlies responses, can yield a more precise index of injury-related functional limitations. We fit IRT models to GOSE interview responses collected at three months post-injury. Each participant's level of functional limitation was estimated from the model (GOSE-IRT) and comparisons were made between IRT-based and standard (GOSE-Ordinal) scores. The IRT scoring resulted in 141 possible scores (vs. 7 GOSE-Ordinal scores in this sample of individuals with GOSE scores ranging between 2 and 8). Moreover, GOSE-IRT scores were significantly more strongly associated with measures of TBI-related symptoms, psychological symptoms, and quality of life. Our findings demonstrate that rescoring the GOSE interview using IRT yields more granular, meaningful measurement of injury-related functional limitations, while adding no additional respondent or examiner burden. This technique may have utility for many applications, such as clinical trials aiming to detect small treatment effects, and small-scale studies that need to maximize statistical efficiency.

Computerized adaptive testing of symptom severity: a registry-based study of 924 patients with trapeziometacarpal arthritis.

We aimed to develop a computerized adaptive testing (CAT) version of the 11 item Patient Evaluation Measure (PEM), using an item response theory model. This model transformed the ordinal scores into ratio-interval scores. We obtained PEM responses from 924 patients with trapeziometacarpal osteoarthritis to build a CAT model and tested its performance on a simulated cohort of 1000 PEM response sets. The CAT achieved high precision (median standard error or measurement 0.26) and reduced the number of questions needed for accurate scoring from 11 to median two. The CAT scores and item-response-theory-based 15-item PEM scores were similar, and a Bland–Altman analysis demonstrated a mean score difference of 0.2 between the CAT and the full-length PEM scores on a scale from 0 to 100. We conclude that the CAT substantially reduced the burden of the PEM while also harnessing the validity of item response theory scoring.

Constructing a Robust Score Scale from IRT Scores with Informed Boundaries

AbstractIn operational testing, item response theory (IRT) models for dichotomous responses are popular for measuring a single latent construct , such as cognitive ability in a content domain. Estimates of , also called IRT scores or , can be computed using estimators based on the likelihood function, such as maximum likelihood (ML), weighted likelihood (WL), maximum a posteriori (MAP), and expected a posteriori (EAP). Although the parameter space of is theoretically unrestricted, the range of finite is constrained by the estimator and test form properties, which is important to consider but often overlooked when developing a score scale for reporting purposes. Irrespective of the estimator or test forms at hand, a common practice is to fix arbitrary points symmetric about zero (e.g., −4 and 4) as anchors for deriving a score transformation, possibly resulting in unintended gaps or truncations at the extremes. Therefore, a systematic framework is proposed for using IRT scores to construct a robust score scale with informed boundaries that are logical and consistent across test forms.