Unidimensional Item Response Theory Models Research Articles

Psychometric validation of the Cystic Fibrosis Impact Questionnaire (CF-IQ): A patient-reported outcome assessing impacts of cystic fibrosis.

The Cystic Fibrosis (CF) Impact Questionnaire (CF-IQ) was qualitatively developed to assess the impact of CF in the context of treatment advancements and increased longevity. This study reports the CF-IQ validation. In this noninterventional validation study, people with CF completed the 40-item CF-IQ and validating patient-reported outcome measures (PROMs) via electronic diaries at enrollment (baseline) and at the 4-week follow-up. Validation consisted of modern methods and focus groups to finalize structural validity, and classical methods to assess internal consistency [1-3], test-retest reliability [4,5], concurrent validity [5], and known-groups validity [5] of the CF-IQ. At baseline, 214 adults completed the survey; 193 completed the follow-up survey. Unidimensional item response theory (IRT) models were separately fit to 5 prespecified domains (Control and Burden of CF Treatment Impacts, Physical Activity Impacts, Social Activity Impacts, Emotional Impacts, and Work/School Limitation Impacts). IRT local dependence (LD) statistics identified 17 redundant items. Two independent CF-patient focus groups (14 total patients) confirmed these findings, and the 17 items were dropped. Each domain defined on the final 23 items achieved the criterion of exact model fit as measured by the root mean squared error of approximation (RMSEA, values = 0), Internal consistency (Cronbach's α) values ranged from 0.81 to 0.89, 4 of 5 domains achieved acceptable test-retest reliability, with intraclass correlation coefficient (ICC) values ≥ 0.7, acceptable concurrent validity was achieved for all domains, and known-groups validity was established. The novel CF-IQ is a psychometrically robust PROM capturing patient-centric impacts of CF in the context of the current standard of care.

Asymptotically Correct Person Fit z-Statistics For the Rasch Testlet Model.

A well-known person fit statistic in the item response theory (IRT) literature is the statistic (Drasgow et al. in Br J Math Stat Psychol 38(1):67-86, 1985). Snijders (Psychometrika 66(3):331-342, 2001) derived , which is the asymptotically correct version of when the ability parameter is estimated. However, both statistics and other extensions later developed concern either only the unidimensional IRT models or multidimensional models that require a joint estimate of latent traits across all the dimensions. Considering a marginalized maximum likelihood ability estimator, this paper proposes and , which are extensions of and , respectively, for the Rasch testlet model. The computation of relies on several extensions of the Lord-Wingersky algorithm (1984) that are additional contributions of this paper. Simulation results show that has close-to-nominal Type I error rates and satisfactory power for detecting aberrant responses. For unidimensional models, and reduce to and , respectively, and therefore allows for the evaluation of person fit with a wider range of IRT models. A real data application is presented to show the utility of the proposed statistics for a test with an underlying structure that consists of both the traditional unidimensional component and the Rasch testlet component.

Item performance of the scale for the assessment and rating of ataxia in rare and ultra-rare genetic ataxias.

The Scale for the Assessment and Rating of Ataxia (SARA) is widely used for assessing the severity and progression of genetic cerebellar ataxias. SARA is now considered a primary end point in several ataxia treatment trials, but its underlying composite item measurement model has not yet been tested. This work aimed to evaluate the composite properties of SARA and its items using item response theory (IRT) and to demonstrate its applicability across even ultra-rare genetic ataxias. Leveraging SARA subscores data from 1932 visits from 990 patients of the Autosomal Recessive Cerebellar Ataxias (ARCA) registry, we assessed the performance of SARA using IRT methodology. The item characteristics were evaluated over the ataxia severity range of the entire ataxia population as well as the assessment validity across 115 genetic ARCA subpopulations. A unidimensional IRT model was able to describe SARA item data, indicating that SARA captures one single latent variable. All items had high discrimination values (1.5-2.9) indicating the effectiveness of the SARA in differentiating between subjects with different disease statuses. Each item contributed between 7% and 28% of the total assessment informativeness. There was no evidence for differences between the 115 genetic ARCA subpopulations in SARA applicability. These results show the good discrimination ability of SARA with all of its items adding informational value. The IRT framework provides a thorough description of SARA on the item level, and facilitates its utilization as a clinical outcome assessment in upcoming longitudinal natural history or treatment trials, across a large number of ataxias, including ultra-rare ones.

How Scoring Approaches Impact Estimates of Growth in the Presence of Survey Item Ceiling Effects.

Survey scores are often the basis for understanding how individuals grow psychologically and socio-emotionally. A known problem with many surveys is that the items are all "easy"-that is, individuals tend to use only the top one or two response categories on the Likert scale. Such an issue could be especially problematic, and lead to ceiling effects, when the same survey is administered repeatedly over time. In this study, we conduct simulation and empirical studies to (a) quantify the impact of these ceiling effects on growth estimates when using typical scoring approaches like sum scores and unidimensional item response theory (IRT) models and (b) examine whether approaches to survey design and scoring, including employing various longitudinal multidimensional IRT (MIRT) models, can mitigate any bias in growth estimates. We show that bias is substantial when using typical scoring approaches and that, while lengthening the survey helps somewhat, using a longitudinal MIRT model with plausible values scoring all but alleviates the issue. Results have implications for scoring surveys in growth studies going forward, as well as understanding how Likert item ceiling effects may be contributing to replication failures.

Investigation of Models Used in Equating Testlet-Based Tests

This study aims to examine the effects of testlets on test equating. For this purpose unidimensional item response theory, two-factor item response theory and testlet response theory models were applied to the testlet-based tests for the estimation of item and ability parameters. In order to equate the tests, the parameters were placed on the common scale using mean-mean, mean-sigma and Stocking-Lord scale transformation methods under the common-item non-equivalent groups design. Then, the equating errors of the models depending on the scale transformation method and the number of testlets were calculated and compared. Equating errors were compared with Root Mean Squared Error. In the study, the science test of the Trends in International Mathematics and Science Study project administered in 2019 was used as the data collection tool. As a result of the study, it was determined that the use of unidimensional item response theory model increased the equating error, while the use of two-factor and testlet response theory models decreased the equating error as the number of testlets in the test increased. In order to compare the models, the correlation between the parameters obtained from the models after scale transformation was examined and it was found that the item parameters were more affected by the model selection than the ability parameter. In addition, it was concluded that the equating errors obtained from the mean-mean and Stocking-Lord scale transformation methods were lower than the mean-sigma method.

Measuring PROMIS pain interference in German patients with chronic conditions: calibration, validation, and cross-cultural use of item parameters

PurposeTo calibrate the item parameters of the German PROMIS® Pain interference (PROMIS PI) items using an item-response theory (IRT) model and investigate psychometric properties of the item bank.MethodsForty items of the PROMIS PI item bank were collected in a convenience sample of 660 patients, which were recruited during inpatient rheumatological treatment or outpatient psychosomatic medicine visits in Germany. Unidimensionality, monotonicity, and local independence were tested as required for IRT analyses. Unidimensionality was examined using confirmatory factor analyses (CFA) and exploratory factor analysis (EFA). Unidimensional and bifactor graded-response IRT models were fitted to the data. Bifactor indices were used to investigate whether multidimensionality would lead to biased scores. To evaluate convergent and discriminant validity, the item bank was correlated with legacy pain instruments. Potential differential item functioning (DIF) was examined for gender, age, and subsample. To investigate whether U.S. item parameters may be used to derive T-scores in German patients, T-scores based on previously published U.S. and newly estimated German item parameters were compared with each other after adjusting for sample specific differences.ResultsAll items were sufficiently unidimensional, locally independent, and monotonic. Whereas the fit of the unidimensional IRT model was not acceptable, a bifactor IRT model demonstrated acceptable fit. Explained common variance and Omega hierarchical suggested that using the unidimensional model would not lead to biased scores. One item demonstrated DIF between subsamples. High correlations with legacy pain instruments supported construct validity of the item bank. T-scores based on U.S. and German item parameters were similar suggesting that U.S. parameters could be used in German samples.ConclusionThe German PROMIS PI item bank proved to be a clinically valid and precise instrument for assessing pain interference in patients with chronic conditions.

이요인 모형 기반 혼합형 검사를 위한 다차원 IRT 척도 연계 방법

Like unidimensional item response theory (IRT) models, bifactor models in multidimensional IRT have a scale indeterminacy problem, and due to this problem scale linking methods are needed to place all bifactor model parameter estimates from separate calibrations on a common ability scale. Four bifactor scale linking methods including the direct least squares (DLS), mean/least squares (MLS), item category response function (ICRF), and test response function (TRF) methods have been presented for use with single-format tests. Parallel to the 2006 paper of Kim and Lee, this paper extends the four scale linking methods to a mixture of bifactor models for mixed-format tests. Each linking method extended is intended to handle mixed-format tests using any mixture of the following bifactor extensions of four unidimensional IRT models: the bifactor three-parameter logistic, bifactor graded response, bifactor generalized partial credit, and bifactor nominal response models. For generality, symmetric criterion functions are proposed for the ICRF and TRF methods. Given two sets of parameter estimates for the common items linking two test forms, each linking method estimates the dilation (slope) and translation (intercept) coefficients of a linear transformation. Simulations are conducted to investigate the performance of the four linking methods. The results indicate that overall, the ICRF method performs very well, the MLS and DLS methods perform well (the MLS method is slightly better than the DLS method), and the TRF method performs poorly in estimating the linking coefficients. The inferiority of the TRF method is mainly due to its poor estimation of the translation coefficients.

Application of longitudinal item response theory models to modeling Parkinson's disease progression.

The Movement Disorder Society revised version of the Unified Parkinson’s Disease Rating Scale (MDS‐UPDRS) parts 2 and 3 reflect patient‐reported functional impact and clinician‐reported severity of motor signs of Parkinson’s disease (PD), respectively. Total scores are common clinical outcomes but may obscure important time‐based changes in items. We aim to analyze longitudinal disease progression based on MDS‐UPRDS parts 2 and 3 item‐level responses over time and as functions of Hoehn & Yahr (H&Y) stages 1 and 2 for subjects with early PD. The longitudinal item response theory (IRT) modeling is a novel statistical method addressing limitations in traditional linear regression approaches, such as ignoring varying item sensitivities and the sum score balancing out improvements and declines. We utilized a harmonized dataset consisting of six studies with 3573 subjects with early PD and 14,904 visits, and mean follow‐up time of 2.5 years (±1.57). We applied both a unidimensional (each part separately) and multidimensional (both parts combined) longitudinal IRT models. We assessed the progression rates for both parts, anchored to baseline H&Y stages 1 and 2. Both the uni‐ and multidimensional longitudinal IRT models indicate significant worsening time effects in both parts 2 and 3. Baseline H&Y stage 2 was associated with significantly higher baseline severities, but slower progression rates in both parts, as compared with stage 1. Patients with baseline H&Y stage 1 demonstrated slower progression in part 2 severity compared to part 3, whereas patients with baseline H&Y stage 2 progressed faster in part 2 than part 3. The multidimensional model had a superior fit compared to the unidimensional models and it had excellent model performance.

Dissecting the Domains of Parkinson's Disease: Insights from Longitudinal Item Response Theory Modeling.

Longitudinal item response theory (IRT) models previously suggested that the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS) motor examination has two salient domains, tremor and nontremor, that progress in time and in response to treatment differently. Apply longitudinal IRT modeling, separating tremor and nontremor domains, to reanalyze outcomes in the previously published clinical trial (Study of Urate Elevation in Parkinson's Disease, Phase 3) that showed no overall treatment effects. We applied unidimensional and multidimensional longitudinal IRT models to MDS-UPDRS motor examination items in 298 participants with Parkinson's disease from the Study of Urate Elevation in Parkinson's Disease, Phase 3 (placebo vs. inosine) study. We separated 10 tremor items from 23 nontremor items and used Bayesian inference to estimate progression rates and sensitivity to treatment in overall motor severity and tremor and nontremor domains. The progression rate was faster in the tremor domain than the nontremor domain before levodopa treatment. Inosine treatment had no effect on either domain relative to placebo. Levodopa treatment was associated with greater slowing of progression in the tremor domain than the nontremor domain regardless of inosine exposure. Linear patterns of progression were observed. Despite different domain-specific progression patterns, tremor and nontremor severities at baseline and over time were significantly correlated. Longitudinal IRT analysis is a novel statistical method addressing limitations of traditional linear regression approaches. It is particularly useful because it can simultaneously monitor changes in different, but related, domains over time and in response to treatment interventions. We suggest that in neurological diseases with distinct impairment domains, clinical or anatomical, this application may identify patterns of change unappreciated by standard statistical methods. © 2022 International Parkinson and Movement Disorder Society.

Item Response Theory True Score Equating for the Bifactor Model Under the Common-Item Nonequivalent Groups Design.

Applying item response theory (IRT) true score equating to multidimensional IRT models is not straightforward due to the one-to-many relationship between a true score and latent variables. Under the common-item nonequivalent groups design, the purpose of the current study was to introduce two IRT true score equating procedures that adopted different dimension reduction strategies for the bifactor model. The first procedure, which was referred to as the integration procedure, linked the latent variable scales for the bifactor model and integrated out the specific factors from the item response function of the bifactor model. Then, IRT true score equating was applied to the marginalized bifactor model. The second procedure, which was referred to as the PIRT-based procedure, projected the specific dimensions onto the general dimension to obtain a locally dependent unidimensional IRT (UIRT) model and linked the scales of the UIRT model, followed by the application of IRT true score equating to the locally dependent UIRT model. Equating results obtained with the two equating procedures along with those obtained with the unidimensional three-parameter logistic (3PL) model were compared using both simulated and real data. In general, the integration and PIRT-based procedures provided equating results that were not practically different. Furthermore, the equating results produced by the two bifactor-based procedures became more accurate than the results returned by the 3PL model as tests became more multidimensional.

Uirt: A command for unidimensional IRT modeling

In this article, I introduce the uirt command, which allows one to estimate parameters of a variety of unidimensional item response theory models (two-parameter logistic model, three-parameter logistic model, graded response model, partial credit model, and generalized partial credit model). uirt has extended item-fit analysis capabilities, features multigroup modeling, allows testing for differential item functioning, and provides tools for generating plausible values with a latent regression conditioning model. I provide examples to illustrate cases where uirt can be especially useful in conducting analyses within the item response theory approach.

Graded response item response theory in scaling suicidal thoughts and behaviors among trauma-exposed women with substance use disorders.

Objective: The co-occurrence of substance use disorders (SUD) and trauma-exposure is a risk factor for suicidal thoughts and behaviors (STB). However, traditional methods of measurement for suicidal thoughts and behaviors are limited by an overreliance on dichotomous (i.e., yes or no) and averaged/summed scale score measurements. Further, among trauma-exposed individuals with SUD, it remains unclear which specific demographic factors, types of SUDs, and trauma sequelae (e.g., posttraumatic stress disorder [PTSD] symptom clusters) may be associated with elevated STB. The present study utilized item response theory to (a) generate empirically derived STB severity scores and, (b) examine which demographic factors, SUD diagnoses, and DSM-IV PTSD symptom clusters are associated with suicidality in a trauma-exposed sample with SUDs. Method: Female trauma-exposed participants with SUDs (N = 544) were recruited from community substance use treatment facilities in the National Drug Abuse Treatment Clinical Trials Network (CTN). Clinician-administered interviews assessed STB, SUDs, and PTSD symptoms. Results: Results indicated that the unidimensional item response theory (IRT) model used to estimate latent STB severity scores fit well, with strong local reliability at higher levels of latent STB severity. Regression predictors of elevated STB severity included younger age, opioid dependence, and higher PTSD reexperiencing symptoms. Conclusions: Clinicians are advised to screen for and target opioid use disorders and reexperiencing symptoms when addressing suicidal thoughts and behavior in trauma-exposed individuals with SUDs. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Investigating Subscores of VERA 3 German Test Based on Item Response Theory/Multidimensional Item Response Theory Models

In this study, the psychometric properties of the listening and reading subtests of the German VERA 3 test were examined using Item Response Theory (IRT) and Multidimensional Item Response Theory (MIRT) models. Listening and reading subscores were estimated using unidimensional Rasch, 1PL, and 2PL models, and total scores on the German test (listening + reading) were estimated using unidimensional and multidimensional IRT models. Various MIRT models were used, and model fit was compared in a cross-validation study. The results of the study showed that unidimensional models of the reading and listening subtests and the German test provided a good overall model-data fit, however, multidimensional models of the subtests provided a better fit. The results demonstrated that, although the subtest scores also fit adequately independently, estimating the scores of the overall test with a model (e.g., bifactor) that includes a general factor (construct) in addition to the subfactors significantly improved the psychometric properties of the test. A general factor was identified that had the highest reliability values; however, the reliabilities of the specific factors were very low. In addition to the fit of the model data, the fit of the persons with IRT/MIRT models was also examined. The results showed that the proportion of person misfit was higher for the subtests than for the overall tests, but the overfit was lower. NA-German students, who did not speak German all-day, had the highest proportion of misfits with all models.

Examining the Robustness of the Graded Response and 2-Parameter Logistic Models to Violations of Construct Normality.

When fitting unidimensional item response theory (IRT) models, the population distribution of the latent trait (θ) is often assumed to be normally distributed. However, some psychological theories would suggest a nonnormal θ. For example, some clinical traits (e.g., alcoholism, depression) are believed to follow a positively skewed distribution where the construct is low for most people, medium for some, and high for few. Failure to account for nonnormality may compromise the validity of inferences and conclusions. Although corrections have been developed to account for nonnormality, these methods can be computationally intensive and have not yet been widely adopted. Previous research has recommended implementing nonnormality corrections when θ is not "approximately normal." This research focused on examining how far θ can deviate from normal before the normality assumption becomes untenable. Specifically, our goal was to identify the type(s) and degree(s) of nonnormality that result in unacceptable parameter recovery for the graded response model (GRM) and 2-parameter logistic model (2PLM).

[RETRACTED ARTICLE] irtplay: An R Package for Unidimensional Item Response Theory Modeling

Item response theory (IRT) is a general framework in which mathematical models are formulated to explain the relationship between an examinee's observable response on an item and the latent ability measured by a test. The application of IRT models and related statistical methods are commonly found in educational and psychological research. An important step in applying IRT models to test data is estimating the IRT model parameters. Accordingly, the successful application of IRT rests on the satisfactory statistical techniques and software for accurately estimating the model parameters. The irtplay R package was developed to provide users with a user-friendly experience and convenience when analyzing test data using unidimensional IRT models. The package can be used to fit the IRT models to a mixture of dichotomous and polytomous item data using marginal maximum likelihood estimation via the expectation-maximization, calibrate pretest items, and estimate examinees' latent ability parameters. In addition, the package provides practical tools that conveniently enable users to conduct many analyses related to IRT such as evaluating IRT model-data fit, analyzing differential item functioning, computing asymptotic variance-covariance matrices of item parameter estimates, calculating the conditional probability distribution of observed scores using the Lord and Wingersky (1984) formula, and importing item and ability parameter estimates from the output of popular IRT software. The main features of the irtplay package are illustrated using three data examples.

Item Response Theory and Differential Item Functioning Analyses With the Suicidal Behaviors Questionnaire-Revised in US and Chinese Samples.

Background: Despite the widespread use of the Suicidal Behaviors Questionnaire-Revised (SBQ-R) and advances in item response theory (IRT) modeling, item-level analysis with the SBQ-R has been minimal. Aims: This study extended IRT modeling strategies to examine the response parameters and potential differential item functioning (DIF) of the individual SBQ-R items in samples of US (N = 320) and Chinese (N = 298) undergraduate students. Method: Responses to the items were calibrated using the unidimensional graded response IRT model. Goodness-of-fit, item parameters, and DIF were evaluated. Results: The unidimensional graded response IRT model provided a good fit to the sample data. Results showed that the SBQ-R items had various item discrimination parameters and item severity parameters. Also, each SBQ-R item functioned similarly between the US and Chinese respondents. In particular, Item 1 (history of attempts) demonstrated high discrimination and severity of suicide-related thoughts and behaviors (STBs). Limitations: The use of cross-sectional data from convenience samples of undergraduate students could be considered a major limitation. Conclusion: The findings from the IRT analysis provided empirical support that each SBQ-R item taps into STBs and that scores for Item 1 can be used for screening purposes.

A Motivational-Developmental Free Response Assessment Through a Bifactor Lens.

The Motivational-Developmental Assessment (MDA) measures a university student’s motivational and developmental attributes by utilizing overlapping constructs measured across four writing prompts. The MDA’s format may lead to the violation of the local item independence (LII) assumption for unidimensional item response theory (IRT) scoring models, or the uncorrelated errors assumption for scoring models in classical test theory (CTT) due to the measurement of overlapping constructs within a prompt. This assumption violation is known as a testlet effect, which can be viewed as a method effect. The application of a unidimensional IRT or CTT model to score the MDA can result in imprecise parameter estimates when this effect is ignored. To control for this effect in the MDA responses, we first examined the presence of local dependence via a restricted bifactor model and Yen’s Q3 statistic. Second, we applied bifactor models to account for the testlet effect in the responses, as this effect is modeled as an additional latent variable in a factor model. Results support the presence of local dependence in two of the four MDA prompts, and the use of the restricted bifactor model to account for the testlet effect in the responses. Modeling the testlet effect through the restricted bifactor model supports a scoring inference in a validation argument framework. Implications are discussed.

Applicability and Efficiency of a Computerized Adaptive Test for the Washington Assessment of the Risks and Needs of Students.

The Washington Assessment of the Risks and Needs of Students (WARNS) is a computer-based assessment created to help courts, schools, and youth service providers determine an adolescent's risks and needs that may lead to truancy, drop out, or delinquency from school. Users are advised to consider the WARNS total score to work with youth. A total score estimate based on fewer items than the full item set may result in less respondent burden, administration time, and fatigue, while not hindering accurate decisions. This simulation study examined the applicability and efficiency of a computerized adaptive test (CAT) to estimate a WARNS total score under a unidimensional item response theory model. The results demonstrate that the CAT provides an accurate estimate of students' risks and needs and reduces the number of items administered for each examinee compared with the existing version. Future directions and limitations of CAT development with the WARNS are discussed.

A Guide for More Accurate and Precise Estimations in Simulative Unidimensional IRT Models

There is a great deal of research about item response theory (IRT) conducted by simulations. Item and ability parameters are estimated with varying numbers of replications under different test conditions. However, it is not clear what the appropriate number of replications should be. The aim of the current study is to develop guidelines for the adequate number of replications in conducting Monte Carlo simulation studies involving unidimensional IRT models. For this aim, 192 simulation conditions which included four sample sizes, two test lengths, eight replication numbers, and unidimensional IRT models were generated. Accuracy and precision of item and ability parameter estimations and model fit values were evaluated by considering the number of replications. In this context, for the item and ability parameters; mean error, root mean square error, standard error of estimates, and for model fit; M_2, 〖RMSEA〗_2, and Type I error rates were considered. The number of replications did not seem to influence the model fit, it was decisive in Type I error inflation and error prediction accuracy for all IRT models. It was concluded that to get more accurate results, the number of replications should be at least 625 in terms of accuracy of the Type I error rate estimation for all IRT models. Also, 156 replications and above can be recommended. Item parameter biases were examined, and the largest bias values were obtained from the 3PL model. It can be concluded that the increase in the number of parameters estimated by the model resulted in more biased estimates.

Measurement of Inter-Individual Variability in Assessing the Quality of Life in Respondents with Celiac Disease

Quality of life of Celiac Disease (CD) patients is affected by constraints in their physical, social and emotional behaviour. Our objective is to assess differences in two relevant dimensions of the Celiac Quality of Life (CQoL) scale, Limitations due to the disease and Dysphoria (i.e., feelings of depression and discomfort), in relation to the perceived social support and some individual and disease-related characteristics. The paper exploits suitable unidimensional Item Response Theory (IRT) models to individually analyse the two mentioned dimensions of the CQoL and Multidimensional Latent Class IRT models for ordinal polytomous items in order to detect sub-populations of CD patients that are homogenous with respect to the perceived CQoL. The latter methods allow to address patients with similar characteristics to the same treatment, performing at the same time a more tailored overture to health promotion programmes. The analysis extracts the relevant patterns and relations among CD patients, disentangling respondents receiving CD diagnosis in adolescence or adult age rather than in childhood (the first perceive high levels of Limitations and Dysphoria), patients with high perceived social support, a factor influencing in a positive way motivation to engage in management of CD-related distress and psychological well-being, and participants who are married or cohabiting. The latter report higher latent trait levels.