Equipercentile Equating Research Articles

Using Equipercentile Equating to Link Scores of the CBCL and SDQ in Residential Youth

ABSTRACT In data-driven residential care settings, it is sometimes necessary to change an established assessment tool due to cost and respondent burden. However, accurate assessment of youth functioning over time or across samples can be compromised when different assessments are used. Hence, the ability to crosswalk between assessments is important. The Child Behavior Checklist (CBCL) and Strengths and Difficulties Questionnaire (SDQ) screen for similar attributes and are highly correlated. The SDQ is shorter and free and can be an alternative to the CBCL. This study links the CBCL and SDQ total problem scores in a sample of 284 high-risk youth in a residential care facility, using equipercentile equating. Youth had both assessments completed on them concurrently by caregivers. The “equate” package in R Studio was used to conduct the analysis. Concordance tables were created for crosswalks from CBCL to SDQ and vice versa. Results showed linked scores that were consistent when equating from each assessment to the other. The concordance tables can serve as a comparison tool between total problem scores of each assessment. In particular, they could be a resource for residential care providers and researchers considering a switch from the CBCL to the SDQ as a resource-effective alternative.

Test Equating by Common Items and Common Subjects: Concepts and Applications

Since the invention of z-scores (standardized scores), comparison among different tests has been widely conducted by test developers, instructors, educational researchers, and psychometricians. Equating, calibration, and moderation are terms used to describe broad levels of possible comparison among educational assessments (Dorans, 2004; Feuer, Holland, Green, Bertenthal, & Hemphill, 1999; Linn, 1993; Mislevy, 1992). Equating is at one end of the linking continuum, involving the most stringent requirements of equivalence among the assessments and examinee populations to be linked, and compares tests that measure the same construct and have been designed to be equivalent. Less equivalent conditions involve calibration, which compares tests that measure the same construct but vary in design or difficulty, and moderation, which compares tests that measure different constructs. Psychometric approaches to linking assessments include linear equating, equipercentile equating, and item response theory (IRT). This article is a practical guide to conducting IRT test equating in two different scenarios:

Conversion From The Korean Dementia Screening Questionnaire To The Informant Questionnaire On Cognitive Decline In The Elderly.

PurposeThe ability to convert scores between cognitive measurements would facilitate the longitudinal assessment of cognition in clinical practice and the comparison and synthesis of cognitive data from international, multicenter, or longitudinal studies. The primary aim of this study was to apply a simple and reliable method for converting scores from the Korean Dementia Screening Questionnaire (KDSQ) to those of the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE).Patients and methodsA total of 627 participants, with and without cognitive dysfunction, received both the KDSQ and the IQCODE at the same visit. The scores of both tools were calculated, and equipercentile equating was used to create a method for converting scores from the KDSQ to the IQCODE.ResultsKDSQ scores were highly correlated with IQCODE scores (Pearson r = 0.905, P < 0.01). We developed scores for converting the KDSQ to the IQCODE using equipercentile equating and log-linear smoothing. We provide an easy-to-use table that enables the conversion of KDSQ scores to IQCODE scores.ConclusionWe delivered a simple and reliable method for converting scores from the KDSQ to the IQCODE. The conversion score table reported here enables direct and easy comparison of these cognitive measurements in older adults.

Prevalence of anxiety in patients with an implantable cardioverter defibrillator: measurement equivalence of the HADS-A and the STAI-S

PurposeThe Hospital Anxiety and Depression Scale (HADS-A) and State-Trait Anxiety Inventory (STAI-S) are popular instruments for assessing anxiety and are considered interchangeable, although little is known about their equivalence. Hence, we examined whether the two instruments are (i) equivalent with respect to determining the prevalence of probable clinical anxiety levels and (ii) reflect variation on a common anxiety attribute.MethodsScore and construct concordance were evaluated using equipercentile equating and bifactor modeling, respectively. Secondary data from the WEBCARE trial and the MIDAS study were used for the current study, where patients implanted with a first-time implantable cardioverter defibrillator completed both the HADS-A and the STAI-S within 10 days post implant.ResultsData from 710 patients were included in the analyses. Results showed that the STAI-S produced a higher prevalence rate than the HADS-A (39% vs. 23%). A crosswalk table was generated with equivalent scores and cutoffs for the HADS-A and STAI-S, respectively. Bifactoring suggested that HADS-A and STAI-S largely tapped into the same generic anxiety attributes.ConclusionsSTAI-S and HADS-A reflect a common anxiety attribute, but using the recommended cutoff scores on the respective measures show very different prevalence rates and would classify patients as anxious with the STAI-S who would not be identified as such with the HADS-A. Clinicians and researchers should be aware of the inequivalence when using these measures for screening and determining the prevalence of probable clinical anxiety levels.

A comparison of linear and equipercentile equating and IRT equating with FIPC across multidimensional test forms for non-equivalent groups

The aim of this study was to compare linear equating (Levine and Tucker), equipercentile equating, and true and observed score IRT equating with fixed item parameter calibration methods when applied to non-equivalent groups taking forms which have various multidimensional structures: equal or unequal total test difficulty and similar or dissimilar difficulty within dimensions across forms. This situation may be common for large-scale test forms that are composed of multiple sub-content areas and are being administered to examinees of mixed abilities at different times. Within the specifications of this study, when forms differ in total difficulty but the difference in difficulty within dimensions is consistent across forms, the linear methods may be preferred. When forms differ in average item difficulty within dimensions, regardless of equal or unequal total test difficulty, the IRT method with fixed item parameter calibration is favoured when data are correlated.

A comparison of linear and equipercentile equating and IRT equating with FIPC across multidimensional test forms for non-equivalent groups

The aim of this study was to compare linear equating (Levine and Tucker), equipercentile equating, and true and observed score IRT equating with fixed item parameter calibration methods when applied to non-equivalent groups taking forms which have various multidimensional structures: equal or unequal total test difficulty and similar or dissimilar difficulty within dimensions across forms. This situation may be common for large-scale test forms that are composed of multiple sub-content areas and are being administered to examinees of mixed abilities at different times. Within the specifications of this study, when forms differ in total difficulty but the difference in difficulty within dimensions is consistent across forms, the linear methods may be preferred. When forms differ in average item difficulty within dimensions, regardless of equal or unequal total test difficulty, the IRT method with fixed item parameter calibration is favoured when data are correlated.

A Review of Test Equating Methods with a Special Focus on IRT-Based Approaches

The overall aim of this work is to review test equating methods with a particularly detailed description of item response theory (IRT) equating. Test score equating is used to compare different test scores from different test forms. Several methods have been developed to conduct equating: traditional methods, kernel method, and IRT equating. We synthetically explain the traditional equating methods which include mean equating, linear equating and equipercentile equating and which have been developed under all the possible data collection designs. We also briefly describe the idea of the kernel method: this is a unified approach to test equating for which recent interesting developments have been proposed. Then we focus on IRT equating, by describing old and new methods: in particular, we define IRT observed-score kernel equating and IRT observed-score equating using covariates, as well as other recent proposals in this field. We conclude the review by describing strengths and weaknesses of the different discussed approaches and by identifying future research topics.

Mapping PROMIS Global Health Items to EuroQol (EQ-5D) Utility Scores Using Linear and Equipercentile Equating.

Mapping Patient-Reported Outcomes Measurement Information System-Global Health (PROMIS-GH) to EuroQol 5-dimension, three-level version (EQ-5D-3L) provides a utility score for use in quality-of-life and cost-effectiveness analyses. In 2009, Revicki et al. mapped the PROMIS-GH items to EQ-5D-3L utilities using linear regression (REVReg). More recently, regression was shown to be ill-suited for mapping to preference-based measures due to regression to the mean. Linear and equipercentile equating are alternative mapping methods that avoid the issue of regression to the mean. Another limitation of the prior models is that ordinal predictors were treated as continuous. Using data collected from the PROMIS Wave 1 sample, we refit REVReg, treating the PROMIS-GH items as categorical variables (CATReg). We applied linear and equipercentile equating to the REVReg model (REVLE, REVequip) and the CATReg model (CATLE, CATequip). We validated and compared the predictive accuracy of these models in a large sample of neurological patients at a single tertiary-care hospital. In the neurological disease patient sample, CATLE produced the strongest correlations between estimated and observed EQ-5D-3L scores and had the lowest mean squared error. The CATequip model had the lowest mean absolute error and had estimated scores that best matched the overall distribution of observed scores. Using linear and equipercentile equating, we created new models mapping PROMIS-GH items to EQ-5D-3L utility scores. EQ-5D-3L utility scores can be more accurately estimated using our models for use in cost-effectiveness studies or studies examining overall health-related quality of life.

Asymptotic Standard Errors of Observed‐Score Equating With Polytomous IRT Models

In observed‐score equipercentile equating, the goal is to make scores on two scales or tests measuring the same construct comparable by matching the percentiles of the respective score distributions. If the tests consist of different items with multiple categories for each item, a suitable model for the responses is a polytomous item response theory (IRT) model. The parameters from such a model can be utilized to derive the score probabilities for the tests and these score probabilities may then be used in observed‐score equating. In this study, the asymptotic standard errors of observed‐score equating using score probability vectors from polytomous IRT models are derived using the delta method. The results are applied to the equivalent groups design and the nonequivalent groups design with either chain equating or poststratification equating within the framework of kernel equating. The derivations are presented in a general form and specific formulas for the graded response model and the generalized partial credit model are provided. The asymptotic standard errors are accurate under several simulation conditions relating to sample size, distributional misspecification and, for the nonequivalent groups design, anchor test length.

Results From the NACC Uniform Data Set Neuropsychological Battery Crosswalk Study.

Four new nonproprietary tests were recommended for use in the National Alzheimer's Coordinating Center's Uniform Data Set Neuropsychological Battery. These tests are similar to previous tests but also allow for continuity of longitudinal data collection and wide dissemination among research collaborators. A Crosswalk Study was conducted in early 2014 to assess the correlation between each set of new and previous tests. Tests with good correlation were equated using equipercentile equating. The resulting conversion tables allow scores on the new tests to be converted to equivalent scores on the previous tests. All pairs of tests had good correlation (ρ=0.68 to 0.78). Learning effects were detected for Logical Memory only. Confidence intervals were narrow at each point estimate, and prediction accuracy was high. The recommended new tests are well correlated with the previous tests. The equipercentile equating method produced conversion tables that provide a useful reference for clinicians and researchers.

An Evaluation of the Single‐Group Growth Model as an Alternative to Common‐Item Equating

AbstractAs an alternative to common‐item equating when common items do not function as expected, the single‐group growth model (SGGM) scaling uses common examinees or repeaters to link test scores on different forms. The SGGM scaling assumes that, for repeaters taking adjacent administrations, the conditional distribution of scale scores in later administration, given the scale score in earlier administration, should generalize from previous repeaters to the repeaters taking the current administration. The current repeaters' scale score distribution is estimated from their earlier scale score distribution. The SGGM scaling first uses previous repeaters' data to estimate the conditional distribution of their later scale scores, given their earlier scale scores. Then the repeaters taking both the current and previous administrations are identified, and their scale score distribution on the current form is estimated based on their previous scale score distribution and the estimated conditional distribution. Finally, a single‐group equipercentile equating is performed between the current‐form repeaters' observed raw score distribution and their estimated scale score distribution to obtain the raw‐to‐scale score conversion. This study evaluated the SGGM scaling performance using the common‐item equating results for a language test as the criterion. The study found that the raw‐to‐scale conversions based on SGGM scaling differed from those based on common‐item equating. However, the SGGM scaling results did not show a systematic bias in either the average or the variability of examinees' scale scores.

The Montreal Cognitive Assessment: Creating a Crosswalk with the Mini-Mental State Examination.

To establish Montreal Cognitive Assessment (MoCA) scores that correspond to well-established cut-points on the Mini-Mental State Examination (MMSE). Cross-sectional observational study. General medical service of a large teaching hospital. Individuals aged 75 and older (N = 199; mean age 84, 63% female). The MoCA (range 0-30) and the MMSE (range 0-30) were administered within 2 hours of each other. The Abbreviated MoCA (A-MoCA; range 0-22) was calculated from the full MoCA. Scores from the three tests were analyzed using equipercentile equating, a statistical method for determining comparable scores on different tests of a similar construct by estimating percentile equivalents. MoCA scores were lower (mean 19.3 ± 5.8) than MMSE scored (mean 24.1 ± 6.6). Traditional MMSE cut-points of 27 for mild cognitive impairment and 23 for dementia corresponded to MoCA scores of 23 and 17, respectively. Scores on the full and abbreviated versions of the MoCA can be linked directly to the MMSE. The MoCA may be more sensitive to changes in cognitive performance at higher levels of functioning.

Comparing apples with oranges? An approach to link TIMSS and the National Educational Panel Study in Germany via equipercentile and IRT methods

The aim of this linking study is to classify the National Educational Panel Study (NEPS) according to the criterion based reference frame of the Trends in International Mathematics and Science Study (TIMSS). One requirement for a strong linking is a high conceptual overlap of the frameworks. The results of the comparison between NEPS and TIMSS indicate a high overlap. To link the studies the equipercentile equating and an IRT linking approach are compared. The results show that both methods showed similar descriptive statistics and satisfying classification consistency. Nevertheless, the equipercentile equating has small advantages and is therefore more suitable. Thus, inferences are possible on a population level, but the results should not be reported or interpreted on an individual level.

The Comparison of Linear, Parallel-Linear, and Equipercentile Equating Methods in the Context of English Tests

This study investigated the adequacy of linking the English test of ITED and the ACT score by using linear, parallel-linear, and equipercentile methods. The results show that the group invariance is satisfied for ACT-E and ITED-EX using the parallel linear or equipercentile method. The reason that this property can hold is probably the content specification of the two tests are very similar to each other. But it does not mean that the score can be used interchangeably of ACT-E and ITED-EX score in practice. Discussions of the results are provided in the end.

Conversion between mini-mental state examination, montreal cognitive assessment, and dementia rating scale-2 scores in Parkinson's disease.

Cognitive impairment is one of the earliest, most common, and most disabling non-motor symptoms in Parkinson's disease (PD). Thus, routine screening of global cognitive abilities is important for the optimal management of PD patients. Few global cognitive screening instruments have been developed for or validated in PD patients. The Mini-Mental State Examination (MMSE), Montreal Cognitive Assessment (MoCA), and Dementia Rating Scale-2 (DRS-2) have been used extensively for cognitive screening in both clinical and research settings. Determining how to convert the scores between instruments would facilitate the longitudinal assessment of cognition in clinical settings and the comparison and synthesis of cognitive data in multicenter and longitudinal cohort studies. The primary aim of this study was to apply a simple and reliable algorithm for the conversion of MoCA to MMSE scores in PD patients. A secondary aim was to apply this algorithm for the conversion of DRS-2 to both MMSE and MoCA scores. The cognitive performance of a convenience sample of 360 patients with idiopathic PD was assessed by at least two of these cognitive screening instruments. We then developed conversion scores between the MMSE, MoCA, and DRS-2 using equipercentile equating and log-linear smoothing. The conversion score tables reported here enable direct and easy comparison of three routinely used cognitive screening assessments in PD patients.

Cross-Validation

The development of the kernel equating (KE) method enhanced the theory of observed-score equating. In KE, discrete test score distributions are converted into continuous distributions through the use of a Gaussian kernel. Traditionally, the optimal bandwidth for a Gaussian kernel was obtained by minimizing a penalty function. In this article, an alternative bandwidth-selection approach for KE was adopted that uses cross-validation (CV) techniques. The method is illustrated through simulations that were conducted with 188 conditions by varying three factors known to influence equating results; these include sample sizes, score distributions, and methods that involve both equating and bandwidth-selection methods. Four equating procedures were considered: traditional equipercentile equating, which uses linear interpolation to make the test distributions continuous; KE with penalty functions; and KE with two newly proposed CV methods. The results were evaluated based on four criteria: bias in continuizing the distributions (i.e., the difference between the estimated and underlying score distributions), the standard error of equating (SEE), the difference between equated scores, and percent relative error (PRE). Overall, the results demonstrate that KE with the two CV methods outperformed the others—the estimated density functions were less biased and the SEEs and PREs were smaller. Equating differences between the different methods were produced, although they were not large. In addition, the bias issues surrounding kernel methods on sample sizes and the shapes of the distributions were addressed and discussed.

THE KERNEL LEVINE EQUIPERCENTILE OBSERVED‐SCORE EQUATING FUNCTION

ABSTRACTIn the framework of the observed‐score equating methods for the nonequivalent groups with anchor test design, there are 3 fundamentally different ways of using the information provided by the anchor scores to equate the scores of a new form to those of an old form. One method uses the anchor scores as a conditioning variable, such as the Tucker method and poststratification equating. A second way to use the anchor scores is as the middle link in a chain of linking relationships, such as chain linear equating and chain equating. The third way to use the anchor scores is in conjunction with the classical test theory, such as Levine observed‐score equating and the newly created hybrid Levine equipercentile equating and poststratification equating based on true anchor scores. The purpose of this paper is to demonstrate that with real data, under certain conditions, hybrid Levine equipercentile equating and poststratification equating based on true anchor scores outperform both poststratification equating and chain equating.

Situations Where It Is Appropriate to Use Frequency Estimation Equipercentile Equating

In operational equating situations, frequency estimation equipercentile equating is considered only when the old and new groups have similar abilities. The frequency estimation assumptions are investigated in this study under various situations from both the levels of theoretical interest and practical use. It shows that frequency estimation equating can be used under circumstances when it is not normally used. To link theoretical results with practice, statistical methods are proposed for checking frequency estimation assumptions based on available data: observed‐score distributions and item difficulty distributions of the forms. In addition to the conventional use of frequency estimation equating when the group abilities are similar, three situations are identified when the group abilities are dissimilar: (a) when the two forms and the observed conditional score distributions are similar the two forms and the observed conditional score distributions are similar (in this situation, the frequency estimation equating assumptions are likely to hold, and frequency estimation equating is appropriate); (b) when forms are similar but the observed conditional score distributions are not (in this situation, frequency estimation equating is not appropriate); and (c) when forms are not similar but the observed conditional score distributions are (frequency estimation equating is not appropriate). Statistical analysis procedures for comparing distributions are provided. Data from a large‐scale test are used to illustrate the use of frequency estimation equating when the group difference in ability is large.

Longitudinal Stability of IRT and Equivalent-Groups Linear and Equipercentile Equating

A multiscale criterion-referenced test that featured two presumably equivalent forms (A and B), was administered to 1,667 Head Start children at each of four points over an academic year. Using a randomly equivalent groups design, three equating methods were applied: common-item IRT equating using concurrent calibration, linear transformation, and equipercentile transformation. The methods were compared by examining mean score differences, weighted mean squared difference, and Kolmogorov's D statistics for each subscale. The results indicated that over time the IRT equating method and conventional equating methods exhibited different patterns of discrepancy between the two test forms. IRT equating yielded marginally smaller form-to-form mean score differences and generated slightly fewer distributional discrepancies between Forms A and B than both linear and equipercentile equating. However, the results were mixed indicating that more studies are needed to provide additional information on the relative merits and weaknesses of each approach.

Observed Score and True Score Equating Procedures for Multidimensional Item Response Theory

The purpose of this research was to develop observed score and true score equating procedures to be used in conjunction with the multidimensional item response theory (MIRT) framework. Three equating procedures—two observed score procedures and one true score procedure—were created and described in detail. One observed score procedure was presented as a direct extension of unidimensional IRT (UIRT) observed score equating and is referred to as the “Full MIRT Observed Score Equating Procedure.” The true score procedure and the second observed score procedure incorporated unidimensional approximation procedures to equate exams using UIRT equating principles. These procedures are referred to as the “Unidimensional Approximation of MIRT True Score Equating Procedure” and the “Unidimensional Approximation of MIRT Observed Score Equating Procedure,” respectively. Three exams were used to conduct UIRT observed score and true score equating, MIRT observed score and true score equating, and equipercentile equating. The equipercentile equating procedure was conducted for the purpose of comparison because this procedure does not explicitly violate the IRT assumption of unidimensionality. Results indicated that the MIRT equating procedures performed more similarly to the equipercentile equating procedure than the UIRT equating procedures, presumably due to the violation of the unidimensionality assumption under the UIRT equating procedures.