Stocking-Lord Methods Research Articles

Klasik Test Kuramı’na ve Madde Tepki Kuramı’na Dayalı Test Eşitleme Yöntemlerinin Karşılaştırılması

This study aims to choose the equating method with the least equating error by using the equating methods in Classical Test Theory and Item Response Theory. In this study, booklet 1 and booklet 3 data were used for PISA (Programme for International Student Assessment) 2012 Mathematics test. Data from Turkey, Indonesia, Shanghai/China and Finland, countries participating in PISA 2012, were selected for this study. Non-equivalent groups design was used in the test equating process. Linear equating methods [Tucker (w1=1, w1=0.5), Levine observed score (w1=1, w1=0.5), Levine true score, Classical Congeneric and Braun-Holland), equipercentile equating methods (pre smoothing according to C6 polynomial degree, beta4, post smoothing according to S 0.05 cubic function, frequency estimation (w1=1, w1=0.5) ] were used in the study. In Classical Test Theory, the least error is obtained from the frequency estimation method with a synthetic universe weight of w1 = 0.5. For the Item Response Theory, the calibration method was first decided, which is the Stocking-Lord method. After the scale transformation was achieved with the Stocking-Lord calibration method, the equating scores were calculated from the IRT's true and observed equating methods. The least error in IRT was obtained from the true score equating method. For error values, error coefficients were calculated according to Newton-Raphson's delta method and bootstrap methods. When the error coefficients (delta and bootstrap) of the equating methods in both theories were compared, it was found that the equating methods based on IRT had fewer errors than the equating methods in CTT, and the method with the least equating error was the IRT true score equating. The least equating error frequency estimation in CTT (w1=0.5) and the most error Levine true score equating method.

Investigation of Multidimensional Scale Transformation Methods Applied to Multidimensional Tests According to Various Conditions

The purpose of this study was to compare the equating errors of item and ability parameters obtained by performing scale transformation methods to two multidimensional test forms under various conditions. Sample size (1000-2000), common item ratio (20% and 40%), correlation between dimensions (0.1-0.5-0.9) and parameter estimation model (2PLM-3PLM) were taken as research conditions. Equating error (RMSE) was used to examine the accuracy of the scale transformation results. It was observed that the RMSE value generally decreased as the sample size and common item ratio increased and the correlation between dimensions decreased. Higher equating errors were obtained when the mean-sigma method was used. In the estimation of the discrimination parameter, lower RMSE values were obtained in 2PLM for all methods. In the estimation of difficulty and ability parameters, lower RMSE values were obtained in 2PLM for Stocking-Lord method and in 3PLM for mean-mean and mean-sigma methods.

Investigation of the Missing Data Imputation Methods on Characteristic Curve Transformation Methods Used in Test Equating

In this research, the aim was to evaluate the effect of zero imputation and multiple imputation missing data handling methods on item response theory (IRT) based test equating methods under different conditions. Data in this study was obtained from the administration of the TIMSS 2019 eighth-grade science test. Data sets were formed by randomly selecting a sample of 1000 students with full data from booklets 7 and 8. By deleting data under a completely random missing data mechanism within the scope of common-item nonequivalent groups (CINEG) design, four different data sets were obtained with the missing data rates of 10% or 20% in the new test or in both tests. The missing da ta problem was solved by using zero imputation and multiple imputation methods from these data sets. In this way, 8 different data sets were formed. Then, scaling transformation was performed by using characteristic curve transformation methods (Haebara, Stocking-Lord). Test equating results were reported in terms of observed scores. The root mean square error (RMSE) was used as the evaluation criterion to determine the error involved in test equating. As a result, it was determined that in the case of 10% missing data in both tests, generally lower RMSE values were obtained. It was observed that the multiple imputation method, one of the methods for handling missing data, was the method that produced RMSE values that were both the lowest and closer to the full data set as a reference value compared to the zero-imputation method. In addition, it was determined that, when compared to the Haebara method, Stocking-Lord method, one of the characteristic curve transformation methods, produced lower RMSE values and these values were closer to the full data set, which was taken as a reference value.

A Comparison of Score Equating Conducted Using Haebara and Stocking Lord Method for Polytomous

<p style="text-align:justify">The purposes of this research are: 1) to compare two equalizing tests conducted with Hebara and Stocking Lord method; 2) to describe the characteristics of each equalizing test method using windows’ IRTEQ program. This research employs a participatory approach as the data are collected through questionnaires based on the National Examination Administration of 2018. The samples are classified into group A and group B respectively by 449 and 502 respondents. This paper discusses how to equalize shared items using the anchor method with a set of instruments in the forms of 35 questionnaire items and 6 shared items. In addition, the researcher also uses PARSCALE to estimate each respondent’s skills and each item’s characteristics. The shared items are eventually equalized using IRTEQ program. The results show that there is a significant difference between those conducted using Haebara method (0.592) which produces bigger mean-sigma value and Stocking & Lord (0.00213). Thus, the results show that the shared testing items may improve respondents’ discrimination and increase the difficulty level (parameter b). Due to the availability of shared items, it is good and appropriate to equalize two different tests on different theta skills.</p>

Evaluating Different Equating Setups in the Continuous Item Pool Calibration for Computerized Adaptive Testing.

The increasing digitalization in the field of psychological and educational testing opens up new opportunities to innovate assessments in many respects (e.g., new item formats, flexible test assembly, efficient data handling). In particular, computerized adaptive testing provides the opportunity to make tests more individualized and more efficient. The newly developed continuous calibration strategy (CCS) from Fink et al. (2018) makes it possible to construct computerized adaptive tests in application areas where separate calibration studies are not feasible. Due to the goal of reporting on a common metric across test cycles, the equating is crucial for the CCS. The quality of the equating depends on the common items selected and the scale transformation method applied. Given the novelty of the CCS, the aim of the study was to evaluate different equating setups in the CCS and to derive practical recommendations. The impact of different equating setups on the precision of item parameter estimates and on the quality of the equating was examined in a Monte Carlo simulation, based on a fully crossed design with the factors common item difficulty distribution (bimodal, normal, uniform), scale transformation method (mean/mean, mean/sigma, Haebara, Stocking-Lord), and sample size per test cycle (50, 100, 300). The quality of the equating was operationalized by three criteria (proportion of feasible equatings, proportion of drifted items, and error of transformation constants). The precision of the item parameter estimates increased with increasing sample size per test cycle, but no substantial difference was found with respect to the common item difficulty distribution and the scale transformation method. With regard to the feasibility of the equatings, no differences were found for the different scale transformation methods. However, when using the moment methods (mean/mean, mean/sigma), quite extreme levels of error for the transformation constants A and B occurred. Among the characteristic curve method the performance of the Stocking-Lord method was slightly better than for the Haebara method. Thus, while no clear recommendation can be made with regard to the common item difficulty distribution, the characteristic curve methods turned out to be the most favorable scale transformation methods within the CCS.

Investigation of Equating Error in Tests with Differential Item Functioning

In this study purposes to indicate the effect of the number of DIF items and the distribution of DIF items in these forms, which be equalized on equating error. Mean-mean, mean-standard deviation, Haebara and Stocking-Lord Methods used in common item design equal groups as equalization methods. The study included six different simulation conditions. The conditions were compared according to the number of DIF items and the distribution of DIF items on tests. The results illustrated that adding DIF items to tests were equated caused an increase in the errors obtained by equating methods. We may state that the change in errors is lowest in characteristic curve transformation methods, largest in moment methods depending on the situations in these conditions.

Asymptotic Variance of Linking Coefficient Estimators for Polytomous IRT Models.

In item response theory (IRT), when two groups from different populations take two separate tests, there is a need to link the two ability scales so that the item parameters of the tests are comparable across the groups. To link the two scales, information from common items are utilized to estimate linking coefficients which place the item parameters on the same scale. For polytomous IRT models, the Haebara and Stocking-Lord methods for estimating the linking coefficients have commonly been recommended. However, estimates of the variance for these methods are not available in the literature. In this article, the asymptotic variance of linking coefficients for polytomous IRT models with the Haebara and Stocking-Lord methods are derived. The results are presented in a general form and specific results are given for the generalized partial credit model. Simulations which investigate the accuracy of the derivations under various settings of model complexity and sample size are provided, showing that the derivations are accurate under the conditions considered and that the Haebara and Stocking-Lord methods have superior performance to several moment methods with performance close to that of concurrent calibration.

Multiple Equating of Separate IRT Calibrations.

When test forms are calibrated separately, item response theory parameters are not comparable because they are expressed on different measurement scales. The equating process includes the conversion of item parameter estimates on a common scale and the determination of comparable test scores. Various statistical methods have been proposed to perform equating between two test forms. This paper provides a generalization to multiple test forms of the mean-geometric mean, the mean-mean, the Haebara, and the Stocking-Lord methods. The proposed methods estimate simultaneously the equating coefficients that permit the scale transformation of the parameters of all forms to the scale of the base form. Asymptotic standard errors of the equating coefficients are derived. A simulation study is presented to illustrate the performance of the methods.

Comparison of Item Response Theory Test Equating Methods for Mixed Format Tests

Article History: Received 30.10.2015 Received in revised form 25.12.2015 Accepted 09.01.2016 Available online 26.02.2016 This study aims to investigate the performance of test equating methods extended to mixed-format tests within the framework of Item Response Theory (IRT). To this end, a simulation study was conducted to compare equating errors of the mean/mean, mean/sigma, robust mean/sigma, Haebara, and Stocking-Lord methods under different conditions. Using 40-item tests, the effects of anchor length (10%, 20%, and 30%) and ability distribution (normal, negatively skewed, and positively skewed) were examined on a sample of 1000 participants. We used the common-item nonequivalent group design. The tests were developed using the three parameter logistic model for dichotomous simulated data and the generalized partial credit model for polytomous simulated data. The results of the study revealed that the robust mean/sigma method generally had the highest equating errors. When all conditions were evaluated, the least equating error occurred with the “Stocking-Lord” method in the case of positively skewed groups and a long anchor test (30%). Moreover, the results indicated that the groups with similar ability distributions (normal-normal, negatively skewednegatively skewed, and positively skewed-positively skewed) produced less equation errors than the groups with different ability distributions (negatively skewed-normal, positively skewed-normal, and positively skewed-negatively skewed). © 2016 IOJES. All rights reserved

New Robust Scale Transformation Methods in the Presence of Outlying Common Items.

Common items play an important role in item response theory (IRT) true score equating under the common-item nonequivalent groups design. Biased item parameter estimates due to common item outliers can lead to large errors in equated scores. Current methods used to screen for common item outliers mainly focus on the detection and elimination of those items, which may lead to inadequate content representation for the common items. To reduce the impact of inconsistency in item parameter estimates while maintaining content representativeness, the authors propose two robust scale transformation methods based on two weighting methods: the Area-Weighted method and the Least Absolute Values (LAV) method. Results from two simulation studies indicate that these robust scale transformation methods performed as well as the Stocking-Lord method in the absence of common item outliers and, more importantly, outperformed the Stocking-Lord method when a single outlying common item was simulated.

Tracking Functional Status Across the Spinal Cord Injury Lifespan: Linking Pediatric and Adult Patient-Reported Outcome Scores

ObjectiveTo use item response theory (IRT) methods to link scores from 2 recently developed contemporary functional outcome measures, the adult Spinal Cord Injury–Functional Index (SCI-FI) and the Pedi SCI (both the parent version and the child version). DesignSecondary data analysis of the physical functioning items of the adult SCI-FI and the Pedi SCI instruments. We used a nonequivalent group design with items common to both instruments and the Stocking-Lord method for the linking. Linking was conducted so that the adult SCI-FI and Pedi SCI scaled scores could be compared. SettingCommunity. ParticipantsThis study included a total sample of 1558 participants. Pedi SCI items were administered to a sample of children (n=381) with SCI aged 8 to 21 years, and of parents/caregivers (n=322) of children with SCI aged 4 to 21 years. Adult SCI-FI items were administered to a sample of adults (n=855) with SCI aged 18 to 92 years. InterventionsNot applicable. Main Outcome MeasuresFive scales common to both instruments were included in the analysis: Wheelchair, Daily Routine/Self-care, Daily Routine/Fine Motor, Ambulation, and General Mobility functioning. ResultsConfirmatory factor analysis and exploratory factor analysis results indicated that the 5 scales are unidimensional. A graded response model was used to calibrate the items. Misfitting items were identified and removed from the item banks. Items that function differently between the adult and child samples (ie, exhibit differential item functioning) were identified and removed from the common items used for linking. Domain scores from the Pedi SCI instruments were transformed onto the adult SCI-FI metric. ConclusionsThis IRT linking allowed estimation of adult SCI-FI scale scores based on Pedi SCI scale scores and vice versa; therefore, it provides clinicians with a means of tracking long-term functional data for children with an SCI across their entire lifespan.

A Comparison of IRT Linking Procedures

Various applications of item response theory often require linking to achieve a common scale for item parameter estimates obtained from different groups. This article used a simulation to examine the relative performance of four different item response theory (IRT) linking procedures in a random groups equating design: concurrent calibration with multiple groups, separate calibration with the Stocking-Lord method, separate calibration with the Haebara method, and proficiency transformation. The simulation conditions used in this article included three sampling designs, two levels of sample size, and two levels of the number of items. In general, the separate calibration procedures performed better than the concurrent calibration and proficiency transformation procedures, even though some inconsistent results were observed across different simulation conditions. Some advantages and disadvantages of the linking procedures are discussed.

Effects on Scale Linking of Different Definitions of Criterion Functions for the IRT Characteristic Curve Methods

Under item response theory, the characteristic curve methods (Haebara and Stocking-Lord methods) are used to link two ability scales from separate calibrations. The linking methods use their respective criterion functions that can be defined differently according to the symmetry- and distribution-related schemes. The symmetry-related scheme relates to which scale, targeted or transformed, should be used for the definition. The distribution-related scheme refers to a way of incorporating underlying ability distributions into the definition. Through simulations, this study examined if a certain optimal combination of the two schemes exists regardless of differences in proficiency distribution between samples involved in scale linking. Concurrent calibration was considered for comparative purposes and, across all nine combinations of proficiency distributions to be linked, its performance was better in linking accuracy than the linking methods. There was no optimal combination of the symmetry- and distribution-related schemes that led to minimal linking error across the nine combinations.