One-parameter Item Response Theory Research Articles

Algebraic knowledge for teaching test: An adaptation study

In this study, the Mathematical Knowledge for Teaching-Elementary Patterns Functions and Algebra-Content Knowledge (MKT-PFA) test, originally developed in English as part of the "Learning Mathematics for Teaching Project" at Michigan University, was adapted into Turkish. The test comprises two equivalent forms, A and B, each translated into Turkish and culturally adapted through consultations with two mathematics education academics and five secondary school math teachers pursuing doctoral studies. A total of 328 pre-service teachers at a Turkish public university's elementary school mathematics teaching department were administered form A (14 questions, 29 items) and form B (12 questions, 27 items) at a one-week interval. Psychometric analyses revealed high reliability (KR-20: A=0.712, B=0.735; Lord reliability: A=0.733, B=0.756), and strong correlations (rpbi) with the original English forms, indicating suitable adaptation. Item difficulties analyzed using a one-parameter Item Response Theory model showed a normal distribution, affirming the tests' validity for assessing pre-service teachers' algebra teaching knowledge in Türkiye.

Exploration and analysis of a generalized one-parameter item response model with flexible link functions.

This paper primarily analyzes the one-parameter generalized logistic (1PGlogit) model, which is a generalized model containing other one-parameter item response theory (IRT) models. The essence of the 1PGlogit model is the introduction of a generalized link function that includes the probit, logit, and complementary log-log functions. By transforming different parameters, the 1PGlogit model can flexibly adjust the speed at which the item characteristic curve (ICC) approaches the upper and lower asymptote, breaking the previous constraints in one-parameter IRT models where the ICC curves were either all symmetric or all asymmetric. This allows for a more flexible way to fit data and achieve better fitting performance. We present three simulation studies, specifically designed to validate the accuracy of parameter estimation for a variety of one-parameter IRT models using the Stan program, illustrate the advantages of the 1PGlogit model over other one-parameter IRT models from a model fitting perspective, and demonstrate the effective fit of the 1PGlogit model with the three-parameter logistic (3PL) and four-parameter logistic (4PL) models. Finally, we demonstrate the good fitting performance of the 1PGlogit model through an analysis of real data.

A Performance-Based Competency Assessment of Pediatric Chest Radiograph Interpretation Among Practicing Physicians.

There is limited knowledge on pediatric chest radiograph (pCXR) interpretation skill among practicing physicians. We systematically determined baseline interpretation skill, the number of pCXR cases physicians required complete to achieve a performance benchmark, and which diagnoses posed the greatest diagnostic challenge. Physicians interpreted 434 pCXR cases via a web-based platform until they achieved a performance benchmark of 85% accuracy, sensitivity, and specificity. Interpretation difficulty scores for each case were derived by applying one-parameter item response theory to participant data. We compared interpretation difficulty scores across diagnostic categories and described the diagnoses of the 30% most difficult-to-interpret cases. 240 physicians who practice in one of three geographic areas interpreted cases, yielding 56,833 pCXR case interpretations. The initial diagnostic performance (first 50 cases) of our participants demonstrated an accuracy of 68.9%, sensitivity of 69.4%, and a specificity of 68.4%. The median number of cases completed to achieve the performance benchmark was 102 (interquartile range 69, 176; min, max, 54, 431). Among the 30% most difficult-to-interpret cases, 39.2% were normal pCXR and 32.3% were cases of lobar pneumonia. Cases with a single trauma-related imaging finding, cardiac, hilar, and diaphragmatic pathologies were also among the most challenging. At baseline, practicing physicians misdiagnosed about one-third of pCXR and there was up to an eight-fold difference between participants in number of cases completed to achieve the standardized performance benchmark. We also identified the diagnoses with the greatest potential for educational intervention.

Psychometric Assessment of Pilot Language and Communication Items on the 2018 and 2019 National Survey of Children's Health.

Until recently, normative data on language and communication development among children in the United States have not been available to inform critical efforts to promote language development and prevent impairments. This study represents the first psychometric assessment of nationally representative data derived from a National Survey of Children's Health (NSCH) pilot measure of language and communication development among children ages 1 to 5 years. We analyzed 14,573 parent responses to language and communication items on the 2018 and 2019 NSCH to evaluate whether the newly added 11 items represent a single latent trait for language and communication development and to determine normative age of success on each item. We applied weighted, one-parameter Item Response Theory to rate and cluster items by difficulty relative to developmental language ability. We examined differential item functioning (DIF) using weighted logistic regression by demographic factors. Together, exploratory factor analysis resulting in a single factor > 1 and explaining 93% of the variance and positive correlations indicated unidimensionality of the measure. Item characteristic curves indicated groupings were overall concordant with proposed milestone ages and representative of an approximate 90% success cut-point by child age. Indicated normative age cut-points for 3 of the items differed slightly from proposed milestone ages. Uniform DIF was not observed and potential nonuniform DIF was observed across 5 items. Results have the potential to enhance understanding of risk and protective factors, inform efforts to promote language and communication development, and guide programmatic efforts on early detection of language delays.

Pediatric Musculoskeletal Radiographs: Anatomy and Fractures Prone to Diagnostic Error Among Emergency Physicians

BackgroundPediatric musculoskeletal (pMSK) radiograph interpretations are common, but the specific radiograph features at risk of incorrect diagnosis are relatively unknown. ObjectiveWe determined the radiograph factors that resulted in diagnostic interpretation challenges for emergency physicians (EPs) reviewing pMSK radiographs. MethodsEPs interpreted 1850 pMSK radiographs via a web-based platform and we derived interpretation difficulty scores for each radiograph in 13 body regions using one-parameter item response theory. We compared the difficulty scores by presence or absence of a fracture and, where applicable, by fracture location and morphology; significance was adjusted for multiple comparisons. An expert panel reviewed the 65 most commonly misdiagnosed fracture-negative radiographs to identify imaging features mistaken for fractures. ResultsWe included data from 244 EPs, which resulted in 185,653 unique interpretations. For elbow, forearm, wrist, femur, knee, and tibia-fibula radiographs, those without a fracture had higher interpretation difficulty scores relative to those with a fracture; the opposite was true for the hand, pelvis, foot, and ankle radiographs (p < 0.004 for all comparisons). The descriptive review demonstrated that specific normal anatomy, overlapping bones, and external artefact from muscle or skin folds were often mistaken for fractures. There was a significant difference in difficulty score by anatomic locations of the fracture in the elbow, pelvis, and ankle (p < 0.004 for all comparisons). Ankle and elbow growth plate, fibular avulsion, and humerus condylar fractures were more difficult to diagnose than other fracture patterns (p < 0.004 for all comparisons). ConclusionsWe identified actionable learning opportunities in pMSK radiograph interpretation for EPs.

A Target Population Derived Method for Developing a Competency Standard in Radiograph Interpretation

Construct For assessing the skill of visual diagnosis such as radiograph interpretation, competency standards are often developed in an ad hoc method, with a poorly delineated connection to the target clinical population. Background Commonly used methods to assess for competency in radiograph interpretation are subjective and potentially biased due to a small sample size of cases, subjective evaluations, or include an expert-generated case-mix versus a representative sample from the clinical field. Further, while digital platforms are available to assess radiograph interpretation skill against an objective standard, they have not adopted a data-driven competency standard which informs educators and the public that a physician has achieved adequate mastery to enter practice where they will be making high-stakes clinical decisions. Approach Operating on a purposeful sample of radiographs drawn from the clinical domain, we adapted the Ebel Method, an established standard setting method, to ascertain a defensible, clinically relevant mastery learning competency standard for the skill of radiograph interpretation as a model for deriving competency thresholds in visual diagnosis. Using a previously established digital platform, emergency physicians interpreted pediatric musculoskeletal extremity radiographs. Using one-parameter item response theory, these data were used to categorize radiographs by interpretation difficulty terciles (i.e. easy, intermediate, hard). A panel of emergency physicians, orthopedic surgeons, and plastic surgeons rated each radiograph with respect to clinical significance (low, medium, high). These data were then used to create a three-by-three matrix where radiographic diagnoses were categorized by interpretation difficulty and significance. Subsequently, a multidisciplinary panel that included medical and parent stakeholders determined acceptable accuracy for each of the nine cells. An overall competency standard was derived from the weighted sum. Finally, to examine consequences of implementing this standard, we reported on the types of diagnostic errors that may occur by adhering to the derived competency standard. Findings To determine radiograph interpretation difficulty scores, 244 emergency physicians interpreted 1,835 pediatric musculoskeletal extremity radiographs. Analyses of these data demonstrated that the median interpretation difficulty rating of the radiographs was −1.8 logits (IQR −4.1, 3.2), with a significant difference of difficulty across body regions (p < 0.0001). Physician review classified the radiographs as 1,055 (57.8%) as low, 424 (23.1%) medium or 356 (19.1%) high clinical significance. The multidisciplinary panel suggested a range of acceptable scores between cells in the three-by-three table of 76% to 95% and the sum of equal-weighted scores resulted in an overall performance-based competency score of 85.5% accuracy. Of the 14.5% diagnostic interpretation errors that may occur at the bedside if this competency standard were implemented, 9.8% would be in radiographs of low-clinical significance, while 2.5% and 2.3% would be in radiographs of medium or high clinical significance, respectively. Conclusion(s) This study’s novel integration of radiograph selection and a standard setting method could be used to empirically drive evidence-based competency standard for radiograph interpretation and can serve as a model for deriving competency thresholds for clinical tasks emphasizing visual diagnosis.

A comparison of Monte Carlo methods for computing marginal likelihoods of item response theory models

Nowadays, Bayesian methods are routinely used for estimating parameters of item response theory (IRT) models. However, the marginal likelihoods are still rarely used for comparing IRT models due to their complexity and a relatively high dimension of the model parameters. In this paper, we review Monte Carlo (MC) methods developed in the literature in recent years and provide a detailed development of how these methods are applied to the IRT models. In particular, we focus on the “best possible” implementation of these MC methods for the IRT models. These MC methods are used to compute the marginal likelihoods under the one-parameter IRT model with the logistic link (1PL model) and the two-parameter logistic IRT model (2PL model) for a real English Examination dataset. We further use the widely applicable information criterion (WAIC) and deviance information criterion (DIC) to compare the 1PL model and the 2PL model. The 2PL model is favored by all of these three Bayesian model comparison criteria for the English Examination data.

Fairness Principle in Accreditation of Health Specialists: The Differential Item Functioning Method

The main purpose of this article is to present a Differential Item Functioning method of item analysis. It is designed to minimize the discriminatory effect of individual items in the accreditation of graduates of medical universities with different training programs. The one-parameter Item Response Theory model is used to align graduates’ rights in accreditation. The measure of the difference in the location of the item characteristic curves constructed for different graduates’ samples is presented using the difficulty estimates. For the development of the methodology, a number of research questions are posed, the solution of which made it possible to analyze the items bias for the “Polyclinic” discipline and the range of item difficulties from 1.5 to 2, 5 logits. The item bank was cleaned and an interpretation of decisions on items release and correction for the cases of different arrangement of their characteristic curves was presented.

Visual acuity and contrast sensitivity are two important factors affecting vision-related quality of life in advanced age-related macular degeneration.

PurposeVision loss from age-related macular degeneration (AMD) has a profound effect on vision-related quality of life (VRQoL). The pupose of this study is to identify clinical factors associated with VRQoL using the Rasch- calibrated NEI VFQ-25 scales in bilateral advanced AMD patients.MethodsWe retrospectively reviewed 47 patients (mean age 83.2 years) with bilateral advanced AMD. Clinical assessment included age, gender, type of AMD, high contrast visual acuity (VA), history of medical conditions, contrast sensitivity (CS), central visual field loss, report of Charles Bonnet Syndrome, current treatment for AMD and Rasch-calibrated NEI VFQ-25 visual function and socioemotional function scales. The NEI VFQ visual function scale includes items of general vision, peripheral vision, distance vision and near vision-related activity while the socioemotional function scale includes items of vision related-social functioning, role difficulties, dependency, and mental health. Multiple regression analysis (structural regression model) was performed using fixed item parameters obtained from the one-parameter item response theory model.ResultsMultivariate analysis showed that high contrast VA and CS were two factors influencing VRQoL visual function scale (β = -0.25, 95% CI-0.37 to -0.12, p<0.001 and β = 0.35, 95% CI 0.25 to 0.46, p<0.001) and socioemontional functioning scale (β = -0.2, 95% CI -0.37 to -0.03, p = 0.023, and β = 0.3, 95% CI 0.18 to 0.43, p = 0.001). Central visual field loss was not assoicated with either VRQoL visual or socioemontional functioning scale (β = -0.08, 95% CI-0.28 to 0.12,p = 0.44 and β = -0.09, 95% CI -0.03 to 0.16, p = 0.50, respectively).ConclusionIn patients with vision impairment secondary to bilateral advanced AMD, high contrast VA and CS are two important factors affecting VRQoL.

Measuring cancer caregiver health literacy: Validation of the Health Literacy of Caregivers Scale-Cancer (HLCS-C) in an Australian population.

Caregivers have been largely neglected in health literacy measurement. We assess the construct validity, and internal consistency of the Health Literacy of Caregivers Scale-Cancer (HLCS-C), and present a revised, psychometrically robust scale. Using data from 297 cancer caregivers (12.4% response rate) recruited from Melbourne, Australia between January-July 2014, confirmatory factor analysis (CFA) was conducted to evaluate the HLCS-C's proposed factor structure. Items were evaluated for: item difficulty, unidimensionality and overall item fit within their domain. Item-threshold-ordering was examined though one-parameter Item Response Theory models. Internal consistency was assessed using Raykov's reliability coefficient. CFA results identified 42 poorly performing/redundant items which were subsequently removed. A 10-factor model was fitted to 46 acceptable items with no correlated residuals or factor cross-loadings accepted. Adequate fit was revealed (χ2WLSMV =1463.807[df=944], p<.001, RMSEA=0.043, CFI=0.980, TLI=0.978, WRMR=1.00). Ten domains were identified: Proactivity and determination to seek information; Adequate information about cancer and cancer management; Supported by healthcare providers (HCP) to understand information; Social support; Cancer-related communication with the care recipient (CR); Understanding CR needs and preferences; Self-care; Understanding the healthcare system; Capacity to process health information; and Active engagement with HCP. Internal consistency was adequate across domains (0.78-0.92). The revised HLCS-C demonstrated good structural, convergent, and discriminant validity, and high internal consistency. The scale may be useful for the development and evaluation of caregiver interventions.

Bayes Factor Covariance Testing in Item Response Models.

Two marginal one-parameter item response theory models are introduced, by integrating out the latent variable or random item parameter. It is shown that both marginal response models are multivariate (probit) models with a compound symmetry covariance structure. Several common hypotheses concerning the underlying covariance structure are evaluated using (fractional) Bayes factor tests. The support for a unidimensional factor (i.e., assumption of local independence) and differential item functioning are evaluated by testing the covariance components. The posterior distribution of common covariance components is obtained in closed form by transforming latent responses with an orthogonal (Helmert) matrix. This posterior distribution is defined as a shifted-inverse-gamma, thereby introducing a default prior and a balanced prior distribution. Based on that, an MCMC algorithm is described to estimate all model parameters and to compute (fractional) Bayes factor tests. Simulation studies are used to show that the (fractional) Bayes factor tests have good properties for testing the underlying covariance structure of binary response data. The method is illustrated with two real data studies.

Comparative Analysis of Classical Test Theory and Item Response Theory Based Item Parameter Estimates of Senior School Certificate Mathematics Examination

The study compared Classical Test Theory (CTT) and Item Response Theory (IRT)-estimated item difficulty and item discrimination indices in relation to the ability of examinees in Senior School Certificate Examination (SSCE) in Mathematics with a view to providing empirical basis for informed decisions on the appropriateness of statistical and psychometric tests. The study adopted ex-post-facto design. A sample of 6,000 students was selected from the population of 35,262 students who sat for the NECO SSCE Mathematics Paper 1 in 2008 in Osun State, Nigeria. An instrument consisting of 60-multiple-choice items, May/June 2008 NECO SSCE Mathematics Paper 1 was used. Three sampling plans: random, gender and ability sampling plans were employed to study the behaviours of the examinees scores under the CTT and IRT measurement frameworks. BILOG-MG 3 was used to estimate the indices of item parameters and SPSS 20 was used to compare CTT- and IRT-based item parameters. The results showed that CTT-based item difficulty estimates and oneparameter IRT item difficulty estimates were comparable (the correlations were generally in the -0.702 to -0.988 range in large sample and -0.622 to - 0.989 range in small sample). Results also indicated that CTT-based and two-parameter IRT-based item discrimination estimates were comparable (the correlations were in the 0.430 to 0.880 ranges in large sample and 0.531 to 0.950 range in small sample). The study concluded that CTT and IRT were comparable in estimating item characteristics of statistical and psychometric tests and thus could be used as complementary procedures in the development of national examinations

Rasch analysis of the Dutch version of the Oxford elbow score

Background:The Oxford elbow score (OES) is a patient-rated, 12-item questionnaire that measures quality of life in relation to elbow disorders. This English questionnaire has been proven to be a reliable and valid instrument. Recently, the OES has been translated into Dutch and examined for its reliability, validity, and responsiveness in a group of Dutch patients with elbow pathology. The aim of this study was to analyze the Dutch version of the OES (OES-DV) in combination with Rasch analysis or the one-parameter item response theory to examine the structure of the questionnaire.Methods:The OES-DV was administered to 103 patients (68 female, 35 male). The mean age of the patients was 44.3 ± 14.7 (range 15–75) years. Rasch analysis was performed using the Winsteps® Rasch Measurement Version 3.70.1.1 and a rating scale parameterization.Results:The person separation index, which is a measure of person reliability, was excellent (2.30). All the items of the OES had a reasonable mean square infit or outfit value between 0.6 and 1.7. The threshold of items were ordered, so the categories can function as intended. Principal component analysis of the residuals partly confirmed the multidimensionality of the English version of the OES. The OES distinguished 3.4 strata, which indicates that about three ranges can be differentiated.Conclusion:Rasch analysis of the OES-DV showed that the data fit to the stringent Rasch model. The multidimensionality of the English version of the OES was partly confirmed, and the four items of the function and three items of the pain domain were recognized as separate domains. The category rating scale of the OES-DV works well. The OES can distinguish 3.4 strata. This conclusion can only be applied to elbow dislocations, which were the largest group of patients studied.

The development of an ICF-oriented, adaptive physician assessment instrument of mobility, self-care, and domestic life

The objective of this study was development and psychometric testing of an adaptive, International Classification of Functioning, Disability, and Health (ICF)-oriented questionnaire to be processed by the rehabilitation physician that aids in assessing mobility, self-care, and domestic life (Moses-Physician). The intent is to develop a physician version, analogous in content to the existing patient questionnaire 'Moses-Patient'. The 58 items of Moses-Patient were converted to an external assessment format without altering the content. The data were compiled for 549 patients with musculoskeletal diseases, 212 patients with cardiac diseases, and 259 neurology rehabilitation patients. Analyses were carried out on the basis of the one-parameter item response theory (Rasch model). Effect sizes and the reliable change index were calculated to test responsiveness. Differential item functioning (DIF) was tested using DIF contrasts, equivalent to Mantel-Haenszel DIF sizes. After the item response theory analysis, 47 of 58 items remained, distributed over 12 scales. The scales are more homogeneous in content than in the patient version because of the omission of 11 items and thus do not cover the ICF categories as broadly. Model fit indices (infit and outfit mean square statistics) were in an acceptable range for all items. Cronbach's alpha was between 0.73 and 0.95. Moreover, there is clear evidence of unidimensionality and sensitivity to change of the scales of Moses-Physician. The item parameters of Moses-Physician are invariant with respect to sex and age for all scales. However, there are clear differences regarding disease groups. The Moses-Physician questionnaire is an adaptive, Rasch-scaled assessment instrument that, to a great extent, covers the contents of the ICF chapters 'mobility', 'self-care', and 'domestic life'.

Differential Item Functioning Analysis of the Preschool Language Scale—4 Between English-Speaking Hispanic and European American Children From Low-Income Families

The study examined whether item bias is present in the Preschool Language Scale—4 (PLS-4). Participants were 440 children (3—5 years old; 86% English-speaking Hispanic and 14% European American) who were enrolled in Head Start programs. The PLS-4 items were analyzed for differential item functioning (DIF) using logistic regression and a one-parameter item response theory (IRT) model. The logistic regression analysis identified Auditory Comprehension (AC) Item 35 as having non-uniform DIF. Using IRT, AC Item 55 and Expressive Communication (EC) Items 30 and 31 displayed statistical evidence of DIF. AC Item 55 and EC Item 30 were more difficult for Hispanic children than for European American children, whereas the opposite was true for EC Item 31. These findings did not find strong evidence of cultural bias for the PLS-4 against English-speaking Hispanic children from low-income families.

Rasch analysis and item reduction of the hypomanic personality scale

The aim of the current study was to reduce the number of items in the 48-item hypomanic personality scale (HPS) and determine whether a unidimensional scale of the hypomanic trait could be derived. Previously collected HPS data from University students ( n = 318) were applied to the Rasch model (one-parameter item response theory). Overall scale and individual item fit statistics were used to judge fit to the model and item maps employed to determine coverage of the trait. Cronbach’s Alpha and correlations with other questionnaires pre- and post-item reduction were evaluated. Rasch analysis indicated that the original HPS was not unidimensional, had significant redundancy and differential item functioning by age and gender. An iterative process of item reduction produced a 20-item HPS (HPS-20) that retained the concepts of the original HPS and had excellent fit to the Rasch model (χ 2 p = 0.27). Unidimensionality of the HPS-20 was confirmed. The traditional psychometric properties of the HPS-20 and coverage of the underlying hypomanic construct were similar to the original. It was possible to derive a unidimensional measure of the hypomanic trait. Further use of the HPS-20 is encouraged as it may increase understanding of the risk factors for affective disorders.

MH4: APPLICATION OF THE RASCH MODEL TO THE SF36 MENTAL HEALTH 5 ITEM SCALE (MH5)

OBJECTIVES: To verify whether the SF36 Mental Health 5 item scale (MH5), an instrument commonly used in mental health research, fulfills criteria of unidimensionality and invariant measurement specified by item response theory (IRT). METHODS: As part of a survey of health care needs among university students in Geneva, Switzerland, 1257 respondents (64% of eligible persons) filled the MH5 (version 2). Each item was scored on a 5-level frequency scale (“never” to “all of the time”). We analyzed these data using both the traditional method (summative scoring), and the polytomous one-parameter IRT (or Rasch) model. RESULTS: In traditional analysis, the MH5 scale performed as expected (single factor, Cronbach alpha 0.85, mean 67.1, standard deviation 17.6, range 0–100). Rasch analysis revealed the good fit of all item characteristic curves. Threshold locations for feeling “nervous” (mental health logits: −3.2, −0.7, 2.0, 3.3), “down in the dumps” (−4.0, −1.7, 0.4, 1.5), “calm and peaceful” (−3.0, −0.6, 0.9, 6.0), “downhearted and blue” (−4.2, −1.6, 1.0, 3.0), and “happy” (−3.1, −0.9, 0.7, 4.2) were all ordinal, consistently with theory, and spread widely over a span of 10.2 logits. While classic MH5 scores and Rasch-based scores were closely correlated (r = 0.98), their relationship was S-shaped: the intervals, in mental health logits, between MH5 scores 0–10, 10–20, etc, up to 90–100 were: 2.23, 1.22, 1.00, 0.90, 0.85, 0.83, 0.90, 1.09, 1.54, and 3.21. CONCLUSIONS: The five MH5 items fit the Rasch model very well. Classic summative scoring tends to compress the measurement scale at its extremities, by assigning scores that are too low at the high end, and too high at the low end. Rasch scoring may render MH5 more sensitive to changes among well populations and among the very sick.

One-Parameter Item Response Theory Models for Psychomotor Tests Involving Repeated, Independent Attempts

Three one-parameter item response theory models are given as viable models that describe the observed score probabilities of an examinee who takes a multiple attempts, single-item test of a psychomotor skill. These models include the Binomial Trials Model, the Poisson Counts Model, and the Inverse Binomial Model. The paper presents these models and offers suggestions for their estimation and subsequent use in the assessment of psychomotor behavior.

The efficiency of a tailored procedure in predicting CPI scale scores

A simple tailored version of the California Psychological Inventory (CPI) was derived from the responses of a calibrating sample of 1,190 subjects using an approach based on a one-parameter item-response theory. This tailored version was administered on a microcomputer to a second independent sample. Validity data were obtained separately from the calibrating and second samples. The mean of the correlations of tailored estimates with full-scale scores was .71 for the calibrating sample and .73 for the second sample. These are comparable with reliability coefficients for the inventory. The average reduction in the number of items given with the tailored version was 71%.

The effects of reducing correlation of external anchors on test equating methods for the equivalent groups and non-equivalent groups designs

Six equating methods were compared: a one-parameter Item Response Theory (IRT) method; two equipercentile methods (direct and by frequency estimation); and three linear methods (Tucker, Levine Equally Reliable and Levine Unequally Reliable) in a situation in which different forms were administered to different groups, thus necessitating the use of an anchor test. The groups were simulated as either equivalent groups or groups of variable ability representing the two types of class groupings that can exist in schools (i.e. parallel or streamed classes). The correlation between the ability measured by an external anchor and the tests to be equated was systematically manipulated. A discrepancy index summarised the discrepancy of each equating method from an IRT criterion, an equipercentile criterion, and from each other. Large discrepancies were interpreted with the aid of graphs and discussed in terms of examinee indifference to the alternative transformations. The direct equipercentile and Levine Unequally Reliable methods were the only methods that consistently increased their level of the discrepancy from criterion following reduction in correlation for the two equatings examined in the equivalent groups design. For the non-equivalent groups design, a reduction in correlation resulted in a systematic effect in favour of those taking an easier form (usually the less able) for all equating methods. What was observed, however, was that for small reductions in correlation, the discrepancy of some of the equating methods from the IRT criterion was reduced. The implications of these findings are discussed and recommendations made for further work.