Anchor Test Research Articles

CHAINED VERSUS POST‐STRATIFICATION EQUATING IN A LINEAR CONTEXT: AN EVALUATION USING EMPIRICAL DATA

ABSTRACTThis study used real data to construct testing conditions for comparing results of chained linear, Tucker, and Levine‐observed score equatings. The comparisons were made under conditions where the new‐ and old‐form samples were similar in ability and when they differed in ability. The length of the anchor test was also varied to enable examination of its effect on the three different equating methods. Two tests were used in the study, and the three equating methods were compared to a criterion equating to obtain estimates of random equating error, bias, and root mean squared error (RMSE). Results showed that for most of the conditions studied, chained linear score equating produced fairly good equating results in terms of low bias and RMSE. In some conditions, Levine‐observed score equating also produced low bias and RMSE. Although the Tucker method always produced the lowest random equating error, it produced a larger bias and RMSE than either of the other equating methods. Based on these results, it is recommended that either chained linear or Levine score equating be used when new‐ and old‐form samples differ in ability and/or when the anchor‐to‐total correlation is not very high.

A Single Population Litmus Test for Linear Scale Alignment Methods: Commentary on Kane, Mroch, Suh, and Ripkey

Kane, Mroch, Suh, and Ripkey (2009) describe what they call five linear equating methods for the nonequivalent groups with anchor test (NEAT) design. The authors embed these methods within a two-di...

Cyclic Load Testing and Ultimate Failure Strength of Suture Anchors in the Acetabular Rim

To determine the resistance to cyclic stress and load-to-failure strength of several suture anchors suitable for hip arthroscopy. Ten polyetheretherketone (PEEK) PushLock, PEEK SutureTak, and Bio-SutureTak anchors (Arthrex, Naples, FL); Lupine Loop BR anchors (DePuy Mitek, Norwood, MA); Bio-Mini Revo anchors (ConMed Linvatec, Largo, FL); and BioRaptor 2.9 AB anchors (Smith & Nephew, Andover, MA) were inserted in rotation into different locations on the rim of 4 matched pairs of potted acetabula clamped to the base of a servohydraulic testing machine. The anchors' sutures were cyclically pulled in line with the insertion angle. Displacement at 100 and 500 cycles, yield load, ultimate failure load, and failure mode were recorded. Statistical analysis was performed. Most of the displacement observed during cyclic loading occurred in the first 100 cycles, except for the BioRaptor 2.9 AB, which showed twice as much displacement at 500 cycles as at 100 cycles. The Lupine Loop BR cyclic displacement was greater than that of the PEEK PushLock (at 100 cycles also), Bio-SutureTak, and PEEK SutureTak at 500 cycles (P < .05). The BioRaptor 2.9 AB cyclic displacement was significantly greater at 100 and 500 cycles than that of the PEEK PushLock and Bio-SutureTak (P < .05). The mean ultimate failure loads for these anchors ranged from 154 N (PEEK SutureTak) to 255 N (Bio-Mini Revo) and was statistically equivalent (P = .139). Correlation analysis showed no significant associations between anchor stiffness, cyclic load displacement, or failure load. Anchors with specific indications for hip arthroscopy showed very little displacement and had consistent failure loads under cyclic conditions in the acetabular rim. All hip anchors except the BioRaptor 2.9 AB showed less than 2.0 mm of displacement after 500 cycles. Biomechanical testing of suture anchors designated for hip arthroscopy should be performed in the acetabular rim before their use. Suture anchors used for shoulder arthroscopy may not perform as well in the hip.

Research and Analysis for Manufactured Housing Foundations: Ground Anchor Verification Testing

The performance of conventional ground anchor assemblies is critical to the overall quality and structural integrity of manufactured housing installations. Consequently, a draft Ground Anchor Assembly Test Protocol (GAATP) has been developed to fulfill the role of a nationally recognized testing as required in the Model Manufactured Home Installation Standards. Therefore, it is important that this test protocol is practical and that it produces reliable and repeatable data characterizing the structural performance of ground anchors. This report provides an assessment of the proposed GAATP based on actual implementation of the test protocol with a variety of conventional ground anchor assemblies, test configurations, and site soil conditions. In addition, a new test rig was developed in compliance with the GAATP rig requirements and implemented in this study to facilitate an efficient and repeatable method of ground anchor testing.

Accumulative Equating Error after a Chain of Linear Equatings

After many equatings have been conducted in a testing program, equating errors can accumulate to a degree that is not negligible compared to the standard error of measurement. In this paper, the author investigates the asymptotic accumulative standard error of equating (ASEE) for linear equating methods, including chained linear, Tucker, and Levine, under the nonequivalent groups with anchor test (NEAT) design. A recursive formula for the ASEE is provided for a series of equatings that makes use of only historical summary statistics. This formula can serve as a new tool to measure the magnitude of equating errors that have accumulated over a series of equatings, and to help monitor and design testing programs.

The Missing Data Assumptions of the NEAT Design and their Implications for Test Equating

The Non-Equivalent groups with Anchor Test (NEAT) design involves missing data that are missing by design. Three nonlinear observed score equating methods used with a NEAT design are the frequency estimation equipercentile equating (FEEE), the chain equipercentile equating (CEE), and the item-response-theory observed-score-equating (IRT OSE). These three methods each make different assumptions about the missing data in the NEAT design. The FEEE method assumes that the conditional distribution of the test score given the anchor test score is the same in the two examinee groups. The CEE method assumes that the equipercentile functions equating the test score to the anchor test score are the same in the two examinee groups. The IRT OSE method assumes that the IRT model employed fits the data adequately, and the items in the tests and the anchor test do not exhibit differential item functioning across the two examinee groups. This paper first describes the missing data assumptions of the three equating methods. Then it describes how the missing data in the NEAT design can be filled in a manner that is coherent with the assumptions made by each of these equating methods. Implications on equating are also discussed.

The Effects of Selection Strategies for Bivariate Loglinear Smoothing Models on NEAT Equating Functions

In this study, eight statistical strategies were evaluated for selecting the parameterizations of loglinear models for smoothing the bivariate test score distributions used in nonequivalent groups with anchor test (NEAT) equating. Four of the strategies were based on significance tests of chi‐square statistics (Likelihood Ratio, Pearson, Freeman‐Tukey, and Cressie‐Read) and four additional strategies were based on different evaluations of the Likelihood Ratio Chi‐Square statistic (Akaike Information Criterion, Bayesian Information Criterion, Consistent Akaike Information Criterion, and an index traced to Goodman). The focus was the implications of the selection strategies’ selection tendencies for the accuracy of chained and poststratification equating functions. The results differentiated the strategies in terms of their tendencies to select models with particular bivariate parameterizations and the implications of these tendencies for equating bias and variability.

Comparisons among Designs for Equating Mixed‐Format Tests in Large‐Scale Assessments

In this study we examined variations of the nonequivalent groups equating design for tests containing both multiple‐choice (MC) and constructed‐response (CR) items to determine which design was most effective in producing equivalent scores across the two tests to be equated. Using data from a large‐scale exam, this study investigated the use of anchor CR item rescoring (known as trend scoring) in the context of classical equating methods. Four linking designs were examined: an anchor with only MC items, a mixed‐format anchor test containing both MC and CR items; a mixed‐format anchor test incorporating common CR item rescoring; and an equivalent groups (EG) design with CR item rescoring, thereby avoiding the need for an anchor test. Designs using either MC items alone or a mixed anchor without CR item rescoring resulted in much larger bias than the other two designs. The EG design with trend scoring resulted in the smallest bias, leading to the smallest root mean squared error value.

A Comparison of Chained Linear and Poststratification Linear Equating Under Different Testing Conditions

In this study I compared results of chained linear, Tucker, and Levine‐observed score equatings under conditions where the new and old forms samples were similar in ability and also when they were different in ability. The length of the anchor test was also varied to examine its effect on the three different equating methods. The three equating methods were compared to a criterion equating to obtain estimates of random equating error, bias, and root mean squared error (RMSE). Results showed that, for most studied conditions, chained linear equating produced fairly good equating results in terms of low bias and RMSE. Levine equating also produced low bias and RMSE in some conditions. Although the Tucker method always produced the lowest random equating error, it produced a larger bias and RMSE than either of the other equating methods. As noted in the literature, these results also suggest that either chained linear or Levine equating be used when new and old form samples differ on ability and/or when the anchor‐to‐total correlation is not very high. Finally, by testing the missing data assumptions of the three equating methods, this study also shows empirically why an equating method is more or less accurate under certain conditions.

What humeri are suitable for comparative testing of suture anchors? An ultrastructural bone analysis and biomechanical study of ovine, bovine and human humeri and four different anchor types

For testing of fixation devices such as suture anchors used in rotator cuff repair often animal bones are used. They are easily obtained, inexpensive and some have been found to be similar to human bone. But can we rely on the results drawn from these studies in our daily surgical practice? The purpose of this study was to compare the trabecular bone mineral density, the trabecular bone volume fraction and the cortical layer thickness in the greater tubercle in different species to evaluate their influence on primary stability of suture anchors under a cyclic loading protocol representing the physiologic forces placed on rotator cuff repairs in vivo. Bovine and ovine humeri are not suitable for suture anchor testing. The statistical significances for pullout forces between the anchors varied from species to species. Therefore, no very applicable information can be obtained from testing suture anchors in ovine or bovine humeri with regard to ultimate failure loads in human humeri. The ultimate failure load seems to depend mainly on the cortical thickness and on the subcortical trabecular bone quality.

Linear Equating for the NEAT Design: Parameter Substitution Models and Chained Linear Relationship Models

This paper analyzes five linear equating models for the nonequivalent groups with anchor test (NEAT) design with internal anchors (i.e., the anchor test is part of the full test). The analysis employs a two-dimensional framework. The first dimension contrasts two general approaches to developing the equating relationship. Under a parameter substitution (PS) approach, estimates of the means and variances for the two tests for some target population are substituted into a generic equating formula; under a chained linear relationship (CLR) approach, expressions for the anchor test scores as functions of total test scores for each of the test forms are simply “equated” to each other. In order to implement either of these approaches, some relationships must be assumed invariant across the groups. The second dimension involves three different choices for the invariant relationships, the regressions of test scores (X or Y) on anchor scores (V), the regression of anchor scores on test scores, or a basic scaling/equating relationship between anchor scores and test scores. If we adopt a scaling/equating relationship of Y with V and X with V as the invariant relationship, the resulting equating relationship is the same for the PS and CLR approaches. So, five distinct regression models yielding five different equating relationships are developed within the two-dimensional framework. The equating relationships for the Tucker, Chained Linear, and Levine Observed-score methods are derived under the PS approach. The equating relationships for the Levine True-score, Chained Linear, and a Tucker-like method (Angoff Design V) are derived under the CLR approach.

An Evaluation of Five Linear Equating Methods for the NEAT Design

This study uses the results of two previous papers (Kane, Mroch, Suh, & Ripkey, this issue; Suh, Mroch, Kane, & Ripkey, this issue) and the literature on linear equating to evaluate five linear equating methods along several dimensions, including the plausibility of their assumptions and their levels of bias and root mean squared difference (RMSD). The methods all employ non-equivalent groups anchor test (NEAT) design, but make different assumptions about the empirical relationship to be generalized across groups. The analyses indicate that the assumptions employed in Levine Observed-score and Levine True-score methods are more plausible than those for a Tucker, Tucker-like, and Chained Linear method, and that the Levine methods generally have lower levels of bias and RMSD than the other three methods. Furthermore, the methods that employed a chained linear relationship (CLR) approach, in which observed relationships between total test scores and anchor test scores are generalized across groups taking various tests, are found to be more consistent with programs in which a series of test forms administered over a period of years are equated to each other, than a parameter substitution (PS) approach, which estimates results for specific synthetic populations. It is argued that the Levine Observed-score and Levine True-score methods have strong advantages over the other methods studied, unless the groups taking the tests to be equated are known to be very similar.

An Empirical Comparison of Five Linear Equating Methods for the NEAT Design

In this study, a data base containing the responses of 40,000 candidates to 90 multiple-choice questions was used to mimic data sets for 50-item tests under the nonequivalent groups with anchor test (NEAT) design. Using these smaller data sets, we evaluated the performance of five linear equating methods for the NEAT design with five levels of group differences and five levels of form differences. The completeness of the data base, with all 40,000 candidates answering all 90 items, allowed us to use a linear equating relationship based on a single group design for the full data base as the criterion for evaluating equating methods with respect to bias, root mean squared difference (RMSD), and differences associated with slopes. All five methods worked well when groups were similar, regardless of form differences, and the Levine methods showed smaller levels of bias and RMSD values than the other methods across a range of group differences. Overall, the Levine methods were most effective in controlling for group differences and, in particular, were most accurate when group differences were moderate to large.

EVALUATING SUBPOPULATION INVARIANCE OF LINKING FUNCTIONS TO DETERMINE THE ANCHOR COMPOSITION FOR A MIXED‐FORMAT TEST

ABSTRACTWe examined the appropriateness of the anchor composition in a mixed‐format test, which includes both multiple‐choice (MC) and constructed‐response (CR) items, using subpopulation invariance indices. We derived linking functions in the nonequivalent groups with anchor test (NEAT) design using two types of anchor sets: (a) MC only and (b) a mix of MC and CR. In each anchor condition, we derived the linking functions separately for males and females and then compared those subpopulation functions to the total group function. In the MC‐only condition, the difference between the subpopulation functions and the total group function was not trivial in a score region that included cut scores, leading to inconsistent pass/fail decisions for low‐performing examinees in particular. Overall, the mixed anchor was a better choice than the MC‐only anchor to achieve subpopulation invariance between males and females. The research reinforces subpopulation invariance indices as a means of determining the adequacy of the anchor.

EFFECT OF REPEATERS ON SCORE EQUATING IN A LARGE‐SCALE LICENSURE TEST

ABSTRACTThis study investigated the subgroup invariance of equating functions for a licensure test in the context of a nonequivalent groups with anchor test (NEAT) design. Examinees who had taken a new, to‐be‐equated form of the test were divided into three subgroups according to their previous testing experience: (a) repeaters who previously took the reference form, to which the new form would be equated; (b) repeaters who previously took any form other than the reference form; and (c) first‐time test takers for whom the new form was the first exposure to the test. Equating functions obtained with two subgroups, all repeaters, and first‐time test takers were similar to those obtained with the total group, supporting score equatability of the two forms. However, when the repeater subgroup was divided into two groups based on the particular form examinees took previously, subgroup equating functions substantially differed from the total‐group equating function, indicating subgroup dependency of score equating. The results indicate that repeater membership needs to be more clearly specified to assess the impact of repeaters on score equating. Such clarification may be especially necessary for high‐stakes licensure tests because repeaters tend to perform more poorly on such tests than first‐time test takers.

COMPARISON OF THE EFFECTS OF DISCRETE ANCHOR ITEMS AND PASSAGE‐BASED ANCHOR ITEMS ON OBSERVED‐SCORE EQUATING RESULTS

ABSTRACTEquating of tests composed of both discrete and passage‐based items using the nonequivalent groups with anchor test (NEAT) design is popular in practice. This study investigated the impact of discrete anchor items and passage‐based anchor items on observed score equating via simulation. Results suggested that an anchor with a larger proportion of passage‐based items and/or a larger degree of local dependence among passage‐based items produces larger equating errors, especially when group ability differences are not minimal. Our findings challenge the common belief that an anchor should be a miniature version of the tests to be equated.

THE EFFECTS OF DIFFERENT TYPES OF ANCHOR TESTS ON OBSERVED SCORE EQUATING

ABSTRACTThis study explores the use of a different type of anchor, a midi anchor, that has a smaller spread of item difficulties than the tests to be equated, and then contrasts its use with the use of a mini anchor. The impact of different anchors on observed score equating were evaluated and compared with respect to systematic error (bias), random equating error (standard errors of equating), and total equating error (RMSE) using empirical data. The results show that the midi anchor generally produces more accurate equating results than the mini anchor—with a few exceptions. Our findings suggest that a midi anchor test would be preferred to a mini anchor test if equating accuracy at the top and at the bottom of the score scale is not a primary concern.

THE MISSING DATA ASSUMPTIONS OF THE NONEQUIVALENT GROUPS WITH ANCHOR TEST (NEAT) DESIGN AND THEIR IMPLICATIONS FOR TEST EQUATING

ABSTRACTThe nonequivalent groups with anchor test (NEAT) design involves missing data that are missing by design. Three popular equating methods that can be used with a NEAT design are the poststratification equating method, the chain equipercentile equating method, and the item‐response‐theory observed‐score‐equating method. These three methods each make different assumptions about the missing data in the NEAT design. Though studies have compared the equating performance of the three methods under the NEAT design, none has examined the missing data assumptions and their implications for such comparisons. The missing data assumptions can affect equating studies because it is necessary to fill in the missing data or their distribution in some way in order to have a true, or criterion, equating function to compare the accuracy and bias of the different methods. If the missing data or their distribution are filled in using missing data assumptions that correspond to a given method, that may favor that method in any comparison with the others. This paper first describes the missing data assumptions of the three equating methods and then performs a fair comparison of the 3 methods using data from 3 different operational tests. For each data set, we examine how the 3 equating methods perform when the missing data satisfy the assumptions made by only 1 of these equating methods. The chain equating method is somewhat more satisfactory overall than the other methods in our fair comparison of the methods; hence, we recommend that equating practitioners seriously consider the chain equating method when using the NEAT design. In addition, we conclude that the results from the different equating methods will tend to agree with each other when proper equating conditions are in place. Moreover, to uncover problems that might not reveal themselves otherwise, it is important for operational testing programs to apply multiple equating methods and study the differences among their results.

CONSTRUCTION OF CHAINED TRUE SCORE EQUIPERCENTILE EQUATINGS UNDER THE KERNEL EQUATING (KE) FRAMEWORK AND THEIR RELATIONSHIP TO LEVINE TRUE SCORE EQUATING

ABSTRACTIn this paper, we develop a new chained equipercentile equating procedure for the nonequivalent groups with anchor test (NEAT) design under the assumptions of the classical test theory model. This new equating is named chained true score equipercentile equating. We also apply the kernel equating framework to this equating design, resulting in a family of chained true score equipercentile equating functions, which include the Levine true score equating model as a special case.

COMPARISONS AMONG DESIGNS FOR EQUATING CONSTRUCTED‐RESPONSE TESTS

ABSTRACTThis study examined variations of a nonequivalent groups equating design used with constructed‐response (CR) tests to determine which design was most effective in producing equivalent scores across the two tests to be equated. Using data from a large‐scale exam, the study investigated the use of anchor CR item rescoring in the context of classical equating methods. Four linking designs were examined: (a) an anchor set containing common CR items, (b) an anchor set incorporating common CR items rescored, (c) an external multiple‐choice (MC) anchor test, and (d) an equivalent groups design incorporating CR items rescored (no anchor test). The use of CR items without rescoring or the use of an external MC anchor resulted in much larger bias than the other two designs. The use of a rescored CR anchor and the equivalent groups design led to similar levels of equating error.