Forms Of Differential Item Functioning Research Articles

An item response theory evaluation of the young mania rating scale and the montgomery-asberg depression rating scale in the systematic treatment enhancement program for bipolar disorder (STEP-BD)

BackgroundThe Young Mania Rating Scale (YMRS) and Montgomery-Asberg Depression Rating Scale (MADRS) are among the most widely used outcome measures for clinical trials of medications for Bipolar Disorder (BD). Nonetheless, very few studies have examined the measurement characteristics of the YMRS and MADRS in individuals with BD using modern psychometric methods. The present study evaluated the YMRS and MADRS in the Systematic Treatment Enhancement Program for BD (STEP-BD) study using Item Response Theory (IRT). MethodsBaseline data from 3716 STEP-BD participants were available for the present analysis. The Graded Response Model (GRM) was fit separately to YMRS and MADRS item responses. Differential item functioning (DIF) was examined by regressing a variety of clinically relevant covariates (e.g., sex, substance dependence) on all test items and on the latent symptom severity dimension, within each scale. ResultsBoth scales: 1) contained several items that provided little or no psychometric information, 2) were inefficient, in that the majority of item response categories did not provide incremental psychometric information, 3) poorly measured participants outside of a narrow band of severity, 4) evidenced DIF for nearly all items, suggesting that item responses were, in part, determined by factors other than symptom severity. LimitationsLimited to outpatients; DIF analysis only sensitive to certain forms of DIF. ConclusionsThe present study provides evidence for significant measurement problems involving the YMRS and MADRS. More work is needed to refine these measures and/or develop suitable alternative measures of BD symptomatology for clinical trials research.

Equating the HBSC Family Affluence Scale across survey years: a method to account for item parameter drift using the Rasch model.

To investigate the measurement invariance (MI) of the Family Affluence Scale (FAS) measured in the Health Behavior in School-aged Children (HBSC) survey, and to describe a method for equating the scale when MI is violated across survey years. This study used a sample of 14,076 Norwegian and 17,365 Scottish adolescents from the 2002, 2006 and 2010 HBSC surveys to investigate the MI of the FAS across survey years. Violations of MI in the form of differential item functioning (DIF) due to item parameter drift (IPD) were modeled within the Rasch framework to ensure that the FAS scores from different survey years remain comparable. The results indicate that the FAS is upwardly biased due to IPD in the computer item across survey years in the Norwegian and Scottish samples. Ignoring IPD across survey years resulted in the conclusion that family affluence is increasing quite consistently in Norway and Scotland. However, the results show that a large part of the increase in the FAS scores can be attributed to bias in the FAS because of IPD across time. The increase in the FAS was more modest in Scotland and slightly negative in Norway once the DIF in the computer item was accounted for in this study. When the comparison of family affluence is necessary over different HBSC survey years or when the longitudinal implications of family affluence are of interest, it is necessary to account for IPD in interpretation of changes in family affluence across time.

How Are the Form and Magnitude of DIF Effects in Multiple-Choice Items Determined by Distractor-Level Invariance Effects?

This article explores how the magnitude and form of differential item functioning (DIF) effects in multiple-choice items are determined by the underlying differential distractor functioning (DDF) effects, as modeled under the nominal response model. The results of a numerical investigation indicated that (a) the presence of one or more nonzero DDF effects implies a nonzero DIF effect; (b) the magnitude of the DDF effects creates an upper bound to the magnitude of the resulting DIF effect; (c) the presence of a large DDF effect does not necessarily imply a large DIF effect; (d) under the condition of constant DDF effects the resulting DIF effect is independent of other item properties (i.e., location and discrimination parameters), but under the condition of varying DDF effects the resulting DIF effect magnitude is dependent on other item properties; and (e) although crossing DIF can only exist with divergent (varying in sign) DDF effects, divergent DDF effects do not always yield crossing DIF. Implications of these findings for evaluating DIF effects and interpreting the causes of DIF effects are discussed.

The Effects of Small Sample Size on Identifying Polytomous DIF Using the Liu–Agresti Estimator of the Cumulative Common Odds Ratio

This study is an evaluation of the behavior of the Liu—Agresti estimator of the cumulative common odds ratio when identifying differential item functioning (DIF) with polytomously scored test items using small samples. The Liu—Agresti estimator has been proposed by Penfield and Algina as a promising approach for the study of polytomous DIF but no simulation study focusing on small samples has analyzed this estimator regarding effect size, Type I error, and power rates. The article begins with a description of this estimator in the context of polytomous DIF. Then it presents the methods and results for a simulation study in which three factors are manipulated: between-group difference in ability distribution, form of DIF introduced into the polytomous item, and sample size. The results of this study indicate that for samples smaller than 200, very little power was observed for statistical significance testing; however, Type I error rates were close to nominal levels, and the recovery of the log odds ratio as an effect size was relatively unaffected by sample size for high discriminating items. Implications for practice are discussed.

Modeling DIF Effects Using Distractor-Level Invariance Effects: Implications for Understanding the Causes of DIF

In 2008, Penfield showed that measurement invariance across all response options of a multiple-choice item (correct option and the J distractors) can be modeled using a nominal response model that included a differential distractor functioning (DDF) effect for each of the J distractors. This article extends this concept to consider how the differential item functioning (DIF) effect (i.e., the conditional between-group differences in the probability of correct response) is determined by the J DDF effects. In particular, this article shows how the DIF effect can be modeled as a function of the J DDF effects and thus reveals the conditions that must hold for uniform DIF, nonuniform DIF, and crossing DIF to exist. The results provide insight into the potential item-level properties that lead to uniform, nonuniform, and crossing DIF. The findings may shed light on the etiology of different forms of DIF, which may help analysts target the particular causes of the DIF effect.

A Comparison of the Logistic Regression and Contingency Table Methods for Simultaneous Detection of Uniform and Nonuniform DIF

In this study, we investigate the logistic regression (LR), Mantel‐Haenszel (MH), and Breslow‐Day (BD) procedures for the simultaneous detection of both uniform and nonuniform differential item functioning (DIF). A simulation study was used to assess and compare the Type I error rate and power of a combined decision rule (CDR), which assesses DIF using a combination of the decisions made with BD and MH to those of LR. The results revealed that while the Type I error rate of CDR was consistently below the nominal alpha level, the Type I error rate of LR was high for the conditions having unequal ability distributions. In addition, the power of CDR was consistently higher than that of LR across all forms of DIF.

Testing Features of Graphical DIF: Application of a Regression Correction to Three Nonparametric Statistical Tests

Inspection of differential item functioning (DIF) in translated test items can be informed by graphical comparisons of item response functions (IRFs) across translated forms. Due to the many forms of DIF that can emerge in such analyses, it is important to develop statistical tests that can confirm various characteristics of DIF when present. Traditional nonparametric tests of DIF (Mantel‐Haenszel, SIBTEST) are not designed to test for the presence of nonuniform or local DIF, while common probability difference (P‐DIF) tests (e.g., SIBTEST) do not optimize power in testing for uniform DIF, and thus may be less useful in the context of graphical DIF analyses. In this article, modifications of three alternative nonparametric statistical tests for DIF, Fisher's χ 2 test, Cochran's Z test, and Goodman's U test (Marascuilo & Slaughter, 1981), are investigated for these purposes. A simulation study demonstrates the effectiveness of a regression correction procedure in improving the statistical performance of the tests when using an internal test score as the matching criterion. Simulation power and real data analyses demonstrate the unique information provided by these alternative methods compared to SIBTEST and Mantel‐Haenszel in confirming various forms of DIF in translated tests.

Differential Item Functioning (DIF) and the Mini-Mental State Examination (MMSE)

Various forms of differential item functioning (DIF) in the Mini-Mental State Examination (MMSE) have been identified. Items have been found to perform differently for individuals of different educational levels, racial/ethnic groups, and/or of groups whose first language is not English. The articles in this section illustrate the use of different methods to examine DIF in relation to English and Spanish language administration of the MMSE. The aim of this article is to provide a context for interpretation of the findings contained in the following set of papers examining DIF in the MMSE. The performance of the MMSE, when administered in English and Spanish, was reviewed. "Translation" has been discussed in the context of measurement bias, illustrating the variability in Spanish translations. Presented are the readability of the MMSE, description of the translation method, the study design and sample for the data set used, together with treatment of missing data, and model assumptions related to the analyses described in the accompanying set of papers examining DIF. The examination of item bias in cognitive impairment assessment instruments has practical and theoretical implications in the context of health disparities. Considerable DIF has been identified in the MMSE. A critical factor that may contribute to measurement bias is language translation and conversion. Once DIF has been established consistently in a measure, decisions regarding adjustments proceed. Perhaps the development of guidelines for appropriate adjustments for DIF correction in self-reported measures represents the next challenge in addressing measurement equivalence in crosscultural research.

A Rasch model analysis to test the cross-cultural validity of the EuroQoL-5D in the Schizophrenia Outpatient Health Outcomes Study.

To investigate the pan-European, cross-cultural validity of the EuroQol-5D (EQ-5D) for assessing quality of life in the Schizophrenia Outpatient Health Outcomes (SOHO) Study. The EQ-5D items investigated were mobility, self-care, usual activities, pain/discomfort and anxiety/depression. A Rasch rating scale model (a form of differential item functioning) was used to identify invariance of item calibrations for the 10 European countries participating in the SOHO study. There was general congruence in the EQ-5D item calibration pattern. The rank of average EQ-5D item calibrations was similar for all countries except Denmark. Denmark showed slight misfits for mobility and pain/discomfort. The EQ-5D is an appropriate measure of health-related quality of life across European countries and translations.

Exploring Differential Item Functioning in a 360-Degree Assessment: Rater Source and Method of Delivery

This research used logistic regression to model item responses from a popular 360-degree assessment-for-development survey. The model used method of survey delivery and rater group to identify items exhibiting differential item functioning (DIF). The methods of survey delivery were pencil-and-paper and online by Web page. The rater groups were self, boss, peer, and direct report. The sample consisted of 374 survey families where a survey family was a set of four surveys: self, boss, peer, and direct report. Half the survey families were pencil-and-paper administration; half were online administration. The flagging procedure used effect size from Wald chi-square statistics. Results indicated little evidence that DIF existed due to method of survey delivery, lending additional support for the use of the Internet to deliver 360-degree surveys. Approximately 10% of items exhibited DIF attributable to rater group, though in every instance the magnitude of the DIF was small, suggesting that the impact of DIF on the 360-degree feedback could be slight. There was no evidence of an interaction between method of delivery and rater group. The maximum likelihood parameter estimates indicated DIF resulted from either hierarchical complexity or contingency theory. This research suggested that such forms of DIF could be naturally occurring phenomena in 360- degree assessment because both contingency and complexity can influence the perceptions by raters of a manager’s performance.