Differential Item Functioning Effect Research Articles

Examining Differential Item Functioning from a Multidimensional IRT Perspective.

Differential item functioning (DIF) is a standard analysis for every testing company. Research has demonstrated that DIF can result when test items measure different ability composites, and the groups being examined for DIF exhibit distinct underlying ability distributions on those composite abilities. In this article, we examine DIF from a two-dimensional multidimensional item response theory (MIRT) perspective. We begin by delving into the compensatory MIRT model, illustrating and how items and the composites they measure can be graphically represented. Additionally, we discuss how estimated item parameters can vary based on the underlying latent ability distributions of the examinees. Analytical research highlighting the consequences of ignoring dimensionally and applying unidimensional IRT models, where the two-dimensional latent space is mapped onto a unidimensional, is reviewed. Next, we investigate three different approaches to understanding DIF from a MIRT standpoint: 1. Analytically Uniform and Nonuniform DIF: When two groups of interest have different two-dimensional ability distributions, a unidimensional model is estimated. 2. Accounting for complete latent ability space: We emphasize the importance of considering the entire latent ability space when using DIF conditional approaches, which leads to the mitigation of DIF effects. 3. Scenario-Based DIF: Even when underlying two-dimensional distributions are identical for two groups, differing problem-solving approaches can still lead to DIF. Modern software programs facilitate routine DIF procedures for comparing response data from two identified groups of interest. The real challenge is to identify why DIF could occur with flagged items. Thus, as a closing challenge, we present four items (Appendix A) from a standardized test and invite readers to identify which group was favored by a DIF analysis.

Bayesian Adaptive Lasso for the Detection of Differential Item Functioning in Graded Response Models

The detection of differential item functioning (DIF) is important in psychological and behavioral sciences. Standard DIF detection methods perform an item-by-item test iteratively, often assuming that all items except the one under investigation are DIF-free. This article proposes a Bayesian adaptive Lasso method to detect DIF in graded response models (GRMs), where the DIF effects for all items can be identified simultaneously. The multiple-group GRMs are specified, and the possible DIF effects for each item are reparameterized using the increment components. Then, a Bayesian adaptive Lasso procedure is developed for parameter estimation, in which DIF effects can be automatically obtained. Our method is evaluated and compared with the commonly used likelihood ratio test method in a simulation study. The results show that our method can recover most model parameters well and has better control of false positive rates in almost all conditions. An application is presented using data from the National Longitudinal Study of Adolescent to Adult Health (Add Health).

DIF Analysis with Unknown Groups and Anchor Items

Ensuring fairness in instruments like survey questionnaires or educational tests is crucial. One way to address this is by a Differential Item Functioning (DIF) analysis, which examines if different subgroups respond differently to a particular item, controlling for their overall latent construct level. DIF analysis is typically conducted to assess measurement invariance at the item level. Traditional DIF analysis methods require knowing the comparison groups (reference and focal groups) and anchor items (a subset of DIF-free items). Such prior knowledge may not always be available, and psychometric methods have been proposed for DIF analysis when one piece of information is unknown. More specifically, when the comparison groups are unknown while anchor items are known, latent DIF analysis methods have been proposed that estimate the unknown groups by latent classes. When anchor items are unknown while comparison groups are known, methods have also been proposed, typically under a sparsity assumption – the number of DIF items is not too large. However, DIF analysis when both pieces of information are unknown has not received much attention. This paper proposes a general statistical framework under this setting. In the proposed framework, we model the unknown groups by latent classes and introduce item-specific DIF parameters to capture the DIF effects. Assuming the number of DIF items is relatively small, an L1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$L_1$$\\end{document}-regularised estimator is proposed to simultaneously identify the latent classes and the DIF items. A computationally efficient Expectation-Maximisation (EM) algorithm is developed to solve the non-smooth optimisation problem for the regularised estimator. The performance of the proposed method is evaluated by simulation studies and an application to item response data from a real-world educational test.

New Dizziness Impact Measures of Positional, Functional, and Emotional Status Were Supported for Reliability, Validity, and Efficiency

ObjectiveTo calibrate the 25 items from the Dizziness Handicap Inventory (DHI) patient-reported outcome measure (PROM), using item response theory (IRT), into 1 or more item banks, and assess reliability, validity, and administration efficiency of scores derived from computerized adaptive test (CAT) or short form (SF) administration modes. DesignRetrospective cohort study. SettingOutpatient rehabilitation clinics. ParticipantsPatients (N=28,815; women=69%; mean age [SD]=60 [18]) included in a large national dataset and assessed for dizziness-related conditions who responded to all DHI items at intake. InterventionsNot applicable. Main Outcome MeasuresIRT model assumptions of unidimensionality, local item independence, item fit, and presence of differential item functioning (DIF) were evaluated. Generated scores were assessed for reliability, validity, and administration efficiency. ResultsPatients were treated in 976 clinics from 49 US states for either vestibular-, brain injury-, or neck-related impairments. Three unidimensional item banks were calibrated, creating 3 distinct PROMs for Dizziness Functional Status (DFS, 13 items), Dizziness Positional Status (DPS, 4 items), and Dizziness Emotional Status (DES, 6 items). Two items did not fit into any domain. A DFS-CAT and a DFS 7-item SF were developed. Except for 2 items by age groups and 1 item by main impairment, no items were flagged for DIF; DIF impact was negligible. Median reliability estimates were 0.91, 0.72, and 0.79 for the DFS, DPS, and DES, respectively. Scores discriminated between patient groups in clinically logical ways and had a large effect size (>0.8), with acceptable floor and ceiling effects (<15%), except for a floor effect for DPS (20.4%). DFS-CAT scores were generated using a median of 8 items; they correlated highly with full-bank scores (r=0.99). ConclusionThe 3 dizziness impact PROMs demonstrated moderate to high reliability, were valid, and highly responsive to change; thus, they are suitable for research and routine clinical administration.

An Analysis of the DIF Sources of ABIDE Mathematics Self-Efficacy Scale by means of a Latent Class Approach

This study aims to identify the sources of Differential Item Functioning (DIF) using the Mixture Ordinal Logistic Regression (Mixture OLR) method, a contemporary approach for detecting DIF. To analyze mathematics self-efficacy, data from a scale comprising 9 items were obtained from 5000 8th-grade students as part of the ABIDE-2016 project. The study compared the presence and extent of DIF by gender using two methods and examined the sources of DIF for items displaying DIF with Mixture OLR. The OLR analysis revealed that five items exhibited DIF at level A, but no DIF was observed with Mixture OLR. Furthermore, it was found that the magnitude of DIF (B) for an item showing DIF at level A changed due to Mixture OLR. The results indicate that the homogeneity of the data affects both the number of items displaying DIF and the magnitude of DIF. Three items did not exhibit significant DIF according to both methods. One significant finding in the study highlights the moderating effect of latent class on item 8, where DIF was observed. However, the source of DIF was not related to gender but rather stemmed from different ecological variables. An analysis of latent class characteristics revealed that students with significant DIF effects had lower absenteeism and fewer siblings. Additionally, students in this class had greater access to books at home and participated in more out-of-school mathematics courses. Surprisingly, these students were found to engage less in social activities. Various factors can influence how students respond to test items, potentially leading to DIF. These factors may include cultural background, gender, social environment, school, teacher, family interest/attitude toward the child, and home climate. Therefore, when developing and administering tests, it is crucial to test for data homogeneity and consider the impact of these variables, in addition to gender, to identify any sources of DIF in test items.

Identification of sources of DIF using covariates in patient-reported outcome measures: a simulation study comparing two approaches based on Rasch family models.

When analyzing patient-reported outcome (PRO) data, sources of differential item functioning (DIF) can be multiple and there may be more than one covariate of interest. Hence, it could be of great interest to disentangle their effects. Yet, in the literature on PRO measures, there are many studies where DIF detection is applied separately and independently for each covariate under examination. With such an approach, the covariates under investigation are not introduced together in the analysis, preventing from simultaneously studying their potential DIF effects on the questionnaire items. One issue, among others, is that it may lead to the detection of false-positive effects when covariates are correlated. To overcome this issue, we developed two new algorithms (namely ROSALI-DIF FORWARD and ROSALI-DIF BACKWARD). Our aim was to obtain an iterative item-by-item DIF detection method based on Rasch family models that enable to adjust group comparisons for DIF in presence of two binary covariates. Both algorithms were evaluated through a simulation study under various conditions aiming to be representative of health research contexts. The performance of the algorithms was assessed using: (i) the rates of false and correct detection of DIF, (ii) the DIF size and form recovery, and (iii) the bias in the latent variable level estimation. We compared the performance of the ROSALI-DIF algorithms to the one of another approach based on likelihood penalization. For both algorithms, the rate of false detection of DIF was close to 5%. The DIF size and form influenced the rates of correct detection of DIF. Rates of correct detection was higher with increasing DIF size. Besides, the algorithm fairly identified homogeneous differences in the item threshold parameters, but had more difficulties identifying non-homogeneous differences. Over all, the ROSALI-DIF algorithms performed better than the penalized likelihood approach. Integrating several covariates during the DIF detection process may allow a better assessment and understanding of DIF. This study provides valuable insights regarding the performance of different approaches that could be undertaken to fulfill this aim.

DIF Statistical Inference Without Knowing Anchoring Items

Establishing the invariance property of an instrument (e.g., a questionnaire or test) is a key step for establishing its measurement validity. Measurement invariance is typically assessed by differential item functioning (DIF) analysis, i.e., detecting DIF items whose response distribution depends not only on the latent trait measured by the instrument but also on the group membership. DIF analysis is confounded by the group difference in the latent trait distributions. Many DIF analyses require knowing several anchor items that are DIF-free in order to draw inferences on whether each of the rest is a DIF item, where the anchor items are used to identify the latent trait distributions. When no prior information on anchor items is available, or some anchor items are misspecified, item purification methods and regularized estimation methods can be used. The former iteratively purifies the anchor set by a stepwise model selection procedure, and the latter selects the DIF-free items by a LASSO-type regularization approach. Unfortunately, unlike the methods based on a correctly specified anchor set, these methods are not guaranteed to provide valid statistical inference (e.g., confidence intervals and p-values). In this paper, we propose a new method for DIF analysis under a multiple indicators and multiple causes (MIMIC) model for DIF. This method adopts a minimal L_1 norm condition for identifying the latent trait distributions. Without requiring prior knowledge about an anchor set, it can accurately estimate the DIF effects of individual items and further draw valid statistical inferences for quantifying the uncertainty. Specifically, the inference results allow us to control the type-I error for DIF detection, which may not be possible with item purification and regularized estimation methods. We conduct simulation studies to evaluate the performance of the proposed method and compare it with the anchor-set-based likelihood ratio test approach and the LASSO approach. The proposed method is applied to analysing the three personality scales of the Eysenck personality questionnaire-revised (EPQ-R).

Comparing different trend estimation approaches in country means and standard deviations in international large-scale assessment studies

One major aim of international large-scale assessments (ILSA) like PISA is to monitor changes in student performance over time. To accomplish this task, a set of common items (i.e., link items) is repeatedly administered in each assessment. Linking methods based on item response theory (IRT) models are used to align the results from the different assessments on a common scale. This work employs the one-parameter logistic (1PL) and the two-parameter logistic (2PL) IRT models as scaling models for dichotomous item response data. The present article discusses different types of trend estimates in country means and standard deviations for countries in ILSA. These types differ in three aspects. First, the trend can be assessed by an indirect or direct linking approach for linking a country’s performance at an international metric. Second, the linking for the trend estimation can rely on either all items or only the link items. Third, item parameters can be assumed to be invariant or noninvariant across countries. It is shown that the most often employed trend estimation methods of original trends and marginal trends can be conceived as particular cases of indirect and direct linking approaches, respectively. Through a simulation study and analytical derivations, it is demonstrated that trend estimates using a direct linking approach and those that rely on only link items outperformed alternatives for the 1PL model with uniform country differential item functioning (DIF) and the 2PL model with uniform and nonuniform country DIF. We also illustrated the performance of the different scaling models for assessing the PISA trend from PISA 2006 to PISA 2009 in the cognitive domains of reading, mathematics, and science. In this empirical application, linking errors based on jackknifing testlets were utilized that adequately quantify DIF effects in the uncertainty of trend estimates.

Universal screening for social-emotional and behavioral risk: Differential item functioning on the SAEBRS.

Universal screening for social-emotional and behavioral (SEB) risk is one strategy for schools to proactively identify students in need of additional supports and services. As schools serve an increasing number of children from racially and culturally diverse backgrounds, further research is needed to examine the differential functioning of brief behavior rating scales. The present study examined differential item functioning (DIF) on the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS)-Teacher Rating Scale. Participants included 11,496 kindergartens through 12th-grade students. DIF analyses were conducted by race/ethnicity, grade level, and biological sex. Results indicated small-to-large effects of DIF for teacher ratings of Black students compared to their non-Black peers on each item resulting in a moderate effect at the test level (Total Behavior [TB] expected test score standardized difference [ETSSD] = -0.67). There was a small-to-moderate effect of DIF for teacher ratings of White students compared to their non-White peers at the test level (TB ETSSD = 0.43). There was a small-to-moderate effect of DIF by biological sex, with teachers rating males differentially with high risk (TB ETSSD = -0.47). There were no significant effects at the test level for differences in ratings by grade level. Future research is needed to identify the factors influencing the interaction between the rater, the student, and the rating scale that could lead to resulting differential functioning. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Effects of DIF in MST routing in ILSAs

The advance and access to technology have allowed for the implementation of adaptive testing in international large-scale assessments (ILSAs). Multistage testing (MST) in ILSAs offers opportunities and advantages compared to linear testing. However, when dozens of highly heterogeneous systems participate in ILSAs, the sources of heterogeneity pose challenges to any cross-cultural measurement endeavor. With the recent implementation of adaptive designs in ILSAs, little is known about differential item functioning (DIF) effects, mainly when an MST design is used in an ILSA context. Through a simulation study grounded on the empirical basis of the Trends in International Mathematics and Science Study (TIMSS) data, this paper examines the impact of DIF in MST routing in ILSA under different routing strategies. Results showed that Merit routing is highly accurate even when the amount and magnitude of DIF is high, whereas suboptimal routing showed poor accuracy across DIF conditions. As expected, Merit routing has better proficiency recovery parameters than a suboptimal routing mechanism. Implications and recommendations for test developers are included in the discussion section.

Assessment of differential item functioning of the PHQ-9, HADS-D and PROMIS-depression scales in persons with and without multiple sclerosis

ObjectiveWe tested for the presence of differential item functioning (DIF) in commonly used measures of depressive symptoms, in people with multiple sclerosis (MS) versus people with a psychiatric disorder without MS. MethodsParticipants included individuals with MS, or with a lifetime history of a depressive or anxiety disorder (Dep/Anx) but no immune-mediated inflammatory disease. Participants completed the Patient Health Questionnaire (PHQ-9), Hospital Anxiety and Depression Scale (HADS), and the Patient Reported Outcome Measurement Information System (PROMIS)-Depression. We assessed unidimensionality of the measures using factor analysis. We evaluated DIF using logistic regression, with and without adjustment for age, gender and body mass index (BMI). ResultsWe included 555 participants (MS: 252, Dep/Anx: 303). Factor analysis showed that each depression symptom measure had acceptable evidence of unidimensionality. In unadjusted analyses comparing the MS versus Dep/Anx groups we identified multiple items with evidence of DIF, but few items showed DIF effects that were large enough to be clinically meaningful. We observed non-uniform DIF for one PHQ-9 item, and three HADS-D items. We also observed DIF with respect to gender (one HADS-D item), and BMI (one PHQ-9 item). For the MS versus Dep/Anx groups, we no longer observed DIF post-adjustment for age, gender and BMI. On unadjusted and adjusted analyses, we did not observe DIF for any PROMIS-D item. ConclusionOur findings suggest that DIF exists for the PHQ-9 and HADS-D with respect to gender and BMI in clinical samples that include people with MS whereas DIF was not observed for the PROMIS-Depression scale.

Using a Generalized Logistic Regression Method to Detect Differential Item Functioning With Multiple Groups in Cognitive Diagnostic Tests.

Items with the presence of differential item functioning (DIF) will compromise the validity and fairness of a test. Studies have investigated the DIF effect in the context of cognitive diagnostic assessment (CDA), and some DIF detection methods have been proposed. Most of these methods are mainly designed to perform the presence of DIF between two groups; however, empirical situations may contain more than two groups. To date, only a handful of studies have detected the DIF effect with multiple groups in the CDA context. This study uses the generalized logistic regression (GLR) method to detect DIF items by using the estimated attribute profile as matching criteria. A simulation study is conducted to examine the performance of the two GLR methods, GLR-based Wald test (GLR-Wald) and GLR-based likelihood ratio test (GLR-LRT), in detecting the DIF items, the results based on the ordinary Wald test are also reported. Results show that (1) both GLR-Wald and GLR-LRT have more reasonable performance in controlling Type I error rates than the ordinary Wald test in most conditions; (2) the GLR method also produces higher empirical rejection rates than the ordinary Wald test in most conditions; and (3) using the estimated attribute profile as the matching criteria can produce similar Type I error rates and empirical rejection rates for GLR-Wald and GLR-LRT. A real data example is also analyzed to illustrate the application of these DIF detection methods in multiple groups.

Effect Sizes for Estimating Differential Item Functioning Influence at the Test Level

Differential item functioning (DIF) is a critical step in providing evidence to support a scoring inference in building a validity argument for a psychological or educational assessment. Effect sizes can assist in understanding the accumulation of DIF at the test score level. The current simulation study investigated the performance of several proposed effect size measures under a variety of conditions. Conditions under study included varied sample sizes, DIF effect sizes, the proportion of items with DIF, and the type of DIF (additive vs. non-additive). DIF effect sizes under study included sDTF%, uDTF%, τ^w2, d, R¯Δ2, IDIF2*, and S−DIF−V. The results of this study suggest that across study conditions, τ^w2, IDIF2*, and d were consistently the most accurate measures of the DIF effects. The effect sizes were also estimated in an empirical example. Recommendations and implications for practice are discussed.

Psychometric property and measurement invariance of internet addiction test: the effect of socio-demographic and internet use variables

BackgroundAccording to the validation literature on items of Young’s Internet Addiction Test (IAT), this study rephrased disputable items to improve the psychometric properties of this Chinese version of IAT and identify the presence of differential item function (DIF) among demographic and Internet use factors; detect the effect of demographic and Internet use factors on IAT after adjusting for DIF.MethodsAn online questionnaire was distributed to college students in Zhe Jiang province in two stage. The 1st phase study collected 384 valid responses to examine the quality of IAT items by using Rasch Model analysis and exploring factor analysis (EFA). The online questionnaire was modified according to the 1st phase study and distributed online for the 2nd phase study which collected a total of 1131 valid responses. The 2nd phase study applied confirmatory factor analysis (CFA) and a multiple indicator multiple causes (MIMIC) model to verify the construct of IAT, potential effect of covariates on IAT latent factors, as well as the effect of differential item functioning (DIF).ResultsRasch model analysis in the 1st phase study indicated a 5-point rating scale was performed better, no sever misfit was found on item. The overall property of Chinese version IAT with the 5-point scale was good to excellent person and item separation (2.66 and 6.86). A three-factor model was identified by EFA. In the 2nd phase study, IAT 13 were detected with DIF for gender in MIMIC model. After correcting DIF effect, the significant demographic and Internet use factors on IAT were time spent online per day, year 3, year 2, general users.ConclusionItem improvement was efficient that the problematic items found in literature was performed good in this study. The overall psychometric property of this Chinese version IAT was good with limited DIF effect in one item. Item improvement on IAT13 was encouraged in the future study to avoid gender bias and benefit for epidemiology on PIU.

Analysis of Race and Sex Bias in the Autism Diagnostic Observation Schedule (ADOS-2)

There are long-standing disparities in the prevalence of autism spectrum disorder (ASD) across race and sex. Surprisingly, few studies have examined whether these disparities arise partially out of systematic biases in the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2), the reference standard measure of ASD. To examine differential item functioning (DIF) of ADOS-2 items across sex and race. This is a cross-sectional study of children who were evaluated for ASD between 2014 and 2020 at a specialty outpatient clinic located in the Mid-Atlantic region of the US. Data were analyzed from July 2021 to February 2022. Child race (Black/African American vs White) and sex (female vs male). Item-level biases across ADOS-2 harmonized algorithm items, including social affect (SA; 10 items) and repetitive/restricted behaviors (RRBs; 4 items), were evaluated across 3 modules. Measurement bias was identified by examining DIF and differential test functioning (DTF), within a graded response, item response theory framework. Statistical significance was determined by a likelihood ratio χ2 test, and a series of metrics was used to examine the magnitude of DIF and DTF. A total of 6269 children (mean [SD] age, 6.77 [3.27] years; 1619 Black/African American [25.9%], 3151 White [50.3%], and 4970 male [79.4%]), were included in this study. Overall, 16 of 140 ADOS-2 diagnostic items (11%) had a significant DIF. For race, 8 items had a significant DIF, 6 of which involved SA. No single item showed DIF consistently across all modules. Most items with DIF had greater difficulty and poorer discrimination in Black/African American children compared with White children. For sex, 5 items showed significant DIF. DIF was split across SA and RRB. However, hand mannerisms evidenced DIF across all 5 algorithms, with generally greater difficulty. The magnitude of DIF was only moderate to large for 2 items: hand mannerisms (among female children) and repetitive interests (among Black/African American children). The overall estimated effect of DIF on total DTF was not large. These findings suggest that the ADOS-2 does not have widespread systematic measurement bias across race or sex. However, the findings raise some concerns around underdetection that warrant further research.

A comparison of methods to address item non-response when testing for differential item functioning in multidimensional patient-reported outcome measures.

Item non-response (i.e., missing data) may mask the detection of differential item functioning (DIF) in patient-reported outcome measures or result in biased DIF estimates. Non-response can be challenging to address in ordinal data. We investigated an unsupervised machine-learning method for ordinal item-level imputation and compared it with commonly-used item non-response methods when testing for DIF. Computer simulation and real-world data were used to assess several item non-response methods using the item response theory likelihood ratio test for DIF. The methods included: (a) list-wise deletion (LD), (b) half-mean imputation (HMI), (c) full information maximum likelihood (FIML), and (d) non-negative matrix factorization (NNMF), which adopts a machine-learning approach to impute missing values. Control of Type I error rates were evaluated using a liberal robustness criterion for α = 0.05 (i.e., 0.025-0.075). Statistical power was assessed with and without adoption of an item non-response method; differences > 10% were considered substantial. Type I error rates for detecting DIF using LD, FIML and NNMF methods were controlled within the bounds of the robustness criterion for > 95% of simulation conditions, although the NNMF occasionally resulted in inflated rates. The HMI method always resulted in inflated error rates with 50% missing data. Differences in power to detect moderate DIF effects for LD, FIML and NNMF methods were substantial with 50% missing data and otherwise insubstantial. The NNMF method demonstrated comparable performance to commonly-used non-response methods. This computationally-efficient method represents a promising approach to address item-level non-response when testing for DIF.

Investigating the Distractors to Explain DIF Effects Across Gender in Large-Scale Tests With Non-Linear Logistic Regression Models

The purpose of this study is to examine the distractors of items that exhibit differential item functioning (DIF) across gender to explain the possible sources of DIF in the context of large-scale tests. To this end, two non-linear logistic regression (NLR) models-based DIF methods (three parameters, 3PL-NLR and four-parameter, 4PL-NLR) were first used to detect DIF items, and the Mantel-Haenszel Delta (MH-Delta) DIF method was used to calculate the DIF effect size for each DIF item. Then, the multinomial log-linear regression (MLR) model and 2-PL nested logit model (2PL-NLM) were applied to items exhibiting DIF with moderate and large DIF effect sizes. The ultimate goals are (a) to examine behaviors of distractors across gender and (b) to investigate if distractors have any impact on DIF effects. DIF results of the Art Section of the General Aptitude Test (GAT-ART) based on both 3PL-NLR and 4PL-NLR methods indicate that only 10 DIF items had moderate to large DIF effects sizes. According to MLR differential distractor functioning (DDF) results, all items exhibited DDF across gender except for one item. An interesting finding of this study is that DIF items related to the verbal analogy and context analysis were in favor of female students, while all DIF items related to the reading comprehension subdomain were in favor of male students, which may signal the existence of content specific DIF or true ability difference across gender. DDF results show that distractors have a significant effect on DIF results. Therefore, DDF analysis is suggested along with DIF analysis since it signals the possible causes of DIF.

Grade-Related Differential Item Functioning in General English Proficiency Test-Kids Listening

Differential Item Functioning (DIF) analysis is always an indispensable methodology for detecting item and test bias in the arena of language testing. This study investigated grade-related DIF in the General English Proficiency Test-Kids (GEPT-Kids) listening section. Quantitative data were test scores collected from 791 test takers (Grade 5 = 398; Grade 6 = 393) from eight Chinese-speaking cities, and qualitative data were expert judgments collected from two primary school English teachers in Guangdong province. Two R packages “difR” and “difNLR” were used to perform five types of DIF analysis (two-parameter item response theory [2PL IRT] based Lord’s chi-square and Raju’s area tests, Mantel-Haenszel [MH], logistic regression [LR], and nonlinear regression [NLR] DIF methods) on the test scores, which altogether identified 16 DIF items. ShinyItemAnalysis package was employed to draw item characteristic curves (ICCs) for the 16 items in RStudio, which presented four different types of DIF effect. Besides, two experts identified reasons or sources for the DIF effect of four items. The study, therefore, may shed some light on the sustainable development of test fairness in the field of language testing: methodologically, a mixed-methods sequential explanatory design was adopted to guide further test fairness research using flexible methods to achieve research purposes; practically, the result indicates that DIF analysis does not necessarily imply bias. Instead, it only serves as an alarm that calls test developers’ attention to further examine the appropriateness of test items.

Robust and Nonrobust Linking of Two Groups for the Rasch Model with Balanced and Unbalanced Random DIF: A Comparative Simulation Study and the Simultaneous Assessment of Standard Errors and Linking Errors with Resampling Techniques

In this article, the Rasch model is used for assessing a mean difference between two groups for a test of dichotomous items. It is assumed that random differential item functioning (DIF) exists that can bias group differences. The case of balanced DIF is distinguished from the case of unbalanced DIF. In balanced DIF, DIF effects on average cancel out. In contrast, in unbalanced DIF, the expected value of DIF effects can differ from zero and on average favor a particular group. Robust linking methods (e.g., invariance alignment) aim at determining group mean differences that are robust to the presence of DIF. In contrast, group differences obtained from nonrobust linking methods (e.g., Haebara linking) can be affected by the presence of a few DIF effects. Alternative robust and nonrobust linking methods are compared in a simulation study under various simulation conditions. It turned out that robust linking methods are preferred over nonrobust alternatives in the case of unbalanced DIF effects. Moreover, the theory of M-estimation, as an important approach to robust statistical estimation suitable for data with asymmetric errors, is used to study the asymptotic behavior of linking estimators if the number of items tends to infinity. These results give insights into the asymptotic bias and the estimation of linking errors that represent the variability in estimates due to selecting items in a test. Moreover, M-estimation is also used in an analytical treatment to assess standard errors and linking errors simultaneously. Finally, double jackknife and double half sampling methods are introduced and evaluated in a simulation study to assess standard errors and linking errors simultaneously. Half sampling outperformed jackknife estimators for the assessment of variability of estimates from robust linking methods.

Rasch analysis of the Brain Injury Screening Tool (BIST) in mild traumatic brain injury

ObjectiveTo evaluate the psychometric properties of the Brain Injury Screening Tool (BIST) symptom scale in a sample of people with a mild Traumatic Brain Injury (mTBI) through Rasch analysis, and to obtain an interval level measurement score for potential clinical use.Materials and methodsData were obtained from 114 adults aged over 16 years, who had experienced at least one mTBI in the past 10 years. Participants were recruited via social media, concussion clinics and sports organisations over a 4-month period between May and September 2020. Participants were asked to compete the symptom scale of the BIST tool via an anonymous online questionnaire. Internal construct validity, dimensionality, person separation index, and differential item functioning of the BIST were examined with Rasch analysis.ResultsBIST in its original form produced a satisfactory item-trait interaction, and good reliability, but was found to be multi-dimensional. Rasch analysis of the full scale with three domains as subtests resulted in acceptable model fit (χ2(6) =3.8, p > 0.05), with good reliability (Person Separation Index = 0.84), and uni-dimensionality. Differential Item Functioning (DIF) analysis displayed no significant DIF effects for sex or age revealing that people responded consistently and similarly to the individual BIST items based on severity of symptom burden.ConclusionsThe 15-item symptom scale of the BIST tool is a psychometrically sound measure of symptom burden following mTBI. The findings provide support for use of both total and sub scale scores for clinical use. Ordinal to interval score conversions are recommended for use when using the scores for research purposes in mTBI.