Impact Of Differential Item Functioning Research Articles

Can personality disorder be accurately assessed in older age? A differential item functioning analysis of ICD-11 inventories.

The contested categorical personality disorder (PD) criteria are not well suited to inform PD diagnoses in older adults. Yet, the classification of PDs is undergoing a critical transition phase with a paradigm shift to a dimensional approach for diagnosing PDs. No special attention was given to the expression of PDs in older age when the dimensional ICD-11 model was developed. Given that PDs are highly prevalent in older adults, there is an urgent need to examine if ICD-11 related instruments are able to adequately assess for PDs in older adults. The age-neutrality of ICD-11 measures was examined in a sample of 208 Dutch community-dwelling adults (N=208, M age=54.96, SD=21.65), matched on sex into 104 younger (age range 18-64) and 104 older (age range 65-93) adults. An instrument is considered not to be age-neutral if a collective large level of differential item functioning (DIF) exists in a group of items of an instrument (i.e., 25% or more with DIF). We therefore set out to detect possible DIF in the following ICD-11 self-report measures: the Standardized Assessment of Severity of Personality Disorder (SASPD), the Personality Inventory for ICD-11 (PiCD), and the Borderline Pattern Scale (BPS). DIF analyses using a non-parametric odds ratio approach demonstrated that SASPD, PiCD, and BPS were age-neutral with less than 25% of items showing DIF. Yet, impact of DIF at scale level, examined by way of differential test functioning (DTF), indicated a DTF effect on the SASPD total score. These results of age-neutrality of the PiCD and BPS are promising for measuring ICD-11 traits and the borderline pattern. Yet, the age-neutral measurement of PD severity requires further research. With a rapidly aging population, its accurate assessment across the entire adult life span, including older age, is a prerequisite for an adequate detection of PDs.

Detecting Rater Bias in Mixed-Format Assessments

ABSTRACT Mixed-format assessments made up of multiple-choice (MC) items and constructed response (CR) items that are scored using rater judgments include unique psychometric considerations. When these item types are combined to estimate examinee achievement, information about the psychometric quality of each component can depend on that of the other. For example, the presence of differential item functioning (DIF) in MC items could compromise the sensitivity of rater effect indicators, such as differential rater functioning (DRF) indices. Likewise, the presence of DRF could compromise the sensitivity of DIF indices. We used real data and a simulation study to consider the impact of DIF on DRF indices, and the impact of DRF on DIF indices. Results indicate some interaction between DIF and DRF that varies across data collection designs. We consider the implications of these results for research and practice.

Can severity of substance use be measured across drug classes? Estimating differential item functioning by drug class in two general measures of substance use severity

BackgroundSubstance use severity is frequently measured using generic (i.e., non-drug specific) items. Yet, the measurement properties of these items must be evaluated for measurement invariance across inidividuals who use differing substances to ensure total scores can be compared across groups. MethodThis study used data from two independent samples (n1 = 474; n2 = 5183) and two measures of general substance use severity with generic items, the Patient Reported Outcomes Measurement Information System (PROMIS) Severity of Substance Use and DAST-10, to examine for differential item functioning (DIF) across substances (i.e., sedatives, opioids, amphetamines, cocaine, and cannabis). We utilized moderated nonlinear factor analysis to estimate DIF. Finally, we compared factor scores across estimation methods with and without accounting for DIF to examine the impact of DIF. ResultsA minority of items showed statistically significant DIF in each scale (Items with DIF: PROMIS Sample 1: 5/37; PROMIS Sample 2: 7/20; DAST-10 Sample 2: 3/10). Factor scores across scoring methods showed extremely high correlations (0.994 – 0.999), estimates of mean differences across substance groups did not vary considerably across scoring methods, but measurement differences were correlated with factor scores. DiscussionThese findings suggest that these two measures of substance use severity can be used across individuals using different substances. Factor scores appear similar across scoring methods and mean differences do not appear to be substantially biases. Measures with generic items may offer a parsimonious alternative to measures with drug specific items but more research is needed to evaluate the robustness of these findings.

Effects of DIF in MST routing in ILSAs

The advance and access to technology have allowed for the implementation of adaptive testing in international large-scale assessments (ILSAs). Multistage testing (MST) in ILSAs offers opportunities and advantages compared to linear testing. However, when dozens of highly heterogeneous systems participate in ILSAs, the sources of heterogeneity pose challenges to any cross-cultural measurement endeavor. With the recent implementation of adaptive designs in ILSAs, little is known about differential item functioning (DIF) effects, mainly when an MST design is used in an ILSA context. Through a simulation study grounded on the empirical basis of the Trends in International Mathematics and Science Study (TIMSS) data, this paper examines the impact of DIF in MST routing in ILSA under different routing strategies. Results showed that Merit routing is highly accurate even when the amount and magnitude of DIF is high, whereas suboptimal routing showed poor accuracy across DIF conditions. As expected, Merit routing has better proficiency recovery parameters than a suboptimal routing mechanism. Implications and recommendations for test developers are included in the discussion section.

A Psychometric Study of the Prosocial Behavior Scale: Differential Item Functioning by Gender.

Some scales co-exist in the literature to measure prosocial behavior in adolescents. Gender differences in prosocial behavior have been a controversial topic of research. To strengthen future research in the area, the psychometric properties of the most used instruments must be guaranteed, especially its gender non-bias. Our study provides psychometric evidence for the Prosocial Behavior Scale in a sample of adolescents, exploring: (a) its factor structure; (b) reliability; (c) gender-related differential item functioning (DIF); (d) nomological validity. A sample of 512 high school students (mean age = 13.62 (SD = 1.34), 51.6% females) participated in the research. Confirmatory Factor Analysis (CFA) was used to test the factor structure of the scale, which adequately fitted the data (χ2 (35) = 152.224, p < 0.001, Comparative Fit Index (CFI) = 0.905, Root Mean Squared Error of Approximation (RMSEA) = 0.085 90%CI [0.072-0.099], Standardized Root Mean Squared Residual (SRMR) = 0.079). Reliability results were good (α = 0.74, ω = 0.74). Regarding the DIF, five items presented some gender-related bias, generally benefiting females. However, the DIF impact could be considered negligible. Correlations with the subdimensions of the psychological capital offered evidence of the nomological validity of the scale. In conclusion, the scale presented adequate psychometric properties that support its ability to effectively assess prosocial behavior and gender differences in the prosocial behavior samples of adolescents. Additionally, the results obtained imply that gender differences in the manifestations of prosocial behavior require measurements that can fairly sample behaviors characteristic of each gender.

Differential Item Functioning of the Jaw Functional Limitation Scale

To assess the differential item functioning (DIF) of the Jaw Functional Limitation Scale (JFLS) due to gender, age, and language (English vs Spanish). JFLS data were collected from a consecutive sample of 2,115 adult dental patients from HealthPartners dental clinics in Minnesota. Participants with missing data were excluded, and analyses were performed using data from 1,678 participants. Whether the item response theory (IRT) model assumptions of essential unidimensionality and local independence held up for the JFLS was examined. Then, using Samejima's graded response model, the IRT log-likelihood ratio approach was used to detect DIF. The magnitude and impact of DIF based on Raju's noncompensatory DIF (NCDIF) cutoff value of 0.096, Cohen's effect sizes, and test (or scale) characteristic curves were also assessed. Essential unidimensionality was confirmed, but locally dependent items were found on the JFLS. A few items were flagged with statistically significant DIF after adjustment for multiple comparisons. The NCDIF indices associated with all DIF items were < 0.096, and they had small effect sizes of ≤ 0.2. The differences between the expected scores shown in the test characteristic curves were little to none. The present results support the use of the JFLS summary score to obtain psychometrically robust score comparisons across English- and Spanish-speaking, male and female, and younger and older dental patients. Overall, the magnitude of DIF was relatively small, and the practical impact minimal.

New Item-response Theory-based Dizziness Impact Measures Were Reliable, Valid, and Efficient

<h3>Research Objectives</h3> Calibrate items from the Dizziness Handicap Inventory (DHI) patient-reported outcome measure (PROM) to create an item response theory (IRT)-based item bank and assess its reliability, validity, and efficiency using computerized adaptive test (CAT) and short form (SF) administration modes. <h3>Design</h3> A retrospective cohort including data from patients assessed for dizziness. <h3>Setting</h3> Outpatient rehabilitation. <h3>Participants</h3> Total cohort consisted of 28,815 adult patients [69% female; mean age (SD) 60 (18)] treated in 976 clinics from 49 states (USA) for either vertigo, brain injury, or neck impairments who responded to all 25 DHI items at intake. <h3>Interventions</h3> Not applicable. <h3>Main Outcome Measures</h3> IRT model assumptions of local independence, unidimensionality, item fit, and absence of differential item functioning (DIF) were evaluated. CAT and SF T-scores (mean=50, SD=10) were assessed for reliability, validity, and administration efficiency. Score known-groups validity was assessed for the ability to discriminate among patient groups by demonstrating expected differing levels of dizziness impact. Floor and ceiling effects were assessed, and responsiveness was evaluated using an effect-size statistic. <h3>Results</h3> Three unidimensional item sets were identified, creating 3 distinct PROMs to assess functional (DF; 13 items), positional (DP; 4 items), and emotional (DE; 6 items) status (higher score=lower impact). Each item fit its IRT model. Reliability estimates were 0.72 to 0.92. Except for DIF by age group for DF and DE, no items were flagged for DIF, with negligible DIF impact. Scores discriminated between patient groups in clinically expected ways. Floor and ceiling effects were acceptable (< 15%) excepting a floor effect for the positional PROM (21.4%). Effect sizes were large (>0.8). A CAT was developed for DF, administering an average of 8.3 items (median=9, range 5 to 10). <h3>Conclusions</h3> The 3 dizziness PROMs demonstrated moderate to high reliability, were valid, highly responsive to change; thus, they are suitable for routine clinical administration. <h3>Author(s) Disclosures</h3> Dr. Kallen consults for Net Health Systems, Inc., that owns the Focus on Therapeutic Outcomes (FOTO) Patient Outcomes system that gathers and manages the data analyzed. The authors have no other conflicts to declare.

Impact of differential item functioning on group score reporting in the context of large-scale assessments

We investigated the potential impact of differential item functioning (DIF) on group-level mean and standard deviation estimates using empirical and simulated data in the context of large-scale assessment. For the empirical investigation, PISA 2018 cognitive domains (Reading, Mathematics, and Science) data were analyzed using Jackknife sampling to explore the impact of DIF on the country scores and their standard errors. We found that the countries that have a large number of DIF items tend to increase the difference of the country scores computed with and without the DIF adjustment. In addition, standard errors of the country score differences also increased with the number of DIF items. For the simulation study, we evaluated bias and root mean squared error (RMSE) of the group mean and standard deviation estimates using the multigroup item response theory (IRT) model to explore the extent to which DIF items create a bias of the group mean scores and how effectively the DIF adjustment corrects the bias under various conditions. We found that the DIF adjustment reduced the bias by 50% on average. The implications and limitations of the study are further discussed.

Psychometric evaluation of the Positive Mental Health (PMH) scale using item response theory

BackgroundThe investigation of patient-reported outcomes and psycho-oncological interventions mainly focuses on psychological distress or psychopathology. However, the recognition of the equal importance of positive mental health (PMH) has increased lately. The PMH-scale is a brief questionnaire allowing to assess well-being in individuals in the general population and in patients. Previous studies evaluated the psychometric properties of the PMH-scale using classical test theory (CTT). This study is the first to investigate the PMH-scale in patients with cancer using item analysis according to the Rasch model.MethodsIn total, N = 357 cancer patients participated in the study. A Rasch analysis of the PMH-scale was conducted including testing of unidimensionality, local independence, homogeneity and differential item functioning (DIF) with regard to age, gender, type of cancer, the presence of metastases, psycho-oncological support, and duration of disease. Additionally, the ordering of the item thresholds as well as the targeting of the scale were investigated.ResultsAfter excluding one misfitting item and accounting for local dependence by forming superitems, a satisfactory overall fit to the Rasch model was established (χ2 = 30.34, p = 0.21). The new PMH-8 scale proved to be unidimensional, and homogeneity of the scale could be inferred. All items showed ordered thresholds, there was no further item misfit. DIF was found for age, but as the impact of DIF was not substantial, no adjustment related to the age-DIF had to be made. The Person Separation Index (PSI = 0.89) was excellent, indicating excellent discriminatory power between different levels of positive mental health. Overall, the targeting of the PMH-8 was good for the majority of the present sample. However, at both ends of the scale item thresholds are missing as indicated by a slight floor effect (1.4%) and a considerable ceiling effect (9.8%).ConclusionsOverall, the results of the analysis according to the Rasch model support the use of the revised PMH-scale in a psycho-oncological context.

Age-neutrality of the SCID-II: Differential Item Functioning in Younger Outpatients (Aged 20-45) and Older Outpatients (Aged 65+)

ABSTRACT Objectives Measurements are often developed for the assessment of personality disorders (PDs) in younger adults and seldom evaluated on the applicability in older adults. Remarkably, research has not yet been conducted into age-group appropriateness of the gold standard for the assessment of PDs, known as Structured Clinical Interview for DSM-IV Personality Disorders (SCID-II). Therefore, the current study empirically investigated the age-neutrality of the PDs assessed by the SCID-II. Methods Age-neutrality was examined in 84 younger adults (aged 20–45 years) and 68 older adults (aged 65–85) by Differential Item Functioning (DIF). The impact of DIF on scale level was further examined using Differential Test Functioning analyzes to examine the impact of the amount of DIF variance in the items on scale level. Results Overall, the great majority, 95.8% of the categorically measured items and 87.5% of the dimensionally measured items, was endorsed in the same way by younger adults and older adults with equal scores on the PD scale. Subsequent analyzes revealed no large DTF for PD scales. Conclusions Overall the SCID-II in an outpatient population is age-neutral for both categorically and dimensionally scored PD scales. Clinical Implications The SCID-II can be used for the assessment of PDs in older adults.

Measurement Disparities in Frailty Among Kidney Transplant Patients: Impact of Differential Item Functioning

Frailty is commonly measured for clinical risk stratification during transplant evaluation and is more prevalent among older, non-White kidney transplant (KT) patients. However, group differences may be partially attributable to misclassification resulting from measurement bias (differential item functioning/DIF). We examined the extent that DIF affects estimates of age, sex, and race differences in frailty (physical frailty phenotype/PFP) prevalence among 4,300 candidates and 1,396 recipients. We used Multiple Indicators Multiple Causes with dichotomous indicators to assess uniform DIF in PFP criteria attributable to age (≥65vs.18-64 years), sex, and race (Black vs.White). Among candidates (mean age=55 years), 41% were female, 46% were Black, and 19% were frail. After controlling for mean frailty level, females were more likely to endorse exhaustion (OR=1.20,p=0.003), but less likely to endorse low activity (OR=0.83,p=0.01). Younger candidates were more likely to endorse weight loss (OR=1.30,p=0.005), exhaustion (OR=1.60,p<0.001), and low activity (OR=1.80,p<0.001). Black candidates were more likely to endorse exhaustion (OR=1.25,p<0.001), but less likely to endorse weakness (OR=0.79,p<0.001). Among recipients (mean age=54 years), 40% were female, 39% were Black, and 15% were frail. Younger recipients were more likely to endorse weight loss (OR=1.55,p=0.005) and low activity (OR=1.61,p=0.02); however, no DIF was detected by sex or race. Results highlight the impact of DIF for specific PFP measures by age, sex, and race among candidates, but only by age for recipients. Further research is needed to ascertain whether candidate- and/or recipient-specific thresholds to correct for DIF could improve risk prediction and equitable access to KT for older, female, and Black candidates.

Translation and cultural validation of the University of Washington Caregiver Stress and Benefit Scales

BackgroundEnglish versions of the University of Washington Caregiver Stress (UW-CSS) and Benefit (UW-CBS) Scales were developed in the United States (US) to measure impact on caregivers of caring for a child/children. Caregiving stress and benefit are important constructs to study worldwide. The purpose of this study was to translate and validate the UW-CSS and UW-CBS into French, German, Italian, and Spanish languages.MethodUW-CSS and UW-CBS were translated using forward and backward translation with reconciliation. Cognitive interviews (CIs) were completed with caregivers of children < 18 years with severe epilepsy. Translated versions were also administered to at least 100 caregivers in each of the four countries: France, Germany, Italy, and Spain. Differential item functioning (DIF) analyses were used to assess linguistic and cultural bias by country. The US development sample of 722 caregivers was used as a comparison sample for DIF analyses. DIF adjusted scores were calculated to determine impact of DIF on the item response theory (IRT)-based T-score. Benefit and stress scores were also calculated and compared across countries and health condition subgroups. Finally, short forms were modified to minimize the impact of DIF on the UW-CSS and UW-CBS T-scores and to reflect feedback from CIs.ResultsInterviews were completed with 47 caregivers (German n = 14; Spanish n = 10; French n = 13; Italian n = 10). UW-CSS and UW-CBS were administered to 456 (German n = 117, Spanish n = 114, French n = 115, Italian n = 110) caregivers of children with and without health conditions. All stress items functioned well in CIs, though results indicated statistically significant DIF for three items in multiple countries and in the overall sample. Four of the 13 benefit items were problematic based on CI feedback, and six items showed DIF in one or more countries or in the combined sample. However, average differences between DIF adjusted and non-adjusted scores were minimal for both scales and all comparisons, indicating the impact of DIF on the total score was negligible.ConclusionModified short forms functioned well in all four of the translated versions. All language versions are freely publicly available.

Evaluations of the sum-score-based and item response theory-based tests of group mean differences under various simulation conditions.

The use of patient-reported outcomes measures is gaining popularity in clinical trials for comparing patient groups. Such comparisons typically focus on the differences in group means and are carried out using either a traditional sum-score-based approach or item response theory (IRT)-based approaches. Several simulation studies have evaluated different group mean comparison approaches in the past, but the performance of these approaches remained unknown under certain uninvestigated conditions (e.g. under the impact of differential item functioning (DIF)). By incorporating some of the uninvestigated simulation features, the current study examines Type I error, statistical power, and effect size estimation accuracy associated with group mean comparisons using simple sum scores, IRT model likelihood ratio tests, and IRT expected-a-posteriori scores. Manipulated features include sample size per group, number of items, number of response categories, strength of discrimination parameters, location of thresholds, impact of DIF, and presence of missing data. Results are summarized and visualized using decision trees.

Impact of DIF on General Factor Mean Comparisons for Bifactor, Ordinal Data

A simulation study was conducted to explore the impact of differential item functioning (DIF) on general factor difference estimation for bifactor, ordinal data. Common analysis misspecifications in which the generated bifactor data with DIF were fitted using models with equality constraints on noninvariant item parameters were compared under data generation conditions varying by sample size, number of response categories, effect size of the general factor mean difference, and the position and the magnitude of DIF. Estimation bias in the general factor mean difference resulting from ignoring DIF mainly depended on the type of parameters with DIF. When ignoring DIF in threshold parameters, very substantial estimation bias was produced, and Type I error rates (powers) were inflated for many conditions. In contrast, when ignoring DIF in general factor loadings, a little estimation bias was produced; when ignoring DIF in specific factor loadings, almost no estimation bias was produced. All Type I error rates fell within required limits when DIF was present in factor loadings, and powers were not influenced by ignoring DIF in factor loadings.

Differential item functioning of the Beck Anxiety Inventory in a rural, multi-ethnic cohort

BackgroundEvaluating measurement bias is vital to ensure equivalent assessment across diverse groups. One approach for evaluating test bias, differential item functioning (DIF), assesses item-level bias across specified groups by comparing item-level responses between groups that have the same overall score. Previous DIF studies of the Beck Anxiety Inventory (BAI) have only assessed bias across age, sex, and disease duration in monolingual samples. We expand this literature through DIF analysis of the BAI across age, sex, education, ethnicity, cognitive status, and test language. MethodsBAI data from a sample (n = 527, mean age=61.4 ± 12.7, mean education=10.9 ± 4.3, 69.3% female, 41.9% Hispanic/Latin American) from rural communities in West Texas, USA were analyzed. Item response theory (IRT) / logistic ordinal regression DIF was conducted across dichotomized demographic grouping factors. The Mann-Whitney U test and Hedge's g standardized mean differences were calculated before and after adjusting for the impact of DIF. ResultsSignificant DIF was demonstrated in 10/21 items. An adverse impact of DIF was not identified when demographics were assessed individually. Adverse DIF was identified for only one participant (1/527, 0.2%) when all demographics were aggregated. LimitationsThese results might not be generalizable to a sample with broader racial representation, more severe cognitive impairment, and higher levels of anxiety. ConclusionsMinimal item-level bias was identified across demographic factors considered. These results support prior evidence that the BAI is valid for assessing anxiety across age and sex while contributing new evidence of its clinical relevance across education, ethnicity, cognitive status, and English/Spanish test language.

Validation of the Four-Dimensional Symptom Questionnaire (4DSQ) in a mental health setting

ABSTRACT Mental health problems are highly prevalent in primary care. Validated tools to detect mental disorders in general practice are needed. The Four-Dimensional Symptom Questionnaire (4DSQ) was designed to help GPs differentiating between psychological distress and psychopathological conditions (depression, anxiety, somatization). The aim of the current study was to examine psychometric properties of the 4DSQ in a mental health setting. Reliability, factorial, construct, and criterion validity of the English translation of the 4DSQ were analyzed in an American sample of 159 patients attending a psychotherapy outpatient clinic. Measurement equivalence across languages was determined by analyzing differential item functioning (DIF) and differential test functioning (DTF) in the American sample and a Dutch mental health sample, matched by age and sex. A confirmatory factor analysis confirmed all 4DSQ subscales to be unidimensional. All 4DSQ subscales revealed excellent reliability (Cronbach’s alpha and McDonald omega ≥.90) and high correlations with a symptom distress subscale of an instrument that is commonly used to monitor psychotherapy progress, the Outcome Questionnaire-45. Eight items were flagged with DIF. The Depression subscale was free of DIF. DTF analyses showed an impact of DIF on scale level for the lower cutoff score of the Distress scale. The 4DSQ Distress score was the best predictor of a mood disorder diagnosis and the Anxiety score outperformed other 4DSQ scales to predict an anxiety disorder. In conclusion, the 4DSQ demonstrates excellent reliability and validity in a mental health setting. Further research is needed to determine reliable cutoff values on 4DSQ subscales to predict psychiatric diagnoses.

The Lower Extremity Physical Function Patient-Reported Outcome Measure Was Reliable, Valid, and Efficient for Patients With Musculoskeletal Impairments

ObjectiveTo calibrate the Lower Extremity Functional Scale (LEFS) items into a regional lower extremity physical function (LEPF) item bank and assess reliability, validity, and efficiency of computerized adaptive test (CAT) and short form (SF) administration modes. DesignRetrospective cohort. SettingData were collected from patients treated in outpatient rehabilitation clinics for musculoskeletal impairments of the hip, knee, foot, and ankle that responded to all 20 LEFS items at intake. ParticipantsPatients aged 14 years or older who started an episode of care during January 2016-October 2019 and identified the lower extremity region as the source of a primary musculoskeletal complaint. Total cohort included 78,186 patients (mean age, 53±19y, range, 14-89y). InterventionsNot applicable. Main Outcome MeasuresItem response theory (IRT) model assumptions of unidimensionality, local item independence, item fit, and presence of differential item functioning (DIF) were studied. LEPF-CAT– and LEPF-SF–generated scores were evaluated. ResultsAn 18-item solution was supported for its unidimensionality and fit to the IRT model, with reliability estimates >0.9 for all administration modes. No DIF impact on LEPF scores was identified. Scores discriminated between multiple patient groups in clinically logical ways and were highly responsive to change, with negligible floor or ceiling effects. CAT scores were generated using an average of 4.9 items (median, 4). ConclusionsThe LEPF scores were reliable, valid, and efficient for assessing perceived physical function of patients with musculoskeletal impairments of the hip, knee, foot, and ankle; thus, it was found suitable for research and routine clinical administration. These findings are limited to the type of patients included in this study, with further validation needed to assess their generalizability.

Differential Item Functioning of the Scales for Assessing Emotional Disturbance-3 for White and Hispanic Students

The present study explored the psychometric properties of ratings made using the Scales for Assessing Emotional Disturbance–Third Edition: Rating Scale (SAED-3 RS), a scale developed for use in identifying school-age students with emotional and behavioral problems. The purposes of the study were to assess differential item functioning (DIF) for SAED-3 RS items between White and Hispanic students and to assess the impact of DIF on SAED-3 RS scale scores. The sample consisted of 979 students without disabilities who were identified as White/Non-Hispanic (71.91%) and Hispanic (28.09%). The data indicated that SAED-3 RS items demonstrated small to negligible levels of DIF and that DIF did not saliently impact scores. The data suggest that items and scores yielded from the SAED-3 RS are consistent in measuring the emotional and behavioral functioning of school-age students from diverse backgrounds. Research limitations, future research directions, and practical use implications for school personnel are discussed.

Differential item functioning of the Scales for Assessing Emotional Disturbance‐3 for White and African American students

AbstractThe present study investigated the psychometric properties of the Scales for Assessing Emotional Disturbance – Third Edition: Rating Scale (SAED‐3 RS), which is designed for use in identifying students with emotional disturbance for special education services. The purposes of this study were to evaluate (a) the measurement invariance of SAED‐3 RS scores between White and African American students and (b) the impact of differential item functioning (DIF) on test scores from the SAED‐3 RS. The sample consisted of 855 K‐12 students from throughout the United States. The findings suggested that SAED‐3 RS items exhibited small to negligible levels of DIF and that DIF did not significantly impact scores. The results supported the SAED‐3 RS, a teacher‐completed rating scale, as relatively consistent in measuring the emotional and behavioral status of school‐age students from different racial backgrounds. Researchers and practitioners can have confidence that scores from the SAED‐3 RS are not substantially affected by DIF when assessing the emotional and behavioral functioning of African American and White school‐age students. Research limitations, future research, and implications for school professionals are discussed.

When Does Differential Item Functioning Matter for Screening? A Method for Empirical Evaluation.

When items in a screening measure exhibit differential item functioning (DIF) across groups (e.g., males vs. females), DIF might affect which individuals are "caught" in the screening. This phenomenon is common, but DIF detection procedures do not typically provide guidance on whether the presence of DIF will meaningfully affect screening accuracy. Millsap and Kwok proposed a method to quantify the impact of DIF on screening accuracy, but their approach had limitations that prevent its use in scenarios where items are discrete. We extend the Millsap and Kwok procedure to accommodate discrete items and provide R functions to apply the procedure to the user's own data. We illustrate our approach using published screening information and evaluate the proposed methodology with a small simulation study. Overall, we encourage researchers to use empirical methods to evaluate the extent to which the presence of DIF in a screening measure materially affects screening performance.