Minimal Differential Item Functioning Research Articles

Establishing language and ethnic equivalence for health-related quality of life item banks and testing their efficiency via computerised adaptive testing simulations.

We aimed to (1) establish linguistic and ethnic equivalence (i.e. lack of bias) for the items in the English and Chinese versions of the Singapore Health and Well Being (SHAWS) Physical Functioning (PF), Positive Mindset (PM) and Social Relationship (SR) item banks (IBs); and (2) evaluate the preliminary efficiency of these IBs using Computer Adaptive Testing (CAT) simulations. In this cross-sectional study, 671, 670, and 672 subjects answered 55, 48 and 30 items of the PF, PM, and SR IBs, respectively. Rasch analysis was conducted to assess each IB's psychometric properties, particularly the presence of differential item functioning (DIF) for language and ethnicity. A set of performance criteria related to removing items that displayed notable DIF were employed. CAT simulations determined the mean number of items for high, moderate, and moderate-low measurement precisions (stopping rule: SEM 0.300, 0.387. 0.521, respectively). Half of subjects were >50 years old (40.9% PF, 42.1% PM, 41.4% SR), Chinese (50.7% PF, 51.0% PM, 50.6% SR) and female (50.0% PF. 49.4% PM, 52.8% SR) respectively. Rasch analysis revealed 4 items with DIF for the PF IB, 9 items with DIF for the PM IB and 2 items with DIF for the SR IB. In CAT simulations, the mean number of items administered was 8.5, 21.6 and 14.5 for the PF, PM and SR IBs, respectively (SEM 0.300), 5.1, 13.0, 8.0 for PF, PM and SR IBs, respectively (SEM 0.387) and 3.1, 5.3 and 4.1 for PF, PM and SR IBs, respectively (SEM 0.521). The PF, PM and SR IBs to measure health-related quality of life revealed minimal DIF for language and ethnicity after remedial efforts. CAT simulations demonstrated that these IBs were efficient, especially when the stopping rule was set at moderate precision, and support the implementation of the SHAWS IBs into routine clinical care.

Dimensionality and Measurement Invariance of the Italian Version of the EORTC QLQ-C30 in Postoperative Lung Cancer Patients.

BackgroundThis study aims to validate and evaluate the psychometric properties and measurement invariance of the Italian version of the European Organization for Research and Treatment of Cancer (EORTC) Quality of Life Questionnaire-Core 30 (QLQ-C30), which is a measure of quality of life (QoL) for lung cancer patients after surgery.MethodsA total of 167 lung cancer patients completed the Italian version of the EORTC QLQ-C30 questionnaire at 30 days after they received a lobectomy. The factor structure of this scale was assessed by performing confirmatory factor analysis (CFA). Measurement invariance was evaluated by considering differential item functioning (DIF) due to age, gender, and type of surgery (i.e., robot- or not robot-assisted).ResultsThe CFA demonstrated the validity of the factor structure of the EORTC QLQ-C30 in assessing overall health and eight distinct subscales of adverse events and functioning. Moreover, the results highlighted a minimal DIF with only trivial consequences on measurement invariance. Specifically, the DIF did not affect the mean differences of latent scores of QoL between patients undergoing robot-assisted surgery or traditional surgery.ConclusionThese findings supported the validity and suitability of the EORTC QLQ-C30 for the assessment of QoL in lung cancer patients of diverse ages and genders undergoing lobectomy with or without robot-assisted surgery.

Measurement of early literacy skills among monolingual English-speaking and Spanish-speaking language-minority children: A differential item functioning analysis

A critical issue in psychological and educational testing is whether assessments provide reliable and valid estimates of ability for different populations of individuals. This issue may be particularly relevant for populations who are not native speakers of the language in which the assessment is written. Therefore, the purpose of this study was to evaluate the utility of a norm-referenced assessment of English early literacy skills for Spanish-speaking language-minority (LM) children. Participants for this study (1221 preschool children, 751 of whom were identified as Spanish-speaking LM children) completed the Phonological Awareness, Print Knowledge, and Definitional Vocabulary subtests of the Test of Preschool Early Literacy (TOPEL). Item response theory analysis was conducted to examine student performance on each subtest, and performance of monolingual English-speaking and Spanish-speaking LM children was compared using differential item functioning (DIF) analysis. Results indicated that there was minimal DIF for the Phonological Awareness and Print Knowledge subtests. Substantially more DIF was evident on the Definitional Vocabulary subtest, although presence of DIF was not consistently in favor of monolingual English-speaking or Spanish-speaking LM children. Moreover, effect size estimates of DIF indicated that, across most test items, the magnitude of DIF was small to moderate. Taken together, these findings indicate that the TOPEL can be used to obtain valid and reliable estimates of Spanish-speaking LM preschoolers’ English early literacy skills.

Measuring dispositional optimism in patients with chronic heart failure and their healthcare providers: the validity of the Life Orientation Test-Revised.

The Life Orientation Test-Revised (LOT-R) measures dispositional optimism (DO) – an individual difference promoting physical and psychological well-being in healthy adults (HAs) as well as in patients with chronic heart failure (CHF) and healthcare providers (HPs). Controversy has arisen regarding the dimensionality of the LOT-R. Whereas DO was originally defined as a one-dimensional construct, empirical evidence suggests two correlated factors in the LOT-R. This study was the first attempt to identify the best factor structure of the LOT-R in patients with CHF and HPs and to evaluate its measurement invariance among subsamples of patients with CHF, HPs, and a normative sample of HAs. Its validity was also evaluated in patients with CHF. The sample comprised 543 participants (34% HAs; 34% HPs; and 32% CHF patients). Congeneric, two correlated factor, and two orthogonal factor models for the LOT-R were compared by performing confirmatory factor analysis (CFA). Measurement invariance was evaluated by considering differential item functioning (DIF) among subsamples of HPs, patients with CHF, and HAs. In patients with CHF, validity was assessed by considering associations with anxiety and depression. The CFA demonstrated the superior fit of the two orthogonal factor model. Moreover, across patients with CHF, HPs, and HAs, the results highlighted a minimal DIF with only trivial consequences. Finally, negative but weak correlations of DO with anxiety and depression confirmed the validity of the LOT-R in patients with CHF. In summary, these findings supported the validity and suitability of the LOT-R for the assessment of DO in patients with CHF, HPs, and HAs.

Content and Construct Validity, Reliability, and Responsiveness of the Rheumatoid Arthritis Flare Questionnaire: OMERACT 2016 Workshop Report.

The Outcome Measures in Rheumatology (OMERACT) Rheumatoid Arthritis (RA) Flare Group was established to develop a reliable way to identify and measure RA flares in randomized controlled trials (RCT). Here, we summarized the development and field testing of the RA Flare Questionnaire (RA-FQ), and the voting results at OMERACT 2016. Classic and modern psychometric methods were used to assess reliability, validity, sensitivity, factor structure, scoring, and thresholds. Interviews with patients and clinicians also assessed content validity, utility, and meaningfulness of RA-FQ scores. People with RA in observational trials in Canada (n = 896) and France (n = 138), and an RCT in the Netherlands (n = 178) completed 5 items (11-point numerical rating scale) representing RA Flare core domains. There was moderate to high evidence of reliability, content and construct validity, and responsiveness. Factor analysis supported unidimensionality. Rasch analysis showed acceptable fit to the Rasch model, with items and people covering a broad measurement continuum and evidence of appropriate targeting of items to people, ordered thresholds, minimal differential item functioning by language, sex, or age. A summative score across items is defensible, yielding an interval score (0-50) where higher scores reflect worsening flare. The RA-FQ received endorsement from 88% of attendees that it passed the OMERACT Filter 2.0 "Eyeball Test" for instrument selection. The RA-FQ has been developed to identify and measure RA flares. Its review through OMERACT Filter 2.0 shows evidence of reliability, content and construct validity, and responsiveness. These properties merit its further validation as an outcome for clinical trials.

Is it possible to develop a cross-country test of social interaction?

Objective: The Evaluation of Social Interaction (ESI) is used in Asia, Australia, North America and Europe. What is considered to be appropriate social interaction, however, differs amongst countries. If social interaction varies, the relative difficulty of the ESI items and types of social exchange also could vary, resulting in differential item functioning (DIF) and test bias in the form of differential test functioning (DTF). Yet, because the ESI scoring criteria are designed to account for culture, the ESI should be free of DIF and DTF. The purpose, therefore, was to determine whether the ESI demonstrates DIF or DTF related to country.Methods: A retrospective, descriptive, cross-sectional study of 9811 participants 2–102 years, 55% female, from 12 countries was conducted using many-facet Rasch analyses. DIF analyses compared paired item and social exchange type values by country against a critical effect size (±0.55 logit). DTF analyses compared paired ESI measures by country to 95% confidence intervals.Results: All paired social exchange types and 98.3% of paired items differed by less than ±0.55 logit. All persons fell within 95% confidence intervals.Conclusions: Minimal DIF resulted in no test bias, supporting the cross-country validity of the ESI.

Cross-Cultural Validation of the Evaluation of Social Interaction (ESI)

Abstract Date Presented 4/8/2016 We used Rasch analyses of participants’ Evaluation of Social Interaction (ESI) scores to test for differential item functioning (DIF), differential social exchange functioning (DSF), and differential test functioning (DTF). We found minimal DIF and no DSF or DTF, concluding that ESI measures are at minimal risk for cross-cultural bias. Primary Author and Speaker: Brett Berg Contributing Author: Anne G. Fisher

Cross-cultural validation of the computerized animated activity questionnaire to assess activity limitations in patients with hip and knee osteoarthritis

Purpose: The computerized Animated Activity Questionnaire (AAQ) for assessing activity limitations in hip and knee osteoarthritis (HKOA) consists of video animations from which patients can choose the animation that best matches their own performance. The AAQ has demonstrated good validity and reliability. Application of the AAQ in international studies, requires good cross-cultural validity, i.e., minimal Differential Item Functioning (DIF) across countries. The aim of this study was to evaluate cross-cultural validity. Methods: Patients in 7 European countries patients completed the AAQ on a computer. Ordinal logistic regression analysis was used to evaluate DIF across languages (Dutch versus 6 other languages). DIF is defined as follows: If a patient in a country has the same level of activity limitation as a patient in the Netherlands (the reference country in which the AAQ is developed), they should score the same on each item of the AAQ. If there is a statistical significant difference between countries, there is DIF. We used frequently described criteria in literature to assess DIF between countries. Criteria for non-uniform DIF were set as a statistical significance (p <0.001) pseudo R-square change according to Nagelkerke with a magnitude larger than 0.035 between the AAQ total score and the country variable (with Dutch as the reference group). Criteria for uniform DIF were set as a statistical significant (p <0.001) odds ratio (OR) of the country variable with a magnitude outside the interval 0.53-1.89. Analyses were adjusted for sex, age, weight, height, and affected joint. The influence of each individual item with DIF on the total score was calculated by means of comparing the correlation between AAQ score with and without the DIF item. A Spearman's correlation of 0.95 or less was interpreted as important influence of the DIF of that item on the total AAQ score. Results: Data of 1239 patients were available. Compared to Dutch (n = 279), none of the 17 items showed DIF in English (n = 202) and French (n = 193). Uniform DIF occurred in the activity 'walking outside' for Spanish versus Dutch (OR 0.28) and Norwegian versus Dutch (OR 0.16), with for both countries scores representing slightly worse functioning compared to Dutch. For Danish versus Dutch, DIF occurred in two items (walking outside on uneven terrain; OR 0.45, walking inside; OR 0.43), also representing slightly worse functioning compared to Dutch. In all these languages, the occurrence of DIF did not influence the total score with correlations of 0.98-0.99 in comparing AAQ scores with and without DIF item (s). For Italian (n = 203) versus Dutch, 6 items showed uniform DIF and 1 item showed non-uniform DIF, which makes it difficult to compare scores obtained in those two countries. Conclusions: Different language versions of the AAQ remain comparable with the original Dutch version except for the Italian version. The AAQ seems to have great potential for international use in research and daily clinical practice, especially in patients with low literacy and non-native speakers, because the use of video animations instead of written text. Future research will focus on responsiveness, interpretation of AAQ scores, and should explore explanations for DIF in Italy by means of qualitative research.

Evaluating the Mathematics Interest Inventory Using Item Response Theory

As the United States has begun to lag behind other developed countries in performance on mathematics and science, researchers have sought to explain this with theories of teaching, knowledge, and motivation. We expand this examination by further analyzing a measure of interest that has been linked to student performance in mathematics and intention to enroll in advanced mathematics classes. Using Item Response Theory (IRT), the purpose of this study was to evaluate the psychometric properties of the Mathematics Interest Inventory (MII). Six hundred sixty-six students (47.6% Hispanic, 48.2% Male) formed the sample. Results revealed minimal differential item functioning (DIF) between genders. Substantial DIF emerged between Hispanic and White students. Implications of the differential functioning and the measurement of interest in mathematics are discussed.

Psychometric evaluation of the Revised Illness Perception Questionnaire (IPQ-R) in cancer patients: Confirmatory factor analysis and Rasch analysis

ObjectiveTo provide new insights into the psychometrics of the revised Illness Perception Questionnaire (IPQ-R) in cancer patients. To undertake, for the first time using data from breast, colorectal and prostate cancer patients, a confirmatory factor analysis (CFA) to assess the validity of the IPQ-R's core seven-factor structure. Also, for the first time in any illness group, to undertake Rasch analysis to explore the extent to which the IPQ-R factors form unidimensional scales, with linear measurement properties and no Differential Item Functioning (DIF). MethodsPatients with potentially curable breast, colorectal or prostate cancer, within 6months post-diagnosis, completed the IPQ-R online (N=531). CFA was conducted, including multi-sample analysis, and for each IPQ-R factor fit to the Rasch model was assessed by examining, amongst other things, item fit, DIF and unidimensionality. ResultsThe CFA showed a moderate fit of the data to the IPQ-R model, and stability across diagnosis, although fit was significantly improved following the removal of selected items. All seven factors achieved fit to the Rasch model, and exhibited unidimensionality and minimal DIF, although in most cases this was after some item rescoring and/or deletion. In both analyses, IPQ-R items 12, 18 and 24 were indicated as misfitting and removed. ConclusionGiven the rigorous standard of Rasch measurement, and the generic nature of the IPQ-R, it stood up well to the demands of the Rasch model in this study. Importantly, the results show that with some relatively minor, pragmatic modifications the IPQ-R could possess Rasch-standard measurement in cancer patients.

An item factor analysis and item response theory-based revision of the Everyday Discrimination Scale.

The Everyday Discrimination Scale (EDS), a widely used measure of daily perceived discrimination, is purported to be unidimensional, to function well among African Americans, and to have adequate construct validity. Two separate studies and data sources were used to examine and cross-validate the psychometric properties of the EDS. In Study 1, an exploratory factor analysis was conducted on a sample of African American law students (N = 589), providing strong evidence of local dependence, or nuisance multidimensionality within the EDS. In Study 2, a separate nationally representative community sample (N = 3,527) was used to model the identified local dependence in an item factor analysis (i.e., bifactor model). Next, item response theory (IRT) calibrations were conducted to obtain item parameters. A five-item, revised-EDS was then tested for gender differential item functioning (in an IRT framework). Based on these analyses, a summed score to IRT-scaled score translation table is provided for the revised-EDS. Our results indicate that the revised-EDS is unidimensional, with minimal differential item functioning, and retains predictive validity consistent with the original scale.

Precision and content range of a parent‐reported item bank assessing lower extremity and mobility skills in children with cerebral palsy

The aim of this study was to determine the psychometric properties, content range, and measurement precision of a lower extremity physical functioning and mobility skills item bank (LE85) in children with cerebral palsy (CP). Lower extremity functioning and mobility skill items were administered to 308 parents of children (169 males, 139 females; mean age 10y 8mo, SD 4y) with spastic CP (145 diplegia, 73 hemiplegia, 89 quadriplegia; [for one person type of CP was unknown]) classified using the Gross Motor Function Classification System (75 level I, 91 level II, 79 level III, 37 level IV, 26 level V). Additional legacy measures were administered to assess concurrent validity. Psychometric characteristics, differential item functioning, content range, and score precision were examined. The LE85 had acceptable psychometric properties. Content range matched the ability range of the sample population and exceeded legacy measures with minimal differential item functioning. The LE85 had good correlation with the Paediatric Outcomes Data Collection Instrument, Functional Independence Measure for Children, Gillette Functional Assessment Questionnaire, and Paediatric Quality of Life Inventory-CP module (range r=0.63-0.86). Precision of the LE85 and 10-item simulated computer adaptive test scores outperformed legacy measures. The LE85 appears to be suitable to administer as a computer adaptive test to measure lower extremity physical functioning and mobility in children with CP.

The Development of an Instrument to Measure Quality of Vision: The Quality of Vision (QoV) Questionnaire

To develop an instrument to measure subjective quality of vision: the Quality of Vision (QoV) questionnaire. A 30-item instrument was designed with 10 symptoms rated in each of three scales (frequency, severity, and bothersome). The QoV was completed by 900 subjects in groups of spectacle wearers, contact lens wearers, and those having had laser refractive surgery, intraocular refractive surgery, or eye disease and investigated with Rasch analysis and traditional statistics. Validity and reliability were assessed by Rasch fit statistics, principal components analysis (PCA), person separation, differential item functioning (DIF), item targeting, construct validity (correlation with visual acuity, contrast sensitivity, total root mean square [RMS] higher order aberrations [HOA]), and test-retest reliability (two-way random intraclass correlation coefficients [ICC] and 95% repeatability coefficients [R(c)]). Rasch analysis demonstrated good precision, reliability, and internal consistency for all three scales (mean square infit and outfit within 0.81-1.27; PCA >60% variance explained by the principal component; person separation 2.08, 2.10, and 2.01 respectively; and minimal DIF). Construct validity was indicated by strong correlations with visual acuity, contrast sensitivity and RMS HOA. Test-retest reliability was evidenced by a minimum ICC of 0.867 and a minimum 95% R(c) of 1.55 units. The QoV Questionnaire consists of a Rasch-tested, linear-scaled, 30-item instrument on three scales providing a QoV score in terms of symptom frequency, severity, and bothersome. It is suitable for measuring QoV in patients with all types of refractive correction, eye surgery, and eye disease that cause QoV problems.

An analysis of test bias and differential item functioning due to race on the Mattis Dementia Rating Scale.

The Mattis Dementia Rating Scale (MDRS) is a commonly used cognitive measure designed to assess the course of decline in progressive dementias. However, little information is available about possible systematic racial bias on the items presented in this test. We investigated race as a potential source of test bias and differential item functioning in 40 pairs of African American and Caucasian dementia patients (N = 80), matched on age, education, and gender. Principal component analysis revealed similar patterns and magnitudes across component loadings for each racial group, indicating no clear evidence of test bias on account of race. Results of an item analysis of the MDRS revealed differential item functioning across groups on only 4 of 36 items, which may potentially be dropped to produce a modified MDRS that may be less sensitive to cultural factors. Given the absence of test bias because of race, the observed racial differences on the total MDRS score are most likely associated with group differences in dementia severity. We conclude that the MDRS shows no appreciable evidence of test bias and minimal differential item functioning (item bias) because of race, suggesting that the MDRS may be used in both African American and Caucasian dementia patients to assess dementia severity.

DIFFERENTIAL ITEM FUNCTIONING FOR MALES AND FEMALES ON SAT®‐VERBAL READING SUBSCORE ITEMS

ABSTRACTThe Reading Comprehension and Sentence Completion items from four forms of the SAT‐verbal sections were examined for differential item functioning (DIF) between male and female test‐takers. In general, minimal DIF was observed for the items analyzed in this investigation. However, among the relatively few items exhibiting substantial DIF, two trends related to science content were noted.An important factor that appears to be connected to DIF on Reading Comprehension items is the distinction between technical and nontechnical reading passage material. Items associated with passages containing technical (as opposed to historical or philosophical) science material are generally more difficult for female examinees. This trend emerged for three of the four forms studied. Data from the fourth form suggested that there may be an interaction between the technical science factor and the passage location. Confirmation of this finding will require a follow‐up study.The main factor that appears to be related to DIF on Sentence Completion items is the distinction between nonscience or surface science references and true science references. Items containing true science references tend to be more difficult for females.Because the majority of items in the Reading subscore exhibited minimal DIF, the conclusions described in this report are based on limited data. Findings from this study also indicate the confounded nature of multiple factors thought to contribute to DIF. There is a need for experimental research in which item features and passage features can be controlled and counterbalanced in variable sections of the SAT.