Differential Item Functioning Research Articles

Measurement invariance and differential item functioning of the functionality appreciation scale (FAS) in Colombia and Spain

The Functionality Appreciation Scale (FAS) is increasingly used in diverse national and linguistic contexts. However, limited work has assessed the extent to which the instrument demonstrates measurement invariance and differential item functioning (DIF) across nations and respondent characteristics. Here, we examined measurement invariance and DIF of the FAS using archival data from adults in Colombia (Mebarak et al., 2023) and Spain (Zamora et al., 2024). Participants included 1420 (women n = 804, men n = 616) respondents from Colombia and 838 (women n = 415, men n = 423) respondents from Spain who completed translations of the FAS. Confirmatory factor analysis supported a unidimensional structure of the FAS in both national groups. Additionally, the FAS achieved full measurement invariance (up to latent mean invariance) across both groups. We also found that the FAS lacked DIF as a function of age, body mass index (BMI), and gender identity across both national groups. Older participants (relative to younger participants), men (relative to women), and participants with lower BMIs (relative to those with higher BMIs) had higher FAS scores. These results support the notion that the FAS is measuring a common underlying construct across these national groups and respondent characteristics.

Generalisability of the Barthel Index and the Functional Independence Measure: robustness of disability measures to Differential Item Functioning

Purpose Differential Item Functioning (DIF), an item malfunctioning, causes Differential Test Functioning (DTF), thus biasing questionnaire measures. The current study evaluates the relationship between DIF and DTF for the Barthel Index and the Functional Independence Measure, likely the most used disability measures. The aim is to understand under which conditions DIF can be ignored as its DTF is negligible. Methods A simulation study was run. Disability measures were obtained for the Barthel Index and FIM motor domain using Rasch analysis with previously published item calibrations. Several DIF scenarios have been assessed. DTF was tolerable if ≤0.50 logits. Results Simulations showed that the larger the DIF, the larger the DTF and that, keeping the overall DIF constant, the total number of items with DIF does not affect DTF. DIF of the items with the lowest or highest calibrations is the most dangerous. The DIF of central items should be so massive to matter in DTF terms that it is unlikely to happen in practice. The FIM robustness to DIF is better than that of the Barthel Index. Conclusions The FIM and the Barthel Index show remarkable robustness to DIF. Thanks to this feature, sample invariant, generalisable disability measures are available.

Uncovering measurement bias: Differential item functioning by income in the Couples Satisfaction Index.

Relationship satisfaction is among the most popular constructs in family science. As the study of families and romantic couples continues to include more diverse samples, it is imperative to ensure the measures scholars use do not significantly vary in psychometric quality across groups. The goal of this study was to examine the psychometric utility of the four-item Couples Satisfaction Index (CSI-4; Funk & Rogge, 2007) for individuals across the income spectrum. This study uses data from the National Couples' Health and Time Study, which sampled married and cohabiting adults in the United States from late 2020 to early 2021. Differential item functioning in the CSI-4 was assessed based on income (N = 3,497) using item response theory. For individuals experiencing lower incomes, the CSI-4 demonstrated differential functioning compared to their more affluent peers, with a bias against individuals experiencing lower incomes equating to average scores 1.38 units lower than higher income participants on the CSI-4 despite having the same true levels of relationship satisfaction. Differential item functioning was less pronounced when comparing item performance of lower- and middle-income participants, equating to lower income participants scoring .21 units lower than middle-income individuals despite having the same true levels of relationship satisfaction. While advancing diversity in family science, researchers should employ psychometric methods to understand how measures perform across groups, which will help prevent the misinterpretation of statistical bias as real group differences. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Reexamining gender differences and the transdiagnostic boundaries of various conceptualizations of perseverative cognition.

Research examining gender differences in perseverative cognition (repetitive, negative, and difficult-to-control thoughts) has focused on depressive rumination and internalizing syndromes. This study examines the transdiagnostic role of depressive rumination, anger rumination, and repetitive negative thinking across gender on internalizing and externalizing symptoms. Utilizing an ethnoracially diverse sample (33% Black, 35% Latinx, 32% White non-Hispanic) of n = 1,187 young adults (49.5% women), we found equivalent instrument functioning across gender for depressive rumination (specifically brooding), anger rumination, and internalizing problems. Differential item functioning was found for repetitive negative thinking and externalizing problems; partial metric and scalar invariance were established for repetitive negative thinking, and partial metric invariance was established for externalizing problems. After accounting for bias in measurement, women engaged in more perseverative cognition, though effects were small for brooding and anger rumination and large for repetitive negative thinking. Different types of perseverations were positively associated with internalizing and externalizing symptoms across gender. Perseverative cognition may be a transdiagnostic mechanism beyond internalizing problems. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Linguistic and cultural validation of symptom questionnaire for visual dysfunctions (SQVD) for psychometric analysis in the Chinese context.

The Spanish scale symptom questionnaire for visual dysfunctions (SQVD) was sinicized and tested for reliability and validity in the Chinese context, employing both classical measurement theory and item response theory. A meticulous translation was conducted using the modified Brislin translation model, with input from experts for cross-cultural debugging and in-depth review. Following a pre-survey study, the Chinese version of the SQVD was finalized. A convenience sampling method was used to select 270 patients from the target group and 252 valid questionnaires were successfully collected. The Rasch model was employed to assess response category functionality, fit statistics, unidimensionality, person and item reliability, separation, targeting, and differential item functioning. Classical test theory was applied to evaluate internal consistency and retest reliability, supplemented by correlation analysis. Job characteristic curves were also plotted to assess diagnostic accuracy. The Chinese SQVD conformed to a unidimensional structure with excellent reliability and validity. Person and item reliabilities were 0.85 and 0.99, respectively, indicating, high stability. Person and item separation indices were 2.37 and 11.54, respectively, signifying strong differentiation ability. Retest reliability was 0.917, further emphasizing the stability of the scale. The area under the receiver operating characteristic curve was 0.908 (95% CI: 0.854, 0.962), with a cutoff value of 7.5 and Youden index of 0.733, highlighting the scale's high diagnostic accuracy. The translated and culturally adapted Chinese SQVD demonstrated excellent psychometric properties. With streamlined items, short assessment time, and high efficiency, the scale is a stable and reliable clinical tool for detecting a variety of conditions related to refractive, regulatory, and binocular vision dysfunction.

Myths about suicide - validating the Hungarian version of the Literacy of Suicide Scale (H-LOSS) on a community sample

BackgroundSuicide was exceptionally high in Hungary in the last century. According to Eurostat, Hungary ranks second in the EU in death by suicide and was among the few countries where the suicidal tendencies increased in 2020. Primary tasks of suicide prevention programs are to develop suicide literacy and dispel myths and misconceptions about suicide. Therefore, the goal of our research was the Hungarian validation of the 26-item Literacy of Suicide Scale (LOSS). Methods: 749 people (382 female (51.0%), 364 male (48.6%), 3 identify as non-binary or other (0.4%); 4 identifying as transgender (0.5%)) participated in our online cross-sectional survey with a mean age of 32.4 years (SD = 14.5 years). The H-LOSS questionnaire was adapted using the 2PL (two-parameter logistic) model with WLSE (weighted least squares) estimation in item response theory method, similarly to the original English version.ResultsScale unidimensionality was confirmed. Model fit indices and internal reliability indicators were acceptable. Item infit and outfit values were adequate, item discrimination values were within range, but one item had extremely high and three items had extremely low item difficulty parameters. Few items had differential item functioning by age, gender and own suidice attempt.ConclusionsThe H-LOSS scale deemed to be appropriate for assessing suicide literacy in Hungarian speaking samples.

Examining measures of schizotypy for gender and racial bias using item response theory and differential item functioning

Schizotypy involves schizophrenia-like traits and symptoms, with the Wisconsin Schizotypy Scales (WSS) being frequently used in previous research. There is some evidence that schizophrenia-spectrum symptom levels, including when using the WSS, might vary both by gender and by race and ethnicity. However, previous research has rarely examined to what extent the WSS show gender and racial bias. Further, this has not been previously examined for the Cognitive Slippage Scale, a measure of disorganized schizotypy. In this study, we examined biases for a subset of items from the WSS in a large sample of undergraduate students (n = 21,829). Using item response theory to test for levels of differential item functioning (DIF), we found some evidence of problematic DIF for all scales, including for negative, positive, and disorganized schizotypy scales. There was evidence of problematic DIF especially by gender and for Black and Multiracial participants. Overall, the current results suggest that gender and/or racial bias on these scales should be an important consideration in using these scales and our results could have implications for assessment of schizophrenia-spectrum symptoms.

Improving outcome measures in late onset Pompe disease: Modified Rasch-Built Pompe-Specific Activity scale.

The Rasch-Built Pompe-Specific Activity (R-PAct) scale is a patient-reported outcome measure specifically designed to quantify the effects of Pompe disease on daily life activities, developed for use in Dutch- and English-speaking countries. This study aimed to validate the R-PAct for use in other countries. Four other language versions (German, French, Italian, and Spanish) of the R-PAct were created and distributed among Pompe patients (≥16 years old) in Germany, France, Spain, Italy, and Switzerland and pooled with data of newly diagnosed patients from Australia, Belgium, Canada, the Netherlands, New Zealand, the USA, and the UK and the original validation cohort (n = 186). The psychometric properties of the scale were assessed by exploratory factor analysis and Rasch analysis. Data for 520 patients were eligible for analysis. Exploratory factor analysis suggested that the items separated into two domains: Activities of Daily Living and Mobility. Both domains independently displayed adequate Rasch model measurement properties, following the removal of one item ("Are you able to practice a sport?") from the Mobility domain, and can be added together to form a "higher order" factor as well. Differential item functioning (DIF)-by-language assessment indicated DIF for several items; however, the impact of accounting for DIF was negligible. We recalibrated the nomogram (raw score interval-level transformation) for the updated 17-item R-PAct scale. The minimal detectable change value was 13.85 for the overall R-PAct. After removing one item, the modified-R-PAct scale is a valid disease-specific patient-reported outcome measure for patients with Pompe disease across multiple countries.

An interplay between cross-cultural and psychometric factors in the Montreal Cognitive Assessment: Experience from the language of a small nation

The present study aimed to test the hypothesis that the total word length on the Memory subtest of the Czech version of the MoCA, which is 12 syllables compared to the English version of 7 syllables, would have a significant effect on Delayed Recall scores compared to the newly created well-balanced version of the test (further MoCA-WLE). In the original Czech version of MoCA, we replaced the 12-syllable word list in the Memory subtest with a 7-syllable list (MoCA-WLE) to make it equivalent to the standard English version in this respect. We analyzed data from 83 participants in the original MoCA group (70.63 ± 7.01 years old, 14.61 ± 3.17 years of education, 30.12% males) and 83 participants in the MoCA-WLE group (70.72 ± 6.95 years old, 14.93 ± 3.48 years of education, 30.12% males). We did not find evidence for a significant word-length effect in the original MoCA versus MoCA-WLE Delayed Recall in either the Mann–Whitney U test (W = 3418.0, p = .932) or multilevel binomial regression (b = 0.10, 95% posterior probability interval [−0.46, 0.68]). The present study shows cross-cultural limits in the adaptation of the test material. The results underline the caveats of such an approach to test adaptation. Fortunately, 12-syllables in the MoCA Memory Czech version versus the original 7-syllable list did not show a detectable word-length effect. We did not find evidence for differential item functioning or cultural item bias. The original MoCA Czech version is psychometrically comparable to the original English version.

Online Psychometric Performance of the Athens Insomnia Scale among Colombian People

Abstract Introduction The Athens Insomnia Scale (AIS) has shown acceptable performance in face-to-face applications. However, knowledge about AIS performance in online applications could be improved. Objective The study aimed to perform confirmatory factor, internal consistency, and differential item functioning analyses of AIS in two Colombian online samples. Material and methods Two samples were recruited online at different times (n = 700, M = 37.05, SD = 12.7 and n = 292, M = 60.4, SD = 17.6). Separated confirmatory factor analyses and Cronbach’s alpha and McDonald’s omega were computed for each factor. Results The data adjusted better for the AIS two-dimensional model in the general and clinical populations. Excellent internal consistency was observed among the general population sample and clinical sample. Conclusions The AIS shows a two-dimensional structure and high internal consistency in two online samples in Colombia. Corroboration of these findings in other samples is necessary.

An investigation into the effect of different missing data imputation methods on IRT-based differential item functioning

The purpose of this study is to examine the effect of missing data imputation methods, namely regression imputation (RI), multiple imputation (MI) and k-nearest neighbor (kNN) on differential item functioning (DIF). In this regard, the datasets used in the research were created by deleting some of the data via the missing completely at random mechanism from the complete datasets obtained from 600 students in Türkiye, the United Kingdom, the USA, New Zealand and Australia, who answered booklets 14 and 15 from the PISA 2018 science literacy test. Data imputation was applied to the datasets through missing data using RI, MI and kNN methods and DIF analysis was performed on all datasets in terms of language and gender variables via Lord’s χ2 method, Raju’s area measurement method and item response theory likelihood ratio method. DIF results from the complete datasets were taken as a reference and they were compared with the results from other datasets. As a result of the research, values close to 10% of accurate imputation were achieved in the RI method depending on language and gen-der variables. In MI and kNN methods, results closest to the complete datasets were obtained at a rate of 5% depending on the language variable. In the MI method, inaccurate results were obtained in all proportions in terms of the gender variable. For the gender variable, the kNN method gave accurate results at rates of 5% and 10%.

Psychometric Properties of Fine Motor Function Measure in Children With Cerebral Palsy: A Rasch Analysis.

Our study utilized Rasch Analysis to examine the psychometric properties of 61-items fine motor function measure (FMFM) in children with cerebral palsy (CP). Partial credit model (PCM) was utilized to test the reliability and validity of FMFM. The response pattern of this samples displayed acceptable fitness to PCM. The analysis results supported the assumption of 1-dimensionality of FMFM. Disordered category thresholds were found in 30 items. Differential item functioning (DIF) was detected in 23 items. Participants with different CP subtypes in different age groups may perform in differently responses patterns. The Rasch analysis produces reliable evidence to support the clinical application of FMFM. Some items may produce inaccurate measurements originated from category structures. Difference in age groups and symptom topography may be associated with variation in fine motor ability among children with CP and leading to unnecessary assessment bias. Hence, FMFM items need modifications to calibrate the former item formulation.

Evaluation of the psychometric properties of the homophobia scale in religious-based university students in Indonesia

This study aims to evaluate the psychometric properties of the homophobia scale in students attending religion-based universities in Indonesia. This research is important as homosexuality is a controversial issue in the country and is still a topic of debate. The Homophobia Scale is a tool that assesses attitudes towards homosexuality through 17 items measuring positive affirmation, negative cognition, and the perceived threat of homosexual behavior. The scale was adapted for the Indonesian context, which is predominantly religious, based on The Heterosexual Attitudes Towards Homosexuality (HATH) Scale and Items, originally translated by bilingual experts. The translated scale was then reviewed for content by psychologists and communication experts, and field-tested for reliability and validity. Data from 327 students aged 18–35 from both state and private religion-based universities were analyzed using Rasch model analyses, including principal component analysis (PCA), reliability analysis, and differential item functioning (DIF) assessment. The study found that the homophobia scale accounted for 42.4 % of the raw variance, indicating its unidimensionality. The scale demonstrated an acceptable level of personal reliability and excellent reliability for individual items. Results revealed significant demographic effects, with age and study program showing more differential item functioning (DIF). Male students were more tolerant towards homosexuals than females. Additionally, students at state universities tended to be more tolerant but held negative views of homosexuality when associated with AIDS. In conclusion, the homophobia scale assessed in this study exhibits promising construct validity and sufficient psychometric properties. The findings indicate that negative stigma towards homosexuals and homophobia still persist among students at religion-based universities in Indonesia, despite limited interaction with homosexuals.

Do conservatives really have an advantage in mental health? Anexamination of measurement invariance.

Many studies have found that conservatives show an advantage in mental health and happiness and various causes of this have been debated (e.g., religiousness, ideology, or genetics). However, not much attention has been given to examining whether this advantage is psychometrically real, or whether it is due to test bias. We analyzed data from two large Finnish surveys of adults (Ns = 848 and 4,978) from Lahtinen (2024), that measured general anxiety and depression symptoms, as well as a new wokeness scale. Using differential item functioning tests, we found no evidence for measurement bias in these scales. The correlation between index scores of wokeness and mental health (internalizing) was -0.36, which increased to -0.41 when measurement error was removed. The association between wokeness and anxiety (r = -0.33, adjusted r = 0.37) was stronger than wokeness and depression (r = 0.20, adjusted r = 0.22).

A person-reported cumulative social risk measure does not show bias by income and education

BackgroundSocial risk such as housing instability, trouble affording medical care and food insecurity are a downstream effect of social determinants of health (SDOHs) and are frequently associated with worse health. SDOHs include experiences of racism, sexism and other discrimination as well as differences in income and education. The collective effects of each social risk a person reports are called cumulative social risk. Cumulative social risk has traditionally been measured through counts or sum scores that treat each social risk as equivalent. We have proposed to use item response theory (IRT) as an alternative measure of person-reported cumulative social risk as IRT accounts for the severity in each risk and allows for more efficient screening with computerized adaptive testing.MethodsWe conducted a differential item functioning (DIF) analysis comparing IRT-based person-reported cumulative social risk scores by income and education in a population-based sample (n = 2122). Six social risk items were analyzed using the two-parameter logistic model and graded response model.ResultsAnalyses showed no DIF on an IRT-based cumulative social risk score by education level for the six items examined. Statistically significant DIF was found on three items by income level but the ultimate effect on the scores was negligible.ConclusionsResults suggest an IRT-based cumulative social risk score is not biased by education and income level and can be used for comparisons between groups. An IRT-based cumulative social risk score will be useful for combining datasets to examine policy factors affecting social risk and for more efficient screening of patients for social risk using computerized adaptive testing.

Bayesian Adaptive Lasso for Detecting Item-Trait Relationship and Differential Item Functioning in Multidimensional Item Response Theory Models.

In multidimensional tests, the identification of latent traits measured by each item is crucial. In addition to item-trait relationship, differential item functioning (DIF) is routinely evaluated to ensure valid comparison among different groups. The two problems are investigated separately in the literature. This paper uses a unified framework for detecting item-trait relationship and DIF in multidimensional item response theory (MIRT) models. By incorporating DIF effects in MIRT models, these problems can be considered as variable selection for latent/observed variables and their interactions. A Bayesian adaptive Lasso procedure is developed for variable selection, in which item-trait relationship and DIF effects can be obtained simultaneously. Simulation studies show the performance of our method for parameter estimation, the recovery of item-trait relationship and the detection of DIF effects. An application is presented using data from the Eysenck Personality Questionnaire.

The Influence of Vision Impairment on the Measurement of Cognition in Older Adults in India: Findings From LASI-DAD.

Prior investigations have not considered whether poor vision biases cognitive testing. However, such research is vital given increasing evidence that vision impairment (VI) may be an important modifiable risk factor for dementia, particularly in low- and middle-income settings where the prevalence of VI is high. This study employed data from 3784 participants in Wave 1 of the Longitudinal Aging Study in India-Diagnostic Assessment of Dementia (LASI-DAD) who underwent both visual acuity and cognitive function testing. We used multiple indicators and multiple causes models to assess differential item functioning (DIF; eg, bias) in cognitive testing by objectively measured distance and near VI. Multivariable linear regression was used to model the association between VI and cognitive factor scores before and after DIF adjustment. Analyses were performed for general cognition and separate cognitive domains, corresponding to memory, language/fluency, executive functioning, and visuospatial performance. Models were adjusted for demographic, health, and socioeconomic covariates. Participants in our sample were 60 and older. Most participants with VI were 60-69 years old (59.6%) and 50.8% were female. Individuals experiencing both distance and near VI tended to be older, have lower educational attainment, be married, reside in rural settings, and belong to lower consumption and BMI categories. Both distance and near VI were associated with poorer cognition before and after DIF-adjustment. Differences between DIF-unadjusted and -adjusted scores were small compared to the standard error of measurement, indicating no evidence of meaningful measurement differences by VI. In well-conducted large-scale surveys, bias in cognitive testing due to VI is likely minimal. Findings strengthen previous evidence on the association between VI and dementia by showing that such associations are unlikely to be attributable to vision-related measurement error in the assessment of cognitive functioning.

Activities-specific balance confidence scale: a Rasch evaluation of the Arabic version in lower-limb prosthetic users

Purpose To evaluate the psychometric properties of the Arabic version of the Activities-Specific Balance Confidence Scale using the 5-option response categories for individuals with lower limb amputation (ABC-5/Ar). Materials and methods This was a methodological study on a convenience sample of individuals with unilateral lower-limb amputation attending outpatient rehabilitation centres in Saudi Arabia and Turkey (N = 155). Rasch analysis (WINSTEPS version 4.6.5) was used to examine the 5-categories rating scale structure, item fit, item difficulty hierarchy, person separation index, unidimensionality, local item dependency, and differential item functioning. Results The ABC-5/Ar 5-response option demonstrated an appropriate model fit. Most items fit the Rasch model, except for item #12 “walk in a crowded mall,” which showed an overfitting value of 0.63. The person separation indices 2.95 (Cronbach’s α = 0.96). Principal component analysis of residuals confirmed the unidimensionality of the scale; however, local dependency was detected between item #14 “Ride in escalator holding rail,” and item #15 “Ride in escalator not holding rail.” Conclusions The findings suggest that the ABC-5/Ar shows promise in assessing balance confidence in Arabic-speaking lower-limb prosthesis users. However, further studies with larger sample sizes and in diverse clinical contexts are needed to confirm its effectiveness in various clinical settings.

SIMEX-Based and Analytical Bias Corrections in Stocking–Lord Linking

Stocking–Lord (SL) linking is a popular linking method for group comparisons based on dichotomous item responses. This article proposes a bias correction technique based on the simulation extrapolation (SIMEX) method for SL linking in the 2PL model in the presence of uniform differential item functioning (DIF). The SIMEX-based method is compared to the analytical bias correction methods of SL linking. It turned out in a simulation study that SIMEX-based SL linking performed best, is easy to implement, and can be adapted to other linking methods straightforwardly.

Revisiting the English Short Measure for Adult Playfulness (SMAP): An Investigation of Reliability, Validity, and Cross-Cultural Comparisons

Adult playfulness describes the personality of the quick initiation and strong intensity of enjoyable experiences coupled with the frequency of engaging in playful behaviors. In addition to examining the reliability and validity of the Short Measure for Adult Playfulness (SMAP), we compared the psychometric properties of the SMAP across (a) 4- and 7-point answer formats; (b) German and English language versions; and (c) gender. The SMAP and criterion validity measures were distributed across three independent samples from Canada (N total = 1,177) and a German sample (N = 660). Results supported a unidimensional solution with strong factor loadings for all items (>.50 across all samples) and high test-retest reliability (r = .67, BF10>100). In item response theory analyses, high discrimination parameters were found across the latent continuum. Although both the four-point and seven-point Likert scale versions showed robust item properties, the seven-point Likert scale version yielded more precise measurement of higher playfulness (θ > 1.50). No significant differential item functioning was located across gender or language. Finally, results supported external construct validity via associations with broad personality traits, humor, and orientations to happiness. The present investigation provides support for the reliability and validity of the English SMAP for concise adult playfulness measurement.