Cross-cultural Differential Item Functioning Research Articles

Dimensionality, Factorial Invariance, and Cross-Cultural Differential Item Functioning of the Short Dark Tetrad (SD4) in Italian, Romanian, and UK Samples

Abstract: The present research tested the dimensionality and cross-cultural differential item functioning of the Short Dark Tetrad (SD4; Paulhus et al., 2021 ) in samples of university students (total N = 804) from Italy ( N = 268), Romania ( N = 313), and the United Kingdom ( N = 223), respectively. Multidimensional Item Response Theory was used to model the data and ordinal logistic regression for differential item functioning analyses. The results showed that a four-factor model was the best fit for the data. However, the model showed metric non-invariance between the three samples, as well as non-negligible differential item functioning for several items in each of the four factors, in line with findings from previous research on cultural differences in antagonistic traits. Implications for theory and research are discussed.

Assessing the differential item functioning of PHQ-9 items for diverse racial and ethnic adults with mental health and/or substance use disorder diagnoses: A retrospective cohort study

BackgroundImproving health equity in depression care and suicide screening requires that measures like the Patient Health Questionnaire 9 (PHQ-9) function similarly for diverse racial and ethnic groups. We evaluated PHQ-9 differential item functioning (DIF) between racial/ethnic groups in a retrospective cohort study of secondary electronic health record (EHR) data from eight healthcare systems. MethodsThe population (n = 755,156) included patients aged 18–64 with mental health and/or substance use disorder (SUD) diagnoses who had a PHQ-9 with no missing item data in the EHR for primary care or mental health visits between 1/1/2009–9/30/2017. We drew two random samples of 1000 from the following racial/ethnic groups originally recorded in EHRs (n = 14,000): Hispanic, and non-Hispanic White, Black, Asian, American Indian/Alaska Native, Native Hawaiian/Other Pacific Islander, multiracial. We assessed DIF using iterative hybrid ordinal logistic regression and item response theory with p < 0.01 and 1000 Monte Carlo simulations, where change in model R2 > 0.01 represented non-negligible (e.g., clinically meaningful) DIF. ResultsAll PHQ-9 items displayed statistically significant, but negligible (e.g., clinically unmeaningful) DIF between compared groups. The negligible DIF varied between random samples, although six items showed negligible DIF between the same comparison groups in both random samples. LimitationsOur findings may not generalize to disaggregated racial/ethnic groups or persons without mental health and/or SUD diagnoses. ConclusionsWe found the PHQ-9 had clinically unmeaningful cross-cultural DIF for adult patients with mental health and/or SUD diagnoses. Future research could disaggregate race/ethnicity to discern if within-group identification impacts PHQ-9 DIF.

A Comparison of Non-verbal Maternal Care of Male and Female Infants in India and the United Kingdom: The Parent-Infant Caregiving Touch Scale in Two Cultures.

Differences in infant caregiving behavior between cultures have long been noted, although the quantified comparison of touch-based caregiving using uniform standardized methodology has been much more limited. The Parent-Infant Caregiving Touch scale (PICTS) was developed for this purpose and programming effects of early parental tactile stimulation (stroking) on infant hypothalamic-pituitary adrenal (HPA)-axis functioning (stress-response system), cardiovascular regulation and behavioral outcomes, similar to that reported in animals, have now been demonstrated. In order to inform future studies examining such programming effects in India, we first aimed to describe and examine, using parametric and non-parametric item-response methods, the item-response frequencies and characteristics of responses on the PICTS, and evidence for cross-cultural differential item functioning (DIF) in the United Kingdom (UK) and India. Second, in the context of a cultural favoring of male children in India, we also aimed to test the association between the sex of the infant and infant “stroking” in both cultural settings. The PICTS was administered at 8–12 weeks postpartum to mothers in two-cohort studies: The Wirral Child Health and Development Study, United Kingdom (n = 874) and the Bangalore Child Health and Development Study, India (n = 395). Mokken scale analysis, parametric item-response analysis, and structural equation modeling for categorical items were used. Items for two dimensions, one for stroking behavior and one for holding behavior, could be identified as meeting many of the criteria required for Mokken scales in the United Kingdom, only the stroking scale met these criteria in the sample from India. Thus, while a comparison between the two cultures was possible for the stroking construct, comparisons for the other non-verbal parenting constructs within PICTS were not. Analyses revealed higher rates of early stroking being reported for the United Kingdom than India, but no sex differences in rates in either country and no differential sex difference by culture. We conclude that PICTS items can be used reliably in both countries to conduct further research on the role of early tactile stimulation in shaping important child development outcomes.

Detecting cross-cultural differential item functioning for increasing validity: An example from the American board of family medicine in-training examination

Background: The present study describes the process used to detect items for cross-cultural differential item functioning (cc-DIF) and attempts to understand cc-DIF by both statistical analysis and content review using a cultural lens. Methods: Data from the 2014 American Board of Family Medicine (ABFM) In-Training Examination (ITE). Results: cc-DIF existed in ten items on the 2014 ABFM ITE and could not be eliminated over the residency program years. International medical school graduates were benefited by seven items, whereas the United States medical school graduates (USMGs) were benefited by three items. Discussion: Cultural specificities and differential content familiarity likely are the primary reasons for items exhibiting cc-DIF. Conclusions: Investigating cc-DIF is recommended for any examination involving multicultural groups. Further, items exhibiting cc-DIF offer opportunities for students to reflect on their implicit cultural differences that may ultimately affect how they practice medicine in a multicultural society.

Adjusting for cross-cultural differences in computer-adaptive tests of quality of life

PurposePrevious studies using the WHOQOL measures have demonstrated that the relationship between individual items and the underlying quality of life (QoL) construct may differ between cultures. If unaccounted for, these differing relationships can lead to measurement bias which, in turn, can undermine the reliability of results.MethodsWe used item response theory (IRT) to assess differential item functioning (DIF) in WHOQOL data from diverse language versions collected in UK, Zimbabwe, Russia, and India (total N = 1332). Data were fitted to the partial credit ‘Rasch’ model. We used four item banks previously derived from the WHOQOL-100 measure, which provided excellent measurement for physical, psychological, social, and environmental quality of life domains (40 items overall). Cross-cultural differential item functioning was assessed using analysis of variance for item residuals and post hoc Tukey tests. Simulated computer-adaptive tests (CATs) were conducted to assess the efficiency and precision of the four items banks.ResultsSplitting item parameters by DIF results in four linked item banks without DIF or other breaches of IRT model assumptions. Simulated CATs were more precise and efficient than longer paper-based alternatives.DiscussionAssessing differential item functioning using item response theory can identify measurement invariance between cultures which, if uncontrolled, may undermine accurate comparisons in computer-adaptive testing assessments of QoL. We demonstrate how compensating for DIF using item anchoring allowed data from all four countries to be compared on a common metric, thus facilitating assessments which were both sensitive to cultural nuance and comparable between countries.

Developing the Polish Educational Needs Assessment Tool (Pol-ENAT) in rheumatoid arthritis and systemic sclerosis: a cross-cultural validation study using Rasch analysis

ObjectivesTo undertake cross-cultural adaptation and validation of the educational needs assessment tool (ENAT) for use with people with rheumatoid arthritis (RA) and systemic sclerosis (SSc) in Poland.MethodsThe study involved two main phases: (1) cross-cultural adaptation of the ENAT from English into Polish and (2) Cross-cultural validation of Polish Educational Needs Assessment Tool (Pol-ENAT). The first phase followed an established process of cross-cultural adaptation of self-report measures. The second phase involved completion of the Pol-ENAT by patients and subjecting the data to Rasch analysis to assess the construct validity, unidimensionality, internal consistency and cross-cultural invariance.ResultsAn adequate conceptual equivalence was achieved following the adaptation process. The dataset for validation comprised a total of 278 patients, 237 (85.3 %) of which were female. In each disease group (145, RA and 133, SSc), the 7 domains of the Pol-ENAT were found to fit the Rasch model, X 2(df) = 16.953(14), p = 0.259 and 8.132(14), p = 0.882 for RA and SSc, respectively. Internal consistency of the Pol-ENAT was high (patient separation index = 0.85 and 0.89 for SSc and RA, respectively), and unidimensionality was confirmed. Cross-cultural differential item functioning (DIF) was detected in some subscales, and DIF-adjusted conversion tables were calibrated to enable cross-cultural comparison of data between Poland and the UK.ConclusionUsing a standard process in cross-cultural adaptation, conceptual equivalence was achieved between the original (UK) ENAT and the adapted Pol-ENAT. Fit to the Rasch model, confirmed that the construct validity, unidimensionality and internal consistency of the ENAT have been preserved.Electronic supplementary materialThe online version of this article (doi:10.1007/s11136-014-0805-6) contains supplementary material, which is available to authorized users.

Calibration of the PROMIS Physical Function Item Bank in Dutch Patients with Rheumatoid Arthritis

ObjectiveTo calibrate the Dutch-Flemish version of the PROMIS physical function (PF) item bank in patients with rheumatoid arthritis (RA) and to evaluate cross-cultural measurement equivalence with US general population and RA data.MethodsData were collected from RA patients enrolled in the Dutch DREAM registry. An incomplete longitudinal anchored design was used where patients completed all 121 items of the item bank over the course of three waves of data collection. Item responses were fit to a generalized partial credit model adapted for longitudinal data and the item parameters were examined for differential item functioning (DIF) across country, age, and sex.ResultsIn total, 690 patients participated in the study at time point 1 (T2, N = 489; T3, N = 311). The item bank could be successfully fitted to a generalized partial credit model, with the number of misfitting items falling within acceptable limits. Seven items demonstrated DIF for sex, while 5 items showed DIF for age in the Dutch RA sample. Twenty-five (20%) items were flagged for cross-cultural DIF compared to the US general population. However, the impact of observed DIF on total physical function estimates was negligible.DiscussionThe results of this study showed that the PROMIS PF item bank adequately fit a unidimensional IRT model which provides support for applications that require invariant estimates of physical function, such as computer adaptive testing and targeted short forms. More studies are needed to further investigate the cross-cultural applicability of the US-based PROMIS calibration and standardized metric.

OP0283 Validation of the Educational Needs Assessment Tool as a Generic Instrument for Rheumatic Diseases in 7 European Countries

ObjectivesTo validate the educational needs assessment tool (The ENAT) as a generic tool with which to assess the educational needs of patients with rheumatic diseases in European Countries.MethodsThe study followed...

A comparison of Rasch item‐fit and Cronbach's alpha item reduction analysis for the development of a Quality of Life scale for children and adolescents

This study compares item reduction analysis based on classical test theory (maximizing Cronbach's alpha - approach A), with analysis based on the Rasch Partial Credit Model item-fit (approach B), as applied to children and adolescents' health-related quality of life (HRQoL) items. The reliability and structural, cross-cultural and known-group validity of the measures were examined. Within the European KIDSCREEN project, 3019 children and adolescents (8-18 years) from seven European countries answered 19 HRQoL items of the Physical Well-being dimension of a preliminary KIDSCREEN instrument. The Cronbach's alpha and corrected item total correlation (approach A) were compared with infit mean squares and the Q-index item-fit derived according to a partial credit model (approach B). Cross-cultural differential item functioning (DIF ordinal logistic regression approach), structural validity (confirmatory factor analysis and residual correlation) and relative validity (RV) for socio-demographic and health-related factors were calculated for approaches (A) and (B). Approach (A) led to the retention of 13 items, compared with 11 items with approach (B). The item overlap was 69% for (A) and 78% for (B). The correlation coefficient of the summated ratings was 0.93. The Cronbach's alpha was similar for both versions [0.86 (A); 0.85 (B)]. Both approaches selected some items that are not strictly unidimensional and items displaying DIF. RV ratios favoured (A) with regard to socio-demographic aspects. Approach (B) was superior in RV with regard to health-related aspects. Both types of item reduction analysis should be accompanied by additional analyses. Neither of the two approaches was universally superior with regard to cultural, structural and known-group validity. However, the results support the usability of the Rasch method for developing new HRQoL measures for children and adolescents.

The Behcet's disease activity index.

To identify a subset of clinical features of Behçet's disease (BD) that can be summated to form an overall index of disease activity appropriate for clinical and research use internationally. Completed Behçet's Disease Current Activity Forms were collected from a total of 524 patients with BD from five countries. The data from 14 questions on the form were subjected to Rasch analysis to establish whether these items form a hierarchical and unidimensional scale of disease activity, both within and between countries. The data showed a good fit to the Rasch model within three countries using a dichotomous scoring function. However, when the data from these three countries were pooled, the fit to the model was poor. Cross-cultural differential item functioning (DIF) was found in seven items in the pooled data. When the items with DIF by country were separated and two items were removed, the resulting 26-item scale showed a good fit to the Rasch model. Within Turkey, Korea and the UK, the 14 items can be summated to give an index of disease activity. Analysis of the pooled data confirmed that the index is not suitable for comparison between countries or for pooling of data in the raw form, but after fitting the data to the Rasch model such comparisons can be made. This gives a scaling tool that is quick and easy to use in the clinical situation.