Evidence Of Item Bias Research Articles

Evidence of item bias in a national flourishing measure for autistic youth.

Flourishing is a positive health indicator that aligns with strengths-based perspectives and measures within autism research. Flourishing indicators were recently included in the National Survey of Children's Health (NSCH) and have been used to evidence disparities in flourishing experienced by autistic children compared to non-autistic peers. Yet, little has been done to examine the utility of standard flourishing items for this population. This study examined the NSCH caregiver-reported flourishing items for measurement item bias. A cross-sectional, representative sample of autistic and non-autistic US children aged 6-17 years (n= 41,691) was drawn from the 2018-2019 NSCH public dataset. A confirmatory factor analysis using a multiple indicators and multiple causes model (MIMIC-CFA) was conducted to (1) test for differential item functioning (DIF; i.e., measurement bias); and (2) estimate latent mean group differences after controlling for DIF. Findings supported a 3-factor (social competence, school motivation, and behavioral control), 10-item model structure consistent with past literature, yet measurement bias was evident for 6 of the 10 items. Persistent group differences, after accounting for DIF and covariates, indicates that caregivers of autistic children perceive their children are experiencing meaningfully lower flourishing outcomes compared to caregivers of non-autistic children. However, evidence of measurement bias for items related to the social competence dimension calls into question the applicability of this measure for autistic children. Further interpretation of group differences and use of this measure should be approached with caution.

Re-engineering the Hong Kong Quality of Life Questionnaire to Assess Cataract Surgery Outcomes

To assess psychometric properties of the Hong Kong Quality of Life Questionnaire (HKQ) using Rasch analysis and use it to assess cataract surgery outcomes. The HKQ was interviewer administered to participants at baseline (preoperative) and 12 months of follow-up (postoperative). Rasch analysis was used to assess and improve psychometric properties of the HKQ using the preoperative data. The responsiveness of the HKQ was assessed on the stacked preoperative and postoperative data, reported with the effect size. One hundred ninety-nine participants (median age: 72 years; interquartile range: 65 to 78 years) with cataract completed the HKQ at the preoperative assessment. There were more women (58%) and most of the participants had bilateral cataract (83.9%). Nearly half of the participants had undergone surgery in one eye (45.2%) followed by bilateral surgery (38.7%). Psychometric assessment guided by Rasch analysis resulted into a 15-item HKQ with promising psychometric properties including adequate measurement precision (2.09), no misfitting items, near perfect targeting (-0.05), unidimensionality, and with no evidence of item bias. For those who completed the HKQ (n = 82, 41.2%) at the 12-month follow-up visit, all groups demonstrated statistically significant gains in the HKQ scores, with the highest gain in participants who had undergone bilateral surgery (effect size: 2.61). The 15-item HKQ was valid and psychometrically sound and might be a highly responsive instrument to measure cataract surgery outcomes in China. This study demonstrated that cataract surgery significantly improves quality of life in the Chinese population. [J Refract Surg. 2018;34(6):413-418.].

Rasch Analysis Supports the Use of the Pain Self-Efficacy Questionnaire

The Pain Self-Efficacy Questionnaire (PSEQ) is used by physical therapists in clinical practice and in research. However, current understanding of the PSEQ's measurement properties is incomplete, and investigators cannot be confident that it provides unbiased information on patient self-efficacy. The aims of this study were: (1) to investigate the scale properties of the PSEQ using Rasch analysis and (2) to determine whether age, sex, pain intensity, pain duration, and pain-related disability bias function of the PSEQ. This was a retrospective study; data were obtained from 3 existing studies. Data were combined from more than 600 patients with low back pain of varying duration. Rasch analysis was used to evaluate targeting, category ordering, unidimensionality, person fit, internal consistency, and item bias. There was evidence of adequate category ordering, unidimensionality, and internal consistency of the PSEQ. Importantly, there was no evidence of item bias. The PSEQ did not adequately target the sample; instead, it targeted people with lower self-efficacy than this population. Item 7 was hardest for participants to endorse, showing excessive positive misfit to the Rasch model. Response strings of misfitting persons revealed older participants and those reporting high levels of disability. The individual items of the PSEQ can be validly summed to provide a score of self-efficacy that is robust to age, sex, pain intensity, pain duration, and disability. Although item 7 is the most problematic, it may provide important clinical information and requires further investigation before its exclusion. Although the PSEQ is commonly used with people with low back pain, of whom the sample in this study was representative, the results suggest it targets patients with lower self-efficacy than that observed in the current sample.

Evaluating fidelity to the wraparound service model for youth: application of item response theory to the Wraparound Fidelity Index.

The wraparound process is a mechanism for multisystem planning and care coordination for youth with serious emotional and behavioral problems. Fidelity monitoring is critical to effective implementation of evidence-based practices in children's mental health, as it helps ensure that complex interventions like wraparound are implemented as intended. The 40-item Wraparound Fidelity Index, Version 4 (WFI-4; Bruns, Burchard, Suter, Leverentz-Brady, & Force, 2004) is the most frequently used measure of fidelity to the wraparound process, but analysis of its psychometric properties is insufficient. An item response theory approach, Rasch partial credit models for ordered polytomous data, was used on ratings from 1,234 facilitators, 1,006 caregivers, and 221 team members, focused on 1,478 youths (55% male). Results indicated the WFI-4 measured a unidimensional construct, with little evidence of item bias and good item and model fit. However, the item information curve was skewed, with most people endorsing high-fidelity responses, and several items had duplicative location estimates. A reduced 20-item measure is proposed. Internal reliability estimates for scores from this reduced measure were approximately equivalent to the longer measure. However, both versions would benefit from additional items located in the highest fidelity area of either version of the scale where scores by greater than half of our sample fall, but only 3 items are located.

Using the 12‐item General Health Questionnaire to screen psychological distress from survivorship to end‐of‐life care: dimensionality and item quality

This study aimed (i) to determine the factor structure of the 12-item General Health Questionnaire (GHQ-12) across the cancer trajectory represented by samples from three cancer care settings and (ii) to appraise the item misfit and differential item functioning (DIF) of the GHQ-12. Data were from cancer outpatient (n = 200), general community (n = 364) and palliative care (n = 150) settings. The factor structure was tested using exploratory factor analysis followed by confirmatory factor analysis. The factors were assessed for correlation using Spearman's ρ. The analyses were run separately for standard GHQ, Likert, modified Likert and chronic GHQ scoring and for the individual cancer settings. The best scoring method within the cancer setting was determined by Akaike's information criterion (AIC). Item misfit (mean square, MNSQ; standardised z-score, ZSTD) and DIF were assessed using the Rasch model. The best scoring method was the chronic GHQ for the cancer outpatient (AIC = -45.8), modified Likert for the general community (AIC = 9.6) and standard GHQ for the palliative care (AIC = -43.0). The GHQ-12 displayed a correlated two-factor structure ('social dysfunction' and 'distress'); Spearman ρ values were 0.69, 0.82 and 0.88 in the cancer outpatient, the general community and the palliative care, respectively. One item in the palliative care indicated misfit (MNSQ = 1.62, ZSTD = 3.0). Five items in the cancer outpatient showed DIF by gender and age. Two items in the palliative care showed DIF by gender. The GHQ-12 was more problematic (less clear factor structure and evidence of item bias) for newly diagnosed patients, less problematic for patients approaching end-of-life and satisfactory for patients between those times.

Applicants faking good: evidence of item bias in the NEO PI-R

Faking good by applicants threatens the validity of using personality measures in selection. Previous research suggests that Conscientiousness is the most easily faked while Openness to Experience is the least easily faked of the Big 5 measures. Structural equation modeling was used to assess the effect of faking on the NEO PI-R facets of these measures. When comparing applicant with student responses, differential item functioning (DIF) was found in four of the Conscientiousness facets and in all the Openness to Experience facets. The practical implications of these findings for the use of personality tests in selection are discussed, together with ideas for future research.

Cross-cultural measurements of psychological well-being: the psychometric equivalence of Cantonese, Vietnamese, and Laotian translations of the Affect Balance Scale.

This paper evaluates the cultural equivalence of Cantonese, Vietnamese, and Laotian translations of the Affect Balance Scale. The scale was completed by 399 Vietnamese, 193 Laotian, 756 Cantonese, and 319 English speakers who were participants in the Clarke Institute-University of Toronto Refugee Resettlement Project (n = 1667). Confirmatory factor analyses indicated a good fit between the hypothesized two-factor model (separate factors for positive and negative affect) across the original English-language version and each of the Asian-language translations. Factorial invariance (numbers and patterns of factor loadings) was evident across all versions of the scale. No evidence of item bias was detected by mixed Language x Item analyses of variance. Acceptable reliability was observed; coefficient alphas ranged from .62 to .72 for positive affect and from .62 to .70 for negative affect items. These findings substantiate the cultural equivalence of the three translations of the scale for population health research. Important future research directions made possible by the availability of culturally equivalent instruments are discussed.

Measurement issues in research on social support and health.

The aims were: (1) to identify methodological problems that may explain the inconsistencies and contradictions in the research evidence on social support and health, and (2) to validate a frequently used measure of social support in order to determine whether or not it could be used in multivariate analyses of population data in research on social support and health. Secondary analysis of data collected in a cross sectional survey of a multistage cluster sample of the population of the United States, designed to study relationships in behavioural, social support and health variables. Statistical models based on item response theory and graph theory were used to validate the measure of social support to be used in subsequent analyses. Data on 1755 men and women aged 20 to 64 years were available for the scale validation. Massive evidence of item bias was found for all items of a group membership subscale. The most serious problems were found in relationship to an item measuring membership in work related groups. Using that item in the social network scale in multivariate analyses would distort findings on the statistical effects of education, employment status, and household income. Evidence of item bias was also found for a sociability subscale. When marital status was included to create what is called an intimate contacts subscale, the confounding grew worse. The composite measure of social network is not valid and would seriously distort the findings of analyses attempting to study relationships between the index and other variables. The findings show that valid measurement is a methodological issue that must be addressed in scientific research on population health.

Evidence of Item Bias in the Verbal Subtests of the WISC-R for Mexican-American Children

The study investigated cultural bias in the verbal items of the Wechsler Intelligence Scale for Children-Revised (WISC-R). Six verbal subtests of the WISC-R were administered to 40 Anglo and 40 Mexican-American subjects matched for grade level, sex, and socioeconomic status. The responses of two groups of subjects to individual items comprising verbal subtests were analyzed by log-linear technique using the likelihood ratio chi-square statistic. The findings revealed that the performance of subjects was homogeneous across two cultural groups on the items comprising Arithmetic, Comprehension, and Digit Span subtests. Ten (13%) of the 79 items comprising Information, Similarities, and Vocabulary subtests were, however, found to be biased against Mexican-American subjects. Three of these items are from the Information subtest, two from Similarities, and the remaining five items from Vocabulary. Implications of these findings for the psychoeducational assessment of minority children were discussed.