Assess Item Bias Research Articles

Development and Validation of a Brief Version of the Vanderbilt Fatigue Scale for Adults: The VFS-A-10.

Listening-related fatigue can be a significant problem for adults who struggle to hear and understand, particularly adults with hearing loss. However, valid, sensitive, and clinically useful measures for listening-related fatigue do not currently exist. The purpose of this study was to develop and validate a brief clinical tool for measuring listening-related fatigue in adults. The clinical scale was derived from the 40-item version of the Vanderbilt Fatigue Scale for Adults (VFS-A-40), an existing, reliable, and valid research tool for measuring listening-related fatigue. The study consisted of two phases. Phase 1 ( N = 580) and Phase 2 ( N = 607) participants consisted of convenience samples of adults recruited via online advertisements, clinical records review, and a pool of prior research participants. In Phase 1, results from item response theory (IRT) analyses of VFS-A-40 items were used to identify high-quality items for the brief (10-item) clinical scale: the VFS-A-10. In Phase 2, the characteristics and quality of the VFS-A-10 were evaluated in a separate sample of respondents. Dimensionality was evaluated using exploratory factor analyses (EFAs) and item quality and characteristics were evaluated using IRT. VFS-A-10 reliability and validity were assessed in multiple ways. IRT reliability analysis was used to examine VFS-A-10 measurement fidelity. In addition, test-retest reliability was assessed in a subset of Phase 2 participants ( n = 145) who completed the VFS-A-10 a second time approximately one month after their initial measure (range 5 to 90 days). IRT differential item functioning (DIF) was used to assess item bias across different age, gender, and hearing loss subgroups. Convergent construct validity was evaluated by comparing VFS-A-10 responses to two other generic fatigue scales and a measure of hearing disability. Known-groups validity was assessed by comparing VFS-A-10 scores between adults with and without self-reported hearing loss. EFA suggested a unidimensional structure for the VFS-A-10. IRT analyses confirmed all test items were high quality. IRT reliability analysis revealed good measurement fidelity over a wide range of fatigue severities. Test-retest reliability was excellent ( rs = 0.88, collapsed across participants). IRT DIF analyses confirmed the VFS-A-10 provided a valid measure of listening-related fatigue regardless of respondent age, gender, or hearing status. An examination of associations between VFS-A-10 scores and generic fatigue/vigor measures revealed only weak-to-moderate correlations (Spearman's correlation coefficient, rs = -0.36 to 0.57). Stronger associations were seen between VFS-A-10 scores and a measure of perceived hearing difficulties ( rs = 0.79 to 0.81) providing evidence of convergent construct validity. In addition, the VFS-A-10 was more sensitive to fatigue associated with self-reported hearing difficulties than generic measures. It was also more sensitive than generic measures to variations in fatigue as a function of degree of hearing impairment. These findings suggest that the VFS-A-10 is a reliable, valid, and sensitive tool for measuring listening-related fatigue in adults. Its brevity, high sensitivity, and good reliability make it appropriate for clinical use. The scale will be useful for identifying those most affected by listening-related fatigue and for assessing benefits of interventions designed to reduce its negative effects.

Cross-cultural adaptation and validation of the Amsterdam Instrumental Activities of Daily Living questionnaire short version German for Switzerland

BackgroundInstrumental Activities of Daily Living (IADL) limitations are associated with reduced health-related quality of life for people with mild cognitive impairment (MCI). For these people, the assessment of IADL is crucial to the diagnostic process, as well as for the evaluation of new interventions addressing MCI. The Amsterdam IADL Questionnaire Short Version (A-IADL-Q-SV) is an established assessment tool with good psychometric properties that has been shown to be robust to cultural differences in Western countries. The aims of this study were to: (1) cross-culturally adapt and validate the A-IADL-Q-SV for the German-speaking population of Switzerland; (2) investigate its cultural comparability; and (3) evaluate further psychometric properties.MethodsThe A-IADL-Q-SV German was pretested on clinicians and participants in a memory clinic setting. The psychometric properties and cultural comparability of the questionnaire were investigated in memory clinic settings including participants with MCI or mild dementia, as well as participants with normal cognition recruited from the community. Item response theory (IRT) was applied to investigate measurement invariance by means of differential item functioning to assess item bias. Additionally, the test–retest reliability on scale level, the construct validity through hypothesis testing and the discriminant validity of the A-IADL-Q-SV German were evaluated.ResultsNinety-six informants of participants with normal cognition, MCI or mild dementia completed the A-IADL-Q-SV German. The basic assumptions for IRT scoring were met. No meaningful differential item functioning for culture was detected between the Swiss and Dutch reference samples. High test–retest reliability on scale level (ICC 0.93; 95% CI 0.9–0.96) was found. More than 75% of the observed correlations between the A-IADL-Q-SV German and clinical measures of cognition and functional status were found to be in the direction and of the magnitude hypothesized. The A-IADL-Q-SV German was shown to be able to discriminate between participants with normal cognition and MCI, as well as MCI and mild dementia.ConclusionsThe A-IADL-Q-SV German is a psychometrically robust measurement tool for a Swiss population with normal cognition, MCI and mild dementia. Thus, it provides a valuable tool to assess IADL functioning in clinical practices and research settings in Switzerland.Trial registration This study was registered retrospectively in July 2019 on ClinicalTrials.gov (NCT04012398).

Understanding the Country's Underachievement in International Assessment: Differential Item and Bundle Functioning Approach

In educational context, Differential Item Functioning (Dif) and Differential Bundle Functioning (Dbf) are usually used in assessing item bias. In present study, it is demonstrated how results of Dif and Dbf analyses can help in better understanding the reasons behind one country’s underachievement in international assessments. In Timss 2011 mathematics assessment, Croatian fourth graders achieved a mean result that was below the international average, while Slovenian and Serbian students achieved mean results that were above the international average. The results of Dif and Dbf analyses indicate that large portion of items exhibiting differences in favour of Slovenian and Serbian students are related to topics that are not taught in Croatian schools during first four grades. The described methodology can be used in investigating other countries’ (under)achievement in international assessments.

Gender Differences in the Endorsement of Symptoms for Depression and Anxiety

Gender-biased items are one explanation for the gender difference in reports of depression and anxiety symptoms. This hypothesis was explored using responses from the Goldberg Anxiety and Depression scales. Responses were drawn from 2 waves of a community-based survey with participants in 3 age cohorts: 20 to 25, 40 to 44, and 60 to 64 at wave 1. The results showed that a 2-factor model fitted the scales well. Multiple group analyses showed that that for all 3 age groups and both time points the fit indices changed minimally between a model that constrained parameters to be equal across gender and a model that permitted variation. These findings indicate that gender differences in the endorsement of items from the Goldberg scales are not because of gender-biased items. This study points to alternate biopsychosocial explanations and highlights the importance of choosing appropriate methods of analysis when assessing item bias.

Mini-Mental Status Examination: Mixed Rasch model item analysis derived two different cognitive dimensions of the MMSE

Objectives This study published in two companion papers assesses properties of the Mini-Mental State Examination (MMSE) with the purpose of improving the efficiencies of the methods of screening for cognitive impairment and dementia. Study Design and Setting An item analysis by conventional and mixed Rasch models was used to explore empirically derived cognitive dimensions of the MMSE, to assess item bias, and to construct diagnostic cut-points. The scores of 1,189 elderly residents were analyzed. Results Two dimensions of cognitive function, which are statistically and conceptually different from those obtained in previous studies, were derived. The corresponding sum scales were (1) age-correlated MMSE scale (A-MMSE scale: orientation to time, attention/calculation, naming, repetition, and three-stage command) and (2) non–age-correlated MMSE scale (B-MMSE scale: orientation to place, registration, recall, reading, and copying). The “writing” item was not included due to differential effects of age and sex. The analysis also showed that the study sample consisted of two cognitively different groups of elderly. Conclusions The findings indicate that a two-scale solution is a stable and statistically supported framework for interpreting data obtained by means of the MMSE. Supplementary analyses are presented in the companion paper to explore the performance of this item response theory calibration as a screening test for dementia.

Scaling the sickness impact profile using item response theory: an exploration of linearity, adaptive use, and patient driven item weights.

The objective of the study was to enhance the clinical interpretation and practicality of the widely used comprehensive Sickness Impact Profile. Item Response Theory (extension of the Rasch model) was used to calibrate the severity of the SIP items, to assess item bias and to construct equally severe short forms of the SIP that can be used interchangeably. The scores of 1507 subjects were analyzed. Of the 127 SIP items, 82 items fitted the extended Rasch model, i.e., the observed proportions of sickness level groups endorsing the items corresponded to the proportions expected by the model. The item severity hierarchy allowed a more straightforward interpretation of the calibrated SIP-82 scores. Some items showed bias in age, gender, or diagnosis groups. The equivalent short forms agreed sufficiently well with the calibrated SIP-82 item pool to be used interchangeably. We observed a moderate correlation between the original SIP item severity weights and the Rasch item severity calibrations (r=0.53). The interpretability and practicality of the SIP was enhanced by the IRT calibration. Using the item calibrations, short forms can be assembled that can be used interchangeably.

Symptom profiles of depression among general medical service users compared with specialty mental health service users.

Since depressive disorders are now eminently treatable and early detection and treatment could bring substantial benefits, it is critical to address alternative presentations of depression in the general medical setting. Concern regarding under-diagnosis of depression in general medical settings has given rise to the question of whether the clinical disorder of depression differs qualitatively or only quantitatively across care settings. Symptom profiles of depression were compared across care sectors to investigate how the presentation of depression among general medical service users might differ qualitatively from specialty mental health service users. Data on depression symptoms within 6 months of interview gathered in three community surveys that were part of the NIMH Epidemiologic Catchment Area Program were analysed using methods developed to assess item bias. The subjects were 4931 and 363 persons who reported a visit to the general medical sector or to specialty mental health respectively, within 6 months of interview. Compared with specialty mental health service users, general medical service users were less likely to present dysphoria (adjusted Odds Ratio, aOR = 0.57; 95% Confidence Interval, CI = 0.38-0.84) and feeling worthless, sinful, or guilty (aOR = 0.63; 95% CI = 0.40-0.98), but were more likely to present fatigue (aOR = 1.71; 95% CI = 1.09-2.69), even after holding constant other characteristics that might influence reporting of symptoms as well as level of depression. These results suggest that there are qualitative differences in depression presenting in general medical care compared with specialty mental health care and call for a re-conceptualization of depression in the general medical setting.

Experiences in the Application of Item Response Theory in Test Construction

Certain potential benefits of using item response theory in test construction are discussed and evaluated using the experience and evidence accumulated during 9 years of using a three-parameter model in the construction of major achievement batteries. We also discuss several cautions and limitations in realizing these benefits as well as issues in need of further research. The potential benefits considered are those of getting sample-free item calibrations and item-free person measurement, automatically equating various tests, decreasing the standard errors of scores without increasing the number of items used by using item pattern scoring, assessing item bias (or differential item functioning) independently of difficulty in a manner consistent with item selection, being able to determine just how adequate a tryout pool of items may be, setting up computer-generated ideal tests drawn from pools as targets for test developers, and controlling the standard error of a selected test at any desired set o...

An Iterative Procedure for Linking Metrics and Assessing Item Bias in Item Response Theory

The presence of biased items may seriously affect methods used to link metrics in item response theory. An iterative procedure designed to minimize this methodological problem was examined in a monte carlo investigation using the two-parameter item re sponse model. The iterative procedure links the scales of independently calibrated parameter estimates using only those items identified as unbiased. Two methods for transforming parameter estimates to a common metric were incorporated into the iterative procedure. The first method links scales by equating the first two moments of the distributions of estimated item diffi culties. The second method determines the linking transformation by minimizing differences across IRT characteristic curve estimates. Results indicate that it erative linking provides a substantial improvement in item bias detection over the noniterative approach. Index terms: Item bias, Item response theory, Iterative method, Linking, Metric linking, Two-parameter item response model.

Internal evidence of bias in the PPVT-R for Anglo-American and Mexican-American children

This study examines item bias on Forms L and M of the Peabody Picture Vocabulary Test-Revised(PPVT-R) for a sample of Anglo-American and Mexican-American children. Analyses of variance (ANOVA) were employed to assess item bias as defined by items X ethnicity interactions. Follow-up analyses were performed using a Bonferroni-type procedure on individual item contrasts. Bias as measured by differences in item difficulty was found in both groups; however, there was no clear pattern of items that were more difficult for either group. The small number of items that were more difficult for one ethnic group than for the other, coupled with the high reliability of performance overall for both groups, suggest that bias in content of the PPVT-R is minimal.

AN ITERATIVE ITEM BIAS DETECTION METHOD

Two strategies for assessing item bias are discussed: methods that compare (transformed) item difficulties unconditional on ability level and methods that compare the probabilities of correct response conditional on ability level. In the present study, the logit model was used to compare the probabilities of correct response to an item by members of two groups, these probabilities being conditional on the observed score. Here the observed score serves as an indicator of ability level. The logit model was iteratively applied: In the Tth iteration, the T items with the highest value of the bias statistic are excluded from the test, and the observed score indicator of ability for the (T + 1)th iteration is computed from the remaining items. This method was applied to simulated data. The results suggest that the iterative logit method is a substantial improvement on the noniterative one, and that the iterative method is very efficient in detecting biased and unbiased items.

Contingency Table Models for Assessing Item Bias

Two strategies for assessing item bias are discussed: Methods based on Item x Group interaction and methods that compare the probabilities of a correct response for different groups conditional on ability level. In latent trait models, correct response probabilties are compared conditional on latent ability; in Scheuneman’s (1979) method these probabilities are compared conditional on the observed test score. Scheuneman’s method is modified to fit the general theory of loglinear and logit models for contingency tables. A distinction is made between uniform and nonuniform item bias, and a method to assess item bias and distinguish between uniform and nonuniform bias is described. Data reported by Scheuneman are used to demonstrate the method in detail; the differences obtained with Scheuneman’s method are discussed. In addition, the method is applied to two tests administered by van der Flier (1980) in Kenya and Tanzania, and the results are compared to those obtained with Scheuneman’s approach.

Contingency Table Models for Assessing Item Bias

Contingency Table Models for Assessing Item Bias