Gender-related Differential Item Functioning Research Articles

Assessing the Construct Validity of the Adult ADHD Self-report Scale for DSM-5 and Prevalence of ADHD in a Danish Population Sample.

This study examines the validity of the ASRS-5 as a new screening tool for ADHD and evaluates its proposed screening cut-off in a general population context. A nationally representative sample of 2,002 individuals aged 18 to 80 years was surveyed using the ASRS-5, with complete data obtained from 714 participants. Psychometric analysis evaluated fit to the Rasch model, response categories, dimensionality, differential item functioning, local dependency, and reliability. A cut-off score of 14, based on a simple additive scale, was applied. The ASRS-5 generally conformed to the expectations of the Rasch model. However, disordered response categories were observed for item 6, and gender-related differential item functioning was noted in items 3, 4, and 6. The ASRS-5 demonstrated a unidimensional construct, and a cut-off score of 14 identified 6.0% of the sample as potential ADHD cases. Although the ASRS-5 aligns well with the Rasch model, certain measurement challenges exist. The recommended cut-off score effectively identifies an ADHD prevalence consistent with the 6.5% reported in the original ASRS-5 study.

Purification procedures used for the detection of gender DIF: Item bias in a foreign language test

Differential item functioning (DIF) detection was handled based on “Mantel-Haenszel (MH)”, “Simultaneous item bias test (SIBTEST)”, “Lord's chi-square”, “Raju's area” methods when item purification was performed or item purification was not performed using real data in current study. After detecting gender-related DIF, expert opinions were taken for bias study. It is important to conduct the gender bias research in the English test when purification is performed and when purification is not performed, as there were DIF studies, but there were not completely similar bias studies in the literature. The sample of the research consists of 7389 students who took the “Transition from Primary to Secondary Education Exam (TPSEE, referred to as “TEOG” in Turkey)” administered in April 2017. When gender-related DIF analysis was performed with the four methods, the results were found to differ partially. DIF analysis results differed in the different conditions item purification was performed or not. Detection of DIF was indicative of possible bias. In the second stage of the study, the opinions of seven experts were taken for item 11, for which DIF was detected at least at B level based on MH, SIBTEST. As a result of expert opinion, it was found that there was no item bias according to gender in any item in the English test. It is recommended that similar bias studies can be conducted for test developers to be aware of the features that may lead to item bias and to construct unbiased items.

Using Rasch Analysis to Assess and Improve the Measurement Properties of a Questionnaire With Few Items: The York Binaural Hearing-Related Quality of Life (YBHRQL) Questionnaire.

The York Binaural Hearing-Related Quality of Life questionnaire is a condition-specific preference-based instrument sensitive to advantages of binaural over monaural hearing. Respondents use 5-point scales to report the difficulty experienced with three dimensions of listening that are easier or more successful when hearing is binaural: understanding speech in spatially separated noise, localizing sources of sound in azimuth, and the associated effort and fatigue. Previously, a preference value was estimated for each combination of dimension and level so that a value of binaural utility could be assigned to a respondent to inform analyses of cost effectiveness. The present objective was to determine whether the questionnaire conforms with the Rasch model sufficiently well for estimates of the binaural abilities of respondents to be obtained on an interval scale to inform parametric analyses of clinical effectiveness. Data were obtained from unilateral cochlear implantees (N = 418; 209 ≤62 years; 209 ≥63 years) and members of the public (N = 325; 207 ≤62 years; 118 ≥63 years). A subset of implantees (N = 118) responded at test and retest. Responses were fitted to the partial credit model using the Extended Rasch Modeling package. Conformity with the model was evaluated in six ways: the ordering of response categories ( Monotonicity ) was assessed with plots of response probability against ability; differential item functioning ( DIF ) was assessed by analyses of variance of standardized response residuals; alignment of participants' abilities with item difficulties ( Targeting ) was assessed with person-item maps; fit to the model ( Fit ) was assessed by comparing the means and variabilities of observed and expected responses, and by comparing observed values with analyses of simulated datasets; the hypothesis that item difficulties and participants' abilities were measured on a single underlying scale ( Unidimensionality ) was assessed with principal components analyses of standardized response residuals. Values of fit statistics were toward the lower end of the acceptable range. Comparisons with analyses of simulated datasets showed that low values were primarily the result of the structural limitation of including only three items. Modal values of the probabilities of response categories were ordered monotonically, but some response thresholds were disordered because of under-use of one category. Pooling categories to correct disordered thresholds resulted in estimates of ability that were less discriminatory of differences within and between groups, and showed less reproducibility between test and retest, than did the original estimates. Neither source-related DIF nor gender-related DIF arose. Uniform age-related DIF arose for the speech-in-noise item and could be managed by resolving the item. The resulting estimates of ability and difficulty were well targeted and unidimensional. The York Binaural Hearing-Related Quality of Life questionnaire, with three items each with five response categories, conforms with the Rasch model sufficiently well to yield practically useful measures of the abilities of participants. The trait measured by the questionnaire aligns with the ability to benefit from binaural hearing. More discriminatory measurement of this ability would be achieved with more items. Nonetheless, the questionnaire possesses the virtue that responses to the same three questions can be scored in different ways to inform parametric analyses of both cost-effectiveness and clinical effectiveness.

A Psychometric Study of the Prosocial Behavior Scale: Differential Item Functioning by Gender.

Some scales co-exist in the literature to measure prosocial behavior in adolescents. Gender differences in prosocial behavior have been a controversial topic of research. To strengthen future research in the area, the psychometric properties of the most used instruments must be guaranteed, especially its gender non-bias. Our study provides psychometric evidence for the Prosocial Behavior Scale in a sample of adolescents, exploring: (a) its factor structure; (b) reliability; (c) gender-related differential item functioning (DIF); (d) nomological validity. A sample of 512 high school students (mean age = 13.62 (SD = 1.34), 51.6% females) participated in the research. Confirmatory Factor Analysis (CFA) was used to test the factor structure of the scale, which adequately fitted the data (χ2 (35) = 152.224, p < 0.001, Comparative Fit Index (CFI) = 0.905, Root Mean Squared Error of Approximation (RMSEA) = 0.085 90%CI [0.072-0.099], Standardized Root Mean Squared Residual (SRMR) = 0.079). Reliability results were good (α = 0.74, ω = 0.74). Regarding the DIF, five items presented some gender-related bias, generally benefiting females. However, the DIF impact could be considered negligible. Correlations with the subdimensions of the psychological capital offered evidence of the nomological validity of the scale. In conclusion, the scale presented adequate psychometric properties that support its ability to effectively assess prosocial behavior and gender differences in the prosocial behavior samples of adolescents. Additionally, the results obtained imply that gender differences in the manifestations of prosocial behavior require measurements that can fairly sample behaviors characteristic of each gender.

Measuring palliative care self-efficacy of intellectual and developmental disability staff using Rasch models.

The objectives of this study were to evaluate the psychometric properties of a palliative care self-efficacy instrument developed for intellectual and developmental disability (IDD) staff using Rasch analysis and assess the change in palliative care self-efficacy between 2 time points using Rasch analysis of stacked data. Staff from 4 nonprofit IDD services organizations in a US Midwestern state (n=98) answered 11 questions with Likert-style responses at baseline and 1-month follow-up post training. Rasch analysis was performed to examine rating scale structure, unidimensionality, local independence, overall model fit, person and item reliability and separation, targeting, individual item and personal fit, differential item functioning (DIF), and change in palliative care self-efficacy between 2 time points. The rating scale structure improved when 5 response categories were collapsed to 3. With the revised 3 response categories, the instrument demonstrated good psychometric properties. Principal components analysis of Rasch residuals supported the assumption of unidimensionality. Model fit statistics indicated an excellent fit of the data to the Rasch model. The instrument demonstrated good person and item reliability and separation. Gender-related DIF was found in 1 item, and work tenure-related DIF in 3 items. Overall palliative care self-efficacy improved between 2 time points. Rasch analysis allowed for a more thorough examination of this palliative care self-efficacy instrument than classical test theory and provided information on rating scale structure, targeting, DIF, and individual persons and items. These recommendations can improve this instrument for research and practical contexts.

Comparison of unweighted and item response theory-based weighted sum scoring for the Nine-Questions Depression-Rating Scale in the Northern Thai Dialect

BackgroundThe Nine-Questions Depression-Rating Scale (9Q) has been developed as an alternative assessment tool for assessing the severity of depressive symptoms in Thai adults. The traditional unweighted sum scoring approach does not account for differences in the loadings of the items on the actual severity. Therefore, we developed an Item Response Theory (IRT)-based weighted sum scoring approach to provide a scoring method that is more precise than the unweighted sum score.MethodsSecondary data from a study on the criterion-related validity of the 9Q in the northern Thai dialect was used in this study. All participants were interviewed to obtain demographic data and screened/evaluated for major depressive disorder and the severity of the associated depressive symptoms, followed by diagnosis by a psychiatrist for major depressive disorder. IRT models were used to estimate the discrimination and threshold parameters. Differential item functioning (DIF) of responses to each item between males and females was compared using likelihood-ratio tests. The IRT-based weighed sum scores of the individual items are defined as the linear combination of individual response weighted with the discrimination and threshold parameters divided by the plausible maximum score based on the graded-response model (GRM) for the 9Q score (9Q-GRM) or the nominal-response model (NRM) for categorical combinations of the intensity and frequency of symptoms from the 9Q responses (9QSF-NRM). The performances of the two scoring procedures were compared using relative precision.ResultsOf the 1,355 participants, 1,000 and 355 participants were randomly selected for the developmental and validation group for the IRT-based weighted scoring, respectively. the gender-related DIF were presented for items 2 and 5 for the 9Q-GRM, while most items (except for items 3 and 6) for the 9QSF-NRM, which could be used to separately estimate the parameters between genders. The 9Q-GRM model accounting for DIF had a higher precision (16.7%) than the unweighted sum-score approach.DiscussionOur findings suggest that weighted sum scoring with the IRT parameters can improve the scoring when using 9Q to measure the severity of the depressive symptoms in Thai adults. Accounting for DIF between the genders resulted in higher precision for IRT-based weighted scoring.

Gender-based Differential Item Functioning Analysis of the Medical Specialization Education Entrance Examination

The Medical Specialization Education Entrance Examination is a national high-stake test for the placement of medical graduates in medical specialization training in Turkey. The purpose of this study is to determine whether the Medical Specialization Education Entrance Examination items display gender-related differential item functioning (DIF) by using Mantel-Haenszel and logistic regression methods. To determine the presence of item bias, content experts reviewed items. The analyzes were conducted on the answers of 11,530 physicians to the Basic Medical Sciences and Clinical Medical Sciences tests of the 2017 Medical Specialization Education Entrance Examination spring term. According to the Mantel-Haenszel method, there were eleven out of 234 items identified as showing B level gender-related DIF. While six of the items functioned in favor of male physicians, five of them were in favor of female physicians. Since the number of items in favor of each gender is close, DIF cancellation occurs. According to content areas, one histology and embryology, one internal medicine, and three gynecology and obstetrics items were in favor of female physicians, one physiology, two medical pharmacology, one pediatrics, and two surgical items were in favor of male physicians. To the experts’ reviews, there are no biased items. The medical specialty preferences of the physicians and content area of the displaying differential item functioning items overlapped.

Examination of Gender-Related Differential Item Functioning Through Poly-BW Indices.

The existing differential item functioning (DIF) detection approaches relying on item difficulty or item discrimination are limited for understanding the associates of DIF items, and consequently, DIF items were conventionally either deleted or ignored. Given the importance of minimizing DIF items in test construction, teachers or testing practitioners need more information regarding possible associates of DIF items. Using an example of a teacher-made mathematics achievement test, this study aimed to examine how the Poly-BW indices (power, defenselessness, disturbance, and hint) contributed to the properties of gender-related DIF items. Data from a 34-item mathematics achievement test that involved 1,439 seventh-grade students from Taiwan (51.01% boys and 48.99% girls) showed that the differences of the defenselessness (mp) and power (cp) indices between men and women served as salient predictors of the DIF measures estimated by the Poly Simultaneous Item Bias Test (Poly-SIBTEST) procedure and with satisfactory accuracy of hit rates. Items with relatively large defenselessness for men were likely to present male-favoring DIFs, whereas items with relatively large power for men were likely to present female-favoring DIFs. The Poly-BW indices yielded directions for modifying items for teachers in practice.

Validation of the inflexible eating questionnaire in a large sample of Chinese adolescents: psychometric properties and gender-related differential item functioning.

The aim of this study was to provide preliminary psychometric evidence for the Chinese version of the Inflexible Eating Questionnaire (C-IEQ) among a large sample of Chinese adolescents. For testing the psychometric properties of the C-IEQ, a total of 2241 (Mage = 13.91years; 46.4% boys) adolescents from mainland China responded to the survey. Confirmatory factor analysis (CFA) was used to examine the factor structure of the C-IEQ. Measurement invariance by gender was examined by both multi-group CFA and differential item functioning (DIF). Convergentvalidity of the C-IEQ was assessed via examining the correlations between the C-IEQ scores and theoretically related constructs (e.g., orthorexia nervosa symptomatology, eating disorder symptomatology, and body image inflexibility). The unidimensional structure with correlated errors of the C-IEQ showed good model fit (χ2 = 1674.05, df = 44, p < 0.001; CFI = 0.93; TLI = 0.91; RMSEA = 0.13; SRMR = 0.06). The C-IEQ had an adequate internal consistency (α = 0.89) and demonstrated strong measurement invariance across genders. Moreover, the scores of the C-IEQ showed significant correlations with theoretically correlated constructs: orthorexia nervosa symptomatology (girls; r = 0.53 p < 0.001, boys; r = 0.45 p < 0.001), eating disorder symptomatology (girls; r = 0.32 p < 0.001, boys; r = 0.25 p < 0.001) body image inflexibility (girls; r = 0.41 p < 0.001, boys; r = 0.36 p < 0.001), suggesting good convergentvalidity of the C-IEQ. The C-IEQ showed good psychometric properties in a sample of Chinese adolescents and can be used in future studies to assess eating-specific psychological inflexibilityamong Chinese adolescents. Level V, cross-sectional descriptive study.

A Short Form of the Chinese Version of the Weinstein Noise Sensitivity Scale through Optimal Test Assembly.

This study developed a short form of the traditional Chinese version of the Weinstein Noise Sensitivity Scale (WNSS) through optimal test assembly (OTA). A total of 1069 Chinese adults (64.8% female) completed the territory-wide cross-sectional study. We first removed Items 12 and 5 which had negative factor loading and gender-related differential item functioning (DIF), respectively. The optimal length was then determined as the minimal one that reasonably resembled the reliability and validity of the scale without DIF items. OTA identified an 8-item WNSS (WNSS-8) which retained 67.2% of the test information of the original 21-item scale and had a Cronbach’s alpha of 0.83. It also showed significant correlations of 0.272 and −0.115 with the neuroticism and extraversion scales of Chinese NEO-Five Factor Inventory, respectively. Adequate model fit of the WNSS-8 was demonstrated by the confirmatory factor analysis. The Chinese WNSS-8 can be used to assess noise sensitivity without compromising reliability and validity.

Gender-Related Differential Item Functioning of 2015 Wassce Core Mathematics Results in Southern Ghana Using Logistic Regression Procedure

Differential item functioning (DIF) occurs when individuals of the same ability level from separate groups have a different probability of answering an item correctly. This study was conducted in two parts: in the first part a real 2015 West African Senior Secondary Certificate Examination (WASSCE) core mathematics test data were analyzed for uniform and nonuniform DIF using binary logistic regression (LR) procedure and in the second part, content analysis of items identified as DIF were classified under the levels of the cognitive domain by experts. Three research questions were formulated for the study. A sample of 4,285 male and 3,712 female candidates were selected from a population of 15,258 candidates who sat for the examination in 2015 from 20 selected schools in Southern Ghana. The instrument for the study was the 50 multiple-choice core mathematics items. The findings showed that there was 43 significant gender differential item functioning items of which 9 were uniform and 34 non-uniform. Also, the content analysis revealed that items that favoured males were mainly number and numeration, algebraic processes, probability and statistics and mensuration whiles plane geometry and coordinate geometry revealed DIF in favour of females. It was concluded that test items used were not free from gender DIF. It was recommended that DIF studies should be conducted by test developers in order to be review or exclude DIF items to enhance fairness in assessment.

Gender-Related Differential Item Functioning of Mathematics Computation Items among Non-native Speakers of English

Gender-Related Differential Item Functioning of Mathematics Computation Items among Non-native Speakers of English

Assessment of Gender-related Differential Item Functioning of Teacher-Made Chemistry test

Assessment of Gender-related Differential Item Functioning of Teacher-Made Chemistry test

Assessing language development in Arabic: The Arabic language: Evaluation of function (ALEF)

Arabic is characterized by extensive dialectal variation, diglossia, and substantial morphological complexity. Arabic lacks comprehensive diagnostic tools that would allow for a systematic evaluation of its development, critical for the early identification of language difficulties in the spoken and written domains. To address this gap, we have developed an assessment battery called Arabic Language: Evaluation of Function (ALEF), aimed at children aged 3 to 11 years. ALEF consists of 17 subtests indexing different language domains, modalities, and associated skills and representational systems. We administered the ALEF battery to native Gulf Arabic-speaking children (n = 467; ages 2.5 to 10.92; 55% boys; 20 children in each 6-month age band) in Saudi Arabia in two data collection waves. Analyses examining the psychometric properties of the instrument indicated that after the removal of misfitting items, the ALEF subtests had reliability coefficients in the range from 0.78 to 0.98, and resulting subtest scores displayed a consistent profile of positive intercorrelations and age effects. Taken together, the results indicate that the ALEF battery has good psychometric properties, and can be used for the purpose of evaluating early language development in Gulf Arabic speaking children, pending further refinement of the test structure, examination of gender-related differential item functioning, and norming.

Detection of Gender-Biased Items in the Peabody Picture Vocabulary Test

This study investigated possible gender bias on a vocabulary test, using a method suggested by Andrich and Hagquist to detect “real” differential item functioning (DIF). A total of 443 adult ESL learners completed all 228 items of the Peabody Picture Vocabulary Test (PPVT-IV). The 310 female and 133 male participants were assumed to be of equal competence, corresponding to levels B1 and B2 on the Common European Framework of Reference for Languages. Male participants outscored female participants, possibly due to the multiple-choice format and to the fact that most gender-biased questions favored men rather than women. Finally, our analysis process yielded only seven items out of 228 as showing gender DIF, which is much lower than the numbers reported in the literature for ESL tests. This low figure suggests that the high number of gender-related DIF items reported in previous research might be attributed to the use of DIF detecting methods that do not take into account artificial DIF stemming from the cross-contamination of test items.

Reasons for Gender-Related Differential Item Functioning in a College Admissions Test

ABSTRACTGender fairness in testing can be impeded by the presence of differential item functioning (DIF), which potentially causes test bias. In this study, the presence and causes of gender-related DIF were investigated with real data from 800 items answered by 250,000 test takers. DIF was examined using the Mantel–Haenszel and logistic regression procedures. Little DIF was found in the quantitative items and a moderate amount was found in the verbal items. Vocabulary items favored women if sampled from traditionally female domains but generally not vice versa if sampled from male domains. The sentence completion item format in the English reading comprehension subtest favored men regardless of content. The findings, if supported in a cross-validation study, can potentially lead to changes in how vocabulary items are sampled and in the use of the sentence completion format in English reading comprehension, thereby increasing gender fairness in the examined test.

Development of a short scale for assessing economic environmental aspects in patients with spinal diseases using Rasch analysis

BackgroundEconomic environmental factors represent important barriers to participation and have deleterious effects on quality of life (QOL) in persons with spinal diseases (SpD). While economic factors are anchored in the International Classification of Functioning, Disability and Health, their influence on QOL and participation from patients’ perspectives is an infrequent focus of research. The aim of the present research is to calibrate a culturally adapted Rasch-based questionnaire assessing economic QOL in patients with SpD.MethodsThe 11-items of the German economic-QOL-scale were answered by 325 patients with SpD on a four-point Likert-scale. Fit to the Rasch measurement model was investigated by testing for stochastic ordering of the items, unidimensionality, local independence, and differential item functioning (DIF).ResultsAfter adjusting for local dependency, fit to the Rasch model was achieved with a non-significant item-trait interaction (chi-squaredf = 20 = 34.8, p = 0.021). The person separation reliability equaled 0.88, the scale was free from age- or gender-related DIF, and unidimensionality could be verified.ConclusionsThe Rasch-based German version of the economic-QOL-scale represents a suitable instrument to investigate the influences of economic factors on patients’ QOL at a group and individual level. It can be easily applied in research and practice and may be administered quickly in combination with other instruments. The short test duration implies a low test burden for patients and a minimum of time expenditure by clinicians when evaluating the results.

Examining gender-related differential item functioning of the Veterans Rand 12-item Health Survey.

Previous research suggests that gender differences in patient-reported outcome measures (PROMs) may reflect measurement bias rather than true differences in underlying health status. The aim of this study is to examine whether the Veterans Rand 12-item Health Survey (VR-12) allows for unbiased comparisons of physical and mental health scores across gender. The VR-12 is a generic PROM consisting of 12 items with 3-6 response options for the measurement of mental and physical health. Study data were from the 2015 Health Outcomes Survey pertaining to the Medicare beneficiaries. A total of 277,518 participants included 116,817 (42.1%) males and 160,701 (57.9%) females. Scale-level and item-level differential functioning methods were applied using multiple-group confirmatory factor analysis and ordinal logistic regression, respectively. The scale-level differential functioning showed support for strict invariance (RMSEA=0.045; CFI=0.995) across gender. Although we found statistically significant differential item functioning for several items, the magnitude was negligible (maximum ΔR 2=0.007). The VR-12 physical and mental health status scores are unbiased with respect to gender.

The Effect of Background Variables on Gender DIF

The purpose of this study is to investigate the presence of differential item functioning (DIF) between gender groups in PISA 2015 science items in nine selected countries. Moreover, the effect of socioeconomic status, reading ability, science self-efficacy and science motivation on the presence of gender-related DIF are examined, respectively. One cluster from computer-based assessment (CBA) is taken into consideration. The countries are selected among the ones that implemented CBA, on the basis of their rank in science achievement. Multiple Indicator Multiple Causes method (MIMIC) is used in our analysis. DIF analysis in the MIMIC involves fit comparisons of both full and reduced models to determine if the items can measure the latent trait equally among the specified groups. The MIMIC analysis is conducted in two steps. First, the items is being tested for showing DIF among gender groups. Then the socioeconomic status, the reading ability, the science self-efficacy and the motivation are added to the model to test gender-related DIF items and their effects, respectively. According to results of the study, gender-related DIF appeared in all of the selected countries vary between two and six items. In four of the countries none of the selected variables significantly affect the presence of gender-related DIF. Instead, in the remaining countries the number of gender-related DIF items is reduced by adding different combinations of the selected variables to the model. The effect of variables which reduce the number of gender-related DIF items will discussed within each country.

The psychometric properties of the “Reading the Mind in the Eyes” Test: an item response theory (IRT) analysis

ABSTRACTIntroduction: The “Reading the Mind in the Eyes” Test (hereafter: Eyes Test) is considered an advanced task of the Theory of Mind aimed at assessing the performance of the participant in perspective-takingthat is, the ability to sense or understand other people’s cognitive and emotional states. In this study, the item response theory analysis was applied to the adult version of the Eyes Test.Methods: The Italian version of the Eyes Test was administered to 200 undergraduate students of both genders (males = 46%). Modified parallel analysis (MPA) was used to test unidimensionality. Marginal maximum likelihood estimation was used to fit the 1-, 2-, and 3-parameter logistic (PL) model to the data. Differential Item Functioning (DIF) due to gender was explored with five independent methods.Results: MPA provided evidence in favour of unidimensionality. The Rasch model (1-PL) was superior to the other two models in explaining participants’ responses to the Eyes Test. There was no robust evidence of gender-related DIF in the Eyes Test, although some differences may exist for some items as a reflection of real differences by group.Conclusions: The study results support a one-factor model of the Eyes Test. Performance on the Eyes Test is defined by the participant’s ability in perspective-taking. Researchers should cease using arbitrarily selected subscores in assessing the performance of participants to the Eyes Test. Lack of gender-related DIF favours the use of the Eyes Test in the investigation of gender differences concerning empathy and social cognition.