Gender Differential Item Functioning Research Articles

Psychometric evaluation of the pictorial scale of perceived movement skill competence in Chinese children: An item response theory analysis.

This study aimed to investigate the psychometric characteristics of the Pictorial Scale of Perceived Movement Skill Competence (PMSC) in Chinese children, utilizing the principles of item response theory (IRT). A total of 284 children (147 boys), aged 4-10years (Mage = 7.39, SD = 2.18), from four schools were recruited for testing, adhering to the principle of random sampling. The Rasch model was applied to evaluate the rating scale validity, local independence, scale dimensionality, person and item fit statistics, and differential item functioning (DIF). (1) The rating scale validity was confirmed by the monotonically increasing trend in both average item difficulty measures and Andrich thresholds. (2) However, the observation that standardized residual correlations for all items remained below 0.3 unequivocally confirmed the presence of local independence among the variables. (3) The scale's dimensions were substantiated, with Rasch measures explaining 40.6% of total variance, exceeding the 40% threshold. Additionally, the first contrast had an eigenvalue of 1.8 and explained 8.80% of the variance, both exceeding the minimum 3:1 ratio. Consequently, the unidimensionality assumption of the item was supported. (4) Person and item fit statistics indicated requisite reliability and separation, with all achieving suitable infit and outfit measures, and point-biserial correlations exceeding 0.55. (5) Regarding gender DIF, it was observed that the majority of items did not exhibit statistically significant gender DIF effects (p > 0.05) except item 8 (Bounce a ball, t = -2.74, p < 0.01, DIF = -0.51). Rasch analysis revealed that the instrument exhibited strong scale validity among Chinese children, adhering to the Rasch model's unidimensionality and local independence principles, with robust person and item reliability and separation. All items demonstrated a good fit. Additionally, the significant gender difference in item 8 (Bounce a ball) warrants further consideration. In conclusion, PMSC is suitable for assessing the PMSC of Chinese children.

Purification procedures used for the detection of gender DIF: Item bias in a foreign language test

Differential item functioning (DIF) detection was handled based on “Mantel-Haenszel (MH)”, “Simultaneous item bias test (SIBTEST)”, “Lord's chi-square”, “Raju's area” methods when item purification was performed or item purification was not performed using real data in current study. After detecting gender-related DIF, expert opinions were taken for bias study. It is important to conduct the gender bias research in the English test when purification is performed and when purification is not performed, as there were DIF studies, but there were not completely similar bias studies in the literature. The sample of the research consists of 7389 students who took the “Transition from Primary to Secondary Education Exam (TPSEE, referred to as “TEOG” in Turkey)” administered in April 2017. When gender-related DIF analysis was performed with the four methods, the results were found to differ partially. DIF analysis results differed in the different conditions item purification was performed or not. Detection of DIF was indicative of possible bias. In the second stage of the study, the opinions of seven experts were taken for item 11, for which DIF was detected at least at B level based on MH, SIBTEST. As a result of expert opinion, it was found that there was no item bias according to gender in any item in the English test. It is recommended that similar bias studies can be conducted for test developers to be aware of the features that may lead to item bias and to construct unbiased items.

Rasch Rating Scale Modelling of the Arabic Version of the Critical Thinking Disposition Scale

Abstract: The tendency to think critically is the motivation of an individual for using critical thinking when faced with a problem that requires a solution, making a decision or evaluating an idea. This study used the Rasch Rating Scale Model (RSM) analysis to examine a set of psychometric properties of an Arabic version of the Critical Thinking Disposition Scale (EMI): items fit, unidimensionality, local independence, equal-item-discriminations, gender differential item functioning, reliability and separation indicators and scale calibra-tion. The findings indicated that EMI showed good compatibility with the RSM as all the items matched the model except for item 11. In addition, the assumptions of the Rasch model which were unidimensionality, local independence, and equal-item-discriminations were realized. The scale had excellent reliability for persons and good reliability for items. The scale showed good separation indicators for items, and excellent separation indicators for persons. The items did not show differential gender performance. The distances be-tween the response categories were appropriate, and the category measurements showed a consistent in-crease.

Rasch Calibration and Differential Item Functioning (DIF) Analysis of the Indonesian National Assessment Program-Language (INAP-L)

This study aimed to evaluate the psychometric properties of the INAP-L instrument by applying the Rasch model. The psychometric evaluation includes item calibration and differential item functioning (DIF) assessment. Participants in this study were 6153 high school students (2326 boys and 3827 girls) aged 14-19 years (mean age = 15.76, SD = 0.78) from 280 schools spread across 34 provinces in Indonesia. The results of the Rasch model analysis show that the assumptions of unidimensionality and local independence are met. The internal consistency analysis of the INAP-L instrument (PSR = 0.80; Ordinal α = 0.81) and item-person targeting showed quite good results. At the item level, it was discovered that four of the forty items did not fit the Rasch model. Meanwhile, the gender-based DIF analysis revealed that all items were free from gender DIF and that two out of forty items showed DIF based on school type. It can be concluded that the INAP-L instrument has good psychometric properties. Furthermore, multilevel analysis was performed to determine the effect of clustering in the data, and it was discovered that INAP-L has a multilevel data structure with ICC = 0.185 and a design effect &gt; 2.00. This means that future research on the relationship among variables on the INAP-L score must consider a multilevel approach. Even though the developer has already published an interim report on INAP-L, the findings of this study can be used as a reference in improving the instrument before conducting INAP-L in the future.

Gender differential item functioning analysis in measuring computational thinking disposition among secondary school students.

Computational thinking refers to the cognitive processes underpinning the application of computer science concepts and methodologies to the methodical approach and creation of a solution to a problem. The study aims to determine how students' cognitive, affective, and conative dispositions in using computational thinking are influenced by a gender. This study used a survey research design with quantitative approach. Five hundred thirty-five secondary school students were sampled using probability sampling with the Computational Thinking Disposition Instrument (CTDI). WINSTEPS version 3.71.0 software was subsequently employed to assess the Gender Differential item functioning (GDIF) including reliability and validity with descriptive statistics were employed to assess students' disposition toward practicing computational thinking. In addition to providing implications for the theory, the data give verifiable research that the CT disposition profile consists of three constructs. In addition, the demonstrated CTDI has good GDIF features, which may be employed to evaluate the efficacy of the application of CT in the Malaysian curriculum by measuring the level of CT in terms of the disposition profile of students.

Measurement properties of the Falls Efficacy Scale-International (FES-I) in persons with late effects of polio: A cross-sectional study.

Fear of falling (FoF) is very common in persons with late effects of polio (LEoP). An internationally recognized rating scale to assess FoF is the Falls Efficacy Scale-International (FES-I). Yet, there is limited knowledge about its measurement properties in persons with LEoP. To investigate the measurement properties of the FES-I (16-item version) and the short FES-I (7-item version) in persons with LEoP. Explorative factor analysis and Rasch model analysis of cross-sectional data. University hospital. A total of 321 persons with LEoP (mean age ± standard deviation [SD] 70 ± 10 years, 173 women). The FES-I and the short FES-I, comprising four response options about concerns of falling ranging from 1 (not at all concerned) to 4 (very concerned). Data were collected by a postal survey. First, a factor analysis was performed to investigate the unidimensionality of the scale. Thereafter, a Rasch model analysis was used to further analyze the measurement properties of the FES-I and the short FES-I, such as local dependency, targeting, hierarchical order of items, Differential Item Functioning (DIF), response category functioning and reliability (Person Separation Index, PSI). Raw score transformation to interval measurements was also performed. The factor analysis revealed that the FES-I was unidimensional, even though the Rasch analysis showed some misfit to the Rasch model and local dependency. Targeting for the FES-I and the short FES-I was somewhat suboptimal as the participants on average reported less FoF than expected. A negligible gender DIF was found for two items in the FES-I and for one item in the short FES-I. Reliability was high (PSI >0.86), and the response category thresholds worked as intended for both the FES-I and the short FES-I. The FES-I and the short FES-I have sufficient measurement properties in persons with LEoP. Both versions can be used to assess fear of falling (or FoF) in this population.

Examining and Improving the Gender and Language DIF in the VERA 8 Tests

The purpose of this study was to examine and improve differential item functioning (DIF) across gender and language groups in the VERA 8 tests. We used multigroup concurrent calibration with full and partial invariance based on the Rasch and two-parameter logistic (2PL) models, and classified students into proficiency levels based on their test scores and previously defined cut scores. The results indicated that some items showed gender- and language-specific DIF when using the Rasch model, but we did not detect large misfit items (suspected as DIF) when using the 2PL model. When the item parameters were estimated using the 2PL model with partial invariance assumption (PI-2PL), only small or negligible misfit items were found in the overall tests for both groups. It is argued in this study that the 2PL model should be preferred because both of its approaches provided less bias. However, especially in the presence of unweighted sample sizes of German and non-German students, the non-German students had the highest misfit item proportions. Although the items with medium or small misfit did not have a significant effect on the scores and performance classifications, the items with large misfit changed the proportions of students at the highest and lowest performance levels.

Psychometric properties of the EURO-D scale of depressive symptomatology: Evidence from SHARE wave 8

BackgroundThe EURO-D is a short scale to measure symptoms of depression, very used in large population surveys. Although there are numerous validation studies, its psychometric properties remain unclear. The two-factor structure (Affective Suffering and Lack of Motivation) is replicated in several studies but with different item compositions, and none reported reliability indices for both factors. For that reason, the aim of this study is to examine the factorial validity of the scale, the reliability of the dimensions, the gender differential item functioning (DIF), and the nomological validity. Methods46,317 participants aged 50 and over (M = 71.33), from which 57.4 % were females, in Wave 8 of the Survey of Health, Aging and Retirement in Europe (SHARE) were included. Instruments: EURO-D, R-UCLA, Self-perceived health index, and indicator of taking drugs for anxiety or depression. Factor Analyses, DIF, Reliability Index and Spearman correlations were estimated. ResultsFactor analysis identified a bifactor structure: a general factor of Depression and two specific factors of Affective Suffering and Lack of Motivation, which reliabilities were 0.83, 0.83 and 0.79 respectively. No relevant DIF item by gender was found, but higher scores were found in women in all factors. Both factors had positive relations with loneliness, taking drugs and self-perceived health. Limitationsthis study has a cross-sectional design; future research may consider the longitudinal stability of the scale. ConclusionsEURO-D shows adequate psychometric properties when a general factor of depression and two specific factors are considered. Women have higher scores on all dimensions.

SPANISH VERSION OF THE ATTITUDE TOWARDS COVID-19 VACCINES SCALE: RELIABILITY AND VALIDITY ASSESSMENT.

Background and Purpose: The negative attitude towards vaccines for coronavirus disease (COVID-19) has motivated the adaptation of instruments for this specific purpose. However, details of the reliability and validity of these scales are unknown. The study aimed to evaluate some indicators of the reliability and validity of the Spanish version of the Attitude towards COVID-19 Vaccines Scale. Methods: A validation study was carried out with 1,136 students of emerging age (18 and 29 years) from a Colombian university; 65.5% were female. Cronbach's alpha and McDonald's omega were calculated for reliability, and exploratory and confirmatory factor analyzes for validity. Additionally, the gender differential item functioning (DIF) was estimated with Kendall's tau b. Results: The Spanish version of Attitude towards COVID-19 Vaccines Scale showed high internal consistency (Cronbach's alpha of .94 and McDonald's omega of .95), a one-dimensional structure with acceptable goodness-of-fit indicators (CFI = .94, TLI = .91, and SRMR = .04), and non-gender DIF (Kendall's tau b between .02 and .06). Conclusions: The Spanish version of the Attitude towards COVID-19 Vaccines Scale presents some appropriate reliability and validity indicators among university emerging adults. These findings should be explored in samples with other characteristics.

You Can't Compare If You Don't Prepare: Differential Item Functioning In Measures of Grit, STEM Self-Regulation, and Motivation

Researchers developing questionnaire measures of personality, motivation, and self-regulation constructs related to students’ achievement and persistence in STEM or other fields rarely have examined whether the items on the measures used are functioning differently across groups, which is necessary for accurate group comparison. The present study assessed differential item functioning (DIF) in two frequently-used measures of such constructs, the Grit-S, the Motivated Strategies for Learning Questionnaire (MSLQ), and a new measure of long-term grit, using a sample of 1,146 ninth and tenth-grade students in the U.S.A. We also assessed gender and racial differences in the patterns of the relationships among the latent constructs, something rarely reported in the literature. Differences in such patterns indicate that the concurrent validity of the construct scores could vary across different groups. Results indicated that numerous items exhibited gender DIF on the Grit-S but not on the MSLQ or long-term grit measure. There were fewer DIF items across racial groups. Patterns of relations among the variables, in general, were weaker for females than males. There were fewer racial groups differences in the patterns of correlations. We discuss the implications of the DIF findings for future research on group comparisons.

Detecting Differential Item Functioning Using Cognitive Diagnosis Models: Applications of the Wald Test and Likelihood Ratio Test in a University Entrance Examination

ABSTRACT The present study aims to examine gender differential item functioning (DIF) in the reading comprehension section of a high stakes test using cognitive diagnosis models. Based on the multiple-group generalized deterministic, noisy “and” gate (MG G-DINA) model, the Wald test and likelihood ratio test are used to detect DIF. The flagged items are further inspected to find the attributes they measure, and the probabilities of correct response are checked across latent profiles to gain insights into the potential reasons for the occurrence of DIF. In addition, attribute and latent class prevalence are examined across males and females. The three items displaying large DIF involve three attributes, namely Vocabulary, Main Idea, and Details. The results indicate that females have lower probabilities of correct response across all latent profiles, and fewer females have mastered all the attributes. Moreover, the findings show that the same attribute mastery profiles are prevalent across genders. Finally, the results of the DIF analysis are used to select models that could replace the complex MG G-DINA without significant loss of information.

Detecting Differential Item Functioning Using SIBTEST, MH, LR and IRT Methods

In this study, differential item functioning (DIF) and differential bundle functioning (DBF) analyses of the Academic Staff and Postgraduate Education Entrance Examination Quantitative Ability Tests were carried out. Mantel-Haenszel, logistic regression, SIBTEST, Item Response Theory-Likelihood Ratio and BILOG-MG DIF Algorithm methods were used for DIF analyses. SIBTEST was the method used for DBF analyses. Data sets for the study came from an earlier application of the examination. Gender DIF analyses showed that eleven items showed DIF. Four of the items favored male applicants, where seven of them favored female applicants. In order to investigate the sources of DIF, we consulted experts. In general, the items which could be solved using routine algorithmic operations and which are presented in the algebraic, abstract format showed DIF in favor of females. The “real-life” word problems favored males. According to DBF analyses, the operations item group favored females and the word problems item group favored males.

Brief Spanish Zung Self-Rating Anxiety Scale: Dimensionality, Internal Consistency, Nomological Validity, and Differential Item Functioning Among Chronic Obstructive Pulmonary Disease Patients in Colombia.

Background and purposes: The brief Spanish Zung Self-Rating Anxiety Scale is a ten-item tool available; however, there is little information about its psychometric performance. The study aimed to test the dimensionality, internal consistency, nomological validity, and gender differential item functioning of the scale in patients with chronic obstructive pulmonary disease (COPD) in Colombia. Methods: A psychometric study was designed. It participated in 408 COPD patients aged between 40 and 102 years. Results: The confirmatory factor analysis showed both one- and two-dimension structures with acceptable goodness-of-fit indicators. The brief Spanish Zung Self-Rating Anxiety Scale presented high internal consistency, acceptable nomological validity, and non-gender differential functioning. Conclusions: The brief Spanish Zung Self-Rating Anxiety Scale presents some good indicators of validity and reliability, and it can be used in the measurement of anxiety in COPD patients.

Determination of Gender Differential Item Functioning in Tegal Students' Scientific Literacy Skills with Integrated Science (SLiSIS) Test Using Rasch Model

The emergence of Differential Item Functioning (DIF) indicates an external bias in an item. This study aims to identify items at scientific literacy skills with integrated science (SLiSIS) test that experience DIF based on gender. Moreover, it is analyzed the emergence of DIF, especially related to the test construct measured, and concluded on how far the validity of the SLiSIS test from the construct validity of consequential type. The study was conducted with a quantitative approach by using a survey or non-experimental methods. The samples of this study were the responses of the SLiSIS test taken from 310 eleventh-grade high school students in the science program from SMA 2 and SMA 3 Tegal. The DIF analysis technique used Wald Test with the Rasch model. From the findings, eight items contained DIF in a 95 % level of trust. In 99 % level of trust, three items contained DIF, items 1, 6, and 38 or 7%. The DIF is caused by differences in test-takers ability following the measured construct, so it is not a test bias. Thus, the emergence of DIF on SLiSIS test items does not threaten the construct validity of the consequential type.

Confirmatory Factor Analysis, Internal Consistency, Gender Differential Item Functioning and Discriminant Validity of the Fear of COVID-5 Scale Amidst Emerging Adult University Students in Mexico.

The study aimed to perform confirmatory factor analysis, internal consistency, gender differential item functioning, and discriminant validity of the Fear of COVID-5 Scale in emerging adult students of a university in Mexico. Confirmatory factor analysis, internal consistency (Cronbach's alpha and McDonald's omega), and gender differential item functioning were estimated (Kendall tau b correlation). The Fear of COVID-5 Scale showed a one-dimension structure (RMSEA = 0.07, CFI = 0.98, TLI = 0.96, and SRMR = 0.02), with high internal consistency (Cronbach's alpha of 0.78 and McDonald's omega of 0.81), non-gender differential item functioning (Kendall tau b between 0.07 and 0.10), and significant discriminant validity (Higher scores for fear of COVID-19 were observed in high clinical anxiety levels). In conclusion, the Fear of COVID-5 Scale presents a clear one-dimension structure similar to a previous study.

Psychometric performance of the brief Francis scale of attitude towards Christianity (Francis-4) in Mexican university students

ABSTRACT The brief Francis scale of attitude towards Christianity (Francis-4) psychometric performance in Mexico is unknown. The objective was to evaluate the dimensionality, homogeneity, gender differential item functioning (DIF), and nomological validity of Francis-4. 1,108 university students between 18 and 60 years of age were evaluated; 80.4% were women (n = 891). CFA was performed with the following fit indicators: chi-square, RMSEA, CFI, TLI, and McDonald’s omega were calculated. For nomological validity, the mean and standard deviation according to gender were compared with Student’s t-test. Gender DIF was quantified with Kendall’s tau-b. The CFA revealed the following goodness-of-fit coefficients: chi-square = 25.45, df = 2, p = .001, RMSEA = .10, 90% CI .07 – .14, CFI = .99, TLI = .99, and SRMR = .01. Cronbach’s alpha of .97 and McDonald’s omega were .97. Scores were significantly higher in women than in men (M = 8.9, SD = 5.4 versus M = 7.6, t = 3.15, p = .002). Kendall’s tau ranged from .07 to .09 indicating absence of gender DIF. Francis-4 is a unidimensional instrument with nomological validity, high internal consistency, and absence of gender DIF. It is recommended to evaluate its performance in other populations.

The self-reflection and insight scale: applying item response theory to craft an efficient short form

The human ability for self-consciousness—the capacity to reflect on oneself and to think about one’s thoughts, experiences, and actions—is central to understanding personality and motivation. The present research examined the psychometric properties of the Self-reflection and Insight Scale (SRIS), a prominent self-report scale for measuring individual differences in private self-consciousness. Using tools from Rasch and item response theory models, the SRIS was evaluated using responses from a large sample of young adults (n = 1192). The SRIS had many strengths, including essentially zero gender-based differential item functioning (DIF), but a cluster of poor performing items was identified based on item misfit, high local dependence, and low item difficulty and discrimination. Based on the IRT analyses, a concise 12-item scale, evenly balanced between self-reflection and insight, was crafted. The short SRIS showed strong dimensionality, reliability, item fit, and local independence as well as essentially no gender DIF. Taken together, the many psychometric strengths of the SRIS support its popularity, and the short form will be useful for research and applied contexts where an efficient, concise version is needed.

Expanding the purview of wellness indicators: validating a new measure that includes attitudes, behaviors, and perspectives

ABSTRACT Objective The present study validated the DeltaQuest Wellness Measure (DQ Wellness), a new 15-item measure of wellness that spans relevant attitudes, behaviors, and perspectives. Design This cross-sectional web-based study recruited chronically-ill patients and/or caregivers (n = 3,961) and a nationally representative comparison group (n = 855). Main Outcome Measures The DQ Wellness assesses: a way of being in the world that involves seeing and embracing the good and expressing kindness toward others; engagement in one’s activities and self-care; downplaying negative thoughts that reduce one’s energy; and an ability to feel joy. Six widely used measures of physical and mental health, cognition, and psychological well-being enabled construct-validity comparisons. Item-response theory (IRT) methods evaluated reliability, factor structure, and differential item functioning (DIF) by gender. Results The DQ Wellness showed strong cross-sectional reliability (marginal reliability = 0.89) and fit a bifactor model (RMSEA = 0.063, CFI = 0.982, TLI = 0.983). The DQ Wellness general score demonstrated construct validity, convergent and divergent validity, unique variance, and known-groups validity, and minimal gender DIF. The study is limited to addressing cross-sectional reliability and validity, and response rates are not known due to the recruitment source. Conclusion The DQ Wellness is a relatively brief measure, taps novel content, and could be useful for observational or interventional studies.

Exploring the Validity of a Comprehensive Listening Test to Identify Differences in Primary School Students’ Listening Skills

ABSTRACT Effective listening comprehension skills are an important prerequisite for the academic success of primary school students. However, the assessment of listening skills in the instructional language appears to have received only scant attention in the literature. Therefore, the goal of the present study was twofold. Firstly, a comprehensive listening test was developed and different aspects of construct validity supporting the use of the listening test were explored. The listening test was administered to 1001 sixth-grade primary school students in Flanders, the Dutch-speaking part of Belgium. Next, the test items were controlled for item difficulty and discrimination, dimensionality, model-data fit, local item independence, monotonicity, and gender differential item functioning. The final listening test consisted of 24 multiple-choice and open-ended test items. Secondly, the listening test was used to identify differences between students’ listening skills based on gender and home language. The results indicate that gender was not significantly related to listening comprehension skills, but L1 Dutch-speaking students significantly outperformed L2 Dutch-speaking students. This study also covers possible further fine-tuning of the instrument.

Validation of Persian Translation of the Intercultural Sensitivity Scale Using Rasch Model

This study subjected the Intercultural Sensitivity scale consisting of 24 items to Rasch analysis in a sample of 200 participants. The scale was translated into Persian and administered to graduate and undergraduate university students. Analysis of data showed that there were no misfitting items. Furthermore, no item manifested gender differential item functioning (DIF). All the thresholds were ordered and respondents could distinguish well between categories of the scale. The results of principal component analysis (PCA) of standardized residuals revealed that there were two contrasts with eigenvalues above two. Deleting positively and negatively loading items separately did not improve model fit. Thus, the content of items was investigated and it became clear that most of the positively loading items covered those items which have negative content and conversely, negatively loading items encompassed the items which have positive content. This brings about two psychometric dimensions in this scale. 1) Wording of the items revealed that the existence of items with negative wording in the scale results in statistical artifacts, and 2) The secondary dimension here could be interpreted to be an artifact of the wording. It was concluded that the scale is unidimensional and enjoys acceptable psychometric properties.