Measurement Invariance Testing Research Articles

Assessing age discrimination in workplaces: psychometric exploration of the Workplace Age Discrimination Scale (WADS-G).

In contemporary workplaces characterized by diverse age groups working collaboratively, the assessment of age discrimination as an interpersonal phenomenon has gained heightened significance. This study focuses on adapting and scrutinizing the psychometric properties of the German iteration of the Workplace Age Discrimination Scale (WADS-G). Comprehensive Confirmatory Factor Analysis (CFA) results affirm a robust fit for the unidimensional model. Convergent validity is established through correlations between WADS-G scores and related instruments, while discriminant validity is evidenced by its lack of association with extraversion. Noteworthy findings include a positive correlation with turnover intention and negative correlations with job satisfaction, occupational self-efficacy, and organizational affective commitment. Despite its merits, the predictive efficacy of the WADS-G is notably inferior when juxtaposed with the Workplace Incivility Scale. Its explanatory power for turnover intention is constrained when accounting for variables such as job satisfaction, work environment, neuroticism, and core self-evaluation. Although measurement invariance testing across gender groups reveals scalar to strict measurement invariance, the examination across age groups indicates metric invariance. However, Confirmatory Factor Analyses for the 18-30 and 50+ age groups, central to the research emphasis, reveal suboptimal model fit. These outcomes prompt a nuanced discussion on whether the WADS-G aptly captures age-discriminatory experiences across diverse age and gender cohorts among employees.

대학생의 학과만족 척도 개발 및 타당화

Objectives This study aims to develop Satisfaction with Academic Department Scale (SADS) and testing measurement invariance by gender and major field for university students in South Korea.
 Methods The survey dataset of 1,775 students was randomly split into two samples. One of them was used for exploratory factor analysis, and the other one was used for confirmatory factor analysis.
 Results The results of the exploratory factor analysis indicated that SADS consisted of four factors (i.e.,‘course satisfaction’, ‘relationship satisfaction’, ‘perception satisfaction’, ‘general satisfaction’). The confirmatory factor analysis confirmed that the SADS consisted of 4 factors with 20 items in satisfactory model fit indices. Moreover, measurement invariance by gender and major field (humanities⋅social science/natural sciences⋅engineering/ arts⋅physical education) were strictly supported. The internal consistencies of the items for each sub-factor of SADS were also reliable.
 Conclusions It is expected that SADS will be useful instrument to measure the satisfaction with academic department for university students regardless of gender and major field.

Measurement Invariance of the Children's Social Desirability Scale-Short Version (CSD-S) Across Gender, Grade Level, and Race/Ethnicity.

Social desirability bias is a methodological challenge with participant self-reports. Social desirability measures are often used to control the potential effects of social desirability bias on participant self-reports, but whether these measures are interpreted similarly across different demographic groups remains unclear. This study examines the factor structure and measurement invariance of the Children's Social Desirability Scale-Short version (CSD-S) across gender, grade level, and race/ethnicity with a large sample of children and adolescents in the United States. Results indicate a close fit to a one-factor model. Tests of measurement invariance show partial strong invariance across gender (male vs. female) and grade level (elementary vs. middle vs. high schoolers) as well as strong invariance across race/ethnicity (Asian vs. Black/African American vs. Hispanic/Latine vs. White vs. Bi/Multiracial). Latent mean differences were found across grade level and race/ethnicity but not gender, with lower grades reporting higher social desirability than upper grades and Bi/Multiracials reporting lower social desirability than Asians and Hispanics/Latines. Findings provide preliminary evidence for the use of CSD-S in detecting social desirability bias and assessing meaningful social desirability differences in diverse children and adolescents.

The Fatigue and Altered Cognition Scale among SARS-CoV-2 Survivors: Psychometric Properties and Item Correlations with Depression and Anxiety Symptoms.

Background/Objectives: This study examined the psychometric properties of the Fatigue and Altered Cognition Scale (FACs) among adult COVID-19 survivors and its unique ability to assess symptomology not accounted for by measures of depression and anxiety. Methods: COVID-19 survivors completed an online survey that included the FACs, a measure of brain fog and central fatigue with 20 items rated on a digital-analog scale. Useable data from 559 participants were analyzed to test the two-factor structure of the FACs, test for measurement invariance by sex and device was used to complete the survey (hand-held, computer), and item correlations with symptoms of depression and anxiety were examined. Results: The two-factor structure of the FACs replicated, supporting the separate assessments of brain fog and fatigue, χ2(164) = 1028.363, p < 0.001, CFI = 0.934, TLI = 0.923, RMSEA = 0.097, SRMR = 0.053. The FACs exhibited invariance at the scalar level, indicating item and factor integrity regardless of sex and device type. Using a correlation > 0.70 as a criterion (i.e., indicating more than 50% shared variance between two items), items on the FACs (assessing fatigue and lack of energy) were highly correlated with feeling tired or having little energy on the depression measure. No other items correlated with any anxiety symptom larger than 0.70. Conclusions: The FACs appears to be a psychometrically sound and efficient measure for use with COVID-19 survivors, assessing symptoms of brain fog and central fatigue that are not attributable to symptoms assessed by established measures of depression and anxiety.

Chinese version of the simplified psychological flexibility scale-6 (C-Psy-Flex): Study of its psychometric properties from the perspective of classical test theory and network analysis

Existing measures of psychological flexibility have demonstrated certain limitations, highlighting the critical need for the development of a more comprehensive assessment tool that encompasses all facets of psychological flexibility. In response to this necessity, we have developed the 6-item Psychological Flexibility Scale (C-Psy-Flex) as a concise instrument designed to evaluate individuals' psychological flexibility comprehensively. The objective of this study was to assess the reliability and validity of the Psy-Flex among Chinese college students and teachers. A total of 1511 valid responses were collected, and participants completed the Chinese version of the Psy-Flex, along with several other questionnaires. Exploratory factor analysis with 6 factors and 6 items was conducted on Sample 1 (n = 756). Confirmatory factor analysis on Sample 2 (n = 755) demonstrated good fit for a single-factor structure (χ2 = 21.17, df = 8, P < 0.001, CFI = 0.989, TLI = 0.980, RMSEA = 0.047, RMSEA 90% CI [0.023, 0.071], SRMR = 0.020). Measurement invariance testing indicated strict equivalence of the Chinese version of Psy-Flex across different gender and sample groups. Psychological flexibility was significantly positively associated with life satisfaction, indicators of physical and mental health, and value questionnaire scores, and significantly negatively associated with negative emotions (depression, anxiety, stress) and experiential avoidance. Network analysis visualization techniques were used to further confirm the factor structure of the scale, providing additional evidence of the scale's structural consistency and internal coherence. The Chinese version of the psychological flexibility scale demonstrated good internal consistency, with a Cronbach's α coefficient of 0.82, 95% CI [0.810, 0.840], and a McDonald's ω coefficient of 0.83, 95% CI [0.811, 0.841]. Overall, the study found that the Chinese version of the Psychological Flexibility Scale demonstrated good reliability and validity.

Structural Validity Evidence for the Oxford Utilitarianism Scale Across 15 Languages

Abstract: Background: The Psychological Science Accelerator (PSA) recently completed a large-scale moral psychology study using translated versions of the Oxford Utilitarianism Scale (OUS). However, the translated versions have no validity evidence. Objective: The study investigated the structural validity evidence of the OUS across 15 translated versions and produced version-specific validity reports. Methods: We analyzed OUS data from the PSA, which was collected internationally on a centralized online questionnaire . We also collected qualitative feedback from experts for each translated version. Results: For each version, we produced version-specific psychometric reports which include the following: (1) descriptive item and demographics analyses, (2) factor structure evidence using confirmatory factor analyses, (3) measurement invariance testing across languages using multiple-group confirmatory factor analyses and alignment optimization, and (4) reliability analyses using coefficients α and ω.

Measurement of adolescent psychological wellbeing: A test of factor structure and measurement invariance.

Gains in holistic approaches to adult mental health have been associated with increasing interest in understanding psychological wellbeing (PWB) among adolescents. Empirical examination of measurement models for PWB in adolescence is lacking. Thus, the current study examined PWB in a longitudinal, diverse sample of 433 adolescents (non-Latinx Black: 37.6%; non-Latinx White: 25.9%; Latinx: 36.5%; Male adolescents: 50.1%). A one-factor, correlated six-factor and hierarchical models were examined across racial/ethnic (White, Black, and Hispanic) and gender (female, male) identities, after which the best fitting model was selected to undergo invariance testing. A one-factor structure was superior, and exhibited strict invariance across racial/ethnic and gender identities at each wave of the study, as well as longitudinal invariance within the entire sample.

Examining the Measurement Invariance and Validity of the SSIS SEL Brief + Mental Health Scales – Student Version in Austria and Germany

The SSIS SEL Brief + Mental Health Scales (SSIS SELb+MHS) are multi-informant assessments developed in the United States to assess the social and emotional learning (SEL) competencies and emotional behavior concerns (EBCs) of school-age youth. Although there are translations of the SEL items of the SSIS SELb+MHS available in other languages, a German translation has never been completed and validated, despite the growing need for SEL and mental health assessment in German-speaking countries. To address this need, this study’s primary purpose was the examination of a German translation of the assessment with a specific focus on measurement invariance and concurrent validity invariance testing with 821 3rd through 6th-grade students in Austria and Germany. Results indicated that the SELb+MHS items clustered into 2 SEL factors and 2 EBC factors. With regard to measurement invariance, the SELb+MHS functioned similarly across both Austria and Germany and full scalar invariance was achieved. Additionally, the overall pattern of concurrent validity relationships was as expected and similar across countries. Implications and future directions are discussed.

Testing for Measurement Invariance of the Counseling Center Assessment of Psychological Symptoms–34 Scores With a University Sample

This study expanded previous psychometric analyses to include gender and race using measurement invariance with the Counseling Center Assessment of Psychological Symptoms (CCAPS-34) on a large sample (n = 4,013) of students referred to a university counseling center. Internal consistency was demonstrated for all subscales with all αs and ωs ≥ .80. The seven-factor model provided a good to excellent fit for all participant subgroup responses. The total sample comparative fit index was .948, the root-mean-square error of approximation was .058 [.057, .059], and the standardized root-mean-square residual was .048. Full measurement invariance was documented across all gender and racial/ethnic identity comparisons (i.e., White, Black, Asian/Pacific Islander, multiracial, Latino/a/x, non-Latino/a/x). The study concluded that CCAPS-34 subscales scores can be used with confidence with college-aged students of diverse backgrounds for screening and outcome assessment purposes.

Difficulties in Emotion Regulation Scale-Short Form (DERS-SF): Psychometric Validation and Measurement Invariance Testing in a Sample of Urban Indian Adolescents

Background: Adolescents are particularly vulnerable to emotion dysregulation (ED), a transdiagnostic marker of psychological disorders with implications for academic achievement, identity development, social bonding, mental health, and the overall well-being and development of adolescents. A well-validated tool for the assessment of ED is imperative for credible advances in research on ED among Indian adolescents. Difficulties in Emotion Regulation Scale-Short Form (DERS-SF) is a promising tool to assess ED but requires proper psychometric validation among Indian adolescents. The current study validated the factor structure and examined psychometric aspects including gender invariance testing of the DERS-SF in Indian adolescents aged between 14 and 20 years. Methods: In a cross-sectional study, a community sample of n = 2079 adolescents completed the English version of the DERS-SF, the Strengths, and Difficulties Questionnaire, the Barratt Impulsiveness Scale, and the Adolescent Problem Behaviour Index. Results: The six-factor model of the DERS-SF exhibited an acceptable model fit in the current study sample. Invariance testing revealed that the DERS-SF is conceptually interpreted in a similar manner by male and female adolescents. Two-way MANOVA indicated gender and age variations on a few DERS-SF subscales. Reliability for DERS-SF was good, α = 0.86. Adequate construct validity was demonstrated by moderate to strong associations of the DERS-SF subscales with measures of internalizing and externalizing symptoms, adolescent problem behaviors, and impulsivity. Conclusions: DERS-SF retained the popular six-factor structure with acceptable psychometric properties and hence can be reliably used with Indian adolescents.

Measurement invairance and construct validity of the Turkish version of the learner autonomy scale in a sample of high and secondary school students

The goal of this research is to adapt the 24-item "Learner Autonomy Scale" developed by Sereti and Giossos (2018) in higher education samples into Turkish by examining the psychometric properties of high school and secondary school samples, and to determine whether these groups are equivalent in terms of measurement invariance. The scale was applied in high school (n = 475) and secondary school (n = 395) samples consisting of 870 students. Different from the original four-factor scale form, EFA applied to both groups revealed a two-factor (factor load range: .308-.775) and 21-item. Correlation values (r = .209-.392, p˂.001) indicate that the factors are not strongly related. The factors produced adequate internal consistency coefficients (α = .706-.866; ω = .708-.871) and were validated by meeting the fit indices accepted in the literature for CFA. Measurement invariance tests revealed strong invariance for the structural and metric tests and partial invariance for the scalar test in high school and secondary school samples. More research is needed to determine why the intersections of items 19, 20, and 21 are not invariant. The main contribution to "learner autonomy" in this study is the adaptation and justification of a valid and reliable measurement tool for determining autonomy in the adolescent age group. The use of the adapted scale in different educational environments and in the examination of "autonomy" by adapting it specific to the field (science, mathematics, etc.) will provide important implications for further theoretical studies.

Application of DeLone and McLean model on students’ E-learning systems during COVID-19 using PLS-SEM

Education is the most important factor for growth of countries’ social and economic development. The purpose of this study is to know online learning experience, system satisfaction and system usage of students in University of Gujrat. Due to COVID-19 pandemic, traditional (face to face) learning method was stopped and educational institutions shifted to online teaching and learning platforms. To gauge online learning system (platforms) a modified DeLone and McLean (D&amp;M) model was used that comprising Instructors’ Information Quality, Instructors’ Service Quality, System Quality, System Usage, System Satisfaction and Academic Achievements factors. The data was collected from 340 students (Batch, 2017-20) using self-administrated questionnaire through Students’ University email medium. Reliability and Validity of the data was tested and found up to the mark. The results of Confirmatory Factor Analysis revealed that all underlying factors achieved measurement invariance test and Fernell and Lacker criteria. Multi-group analysis showed that students of BS4th and BS8th/MSc4th have different opinion with respect to instructors’ information quality towards system satisfaction. Also, students of BS6th and BS8th/MSc 4th have different view according to system usage and their academic achievements. Further, PLS-SEM results showed that students of BS6th and BS8th/MSc4th reported significant positive impact of System Usage on their Academic Achievements. Furthermore, results depicted that a week negative relationship exist between instructors’ service quality with system satisfaction.

Examining measurement invariance of appetitive trait and ARFID symptom measures by food security status

ObjectiveMeasures assessing appetitive traits (i.e., individual differences in the desire to consume food) and disordered eating have generally been developed in predominantly food-secure populations. The current study aims to test measurement invariance (MI) for a measure of appetitive traits and a measure of Avoidant Restrictive Food Intake Disorder (ARFID) symptomology across food security status. MethodData from a sample of mothers (n = 634) and two undergraduate samples (n = 945 and n = 442) were used to assess MI for the Adult Eating Behavior Questionnaire (AEBQ), which measures appetitive traits, and the Nine Item ARFID Screen (NIAS), which measures ARFID symptomology. Current food security was assessed using the 18-item USDA Household Food Security Survey Module, which was dichotomized into two groups: 1) the ‘food insecure’ group included marginal, low, and very low food security and 2) the ‘food secure’ group included high food security. Overall and multi-group confirmatory factor analyses were conducted separately for each measure in each sample. ResultsResults demonstrated scalar (i.e., strong) MI for both measures across samples, indicating that these measures performed equivalently across food-secure and food-insecure individuals. ConclusionFindings suggest that differences in appetitive traits by food security status observed in prior research are not artifacts of measurement differences, but instead reflect true differences. Additionally, past mixed results regarding the relationship between food insecurity (FI) and ARFID symptomology are not likely driven by measurement error when using the NIAS.

Sustainability-related product satisfaction – Development and application of a multi-dimensional measurement instrument

Given the increased pressure of customers on companies to engage in sustainability efforts, satisfying their sustainability expectations for their products is crucial. However, research on customer satisfaction so far has widely neglected sustainability concerns. Based on the sustainability literature we conceptualize a multi-dimensional measurement instrument for sustainability-related product satisfaction (SPS). Scale purification and validation using three independent samples support six subdimensions: sustainable product value, sustainable product manufacturing, sustainable product use, sustainable product life-cycle, product accessibility, and sustainable product end-of-life. The employed MIMIC approach lends support to the conceptualization of SPS as a Type II second-order factor model. Measurement invariance tests support the scale's cross-regional equivalence. When assessing criterion-related validity, we find that compared with other facets of product satisfaction, SPS is strongly related to increasing word-of-mouth but less so to customer loyalty. The availability of sustainability-related information is positively associated with SPS, which in turn contributes to overall product satisfaction.

Associations between school-wide practices and school-wide bullying: Advancing the cross-country understanding of teachers' perspectives from the U.S. and China.

To understand the role of different school-wide practices in school-wide bullying prevention in the global context, this present study was guided by the social-ecological framework to examine cross-country similarities and differences in the association between three forms of school-wide practices (i.e., punitive, positive, and social and emotional learning [SEL] practices) and school-wide bullying between United States (n = 1,833) and Chinese (n = 1,627) teachers from middle and high schools. Measurement invariance tests confirmed that the associations between the three forms of school-wide practices and school-wide bullying were comparable across the two countries. Results of multilevel analyses suggested that more frequent positive practices at the between-school level were associated with increased school-wide bullying in the United States but decreased school-wide bullying in the Chinese. Also, more frequent punitive practices at the within-school level were associated with increased school-wide bullying in both the U.S. and Chinese samples, and this positive association between punitive practices and school-wide bullying was significantly stronger in the Chinese sample than in the U.S. sample. More frequent punitive practices at the between-school level were also associated with increased school-wide bullying in the U.S. sample, but not in the Chinese sample. Moreover, the frequent SEL practices at the within-school level were significantly associated with decreased school-wide bullying in the United States, but not in China; the frequent SEL practices at the between-school level were associated with decreased school-wide bullying in the United States but increased school-wide bullying in China. Implications for school-wide practices in bullying prevention and intervention with sociocultural considerations were discussed. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A bifactor representation of the Center for Epidemiological Studies Depression Scale for children: gender and age invariance and implications for adolescents’ social and academic adjustment

BackgroundThe Center for Epidemiological Studies Depression Scale for Children (CES-DC) is a widely used scale for screening depressive symptoms in children and adolescents. This study aims to uncover the optimal factor structure of the DES-DC and presents an alternate conceptualization of adolescent depression by estimating bifactor models and several competing models using a sample of Chinese adolescents.MethodsThe participants were 533 adolescents (49.7% boys, 49.7% girls, 3 participants did not report) between 12 and 18 years of age attending public secondary schools in the middle part of mainland China. Data were collected in classrooms using a questionnaire survey. A structural equation modeling approach was used to estimate and compare a series of competing models for the DES-DC.ResultsA Bifactor exploratory structural equation model (Bi-ESEM) with the best model fit was retained for representing the current data. Tests of measurement invariance demonstrated strict measurement equivalence across gender and age. No gender and age differences have been found in the general depression factor. Findings provided evidence for the composite reliability and construct validity of DES-DC. Depressive symptoms positively related to the Big Five trait neuroticism, negative emotions, loneliness, social anxious behaviors, and disruptive behaviors during school and negatively related to agreeableness, conscientiousness, extraversion, physical health status, school connectedness, and academic self-efficacy.ConclusionsThis study provides support for the one general factor construct of the CES-DC and the continuum concept structure of adolescent depression. Moreover, this research offers empirical evidence for comparing depression symptoms among adolescent populations with diverse genders and age groups. Additionally, the findings replicate and expand upon the implications of depressive symptoms on adolescents’ traits, well-being, social-relational adjustment, and academic adaptation.

Psychometric properties of the Chinese version of the Gudjonsson compliance scale: scale validation and associations with mental health

BackgroundTrait compliance involves people reacting favorably to demands made by others across different situations. This may lead to susceptibility to external pressures, exploitation, and manipulation. Moreover, trait compliance was found to correlate with various mental health outcomes, such as depression and anxiety. The Gudjonsson Compliance Scale (GCS) is an efficient tool for assessing trait compliance in Western contexts. To date, no study has validated the psychometric properties of the GCS in Chinese populations.MethodsTwo college student samples from China were recruited. The first sample (N = 4,276) was used to conduct exploratory factor analysis. The second (N = 4,356) was used to perform a confirmatory factor analysis. The reliability, measurement invariance, and correlational tests were conducted on the two combined samples.ResultsThe Chinese GCS showed a 3-factor structure, with two items deleted. Reliability was supported by moderate-to-good internal consistency of the three-factor scales and good internal consistency on the full scale. Strong measurement invariance across sex, ethnicity, and group recruitment was supported. Scores of the total scale and factor scales were found to significantly associated with several mental health problems.ConclusionsThe Chinese version of the GCS appears to be a valid and reliable instrument for measuring trait compliance and could promote both the assessment and research on compliance in Chinese population.

Psychometric properties of the Chinese version of the 15-item Need for Closure Scale: Scale validation and associations with mental health.

Need for closure (NFC) has been found to be implicated in different forms of psychopathology. The 15-item Need for Closure Scale (NFCS) is an efficient and easy tool for assessing individuals' NFC in Western contexts. However, the psychometric properties of the 15-item NFCS have not yet been validated in Chinese populations. Two different samples of university students from China were recruited in this study. The first sample (N = 5080, 49.9% females) was used to conduct exploratory factor analysis and reliability analysis. The second sample (N = 3968, 64.2% females) was used to perform confirmatory factor analysis, exploratory structural equation modeling (ESEM), and bifactor models, followed by tests of measurement invariance and criterion validity. The full scale showed good internal consistency. The bifactor-ESEM result with a general factor and four specific factors was chosen as our final model. Strong measurement invariance across sex and ethnicity groups was supported. Evidence was obtained for the criterion validity of NFCS scores with respect to depression, anxiety, and psychological distress. The Chinese NFCS appears to be a valid and reliable instrument for measuring the NFC, which could promote both the assessment and research of the NFC in Chinese populations.

Sexual Consent, Alcohol Use, and Greek Membership: Testing Measurement Invariance and Group Mean Differences of Two Sexual Consent Measures

ABSTRACT Research into sexual consent among college students often focuses on gender and Greek community involvement differences. However, few studies have validated sexual consent measures used for such comparisons. The present study applied a structural equation modeling (SEM) framework to assess the psychometric properties and measurement invariances of two prevalent sexual consent measures across Greek membership and gender groups – the Sexual Consent-Related Behavior Intentions Scale and the Alcohol and Sexual Consent Scale. After establishing measurement invariance, the latent group means were tested between genders and Greek community status. The results with 501 college men (318 fraternity members and 183 non-fraternity members) and 1506 college women (1187 sorority women and 319 nonsorority women) suggested that both scales achieved scalar invariance, permitting confident usage for comparisons across Greek memberships and genders. The latent mean analyses revealed significant differences in intentions to negotiate sexual consent and beliefs regarding alcohol-involved sexual assault among the different groups.

Harmonisation of assessments of attention, social, emotional, and behaviour problems using the Child Behavior Checklist and the Strengths and Difficulties Questionnaire

AbstractObjectivesRetrospective harmonisation of data obtained through different instruments creates measurement error, even if the underlying concepts are assumed the same. We tested a novel method for item‐level data harmonisation of two widely used instruments that measure emotional and behavioural problems: the Child Behavior Checklist (CBCL) and the Strengths and Difficulties Questionnaire (SDQ).MethodsItem content of the CBCL and SDQ was mapped onto four dimensions: emotional problems, peer relationship problems, hyperactivity/inattention and conduct problems. A diverse test sample was drawn from four prospective longitudinal birth cohort studies in Australia and Europe who used one or both instruments. The pooled sample included 5188 data points assessing children and adolescents aged 6–13 years (N = 257–704 participants per cohort). Measurement invariance was assessed using latent variable multi‐group confirmatory factor analysis.ResultsFifteen items from the CBCL and SDQ were mapped onto four dimensions allowing for measurement invariance testing as part of a stepwise process. Partial strict invariance between CBCL and SDQ assessments was established for all four dimensions.ConclusionsThe harmonised dimensions of emotional, peer relationship, hyperactivity/inattention and conduct problems are invariant across the CBCL and SDQ suggesting that these dimensions can be reliably compared with limited measurement error.