Invariance Testing Research Articles

Testing for Measurement Invariance of the Counseling Center Assessment of Psychological Symptoms–34 Scores With a University Sample

This study expanded previous psychometric analyses to include gender and race using measurement invariance with the Counseling Center Assessment of Psychological Symptoms (CCAPS-34) on a large sample (n = 4,013) of students referred to a university counseling center. Internal consistency was demonstrated for all subscales with all αs and ωs ≥ .80. The seven-factor model provided a good to excellent fit for all participant subgroup responses. The total sample comparative fit index was .948, the root-mean-square error of approximation was .058 [.057, .059], and the standardized root-mean-square residual was .048. Full measurement invariance was documented across all gender and racial/ethnic identity comparisons (i.e., White, Black, Asian/Pacific Islander, multiracial, Latino/a/x, non-Latino/a/x). The study concluded that CCAPS-34 subscales scores can be used with confidence with college-aged students of diverse backgrounds for screening and outcome assessment purposes.

Chinese version of the simplified psychological flexibility scale-6 (C-Psy-Flex): Study of its psychometric properties from the perspective of classical test theory and network analysis

Existing measures of psychological flexibility have demonstrated certain limitations, highlighting the critical need for the development of a more comprehensive assessment tool that encompasses all facets of psychological flexibility. In response to this necessity, we have developed the 6-item Psychological Flexibility Scale (C-Psy-Flex) as a concise instrument designed to evaluate individuals' psychological flexibility comprehensively. The objective of this study was to assess the reliability and validity of the Psy-Flex among Chinese college students and teachers. A total of 1511 valid responses were collected, and participants completed the Chinese version of the Psy-Flex, along with several other questionnaires. Exploratory factor analysis with 6 factors and 6 items was conducted on Sample 1 (n = 756). Confirmatory factor analysis on Sample 2 (n = 755) demonstrated good fit for a single-factor structure (χ2 = 21.17, df = 8, P < 0.001, CFI = 0.989, TLI = 0.980, RMSEA = 0.047, RMSEA 90% CI [0.023, 0.071], SRMR = 0.020). Measurement invariance testing indicated strict equivalence of the Chinese version of Psy-Flex across different gender and sample groups. Psychological flexibility was significantly positively associated with life satisfaction, indicators of physical and mental health, and value questionnaire scores, and significantly negatively associated with negative emotions (depression, anxiety, stress) and experiential avoidance. Network analysis visualization techniques were used to further confirm the factor structure of the scale, providing additional evidence of the scale's structural consistency and internal coherence. The Chinese version of the psychological flexibility scale demonstrated good internal consistency, with a Cronbach's α coefficient of 0.82, 95% CI [0.810, 0.840], and a McDonald's ω coefficient of 0.83, 95% CI [0.811, 0.841]. Overall, the study found that the Chinese version of the Psychological Flexibility Scale demonstrated good reliability and validity.

Examining the Measurement Invariance and Validity of the SSIS SEL Brief + Mental Health Scales – Student Version in Austria and Germany

The SSIS SEL Brief + Mental Health Scales (SSIS SELb+MHS) are multi-informant assessments developed in the United States to assess the social and emotional learning (SEL) competencies and emotional behavior concerns (EBCs) of school-age youth. Although there are translations of the SEL items of the SSIS SELb+MHS available in other languages, a German translation has never been completed and validated, despite the growing need for SEL and mental health assessment in German-speaking countries. To address this need, this study’s primary purpose was the examination of a German translation of the assessment with a specific focus on measurement invariance and concurrent validity invariance testing with 821 3rd through 6th-grade students in Austria and Germany. Results indicated that the SELb+MHS items clustered into 2 SEL factors and 2 EBC factors. With regard to measurement invariance, the SELb+MHS functioned similarly across both Austria and Germany and full scalar invariance was achieved. Additionally, the overall pattern of concurrent validity relationships was as expected and similar across countries. Implications and future directions are discussed.

Structural Validity Evidence for the Oxford Utilitarianism Scale Across 15 Languages

Abstract: Background: The Psychological Science Accelerator (PSA) recently completed a large-scale moral psychology study using translated versions of the Oxford Utilitarianism Scale (OUS). However, the translated versions have no validity evidence. Objective: The study investigated the structural validity evidence of the OUS across 15 translated versions and produced version-specific validity reports. Methods: We analyzed OUS data from the PSA, which was collected internationally on a centralized online questionnaire . We also collected qualitative feedback from experts for each translated version. Results: For each version, we produced version-specific psychometric reports which include the following: (1) descriptive item and demographics analyses, (2) factor structure evidence using confirmatory factor analyses, (3) measurement invariance testing across languages using multiple-group confirmatory factor analyses and alignment optimization, and (4) reliability analyses using coefficients α and ω.

TIME-VARYING PARAMETER REGRESSIONS WITH STATIONARY PERSISTENT DATA

We consider local level and local linear estimators for estimation and inference in time-varying parameter (TVP) regressions with general stationary covariates. The latter estimator also yields estimates for parameter derivatives that are utilized for the development of time invariance tests for the regression coefficients. Our theoretical framework is general enough to allow for a wide range of stationary regressors, including stationary long memory. We demonstrate that neglecting time variation in the regression parameters has a range of adverse effects in inference, in particular, when regressors exhibit long-range dependence. For instance, parametric tests diverge under the null hypothesis when the memory order is strictly positive. The finite sample performance of the methods developed is investigated with the aid of a simulation experiment. The proposed methods are employed for exploring the predictability of SP500 returns by realized variance. We find evidence of time variability in the intercept as well as episodic predictability when realized variance is utilized as a predictor in TVP specifications.

Specific Internet Disorders in University Students in Taiwan and Hong Kong: Psychometric Properties with Invariance Testing for the Traditional Chinese Version of the Assessment of Criteria for Specific Internet-Use Disorders (ACSID-11)

Specific Internet Disorders in University Students in Taiwan and Hong Kong: Psychometric Properties with Invariance Testing for the Traditional Chinese Version of the Assessment of Criteria for Specific Internet-Use Disorders (ACSID-11)

Difficulties in Emotion Regulation Scale-Short Form (DERS-SF): Psychometric Validation and Measurement Invariance Testing in a Sample of Urban Indian Adolescents

Background: Adolescents are particularly vulnerable to emotion dysregulation (ED), a transdiagnostic marker of psychological disorders with implications for academic achievement, identity development, social bonding, mental health, and the overall well-being and development of adolescents. A well-validated tool for the assessment of ED is imperative for credible advances in research on ED among Indian adolescents. Difficulties in Emotion Regulation Scale-Short Form (DERS-SF) is a promising tool to assess ED but requires proper psychometric validation among Indian adolescents. The current study validated the factor structure and examined psychometric aspects including gender invariance testing of the DERS-SF in Indian adolescents aged between 14 and 20 years. Methods: In a cross-sectional study, a community sample of n = 2079 adolescents completed the English version of the DERS-SF, the Strengths, and Difficulties Questionnaire, the Barratt Impulsiveness Scale, and the Adolescent Problem Behaviour Index. Results: The six-factor model of the DERS-SF exhibited an acceptable model fit in the current study sample. Invariance testing revealed that the DERS-SF is conceptually interpreted in a similar manner by male and female adolescents. Two-way MANOVA indicated gender and age variations on a few DERS-SF subscales. Reliability for DERS-SF was good, α = 0.86. Adequate construct validity was demonstrated by moderate to strong associations of the DERS-SF subscales with measures of internalizing and externalizing symptoms, adolescent problem behaviors, and impulsivity. Conclusions: DERS-SF retained the popular six-factor structure with acceptable psychometric properties and hence can be reliably used with Indian adolescents.

Measurement invairance and construct validity of the Turkish version of the learner autonomy scale in a sample of high and secondary school students

The goal of this research is to adapt the 24-item "Learner Autonomy Scale" developed by Sereti and Giossos (2018) in higher education samples into Turkish by examining the psychometric properties of high school and secondary school samples, and to determine whether these groups are equivalent in terms of measurement invariance. The scale was applied in high school (n = 475) and secondary school (n = 395) samples consisting of 870 students. Different from the original four-factor scale form, EFA applied to both groups revealed a two-factor (factor load range: .308-.775) and 21-item. Correlation values (r = .209-.392, p˂.001) indicate that the factors are not strongly related. The factors produced adequate internal consistency coefficients (α = .706-.866; ω = .708-.871) and were validated by meeting the fit indices accepted in the literature for CFA. Measurement invariance tests revealed strong invariance for the structural and metric tests and partial invariance for the scalar test in high school and secondary school samples. More research is needed to determine why the intersections of items 19, 20, and 21 are not invariant. The main contribution to "learner autonomy" in this study is the adaptation and justification of a valid and reliable measurement tool for determining autonomy in the adolescent age group. The use of the adapted scale in different educational environments and in the examination of "autonomy" by adapting it specific to the field (science, mathematics, etc.) will provide important implications for further theoretical studies.

Rethinking Measurement Equivalence in Comparative Political Research

A growing number of studies test key cultural constructs used in comparative political research for invariance across countries, only to declare these constructs incomparable if the invariance tests fail. The assumption underlying this kind of conclusion is that between-country differences in within-country factor solutions reflect cross-cultural inequivalence of the respective survey items. Using a Bayesian approach, we explore variation in measurement quality across countries focusing on sexual emancipation. We find that between-country differences in factor solutions reflect little more than imbalances in within-country dispersions of the construct items. As we demonstrate, discrete and bounded response scales used in standard surveys enforce a mathematical relationship between extreme country means, low within-country dispersions, and weak factor solutions. We formalize this insight by introducing a between–within model of cultural variation. Our contribution calls into question the practice of declaring constructs to be incomparable based on failed invariance tests.

Application of DeLone and McLean model on students’ E-learning systems during COVID-19 using PLS-SEM

Education is the most important factor for growth of countries’ social and economic development. The purpose of this study is to know online learning experience, system satisfaction and system usage of students in University of Gujrat. Due to COVID-19 pandemic, traditional (face to face) learning method was stopped and educational institutions shifted to online teaching and learning platforms. To gauge online learning system (platforms) a modified DeLone and McLean (D&amp;M) model was used that comprising Instructors’ Information Quality, Instructors’ Service Quality, System Quality, System Usage, System Satisfaction and Academic Achievements factors. The data was collected from 340 students (Batch, 2017-20) using self-administrated questionnaire through Students’ University email medium. Reliability and Validity of the data was tested and found up to the mark. The results of Confirmatory Factor Analysis revealed that all underlying factors achieved measurement invariance test and Fernell and Lacker criteria. Multi-group analysis showed that students of BS4th and BS8th/MSc4th have different opinion with respect to instructors’ information quality towards system satisfaction. Also, students of BS6th and BS8th/MSc 4th have different view according to system usage and their academic achievements. Further, PLS-SEM results showed that students of BS6th and BS8th/MSc4th reported significant positive impact of System Usage on their Academic Achievements. Furthermore, results depicted that a week negative relationship exist between instructors’ service quality with system satisfaction.

From identity-based distress to thinking “I am better off being dead:” Minority stress, posttraumatic cognitions, and suicidal ideation

BackgroundSuicidal ideation (SI) disproportionately impacts individuals with minoritized race/ethnicity, gender, and sexual orientation. Minority stress – i.e., traumatic, insidious distress that results from acts of discrimination – may lead to the formation of posttraumatic cognitions that may generalize to suicidal ideation, elevating SI risk in minoritized populations. The current study aimed to test this potential relationship by examining whether minority stress and posttraumatic cognitions accounted for the association between discrimination and SI. MethodsSeries of structural equation models, including multigroup confirmatory factor analyses conducted to test invariance of latent constructs, were estimated on cross-sectional data collected from minoritized young adults (n = 337). ResultsResults supported the hypothesized model: experience of discrimination indirectly associated with SI via correlations shared between minority stress and posttraumatic cognitions. Experiences of discrimination lacked a significant correlation with SI while accounting for minority stress and posttraumatic cognition variance. Invariance testing conducted to account for applicability of the model across race, ethnicity, sexual orientation, and plurality of minoritized identities all demonstrated that the model was applicable across these identity dimensions. LimitationsGranular inspection of identity dimensions was infeasible due to sample size and causal inferences cannot be drawn given cross-sectional nature of the data used. ConclusionsPosttraumatic cognitions within the context of discrimination may be effective treatment targets for minoritized individuals who present with minority stress and SI. Future studies should aim to replicate such findings longitudinally to infer temporality.

Sustainability-related product satisfaction – Development and application of a multi-dimensional measurement instrument

Given the increased pressure of customers on companies to engage in sustainability efforts, satisfying their sustainability expectations for their products is crucial. However, research on customer satisfaction so far has widely neglected sustainability concerns. Based on the sustainability literature we conceptualize a multi-dimensional measurement instrument for sustainability-related product satisfaction (SPS). Scale purification and validation using three independent samples support six subdimensions: sustainable product value, sustainable product manufacturing, sustainable product use, sustainable product life-cycle, product accessibility, and sustainable product end-of-life. The employed MIMIC approach lends support to the conceptualization of SPS as a Type II second-order factor model. Measurement invariance tests support the scale's cross-regional equivalence. When assessing criterion-related validity, we find that compared with other facets of product satisfaction, SPS is strongly related to increasing word-of-mouth but less so to customer loyalty. The availability of sustainability-related information is positively associated with SPS, which in turn contributes to overall product satisfaction.

Associations between school-wide practices and school-wide bullying: Advancing the cross-country understanding of teachers' perspectives from the U.S. and China.

To understand the role of different school-wide practices in school-wide bullying prevention in the global context, this present study was guided by the social-ecological framework to examine cross-country similarities and differences in the association between three forms of school-wide practices (i.e., punitive, positive, and social and emotional learning [SEL] practices) and school-wide bullying between United States (n = 1,833) and Chinese (n = 1,627) teachers from middle and high schools. Measurement invariance tests confirmed that the associations between the three forms of school-wide practices and school-wide bullying were comparable across the two countries. Results of multilevel analyses suggested that more frequent positive practices at the between-school level were associated with increased school-wide bullying in the United States but decreased school-wide bullying in the Chinese. Also, more frequent punitive practices at the within-school level were associated with increased school-wide bullying in both the U.S. and Chinese samples, and this positive association between punitive practices and school-wide bullying was significantly stronger in the Chinese sample than in the U.S. sample. More frequent punitive practices at the between-school level were also associated with increased school-wide bullying in the U.S. sample, but not in the Chinese sample. Moreover, the frequent SEL practices at the within-school level were significantly associated with decreased school-wide bullying in the United States, but not in China; the frequent SEL practices at the between-school level were associated with decreased school-wide bullying in the United States but increased school-wide bullying in China. Implications for school-wide practices in bullying prevention and intervention with sociocultural considerations were discussed. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Factor structure and longitudinal invariance for the Chinese Mainland version of the Edinburgh postnatal depression scale during pregnancy

BackgroundThere are inconsistent results on the Edinburgh Postnatal Depression Scale's (EPDS) factor structure and longitudinal invariance among different cultures. Furthermore, limited relevant studies in Chinese pregnant women exist. PurposeTo test the factor structure of the Chinese Mainland EPDS during pregnancy and conduct longitudinal invariance analyses. MethodsA national multi-centre cohort study was conducted among 1207 pregnant women selected consecutively by convenience sampling from five hospitals in Zhuhai, Taiyuan, Haidian, Changchun, and Shenzhen in China between August 2015 and October 2016. Depression was measured by the EPDS during gestational weeks 10–13, 15–18, 23–25, 30–32 and 36–37, respectively.s ResultsThree factors with eigenvalues nearly larger than 1.0 were optimal for the Chinese Mainland EPDS, labelled “anxiety,” “anhedonia,” and “depression,” and contained items 3–5, 1–2, and 6–10, respectively. The confirmatory factor analysis results of standardized root mean square residual (SRMR) = 0.034, root mean square error of approximation (RMSEA) = 0.049, comparative fit index (CFI) = 0.968, Tucker-Lewis index (TLI) = 0.954, and χ2, p < 0.05 indicated good fit. For the longitudinal invariance tests, the configural invariance was met, with the CFI and TLI both higher than 0.90 and the RMSEA lower than 0.08 (CFI = 0.919, TLI = 0.908, and RMSEA = 0.034). The metric-, scalar-, and strict invariances were met. ConclusionsThe three-factor model of the Chinese Mainland EPDS is invariant in pregnancy, suggesting stability and comparability in identifying the women screened positive at different points during pregnancy and making the scale feasible to screen prenatal depression and anxiety simultaneously.

The Short Version of the Italian Maastricht Vital Exhaustion Questionnaire (MVEQ): Psychometric Properties and Relationships with Anxiety, Depression, and Stress in a Community Sample of Older Adults

Vital Exhaustion (VE) refers to a physical and mental state of excessive fatigue, feelings of demoralization, hopelessness, and increased irritability. The short form of the Maastricht Vital Exhaustion Questionnaire (MVEQ) is a widely used measure to assess VE. Despite its utility is broadly recognized, the validity and reliability of the scale have yet to be examined in the Italian context. The present study aimed to evaluate the psychometric properties of the shortened MVEQ in a community sample of Italian older adults. A total of 722 older adults (Mage = 72.97, SD = 7.71; 60.4% females) completed the MVEQ, as well as other self-report questionnaires assessing anxiety, depression and stress in order to evaluate the criterion-related validity of the scale. A confirmatory factor analysis (CFA) was conducted to examine the original MVEQ latent structure. Internal consistency was assessed through model-based omega coefficient. Test-retest reliability was examined by re-administering the MVEQ after three months to a subsample of 568 participants. Factorial invariance tests across gender were conducted by means of multi-group CFAs. The one-factor model showed an acceptable fit to the data. The MVEQ yielded a reliable total score (ω = 0.822) and showed moderate-to-large correlations with measures of anxiety, depression, and stress (r range 0.30 to 0.75, ps < 0.001). Test-retest reliability was supported by an Intraclass Correlation Coefficient (ICC) of 0.661. Lastly, the scale was factorially invariant across gender. Overall, the MVEQ provided evidence of reliability and criterion-related validity in a sample of Italian older adults and may be useful for both clinical and research practices.

A bifactor representation of the Center for Epidemiological Studies Depression Scale for children: gender and age invariance and implications for adolescents’ social and academic adjustment

BackgroundThe Center for Epidemiological Studies Depression Scale for Children (CES-DC) is a widely used scale for screening depressive symptoms in children and adolescents. This study aims to uncover the optimal factor structure of the DES-DC and presents an alternate conceptualization of adolescent depression by estimating bifactor models and several competing models using a sample of Chinese adolescents.MethodsThe participants were 533 adolescents (49.7% boys, 49.7% girls, 3 participants did not report) between 12 and 18 years of age attending public secondary schools in the middle part of mainland China. Data were collected in classrooms using a questionnaire survey. A structural equation modeling approach was used to estimate and compare a series of competing models for the DES-DC.ResultsA Bifactor exploratory structural equation model (Bi-ESEM) with the best model fit was retained for representing the current data. Tests of measurement invariance demonstrated strict measurement equivalence across gender and age. No gender and age differences have been found in the general depression factor. Findings provided evidence for the composite reliability and construct validity of DES-DC. Depressive symptoms positively related to the Big Five trait neuroticism, negative emotions, loneliness, social anxious behaviors, and disruptive behaviors during school and negatively related to agreeableness, conscientiousness, extraversion, physical health status, school connectedness, and academic self-efficacy.ConclusionsThis study provides support for the one general factor construct of the CES-DC and the continuum concept structure of adolescent depression. Moreover, this research offers empirical evidence for comparing depression symptoms among adolescent populations with diverse genders and age groups. Additionally, the findings replicate and expand upon the implications of depressive symptoms on adolescents’ traits, well-being, social-relational adjustment, and academic adaptation.

Psychometric properties of the Chinese version of the Gudjonsson compliance scale: scale validation and associations with mental health

BackgroundTrait compliance involves people reacting favorably to demands made by others across different situations. This may lead to susceptibility to external pressures, exploitation, and manipulation. Moreover, trait compliance was found to correlate with various mental health outcomes, such as depression and anxiety. The Gudjonsson Compliance Scale (GCS) is an efficient tool for assessing trait compliance in Western contexts. To date, no study has validated the psychometric properties of the GCS in Chinese populations.MethodsTwo college student samples from China were recruited. The first sample (N = 4,276) was used to conduct exploratory factor analysis. The second (N = 4,356) was used to perform a confirmatory factor analysis. The reliability, measurement invariance, and correlational tests were conducted on the two combined samples.ResultsThe Chinese GCS showed a 3-factor structure, with two items deleted. Reliability was supported by moderate-to-good internal consistency of the three-factor scales and good internal consistency on the full scale. Strong measurement invariance across sex, ethnicity, and group recruitment was supported. Scores of the total scale and factor scales were found to significantly associated with several mental health problems.ConclusionsThe Chinese version of the GCS appears to be a valid and reliable instrument for measuring trait compliance and could promote both the assessment and research on compliance in Chinese population.

Psychometric properties of the Chinese version of the 15-item Need for Closure Scale: Scale validation and associations with mental health.

Need for closure (NFC) has been found to be implicated in different forms of psychopathology. The 15-item Need for Closure Scale (NFCS) is an efficient and easy tool for assessing individuals' NFC in Western contexts. However, the psychometric properties of the 15-item NFCS have not yet been validated in Chinese populations. Two different samples of university students from China were recruited in this study. The first sample (N = 5080, 49.9% females) was used to conduct exploratory factor analysis and reliability analysis. The second sample (N = 3968, 64.2% females) was used to perform confirmatory factor analysis, exploratory structural equation modeling (ESEM), and bifactor models, followed by tests of measurement invariance and criterion validity. The full scale showed good internal consistency. The bifactor-ESEM result with a general factor and four specific factors was chosen as our final model. Strong measurement invariance across sex and ethnicity groups was supported. Evidence was obtained for the criterion validity of NFCS scores with respect to depression, anxiety, and psychological distress. The Chinese NFCS appears to be a valid and reliable instrument for measuring the NFC, which could promote both the assessment and research of the NFC in Chinese populations.

Making Nations Great Again: National Narcissism and the Left, the Right, and the Extreme in the European Context

Considering current world politics, investigating people’s tendency to believe in the greatness of their nation contingent on external validation seems ever so relevant. Thus, we examined the relationship between the direction and extremity of political orientation and national narcissism (NN) on European quota-representative samples (N = 15,882). Although the relationships between ideological extremity and NN were established, they were much weaker than the positive relationship between political orientation and NN. Testing for model invariance across Western and Eastern European samples suggested differences in the predictive strength of political orientation on NN, with a weaker association in Eastern Europe. Further analyses, including a quadratic examination of political ideology’s relation to NN, supported the rigidity-of-the-right hypothesis, highlighting the stronger presence of NN among right-leaning individuals. This pattern persisted despite varying European socio-cultural backgrounds, suggesting a transcultural aspect of NN within the political right spectrum. Our research underscores the importance of considering socio-political context when assessing the interplay between political ideology and NN, setting a starting point for further, more nuanced research.

Assessing the Structural Validity of the Knee Injury and Osteoarthritis Outcome Score Scale.

The Knee Injury and Osteoarthritis Outcome Score (KOOS) scale is used to assess patient perspectives on knee health. However, the structural validity of the KOOS has not been sufficiently tested; therefore, our objective was to assess the KOOS in a large, multi-site database of patient responses who were receiving care for knee pathology. A cross-sectional study was conducted using the Surgical Outcome System (SOS) database. A confirmatory factor analysis (CFA) was conducted to assess the proposed five-factor KOOS using a priori cut-off values. Because model fit indices were not met, a subsequent exploratory factor analysis (EFA) was conducted to identify a parsimonious model. The resulting four-factor structure (i.e., KOOS SF-12) was then assessed using CFA and subjected to multigroup invariance testing. The original KOOS model did not meet rigorous CFA fit recommendations. The KOOS SF-12 did meet model fit recommendations and passed all invariance testing between intervention procedure, sex, and age groups. The KOOS failed to meet model fit recommendations. The KOOS SF-12 met model fit recommendations, maintained a multi-factorial structure, and was invariant across all tested groups. The KOOS did not demonstrate sound structural validity. A refined KOOS SF-12 model that met recommended model fit indices and invariance testing criteria was identified. Our findings provide initial support for a multidimensional KOOS structure (i.e., KOOS SF-12) that is a more psychometrically sound instrument for measuring patient-reported knee health.