Establishing Measurement Invariance Research Articles

Age-related differences in trait affect: Establishing measurement invariance of the Positive and Negative Affect Schedule (PANAS).

The present study examined trajectories of trait positive and trait negative affect as measured by the Positive and Negative Affect Schedule across the lifespan. Increasing levels of measurement invariance of the Positive and Negative Affect Schedule were tested across a wide age range (18-99 years of age; 10 age groups) in a large sample (N = 3,309; 65.82% women; 74.22% with a university entrance diploma; 92.23% with German as their mother tongue). Strong invariance was established so that the latent factor means of positive and negative affect could be meaningfully compared across age groups. Age had a small effect on positive affect, which showed a reversed U-shaped function with a maximum between the ages of 40 and 50. For negative affect, there was a strong cubic age effect. Specifically, negative affect decreased until about the age of 70 and then increased again. We also found a negative covariance between positive and negative affect, which first increased slightly and then decreased across age groups until becoming positive in the oldest age group. In summary, the present study supports previous findings on age trajectories of positive and negative affect but also highlights the importance of testing for measurement invariance. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Translation, transcultural adaptation, and convergent validity of the Arabic version of the Mukbang addiction scale

IntroductionThe present study evaluated the psychometric properties of the Arabic translation of the Mukbang Addiction Scale (MAS) among Arabic-speaking adults from the general population. Specifically, it aimed to assess the factorial structure through a confirmatory factor analysis, determine the composite reliability through Cronbach alpha and McDonald’s omega scores, assess gender invariance, and evaluate the convergent validity by examining its correlation to eating addiction and psychological distress.MethodsA total of 370 individuals with a mean age of 21.94 ± 2.29 years participated in this study, which was conducted using an online platform. The participants were surveyed on demographic information, mukbang addiction, food addiction, and psychological distress. Translation was conducted using the forward and backward technique.ResultsThe findings demonstrated excellent internal consistency of the Arabic MAS (McDonald’s omega coefficient = 0.93). Confirmatory factor analyses validated the one-factor structure of the scale, while establishing measurement invariance across sex at the configural, metric, and scalar levels. No sex differences were observed in the Mukbang addiction levels. Lastly, the MAS scores were significantly and positively correlated with food addiction and psychological distress, supporting their convergent validity.ConclusionThe current research provides evidence supporting the reliability and validity of the Arabic version of the MAS as a self-report method for assessing addictive Mukbang watching. While further validations are needed to corroborate the present findings, this measure can be effectively utilized across different fields, including schools, mental health centers, and researchers aiming to understand this global phenomenon.Plain English SummaryMukbang refers to individuals or hosts consuming large quantities of food while interacting with their audiences through recorded video or a live stream. The Mukbang phenomenon has gained substantial popularity among young individuals over the past years, rising concerns about its potential impact on their eating habits and health, especially when overconsumed. Overconsumption of mukbang content, or Mukbang addiction, aligns with the core features of addiction, including compulsive engagement, tolerance, and withdrawal symptoms, making it comparable to other addictive behaviors. Mukbang addiction can lead to detrimental effects on mental and physical health, including neglect of balanced nutrition, disordered eating habits, obesity, feelings of guilt, shame, poor self-esteem, distorted self-images, body dissatisfaction, heightened anxiety or depression, and social isolation. Given these significant impacts, valid and reliable tools are crucial to enable an accurate assessment of Mukbang addiction. This study proposes to translate, adapt and examine the psychometric properties of the Mukbang Addiction Scale (MAS) in a sample of Arabic-speaking individuals from the general population. Findings showed that the six MAS items loaded on a single factor with strong internal consistency and good convergent validity, preliminarily indicating its potential validity and reliability for assessing addictive mukbang-watching among Arabic-speaking individuals.

Validation of Romantic Partner Conflict Scale (RPCS) in Pakistani married couples: Establishing measurement invariance

Validation of Romantic Partner Conflict Scale (RPCS) in Pakistani married couples: Establishing measurement invariance

Measuring identification with narrative characters: the development and validation of a new scale

During narrative reception, one psychological response audiences may experience toward story characters is identification, which involves a sense of merging between self and character. Given the lack of formally validated measures of this construct in the literature, the current paper introduces a new 12-item scale for measuring identification. Scale development and validation took place over three sequential studies. Exploratory factor analysis in Study 1 (N = 224) indicated four related factors: merging, perspective-taking, understanding, and emotional involvement. In Study 2 (N = 191), confirmatory factor analysis suggested that a second-order four-factor model provided a good fit to the data and a more parsimonious explanation of the scale’s factor structure compared to a first-order model. In addition, the overall scale and subscales demonstrated adequate internal consistency and correlated in the expected directions with theoretically relevant and irrelevant constructs. Using a more demographically diverse sample, Study 3 (N = 290) established measurement invariance of the scale across two narratives in terms of configural, metric, and partial scalar invariance, and provided further support for its factor structure, reliability, and validity.

Measurement invariance of cognitive and affective job insecurity: A cross-national study

Empirical evidence of established measurement invariance of job insecurity measures may enhance the practical utility of job insecurity as a valid predictor when utilised over different cross-national samples. This study investigated the measurement invariance of the nine-item versions of the Job Security Index (a measure of cognitive job insecurity) and the Job Security Satisfaction Scale (a measure of affective job insecurity), across four countries (i.e. the United States, N = 486; China, N = 629; Italy, N = 482 and South Africa, N = 345). Based on a novel bifactor-(S-1) model approach we found evidence for partial metric, partial scalar and partial strict invariance of our substantive bifactor-(S-1) structure. The results extend measurement invariance research on job insecurity with obvious pragmatic implications (e.g. scaling units, measurement bias over cross-national interpretations).Contribution: This research provides evidence to support the applied use of cross-national comparisons of job insecurity scores across the nationalities included in this study. Theoretically, this research advances the debate about the nature of the relationship between cognitive and affective job insecurity, suggesting that in this cross-national dataset, a model where cognitive job insecurity is specified as the reference domain outperforms a model where affective job insecurity is assigned this status. Practically, it demonstrates that it is sensible and necessary to differentiate between cognitive and affective job insecurity and include measures of both constructs in future research on the construct.

Mental health during ecological crisis: translating and validating the Hogg Eco-anxiety Scale for Argentinian and Spanish populations

BackgroundEco-anxiety is increasingly recognized as a shared experience by many people internationally, encompassing fear of environmental catastrophe and anxiety about ecological crises. Despite its importance in the context of the changing climate, measures for this construct are still being developed in languages other than English.MethodsTo contribute to global eco-anxiety research, we translated the Hogg Eco-Anxiety Scale (HEAS) into Spanish, creating the HEAS-SP. We validated this measure in samples from both Argentina (n = 990) and Spain (n = 548), performing measurement invariance and confirmatory factor analyses. Internal consistency of the scale and score stability over time were investigated through reliability analyses. Differences in eco-anxiety across sociodemographic variables were explored through Student’s t-tests and Pearson’s r tests.ResultsThe four-factor model of the HEAS-SP comprising affective and behavioural symptoms, rumination, and anxiety about personal impact demonstrated excellent model fit. We found good internal consistency for each subscale, and established measurement invariance between Spanish and Argentine samples, as well as across genders and participants’ age. Spanish participants reported higher scores on the affective symptoms and personal impact anxiety factors compared to the Argentinian sample. Also, men reported lower levels than women on the subscales of affective symptoms, rumination, and personal impact anxiety. It was found that the relationship between both age and personal impact anxiety and age and affective symptoms varies significantly depending on the gender of the individuals. Younger participants tended to report higher scores on most dimensions of eco-anxiety.ConclusionsThese findings enhance the global initiative to investigate, explore and therefore comprehend eco-anxiety by introducing the first valid and reliable Spanish-language version of this psychometric instrument for its use within Spanish and Argentinian populations. This study augments the body of evidence supporting the robust psychometric properties of the HEAS, as demonstrated in prior validations for Australian, Turkish, Portuguese, German, French, and Italian populations.

Teaching Burnout in Student-Teachers and Professional Teachers: Teaching Burnout Across Teacher Groups, Genders and the Link of Teaching Burnout to Teaching Satisfaction

Although teaching burnout is considered a risk factor for teaching satisfaction, there remains a lack of comprehensive understanding regarding its impact on both student-teachers and professional teachers. The Maslach Burnout Inventory stands out as a widely recognized tool for measuring burnout, yet little is known about its measurement invariance across teacher groups and teachers’ genders. The current study therefore established measurement invariance across teacher groups and genders to reveal possible differences in the experience of teaching burnout across teacher groups and genders. Moreover, the current study investigate whether teaching burnout is a risk factor for teaching satisfaction in the teacher groups. 649 participants took part in the study (12.8% males, Mage = 34.6 (SD = 11.1). Results indicated that the MBI was invariant across teacher groups and genders. Student-teacher groups experienced less exhaustion and less professional efficacy. No gender difference in the experience of different aspects of burnout was found. Finally, exhaustion and cynicism significantly and uniquely negatively predicted teaching satisfaction, whereas professional efficacy significantly and uniquely positively teaching satisfaction. Student-teachers should be equipped with appropriate knowledge and skills to handle burnout in their future teaching career during their teacher training program.

Evaluating the Psychometric Properties of the Sexual and Gender Minority Adverse Childhood Experiences Scale Among Sexual Minority Men and Women.

Adverse childhood experiences (ACEs) are known determinants of negative health outcomes. Sexual and gender minority (SGM) individuals have higher ACE scores than non-SGM individuals. The SGM-ACE scale was developed to better assess this population but is not yet validated in SGM subgroups. This study aims to validate the sexual and gender minority adverse childhood experiences (SGM-ACE) scale among sexual minority men (SMM) and sexual minority women (SMW), while testing measurement invariance across both groups. A cross-sectional survey included 530 sexual minority adults (265 men, 265 women) in the United States. Cronbach's alpha established internal consistency reliability. Validity was assessed via confirmatory factor analysis (CFA) for the SGM-ACE's theoretical structure and Pearson's correlations for concurrent validity with substance use outcomes (alcohol, cannabis, and drugs). Multigroup structural equation modeling (SEM) determined measurement invariance between SMM and SMW. The CFA of the original model exhibited good fit. Fit was improved after removing the institutionalization item, chi-square (χ2) = 14.26, degrees of freedom (df) = 9, p = 0.113, minimum discrepancy (CMIN/df) = 1.59, comparative fit index (CFI) = 0.99, root mean square error of approximation (RMSEA) = 0.03, 90% confidence interval (CI): (0.00-0.06), and standardized root mean square residual (SRMR) = 0.02. Internal consistency reliability was established (α = 0.78). SGM-ACE exhibited weak, yet significant relationships with each substance use outcome. Multigroup SEM indicated measurement invariance between SMM and SMW. This study provides psychometric validation of the SGM-ACE, establishing measurement invariance between SMM and SMW. Future research should explore its utility in diverse SGM minority subgroups.

Toward Multidimensional Message Tailoring to Address COVID-19 and Influenza Vaccine-Hesitancy: A Latent Profile Analysis Approach

ABSTRACT Vaccines remain the best strategy as the COVID-19 pandemic enters into later stages and governments begin to shed pandemic-control measures. Vaccine hesitancy continues to be a major obstacle in efforts to end the pandemic. This study reports formative evaluation research that adopted a multidimensional approach using latent profile analysis to audience segmentation and message targeting. Within the framework of the integrated behavioral model, data were collected from a US national survey to explore the dimensions in which vaccine-confident vs. -hesitant individuals differed significantly across the topics of COVID-19 and influenza. Latent profile analyses were performed to identify subgroups and establish measurement invariance between COVID-19 and influenza vaccines. Matching message strategies were proposed for the distinctive characteristics of the subgroups for both topics and to be tested in future research.

Sexual Consent, Alcohol Use, and Greek Membership: Testing Measurement Invariance and Group Mean Differences of Two Sexual Consent Measures

ABSTRACT Research into sexual consent among college students often focuses on gender and Greek community involvement differences. However, few studies have validated sexual consent measures used for such comparisons. The present study applied a structural equation modeling (SEM) framework to assess the psychometric properties and measurement invariances of two prevalent sexual consent measures across Greek membership and gender groups – the Sexual Consent-Related Behavior Intentions Scale and the Alcohol and Sexual Consent Scale. After establishing measurement invariance, the latent group means were tested between genders and Greek community status. The results with 501 college men (318 fraternity members and 183 non-fraternity members) and 1506 college women (1187 sorority women and 319 nonsorority women) suggested that both scales achieved scalar invariance, permitting confident usage for comparisons across Greek memberships and genders. The latent mean analyses revealed significant differences in intentions to negotiate sexual consent and beliefs regarding alcohol-involved sexual assault among the different groups.

MIsgivings about measurement invariance

This paper critically evaluates the conventional insistence on establishing measurement invariance (MI) in cross-cultural psychology. We argue that complex and seemingly arbitrary benchmarks for assessing MI can be unrealistic and effectively prohibit meaningful research. The widespread use of various MI criteria creates unnecessary and often unattainable hurdles for cross-cultural researchers who have made the effort to collect data in multiple cultural contexts. Additionally, the prohibitionist tone of discussions surrounding MI is unhelpful, unscientific, and discouraging. We argue that emerging findings that cultural differences might not be as widespread or profound as once assumed imply that significant cross-cultural differences in measurement should not be the default assumption. Additionally, we advocate a shift towards external validity as a more useful metric of measurement quality. Our overall message is that researchers who go to the considerable trouble of gathering data in more than one country should not be disadvantaged compared to researchers who avoid cross-cultural complications by gathering data only at their home campus.

Psychometric properties of an Arabic translation of the long (12 items) and short (7 items) forms of the Violent Ideations Scale (VIS) in a non-clinical sample of adolescents

BackgroundAccurately measuring violent ideations would be of particular high relevance in Arab countries, which are witnessing an unprecedented increase in violence rates among adolescents because of the widespread social, economic and political unrest in the region. Therefore, the present study aimed to test the psychometric properties of an Arabic translation of the 12-item and the 7-item forms of the Violent Ideations Scale (VIS and VIS-SF) in a sample on non-clinical Arabic-speaking adolescents.MethodsFive hundred seventy-seven community adolescents (mean age of 15.90 ± 1.73 years, 56.5% females) answered an anonymous online survey comprising an Arabic translation of the Violent Ideations Scale (VIS) and a measure of physical aggression.ResultsConfirmatory factor analyses (CFA) offered support for the single-factor structure of the Arabic VIS and the VIS-SF. Both the long and short forms of the scale yielded excellent internal consistency, with McDonald’s ω coefficients of 0.96 and 0.94 and Cronbach’s α coefficients of 0.96 and 0.94, respectively. Multi-group CFA established measurement invariance across gender groups. Finally, results revealed significant and positive correlations between the two forms of the VIS and physical aggression scores, thus supporting concurrent validity.ConclusionBoth the VIS and VIS-SF have demonstrated good psychometric properties in their Arabic versions, and suitability for sound assessment of violent ideations. We therefore expect that these measures assist clinicians in risk assessment and management of violence, and help foster research in this area in Arab countries.

Supplemental Material for Age-Related Differences in Trait Affect: Establishing Measurement Invariance of the Positive and Negative Affect Schedule (PANAS)

Supplemental Material for Age-Related Differences in Trait Affect: Establishing Measurement Invariance of the Positive and Negative Affect Schedule (PANAS)

Measurement Invariance of the Hospice Quality of Life Index-14 in Lung Cancer and Nonlung Cancer Patients Admitted to Hospice.

Background and Purpose: Establishing measurement invariance (MI) is important in the questionnaire validation process. This study examined the MI of the Hospice Quality of Life Index-14 (HQLI-14) when comparing hospice patients with lung cancer and those with non-lung cancers. Methods: The HQLI-14 contains 14 items to measure multidimensional concepts of quality of life. A series of confirmatory factor analyses were performed to test configural, metric, and scalar invariance. Results: The MI of the HQLI-14 was supported by increasing equality constraints on item parameters between groups. Although the configural and metric invariances were both supported, one item regarding breathlessness was noninvariant between the groups with lung and nonlung cancers. Conclusions: The HQLI-14 shows early evidence of meeting the requirements for configural, metric, and partial scalar invariance. It may be used to make meaningful comparisons between patients with lung cancer and nonlung cancers.

Volunteering in three cohorts of Czech adolescents: the socialization roles of family, school, and associations

This study aims to identify the proximal socialization processes that are consistently related to adolescents’ volunteering across changing macrosocial conditions. We focused on three groups of Czech adolescents: one living in an early post-communist society (1995) and two living in developed democratic conditions (2010 and 2019). Using structural equation modeling, we analyzed survey-based data from 14-year to 17-year-olds (Ns = 833, 893, and 2,836, respectively). Identical scales with established measurement invariance were used. The results showed that volunteering in all three groups was consistently predicted by prosocial parenting and adolescents’ associational involvement. Another consistent predictor was democratic school climate, although its effect gradually declined across the cohorts. Contrary to our expectations, we found no consistent effects of family cohesion, parental education, or family economic well-being. Overall, adolescents’ volunteering seems to be stimulated by environments promoting prosocial and democratic values and providing sufficient opportunities to become involved with different associations.

Measuring dissociation across adolescence and adulthood: developing the short-form Černis Felt Sense of Anomaly scale (ČEFSA-14).

Dissociation may be important across many mental health disorders, but has been variously conceptualised and measured. We introduced a conceptualisation of a common type of dissociative experience, 'felt sense of anomaly' (FSA), and developed a corresponding measure, the Černis Felt Sense of Anomaly (ČEFSA) scale. We aimed to develop a short-form version of the ČEFSA that is valid for adolescent and adult respondents. Data were collected from 1031 adult NHS patients with psychosis and 932 adult and 1233 adolescent non-clinical online survey respondents. Local structural equation modelling (LSEM) was used to establish measurement invariance of items across the age range. Ant colony optimisation (ACO) was used to produce a 14-item short-form measure. Finally, the expected test score function derived from item response theory modelling guided the establishment of interpretive scoring ranges. LSEM indicated 25 items of the original 35-item ČEFSA were age invariant. They were also invariant across gender and clinical status. ACO of these items produced a 14-item short-form (ČEFSA-14) with excellent psychometric properties (CFI=0.992; TLI=0.987; RMSEA=0.034; SRMR=0.017; Cronbach's alpha=0.92). Score ranges were established based on the expected test scores at approximately 0.7, 1.25 and 2.0 theta (equivalent to standard deviations above the mean). Scores of 29 and above may indicate elevated levels of FSA-dissociation. The ČEFSA-14 is a psychometrically valid measure of FSA-dissociation for adolescents and adults. It can be used with clinical and non-clinical respondents. It could be used by clinicians as an initial tool to explore dissociation with their clients.

18 Measurement Invariance of ImPACT in Bilingual and Monolingual High School Athletes

Objective:Previous studies have found differences between monolingual and bilingual athletes on ImPACT, the most widely used sport-related concussion (SRC) assessment measure. Most recently, results suggest that monolingual English-Speaking athletes outperformed bilingual English- and Spanish-speaking athletes on Visual Motor Speed and Reaction Time composites. Before further investigation of these differences can occur, measurement invariance of ImPACT must be established to ensure that differences are not attributable to measurement error. The current study aimed to 1) replicate a recently identified four-factor model using cognitive subtest scores of ImPACT on baseline assessments in monolingual English-Speaking athletes and bilingual English- and Spanish-speaking athletes and 2) to establish measurement invariance across groups.Participants and Methods:Participants included high school athletes who were administered the ImPACT as part of their standard pre-season athletic training protocol in English. Participants were excluded if they had a self-reported history of concussion, Autism, ADHD, learning disability or treatment history of epilepsy/seizures, brain surgery, meningitis, psychiatric disorders, or substance/alcohol use. The final sample included 7,948 monolingual English-speaking athletes and 7,938 bilingual English- and Spanish-speaking athletes with valid baseline assessments. Language variables were based on self-report. As the number of monolingual athletes was substantially larger than the number of bilingual athletes, monolingual athletes were randomly selected from a larger sample to match the bilingual athletes on age, sex, and sport. Confirmatory factor analysis (CFA) was used to test competing models, including one-factor, two-factor, and three-factor models to determine if a recently identified four-factor model (Visual Memory, Visual Reaction Time, Verbal Memory, Working Memory) provided the best fit of the data. Eighteen subtest scores from ImPACT were used in the CFAs. Through increasingly restrictive multigroup CFAs (MGCFA), configural, metric, scalar, and residual levels of invariance were assessed by language group.Results:CFA indicated that the four-factor model provided the best fit in the monolingual and bilingual samples compared to competing models. However, some goodness-of-fit-statistics were below recommended cutoffs, and thus, post-hoc model modifications were made on a theoretical basis and by examination of modification indices. The modified four-factor model had adequate to superior fit and met criteria for all goodness-of-fit indices and was retained as the configural model to test measurement invariance across language groups. MGCFA revealed that residual invariance, the strictest level of invariance, was achieved across groups.Conclusions:This study provides support for a modified four-factor model as estimating the latent structure of ImPACT cognitive scores in monolingual English-speaking and bilingual English- and Spanish-speaking high school athletes at baseline assessment. Results further suggest that differences between monolingual English-speaking and bilingual English- and Spanish-speaking athletes reported in prior ImPACT studies are not caused by measurement error. The reason for these differences remains unclear but are consistent with other studies suggesting monolingual advantages. Given the increase in bilingual individuals in the United States, and among high school athletics, future research should investigate other sources of error such as item bias and predictive validity to further understand if group differences reflect real differences between these athletes.

An Intersectional Examination of Economic Hardship and Individualized Education Program Meeting Participation

In this preregistered study, we confirmed a transition-planning construct and determined its relationship with an established economic-hardship construct using parent- and youth-reported data from the National Longitudinal Transition Study 2012 and following established quality indicators of preregistered secondary data analyses studies. Transition planning was made up of seven items relevant to individualized education program (IEP) meeting participation. We examined measurement invariance on the basis of disability category and race/ethnicity as well as the intersection of these student characteristics in order to make unbiased group comparisons on key latent parameters. Contrary to our hypotheses, there was no meaningful correlation between transition planning and economic hardship. Moreover, we were unable to establish measurement invariance of transition planning because the item “youth's role in IEP meeting” functioned differently on the basis of disability category. Implications for practice are discussed regarding how decisions about IEP meeting participation are made, by whom and why, and whether evidence-based practices are effectively disseminated. Future research should ensure adequate sampling plans for national survey attempts in order to conduct an in-depth examination based on intersectional student characteristics.

A Bayesian region of measurement equivalence (ROME) approach for establishing measurement invariance.

Measurement invariance research has focused on identifying biases in test indicators measuring a latent trait across two or more groups. However, relatively little attention has been devoted to the practical implications of noninvariance. An important question is whether noninvariance in indicators or items results in differences in observed composite scores across groups. The current study introduces the Bayesian region of measurement equivalence (ROME) as a framework for visualizing and testing the combined impact of partial invariance on the group difference in observed scores. Under the proposed framework, researchers first compute the highest posterior density intervals (HPDIs)-which contain the most plausible values-for the expected group difference in observed test scores over a range of latent trait levels. By comparing the HPDIs with a predetermined range of values that is practically equivalent to zero (i.e., region of measurement equivalence), researchers can determine whether a test instrument is practically invariant. The proposed ROME method can be used for both continuous indicators and ordinal items. We illustrated ROME using five items measuring mathematics-specific self-efficacy from a nationally representative sample of 10th graders. Whereas conventional invariance testing identifies a partial strict invariance model across gender, the statistically significant noninvariant items were found to have a negligible impact on the comparison of the observed scores. This empirical example demonstrates the utility of the ROME method for assessing practical significance when statistically significant item noninvariance is found. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Alternatives to Weighted Item Fit Statistics for Establishing Measurement Invariance in Many Groups

Evaluating scale comparability in international large-scale assessments depends on measurement invariance (MI). The root mean square deviation (RMSD) is a standard method for establishing MI in several programs, such as the Programme for International Student Assessment and the Programme for the International Assessment of Adult Competencies. Previous research showed that the RMSD was unable to detect departures from MI when the latent trait distribution was far from item difficulty. In this study, we developed three alternative approaches to the original RMSD: equal, item information, and b-norm weighted RMSDs. Specifically, we considered the item-centered normalized weight distributions to compute the item characteristic curve difference in the RMSD procedure more efficiently. We further compared all methods’ performance via a simulation study and the item information and b-norm weighted RMSDs showed the most promising results. An empirical example is demonstrated, and implications for researchers are discussed.