Evidence Of Measurement Invariance Research Articles

Evidence of Internal Structure Validity, Measurement Invariance, Convergent Validity, Clinical Validity, and Reliability of the Woman Abuse Screening Tool in Peruvian Women.

Intimate partner violence (IPV) is the most common type of violence committed against women and results in serious personal, familial, social, and economic consequences; thus, there is a need to detect IPV early. One test for detecting IPV is the Woman Abuse Screening Tool (WAST). The objective of the study was to obtain evidence of the validity of the WAST in terms of its internal structure, measurement invariance, convergent validity, clinical validity, and reliability. A total of 670 women who have or have had a heterosexual relationship participated in the study. The first stage included 513 adult women who partook of health services (M age = 35.5 years, SD age = 10 years). The second stage included 160 women, 80 diagnosed with IPV (M age = 34.2 years, SD age = 11.8 years) and 80 who were not (M age = 31.8 years, SD age = 11.5 years). We used multivariate and receiver operating characteristic curve analyses. We found evidence that the WAST demonstrates a one-dimensional structure; evidence of measurement invariance regarding cohabitation with the partner, length of the relationship, and age; evidence of convergent validity in terms of correlations with emotional dependence, self-compassion, expressive suppression, anxiety, and depression; and evidence of clinical validity in terms of the high probability of detecting positive cases of IPV. These properties support the use of the WAST for detecting possible cases of IPV, which will allow timely intervention. This instrument can also be used in larger studies on IPV in the Spanish-speaking population.

Chronic and contextual identity salience: Assessing dual-dimensional salience with the Identity Salience Questionnaire (ISQ)

ABSTRACT Identity Salience is a common construct within social identity research. However, researchers note that it is poorly defined and inconsistently operationalized. We posit that identity salience comprises two elements: chronic (perpetually thinking about the identity) and contextual (only thinking about the identity when prompted) salience. We present evidence for this claim through the development and validation of the dual-dimensional Identity Salience Questionnaire (ISQ). Studies 1-2 (Ns=414; 1,069) provide exploratory and confirmatory factor analytic evidence of the ISQ among LGBTIQA+ participants. Study 2 also provides evidence of measurement invariance, convergent, predictive, and discriminant validity, and internal reliability. Study 3 (N=318) indicates strong test-retest reliability. Study 4 (N=107 social psychologists) confirms the ISQ’s content validity. Future research for the ISQ is discussed.

Item Response Theory Analysis and Measurement Invariance Testing of the Cultural Humility and Enactment Scale

Objective This study aimed to validate the Cultural Humility and Enactment Scale (CHES) through (a) examining its factor structure with multiple samples; (b) employing item response theory (IRT) analysis to examine its item-level characteristics; (c) reducing potential redundancies among items; and (d) conducting measurement invariance (MI) testing. Method The sample included a total of 610 individuals with diverse demographic backgrounds recruited from Mturk. Analyses included exploratory factor analysis (EFA), IRT analysis, confirmatory factor analysis (CFA), and multi-group CFA. Results We replicated the three-factor structure and retained the 21-item CHES (CHES-21), which showed equivalent information coverage and superior model fit compared to the original 29-item version. We also obtained evidence for MI of the CHES-21 between White and nonwhite clients and between male and non-male clients. Conclusion Evidencing reliability and validity with a sample of adult clients, the CHES-21 can be effectively incorporated into counseling practice and counselor training.

Has human progress stagnated in recent decades? Evaluating Seligman’s (2021) model of agency and its correlates using the MIDUS three-wave longitudinal study

ABSTRACT Grounded in Seligman’s tripartite model of agency, the authors developed the 10-item Agency Scale. Using data from three waves (1994–1996, 2004–2006, 2013–2014) of the MIDUS national longitudinal study, results from midlife adults (N = 2,717) supported a three-factor structure, corresponding to self-efficacy, future-minded planning (as a measure of optimism), and imagination. Evidence for measurement invariance, construct validity, and reliability was provided. The three subscales of the Agency Scale uniquely and positively predicted effort, persistence, generativity (as an indicator of innovation), and perceived progress. Supporting Seligman’s hypothesis, sequential mediation analyses revealed that effort, persistence, and generativity significantly mediated the self-efficacy-perceived progress, future-minded planning-perceived progress, and imagination-perceived progress relations, respectively. Significant mean differences across three timepoints indicated an overall declining trend in scores of agency (including self-efficacy, future-minded planning, and imagination) and perceived progress. Collectively, these findings provide support for Seligman’s tripartite model of agency.

Using Measurement Invariance to Explore the Source of Variation in Basic Medical Science Students’ Evaluation of Teaching Effectiveness

Introduction: Many research studies have shown that students' evaluations of teaching(SET) are affected by different variables without testing the requirement of fair comparisons. These studies have not tested the measurement equivalency of SET surveys according to these variables. Measurement equivalency of SET refers to whether a SET survey is interpreted similarly across different groups of individuals (Variable Levels). Without evidence of measurement invariance across different variables under investigation, the SET ratings should not be compared across these variables and this is the goal of this study.
 Methods: Measurement Invariance analysis of SET survey was investigated using 1649 responses to SET of four different medical core courses offered by the College of Science and College of Medicine and from different levels.
 Results: The results showed the existence of teaching practices in the SET survey that are not equivalently loaded on its factor across the levels of targeted variables, and the college offered medical courses were a source of variation in basic medical science students’ evaluation of teaching effectiveness. On the other hand, teaching practices in the SET survey are equivalently loaded on its factor across course levels.
 Discussion: The study results showed that the SET of medical courses is comparable to the courses only taught by the College of Medicine. These results provide evidence that medical courses are different from other courses offered by other colleges. This means that comparing SET of the College of Medicine with other colleges and colleges of medicine needs to compare SET results at the college level only.

Psychometric properties of the Career Adapt-Abilities Scale in Italian young people not in education, employment, or training (NEET)

AbstractThe Career Adapt-Abilities Scale (CAAS) is one of the most widely used measures to assess career adaptabilities. Despite intensive worldwide use across heterogeneous populations, no study has assessed its factorial validity in the not in education, employment, or training (NEET) population. This study aims to extend evidence of CAAS factorial validity by assessing the Italian version for NEET. Participants (N = 318) completed the Italian CAAS. Confirmatory factor analyses confirmed the hierarchical factor structure of this version. Evidence of measurement invariance across age and sex groups were observed. The results supported the CAAS as a reliable and good psychometrically-based assessment of career adaptabilities in the Italian NEET population.

Testing for intersectional measurement invariance with the alignment method: Evaluation of the 8-item patient health questionnaire.

To demonstrate the use of the alignment method to evaluate whether surveys function similarly (i.e., have evidence of measurement invariance) across culturally diverse intersectional groups. Intersectionality theory recognizes the interconnected nature of social categories such as race, gender, ethnicity, and socioeconomic status. A total of 30,215 American adult's responses to the eight-item Patient Health Questionnaire depression assessment scale (PHQ-8) from the 2019 National Health Interview Survey (NHIS). Using the alignment method, we examined the measurement invariance (equivalence) of the PHQ-8 depression assessment scale across 16 intersectional subgroups defined at the intersection of age (under 52, 52 and older), gender (male, female), race (Black, non-Black), and education (no bachelor's degree, bachelor's degree). Overall, 24% of the factor loadings and 5% of the item intercepts showed evidence of differential functioning across one or more of the intersectional groups. These levels fall beneath the benchmark of 25% suggested for determining measurement invariance with the alignment method. The results of the alignment study suggest that the PHQ-8 functions similarly across the intersectional groups examined, despite some evidence of different factor loadings and item intercepts in some groups (i.e., noninvariance). By examining measurement invariance through an intersectional lens, researchers can investigate how a person's multiple identities and social positions possibly contribute to their response behavior on an assessment scale.

Measurement invariance of the PHQ-9 and GAD-7 across males and females seeking treatment for common mental health disorders

BackgroundThe nine-item Patient Health Questionnaire (PHQ-9) and the seven-item Generalised Anxiety Disorder scale (GAD-7) are routinely used in research and clinical practice. Whilst measurement invariance of these measures across gender has been demonstrated individually in general population studies and clinical samples, less is known about invariance of the distinct but correlated latent factors (‘depression’ and ‘anxiety’). The current study assessed measurement invariance of these constructs across males and females seeking treatment for common mental health disorders.MethodsData were provided from eight psychological treatment services in London, England. Data from initial assessments with the services where individual items on the PHQ-9 and GAD-7 were available were included in analyses. Measurement invariance was explored across self-identified genders, with ‘male’ and ‘female’ categories available in the dataset. Sensitivity analyses were conducted using propensity score matching on sociodemographic and clinical variables.ResultsData were available for 165,872 patients (110,833 females, 55,039 males). There was evidence of measurement invariance between males and females in both the full sample and a propensity score matched sample (n = 46,249 in each group).ConclusionsMeasurement invariance of the correlated depression and anxiety factors of PHQ-9 and GAD-7 were indicated in this sample of individuals seeking psychological treatment for CMHDs. These results support the use of these measures in routine clinical practice for both males and females. This is of particular importance for assessing the prevalence of clinically significant levels of symptoms as well as comparing treatment outcomes across genders.

Examining Measurement Invariance in Bayesian Item Response Theory Models: A Simulation Study

The aim of the study is to determine a measurement invariance cut-off point based on item parameter differences in Bayesian Item Response Theory Models. Within this scope, the Bayes factor is estimated for testing measurement invariance. For this purpose, a simulation study is conducted. The data were generated in the R software for each simulation condition under the one-parameter logistic model for 10 binary (1-0 scored) items. The invariance test was performed for various group sizes (n=500, 1000, 1500 and 2000) and difficulty parameters (dk=0, dk=0.1, dk=0.3, dk=0.5 and dk=0.7). The Bayesian analyzes were performed on the WINBUGS using the codes written in the R. A Bayes factor that provides evidence for measurement invariance was calculated depending on the parameter differences. The Savage–Dickey density ratio, one of the MCMC sampling schemas, was used to calculate the Bayes factor. As a result, if the item parameter difference is dk=0.3 and group sizes are 1500 or larger, the measurement invariance cannot be achieved. However, for small sample sizes (n=1000 or less) measurement invariance interpretation should be done carefully. When the dk=0.5, there are invariant items only in n=500. According to Bayes factor test results, evidence has been produced when dk=0.7 that measurement invariance cannot be achieved.

Measurement invariance of six language versions of the post-traumatic stress disorder checklist for DSM-5 in civilians after traumatic brain injury

Traumatic brain injury (TBI) is frequently associated with neuropsychiatric impairments such as symptoms of post-traumatic stress disorder (PTSD), which can be screened using self-report instruments such as the Post-Traumatic Stress Disorder Checklist for DSM-5 (PCL-5). The current study aims to inspect the factorial validity and cross-linguistic equivalence of the PCL-5 in individuals after TBI with differential severity. Data for six language groups (n ≥ 200; Dutch, English, Finnish, Italian, Norwegian, Spanish) were extracted from the CENTER-TBI study database. Factorial validity of PTSD was evaluated using confirmatory factor analyses (CFA), and compared between four concurrent structural models. A multi-group CFA approach was utilized to investigate the measurement invariance (MI) of the PCL-5 across languages. All structural models showed satisfactory goodness-of-fit with small between-model variation. The original DSM-5 model for PTSD provided solid evidence of MI across the language groups. The current study underlines the validity of the clinical DSM-5 conceptualization of PTSD and demonstrates the comparability of PCL-5 symptom scores between language versions in individuals after TBI. Future studies should apply MI methods to other sociodemographic (e.g., age, gender) and injury-related (e.g., TBI severity) characteristics to improve the monitoring and clinical care of individuals suffering from PTSD symptoms after TBI.

Evaluating Gender Differences in Problematic Smartphone Use

Abstract: The Smartphone Addiction Inventory (SPAI) is widely used to measure problematic smartphone use (PSU). Although the SPAI has been translated and validated in different countries, its measurement invariance across gender has received little research attention. This study aimed to examine whether men and women interpreted the Italian version of the SPAI (SPAI-I) similarly and, consequently, whether the observed gender differences in SPAI scores, which have been shown in previous studies, could be due to true differences, rather than to differences in measurement. Six hundred nineteen Italian young adults ( Mage = 22.02 ± 2.63; 55.7% women) took part in the study and completed the SPAI-I. Multigroup CFA was applied to test the measurement invariance across gender, and the item parameter invariance was investigated with the item-response theory (IRT) differential item functioning (DIF) method for multidimensional models. Evidence of measurement invariance across gender was found. Only one item (i.e., item 14, “The idea of using smartphone comes as the first thought on mind when waking up each morning”) out of 24 items of the SPAI-I showed DIF with a large effect size. Gender-related differences found with the SPAI-I reflect true differences in smartphone overuse rather than specific characteristics of the measure.

Psychometric Investigation of the Workplace Social Courage Scale (WSCS): New Evidence for Measurement Invariance and IRT Analysis.

This study, after presenting a review of the existent literature on courage and social courage in the workplace, has the purpose of providing new evidence about the psychometric properties of an Italian-language version of the Workplace Social Courage Scale (WSCS), verifying its measurement invariance across gender and the discrimination properties of its items through IRT analysis. The aim of the research is testing the Italian version of the WSCS; for this scope, four studies have been conducted on four different samples analyzing the factorial structure, the internal consistency, the measurement invariance across gender, and the convergent and concurrent validity. The results support the psychometric properties in terms of factor structure, reliability, validity, and utility, showing positive relationships with the criterion variables: satisfaction of work-related basic needs, prosocial rule breaking and work performance. The current study extends prior findings by providing further insights about the construct of courage and social courage in the workplace, especially in the Italian context. As, to date, little is known about the impact of social courage on work and organizational outcomes, the availability of a reliable, valid, and cross-culturally supported instrument can promote the role of this construct in positive organizational behavior research.

Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S): Evidence of Measurement Invariance Across Five Countries.

PurposeThe percentage of individuals who were fully vaccinated against COVID-19 was 53% worldwide, 62% in Asia, and 11% in Africa at the time of writing (February 9, 2022). In addition to administrative issues, vaccine hesitancy is an important factor contributing to the relatively low rate of vaccination. The Motors of COVID-19 Vaccination Acceptance Scale (MoVac-COVID19S) was developed to assess COVID-19 vaccination acceptance levels. However, it has only been tested among Taiwanese, mainland Chinese, and Ghanaian populations (Chen et al, 2021; Fan et al, 2021; Yeh et al, 2021). Therefore, the present study examined the construct validity and measurement invariance of the MoVac-COVID19S among individuals from five countries (ie, Taiwan, mainland China, India, Ghana, and Afghanistan).Participants and MethodsA cross-sectional survey study recruited 6053 participants across five countries who completed the survey between January and March 2021. Confirmatory factor analysis (CFA) fit indices were used to examine factor structure and measurement invariance across the five countries.ResultsThe fit indices of the CFA were relatively good across the countries except for the root mean square error of approximation (RMSEA). Moreover, the four-factor structure (either nine or 12 items) had a better fit than the one-factor structure. However, the four-factor model using nine MoVac-COVID19S items was the only model that had measurement invariance support for both factor loadings and item intercepts across the five countries.ConclusionThe present study confirmed that the MoVac-COVID19S has acceptable psychometric properties and can be used to assess an individual’s willingness to get COVID-19 vaccination.

The Unified Narcissism Scale: Moving towards an integrated measure of narcissism

Narcissism as a psychological construct has had a contentious past both in its conceptualization and measurement. There is an emerging consensus that narcissism consists of grandiose and vulnerable subtypes, which share a common core. In the present research (N = 1002), we constructed a new measure of unified narcissism that reflects these contemporary understandings using items from the most widely used measures of grandiose and vulnerable narcissism: the Narcissistic Personality Inventory (NPI; Raskin & Terry, 1988, https://doi.org/10.1037/0022-3514.54.5.890), and the Pathological Narcissism Inventory (PNI; Pincus et al., 2009, https://doi-org/10.1037/a0016530). We used classical test theory and item response theory approaches to devise a 29-item Unified Narcissism Scale. The scale showed good internal consistency, and convergent and discriminant validity, and showed evidence of measurement invariance between men and women. This research gave strong support for the structure, reliability, and validity of the unified measure, which offers a promising avenue for further enhancing our knowledge of narcissism.

COVID-19 Bereavement in Ten Latin American Countries: Measurement Invariance of the Pandemic Grief Scale and Its Relation to Suicidal Ideation.

The present study aimed to evaluate the cross-cultural measurement invariance of the Pandemic Grief Scale (PGS) in ten Latin American countries. A total of 2,321 people who had lost a family member or other loved one due to COVID-19 participated, with a mean age of 34.22 years old (SD = 11.99). In addition to the PGS, a single item of suicidal ideation was applied. The unidimensional model of the PGS had adequate fit in most countries and good reliability estimates. There was evidence of measurement invariance by country and gender. Also, a one-point increase in the PGS was associated with an almost twofold increase in the odds of suicidal ideation. Scores greater than or equal to 4 on the PGS are proposed as a cut off to identify individuals with suicidal ideation. Strong evidence of the cross-cultural validity of the PGS is provided.

Invariance of the Household Food Insecurity Access Scale Across Different Groups of Adolescents and Young Adults.

Background:Cross-group comparisons of household food insecurity and its associations using multiple-item scales assume that scale scores can be interpreted as identical across groups. However, scores should not be interpreted as identical across groups without evidence of measurement invariance. Noninvariant measures indicate that the underlying construct may be different across groups.Objective:To determine whether the Household Food Insecurity Access Scale (HFIAS) is invariant across different groups of Ghanaian and South African youth aged 15 to 24.Methods:We analyzed cross-sectional quantitative data from 1437 and 4165 young South Africans and Ghanaians, respectively. Multi-group confirmatory factor analysis was used to examine whether the HFIAS was invariant across different groups of youth, including sex (male or female), age group (middle adolescence, late adolescence, or emerging adulthood), and receipt of child support grant (yes or no). We assessed 3 levels of invariance: configural, metric, and scalar. The model fit between nested models was compared using χ2 difference testing.Results:Invariance tests indicated that the HFIAS had configural, metric, and scalar invariance across different groups of Ghanaian and South African youth. Model fit statistics across all invariance levels indicated good fit of our hypothesized model with the observed data. χ2 difference testing results were not statistically significant across all nested models.Conclusions:Food insecurity, as measured by the HFIAS, meant the same thing for different groups of Ghanaian and South African youth. Evidence of invariance means that the HFIAS scores could be interpreted as identical across youth groups in our study.

Evidence of measurement invariance across gender for the Force Concept Inventory

Gender differences in performance on the Force Concept Inventory are likely not a result of different, gender-specific factor structures of the assessment.

Evidence for Measurement Invariance and Psychometric Reliability for Scores on the PHQ-4 From a Rural and Predominately Hispanic Community

The factor structure, measurement invariance, and internal consistency of the Patient Health Questionnaire for Depression and Anxiety (PHQ-4) was examined with a rural, predominately Hispanic sample (N = 711). Findings supported use of a one-factor model across gender, age groups, and Spanish-speaking groups. Counseling practice and research implications are discussed.

Investigating the validity of the Wong and Law Emotional Intelligence Scale in a Nepali student sample

This paper investigates the cross‐cultural validity of the Wong and Law Emotional Intelligence Scale. Two samples of university students were recruited: 504 from a Nepali university and 260 from a UK university. In relation to culture, structural equation modelling analyses provided support for the scale's configural invariance and the configural, metric, and scalar invariance of two if its subscales. Evidence for measurement invariance was also found in relation to gender in both samples. Tentative analyses suggested that the correlation between self and other emotion appraisal was stronger among UK participants and that UK participants scored higher on the Other Emotion Appraisal subscale. No gender differences on emotional intelligence were found in the Nepali sample, while among UK students, males scored higher on Regulation of Emotion and lower on Other Emotion Appraisal than females. In the Nepali sample, science students scored lower on various aspects of emotional intelligence than humanities students.

The multidimensional school climate questionnaire (MSCQ) parent-version: Factorial structure and measurement invariance

ABSTRACT The current study aimed to test the factorial structure and psychometric properties of a parent version of the Multidimensional School Climate Questionnaire (MSCQ), a multi-informant and multidimensional measure of school climate recently developed and validated in Italian for a student version, by also providing evidence of measurement invariance between students and parents. Participants were 320 parents, mostly mothers, and 339 students enrolled in four middle schools in Northern Italy. The confirmatory factor analysis (CFA) showed a good fit for the expected six-dimension model (Student Support, Home-school Relations, Student Relations, Student-Teacher Relations, Educational Climate, Justice). All factors were positively correlated with each other and reported good Cronbach’s alphas and composite reliability scores (ω). Full configural and metric and partial scalar invariance were achieved between parents and students. These findings confirm that the parent version of the MSCQ is a psychometrically sound measure to assess multidimensional perceptions of school climate. Lastly, limitations and practical implications are discussed.