Assess Measurement Invariance Research Articles

How do narcissists perceive personality items? Measurement invariance of a Big Five scale across low and high narcissism groups

Abstract Two forms of narcissism were found to be associated with various personality traits. Grandiose narcissism typically correlates with high levels of extraversion, openness and emotional stability, whereas vulnerable narcissism correlates positively with neuroticism and negatively with extraversion. Both types of narcissism are associated with disagreeableness. Studies revealed that narcissists, at least grandiose ones, exhibit distorted self-views. Thus, a question about potential response bias of narcissistic individuals on self-report measures seems relevant. The current study assessed measurement invariance in the five-factor model scale across groups with low and high narcissism, separately for vulnerable and grandiose type. From a larger sample (n = 755) extreme groups were selected basing on the respective narcissism scale. The results indicated that the Big Five questionnaire was measurement invariant across narcissism groups with respect to configural and metric equivalence; however, it showed a lack of scalar (intercept) invariance. Individuals scoring high on grandiose narcissism exhibited higher thresholds on extraversion, emotional stability, conscientiousness and intellect in comparison to people with a low level of grandiose narcissism. Individuals with a high level of vulnerable narcissism obtained lower intercepts on all scales, besides intellect, in comparison to individuals low in vulnerable narcissism.

Simplifying the Assessment of Measurement Invariance over Multiple Background Variables: Using Regularized Moderated Nonlinear Factor Analysis to Detect Differential Item Functioning

Determining whether measures are equally valid for all individuals is a core component of psychometric analysis. Traditionally, the evaluation of measurement invariance (MI) involves comparing independent groups defined by a single categorical covariate (e.g., men and women) to determine if there are any items that display differential item functioning (DIF). More recently, Moderated Nonlinear Factor Analysis (MNLFA) has been advanced as an approach for evaluating MI/DIF simultaneously over multiple background variables, categorical and continuous. Unfortunately, conventional procedures for detecting DIF do not scale well to the more complex MNLFA. The current manuscript therefore proposes a regularization approach to MNLFA estimation that penalizes the likelihood for DIF parameters (i.e., rewarding sparse DIF). This procedure avoids the pitfalls of sequential inference tests, is automated for end users, and is shown to perform well in both a small-scale simulation and an empirical validation study.

The emotion regulation questionnaire: Factor structure and measurement invariance in an Italian sample of community dwelling adults

The Emotion Regulation Questionnaire (ERQ) is a widely-used measure to assess how often individuals use two regulatory strategies, namely cognitive reappraisal and expressive suppression, in everyday life. Recent research, however, has shown inconsistencies when testing the factorial structure of the ERQ in community samples compared to validation studies conducted on samples of undergraduate students. These inconsistences raise issues about the use of the ERQ to measure emotion regulation strategies in non-student samples. The current study tested the factor structure of the ERQ in an Italian community sample (N = 415) and in a sample of Italian undergraduate students (N = 371), assessing measurement invariance across the two groups. The results showed poor model fit for the original factor structure of the ERQ in the community sample and barely adequate fit in the student sample. However, after dropping two items, good model fit was obtained for both student and community samples. The eight-item ERQ was then found to be equivalent across gender and age. Findings from latent mean comparison showed that females reported less frequent use of suppression and more frequent use of reappraisal than males. Finally, the reappraisal and suppression scales showed significant associations with several domains of individuals’ well-being. Although further validity studies are required, these findings suggest that a shortened, 8-item version of the ERQ may be a valid instrument with psychometric properties comparable to the original version of the ERQ.

A Note on Using Weighted Sum Scores in the P‐DIF Statistic

AbstractThe Mantel–Haenszel delta difference (MH D‐DIF) and the standardized proportion difference (STD P‐DIF) are two observed‐score methods that have been used to assess differential item functioning (DIF) at Educational Testing Service since the early 1990s. Latent‐variable approaches to assessing measurement invariance at the item level have been proposed and studied since then as well. Previous research showed that using the weighted sum score as the matching variable may close the gap between the MH D‐DIF statistic based on observed scores and its counterpart based on latent ability. In this study, we show that weighting reduces the difference between the STD P‐DIF statistic and its counterpart based on latent ability. In addition, we discuss the factors that influence the gap and examine approaches that may facilitate the use of weighted STD P‐DIF.

Cultural differences in positive psychotic experiences assessed with the Community Assessment of Psychic Experiences-42 (CAPE-42): a comparison of student populations in the Netherlands, Nigeria and Norway

BackgroundPrevious studies have suggested that culture impacts the experience of psychosis. The current study set out to extend these findings by examining cultural variation in subclinical positive psychotic experiences in students from The Netherlands, Nigeria, and Norway. Positive psychotic experiences were hypothesized to (i) be more frequently endorsed by, and (ii) cause less distress in Nigerian vs. Dutch and Norwegian students.MethodsPsychology students, aged 18 to 30 years, from universities in the Netherlands (n = 245), Nigeria (n = 478), and Norway (n = 162) were assessed cross-sectionally with regard to the frequency of subclinical positive psychotic experiences and related distress, using the Community Assessment of Psychic Experiences (CAPE-42). Multi-group confirmatory factor analysis and multivariate analysis of covariance were performed to assess measurement invariance of the positive symptom dimension (CAPE-Pos) and compare mean frequency and associated distress of positive psychotic experiences across study samples.ResultsOnly CAPE-Pos items pertaining to the dimensions ‘strange experiences’ and ‘paranoia’ met assumptions for (partial) measurement invariance. Frequencies of these experiences were higher in the Nigerian sample, compared to both the Dutch and Norwegian samples, which were similar. In addition, levels of experience-related distress were similar or higher in the Nigerian sample compared to respectively the Dutch and Norwegian samples.ConclusionAlthough positive psychotic experiences may be more commonly endorsed in non-Western societies, our findings do not support the notion that they represent a more benign, and hence less distressing aspect of human experience. Rather, the experience of psychotic phenomena may be just as, if not more, distressing in African than in European culture. However, observed differences in CAPE-Pos frequency and distress between samples from different cultural settings may partly reflect differences in the measure rather than in the latent trait. Future studies may therefore consider further cross-cultural adaptation of CAPE-42, in addition to explicitly examining cultural acceptance of psychotic phenomena, and environmental and other known risk factors for psychosis, when comparing and interpreting subclinical psychotic phenomena across cultural groups.

Does the Spanish version of the SWLS measure the same in Spain and Peru?

Satisfaction with life is a measure of protection in older adults. There lies the importance of providing quality instruments. The aim of the study was to evaluate the invariance of the life satisfaction scale (SWLS) in two samples of older adults in Spain and Peru. The participants were 857 older adults in Spain (mean age=68.23 years, SD=5.93) and 336 older adults in Peru (average age=72.42, SD=7.07). All multi-group confirmatory factor analyzes were estimated in Mplus 8.0. The results indicate the presence of a strict invariance of the one-dimensional structure of the SWLS in samples of older adults in Spain and Peru, which allows for meaningful comparisons of latent means and covariances. Comparison of latent means showed small differences in the construct between the cultural groups. The SWLS is a valid instrument for intercultural comparisons between Spanish and Peruvian population. The measurement invariance assessment contributes to a better understanding of life satisfaction in populations from different cultural contexts.

Evidence for essential unidimensionality of AUDIT and measurement invariance across gender, age and education. Results from the WIRUS study

Globally, alcohol use is among the most important risk factors related to burden of disease, and commonly emerges among the ten most important factors. Also, alcohol use disorders are major contributors to global burden of disease. Therefore, accurate measurement of alcohol use and alcohol-related problems is important in a public health perspective. The Alcohol Use Identification Test (AUDIT) is a widely used, brief ten-item screening instrument to detect alcohol use disorder. Despite this the factor structure and comparability across different (sub)-populations has yet to be determined. Our aim was to investigate the factor structure of the AUDIT-questionnaire and the viability of specific factors, as well as assessing measurement invariance across gender, age and educational level. We employed data (N = 4,318) from the ongoing screening study in the Norwegian national WIRUS project. We used Confirmatory Factor Analysis (CFA) to establish the factor structure of the AUDIT. Next, we investigated the viability of specific factors in a bi-factor model, and assessed measurement invariance of the preferred factor structure. Our findings indicate the AUDIT is essentially unidimensional, and that comparisons can readily be done across gender, age and educational attainment. We found support for a one-factor structure of AUDIT. To the best of our knowledge, this is the first study to investigate the viability of specific factors in a bi-factor model as well as evaluating measurement invariance across gender, age and educational attainment for the AUDIT questionnaire. Therefore, further studies are needed to replicate our findings related to essential unidimensionality.

Comparable or not? Assessing measurement invariance of a modified Center for Epidemiologic Studies–Depression Scale between sexual minority and nonsexual minority men in five Asia-Pacific countries.

Comparable or not? Assessing measurement invariance of a modified Center for Epidemiologic Studies–Depression Scale between sexual minority and nonsexual minority men in five Asia-Pacific countries.

Invarianza de Medición y Estructural de la Escala Básica de Empatía Breve (BES-B) en Niños y Adolescentes Peruanos

El objetivo del presente estudio fue examinar la invarianza de medición y la invarianza estructural de una medida de empatía, para niños y adolescentes peruanos. Los participantes fueron 417, provenientes de Lima Metropolitana. Se aplicó la Escala Básica de Empatía Breve (BES-B) de manera estandarizada. El análisis consistió en aplicar el modelamiento de ecuaciones estructurales para evaluar la invarianza de medición y las medias latentes en los constructos de empatía afectiva y cognitiva. Los resultados muestran que la invarianza de medición se mantiene satisfactoriamente de acuerdo al género y la edad; las diferencias en las medias latentes muestran ser entre fuertes o moderadas respecto a la edad y el sexo. Se discuten las implicaciones teóricas y prácticas.

Factor Structure and Measurement Invariance Across Gender Groups of the 15-Item Geriatric Depression Scale Among Chinese Elders.

The 15-item Geriatric Depression Scale (GDS-15) is widely used to screen depression among elders. But the factor structure of the Chinese version GDS-15 remains unclear. This study was conducted to determine the best-fit factor structure of GDS-15 and to assess measurement invariance across gender groups in a sample of Chinese elders recruited from Mainland China (final sample N = 2428). The best-fit factor structure was examined by confirmatory factor analysis (CFA). Multigroup CFA was utilized to test the measurement invariance across genders of the factor structure. The results of CFA revealed that a three-factor model, including life satisfaction (four items), general depressive affect (seven items), and withdrawal (three items), fits the structure of the GDS-15 best. Measurement invariance across genders was supported, fully assuming different degrees of invariance.

Measurement invariance across young adults from Hong Kong and Taiwan among three internet-related addiction scales: Bergen Social Media Addiction Scale (BSMAS), Smartphone Application-Based Addiction Scale (SABAS), and Internet Gaming Disorder Scale-Short Form (IGDS-SF9) (Study Part A)

Internet addiction has been found to be prevalent worldwide, including Asian countries, and related to several negative outcomes and other behavioral addictions. The Bergen Social Media Addiction Scale (BSMAS), Smartphone Application-Based Addiction Scale (SABAS), and nine-item Internet Gaming Disorder Scale-Short Form (IGDS-SF9) have been extensively used to assess internet-related addictions. However, the three aforementioned instruments have rarely been used in Asian countries. The aim of the present study was to investigate whether the BSMAS, SABAS, and IGDS-SF9 were appropriate for use in heterogeneous subsamples from Hong Kong and Taiwan. University students from Hong Kong (n = 306) and Taiwan (n = 336) were recruited via an online survey. Multigroup confirmatory factor analysis (MGCFA) was used to assess measurement invariance of the BSMAS, SABAS, and IGDS-SF9 across the two subcultures. The original unidimensional structures of BSMAS, SABAS and IGDS-SF9 were confirmed through confirmatory factorial analysis in both subcultures. The MGCFA results showed that the unidimensional structures of the BSMAS and IGDS-SF9 were invariant across the two Chinese cultural areas (Hong Kong and Taiwan). However, the measurement invariance of the SABAS was established after some model modifications. In conclusion, the present study found that the Chinese BSMAS, SABAS, and IGDS-SF9 were all adequate instruments to validly assess internet-related addictions among university students. The three brief instruments used for assessing addictions to social media, smartphone applications, and online gaming are valid and psychometrically robust across two Chinese subcultures and can be used by healthcare professionals in these regions.

Age-specific norms and validation of the German SDQ parent version based on a nationally representative sample (KiGGS).

The Strengths and Difficulties Questionnaire (SDQ) is the most widely used mental health screening instrument for children and adolescents. It is a short questionnaire including 25 items that can be answered by parents, teachers or children. There are two studies which report norms for the German SDQ parent version. They do not include children younger than 6years. Moreover, whether the German SDQ parent version is measurement invariant across age has not yet been investigated. The absence of measurement invariance across age would support the use of age-specific norms that are not yet available for the German SDQ parent version. We used data of the German Health Interview and Examination Survey for Children and Adolescents (KiGGS), a nationally representative survey including 14,835 children aged 3-17years, to assess measurement invariance of the German SDQ parent version across the full age range. Multi-group confirmatory factor analysis revealed that the hyperactivity and emotional symptoms subscales are not comparable between children of different ages. This supports the use of age-specific norms for these two subscales and for the total SDQ. We used methods of centile estimation to smoothly model the centiles of the SDQ totalscore and the subscale scores in dependence on age. These age-specific centiles reflect the developmental course of SDQ problems in children (including preschoolers) and adolescents living in Germany. They can be used to identify children and adolescents with abnormal behaviour, while accounting for the developmental course of emotional and behaviour problems.

Multigroup Analysis and Measurement Equivalence: Korean And Chinese Consumers of Korean Cosmetics

We examined the concepts and importance of measurement equivalence tests and how failing to conduct these tests can hinder multigroup analyses, particularly for cross-cultural or cross-national studies. To do so, we developed a conceptual model of Korean cosmetics comprising product characteristics, consumer satisfaction, and repurchase intention, and administered a survey assessing these variables to 356 Korean and 358 Chinese consumers. We then conducted confirmatory factor analyses and multigroup analyses. The measurement equivalence test results revealed significant differences in the measurement methods for Korean and Chinese consumers, necessitating the development of separate modified models. Specifically, the model using cosmetic product characteristics as a single factor was more suitable for Chinese than for Korean consumers. Therefore, we were able to develop separate research models appropriate for each group. Our results also highlight the importance of assessing measurement invariance for social science researchers desiring to perform multigroup analyses in cross-national studies.

Indicators of (in)tolerance toward immigrants among European youth: an assessment of measurement invariance in ICCS 2016

BackgroundPromoting tolerance is an important goal of European education policies focused on education for democratic citizenship and human rights. In this article, we argue that cross-cultural comparability must be empirically assessed and ensured for the measurement of highly relevant indicators that serve to monitor inter-European and international differences in young people’s tolerant attitudes toward immigrants.MethodsUsing the framework and data provided by the recent International Civic and Citizenship Education Study (ICCS 2016), we examine the extent to which average comparisons of cross-national differences in young people’s tolerant attitudes toward immigrants are empirically justified. Multiple-group confirmatory factor analysis (MGCFA) is applied to estimate the measurement model of the concept and test its measurement invariance across fourteen European countries.ResultsIn line with prior research, our findings show that cross-cultural comparability can be achieved with some modifications. Results of measurement invariance analysis point to the achievement of full scalar invariance with the implication that average scores can be validly compared across the European educational systems under investigation. These findings are largely corroborated by robustness analyses.ConclusionsWe conclude by providing information on further scale refinement and improvement. Limitations and implications for further research are outlined and discussed.

Confirmatory factor analysis and measurement invariance of the eating disorders examination-questionnaire across four male samples in Argentina.

The eating disorder examination-questionnaire (EDE-Q) is among the most widely used instruments in eating disorder research and clinical practice. However, the underlying structure remains a source of confusion, and contradictory results have emerged in studies among male populations. In the current study, we examined previously proposed models of EDE-Q structure in four community samples of Argentinian men. A series of confirmatory factor analyses (CFAs) were performed for five previous factor structure models of the EDE-Q among 232 Argentinian male university students, 277 weightlifters, 275 cross-fit users, and 202 athletes. A multigroup CFA was conducted in the model we retained, to assess measurement invariance across groups. A respecified model of the brief eight-item one-factor proposal provided acceptable fit to the data over the original four-factor structure and three other proposed models. Results from the multigroup CFA showed that the retained model was invariant across samples. Our results provide support for retaining a one-factor EDE-Q structure over a multifactor solution for research purposes among male community samples in Argentina. These data underscore the importance of undertaking psychometric assessment of eating disorder symptom measures before their utilization in specific populations.

Multicultural Validation of the Zuckerman-Kuhlman-Aluja Personality Questionnaire Shortened Form (ZKA-PQ/SF) Across 18 Countries.

The aim of this study was to assess the psychometric properties of the Zuckerman-Kuhlman-Aluja Personality Questionnaire shortened form (ZKA-PQ/SF) in 18 cultures and 13 languages of different African, American, Asian, and European cultures and languages. The results showed that the five-factor structure with 20 facets replicated well across cultures with a total congruence coefficient of .97. A confirmatory factor analysis (CFA) resulted in adequate fit indices for the five factors based on the comparative fit index (CFI), Tucker-Lewis index (TLI; >.90), and RMSEA (.031-.081). A series of CFA to assess measurement invariance across cultures resulted in adequate CFIs and TLIs for configural and metric invariance. However, factors did not show scalar invariance. Alpha internal consistencies of five factors ranged between .77 (Sensation Seeking) and .86 (Neuroticism). The average alpha of the 20 facets was .64 with a range from .43 (SS4) to .75 (AG1). Nevertheless, alpha reliabilities were lower in some facets and cultures, especially for Senegal and Togo. The average percentage of the variance explained based on the adjusted R2 was 2.9%, 1.7%, and 5.1% for age, sex, and, cultures, respectively. Finally, multidimensional scaling suggested that geographically or culturally close cultures share mean profile similarities.

Assessing callous-unemotional traits: development of a brief, reliable measure in a large and diverse sample of preadolescent youth.

Callous-unemotional (CU) traits are critical to developmental, diagnostic, and clinical models of antisocial behaviors (AB). However, assessments of CU traits within large-scale longitudinal and neurobiologically focused investigations remain remarkably sparse. We sought to develop a brief measure of CU traits using items from widely administered instruments that could be linked to neuroimaging, genetic, and environmental data within already existing datasets and future studies. Data came from a large and diverse sample (n = 4525) of youth (ages~9-11) taking part in the Adolescent Brain and Cognitive Development (ABCD) Study. Moderated nonlinear factor analysis was used to assess measurement invariance across sex, race, and age. We explored whether CU traits were distinct from other indicators of AB, investigated unique links with theoretically-relevant outcomes, and replicated findings in an independent sample. The brief CU traits measure demonstrated strong psychometric properties and evidence of measurement invariance across sex, race, and age. On average, boys endorsed higher levels of CU traits than girls and CU traits were related to, yet distinguishable from other indicators of AB. The CU traits construct also exhibited expected associations with theoretically important outcomes. Study findings were also replicated across an independent sample of youth. In a large, multi-site study, a brief measure of CU traits can be measured distinctly from other dimensions of AB. This measure provides the scientific community with a method to assess CU traits in the ABCD sample, as well as in other studies that may benefit from a brief assessment of CU.

Measurement Invariance of a Direct Behavior Rating Multi Item Scale across Occasions

Direct Behavior Rating (DBR) as a behavioral progress monitoring tool can be designed as longitudinal assessment with only short intervals between measurement points. The reliability of these instruments has been mostly evaluated in observational studies with small samples based on generalizability theory. However, for a standardized use in the pedagogical field, a larger and broader sample is required in order to assess measurement invariance between different participant groups and over time. Therefore, we constructed a DBR, the Questionnaire for Monitoring Behavior in Schools (QMBS) with multiple items to measure the occurrence of specific externalizing and internalizing student classroom behaviors on a Likert scale (1 = never to 7 = always). In a pilot study, two trained raters observed 16 primary education students and rated the student behavior over all items with a satisfactory reliability. In the main study, 108 regular primary school students, 97 regular secondary students, and 14 students in a clinical setting were rated daily over one week (five measurement points). Item response theory (IRT) analyses confirmed the technical adequacy of the instrument and latent growth models demonstrated the instrument’s stability over time. Further development of the instrument and study designs to implement DBRs is discussed.

A Monte Carlo Simulation Study to Assess The Appropriateness of Traditional and Newer Approaches to Test for Measurement Invariance

Several structural equation modeling (SEM) strategies were developed for assessing measurement invariance (MI) across groups relaxing the assumptions of strict MI to partial, approximate, and partial approximate MI. Nonetheless, applied researchers still do not know if and under what conditions these strategies might provide results that allow for valid comparisons across groups in large-scale comparative surveys. We perform a comprehensive Monte Carlo simulation study to assess the conditions under which various SEM methods are appropriate to estimate latent means and path coefficients and their differences across groups. We find that while SEM path coefficients are relatively robust to violations of full MI and can be rather effectively recovered, recovering latent means and their group rankings might be difficult. Our results suggest that, contrary to some previous recommendations, partial invariance may rather effectively recover both path coefficients and latent means even when the majority of items are noninvariant. Although it is more difficult to recover latent means using approximate and partial approximate MI methods, it is possible under specific conditions and using appropriate models. These models also have the advantage of providing accurate standard errors. Alignment is recommended for recovering latent means in cases where there are only a few noninvariant parameters across groups.

High-Stakes Testing Case Study: A Latent Variable Approach for Assessing Measurement and Prediction Invariance.

The existence of differences in prediction systems involving test scores across demographic groups continues to be a thorny and unresolved scientific, professional, and societal concern. Our case study uses a two-stage least squares (2SLS) estimator to jointly assess measurement invariance and prediction invariance in high-stakes testing. So, we examined differences across groups based on latent as opposed to observed scores with data for 176 colleges and universities from The College Board. Results showed that evidence regarding measurement invariance was rejected for the SAT mathematics (SAT-M) subtest at the 0.01 level for 74.5% and 29.9% of cohorts for Black versus White and Hispanic versus White comparisons, respectively. Also, on average, Black students with the same standing on a common factor had observed SAT-M scores that were nearly a third of a standard deviation lower than for comparable Whites. We also found evidence that group differences in SAT-M measurement intercepts may partly explain the well-known finding of observed differences in prediction intercepts. Additionally, results provided evidence that nearly a quarter of the statistically significant observed intercept differences were not statistically significant at the 0.05 level once predictor measurement error was accounted for using the 2SLS procedure. Our joint measurement and prediction invariance approach based on latent scores opens the door to a new high-stakes testing research agenda whose goal is to not simply assess whether observed group-based differences exist and the size and direction of such differences. Rather, the goal of this research agenda is to assess the causal chain starting with underlying theoretical mechanisms (e.g., contextual factors, differences in latent predictor scores) that affect the size and direction of any observed differences.