Measurement Invariance Research Articles

Psychometric properties of the Arabic version of the Nightmares Distress Questionnaire (NDQ-AV) in a community sample of adolescents

BackgroundThe Nightmare Distress Questionnaire (NDQ), known and commonly used for its adequate psychometric properties, is the most widely used instrument for the measurement of nightmare distress. This study aimed to assess the psychometric properties of a newly translated Arabic version of the Nightmare Distress Questionnaire (NDQ-AV).MethodsA total of 546 Lebanese adolescents was recruited for this study and completed the NDQ-AV, the Insomnia Severity Index (ISI), and the eight-item Depression, Anxiety and Stress Scale (DASS-8).ResultsThe results of the exploratory factor analysis (EFA) and the confirmatory factor analysis (CFA) supported both a one-factor model and a two-factor model of the NDQ-AV, with the latter consisting of seven items within both factors. The first factor was referred to as the “general nightmare distress and coping” while the second was named “nightmare impact and perception”. The reliability of the scale was excellent (α = 0.930 and ω = 0.915). Moreover, measurement invariance was shown across gender, demonstrating that this measure performs consistently for both men and women. Additionally, the NDQ-AV scores exhibited excellent reliability alongside factorial and concurrent validity.ConclusionIn conclusion, these results support the psychometric validity of the Arabic version of the NDQ. The availability of the NDQ-AV is expected to facilitate the understanding of nightmare distress within the Lebanese context.

Measurement invariance and Polish norms for the Perth Empathy Scale (PES)

Measurement invariance and Polish norms for the Perth Empathy Scale (PES)

Exploring Chinese and Iranian EAP students’ oral communication apprehension in English: A cross-cultural mixed-methods study

The role of students' emotions in their academic performance has been widely endorsed in the literature after the rise of positive psychology. However, the scholarly context of English for academic purposes (EAP) has limitedly inspected the status of students' emotions while learning English. One such emotion is oral communication apprehension (OCA), which is a widespread pitfall among EAP students. However, the causing factors and coping strategies of OCA across cultural contexts have been overlooked, to date. To fill this gap, this mixed-methods study scrutinized the level of OCA, leading factors, and coping strategies among 1618 Chinese and Iranian EAP students via questionnaires and semi-structured interviews. The results of the measurement invariance test and independent samples t-test revealed a significant difference between Chinese and Iranian EAP students’ OCA with Chinese participants having a higher OCA. Furthermore, the results of descriptive statistics and thematic analysis demonstrated that Chinese and Iranian EAP students had analogous perceptions regarding the causes and solutions of OCA. They enumerated several instructional and psychological factors that generated OCA and proposed similar coping strategies to curb it. The study provides some practical implications for EAP teachers, students, and teacher trainers, who can realize the criticality of emotions in EAP.

The effects of facial appearance on measures of generalized trust

Research suggests various associations between generalized trust and a wide range of economic, political, and social dimensions. Despite its importance, there is considerable debate about how best to measure generalized trust. One recent solution operationalizes generalized trust as the average of trust ratings across a small set of trust domains and human faces. Here, we investigate whether heterogeneity in facial appearance affects the psychometric properties of these new instruments. In a survey experiment conducted with a sample of U.S. adults (n = 5001), we randomly assigned respondents to one of five conditions that varied the features of human and AI-synthesized faces. Irrespective of the condition, respondents rated each face along four trust domains. We find that facial heterogeneity has negligible effects on the measurement validity and measurement equivalence of these new instruments. Small differences are observed on a subset of faces for some psychometric tests. These findings contribute to a growing body of work using faces to measure generalized trust, and demonstrate the utility of using AI-synthesized faces in social science research more broadly.

Psychometric evaluation of the german version of the parent-adolescent communication scale.

The quality of dyadic Parent-Child Communication (PCC) between parents and their adolescent children may be important for the adolescent's general development and mental health during adolescence. Since German instruments for measuring PCC are still lacking, this study aimed to validate the German translation of the Parent-Adolescent Communication Scale (PACS), an internationally established self-report questionnaire. To examine the theoretical-based two-factor model of the PACS in a German population sample of 1044 parent-adolescent dyads, confirmatory factor analyses were used. Moreover, the cross-group measurement invariance of the instrument was assessed. Two clinical samples were used to assess discriminative validity. Bland-Altman plots were used to asses parent-adolescent agreement. The two-factor structure of the PACS was supported. Scalar measurement invariance was established for parent age groups and adolescent gender but not for adolescent age groups and parent gender. Convergent and divergent validity were established. Discriminative validity was only established for the adolescent version in the adolescent psychiatry sample. Test-retest reliability was satisfactory, and rater agreement was medium. The German translation of the PACS is a reliable and valid measurement to assess parent-adolescent communication quality per self-report. Research directions include the concordance discrepancy in clinical samples and further investigation of changes in parents' and adolescents' perspectives on communication problems and open communication. Registration: NCT05332236 (clinicaltrials.gov), first registered 2022-04-11.

Validity and Reliability of the Chinese Version of Digital Hoarding Scale

Objectives: To facilitate research on digital hoarding in China, this study evaluated the psychometric properties of the Digital Hoarding Scale (DHS). Methods: A total of 618 participants were recruited through an online survey. The factorial structure, measurement invariance (MI), reliability, and validity of the DHS were tested. Results: The Chinese version of DHS was best represented by a bifactor exploratory structural equation modeling representation, which contains one general factor and three specific factors. The MI of the DHS was established across gender and age groups. Reliabilities of the specific and general factors were demonstrated by high values of omega coefficients. Moreover, the criterion validity was supported by the positive associations between DHS and physical hoarding disorder, obsessive-compulsive disorder, and anxiety. Conclusions: The DHS is a reliable tool for future studies investigating digital hoarding in Chinese.

Aggression of electric bike riders in China: comparisons of the Cycling Anger Expression Inventory toward drivers, pedestrians, and other riders

ABSTRACT Electric bikes (e-bikes) account for an unexpected proportion of traffic crash injuries and deaths in China. There is an urgent need to study the factors influencing the aggressive behaviours of e-bike riders and reduce their potential involvement in crashes. In addition, aggression varies when directed at different types of road users, particularly when expressed by vulnerable groups. This study aimed to validate the Cycling Anger Expression Inventory (CAX) for e-bikers and explore whether the expression of aggression in e-bikers differs depending on the recipients, such as drivers, pedestrians, or other riders. The analysis examined the aggression specificity of e-bikers by employing tests of measurement invariance (MI). These findings indicate that adaptive/constructive expression has emerged as a mainstream strategy for managing e-biker anger. E-bikers exhibited the highest levels of aggression toward drivers. These results provide a strong basis for the prevention and intervention of aggression and accidents among Chinese e-bikers.

Testing for Measurement Invariance (MI): Do the Structures of Microaggression, Discrimination, and Resilience Among Black Women Living with HIV Remain the Same Across Time?

Assessing measurement invariance and the interplay of discrimination, microaggressions, and resilience among Black women living with HIV (BWLWH) across time utilizing latent class and repeated measure analysis may provide novel insights. A total of 151 BWLWH in a southeastern U.S. city completed surveys focused on multiple forms of microaggressions and discrimination (race, gender, sexual orientation, or HIV-related) and resilience factors (social support, self-efficacy, post-traumatic growth) at baseline, 3months, and 6months. To capture the psychosocial domains of discrimination, microaggressions, and resilience, three latent factors were developed and measured across three time points. Latent class analysis was also conducted to identify and compare meaningful subgroups based on varying levels of discrimination, microaggressions, and resilience reported. Three latent classes were created. MI testing suggested that measurement invariance was partially met (established metric invariance and scalar invariance), and it is possible to compare factor means of discrimination, microaggressions, and resilience across time. Latent factor mean scores of microaggressions and discrimination decreased after 3 and 6months and increased for resilience after 6months and varied over time across the three latent classes identified. The subgroup with the lowest level of discrimination and microaggressions and the highest level of resilience reported at baseline, experienced increases in resilience after months 3 and 6. Clinical interventions, research, and policies aimed at promoting resilience and reducing structural and social barriers linked to racism, sexism, HIV stigma, and classism are needed to improve the health and well-being of BWLWH.

Does the Bayley-4 measure the same constructs across girls and boys and infants, toddlers, and preschoolers?

This study tested the assumption that the Bayley Scales of Infant and Toddler Development, Fourth Edition (Bayley-4) functions similarly for boys and girls and for four age groups. The Bayley-4 American norming sample of 1,700 children ages 0-42 months (3.5 years) was used, which included 50% boys and girls. Fifty-three percent of the children identified as White, 22.1% as Hispanic, 12.5% as Black, 8.5% as other, and 4.0% as Asian. A confirmatory factor analysis demonstrated the three-factor structure of cognitive, language, and motor abilities fit the data well (comparative fit index = .99, root-mean-square of error of approximation = .08, standardized root-mean-square residual = .02) and fit significantly better than the two- and one-factor models. The correlations between the latent factors were moderate (r = .73) to large sized (r = .81). Measurement and structural invariance were tested for boys and girls and four age groups (0-5, 6-13, 14-25, and 26-42 months). Residual invariance was supported for girls and boys, and intercept invariance was supported for the four age groups. The measurement invariance results suggest the Bayley-4 is not biased toward these gender and age groups, and group comparisons and decision making can be made with the Bayley-4 scores. Structural invariance findings suggested some differences for gender and age groups. The relations between the cognitive, language, and motor factors and factor variances were equal across girls and boys but differed significantly across the four age groups. Girls scored significantly higher on the three latent means, but these differences were small to negligible. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Mindful Mating: Testing Measurement Invariance and Associations Between Relationship Mindfulness and Relationship Quality Among White and Black Women

Mindful Mating: Testing Measurement Invariance and Associations Between Relationship Mindfulness and Relationship Quality Among White and Black Women

Is the use of multiple-choice items and a holistically-scored paragraph translation task fair? Examining a large-scale translation subtest

AbstractThis study investigated the test fairness of the translation section of a large-scale English test in China by examining its Differential Test Functioning (DTF) and Differential Item Functioning (DIF) across gender and major. Regarding DTF, the entire translation section exhibits partial strong measurement invariance across female and male test takers, while exhibiting full measurement invariance across test takers in (1) arts & humanities and social sciences (A&HSS) and (2) science, technology, engineering or mathematics (STEM) majors. No major-based DIF was detected in this study. Objective test items tend to favor male test takers, while direct translation test task was more favorable to females. Combining the DIF and DTF results, there may be a cancelation effect in our case. However, the effect size of DIF is either negligible or slight to moderate, indicating minimal impact on the overall fairness of the translation test task. This study further discusses the necessity of exploring the source of DIF and the importance of combining DIF and DTF for test fairness research.

Psychopathic Traits in Adult versus Adolescent Males: Measurement Invariance across the PCL-R and PCL:YV.

Both the Psychopathy Checklist-Revised (PCL-R) and the Psychopathy Checklist: Youth Version (PCL:YV), respectively, are established instruments for assessment of psychopathy and development of psychopathic propensity. To reliably compare scores from both instruments, measurement invariance must be established. The current study involved a combined sample of 1091 male participants (adults = 813; adolescents = 278) from correctional facilities in New Mexico. An exploratory structural equation modeling (ESEM) framework was used to test for measurement invariance. The four-factor ESEM model demonstrated good fit for the combined and individual samples. Results from the multiple group ESEM provide evidence for generally strong invariance, with equivalent factor loadings and thresholds. Adolescents exhibited decreased latent interpersonal traits but increased latent features on other PCL factors (affective, lifestyle, and antisocial) compared to adults. Findings suggest that the four-factor model and the measurement of psychopathic traits remain consistent across age groups. Implications of the findings within research and clinical contexts are discussed.

Development and Validation of the Perceived Deepfake Trustworthiness Questionnaire (PDTQ) in Three Languages

Exposure to false information is becoming a common occurrence in our daily lives. New developments in artificial intelligence are now used to produce increasingly sophisticated multimedia false content, such as deepfakes, making false information even more challenging to detect and combat. This creates expansive opportunities to mislead individuals into believing fabricated claims and negatively influence their attitudes and behavior. Therefore, a better understanding of how individuals perceive such content and the variables related to the perceived trustworthiness of deepfakes is needed. In the present study, we developed and validated the Perceived Deepfake Trustworthiness Questionnaire (PDTQ) in English, Italian, and Slovene. This was done in three phases. First, we developed the initial pool of items by reviewing previous studies, generating items via interviews and surveys, and employing artificial intelligence. Second, we shortened and adapted the questionnaire according to experts’ evaluation of content validity and translated the questionnaire into Italian and Slovene. Lastly, we evaluated the psychometric characteristics via a cross-sectional study in three languages (N = 733). The exploratory factor analyses suggested a two-factor solution, with the first factor measuring the perceived trustworthiness of the content and the second measuring the perceived trustworthiness of the presentation. This factorial structure was replicated in confirmatory factor analyses. Moreover, our analyses provided support for PDTQ’s reliability, measurement invariance across all three languages, and its construct and incremental validity. As such, the PDTQ is a reliable, measurement invariant, and valid tool for comprehensive exploration of individuals’ perception of deepfake videos.

How should coronaphobia Be conceptualized? Evaluating competing models of the COVID-19 Phobia Scale using confirmatory factor analysis

Coronaphobia represents the fears and phobias attributed to the COVID-19 virus and pandemic. The COVID-19 Phobia Scale, previously validated as a four-factor structure, is a widely used multidimensional measure to assess coronaphobia. The current study scrutinized various competing factor structures of this instrument to identify the optimal psychometric representation of coronaphobia. Adults (N = 412) ranging in age from 19 to 84 years completed the COVID-19 Phobia Scale. Several theoretically plausible factor structures of coronaphobia were tested and compared using confirmatory factor analysis: four-factor structure, one-factor structure, higher-order factor structure, and five-factor bifactor structure. The one-factor structure showing an overarching factor of COVID-19 phobia revealed the poorest fit to the data. The five-factor bifactor structure that allowed every item to double-load on the COVID-19 general phobia factor alongside one of the COVID-19 specific phobia factors (psychological phobia, psycho-somatic phobia, economic phobia, or social phobia) produced the strongest fit indices and was superior to the widely accepted four-factor structure. The five-factor bifactor structure demonstrated multi-group measurement invariance across gender, race, age, and state urbanization. Furthermore, the general phobia factor and the psycho-somatic phobia factor from the five-factor bifactor structure uniquely predicted present moment state anxiety using structural equation modeling. This psychometric investigation underscores that coronaphobia is experienced as a general pandemic fear that is simultaneously experienced with specific domains of pandemic fears. Theoretical and methodological insights are offered for conceptualizing and measuring coronaphobia and understanding how pandemic phobias differ from traditional phobias.

Internalized Shame Among Justice-Involved Women in Substance Use Disorder Treatment: Measurement Invariance and Changes During Treatment.

Shame is one of the leading barriers to successful recovery in substance use treatment settings. This secondary analysis study examined measurement invariance of the Internalized Shame Scale (ISS) and explored changes in shame during treatment. Participants (N=105) in the parent study were recruited from a nonprofit residential treatment center for justice-involved women and were randomized to receive mindfulness-based relapse prevention or relapse prevention treatment. A series of confirmatory factor analyses were used to assess measurement invariance in a one-factor measurement model of the ISS. Latent growth curve modeling was used to examine change in shame over time. Our findings support the assumption of measurement invariance across multiple time points and across treatment conditions, supporting comparisons of stigma scores across groups and over time. Although we observed significant reductions in shame from pre- to post-treatment, there were no differences across treatment conditions. Additional research is needed to determine how distinct treatment components relate to reductions in shame among individuals receiving treatment for a substance use disorder.

Measurement invariance of the Coparenting Relationship Scale (CRS) across 10 countries.

The purpose of this study was to assess the factor structure and the measurement invariance of the Coparenting Relationship Scale (CRS) across 10 countries based on the seven-factor coparenting model (i.e., Coparenting Agreement, Coparenting Closeness, Exposure to Conflict, Coparenting Support, Endorsement of Partner's Parenting; Division of Labor) proposed by Feinberg (2003). The results of research on coparenting from numerous countries have documented its foundational importance for parent mental health, family relationship quality, child development, and psychopathology. Yet, a cross-country perspective is still lacking. Such a perspective can provide insight into which dimensions of coparenting are universally recognized and which are especially prone to variation. A unique multinational data set, comprised of 15 individual studies collected across 10 countries (Belgium, Brazil, China, Israel, Italy, Japan, Portugal, Switzerland, Turkey, USA) in nine languages was established (N = 9,292; 51.1% mothers). Measurement invariance analyses were conducted. A six-factor structure (original seven factors minus Division of Labor) of the measure was consistent across the different contexts and measurement invariance was achieved at the configural level. There was no support for metric or scalar invariance. These findings provide a basis for the CRS to be used across countries and should inspire future quantitative and qualitative research in cross-country coparenting research to understand what aspects are universal and what aspects of coparenting are linked to specific material, relational, or ideational conditions that underlie high-quality coparenting. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Measuring sexual behavior stigma among cisgender men who have sex with men: an assessment of cross-country measurement invariance.

Globally, cisgender men who have sex with men experience sexual stigma, but limited investigation of cross-population scale performance hinder comparisons. As measurement invariance is a necessary but seldom-established criterion of valid cross-cultural comparisons, we assessed invariance in scales of stigma related to sexual behavior across 9 countries. This secondary analysis used data collected from adult (mean age=29.6, standard deviation=12.5) cisgender men who have sex with men (n=8,669) in studies from 6 West African, 2 Southern African, and 1 North American country from 2012-2016. A common item set assessed 2 sexual behavior stigma domains. A sequential process was used to test the factor structure and measurement invariance, which included multigroup confirmatory factor analyses (CFA). Individual countries, items, living with HIV, and disclosure were explored as possible sources of noninvariance. Goodness-of-fit statistics indicated adequate fit of the same 2-factor model in 7 of the 9 countries. The chi2 difference test comparing a constrained and unconstrained 7-country model in which loadings and thresholds were freely estimated was significant (p<0.001), indicating metric and scalar noninvariance, but removing the US provided evidence of invariance and freeing certain items led to a finding of partial invariance. Sexuality disclosure exhibited a direct relationship with select stigma items in several countries. Our findings point to the utility of the two stigma scale dimensions in making cross-country comparisons, but also to the necessity of assessing invariance with explicit attention to several factors including differential disclosure of sexuality across contexts to ensure valid comparisons.

The Impact of “Negligible” Cross-Loadings in Investigations of Measurement Invariance with MGCFA and MGESEM

Measurement invariance (MI) is an essential part of validity evidence concerned with ensuring that tests function similarly across groups, contexts, and time. Most evaluations of MI involve multigroup confirmatory factor analyses (MGCFA) that assume simple structure. However, recent research has shown that constraining non-target indicators to zero when cross-loadings are present results in biased estimates of latent variable associations. Using Monte Carlo simulation, we investigate the behavior of fit statistics for identifying non-invariance when the target measurement model is the same for both groups, and the source of non-invariance is the presence and magnitude of non-zero cross-loadings in one group, but not in another. We consider differences between MGCFA and multigroup ESEM (MGESEM), and combined and separate group tests of configural invariance. Implications for applied researchers are provided.

Assessing and accounting for measurement in intensive longitudinal studies: current practices, considerations, and avenues for improvement.

Intensive longitudinal studies, in which participants complete questionnaires multiple times a day over an extended period, are increasingly popular in the social sciences in general and quality-of-life research in particular. The intensive longitudinal methods allow for studying the dynamics of constructs (e.g., how much patient-reported outcomes vary across time). These methods promise higher ecological validity and lower recall bias than traditional methods that question participants only once, since the high frequency means that participants complete questionnaires in their everyday lives and do not have to retrospectively report about a large time interval. However, to ensure the validity of the results obtained from analyzing the intensive longitudinal data (ILD), greater awareness and understanding of appropriate measurement practices are needed. We surveyed 42 researchers experienced with ILD regarding their measurement practices and reasons for suboptimal practices. Results showed that researchers typically do not use measures validated specifically for ILD. Participants assessing the psychometric properties and invariance of measures in their current studies was even less common, as was accounting for these properties when analyzing dynamics. This was mainly because participants did not have the necessary knowledge to conduct these assessments or were unaware of their importance for drawing valid inferences. Open science practices, in contrast, appear reasonably well ingrained in ILD studies. Measurement practices in ILD still need improvement in some key areas; we provide recommendations in order to create a solid foundation for measuring and analyzing psychological constructs.

Systematic review following COSMIN guidelines: Short forms of Zarit Burden Interview

This comprehensive review assessed the psychometric properties of abbreviated versions of the Caregiver Burden Instrument (ZBI-22). Initially, 40 articles that met the inclusion criteria were identified through a systematic search of four databases. Additionally, 26 articles were included through manual searches, totaling 66 articles in the analysis. Different versions of instruments measuring caregiver burden were examined, considering item variability and differences in factor structures. Although most measures exhibited satisfactory content validity, as well as construct validity supported by high internal consistencies, it is important to note that measurement invariance, criterion validity and test-retest reliability were not established for all measures analyzed. Furthermore, structural validity was not satisfactory for all versions. Research and clinical practice could benefit from a standardized approach that allows for a more accurate and consistent assessment of caregiver strain.