Support For Measurement Invariance Research Articles

Psychometric properties of a Mandarin Chinese version of the Body Appreciation Scale-2 among Chinese adolescents

In the present study, we explored the factor structure and psychometric properties of the Mandarin Chinese BAS-2 among adolescents residing in the Chinese mainland. Exploratory factor analysis in Study 1 (N = 790; 396 girls, 394 boys) supported the unidimensionality of the Mandarin Chinese BAS-2 among Chinese adolescents. Internal consistency reliability was upheld via McDonald’s omega. Convergent validity was supported by its moderate-to-strong relationships with body satisfaction, functionality satisfaction, self-esteem, life satisfaction, positive affect, and negative affect, while its small-to-moderate correlation with social desirability provided somewhat weaker discriminant validity support. Criterion-related validity was upheld by its inverse correlation with eating disorder symptomatology and positive correlation with intuitive eating. It explained unique variance in self-esteem (for girls and boys), eating disorder symptomatology (for girls), and intuitive eating (for boys) beyond age, body satisfaction, and functionality satisfaction, providing incremental validity evidence. A subsample of 134 girls and 114 boys completed the Mandarin Chinese BAS-2 again after three months, and test-retest reliability was upheld. The confirmatory factor analysis in Study 2 (N = 337; 192 girls, 145 boys) replicated the unidimensional structure and supported measurement invariance across gender. Collectively, the present study supported the unidimensionality, reliability, and validity of the Mandarin Chinese BAS-2′s scores among Chinese adolescents.

Measurement invariance of the Depressive Symptom Inventory-Suicidality Scale across race, ethnicity, sexual orientation, and plurality of minoritized identities.

Measurement invariance across diverse groups, though crucial for determining the generalizability of a measure, has not yet been tested in many assessments of suicidal thoughts. The present study assessed the measurement invariance and psychometric properties of one such assessment, the Depressive Symptom Inventory-Suicidality Subscale, across multiple identity dimensions in a large data set (n = 1,118) that combined three diverse samples. Findings supported measurement invariance of the scale by race, ethnicity, and sexual orientation when comparing those who were majority-aligning with their minoritized counterparts, as well as good internal consistency and expected convergent validity. The expected one-factor structure fit well for all three of the samples assessed. Overall, the results support measurement invariance and generalizability of the Depressive Symptom Inventory-Suicidality Subscale. Future studies should continue assessing measures of suicidal thoughts and behaviors through testing invariance across identities, especially as it pertains to specific identity subgroups and their intersections. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Measurement invariance of the Marijuana Motives Measure among men and women using Stop Cannabis App

BackgroundMotives to use cannabis play a central role in the development and maintenance of problematic cannabis use and previous studies stressed sex-related differences on motives to use cannabis. However, motives cannot be validly compared in men and women without first establishing the measurement invariance across sex. Therefore, the aim of the study is to (1) examine for the first time the measurement and structural invariance of the Marijuana Motives Measure (MMM) across sex, and (2) to investigate the motives for cannabis use that best explain problematic use. Methods2951 (41.7% women) users of the “Stop cannabis” smartphone app of which 99.8% reported having used cannabis in the last three months completed an online MMM and ASSIST to assess the severity of their problematic cannabis use. ResultsMultigroup confirmatory factor analyses supported measurement invariance across sex, whereas structural invariance was not confirmed. Indeed, group comparisons indicated that women reported greater coping motives then men whereas men showed greater social motives than women. A multiple linear regression analysis showed that only coping and conformity motives were significantly associated with greater problematic cannabis use, whereas neither sex nor the sex by motives interactions were significantly related to problematic cannabis use. ConclusionsThe MMM appears to function comparably across men and women. Therefore, sex-related comparisons on the questionnaire can be considered valid. Coping and conformity motives may play a central role part in the development of marijuana use problems which may hold implications for intervention development and public policy.

Examining the assumption of measurement invariance in job performance ratings across time: The role of rater experience

AbstractPerformance appraisals are widely used in organizations and most typically involve raters evaluating groups of subordinates along a set of items designed to represent job performance over a predetermined period (e.g., annually). A defining but often overlooked characteristic of performance appraisals is that they are cyclical. Since raters conduct appraisals over many cycles, it may be that measures of job performance are not equivalent across time. This is important because changes or differences in aggregated performance ratings can only be meaningfully interpreted if raters' definitions of job performance, interpretation of what the items mean, and their view of what constitutes the different levels of performance remain unchanged over time, unaffected by their experience with appraisals. Although critical to the interpretation of job performance scores, measurement invariance concerns are generally absent from the literature. The current research investigated the extent to which rater experience affected the conceptualization and measurement of performance using performance data from a major South American company which comprised information from raters and ratees through several appraisal cycles. In the between‐rater design, measurement invariance was analyzed using ratings of one performance appraisal cycle from 514 raters divided into groups according to their level of experience. The within‐rater design analyzed ratings from the same 80 raters in their first three appraisal cycles. In the between‐rater analysis, data supported measurement invariance across raters with different levels of experience. Results from the within‐rater analysis suggested that the job performance factor structure was not the same across cycles. Implications for research and practice are discussed.

Predicting and Moderating COVID-Fear and Stress among College Students in Argentina and the USA.

The COVID-19 pandemic negatively affected mental health worldwide and college students were particularly vulnerable to its adverse effects. This longitudinal study was designed to highlight and compare the COVID experiences of college students in Argentina and the USA (N = 361). Specifically, we examined individual factors (gender, emotional regulation, and social support) assessed prior to the pandemic for their role as predictors or moderators of COVID-fear and psychological stress during the first months of the pandemic. The results supported measurement invariance for brief measures of COVID-fear and indicated that, overall, COVID-fear was highest during the second wave of the study (March-April 2020), lowest during the third wave (June 2020), and then rose again during the fourth wave (September 2020). Several interaction effects emerged, revealing important country-level differences in COVID-fear effects for the emotion regulation and social support factors. More so in the Argentina sample than in the USA sample, higher levels of social support at Time 1 were associated with increases in the effect of COVID-fear on stress among students. We discussed the implications of these and other findings for future cross-cultural pandemic-related stress studies.

Development of the Japanese Version of Pregnancy-Related Anxiety Questionnaire-Revised-2: Measurement and Psychometric Properties.

Tools to evaluate pregnancy-specific anxiety are lacking in Japan. This study aimed to develop a Japanese version of the Pregnancy-Related Anxiety Questionnaire-Revised-2. After scale translation and cognitive interviews, we conducted a cross-sectional study among 120 ≥18-year-old, singleton (pregnant with one baby) Japanese women before 15 weeks of pregnancy, recruited from four facilities. A total of 112 women completed the questionnaires. We tested the internal consistency, measurement error and reliability, structural validity, measurement invariance across nulliparous and parous women, construct validity by calculating omega, standard error of measurement (SEM), intraclass correlation coefficient (ICC), confirmatory factor analysis (CFA), multigroup CFA, multitrait-scaling analysis, correlational analyses with other measurements, and t-test to compare nulliparous and parous groups. Omega was 0.90 for the total score. SEM was 3.4 and ICC was 0.76. The CFA revealed an optimal fit for the three-factor model based on the original scale. Multigroup CFA supported measurement invariance across the nulliparous and parous groups, and multitrait-scaling analysis revealed 100% scaling success. The correlation coefficients with other scales of childbirth anxiety and general anxiety were 0.70 and 0.24. The mean total score of the nulliparous women was higher than that of the parous women (34.5 vs. 30.3, p = 0.001). Therefore, the scale was determined to have good validity and reliability.

Psychometric properties of the Bergen Facebook Addiction Scale for Portuguese adults

Background: The research about online behavior addiction suggests the importance of improving assessment methods and deepening the knowledge about this phenomenon.Goals: To characterize main psychometric properties of the Bergen Facebook Addiction Scale (BFAS) for Portuguese adults.Methods: This cross-sectional study design involved Portuguese participants who were selected by snowball-convenience sampling. Data were collected through online survey of sociodemographic and BFAS, with invitations distributed by Facebook. Data analysis was performed through descriptive and reliability tests, as well factorial and invariance analysis.Results: The sample was composed of 444 Portuguese people between 18 and 73 years old. Reliability estimate for the one-factor solution was .795, suggesting good internal consistency reliability properties. Confirmatory factor analysis revealed congruence with original unifactorial model of BFAS. Multigroup analysis supported measurement invariance across sex and education level.Discussion: These results suggest reliability and validity. Invariance evidence was also confirmed for the BFAS in a Portuguese sample of Facebook users. These results contribute to the validation process of the Portuguese version of the BFAS, encouraging further studies with different clinical and non-clinical groups.

Measuring Resilience in Long-term Sick-listed Individuals: Validation of the Resilience Scale for Adults.

Return to work from long-term sick leave is influenced by personal and social factors, which can be measured by resilience, a construct that describe healthy adaptation against adversity. This study aimed to validate the validity and psychometric properties of the resilience scale for adults in a sample of long-term sick-listed individuals, and to investigate measurement invariance when compared with a university student sample. Confirmatory factor analysis was used on a sick-listed sample (n = 687) to identify the scale?s factor structure, and comparison with a university student sample (n = 241) was utilized to determine measurement invariance. Results show that a slightly modified factor structure, in accordance with previous research, achieved acceptable fit in the sick-listed sample, while comparisons with the student sample supported measurement invariance. This means that the study to a large degree support the factor structure of the resilience scale for adults in long-term sick-listed. Furthermore, the results indicate that the scale is similarly understood among long-term sick-listed as in a previously validated student sample. Thus, the resilience scale for adults can be a valid and reliable measure of protective factors in the long-term sickness absence and return to work context, and the subscale and total score can be interpreted similarly in long-term sick-listed as in other populations.

Longitudinal measurement invariance of the Working Alliance Inventory - Short form across coaching sessions

BackgroundThroughout the psychotherapeutic and coaching literature, the client-therapist or coach-coachee working alliance has been highlighted as key force driving positive outcome. The Working Alliance Inventory Short form (WAI-S) for coaching charts the quality of working alliance throughout coaching sessions and is broadly applied in coaching research. Due to a shortfall in research on psychometric properties of the WAI-S, the purpose of this study was to examine (a) if the theorized three-factor structure of the 12-item WAI-S forms a solid representation of the dimensions of working alliance in coaching, and (b) longitudinal measurement invariance (LMI) of the WAI-S.MethodData were collected in a two-wave study design comprising a main study sample of N = 690 Dutch coachees that completed the questionnaire at the first measurement, of which N = 490 also completed the questionnaire at the second measurement. Post hoc sensitivity analysis was performed based on the original sample, lacking additional information on covariates, and included both completers and dropouts, comprising N = 1986 respondents at T1, and N = 1020 respondents at T2.ResultsConfirmatory factor analyses evidenced best fit of the three-factor model in comparison to one-, and two-factor models at both time points. Despite the fact that multigroup confirmatory factor analysis detected non-invariant intercepts, our findings overall supported measurement invariance across coaching sessions.ConclusionsAs decisions in both clinical and scientific practices generally rely on outcome assessment of interpersonal change in scores on the same measure over time, we believe our findings to be of contributing value to the consolidation of interpretation and accuracy of scorings on the WAI-S in coaching.

Factorial Structure of the Parent-Reported Version of the Inventory of Callous-Unemotional Traits Among Belgian Children: A Theory-Based Model.

The factorial structure of the Inventory of Callous-Unemotional traits (ICU) is still under debate in the current literature and the published models are predominantly based on the empirical results of the statistical analyses rather than on a strong theoretical background. Aimed at overcoming these limits, the current study examined a factorial structure initiated by a theoretical framework for the parent-version of the ICU, based on a community sample of Belgian children aged 3–9 (N = 437; M age = 5.59; 54.7% boys). Further, the current study investigated measurement invariance across age and gender, and the external validity of this structure. Confirmatory factor analysis (CFA) indicated that empirical factor models from the current literature demonstrated a relatively poor fit to the data. Alternative models were built based on theory, specifically criteria from the DSM-V specifier “with limited prosocial emotions.” CFA supported an 18-item second order model with three first order factors (Lack of conscience, Unconcern about performance, Lack of emotional expression), a second order latent factor (General dimension of CU traits) and a methodological factor encompassing negatively worded items. Results supported measurement invariance across child gender, and to a lesser extent across age. As expected, the general dimension correlated with measures of aggressive behavior, attention problems, internalizing behavior and empathy. The Lack of emotional expression subfactor showed a different pattern of associations in comparison to the two other subfactors. The implications of these findings are discussed, specifically in relation to the DSM-V LPE specifier.

Contexts of Concealment: Initial Validation of Three Disclosure Avoidance Process Measures

The choice to withhold subjective distress reflects a multifactorial decision highly sensitive to context. Unfortunately, existing measures are built on unidimensional models (i.e., concealment-disclosure as a single, bipolar dimension) and operationalize the construct as a stable trait. In this article, we outline the development and initial validation of a self-report inventory that accounts for problem-context and assesses multiple processes central to both the concealment and disclosure of emotional distress. Exploratory analysis of pilot items in Study 1 (male student sample; N = 373) guided subsequent item development and revisions to our conceptual model. In Study 2 (mixed-gender community sample; N = 297), we refined the item-pool based on additional tests of latent scale structure and associations with concurrent criteria. In a final validation sample (Study 3; international community participants; N = 978), confirmatory factor analyses corroborated our hypothesized three-factor model (Privacy Management, Disclosure Desire, and Social Fear) and supported measurement invariance by sex. The three Contexts of Concealment Scales (CCS) were internally consistent and associated in expected directions with external indices of concurrent concealment, disclosure, depression, anxiety, loneliness, experiential avoidance, and self-stigma.

Measuring Perceived Attributes of the Interactive Whiteboard for the Mathematics Class

The main purpose of the current study was to develop and validate a scale of perceived attributes of the interactive whiteboard (IW) for the mathematics class. Rogers’ Diffusion of Innovations Theory served as the theoretical framework. Two groups of participants in Turkey were employed in this study. The first group consisted of 350 middle school students (177 female, 173 male). The second group consisted of 557 high school students (288 female, 269 male). The results of confirmatory factor analysis supported construct validity. However, not enough evidence was available to support its convergent and discriminant validity. The results of multigroup analysis supported measurement invariance across two schools—one middle school and one high school. The perceptions of students toward IW were generally positive. We believe that the scale developed for this study can be used in schools in which IW has been implemented and may contribute to educational technology reform efforts by providing metrics for the degree of success in implementing IW specifically and possibly other forms of instructional technology.

Revisiting Shyness and Sociability in Schizophrenia: A Psychometric Examination of Measurement Invariance and Mean Level Differences

Although there is a long and rich empirical history of demonstrating differences on psychological self-report measures between people with schizophrenia and healthy controls, the question of whether both groups respond to psychological measures in the same way has gone largely unexplored. That is, is there measurement equivalence, or invariance, across the samples? To our knowledge, there have been no published studies on measurement equivalency in personality measures across groups diagnosed with and without schizophrenia. Here we examined the question of measurement invariance on two widely used questionnaires assessing temperament, the Cheek and Buss Shyness and Sociability Scales (CBSHY and CBSOC, respectively) between 147 stable adult outpatients with schizophrenia and 147 healthy age- and sex-matched controls. Results supported measurement invariance of the CBSHY and CBSOC across our clinical and non-clinical groups. These findings suggested that stable adult outpatients with schizophrenia and age- and sex-matched controls respond to the shyness and sociability items in the same way. We found that adults with schizophrenia reported higher levels of shyness and lower levels of sociability than healthy controls, consistent with prior studies. Findings are discussed concerning their relevance more broadly to self-report assessments of personality and psychological traits in clinical populations.

Can We Apply WHOQOL-AGE to Asian Population? Verifying Its Factor Structure and Psychometric Properties in a Convenience Sample From Taiwan.

Objectives: To translate and validate a recently developed quality of life instrument (WHOQOL-AGE) on geriatric population into Chinese.Method: Using cross-sectional observational design, the WHOQOL-AGE was conducted among older people through interview. Confirmatory factor analysis (CFA) was used to examine the factor structure and multigroup CFA used to examine the measurement invariance.Results: Through convenience sampling, 522 older adults (mean age = 73.42) participated in the study. Among them, 194 were males, 213 had an educational level at primary school or below, 398 were residing in the community, and 307 were aged 70 years or above. A bifactor structure (items Q1–Q8 are embedded in the factor 1; items Q9–Q13 embedded in the factor 2; and all the items embedded in an additional construct of QoL) was confirmed by the CFA in both the entire sample (χ2 = 25.4; df = 51; p = 0.999) and the subgroup sample with age 70 years or above (χ2 = 25.28; df = 51; p = 1.000). Multigroup CFAs results supported the measurement invariance for the WHOQOL-AGE across genders, having different educational levels, living in different settings and age groups. It also shows good known-groups validity.Conclusions: The promising psychometric properties of the WHOQOL-AGE were found in our convenience sample of older Taiwanese. The supported measurement invariance indicates that the older people in different conditions of gender, educational level, and living setting interpret the WHOQOL-AGE similarly. However, our results should be interpreted with cautious because of the sample representativeness.

The German Multi-Dimensional Perceived Autonomy Support Scale for Physical Education: Adaption and Validation in a Sample of Lower Track Secondary School Students.

Teachers’ autonomy support (AS) in physical education (PE) has positive effects on students’ affective and behavioral outcomes in PE. Even though the existence of three different dimensions of AS, namely cognitive, organizational and procedural AS has been suggested in educational settings, there is a lack of multidimensional instruments for the assessment of autonomy-supportive teaching in PE. The aim of this study was to validate the German Multi-Dimensional Perceived Autonomy Support Scale for Physical Education (MD-PASS-PE). The sample comprised 1030 students of grades 6 through 10. Internal consistency was used to test the reliability of the assumed subscales. Factorial validity and measurement invariance across gender and age were examined by confirmatory factor analyses. Structural equation modeling was used to evaluate criterion validity. The subscales exhibited acceptable to good internal consistency. The assumed three-factor structure was confirmed within a bi-factor model including a general factor and three specific group factors. Results strongly supported measurement invariance across gender while tentatively suggesting measurement invariance across age. Criterion validity was supported as the MD-PASS-PE explained 15% and 14% of the variance in the constructs of self-efficacy and intrinsic value, respectively. The German MD-PASS-PE provides PE teachers with deeper insights into their autonomy-supportive teaching behavior, helping them to support their students’ autonomy in a holistic way.

Pain Anxiety Symptoms Scale–20: An empirical evaluation of measurement invariance across race/ethnicity, sex, and pain.

Pain-related anxiety, defined as fear of pain and pain-related sensations, is a transdiagnostic individual difference factor associated with pain-related problems, addictive disorders, and physical impairment among nonclinical and clinical populations. Pain-related anxiety is most commonly measured using the Pain Anxiety Symptoms Scale-20 (PASS-20). It was hypothesized that the data would provide evidence for a higher order PASS-20 factor structure and this structure would be invariant across race/ethnicity and sex. Therefore, the current study examined measurement invariance of the PASS-20 across a large (n = 3,455) diverse sample (Mage = 21.49, SD = 4.24, 73.7% female, 33.5% Hispanic, 29.3% Asian/Pacific Islander, 22.4% White, and 14.8% Black/African American) of young adults. Results supported measurement invariance across all race/ethnicity and sex groups for the PASS-20 total score, but results were inconsistent for the lower order factors. The PASS-20 total score showed good internal consistency and evidence of convergent and divergent validity with established constructs. These results provide empirical support only for the higher order factor structure of the PASS-20 and support its use across race/ethnicity and sex. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The brief (seven-item) eating disorder examination-questionnaire: Evaluation of a non-nested version in men and women.

Several recent studies have examined the psychometric properties of brief measures of eating disorder attitudes based on the Eating Disorder Examination Questionnaire (EDE-Q). A seven-item version (the EDE-Q7) has been proposed but, as yet, has only been investigated by looking at the items when presented as part of the longer EDE-Q (i.e., as a nested version). The current study presented the EDE-Q7 as a standalone instrument and examined factor structure fit and measurement invariance across male and female genders. University students (244 women; 155 men; 1 did not identify with either gender) completed questionnaires as part of two independent studies. All individuals completed the EDE-Q7 and measures of eating disorder behaviors. In a mixed-gender subsample (n = 286), measures of depression and eating disorder-specific quality of life were also included. Confirmatory factor analysis of the EDE-Q7 was conducted on males and females independently, in addition to estimates of internal consistency reliability and validity. Measurement invariance was assessed through multigroup confirmatory factor analysis. The EDE-Q7 demonstrated good internal consistency and findings supported measurement invariance by gender. In a mixed-gender subsample, the measure showed positive associations with depression and both eating disorder behaviors and eating disorder-specific quality of life. The present study adds to the literature supporting the psychometric properties of the EDE-Q7, extending this to use of the questionnaire as a standalone instrument. Measurement invariance suggests that the measure may be appropriate for college-age men and women, although future studies should establish psychometric properties more fully.

Advances in the assessment of self-determination: internal structure of a scale for people with intellectual disabilities aged 11 to 40.

Advances in theoretical frameworks of self-determination require the development of new assessment instruments. This study examines the dimensional structure of a self-determination scale and analyses the factorial invariance of its measurement across age and gender. The AUTODDIS Scale was used to assess the self-determination of 541 people with intellectual disabilities aged from 11 to 40. Different models (correlational and hierarchical structures) of the scale were tested. The correlational model obtained from the exploratory structural equation model approach provided the best fit for the data. The results also supported measurement invariance across youths (aged 11 to 21years) and adults (aged 21 to 40years) and across genders. This study contributes to international research on self-determination and the development of assessment tools in this field, offering a better understanding of this multifaceted and complex construct. The results provide construct validity evidence regarding a new measurement tool tested across people aged 11 to 40, using information from third parties. However, further research is needed to explore the best ways to understand and assess the different factors related to self-determination.

An Assessment of Factors Related to Ocean Literacy Based on Gender-Invariance Measurement.

This study sought to gain a more comprehensive understanding of how the factors of parental education level and student attitude toward the ocean influence the ocean literacy of students in Taiwan after establishing measurement invariance across genders. The analyzed data were collected from self-reported questionnaires filled out by students aged 16–18 years old. The students’ ocean literacy was used as the outcome variable, while parental education level and student attitude toward the ocean were employed as the independent variables. The effects of parental education level and student attitude toward the ocean on ocean literacy were estimated with a multi-group structural equation model. Of the final total of 945 valid respondents in this study, 58.1% were male and 41.9% were female. The results from the multiple-group analysis supported measurement invariance across the genders. After establishing gender invariance, it was further found that higher degrees of parental education level and student attitude toward the ocean were positively related to ocean literacy. A considerable contribution was detected between parental education level and ocean literacy that was indirectly related through student attitude toward the ocean in the female student.

Factor structure and psychometric properties of a Persian translation of the Epworth Sleepiness Scale for Children and Adolescents.

Background: Given the high prevalence of excessive daytime disorder (EDS) among children and adolescents, daytime sleepiness should be effectively measured for them to design appropriate intervention program. However, the commonly used instrument Epworth Sleepiness Scale for Children and Adolescents (ESS-CHAD) has little information in its psychometric properties. This study aimed to apply 2 different test theories to examine the psychometric properties of the Persian ESS-CHAD among a large sample of Iranian adolescents and children.Methods: In this methodological study, participants from 8 high schools (n=1371; 700 males), in Qazvin, Iran, completed the ESS-CHAD, a background information sheet, and Insomnia Severity Index (ISI). The ESS-CHAD was translated by using a forward-backward translation method.Two weeks later, the participants completed the ESS-CHAD again. Internal consistency using Cronbach’s alpha, test-retest reliability using intraclass correlation coefficient (ICC), regression analysis testing the correlation between ESS-CHAD and ISI, Confirmatory factor analysis (CFA)with measurement invariance, Rasch analysis with differential item functioning (DIF), and latent class analysis (LCA) were used to examine the psychometric properties of the ESS-CHAD. Results: The internal consistency (α=0.79), test-retest reliability (ICC=0.84), regression findings(β=0.39, P<0.001), CFA (comparative fit index [CFI])=0.974, root-mean square error of approximation [RMSEA]=0.040), supported measurement invariance (∆CFI=-0.009 to 0.007,∆RMSEA=-0.009 to 0.001), Rasch analysis (infit mean square=0.88 to 1.31, outfit mean square=0.68 to 1.19), and no substantial DIF (DIF contrast=-0.43 to 0.38) all indicated that ESSCHAD is a reliable and valid instrument. The LCA further classified the sample into 2 distinctclasses. Conclusion: Persian ESS-CHAD could be used to assess daytime sleepiness for adolescents who are speaking Persian.