Measurement Invariance Study Research Articles

Online racial discrimination scale: A study of measurement invariance across racial-ethnic and gender groups.

Reliable and valid tools assessing online racism exposures in youth of color are needed. The present study examined the validity, internal consistency, and measurement invariance of the modified individual and vicarious Online Racial Discrimination (ORD) subscales from the Online Victimization Scale (OVS; Tynes et al., 2010). Using a nationally representative sample of Black and Latinx adolescents (N = 696), this study employed an exploratory structural equation modeling (ESEM) to (a) confirm the factor structure with the updated individual and vicarious ORD subscales, (b) test the measurement invariance across racial-ethnic and gender groups, and (c) investigate the construct validity. Findings revealed that the two-factor model provided good fit of the data and good conceptual clarity consistent with our original framework. Additionally, the individual and vicarious ORD subscales met the criteria for scalar measurement invariance across racial-ethnic groups. These subscales also met criteria for metric invariance across gender groups, and this finding cautions users from comparing (adjusted-)means of intercepts across gender groups given that scalar measurement invariance was not supported. The ORD scale is a reliable and valid measure for assessing experiences of online racial discrimination in Black and Latinx adolescents. These findings provide further support for the ORD scale's psychometric properties to help advance this line of research. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Cultural adaptation of the DESSA high-school student self report for chinese adolescents

Social and emotional skills play a pivotal role in an adolescent’s healthy development. In recent years, there has been a greater focus on the international expansion of social and emotional learning (SEL) efforts to support students’ well-being. SEL efforts in China for example, have expanded significantly to address the increasing trends in the rates of anxiety and depression among adolescents. Despite the increased focus on SEL, there remains a lack of standardized measures to inform these efforts. This study describes the application of the cultural adaptation process for assessments outlined by the International Test Commission to create a Chinese version of the DESSA High School Edition Student Self-Report (DESSA SSR). The DESSA SSR is a standardized, norm-referenced, strength-based behavior rating scale that measures the social and emotional competence of students in 9th through 12th grades. In addition to a rigorous translation process, expert cultural review and a measurement invariance study were conducted. Findings support the use of the DESSA SSR with Chinese students.

Attitudes justifying partner violence against women in Latin-American and Caribbean countries: A measurement invariance study.

Attitudes justifying partner violence against women in Latin-American and Caribbean countries: A measurement invariance study.

Assessing Measurement Invariance in ASWB Exams: Regulatory Research Proposal to Advance Equity

ABSTRACT Purpose Social workers from minoritized racial, ethnic, linguistic, and age groups are far less likely to pass licensing examinations required to practice. Using a simulated data set, our study investigates measurement equivalence, or invariance, of social work licensing exams. Materials For this analysis, we simulated responses to 15 multiple-choice questions which were scored as either correct or incorrect using the R mirt package and used mirt to fit a 2-parameter logistic model (2PL) to the response data. We generated the data so that five items could demonstrate DIF and calculated their impact on the test characteristic curves and item characteristic curves. Results Small amounts of differential item functioning added up into differential test functioning, but the effect size was small. This result is one potential outcome of an analysis of ASWB exams. Discussion Most studies evaluating test characteristic curves demonstrate small effect sizes. Measuring the test characteristic curve and the test information curve will help to investigate content-irrelevant sources of variance in the exams, including unfairness, unreliability, and invalid pass scores. Conclusion Differential test functioning is a core part of measurement invariance studies. Psychometric standards require test developers to assess measurement invariance at both the item-level and test-level to protect themselves from accusations of bias.

Investigation of the Measurment Invairance of the Social Media Addiction Scale

This study aims to examine the measurement invariance of the Social Media Addiction Scale (SMAS) in terms of gender, time spent on social media accounts, and the number of social media accounts. Invariance analyses conducted within the scope of the research were carried out on 672 participants. Measurement invariance studies were examined separately for all measurement models presented in SMAS and for each sub-factor in the scale. As a result of the analyses, it was revealed that the psychometric properties obtained from the measurement model may show bias according to the relevant subgroups for the SMAS, in cases where the model fits up to the configural and metric stages. Comparison of the scores obtained from this measurement tool can be made, but careful interpretation should be made, keeping in mind that the items may behave biased according to gender on an item basis. For the SMAS, it has been demonstrated that in cases where the model fits up to the configural, metric, and scalar invariance stages, comparisons of the psychometric properties obtained from the measurement model can be made without bias according to the relevant subgroups. In addition, in cases where scalar invariance is met, the scores obtained from the measurement tool can be compared and comments can be made on an item basis according to the relevant subgroups. Finally, it could be stated that any comparison made according to the subgroups tested using the SMAS would be meaningless in cases where even configural invariance is not accepted.

The contemporary leadership behavior of school principals scale according to teacher's perceptions: Validity and reliability study

School principals are expected to possess modern leadership abilities that place an emphasis on ideas like collaboration, innovation, technological advancement, and egalitarianism. The objective of this study was to establish the psychometric features of the scale of current leadership behaviors of school principals (SCLBSP) developed in accordance with teacher perspectives. For the scale's content validity, eight experts were contacted, and for each item, the content validity ratio (CVR) and content validity index (CVI) were computed. Two different samples served as the basis for the scale development investigation. 253 teachers' worth of data were utilized in the exploratory factor analysis (EFA), whereas 215 teachers' worth of data were used in the confirmatory factor analysis (CFA) and measurement invariance studies. Cronbach's alpha, McDonald's omega, Split-Half method coefficients, composite reliability (CR), and average variance extracted (AVE) values were determined to determine the scale's reliability. The studies produced a single-factor structure with 34 items that explained 74.4% of the total variation. All SCLBSP items were found to have high levels of discrimination, and the reliability and validity of the entire scale were also found to be high.

Is the meaning of subjective well-being similar in Latin American countries? A cross-cultural measurement invariance study of the WHO-5 well-being index during the COVID-19 pandemic

BackgroundThere is an urgent need to assess changes in well-being on a multinational scale during the COVID-19 pandemic, thus culturally valid scales must be available.MethodsWith this in mind, this study examined the invariance of the WHO well-being index (WHO-5) among a sample of 5183 people from 12 Latin Americans countries (Argentina, Bolivia, Chile, Colombia, Cuba, Ecuador, El Salvador, Guatemala, Mexico, Paraguay, Peru, and Uruguay).ResultsThe results of the present study indicate that the WHO-5 is strictly invariant across samples from different Latin American countries. Furthermore, the results of the IRT analysis indicate that all items of the WHO-5 were highly discriminative and that the difficulty required to respond to each of the five items is ascending. Additionally, the results indicated the presence of moderate and small size differences in subjective well-being among most countries.ConclusionThe WHO-5 is useful for assessing subjective well-being in 12 Latin American countries during the COVID-19 pandemic, since the differences between scores can be attributed to differences in well-being and not in other characteristics of the scale.

Perceived Teacher Informal Relationship Scale: A scale development and measurement invariance study.

This work is intended to develop a measuring tool for determining teacher perception of informal relationships. The pool of item created by researchers through a literature review has been presented with expert assessment of the validity of content, face and meaning, and a draft scale form has been created by making necessary revisions to the feedbacks. The draft form was applied to 214 teachers working in the central districts of Diyarbakır, exploratory factor analysis was made on the obtained data set and a six-dimensional scale structure consisting of 20 items was determined. In order to verify this structure, data were collected from 306 teachers working in the central districts of Diyarbakir, and the six-factor scale structure was confirmed based on the goodness of fit values estimated by confirmatory factor analysis. In addition, AVE (Average Variance Extracted), root of AVE, composite reliability and correlation among factors were checked, and it was seen that the scale provided the convergent and discriminant validity conditions as a result of the values reached. For reliability analysis, Cronbach's Alpha coefficients and composite reliability values were checked together, and it was seen that the scale had sufficient reliability values. The measurement invariance of the scale was tested according to the categories of gender (female-male), marital status (married-single), level of employment (primary school, secondary school and high school) and seniority (1-10 years, 11-20 years, 21 and above) and the formality of the scale, metric, scalar and strict invariance conditions were found to satisfy. Consequently, it was concluded that the scale in question is a valid and reliable scale, which can be used to measure teacher perception of informal relationships.

Prioritize positivity in Italians: a validation and measurement invariance study of an italian version of the prioritizing positivity scale

Prioritize positivity in Italians: a validation and measurement invariance study of an italian version of the prioritizing positivity scale

Factor Structure and Measurement Invariance of the Spanish Burnout Inventory Among Professionals Across 17 Countries and Regions

Studies on the prevalence of burnout in professionals in service organizations who work in direct contact with the clients or users of the organization have concluded that burnout is a serious health disorder that has increased due to the COVID-19 pandemic. A significant advantage of the Spanish Burnout Inventory (SBI) over other instruments is that it provides a broader conceptualization of burnout by including feelings of guilt as a dimension of burnout to explain its development. However, the measurement invariance of the SBI across countries has not been investigated. The purpose of this study was to test the measurement invariance of the SBI among professionals across 17 countries and regions in Europe, Latin America, and Asia, and in different languages. All the countries showed a good fit to the four-factor model, except the Indian sample, which was excluded from the measurement invariance study. Using the alignment method, it was possible to verify the scalar measurement invariance of the four SBI factors across 15 countries and one Spanish region (16 samples). The comparison of estimated latent means indicates that France is the country with the lowest scores on the Enthusiasm factor and the highest scores on the negative factors (Exhaustion, Indolence, and Guilt). In contrast, the Andean countries, Colombia, Peru, and Ecuador, show the highest latent means on the Enthusiasm factor and the lowest means on the negative factors. These results support the validity of the SBI in the countries and regions in Europe and Latin America included in this study.

Implications of A Measurement Invariance Study by Age Cohorts for the Life Satisfaction Survey for Apostolic Women Religious (LSSAWR)

A measurement invariance study was performed on the Life Satisfaction Survey for Apostolic Women Religious (LSSAWR) across age cohorts to support its continued use to assess satisfaction with religious life across an individual sister’s life span, and to conduct intergenerational comparisons within and across congregations worldwide. Unfortunately, measurement invariance (MI) is often assumed rather than tested but is important to determine when comparisons are conducted across groups. Establishing MI produces confidence that the differences observed are a result of real differences between groups rather than a result of group membership. In general, the current study provides evidence that the LSSAWR is MI for life satisfaction across the Silent, Baby Boomer, and Generation X cohorts and should be robust to many types of analyses. Therefore, the continued use of the LSSAWR to provide feedback to individual Sisters and congregations of women religious regarding commitment to religious life and overall life satisfaction is supported. The most notable result was two of the five dimensions of the scale were statistically indistinguishable for the Silent generation, but not for the Baby Boomer or Generation X cohorts. This article discusses the importance of measurement invariance studies and implications for instruments used across the life span with items that could be age sensitive. 
  

Forgiveness in the Context of Divorce: A Cross-Cultural Measurement Invariance Study via Multiple-Group Factor Analysis (CFA) across Chile and Spain.

(1) Background: Current research on the factors involved in the adaptation process to divorce or separation has explored cross-cultural differences. An initial step in the cross-cultural field is to investigate whether the measurements applied are comparable in different cultural contexts. The aim of the present study is to test the measurement invariance of the Questionnaire of Forgiveness in Divorce-Separation (CPD-S); (2) Methods: The CPD-S was completed by 556 (M = 44.52, SD = 10.18) and 240 (M = 41.44, SD = 7.87) Chilean and Spanish divorced individuals, respectively. Confirmatory factor analyses in single samples and measurement invariance testing in a multi-group framework were conducted to test the cross-group equivalence; (3) Results: The single-factor structure of the CPD-S was supported in both countries. Measurement invariance analysis demonstrated that the CPD-S had partial scalar measurement invariance; (4) Conclusions: The evidence supports the conclusion that CPD-S operates similarly across both countries. Findings are discussed from a cross-cultural and methodological perspective.

PISA 2015 Reading Test Item Parameters Across Language Groups: A measurement Invariance Study with Binary Variables

Large-scale international assessments, including PISA, might be useful for countries to receive feedback on their education systems. Measurement invariance studies are one of the active research areas for these assessments, especially cross-cultural and linguistic comparability have attracted attention. PISA questions are prepared in the English language, and students from many countries answer the translated form. In this respect, the purpose of our study is to investigate whether there is a measurement invariance problem across native English and non-native English speaker groups in the PISA-2015 reading skills subtest. The study sample included students from Canada, the USA, and the UK as the native speaker group and students from Japan, Thailand, and Turkey as the non-native speaker group. Measurement invariance studies taking into account the binary structure of the data set for these two groups revealed that eight of the twenty-eight items in the PISA-2015 reading skills test had possible limitations in equivalence

Comprehensive assessment of Acceptance and Commitment Therapy processes (CompACT): Measure refinement and study of measurement invariance across Portuguese and UK samples

The need for a transnational validation is imperative at the stage of development of the CompACT, a self-report measure of psychological flexibility. This study aimed to translate, validate and test the factor structure of the Portuguese version of the CompACT and to conduct a measurement invariance analysis comparing the scale's performance in Portuguese and UK samples.Results from an Exploratory Factor Analysis demonstrated that the Portuguese version of the CompACT statistically performed better without 5 items from the Openness to Experience subscale. The 18-item Portuguese-adapted CompACT presented significant correlations in the expected directions and with the expected magnitudes with AAQ-II, CFQ-7, MAAS, CAQ-8, and DASS-21. Partial metric invariance was demonstrated between the Portuguese-adapted 18-item CompACT and the original CompACT in a UK sample. The non-correspondence between responses to these versions may be due to differences between the Portuguese and British cultures.This study contributes with the adaptation of the original CompACT to the Portuguese language and with the refinement of this instrument to an 18-item measure of psychological flexibility, that appears to be adequate for use in Portuguese samples. The lack of complete metric invariance of the CompACT found across the Portuguese and UK samples highlights the importance of psychometrically analyzing psychological instruments before use in cultural contexts distinct from the one targeted in the measure's original validation study.

Examining the Measurement Invariance of TIMSS 2015 Mathematics Liking Scale through Different Methods

Studies aiming to make cross-cultural comparisons first should establish measurement invariance in the groups to be compared because results obtained from such comparisons may be artificial in the event that measurement invariance cannot be established. The purpose of this study is to investigate the measurement invariance of the data obtained from the "Mathematics Liking Scale" in TIMSS 2015through Multiple Group CFA, Multiple Group LCA and Mixed Rasch Model, which are based on different theoretical foundations and to compare the obtained results. To this end, TIMSS 2015 data for students in the USA and Canada, who speak the same language and data for students in the USA and Turkey, who speak different languages, are used. The study is conducted through a descriptive study approach. The study revealed that all measurement invariance levels were established in Multiple Group CFA for the USA-Canada comparison. In Multiple Group LCA, on the other hand, measurement invariance was established up to partial homogeneity. However, it was not established in the Mixed Rasch Model. As for the USA-Turkey comparison, metric invariance was established in Multiple Group CFA whereas in Multiple Group LCA it stopped at the heterogeneity level. Measurement invariance for data failed to be established for the relevant sample in the Mixed Rasch Model. The foregoing findings suggest that methods with different theoretical foundations yield different measurement invariance results. In this regard, when deciding on the method to be used in measurement invariance studies, it is recommended to examine the necessary assumptions and consider the variable structure.

The Financial Identity Scale (FIS): A multinational validation and measurement invariance study among emerging adults

The transition from financial dependence on one’s parents to financial self-sufficiency is one of the most relevant transitions during emerging adulthood. It is important to have an instrument able to assess emerging adults’ financial capabilities and to detect its change over time. The current article aims to collect international evidence of the Financial Identity Scale (FIS) validity and reliability. Cross-sectional data collected from 2,501 emerging adults aged 18–25 and belonging to three different countries—U.S. (n = 1,535), Italy (n = 485), and Lithuania (n = 481)—were adopted to test score structure validity, generalizability, sensitivity to difference, criterion-related validity, and internal consistency. Instead, four-wave longitudinal data, available for the American sample only (n = 1,900), were adopted to test FIS structural stability and sensitivity to change. As recommended by the contemporary view of validity, different structural equation models were performed. Findings suggest that FIS scores are valid and reliable. The implications for researchers and practitioners are discussed.

Valid group comparisons can be made with the Patient Health Questionnaire (PHQ-9): A measurement invariance study across groups by demographic characteristics.

ObjectiveAnalyze the measurement invariance and the factor structure of the Patient Health Questionnaire-9 (PHQ-9) in the Peruvian population.MethodSecondary data analysis performed using cross-sectional data from the Health Questionnaire of the Demographic and Health Survey in Peru. Variables of interest were the PHQ-9 and demographic characteristics (sex, age group, level of education, socioeconomic status, marital status, and area of residence). Factor structure was evaluated by standard confirmatory factor analysis (CFA), and measurement invariance by multi-group CFA, using standard goodness-of-fit indices criteria for interpreting results from both CFAs. Analysis of the internal consistency (α and ω) was also pursued.ResultsData from 30,449 study participants were analyzed, 56.7% were women, average age was 40.5 years (standard deviation (SD) = 16.3), 65.9% lived in urban areas, 74.6% were married, and had 9 years of education on average (SD = 4.6). From standard CFA, a one-dimensional model presented the best fit (CFI = 0.936; RMSEA = 0.089; SRMR = 0.039). From multi-group CFA, all progressively restricted models had ΔCFI<0.01 across almost all groups by demographic characteristics. PHQ-9 reliability was optimal (α = ω = 0.87).ConclusionsThe evidence presents support for the one-dimensional model and measurement invariance of the PHQ-9 measure, allowing for reliable comparisons between sex, age groups, education level, socioeconomic status, marital status, and residence area, and recommends its use within the Peruvian population.

Is the endorsement of the Attention Deficit Hyperactivity Disorder symptom criteria ratings influenced by informant assessment, gender, age, and co-occurring disorders? A measurement invariance study.

This study aims to ascertain whether the differences of prevalence and severity of attention deficit hyperactivity disorder (ADHD) are true or whether children are perceived and rated differently by parent and teacher informant assessments (INFAs) according to gender, age, and co-occurring disorders, even at equal levels of latent ADHD traits. Use of latent trait models (for binary responses) to evaluate measurement invariance in children with ADHD and their siblings from the International Multicenter ADHD Gene data. Substantial measurement noninvariance between parent and teacher INFAs was detected for seven out of nine inattention (IA) and six out of nine hyperactivity/impulsivity (HI) items; the correlations between parent and teacher INFAs for six IA and four HI items were not significantly different from zero, which suggests that parent and teacher INFAs are essentially rating different kinds of behaviours expressed in different settings, instead of measurement bias. However, age and gender did not affect substantially the endorsement probability of either IA or HI symptom criteria, regardless of INFA. For co-occurring disorders, teacher INFA ratings were largely unaffected by co-morbidity; conversely, parental endorsement of HI symptoms is substantially influenced by co-occurring oppositional defiant disorder. Our findings suggest general robustness of Diagnostic and Statistical Manual of Mental Disorders ADHD diagnostic items in relation to age and gender. Further research on classroom presentations is needed.

German Translation of the Satisfaction With Life Scale for Children and Adolescents

The research in the field of positive psychology is alive for children and adolescents, and especially the effects of intervention studies are promising. While a large number of instruments exist for adults, there is a need for greater attention to basic measurement work of constructs for children. The Satisfaction With Life Scale is one of the most common questionnaires used in the field of positive psychology. Whereas the adult version is translated into many languages, the child version has only been translated into a few languages yet. This is a limitation for countries where young students are not familiar with the English language. The aim of the present study was to translate the Satisfaction With Life Scale–child version (SWLS-C) into German with the use of a backtranslation-proceeding (Study 1) and to validate the questionnaire with a sample of 1,099 students aged 8 to 17 years (Study 2). This included retesting of reliability as well as concurrent and discriminant validity by additionally measuring several established constructs of positive psychology. Model fit indices, internal consistencies, retest reliability as well as concurrent and discriminant validity evidence were satisfying. The German version of the SWLS-C can be used in future studies, but to compare SWLS-C data of different nationalities more translations of the questionnaire as well as crosscultural measurement invariance studies are needed.

Expression and Level of Test Anxiety in a Sample of Elementary Students

This study examined the expression and level of test anxiety in a sample of 1221 elementary school students. The Test Anxiety Scale for Elementary Students (TAS-E), a new multidimensional measure developed based on recent conceptualizations of the test anxiety construct, was administered to elementary school-age students. The TAS-E consists of four test anxiety (Physiological Hyperarousal, Social Concerns, Task Irrelevant Behaviors, and Worry) subscales and a Total Test Anxiety scale. The results of four multi-group confirmatory factor analyses and latent mean analyses supported invariance between groups in the four multi-group comparisons conducted and found grade and gender differences in the level and expression of test anxiety among elementary students in grades 2-5 on the TAS-E. This is the first measurement invariance study to include students as young as grade 2 on a new multidimensional measure of test anxiety. Implications of the findings of the study for mental health professionals and educators who work with elementary school-age students are discussed.