Measurement Equivalence/invariance Research Articles

Derivation, and establishment of the validity and reliability, of the CASP-11-SG quality of life scale among community-dwelling older adults.

The number and proportion of older adults, aged ≥ 60years, in Singapore is rapidly increasing. A valid and reliable quality of life (QoL) measure will enable assessment of their situation and help evaluate social and clinical interventions, potentially improving care. This study aims to evaluate the validity and reliability of the control, autonomy, self-realization, and pleasure (CASP)-12v.3 QoL scale and establish a scale suitable for use among older adults in Singapore. Data from 3526 community-dwelling older adults from a national survey was used. Measurement properties of the CASP-12v.3 scale were evaluated. Confirmatory factor analysis (CFA; testing single- and two-factor models with residual covariances for negatively worded items and a bifactor model) was performed in half of the sample and exploratory factor analysis (EFA) was performed in the other half. The results led to revised CFA models and the CASP-11-SG scale. The CASP-11-SG scale's measurement properties, convergent, and known-groups validity, and measurement equivalence/invariance (ME/I) across English and Chinese languages were evaluated. Item 3 'I feel free to plan for the future' of the CASP-12v.3 scale had low correlation with other items of the control/autonomy subscale, low item-total correlation and high item-scale correlation. While CFA and EFA supported the two-factor model, removing item 3 improved model fit. The resulting CASP-11-SG scale (Cronbach's alpha: 0.81) demonstrated convergent and known-groups validity and partial ME/I across English and Chinese languages. The CASP-11-SG scale, with satisfactory psychometric properties, can be used for assessing QoL among older adults in Singapore.

The bi-factor structure of the 17-item Hamilton Depression Rating Scale in persistent major depression; dimensional measurement of outcome

The 17-item Hamilton Depression Rating Scale (HDRS17) is used world-wide as an observer-rated measure of depression in randomised controlled trials (RCTs) despite continued uncertainty regarding its factor structure. This study investigated the dimensionality of HDRS17 for patients undergoing treatment in UK mental health settings with moderate to severe persistent major depressive disorder (PMDD). Exploratory Structural Equational Modelling (ESEM) was performed to examine the HDRS17 factor structure for adult PMDD patients with HDRS17 score ≥16. Participants (n = 187) were drawn from a multicentre RCT conducted in UK community mental health settings evaluating the outcomes of a depression service comprising CBT and psychopharmacology within a collaborative care model, against treatment as usual (TAU). The construct stability across a 12-month follow-up was examined through a measurement equivalence/invariance (ME/I) procedure via ESEM. ESEM showed HDRS17 had a bi-factor structure for PMDD patients (baseline mean (sd) HDRS17 22.6 (5.2); 87% PMDD >1 year) with an overall depression factor and two group factors: vegetative-worry and retardation-agitation, further complicated by negative item loading. This bi-factor structure was stable over 12 months follow up. Analysis of the HDRS6 showed it had a unidimensional structure, with positive item loading also stable over 12 months. In this cohort of moderate-severe PMDD the HDRS17 had a bi-factor structure stable across 12 months with negative item loading on domain specific factors, indicating that it may be more appropriate to multidimensional assessment of settled clinical states, with shorter unidimensional subscales such as the HDRS6 used as measures of change.

Measurement invariance tests of revisions to archaically worded items in the Mach IV scale.

The Machiavellian IV [1] instrument, developed almost 50 years ago to measure trait Machiavellianism and still in wide use in personality research, uses item wording that is not gender-neutral, makes use of idiomatic expressions, and includes archaic references. In this two-sample study, exploratory factor analysis (EFA) was conducted on one sample to examine the structure of responses to the Mach IV. In an independent second sample the resulting EFA structure was analyzed using confirmatory factor analysis-based measurement equivalence/invariance (ME/I) tests in a control group with the original archaic items and a treatment group with eight items rewritten in a more modern vernacular. Specific model testing steps [2] and statistical tests [3] were applied in a bottom-up approach [4] to ME/I tests on these two versions of the Mach IV. The two versions were found to have equal form, equal factor loadings, but unequal indicator error variances. Subsequent item-by-item tests of error invariance resulted in substantial decrements to fit for three revised items suggesting that the error associated with these items was not equal across the two versions.

The validity of the Strengths and Difficulties Questionnaire (SDQ) for children with ADHD symptoms

BackgroundThe Strengths and Difficulties Questionnaire (SDQ) is widely used to assess child and adolescent mental health problems. However, the factor structure of the SDQ is subject to debate and there is limited evidence investigating measurement equivalence invariance (ME/I) between treatment groups, informants, and across time.MethodA randomised controlled trial (RCT) recruited 250 participants (6–17 years) who had been referred for an attention deficit hyperactivity disorder (ADHD) assessment. Participants and their clinician either received or did not receive a QbTest report (computer task measuring attention, impulsivity and activity). Parents and teachers completed the SDQ at baseline and 6-months later. This study aimed to understand the factor structure of the SDQ in a clinic referred ADHD sample, and validate the scale as a screening/diagnostic aide and as a measure of treatment outcome both in clinical and research settings. Exploratory Structural Equation Modelling (ESEM) was performed to examine the factor structure, and ME/I was assessed between treatment groups, informants, and time points. The criterion validity of the SDQ predictive algorithm for ADHD was compared with clinician and research diagnoses using logistic regression and tests of diagnostic accuracy.ResultsA 5-factor structure provided the best fit with strong factorial invariance between treatment groups and across time points, but not across informants (parent and teacher ratings). SDQ ratings of ‘probable’ hyperactivity disorder were good predictors of clinical (OR = 10.20, 95%CI 2.18–48.71,p = 0.003) and research diagnoses of ADHD (OR = 6.82, 95%CI 1.95–23.84,p = 0.003), and research diagnoses of Hyperkinetic disorder (OR = 4.02, 95%CI 1.13–14.25,p = 0.031). Further examination of the SDQ hyperactivity ‘probable’ rating showed good specificity (84.5%-74.5%) but poor sensitivity (45.0–42.5%) for ADHD.ConclusionThe findings indicate the SDQ is a valid outcome measure for use in RCTs and clinical settings. However, care should be taken when using the SDQ predictive algorithm to screen for ADHD in clinically referred samples.

Testing Measurement Invariance with Ordinal Missing Data: A Comparison of Estimators and Missing Data Techniques

Ordinal missing data are common in measurement equivalence/invariance (ME/I) testing studies. However, there is a lack of guidance on the appropriate method to deal with ordinal missing data in ME/I testing. Five methods may be used to deal with ordinal missing data in ME/I testing, including the continuous full information maximum likelihood estimation method (FIML), continuous robust FIML (rFIML), FIML with probit links (pFIML), FIML with logit links (lFIML), and mean and variance adjusted weight least squared estimation method combined with pairwise deletion (WLSMV_PD). The current study evaluates the relative performance of these methods in producing valid chi-square difference tests () and accurate parameter estimates. The result suggests that all methods except for WLSMV_PD can reasonably control the type I error rates of tests and maintain sufficient power to detect noninvariance in most conditions. Only pFIML and lFIML yield accurate factor loading estimates and standard errors across all the conditions. Recommendations are provided to researchers based on the results.

Ethnic identity: Factor structure and measurement invariance across ethnic groups.

Considering a historically diversified (and growing) population in the United States, one's ethnic identification is often an important psychological-as well as social and political-construct because it can serve as a hindrance to interpersonal interaction. Despite the importance of ethnic identity in psychological research, the most widely developed ethnic identity measurement tool, the Multigroup Ethnic Identity Measure (MEIM; Phinney, 1992), lacks consensus regarding its psychometric properties. The purpose of this article is to identify the factor structure of this measure and identify whether it exhibits measurement equivalence/invariance (ME/I) across ethnicities. The current findings offer several contributions to the state of the literature. First, our data suggests a two-factor model, including affirmation/commitment and exploration factors, is the most appropriate structure when considering fit and parsimony indices via confirmatory factor analysis. Second, configural and metric measurement equivalence was found across Caucasian and non-Caucasian participants. Interestingly, partial scalar invariance was established when comparing Caucasians with the minority groups with the exception of the Hispanic subgroup, which exhibited no scalar invariance. Third, differences in ethnic identity factor means were found, especially across Caucasians and African Americans. In conclusion, the use of the two-factor model of the MEIM is recommended, and results suggest that the MEIM is an appropriate measure of ethnic identity in most ethnic groups. Limitations and future research are also discussed. (PsycINFO Database Record

Factor structure and longitudinal measurement invariance of PHQ-9 for specialist mental health care patients with persistent major depressive disorder: Exploratory Structural Equation Modelling

BackgroundThe Patient Health Questionnaire-9 (PHQ-9) is a widely used instrument for measuring levels of depression in patients in clinical practice and academic research; its factor structure has been investigated in various samples, with limited evidence of measurement equivalence/invariance (ME/I) but not in patients with more severe depression of long duration. This study aims to explore the factor structure of the PHQ-9 and the ME/I between treatment groups over time for these patients. Methods187 secondary care patients with persistent major depressive disorder (PMDD) were recruited to a randomised controlled trial (RCT) with allocation to either a specialist depression team arm or a general mental health arm; their PHQ-9 score was measured at baseline, 3, 6, 9 and 12 months. Exploratory Structural Equational Modelling (ESEM) was performed to examine the factor structure for this specific patient group. ME/I between treatment arm at and across follow-up time were further explored by means of multiple-group ESEM approach using the best-fitted factor structure. ResultsA two-factor structure was evidenced (somatic and affective factor). This two-factor structure had strong factorial invariance between the treatment groups at and across follow up times. LimitationsParticipants were largely white British in a RCT with 40% attrition potentially limiting the study's generalisability. Not all two-factor modelling criteria were met at every time-point. ConclusionPHQ-9 has a two-factor structure for PMDD patients, with strong measurement invariance between treatment groups at and across follow-up time, demonstrating its validity for RCTs and prospective longitudinal studies in chronic moderate to severe depression.

Validity and Reliability of the Positive Aspects of Caregiving (PAC) Scale and Development of Its Shorter Version (S-PAC) Among Family Caregivers of Older Adults.

To (a) assess the validity and reliability of the 9-item Positive Aspects of Caregiving (PAC) scale among a national sample of caregivers for older adults with functional limitations, (b) develop a shorter version (short-PAC [S-PAC] scale) and assess its psychometric properties, and (c) investigate both scales' measurement equivalence/invariance (ME/I) across language of administration (Chinese/English/Malay). Scale/item measurement property assessment, confirmatory factor analysis (CFA), testing the "original" 2-factor model (6 items: first factor; 3 items: second factor), and exploratory FA (EFA) of the 9-item PAC scale was done. Consequently, alternate CFA models were tested. The S-PAC was developed and subjected to CFA. For both scales, convergent (correlation with caregiver esteem) and divergent (correlation with caregiver depressive symptoms) validity, and language ME/I was assessed. For the 9-item PAC scale, the "original" 2-factor CFA model had a poor fit; its EFA and scale/item measurement properties supported a single factor. Among alternate CFA models, a bi-factor model (all nine items: first factor [overall PAC]; six items: second factor [self-affirmation]; three items: third factor [outlook-on-life]) had the best fit. The bi-factor CFA model also had a good fit for the S-PAC scale, developed after eliminating 2 items from the 9-item PAC scale. Both scales demonstrated convergent and divergent validity, and partial ME/I across language of administration. Both the 9-item PAC and 7-item S-PAC scales can be used to assess positive feelings resulting from care provision among family caregivers of older adults with functional limitations.

Measurement equivalence across gender and education in the WHOQOL-BREF for community-dwelling elderly Taiwanese.

The WHOQOL-BREF, a generic quality of life (QoL) instrument, has been widely used clinically and for research on older populations. However, its measurement equivalence/invariance (ME/I) has not been well examined for the elderly (≥ 65 years) across some different demographics. The data were derived from a cross-sectional study with a convenience sampling design in Taiwan. We enrolled 244 elderly participants: men = 143 (58.6%); educational level ≤ primary school = 121 (49.6%). The ME/I was examined using multiple group confirmatory factor analysis (MGCFA) across gender and educational level. The fit indices were satisfactory for the configural models of gender and educational level (standardized root mean square residual [SRMR] = 0.0742 and 0.0770; root mean square error of approximation [RMSEA] = 0.0655 and 0.0686; comparative fit index [CFI] = 0.953). In addition, MGCFAs showed that ME/I was supported across gender (ΔSRMR = 0.001 to 0.019; ΔRMSEA = -0.003 to 0.001; ΔCFI = -0.003 to 0.000) and educational level (ΔSRMR = 0.002 to 0.006; ΔRMSEA = -0.002 to 0.004; ΔCFI = -0.007 to 0.000). The WHOQOL-BREF Taiwan version is appropriate for combined use and for comparisons in older people across gender and different educational levels.

Measurement and Predictive Invariance of a Work–Life Boundary Measure Across Gender

Pervasive gender norms influence the way in which men and women create and experience work–life boundaries and work–life conflict, which may lead to differing conceptions of work–life boundary management. Measurement equivalence/invariance (ME/I) and predictive invariance analyses were conducted on the WorkLife IndicatorTM (WLI), a measure of work–life boundary management, to determine if men and women maintain similar conceptualizations of work–life boundary constructs and if work–life outcomes are predicted equally well by WLI scales across gender. Three of the five WLI scales exhibited ME/I; however, half of the WLI scale–criterion pairings failed to demonstrate predictive invariance. These results indicate that measurement and/or predictive invariance cannot be assumed when using work–life boundary assessments across gender.

Measurement properties of the sedentary behavior strategy self-management instrument in African-American breast cancer survivors.

To examine the validity and reliability of a modified Sedentary Behavior Strategy Self-Management Scale (SBSMS) in a sample of breast cancer survivors. A total of 291 African-American (AA) breast cancer survivors completed the SBSMS, which was subjected to tests of reliability, structural validity, and tests of measurement equivalence/invariance (ME/I). A revised measurement model fit the data and demonstrated internal reliability and structural validity. Tests for ME/I revealed that the revised model had appropriate levels of invariance among weight status, educational, and years out from diagnosis groups, but not among age groups. The reliability and structural validity of the instrument was supported overall; however, revisions may be needed to support its validity in older AA breast cancer survivors.

Cross-cultural equivalence and latent mean differences of the Negotiation Style Profile (NSP-12) in Taiwan and US managers

Casse's (1981) Negotiation Style Profile (NSP) measures perceiving (intuitive and factual) and processing (normative and analytical) behaviors used during intercultural negotiations. The NSP has been used across cultures in global training and consulting, and in research; however, measurement equivalence/invariance (MI) across national cultures has not been reported for the NSP of any length. An online survey was used to examine the NSP-12 in two cultural samples that represented individual-collectivist and low-high context cultures and two geographic regions. A stratified, random sample of 330 CEO's and managers from publicly traded companies of Taiwan and the United States (US) was selected. Multi-group confirmatory factor analysis (MGCFA) established levels of MI for each of the four NSP-12 styles across the two cultures: “normative style” (strict MI), “factual style” (partial scalar and partial strict MI), “analytical style” (partial metric and partial scalar MI), and the “intuitive style” (metric MI). Latent mean differences showed US managers used factual and analytic styles more frequently than Taiwan managers, consistent with individualistic and low context cultures. Furthermore, Taiwan managers used normative and intuitive styles more frequently than US managers, consistent with collectivist and high context cultures. Limitations and recommendations are discussed.

The Robustness of ME/I Evaluations to Among-Group Dependence

This study evaluates the robustness of multisample confirmatory factor analysis approaches to measurement equivalence/invariance (ME/I) evaluations under a specific model misspecification: when independence among the groups is assumed, but groups are dependent. Monte Carlo simulation is used to investigate the robustness of several ME/I evaluation procedures including the likelihood ratio test (), ΔCFI, ΔSRMR, and ΔRMSEA. The effect of this model misspecification on the factor loadings and their standard errors is also considered. Assuming the groups are independent when they are not is shown to have no practical effect on the results of the , ΔCFI, ΔSRMR, or ΔRMSEA procedures because the effect of model misspecification is canceled out through differencing. Similarly, the estimated factor loadings and standard errors are not significantly affected by incorrectly assuming among-group dependence. These results are used to develop recommendations for researchers who conduct multigroup analyses using structural equation models with samples in which independence is unclear.

Measurement Equivalence across Child Self-Reports and Parent-Proxy Reports in the Chinese Version of the Pediatric Quality of Life Inventory Version 4.0

An important assumption for comparing children's quality of life (QoL) between children's and parents' perceptions is that measurement equivalence/invariance (ME/I) exists. The ME/I across the child- and parent-reported Chinese PedsQL was examined, and the latent means between child self-reports and parent-proxy reports were compared. Third-grade to sixth-grade children (n = 519) and their parents (n = 270) respectively completed the child- and parent-reported PedsQL. Seventy-eight parents completed parent-proxy reports twice. Full ME/I across child and parent reports was found in first- and second-order factor loadings. Partial ME/I was supported in item intercepts and item residual variances. The latent means of child self-reports and of parent-proxy reports were not significantly different, which suggested interchangeability between child- and parent-reported PedsQL. The ME/I results support the use of PedsQL scores to compare children's and parents' perceptions of children's QoL.

Null Results in Assessing Survey Score Comparability: Illustrating Measurement Invariance Using Item Response Theory

In using organizational surveys for decision-making, it is essential to consider measurement equivalence/invariance (ME/I), which addresses the questions of whether score differences are attributable to differences in the latent variable we intend to measure, or attributable to confounding differences in measurement properties. Due to the tendency for null results to remain unpublished, most articles have focused on findings of, and reasons for violations of ME/I. On the other hand, little is available to practitioners and researchers concerning situations where ME/I can be expected to uphold. This is especially disconcerting due to the fact that the null is the desired result in such analyses, and allows for unfettered observed-score comparisons. This special issue presents a unique opportunity to provide such a discussion using real-world examples from an organizational culture survey. In doing so we hope to clear up confusion surrounding the concept of ME/I, when it can be expected, and how it relates to actual differences in scores. First, we review the basic tenets and past findings focusing on ME/I, and discuss the item response theory differential item functioning framework used here. Next, we show ME/I being upheld using organizational survey data wherein violations of ME/I would reasonably not be expected (i.e., the null hypothesis was predicted and supported), and simulate the consequences of ignoring ME/I. Finally, we suggest a set of conditions wherein ME/I is likely to be upheld.

Measurement Equivalence/Invariance A Requirement To Conducting Cross-Groups Comparisons

The establishment of Measurement Equivalence/Invariance (ME/I) is a logical prerequisite to conducting substantive cross-groups comparisons, for example tests of group mean differences, invariance of structural parameter estimates. In agreement with this argument, this paper aims to (1) demonstrate the stages to establish ME/I test, (2) provide an example of ME/I application on customer satisfaction survey across three groups of consumer, (3) demonstrate the steps should be carried out if the measurement were not perceived equivalently by respondents. Conclusions and future research directions are presented.

Validity and reliability of the Caregiver Reaction Assessment scale among primary informal caregivers for older persons in Singapore

Previous studies evaluating the validity and reliability of the Caregiver Reaction Assessment (CRA) scale (24 items; five subscales: schedule, health, finances, family support, and esteem) in different countries are not fully congruent. This article assesses the validity and reliability of the CRA among informal caregivers of older persons in Singapore. Data from a national survey of 1190 primary informal caregivers of Singaporeans aged ≥75 years with ≥1 activity of daily living limitation was analyzed. Fit of the five-factor model was tested in half of the sample using confirmatory factor analysis (CFA) and the other half subjected to exploratory factor analysis (EFA). The CRA was modified accordingly and again subjected to CFA. The CRA's measurement equivalence/invariance (ME/I) across language of administration (Chinese/English/Malay) was assessed. CFA showed a poor fit for the five-factor model. EFA suggested the presence of four factors, three items to have neither sufficient nor unique factor loadings and items on two of the subscales to load on a single factor. CFA of the ‘modified’ CRA (21 items; four subscales: schedule and health, finances, family support, and esteem) suggested a better fit for the four-factor model than for the five-factor model. ME/I analysis supported partial invariance of the CRA across language of administration. The CRA scale should be assessed for relevance in Asian settings. With the suggested modifications, it is suitable for assessing negative and positive effects of caregiving among informal caregivers of older persons with activity limitations in Singapore.

A Direct Comparison Approach for Testing Measurement Invariance

Measurement equivalence/invariance (ME/I) is a condition that should be met before meaningful comparisons of survey results across groups can be made. As an alternative to the likelihood ratio test (LRT), the change in comparative fit index (ΔCFI) rules of thumb, and the modification index (MI), this teaching note demonstrates the procedures for establishing bias-corrected (BC) bootstrap confidence intervals for testing ME/I. Unlike the LRT and ΔCFI methods, which need a different model estimation per item, the BC bootstrap confidence intervals approach can examine item-level ME/I tests using a single model. This method greatly simplifies the search for an invariant item as the reference indicator in the factor-ratio test. Also demonstrated here is how the factor-ratio test and the list-and-delete method can be extended from the metric invariance test to the scalar invariance test. Finally, the BC bootstrap confidence interval procedures for comparing uniqueness variances, factor variances, factor covariances, and latent means across groups are shown.

The Effect of Among-Group Dependence on the Invariance Likelihood Ratio Test

When comparing latent variables among groups, it is important to first establish the equivalence or invariance of the measurement model across groups. Confirmatory factor analysis (CFA) is a commonly used methodological approach to examine measurement equivalence/invariance (ME/I). Within the CFA framework, the chi-square goodness-of-fit test and chi-square difference tests are used to evaluate ME/I, and these tests rely on the assumption of independence among groups. Limitations in the study design can hinder the practicality of the independence among groups assumption. This article illustrates, algebraically, the effects of violations independence on the chi-square goodness-of-fit and chi-square difference test statistics in a multigroup CFA.

The Factor Structure and Factorial Invariance for the Decisional Balance Scale for Adolescent Smoking

The transtheoretical model is a framework to explain smoking uptake and cessation in adolescence. Decisional balance is proposed as a driver of stage movement. The purpose of this study was to examine the factor structure and measurement equivalence/invariance (ME/I) of the decisional balance scale. In this study, we used confirmatory factor analysis followed by measurement equivalence/invariance testing to examine the factorial validity of the decisional balance scale in adolescent smokers and nonsmokers. Unlike previous studies, we found that a four-factor solution splitting cons into esthetic and health cons significantly improved the fit of model to the data. ME/I testing showed that the same structure and measurement model held for both smokers and nonsmokers, girls and boys, and across the three occasions the scale was administered. Cons showed strong evidence that it constituted two separate first order factors. Decisional balance for smoking in adolescence has good evidence of factorial validity.