Strict Factorial Invariance Research Articles

Psychometric Findings for a Spanish Translation of the Diabetes Self-Management Profile (DSMP-Parent-Sp)

OBJECTIVEFew validated measures exist to evaluate self-management of diabetes in families with limited English proficiency. The present study evaluated the psychometric properties and the factorial equivalence of a Spanish translation of the parent report version of the Diabetes Self-Management Profile (DSMP-Parent-Sp).RESEARCH DESIGN AND METHODSHispanic families of youth (mean 13.7 years old) with type 1 diabetes were recruited from three clinics in South Florida and represented a wide range of nationalities and acculturation levels. A total of 127 parents reported on their child's self-management behaviors using either the original DSMP-Parent (59.8%) or the DSMP-Parent-Sp (40.2%). In addition, youth reported their self-management using the original DSMP in English, and physicians rated their perceptions of the youth's self-management. Glycemic control was indexed by A1C in the past 3 months and collected from medical chart review.RESULTSItem analysis confirmed that the DSMP-Parent-Sp items related to the overall composite score in expected ways, and internal consistency estimates were adequate. Paired correlations demonstrated strong parent-child concordance and a significant relationship with physician perceptions of self-management. Evidence of concurrent and convergent validity, as well as “strict factorial invariance,” was demonstrated.CONCLUSIONSThese preliminary findings indicate that the DSMP-Parent-Sp is a reliable and valid parent report measure of the diabetes self-management behaviors of Hispanic youths. In addition, there is preliminary evidence that the translated measure may be considered equivalent to the original English measure when used to measure self-management in Hispanic youth with diabetes.

Measurement equivalence of neighborhood quality measures for European American and Mexican American families

The factorial and construct equivalence of subscales assessing parents' and children's perceptions of the quality of their neighborhood was examined in Mexican American and European American families. All subscales (dangerous people in the neighborhood, sense of safety in the neighborhood, quality of the physical environment) demonstrated adequate partial factorial invariance across English- and Spanish-speaking Mexican American and European American families. Reports by children about dangerous people in the neighborhood was the closest to achieving strict factorial invariance, and the only one of the four dimensions to achieve invariance in the validity analyses across Mexican American and European American families. The implications of using these self-report neighborhood quality measures in studies of multiple cultural or language groups are discussed.

Longitudinal factorial invariance of the PedsQL™ 4.0 Generic Core Scales Child Self-Report Version: One year prospective evidence from the California State Children’s Health Insurance Program (SCHIP)

The measurement of health-related quality of life (HRQOL) in pediatric medicine and health services research has grown significantly over the past decade. The paradigm shift toward patient-reported outcomes (PROs) has provided the opportunity to emphasize the value and critical need for pediatric patient self-report. In order for changes in HRQOL/PRO outcomes to be meaningful over time, it is essential to demonstrate longitudinal factorial invariance. This study examined the longitudinal factor structure of the PedsQL 4.0 Generic Core Scales over a one-year period for child self-report ages 5-17 in 2,887 children from a statewide evaluation of the California State Children's Health Insurance Program (SCHIP) utilizing a structural equation modeling framework. Specifying four- and five-factor measurement models, longitudinal structural equation modeling was used to compare factor structures over a one-year interval on the PedsQL 4.0 Generic Core Scales. While the four-factor conceptually-derived measurement model for the PedsQL 4.0 Generic Core Scales produced an acceptable fit, the five-factor empirically-derived measurement model from the initial field test of the PedsQL 4.0 Generic Core Scales produced a marginally superior fit in comparison to the four-factor model. For the five-factor measurement model, the best fitting model, strict factorial invariance of the PedsQL 4.0 Generic Core Scales across the two measurement occasions was supported by the stability of the comparative fit index between the unconstrained and constrained models, and several additional indices of practical fit including the root mean squared error of approximation, the non-normed fit index, and the parsimony normed fit index. The findings support an equivalent factor structure on the PedsQL 4.0 Generic Core Scales over time. Based on these data, it can be concluded that over a one-year period children in our study interpreted items on the PedsQL 4.0 Generic Core Scales in a similar manner.

Psychometric Evaluation of the Exercise Identity Scale Among Greek Adults and Cross-Cultural Validity

The present study reported on translating the Exercise Identity Scale (EIS: Anderson & Cychosz, 1994) into Greek and examining its psychometric properties and cross-cultural validity based on U.S. individuals' EIS responses. Using four samples comprising 33, 103, and 647 Greek individuals, including exercisers and nonexercisers, and a similar sample comprising 800 U.S. individuals, the concurrent validity, factor structure, internal reliability, test-retest reliability, external validity, gender invariance, and cross-cultural validity of the EIS responses were examined using confirmatory factor analytical procedures. The results supported the concurrent validity, an adequate unidimensional factor structure for the translated EIS and the internal reliability and test-retest reliability over a 6-week interval. Further, cross-gender configural, partial metric, partial strong factorial, and partial strict factorial invariance and cross-cultural configural and partial metric invariance supported the cross-cultural equivalence of the EIS versions. Moreover, the external validity of the translated EIS responses was also supported. Overall, the findings supported the validity of the exercise identity construct outside North American boundaries and the EIS items' equivalence, providing initial evidence for its cross-cultural applicability.

Psychometric Evaluation of the Exercise Identity Scale Among Greek Adults and Cross-Cultural Validity

The present study reported on translating the Exercise Identity Scale (EIS: Anderson & Cychosz, 1994) into Greek and examining its psychometric properties and cross-cultural validity based on U.S. individuals' EIS responses. Using four samples comprising 33, 103, and 647 Greek individuals, including exercisers and nonexercisers, and a similar sample comprising 800 U.S. individuals, the concurrent validity, factor structure, internal reliability, test-retest reliability, external validity, gender invariance, and cross-cultural validity of the EIS responses were examined using confirmatory factor analytical procedures. The results supported the concurrent validity, an adequate unidimensional factor structure for the translated EIS and the internal reliability and test-retest reliability over a 6-week interval. Further, cross-gender configural, partial metric, partial strong factorial, and partial strict factorial invariance and cross-cultural configural and partial metric invariance supported the cross-cultural equivalence of the EIS versions. Moreover, the external validity of the translated EIS responses was also supported. Overall, the findings supported the validity of the exercise identity construct outside North American boundaries and the EIS items' equivalence, providing initial evidence for its cross-cultural applicability.

Factorial Invariance of the PedsQL™ 4.0 Generic Core Scales Child Self-Report Across Gender: A Multigroup Confirmatory Factor Analysis with 11,356 Children Ages 5 to 18

The measurement of health-related quality of life (HRQOL) in pediatric medicine and health services research has grown significantly over the past decade. The paradigm shift toward patient-reported outcomes (PROs) in clinical trials and population studies has provided the opportunity to emphasize the value and critical need for pediatric patient self-report. In order for HRQOL/PRO outcome comparisons to be meaningful for gender group analyses, it is essential to demonstrate the factorial invariance of an instrument. This study examined factorial invariance of the 23-item PedsQL™ 4.0 Generic Core Scales across gender groups for child self-report ages 5–18. Multigroup Confirmatory Factor Analysis (MGCFA) was performed specifying a five-factor model across gender groups. The analyses were based on 11,356 children recruited from clinic, school, and community populations. Strict factorial invariance across gender groups was demonstrated based on stability of the Comparative Fit Index (CFI) between the models, and indices of practical fit including the Root Mean Squared Error of Approximation (RMSEA), the Non-Normed Fit Index (NNFI), and the Parsimony Normed Fit Index (PNFI). The findings support an equivalent five-factor structure on the PedsQL™ 4.0 Generic Core Scales across gender groups. Based on these data, it can be concluded that boys and girls studied interpreted items on the PedsQL™ 4.0 Generic Core Scales in a similar manner. These findings have implications for studies of gender differences in health outcomes.

Invariance in Measurement and Prediction Revisited

Borsboom (Psychometrika, 71:425–440, 2006) noted that recent work on measurement invariance (MI) and predictive invariance (PI) has had little impact on the practice of measurement in psychology. To understand this contention, the definitions of MI and PI are reviewed, followed by results on the consistency between the two forms of invariance in the general case. The special parametric cases of factor analysis (strict factorial invariance) and linear regression analyses (strong regression invariance) are then described, along with findings on the inconsistency between the two forms of invariance in this context. Two numerical examples of inconsistency are reviewed in detail. The impact of violations of MI on accuracy of selection is illustrated. Finally, reasons for the slow dissemination of work on invariance are discussed, and the prospects for altering this situation are weighed.

A Comparison of Latent Growth Models for Constructs Measured by Multiple Items

Univariate latent growth modeling (LGM) of composites of multiple items (e.g., item means or sums) has been frequently used to analyze the growth of latent constructs. This study evaluated whether LGM of composites yields unbiased parameter estimates, standard errors, chi-square statistics, and adequate fit indexes. Furthermore, LGM was compared with 2 alternatives: LGM with fixed error variances and the curve-of-factors model (McArdle, 1988), which is a multivariate latent growth model. It was found that the 2 univariate models only yield adequate results when the items are essentially tau-equivalent and there is strict factorial invariance. The curve-of-factors model was found to produce adequate results in all conditions, but it usually requires large sample sizes.

Cross‐Sectional Age Differences and Longitudinal Age Changes of Personality in Middle Adulthood and Old Age

The present study examines different aspects of personality continuity (or change) in middle adulthood and old age both cross-sectionally and longitudinally. The sample comprised 445 middle-aged (42-46 years) and 420 older (60-64 years) participants, reassessed after a 4-year interval. Personality was measured using the NEO-FFI personality inventory. After having established strict factorial invariance, factor covariances were found to be equal for both age groups and at both testing occasions, indicating perfect structural continuity of personality. A number of age differences in personality emerged at both measurement occasions. Longitudinally, in both age groups, an average decline in Neuroticism was observed. Longitudinal stability coefficients were around .80 in middle-aged and old participants, implying high, but not perfect, differential continuity. With respect to continuity of divergence, statistically significant cross-sectional age differences were found for the variance of Openness at both measurement occasions. Eventually, concerning specific versus general continuity, a variety of medium effect-sized correlated changes in the Big Five personality domains across the 4-year period was established, implying that personality changes share a certain amount of commonality.

An Essay on Measurement and Factorial Invariance

Analysis of subgroups such as different ethnic, language, or education groups selected from among a parent population is common in health disparities research. One goal of such analyses is to examine measurement equivalence, which includes both qualitative review of the meaning of items as well as quantitative examination of different levels of factorial invariance and differential item functioning. The purpose of this essay is to review the definitions and assumptions associated with factorial invariance, placing this formulation in the context of bias, fairness, and equity. The connection between the concepts of factorial invariance and item bias (differential item functioning) using a variant of item response theory is discussed. The situations under which different forms of invariance (weak, strong, and strict) are required are discussed. Establishing factorial invariance involves a hierarchy of levels that include tests of weak, strong, and strict invariance. Pattern (metric or weak) factorial invariance implies that the regression slopes are invariant across groups. Pattern invariance requires only invariant factor loadings. Strong factorial invariance implies that the conditional expectation of the response, given the common and specific factors, is invariant across groups. Strong factorial invariance requires that specific factor means (represented as invariant intercepts) also be identical across groups. Strict factorial invariance implies that, in addition, the conditional variance of the response, given the common and specific factors, is invariant across groups. Strict factorial invariance requires that, in addition to equal factor loadings and intercepts, the residual (specific factor plus error variable) variances are equivalent across groups. The concept of measurement invariance that is most closely aligned to that of item response theory considers the latent variable as a common factor measured by manifest variables; the specific factors can be characterized as nuisance variables. Invariance of factor loadings across studied groups is required for valid comparisons of scale score or latent variable means. Strong and strict invariance may be less important in the context of basic research in which group differences in specific factors are indicative of individual differences that are important for scientific exploration. However, for most applications in which the aim is to ensure fairness and equity, strict factorial invariance is required. Health disparities research often focuses on self-reported clinical outcomes such as quality of life that are not observed directly. Latent variable models such as factor analyses are central to establishing valid assessment of such outcomes.

Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework.

Comparative public health research makes wide use of self-report instruments. For example, research identifying and explaining health disparities across demographic strata may seek to understand the health effects of patient attitudes or private behaviors. Such personal attributes are difficult or impossible to observe directly and are often best measured by self-reports. Defensible use of self-reports in quantitative comparative research requires not only that the measured constructs have the same meaning across groups, but also that group comparisons of sample estimates (eg, means and variances) reflect true group differences and are not contaminated by group-specific attributes that are unrelated to the construct of interest. Evidence for these desirable properties of measurement instruments can be established within the confirmatory factor analysis (CFA) framework; a nested hierarchy of hypotheses is tested that addresses the cross-group invariance of the instrument's psychometric properties. By name, these hypotheses include configural, metric (or pattern), strong (or scalar), and strict factorial invariance. The CFA model and each of these hypotheses are described in nontechnical language. A worked example and technical appendices are included.

Analysis of factorial invariance across gender in the Taiwan version of the Satisfaction with Life Scale

The purpose of this study was to examine factorial invariance of the 5-item Satisfaction with Life Scale-Taiwan version (SWLS) across gender among university students in Taiwan. A total of 476 undergraduate students (males = 207 and females = 269) at National Taiwan University completed the Taiwan version of the SWLS. Prior to any invariance analysis, one-factor and two-factor models ( present and past factors) were applied to each group. The confirmatory factor analysis results revealed that both the one-factor model and the two-factor model were acceptable for each group. However, since the “present” and “past” factors were highly correlated in both groups (>0.90), the one-factor model was accepted upon consideration of the principle of parsimony. Further, based on the one-factor model, results from multi-sample analysis revealed that the Taiwan version of the SWLS has the property of strict factorial invariance across gender, including invariance of factor loadings, unique variances and factor variance.

Testing dynamic relations between perceived competence and fear of failure in young athletes

Conceptually, achievement motives are stable phenomena; however, recent evidence suggests that fear of failure (FF) levels decrease slightly as participants gain experience with an activity. One plausible explanation for this decrease in FF is that practice (and consequential reinforcement and approval from coaches) leads to increased perceptions of competence and concomitant reductions in the perceived likelihood and aversiveness of failing. Boys and girls in a recreational summer swimming program ( N = 165) completed measures of FF and perceived competence (PC) at the beginning, middle, and end of a 6-week swimming season. The FF and PC measures exhibited strong and strict factorial invariance, respectively. As expected, FF scores exhibited a slight but significant decrease over the season whereas PC growth trajectories varied significantly. Neither initial levels of nor subsequent changes in PC significantly predicted the rate at which FF scores changed over six weeks. These findings indicated that changes in PC do not confound evaluations of change in FF during youth sport seasons.

Weak measurement invariance with respect to unmeasured variables: An implication of strict factorial invariance

Measurement bias refers to systematic differences across subpopulations in the relation between observed test scores and the latent variant underlying the test scores. Comparisons of subpopulations with the same score on the latent variable can be expected to have the same observed test score. Measurement invariance is therefore one of the key issues in psychological testing. It has been established that strict factorial invariance (SFI) with respect to a selection variable V almost certainly implies weak measurement invariance with respect to V: given SFI, means and variances of observed scores do not depend on V. It is shown that this result can be extended. SFI in groups derived by selection on V has implications not only for V but also for potentially biasing variables W, if W and the selection variable V and/or if W and the factor underlying the observed test scores are statistically dependent. Given SFI with respect to V and prior knowledge concerning these dependencies, it is not necessary to measure and model variables W in order to exclude them as potentially biasing variables if the investigation focuses on groups selected on V.

Factor Structure of the Spanish Version of the Ways of Coping Questionnaire

A Spanish translation of the Ways of Coping (WOC‐S) questionnaire was administered to three samples of Latinos sampled from Mexico City, Puerto Rico, and the San Francisco Peninsula, respectively. The factor structure of the WOC‐S was assessed with both exploratory and confirmatory factor analyses. The factor structure that emerged was found to have important similarities and differences compared to that previously reported for members of the predominant United States culture. Across the sampled Latino groups, the WOC‐S exhibited evidence of both strong and strict factorial invariance (Meredith, 1993), suggesting it is appropriate for cross‐cultural research assessing coping within diverse Latino populations.

Investigating Spearman's Hypothesis by Means of Multi-Group Confirmatory Factor Analysis

Differences between blacks and whites on cognitive ability tests have been attributed to a fundamental difference between these groups in general intelligence (or g, as it is denoted). The hypothesized difference in g gives rise to Spearman's hypothesis, which states that the differences in the means of the tests are related to the tests' factor loadings on g. Jensen has investigated this hypothesis by correlating differences in means and tests' g loadings. The aim of the present article is to investigate B-W differences using multi-group confirmatory factor analysis. The advantages of multi-group confirmatory factor analysis over Jensen's test of Spearman's hypothesis are discussed. A published data set is analyzed. Strict factorial invariance is tested and judged to be tenable. Various models are tested, which do and do not incorporate g. It is observed that it is difficult to distinguish between several hypotheses, including and excluding g, concerning group differences. The inability to distinguish between competing models using multi-group confirmatory factor analysis makes it difficult to draw clear conclusions about the exact nature of black-white differences in cognitive abilities. The implications of the results for Jensen's test of Spearman's hypothesis are discussed.

A robust five-factor structure of the 16PF: Strong evidence from independent rotation and confirmatory factorial invariance procedures

Establishing factorial invariance across samples and forms has important implications for efforts to build a taxonomy of personality traits. These methods provide a strong test of measurement properties and, if factorial invariance obtains, allow meaningful factor comparisons across groups. Results are presented for an invariant second-order factor structure in two forms (Form C and Clinical Analysis Questionnaire) of the 16PF and across six independent samples: four samples consisting of male and female police applicants (combined N = 15,332) and two samples of male felons (combined N = 15,460). Independently rotated common factor analyses indicated that six previously reported factors (Extraversion, Anxiety, Control, Independence, Sensitive Awareness, and Intelligence) were well-replicated in all samples. A restricted factor solution (salient loadings only) with identical factor loadings (metric factorial invariance) was found to hold remarkably well across all samples. Further constraints on the factor model known as ‘strict factorial invariance’ (Meredith, 1993; invariant factor loadings, mean intercepts, and unique variances) were found to provide a good fit to the data within police applicant and felon samples. The findings from this study offer strong evidence in support of a stable five-factor personality structure of the 16PF across different forms and people.

Measurement invariance, factor analysis and factorial invariance

Several concepts are introduced and defined: measurement invariance, structural bias, weak measurement invariance, strong factorial invariance, and strict factorial invariance. It is shown that factorial invariance has implications for (weak) measurement invariance. Definitions of fairness in employment/admissions testing and salary equity are provided and it is argued that strict factorial invariance is required for fairness/equity to exist. Implications for item and test bias are developed and it is argued that item or test bias probably depends on the existence of latent variables that are irrelevant to the primary goal of test constructers.