Psychometric Properties of the Modified Illinois Bully/Victim/Fight Scale for Youth in Restrictive Education Settings

Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), path analysis, and structural equation modeling (SEM) have long histories in clinical research. Although CFA has largely superseded EFA, CFAs of multidimensional constructs typically fail to meet standards of good measurement: goodness of fit, measurement invariance, lack of differential item functioning, and well-differentiated factors in support of discriminant validity. Part of the problem is undue reliance on overly restrictive CFAs in which each item loads on only one factor. Exploratory SEM (ESEM), an overarching integration of the best aspects of CFA/SEM and traditional EFA, provides confirmatory tests of a priori factor structures, relations between latent factors and multigroup/multioccasion tests of full (mean structure) measurement invariance. It incorporates all combinations of CFA factors, ESEM factors, covariates, grouping/multiple-indicator multiple-cause (MIMIC) variables, latent growth, and complex structures that typically have required CFA/SEM. ESEM has broad applicability to clinical studies that are not appropriately addressed either by traditional EFA or CFA/SEM.

Coaching competency and (exploratory) structural equation modeling: A substantive-methodological synergy

Allen and Meyer's(1990) 3-component model of organizational commitment(OC) was investigated using exploratory structural equation modeling(ESEM) with samples of full-time social workers at social welfare organizations in South Korea. Confirmatory factor analysis(CFA) has been at the heart of testing factor structure of the 3-component model in organizational commitment research wherein each scale of the affective, continuance, and normative commitment is reported to measure conceptually and empirically separable construct. The present study applied ESEM, specifically ‘intra-scale’ and ‘inter-scale’ ESEM, to scrutinize factor structure of the 3-component scales. ESEM methodology uses exploratory approach in that all cross-loadings are estimated between each measure and factors, with uniqueness correlated according to the researcher's hypotheses as in CFA. In this respect, ESEM can be viewed as an open approach to item analysis distinguished from the conventional (closed) approach, such as EFA and CFA. This study provided detailed assessment of the 3-component model through comparisons of factor structures estimated by EFA, CFA and ‘intra’ ESEM, followed by the ‘inter’ ESEM conducted on all other variables(assumed to be similar constructs with or antecedents of OC). As a result, the ‘intra-scale’ ESEM showed a substantially better fit and yielded more discriminated factors(less correlated) than did EFA and CFA that are models for planned scale. The ‘inter-scale’ ESEM revealed how seriously method effect can distort an original factor structure in empirical data measured together with multiple scales of other constructs. Using ESEM has advantages of estimating common factor structures, controlling for common method effect that are typically included in measures in applied research. Also, it allows for much more possibilities that each item can measure multiple constructs so as to reveal more realistic factor structures. Taken together, the present results suggest a need to conceptualize and validate a new scale for organizational commitment reflecting Korean culture.

Allen and Meyer's(1990) 3-component model of organizational commitment(OC) was investigated using exploratory structural equation modeling(ESEM) with samples of full-time social workers at social welfare organizations in South Korea. Confirmatory factor analysis(CFA) has been at the heart of testing factor structure of the 3-component model in organizational commitment research wherein each scale of the affective, continuance, and normative commitment is reported to measure conceptually and empirically separable construct. The present study applied ESEM, specifically ‘intra-scale’ and ‘inter-scale’ ESEM, to scrutinize factor structure of the 3-component scales. ESEM methodology uses exploratory approach in that all cross-loadings are estimated between each measure and factors, with uniqueness correlated according to the researcher's hypotheses as in CFA. In this respect, ESEM can be viewed as an open approach to item analysis distinguished from the conventional (closed) approach, such as EFA and CFA. This study provided detailed assessment of the 3-component model through comparisons of factor structures estimated by EFA, CFA and ‘intra’ ESEM, followed by the ‘inter’ ESEM conducted on all other variables(assumed to be similar constructs with or antecedents of OC). As a result, the ‘intra-scale’ ESEM showed a substantially better fit and yielded more discriminated factors(less correlated) than did EFA and CFA that are models for planned scale. The ‘inter-scale’ ESEM revealed how seriously method effect can distort an original factor structure in empirical data measured together with multiple scales of other constructs. Using ESEM has advantages of estimating common factor structures, controlling for common method effect that are typically included in measures in applied research. Also, it allows for much more possibilities that each item can measure multiple constructs so as to reveal more realistic factor structures. Taken together, the present results suggest a need to conceptualize and validate a new scale for organizational commitment reflecting Korean culture.

Fatigue is among the most disabling symptoms in patients with multiple sclerosis (PwMS). The common distinction between cognitive and motor fatigue is typically incorporated in self-rating instruments, such as the Chalder Fatigue Questionnaire (CFQ), the Fatigue Scale for Motor and Cognitive Functions (FSMC) or the Modified Fatigue Impact Scale (MFIS). The present study investigated the factor structure of the CFQ, the FSMC and the MFIS utilizing exploratory (EFA) and confirmatory factor analysis (CFA) as well as exploratory structural equation modeling (ESEM). Data of 1.403 PwMS were analyzed, utilizing four samples. The first sample (N = 605) was assessed online and split into two stratified halves to perform EFA, CFA, and ESEM on the CFQ and FSMC. The second sample (N = 293) was another online sample. It served to calculate CFA and ESEM on the CFQ and FSMC. The third sample was gathered in a clinical setting (N = 196) and analyzed by applying CFA and ESEM to the FSMC. The fourth sample (N = 309) was assessed in a clinical setting and allowed to run a CFA and ESEM on the MFIS. Proposed factor structures of all questionnaires were largely confirmed in EFA. However, none of the calculated CFAs and ESEMs could verify the proposed factor structures of the three measures, even with oblique rotation techniques. The findings might have implications for future research into the pathophysiological basis of MS-related fatigue and could affect the suitability of such measures as outcomes for treatment trials, presumably targeting specific sub-components of fatigue.

While exploratory factor analysis (EFA) provides a more realistic presentation of the data with the allowance of item cross-loadings, confirmatory factor analysis (CFA) includes many methodological advances that the former does not. To create a synergy of the two, exploratory structural equation modeling (ESEM) was proposed as an alternative solution, incorporating the advantages of EFA and CFA. The present investigation is thus an illustrative demonstration of the applicability and flexibility of ESEM. To achieve this goal, we compared CFA and ESEM models, then thoroughly tested measurement invariance and differential item functioning through multiple-indicators-multiple-causes (MIMIC) models on the Passion Scale, the only measure of the Dualistic Model of Passion (DMP) which differentiates between harmonious and obsessive forms of passion. Moreover, a hybrid model was also created to overcome the drawbacks of the two methods. Analyses of the first large community sample (N = 7,466; 67.7% females; Mage = 26.01) revealed the superiority of the ESEM model relative to CFA in terms of improved goodness-of-fit and less correlated factors, while at the same time retaining the high definition of the factors. However, this fit was only achieved with the inclusion of three correlated uniquenesses, two of which appeared in previous studies and one of which was specific to the current investigation. These findings were replicated on a second, comprehensive sample (N = 504; 51.8% females; Mage = 39.59). After combining the two samples, complete measurement invariance (factor loadings, item intercepts, item uniquenesses, factor variances-covariances, and latent means) was achieved across gender and partial invariance across age groups and their combination. Only one item intercept was non-invariant across both multigroup and MIMIC approaches, an observation that was further corroborated by the hybrid model. While obsessive passion showed a slight decline in the hybrid model, harmonious passion did not. Overall, the ESEM framework is a viable alternative of CFA that could be used and even extended to address substantially important questions and researchers should systematically compare these two approaches to identify the most suitable one.

Boredom has recently become the subject of inquiry in L2 studies, which has resulted, among others, in the development and validation of several boredom-measuring scales, mostly through exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). However, such analytical procedures are not free from limitations and exploratory structural equation modeling (ESEM) can be applied to overcome them. CFA has also been used to validate the Boredom in Learning English Outside of School Questionnaire (BLEOS), an instrument which taps into recently conceptualized after-class boredom, revealing the extent to which students are likely to feel bored when they attempt to practice the target language outside the classroom as well as factors underpinning this negative emotion in such contexts. The present study sought to revisit the BLEOS scale to determine (1) whether the factor structure of the BLEOS is supported by ESEM and bifactor ESEM, (2) how stable the results are across gender, and (3) the criterion-related validity of the BLEOS scale. In doing so, the CFA and ESEM models (i.e., standard and bifactor CFA as well as standard and bifactor ESEM models) were compared to identify which model(s) would exhibit better fit indices for the measure in question. A total of 433 students majoring in English, 120 males and 313 females, participated in this study. It was revealed that while the CFA model provided an inflated factor correlation and unacceptable model fit to the data, ESEM (especially bifactor ESEM) indicated a realistic representation of the data with a good fit. The bi-factor ESEM model of the BLEOS turned out to be stable across gender. Finally, the structural bifactor ESEM model of BLEOS proved to explain learning outcomes modestly.

This study is a methodological-substantive synergy, demonstrating the power and flexibility of exploratory structural equation modeling (ESEM) methods that integrate confirmatory and exploratory factor analyses (CFA and EFA), as applied to substantively important questions based on multidimentional students' evaluations of university teaching (SETs). For these data, there is a well established ESEM structure but typical CFA models do not fit the data and substantially inflate correlations among the nine SET factors (median rs = .34 for ESEM, .72 for CFA) in a way that undermines discriminant validity and usefulness as diagnostic feedback. A 13-model taxonomy of ESEM measurement invariance is proposed, showing complete invariance (factor loadings, factor correlations, item uniquenesses, item intercepts, latent means) over multiple groups based on the SETs collected in the first and second halves of a 13-year period. Fully latent ESEM growth models that unconfounded measurement error from communality showed almost no linear or quadratic effects over this 13-year period. Latent multiple indicators multiple causes models showed that relations with background variables (workload/difficulty, class size, prior subject interest, expected grades) were small in size and varied systematically for different ESEM SET factors, supporting their discriminant validity and a construct validity interpretation of the relations. A new approach to higher order ESEM was demonstrated, but was not fully appropriate for these data. Based on ESEM methodology, substantively important questions were addressed that could not be appropriately addressed with a traditional CFA approach.

NEO instruments are widely used to assess Big Five personality factors, but confirmatory factor analyses (CFAs) conducted at the item level do not support their a priori structure due, in part, to the overly restrictive CFA assumptions. We demonstrate that exploratory structural equation modeling (ESEM), an integration of CFA and exploratory factor analysis (EFA), overcomes these problems with responses (N = 3,390) to the 60-item NEO-Five-Factor Inventory: (a) ESEM fits the data better and results in substantially more differentiated (less correlated) factors than does CFA; (b) tests of gender invariance with the 13-model ESEM taxonomy of full measurement invariance of factor loadings, factor variances-covariances, item uniquenesses, correlated uniquenesses, item intercepts, differential item functioning, and latent means show that women score higher on all NEO Big Five factors; (c) longitudinal analyses support measurement invariance over time and the maturity principle (decreases in Neuroticism and increases in Agreeableness, Openness, and Conscientiousness). Using ESEM, we addressed substantively important questions with broad applicability to personality research that could not be appropriately addressed with the traditional approaches of either EFA or CFA.

Revisiting boredom in practical English language classes via exploratory structural equation modeling

This special issue of Education and Treatment of Children explores the use of positive behavior interventions and supports (PBIS) as a means of prevention for high-risk youth being served in non-traditional, more restrictive educational settings including alternative education (AE), residential, and juvenile justice (JJ) settings. PBIS is a multi-tiered framework differentiating interventions and intensity of delivery based on student needs and data; and is applicable across all education settings. Currently, the PBIS framework has been implemented in many traditional education settings and has recently been adopted and adapted to non-traditional settings. Youth within these settings present a wide range of academic and behavioral deficits and excesses that could benefit from the tiered support within the PBIS framework. The goal of this special issue is to provide empirical and practical information on the PBIS framework to educators and a wide-range of service providers (e.g., behavior specialists, counselors, mental health, advocacy, and policy organization personnel) who work with high-risk youth in AE, residential, and JJ settings to improve youth outcomes and teacher effectiveness. In an effort to (a) support the continued and extended use of PBIS in these settings; (b) focus on the utility of PBIS as a method of prevention, in particular as a means to address the school-to-prison pipeline phenomenon of more restrictive placements and possible incarceration; and (c) provide resources and research directions for the field, we have assembled articles that address implementation of PBIS in AE settings across the tiers and provide lessons learned from research and implementation. To begin, we offer articles that provide an overview of the overarching goals of and need for PBIS in restrictive educational settings. Simonsen and Sugai offer a rationale for why PBIS is needed in restrictive educational settings by linking the broader PBIS literature to the needs of high-risk youth in these settings and how interventions can be intensified across the three tiers. To further contextualize this need, Benner and colleagues offer support for how PBIS can be used to bridge the achievement gap between high-risk youth and their typically developing peers through the use of effective instructional practices. Swain-Bradway and colleagues present common facilitators and barriers from stakeholder interviews of administrators and PBIS team members currently implementing PBIS in AE, residential, and JJ settings to guide future implementation of PBIS across these settings. Next, in an effort to provide empirical support and lessons learned from implementing PBIS across the tiers, we offer examples at the primary (tier I), secondary (tier II), and tertiary (tier III) tiers. …

The most popular measures of multidimensional constructs typically fail to meet standards of good measurement: goodness of fit, measurement invariance, lack of differential item functioning, and well-differentiated factors that are not so highly correlated as to detract from their discriminant validity. Part of the problem, the authors argue, is undue reliance on overly restrictive independent cluster models of confirmatory factor analysis (ICM-CFA) in which each item loads on one, and only one, factor. Here the authors demonstrate exploratory structural equation modeling (ESEM), an integration of the best aspects of CFA and traditional exploratory factor analyses (EFA). On the basis of responses to the 11-factor Motivation and Engagement Scale ( n = 7,420, Mage = 14.22), we demonstrate that ESEM fits the data much better and results in substantially more differentiated (less correlated) factors than corresponding CFA models. Guided by a 13-model taxonomy of ESEM full-measurement (mean structure) invariance, the authors then demonstrate invariance of factor loadings, item intercepts, item uniquenesses, and factor variancescovariances, across gender and over time. ESEM has broad applicability to other areas of research that cannot be appropriately addressed with either traditional EFA or CFA and should become a standard tool for use in psychometric tests of psychological assessment instruments.

The Statistics Anxiety Rating Scale (STARS) is a 51-item scale commonly used to measure college students’ anxiety regarding statistics. To date, however, limited empirical research exists that examines statistics anxiety among ethnically diverse or first-generation graduate students. We examined the factor structure and reliability of STARS scores in a diverse sample of students enrolled in graduate courses at a Minority-Serving Institution (n = 194). To provide guidance on assessing dimensionality in small college samples, we compared the performance of best-practice factor analysis techniques: confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and Bayesian structural equation modeling (BSEM). We found modest support for the original six-factor structure using CFA, but ESEM and BSEM analyses suggested that a four-factor model best captures the dimensions of the STARS instrument within the context of graduate-level statistics courses. To enhance scale efficiency and reduce respondent fatigue, we also tested and found support for a reduced 25-item version of the four-factor STARS scale. The four-factor STARS scale produced constructs representing task and process anxiety, social support avoidance, perceived lack of utility, and mathematical self-efficacy. These findings extend the validity and reliability evidence of the STARS inventory to include diverse graduate student populations. Accordingly, our findings contribute to the advancement of data science education and provide recommendations for measuring statistics anxiety at the graduate level and for assessing construct validity of psychometric instruments in small or hard-to-survey populations.

BackgroundAdolescents’ body image concerns are related to their mental and physical health. Reliable instruments are fundamental to understanding body image concerns, but there are some concerns about the adolescent body image instruments currently used in China. One of the obvious concerns is the inadequate psychometric properties of the extant Chinese translations of the Body Esteem Scale for Adolescents and Adults (BESAA), such as the lack of evaluation of factorial validity and construct validity. BESAA is an instrument that encompasses partly positive body image and partly negative body image, and the broad conceptualization of appearance is considered one of its unique values. The main purpose of this study was to translate the BESAA into Chinese and to preliminarily evaluate its reliability and validity in a sample of Chinese adolescents.MethodsA total of 1368 adolescents (age: mean = 14.94, standard deviation = 2.08; 70.18% females, 29.82% males) were recruited through a convenience sampling method. Exploratory Factor Analysis (EFA), Confirmatory Factor Analysis (CFA), Exploratory Structural Equation Modelling (ESEM), test–retest reliability, convergent validity, measurement invariance, and McDonald’s Omega were used to evaluate the psychometric properties of the Chinese version of the BESAA. The total sample was used for the item analysis, convergent validity, and reliability. The total sample was randomly divided into two equal-sized subsamples; Sample 1 (n = 684; girls n = 459, boys n = 225) was used for the EFA, while Sample 2 (n = 684; girls n = 501, boys n = 183) was used for the CFA, the ESEM, and the measurement invariance.ResultsA three-factor, 15-item model was obtained through EFA. The model showed satisfactory goodness of fit in terms of both CFA and ESEM (χ2 = 172.071, df = 63, χ2/df = 2.73 (p < 0.001), Comparative fit index (CFI) = 0.972, Tucker-Lewis index (TLI) = 0.953, Root mean square error of approximation (RMSEA) = 0.050, and Standardized root mean residual (SRMR) = 0.024). The Chinese version of the BESAA demonstrated good internal consistency (McDonald’s Omega: 0.881), and its convergent validity is supported.ConclusionThe current findings provide evidence for the validity and reliability of the Chinese version of the BESAA, and support the use of the BESAA among Chinese adolescents.

Traditionally, assessments of factor validity of body image instruments have relied on exploratory or confirmatory factor analysis. However, the emergence of exploratory structural equation modeling (ESEM), a resurgence of interest in bifactor models, and the ability to combine both models (bifactor-ESEM) is beginning to shape the future of body image research. For these analytic approaches to truly advance body image research, scholars will need to have a deep understanding of their use and application. To facilitate such understanding, we describe ESEM and bifactor-ESEM models for body image researchers and provide them with the tools they need to apply these methods in their own work. Specifically, we provide an overview of ESEM and bifactor-ESEM models, and describe their broad applicability to body image research. Next, we describe how ESEM and bifactor models can be used and, using an existing dataset of responses to the Acceptance of Cosmetic Surgery Scale, demonstrate how ESEM and bifactor-ESEM models can be deployed. To facilitate wider application of these ideas, we provide our Mplus syntax (inputs) in Supplementary Materials. Through this manuscript, we hope to assist researchers to better understand the strengths ESEM and bifactor models, and to use these approaches in their own work.