Multiple-group Confirmatory Factor Analysis Research Articles

Examining Appropriacy of CFI and TLI Cutoff Value in Multiple-Group CFA Test of Measurement Invariance to Enhance Accuracy of Test Score Interpretation

The most common effect size when using a multiple-group confirmatory factor analysis approach to measurement invariance is ΔCFI and ΔTLI with a cutoff value of 0.01. However, this recommended cutoff value may not be ubiquitously appropriate and may be of limited application for some tests (e.g., measures using dichotomous items or different estimation methods, sample sizes, or model complexity). Moreover, prior cutoff value estimations often have ignored consequences resulting in using measures that more accurately estimate countries’ or learners’ proficiency for some countries or groups versus others. In this study, we investigate whether the cutoff value proposed by Cheung and Rensvold (ΔCFI or ΔTLI > 0.01) is appropriate across educational measurement contexts. Specifically, we investigated the performance of ΔCFI and ΔTLI in capturing LOI at the scalar level in dichotomous items within item response theory on groups whose test characteristic curves differed by 0.5. Simulation results showed that the proposed cutoff value of 0.01 in ΔCFI and ΔTLI was not appropriate to capture LOI under the study conditions, which may result in the misinterpretation of test results or inaccurate inferences.

Investigation of The Measurement Invariance of Affective Characteristics Related to TIMSS 2019 Mathematics Achievement by Gender

This research examines whether the affective characteristics of the TIMSS 2019 Turkey mathematics application provide measurement invariance according to gender. The research sample consists of 4048 8th-grade students participating in the TIMSS in 2019. Research data were downloaded from the international website of TIMSS. The research data collection tools are “Sense of School Belonging”, “Students Confident in Mathematics”, “Students Like Learning Mathematics”, and “Students Value Mathematics” scales. Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) were performed in the context of validity analyses to examine measurement invariance. In terms of reliability, the Cronbach Alfa internal consistency coefficient was calculated. Accordingly, out of the four scales in the study, only “Students Confident in Mathematics” scale could not be confirmed in confirmatory factor analysis. Therefore, while “Students Confident in Mathematics” scale was not examined for measurement invariance, the other three scales were examined within the scope of measurement invariance. For measurement invariance, research data were tested with Multiple Group Confirmatory Factor Analysis (MG-CFA), one of the Structural Equation Modeling (SEM) techniques. As a result of the analyses, while the strict invariance model was provided in “Students Like Learning Mathematics” scale and “Students Value Mathematics” scale, strong invariance/scale invariance model was provided in “Sense of School Belonging” scale. It was concluded that there was no gender bias in the three scales for which MG-CFA was performed, and the mean scores were comparable according to gender. In this context, it can be said that “Sense of School Belonging”, “Students Like Learning Mathematics”, and “Students Value Mathematics” scales are valid in determining the differences according to gender.

Validating a Short Version of the Intolerance of Uncertainty Scale-12 (IUS-12)

Background: This study aims to validate the short version of the Intolerance of Uncertainty Scale-12 (IUS-12).Methods: A comprehensive set of questionnaires, including the IUS-12, STAI-T, PSWQ, CES-D, and K-CSI, was administered and completed by 404 participants. The average age of the participants was 31.29 years (SD=9.18), comprising 114 men, 287 women, and 3 individuals identifying with the other gender. We conducted a factor analysis on the collected data and compared the fit indices of one-factor, two-factor, and bifactor models. Furthermore, a multiple-group confirmatory factor analysis evaluated measurement invariance between the men’s and women’s groups. Correlational analyses were also conducted.Results: Data from the study show that the bifactor model was the most appropriate, demonstrating good internal consistency. The Omega coefficients and explained common variance were computed to evaluate the dimensionality of the IUS-12, validating the use of the total score of the scale. Its factor model also showed that the measurement invariance sustained between two gender groups. The correlation analyses between the IUS-12 and other scales assessing attributes such as trait anxiety, worry, depression, and avoidance coping strategy supported convergent validity.Conclusions: These findings suggest that the IUS-12 is a reliable and valid measure for assessing the intolerance of uncertainty. The study also discusses its limitations and offers recommendations for future research endeavors.

Ensuring cross-cultural data comparability by means of anchoring vignettes in heterogeneous refugee samples

BackgroundConfigural, metric, and scalar measurement invariance have been indicators of bias-free statistical cross-group comparisons, although they are difficult to verify in the data. Low comparability of translated questionnaires or the different understanding of response formats by respondents might lead to rejection of measurement invariance and point to comparability bias in multi-language surveys. Anchoring vignettes have been proposed as a method to control for the different understanding of response categories by respondents (the latter is referred to as differential item functioning related to response categories or rating scales: RC-DIF). We evaluate the question whether the cross-cultural comparability of data can be assured by means of anchoring vignettes or by considering socio-demographic heterogeneity as an alternative approach.MethodsWe used the Health System Responsiveness (HSR) questionnaire and collected survey data in English (n = 183) and Arabic (n = 121) in a random sample of refugees in the third largest German federal state. We conducted multiple-group Confirmatory Factor Analyses (MGCFA) to analyse measurement invariance and compared the results when 1) using rescaled data on the basis of anchoring vignettes (non-parametric approach), 2) including information on RC-DIF from the analyses with anchoring vignettes as covariates (parametric approach) and 3) including socio-demographic covariates.ResultsFor the HSR, every level of measurement invariance between the Arabic and English languages was rejected. Implementing rescaling or modelling on the basis of anchoring vignettes provided superior results over the initial MGCFA analysis, since configural, metric and – for ordered categorical analyses—scalar invariance could not be rejected. A consideration of socio-demographic variables did not show such an improvement.ConclusionsSurveys may consider anchoring vignettes as a method to assess cross-cultural comparability of data, whereas socio-demographic variables cannot be used to improve data comparability as a standalone method. More research on the efficient implementation of anchoring vignettes and further development of methods to incorporate them when modelling measurement invariance is needed.

Measurement Invariance of the Neurobehavioral Symptom Inventory in Male and Female Million Veteran Program Enrollees Completing the Comprehensive Traumatic Brain Injury Evaluation.

This study evaluated measurement invariance across males and females on the Neurobehavioral Symptom Inventory (NSI) in U.S. military veterans enrolled in the VA Million Veteran Program. Participants (N = 17,059; males: n = 15,450; females: n = 1,609) included Veterans who took part in the VA Traumatic Brain Injury (TBI) Screening and Evaluation Program and completed the NSI. Multiple-group confirmatory factor analyses investigated measurement invariance of the NSI 4-factor model. The configural (comparative fit index [CFI] = 0.948, root mean square error of approximation [RMSEA] = 0.060) and metric (CFI = 0.948, RMSEA = 0.058) invariance models showed acceptable fit. There was a minor violation of scalar invariance (Δχ2 = 232.50, p < .001); however, the degree of noninvariance was mild (ΔCFI = -0.002, ). Our results demonstrate measurement invariance across sex, suggesting that the NSI 4-factor model can be used to accurately assess symptoms in males and females following TBI. Findings highlight the importance of considering validity of measurement across study groups to increase confidence that a measure is interpreted similarly by respondents from different subgroups.

&lt;p&gt;A bifactor-based Academic PsyCap Questionnaire for students and fresh graduates&lt;/p&gt;

Introduction. In the last two decades, great attention has been devoted to the four dimensions of the psychological capital (PsyCap): self-efficacy, hope, optimism, and resilience. These individual differences have been found to positively influence both academic and professional outcomes. This work aims to develop a bifactor-based version of the Academic PsyCap Questionnaire (APCQ), which is an instrument specifically meant for students and fresh graduates. Method. Cross-sectional data were collected in a large sample of Italian fresh graduates (N = 1,603). Data were analysed through exploratory and confirmatory factor analysis. The invariance of the APCQ across males and females and across bachelor and master graduates was tested through Multiple-Group Confirmatory Factor Analyses. The relationships of the scores at the APCQ and its four scales with the soft skills were explored through Pearson’s correlations. Results. The questionnaire includes 20 items that have been found to adequately assess the four dimensions of self-efficacy, resilience, optimism, and hope, as well as the general factor of psychological capital. The bifactor structure of the instrument is in line with the definition of PsyCap in the literature. The APCQ was found to be invariant across genders and levels of academic degrees. Moreover, the APCQ scales were found to be positively associated with a set of soft skills that are relevant to achieving academic and professional success. Discussion and Conclusion. The paper provides researchers, practitioners, and educators with a valid and reliable instrument that may help them to effectively evaluate PsyCap dimensions in students and fresh graduates in order to design effective interventions and tailored educational programs.

Measurement invariance in the assessment of parenting practices: A cross-cultural comparison of China and the United States.

Assessing parenting practices in a culturally informed manner is critical to clinical practice when working with families. Although many parenting measures have been translated into Chinese, limited evidence for measurement invariance is available. The present study aims to assess the measurement invariance of positive and negative parenting practices across Mandarin-speaking families living in Mainland China and English-speaking families living in the United States. Three thousand seven parents of children ages 6-12 years (770 English-speaking: parent Mage = 35.15 years, SD = 7.96; child Mage = 9.50 years, SD = 4.27; 2,237 Chinese-speaking: parent Mage = 38.46 years, SD = 4.42; child Mage = 9.40 years, SD = 1.78) completed the Multidimensional Assessment of Parenting Scale as a part of two separate research protocols. Multiple-group confirmatory factor analyses (CFAs) were used, and the source of invariance at the factor and item levels was examined. CFA revealed that a seven-factor solution was feasible across both samples, as evidenced by configural and metric invariance. We found a lack of scalar invariance; thus, we constructed a partial scalar invariance model and presented latent means, correlations, and variances of the seven subscales. Item-level parameter estimates and content analyses revealed potentially different item interpretations of the measure. The lack of scalar invariance suggests that researchers should not use mean differences (e.g., from simple t tests) for cross-cultural comparisons using common parenting questionnaires. Instead, we recommend analyzing data utilizing latent variable modeling (e.g., structural equation modeling) and future directions for improving measures as part of larger efforts for promoting inclusive parenting science. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

New cut-off points of PHQ-9 and its variants, in Costa Rica: a nationwide observational study

The PHQ-9 questionnaire is a screening test worldwide used to measure depression. But it cannot be used in Costa Rica, due to the fact that it has not previously been validated for its population. The present study aims to show the validation of the PHQ-9 questionnaire and its variants (PHQ-2, PHQ-4, PHQ-8) in a population sample of adults residing in Costa Rica. A sample was collected (n = 1162) using a self-administered questionnaire. Confirmatory Factor Analysis (CFA), Receiver Operating Characteristic (ROC) curve, and Multiple Group Confirmatory Factor Analysis (MGCFA) were tested. One factor was found that explained 73.33% of the variance with excellent internal consistency (α = 0.928). Goodness-of-fit measures were adequate (RMSEA = 0.107; CFI = 0.948), as was diagnostic power at a cut-off of 10 (78.60 for Sensitivity and 27.95 for 1-Specificity). External validation indices were good (r = 0.843 with GAD-7, r = − 0.647 with RS14, and r = 0.301 with FCV19S), and the model showed invariance by sex (∆χ2 = 27.90; df = 27; p < 0.001). Additionally, new cut-off points were proposed for PHQ-9 and its variants for Costa Rican male, female, and general populations. The PHQ-9 and its variants (PHQ-2, 4, and 8) are valid tools for detecting depression (and anxiety for PHQ-4) in Costa Rican population. In addition, new cut-off points differentiated by sex are proposed.

Cross-Country (Brazil and Iran) Invariance of Fractionation of Executive Functions in Early Adolescence

ABSTRACT Cultural background can influence cognition, including executive functions (EFs), abilities that encompass skills responsible for self-regulation of thoughts and behavior. The seminal unity and diversity model of EFs proposes the existence, in adulthood, of at least three correlated but separable EF latent (shared variance in more than one task/indicator) domains: inhibition, updating and shifting. However, evidence of the cross-cultural generality of this framework is lacking, especially in adolescence, an age during which these domains become more clearly separable. We tested whether this EF fractionation could be observed in early adolescents (9- to 15-year-olds) from metropolitan areas in Brazil (São Paulo) and Iran (Tehran) (total sample: 739; 407 Iranians; 358 girls). Participants carried out two open-access tasks that are representative of each EF domain and that were adapted to each cultural context. Seven latent model configurations were tested. The three-correlated latent factor structure had adequate fit, and multiple-group confirmatory factor analysis invariance testing showed invariance for country at the level of the latent factor structure (configural), factor loadings (metric), and partial invariance at the intercept (scalar) level. Iranians had higher scores in all domains. Multiple indicators multiple causes invariance testing showed model invariance across age (except for one task) and parental education. Performance in all domains improved with age and only minimally with parental schooling. We conclude that EF fractionation into three domains is present in the first half of adolescence in two samples from underrepresented populations in the literature, suggesting a potential generality of EF latent unity/diversity development at this age.

Development and initial validation of the Career Self-Management scale for Chinese coaches.

The purpose of the study was to develop and initially validate a context-specific scale assessing Career Self-Management for Chinese coaches (Career Self-Management Scale-CC; CSMS-CC). Firstly, qualitative data obtained from in-depth interview with coaches were contently analyzed to generate potential CSMS-CC items. The content validity of the items was evaluated by a panel of experts. Secondly, the factor structure and item performance of the CSMS-CC were examined using exploratory factor analysis (EFA) and internal consistency reliability of its subscales were evaluated in sample 1 (n = 229, 24.01% females). Thirdly, factor structure of the CSMS-CC was further examined using confirmatory factor analysis (CFA) in sample 2 (n = 295, 32.54% females). Internal consistency reliability was evaluated using Cronbach' alpha coefficient and composite reliability. Nomological validity was examined using Pearson correlation and structural equation modeling (SEM) by investigating the correlations between CSMS-CC subscales with career success. Finally, measurement invariance and latent mean difference of the CSMS-CC was examined across gender, professional title and coaching class using multiple-group CFA (MGCFA). Based on the results of the content analysis and content validity evaluation, 18 CSMS-CC items were retained for further analysis. Results of EFA in sample 1 revealed that eight items were problematic and removed. The second round of EFA revealed that three components were retained and labelled as Networking Behavior (4 items), Training Exploration (3 items), and Guanxi Development (3 items). Results of CFA in sample 2 suggested that the 10-item three-correlated-factors model of CSMS-CC demonstrated acceptable model fit to the data, χ2 = 135.01, df = 32, p < 0.01, CFI = 0.91, TLI = 0.90, SRMR = 0.05, RMSEA = 0.092 (90% CI = 0.076-0.108). Composite reliability (ranging from 0.84 to 0.88) and Cronbach's alpha coefficients (ranging from 0.78 to 0.81) of three subscales were found satisfactory. Nomological validity was supported by the results that total score and subscale scores of the CSMS-CC were significantly associated with internal marketability and external marketability. It was found that the CSMS-CC measurement model was strict invariant across gender, professional title and coaching class. Significant differences on all three subscales across professional title and on Guanxi development across coaching class were revealed. Results of this study provided initial support for the psychometric properties of the 10-item CSMS-CC, which suggested that the CSMS-CC could be used for measuring the career self-management of Chinese coaches.

Psychometric properties of the Coronavirus Anxiety Scale based on Classical Test Theory (CTT) and Item Response Theory (IRT) models among Chinese front-line healthcare workers

BackgroundSince March 2022, the COVID-19 epidemic has rebounded widely and frequently in China. Healthcare workers have faced grand challenges such as soaring COVID-19 patients, being busy with the nucleic acid screening of all the populations in the epidemic areas every day, and testing positive for COVID-19, all of which contributed to anxiety easily according to the Conservation of Resources theory. However, anxiety among healthcare workers is not only associated with personal health but also adversely affects the quality of health services. Therefore, it is crucial to search for suitable tools to monitor the anxiety related to COVID-19 among healthcare workers. The current study aimed to test the Coronavirus Anxiety Scale (CAS) in Chinese healthcare workers.MethodsThe current study employed a cross-sectional design. The CAS was translated into Chinese. Then, according to Classical Test Theory (CTT) and Item Response Theory (IRT) models, the psychometric properties of the Chinese version were measured among 811 healthcare workers.ResultsThe split‐half reliability was 0.855. The Cronbach’s α coefficient was 0.895. The retest coefficient was 0.901 with 10 days as the retest interval. The content validity index was 0.920. In exploratory factor analysis, one common factor was extracted and explained 72.559% of the total variance. All item load values on the common factor ranged from 0.790 to 0.885, and the communality of each item ranged from 0.625 to 0.784. With confirmatory factor analysis, the single factor model showed an excellent goodness-of-fit, chi-square/degree of freedom (χ2/df) = 3.339, goodness of fit index (GFI) = 0.992, adjusted goodness of fit index (AGFI) = 0.975, root-mean-square error of approximation (RMSEA) = 0.054, root mean square residual (RMR) = 0.005, incremental fit index (IFI) = 0.967, Tucker-Lewis index (TLI) = 0.932, and comparative fit index (CFI) = 0.966. The multiple-group confirmatory factor analysis revealed the invariance measuring anxiety of COVID-19 was in similar ways across ages, hospital degrees, and professional titles. With convergent validity, the CAS was positively correlated with post-traumatic stress disorder (r = 0.619, P < 0.001), fear of COVID (r = 0.550, P < 0.001), and depression (r = 0.367, P < 0.001). According to IRT models, the results showed that all item discrimination parameters were higher than 1.70 and difficulty parameters ranged from 1.13 to 2.83.ConclusionThe Chinese version of CAS has good psychometric properties in healthcare workers after China adjusted the COVID-19 management measures during the COVID-19 Omicron epidemic, and can be used for assessing the anxiety associated with COVID-19 in Chinese healthcare workers.

Paper-based and computer-based admission tests for medicine – are they equivalent?

Admission tests are among the most widespread and effective criteria for student selection in medicine in Germany. As such, the Test for Medical Studies (TMS) and the Hamburg Assessment Test for Medicine, Natural Sciences (HAM-Nat) are two major selection instruments assessing applicants’ discipline-specific knowledge and cognitive abilities. Both are currently administered in a paper-based format and taken by a majority of approximately 40,000 medicine applicants under high-stakes conditions yearly. Computer-based formats have not yet been used in the high-stakes setting, although this format may optimize student selection processes substantially. For an effective transition to computer-based testing, the test formats’ equivalence (i.e., measurement invariance) is an essential prerequisite. The present study examines measurement invariance across test formats for both the TMS and HAM-Nat. Results are derived from a large, representative sample of university applicants in Germany. Measurement invariance was examined via multiple-group confirmatory factor analysis. Analyses demonstrated partial scalar invariance for both admission tests indicating initial evidence of equivalence across test formats. Generalizability of the results is examined, and implications for the transition to computer-based testing are discussed.

Measurement Invariance of Scores on the Somatic Symptom Scale-8: National Sample of Non-Clinical Adults in the United States

Objective The aim of the present study was to test the factorial invariance and convergent validity evidence of scores on the Somatic Symptom Scale (SSS)-8 with a national sample of adults living in the United States. Method A data collection contracting service was hired to recruit a national, non-clinical sample (N = 821) of adults in the United States stratified by the census data for age, gender, ethnicity, and geographic location. Results Factorial invariance testing via multiple-group confirmatory factor analysis revealed strong factorial invariance evidence (configural, metric, and scalar) of SSS-8 scores across gender, ethnicity, help-seeking history, education, and income. Convergent validity testing displayed strong correlations between SSS-8 scores and the following established measures: Patient Health Questinaire-9, Generalized Anxiety Disorder-7, and Mental Health Inventory-5. Conclusion Collectively, the results suggest that professional counselors can use the SSS-8 to enhance somatic symptomology screening efforts with adults living in the United States.

Measurement invariance testing using confirmatory factor analysis and alignment optimization: A tutorial for transparent analysis planning and reporting.

Measurement invariance-the notion that the measurement properties of a scale are equal across groups, contexts, or time-is an important assumption underlying much of psychology research. The traditional approach for evaluating measurement invariance is to fit a series of nested measurement models using multiple-group confirmatory factor analyses. However, traditional approaches are strict, vary across the field in implementation, and present multiplicity challenges, even in the simplest case of two groups under study. The alignment method was recently proposed as an alternative approach. This method is more automated, requires fewer decisions from researchers, and accommodates two or more groups. However, it has different assumptions, estimation techniques, and limitations from traditional approaches. To address the lack of accessible resources that explain the methodological differences and complexities between the two approaches, we introduce and illustrate both, comparing them side by side. First, we overview the concepts, assumptions, advantages, and limitations of each approach. Based on this overview, we propose a list of four key considerations to help researchers decide which approach to choose and how to document their analytical decisions in a preregistration or analysis plan. We then demonstrate our key considerations on an illustrative research question using an open dataset and provide an example of a completed preregistration. Our illustrative example is accompanied by an annotated analysis report that shows readers, step-by-step, how to conduct measurement invariance tests using R and Mplus. Finally, we provide recommendations for how to decide between and use each approach and next steps for methodological research. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Assessing measurement invariance of the SEGUE framework among Chinese and Ghanaian patients: A cross-sectional study

ObjectiveMeasurement invariance assesses the psychometric equivalence of a construct across groups or across time. This study aimed to find out whether the newly modified 5-point Likert scale SEGUE Framework for evaluating doctor's communication skills demonstrates measurement invariance among Chinese and Ghanaian patients. MethodsMeasurement invariance tests were conducted using two samples collected from the two cultures respectively (China: N = 598 and Ghana: N = 591). Confirmatory Factor Analysis (CFA) was used to test for the factor structure. Multiple-group CFA was used to test the measurement invariance of the SEGUE between the two samples. ResultsThe same five-factor model showed acceptable model fit in both Chinese and Ghanaian samples. Cross-cultural measurement invariance tests revealed that scalar invariance was observed across the two samples. ConclusionCross-cultural adaptations of instruments should exhibit measurement invariance to ensure that measures are interpreted and used consistently. Findings of our study suggest that the SEGUE Framework is invariant across the two cultures, which increases confidence in data collection and comparison. InnovationThis is the first study to establish measurement invariance among Chinese and Ghanaians using the SEGUE Framework which assesses doctors' communication from patients' perspectives.

Measurement Invariance of the Intelligence and Development Scales – 2 Across Language Versions, Gender, and Age

Abstract: We examined the factor structure and measurement invariance of the intelligence and basic skills domains of the Intelligence and Development Scales – 2 (IDS-2) with the Dutch ( N = 1,665) and German ( N = 1,672) standardization samples. First, we tested five competing models using confirmatory factor analysis (CFA) on the Dutch data: two empirically based, derived earlier from the German data, and three theoretically based (IDS-2 and two Cattell–Horn–Carroll-based). Subsequently, we evaluated the measurement invariance of the final model across the Dutch and German versions and gender using multiple-group CFA and across age using local structural equation modeling. A second-order model with six first-order factors best represented the Dutch IDS-2 structure. Five IDS-2 factors were confirmed, but Visual Processing and Abstract Reasoning, and the intelligence and basic skills domains were not separable. This model displayed full invariance across the language versions and was largely invariant across gender and age (7–20 years). Thus, scores derived according to this model are comparable across these language versions, gender, and age. The strong general intelligence factor and weak broad ability factors ask for precaution when basing clinical interpretation on the broad ability factors.

Measuring Rape Empathy Among University Men in Vietnam

Abstract. Background: Empathy for rape victims is a correlate of sexual aggression and a target of interventions to reduce sexual violence on university campuses. Methods: We administered 17 adapted items from the Rape Empathy Scale to 793 first-year male undergraduates in Hanoi, Vietnam, in September 2020 and March-May 2020. Exploratory/confirmatory factor analysis (EFA/CFA) assessed factor structure and multiple group confirmatory factor analysis assessed measurement equivalence across those reporting different frequencies of sexually violent behavior and over time. Results: The final unidimensional, 10-item scale demonstrated equivalence across groups and over time. Correlations of the scale with rape myths and knowledge of the legality and harm of sexual violence suggested discriminant validity. Latent victim empathy was lower among participants who reported any recent sexually violent behavior. Conclusions: The adapted, validated Rape Empathy Scale-10 provides a useful tool for measuring rape empathy in sexual violence prevention interventions in Southeast Asia.

Personality inventory for DSM–5-Brief Form (PID-5-BF): Measurement invariance across men and women.

The study examined the measurement invariance (configural, metric, scalar, and residual) of the Personality Inventory for Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) Brief Form (PID-5-BF; Krueger et al., 2013) across gender for the theorized Five-Factor oblique model. A large group of adults (N = 502), with ages ranging from 18 to 67 years, from the Australian general community completed the PID-5-BF. When the Δχ² test was applied, multiple-group confirmatory factor analysis supported configural invariance and full metric and residual invariance. For scalar invariance, at least seven items (constituting 13 thresholds) across the different dimensions lacked invariance for one or more of its thresholds. In addition, controlling all these noninvariance thresholds, women had higher latent mean scores for Negative Affect, and men had higher scores for Antagonism and Disinhibition. When the Δcomparative fit index (ΔCFI) test was applied, the findings also supported the configural invariance model and the full metric and residual invariance models. With the exception of one threshold, all the other thresholds were also invariant. The psychometric and practical implications of the findings are discussed. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy

BackgroundValid and reliable measurement of implementation strategies is essential to advancing implementation science; however, this area lags behind the measurement of implementation outcomes and determinants. Clinical supervision is a promising and highly feasible implementation strategy in behavioral healthcare for which pragmatic measures are lacking. This research aimed to develop and psychometrically evaluate a pragmatic measure of clinical supervision conceptualized in terms of two broadly applicable, discrete clinical supervision techniques shown to improve providers’ implementation of evidence-based psychosocial interventions—(1) audit and feedback and (2) active learning.MethodsItems were generated based on a systematic review of the literature and administered to a sample of 154 outpatient mental health clinicians serving youth and 181 community-based mental health providers serving adults. Scores were evaluated for evidence of reliability, structural validity, construct-related validity, and measurement invariance across the two samples.ResultsIn sample 1, confirmatory factor analysis (CFA) supported the hypothesized two-factor structure of scores on the Evidence-Based Clinical Supervision Strategies (EBCSS) scale (χ2=5.89, df=4, p=0.208; RMSEA=0.055, CFI=0.988, SRMR=0.033). In sample 2, CFA replicated the EBCSS factor structure and provided discriminant validity evidence relative to an established supervisory alliance measure (χ2=36.12, df=30, p=0.204; RMSEA=0.034; CFI=0.990; SRMR=0.031). Construct-related validity evidence was provided by theoretically concordant associations between EBCSS subscale scores and agency climate for evidence-based practice implementation in sample 1 (d= .47 and .55) as well as measures of the supervision process in sample 2. Multiple group CFA supported the configural, metric, and partial scalar invariance of scores on the EBCSS across the two samples.ConclusionsScores on the EBCSS provide a valid basis for inferences regarding the extent to which behavioral health providers experience audit and feedback and active learning as part of their clinical supervision in both clinic- and community-based behavioral health settings.Trial registrationClinicalTrials.gov NCT04096274. Registered on 19 September 2019.

Development of a latent dementia index in the aging, demographics, and memory study: Validation and measurement invariance by sex.

Latent variable models can create a latent dementia index (LDI) using cognitive and functional ability to approximate dementia likelihood. The LDI approach has been applied across diverse cohorts. It is unclear whether sex affects its measurement properties. We use Wave A (2001-2003) of the Aging, Demographics, and Memory Study (n=856). Multiple group confirmatory factor analysis (CFA) was used to test measurement invariance (MI) using informant-reported functional ability and cognitive performance tasks, which we group into verbal, nonverbal, and memory. Partial scalar invariance was found, allowing for testing sex differences in LDI means (MDiff=0.38). The LDI correlated with consensus panel dementia diagnosis, Mini-Mental State Examination (MMSE), and dementia risk factors (low education, advanced age, and apolipoprotein ε4 [APOE-ε4] status) for men and women. The LDI validly captures dementia likelihood to permit estimation of sex differences. LDI sex differences indicate higher dementia likelihood in women, potentially due to social, environmental, and biological factors.