Categorical Confirmatory Factor Analysis Research Articles

A Psychometric Investigation of the Painful and Provocative Events Scale: Moving Forward

Acquired capability for suicide is posited to develop through repeated exposure to physically painful and psychologically provocative events; however, there is limited research evaluating the assessment of painful and provocative events. The current study examined the factor structure, validity, and reliability of the Painful and Provocative Events Scale (PPES). Categorical exploratory and confirmatory factor analyses were conducted in a sample of undergraduate students (N = 732). A 15-item PPES with a 3-factor solution best fits the data; however, both the subscales derived from factor analyses yielded inconsistent validity and reliability. Despite addressing limitations of previous psychometric investigations of the PPES, the inconsistent findings of the current study suggest the theoretical reconceptualization of painful and provocative events and the subsequent development of a valid and reliable assessment of painful and provocative events is needed to take the place of current versions of the PPES.

Robustness of Parameter Estimation to Assumptions of Normality in the Multidimensional Graded Response Model

ABSTRACTA central assumption that is implicit in estimating item parameters in item response theory (IRT) models is the normality of the latent trait distribution, whereas a similar assumption made in categorical confirmatory factor analysis (CCFA) models is the multivariate normality of the latent response variables. Violation of the normality assumption can lead to biased parameter estimates. Although previous studies have focused primarily on unidimensional IRT models, this study extended the literature by considering a multidimensional IRT model for polytomous responses, namely the multidimensional graded response model. Moreover, this study is one of few studies that specifically compared the performance of full-information maximum likelihood (FIML) estimation versus robust weighted least squares (WLS) estimation when the normality assumption is violated. The research also manipulated the number of nonnormal latent trait dimensions. Results showed that FIML consistently outperformed WLS when there were one or multiple skewed latent trait distributions. More interestingly, the bias of the discrimination parameters was non-ignorable only when the corresponding factor was skewed. Having other skewed factors did not further exacerbate the bias, whereas biases of boundary parameters increased as more nonnormal factors were added. The item parameter standard errors recovered well with both estimation algorithms regardless of the number of nonnormal dimensions.

Development of a psychological test to measure ability-based emotional intelligence in the Indonesian workplace using an item response theory.

This study aimed to develop an emotional intelligence (EI) test that is suitable to the Indonesian workplace context. Airlangga Emotional Intelligence Test (Tes Kecerdasan Emosi Airlangga [TKEA]) was designed to measure three EI domains: 1) emotional appraisal, 2) emotional recognition, and 3) emotional regulation. TKEA consisted of 120 items with 40 items for each subset. TKEA was developed based on the Situational Judgment Test (SJT) approach. To ensure its psychometric qualities, categorical confirmatory factor analysis (CCFA) and item response theory (IRT) were applied to test its validity and reliability. The study was conducted on 752 participants, and the results showed that test information function (TIF) was 3.414 (ability level = 0) for subset 1, 12.183 for subset 2 (ability level = −2), and 2.398 for subset 3 (level of ability = −2). It is concluded that TKEA performs very well to measure individuals with a low level of EI ability. It is worth to note that TKEA is currently at the development stage; therefore, in this study, we investigated TKEA’s item analysis and dimensionality test of each TKEA subset.

Measurement invariance of the Depression Anxiety Stress Scales-21 across medical student genders.

ObjectivesThis study aimed to assess whether male and female Iranian medical students perceived the meaning of the items in the Depression Anxiety Stress Scales-21 consistently. MethodsA convenience sample of 783 preclinical medical students from the first to sixth semester was invited to this cross-sectional study. Of the 477 respondents, 238 were male and 239 were female. All participants completed the Persian version of the Depression Anxiety Stress Scales-21. The graded response model was used to assess measurement invariance of the instrument across the gender groups. Categorical confirmatory factor analysis was used to evaluate the construct validity of the measure. Moreover, internal consistency was assessed via Cronbach's Alpha. ResultsStatistically significant differential item functioning was flagged for just item 6 in the depression subscales (c2=6.5, df=1, p=0.011). However, removing or retaining the item 6 in the stress subscale did not change our findings significantly, when we compared stress scores across two genders. The results of categorical confirmatory factor analysis supported the fit of the three-factor model of Depression Anxiety Stress Scales-21. Moreover, Cronbach’s alpha was greater than 0.7 in depression, anxiety and stress subscales. ConclusionsThis study revealed that Depression Anxiety Stress Scales-21 is an invariant measure across male and female medical students. Hence, this reliable and valid instrument can be used for meaningful comparison of distress scores between medical student genders. Gender comparisons of medical students’ psychological profiles provide a better insight into gender influences on the outcome of medical education and medical practice.

中学生対象日本語版BRIEF構成概念妥当性の検証と標準化

This study sought to examine the reliability and validity of the Japanese version of the Behavior Rating Inventory of Executive Function (J-BRIEF). In this study, BRIEF was administered to evaluate executive function in everyday life in 91 subjects with autism spectrum disorder (ASD; ages 12–15; 72 boys) and 2,230 community samples (CS; ages 12–15; 1,083 boys). For this purpose, we applied categorical confirmatory factor analysis, which revealed that the scale was composed of two factors and eight subscales of the high test-retest stability. Reliability was confirmed using an external criterion (ADHD-Rating scale: ADHD-RS). Receiver operating characteristic analysis revealed an optimal cut-off of 118.5 (sensitivity = 0.811, specificity = 0.828). This study confirmed the reliability and the validity of J-BRIEF.

Assessing the Sensitivity of Weighted Least Squares Model Fit Indexes to Local Dependence in Item Response Theory Models

Given the relationships of item response theory (IRT) models to confirmatory factor analysis (CFA) models, IRT model misspecifications might be detectable through model fit indexes commonly used in categorical CFA. The purpose of this study is to investigate the sensitivity of weighted least squares with adjusted means and variance (WLSMV)-based root mean square error of approximation, comparative fit index, and Tucker–Lewis Index model fit indexes to IRT models that are misspecified due to local dependence (LD). It was found that WLSMV-based fit indexes have some functional relationships to parameter estimate bias in 2-parameter logistic models caused by violations of LD. Continued exploration into these functional relationships and development of LD-detection methods based on such relationships could hold much promise for providing IRT practitioners with global information on violations of local independence.

Development and Validation of the Adolescent Psychological Need Support in Exercise Questionnaire.

Grounded within self-determination theory (SDT; Deci & Ryan, 2000; Ryan & Deci, in press), three studies were conducted to develop and psychometrically test a measure of adolescents' perceptions of psychological need support for exercise (viz., for autonomy, competence, and relatedness): the Adolescent Psychological Need Support in Exercise Questionnaire (APNSEQ). In Study 1, 34 items were developed in collaboration with an expert panel. Through categorical confirmatory factor analysis and item response theory, responses from 433 adolescents were used to identify the best fitting and performing items in Study 2. Here, a three-factor nine-item measure showed good fit to the data. In Study 3, responses from an independent sample of 373 adolescents provided further evidence for the nine-item solution as well as for internal consistency, criterion validity, and invariance across gender and social agent (friends, family, and physical education teacher). The APNSEQ was supported as a measure of adolescents' perceptions of psychological need support within the context of exercise.

EHealth literacy in chronic disease patients: An item response theory analysis of the eHealth literacy scale (eHEALS)

BackgroundChronic disease patients are affected by low computer and health literacy, which negatively affects their ability to benefit from access to online health information. ObjectiveTo estimate reliability and confirm model specifications for eHealth Literacy Scale (eHEALS) scores among chronic disease patients using Classical Test (CTT) and Item Response Theory techniques. MethodsA stratified sample of Black/African American (N=341) and Caucasian (N=343) adults with chronic disease completed an online survey including the eHEALS. Item discrimination was explored using bi-variate correlations and Cronbach’s alpha for internal consistency. A categorical confirmatory factor analysis tested a one-factor structure of eHEALS scores. Item characteristic curves, in-fit/outfit statistics, omega coefficient, and item reliability and separation estimates were computed. ResultsA 1-factor structure of eHEALS was confirmed by statistically significant standardized item loadings, acceptable model fit indices (CFI/TLI>0.90), and 70% variance explained by the model. Item response categories increased with higher theta levels, and there was evidence of acceptable reliability (ω=0.94; item reliability=89; item separation=8.54). ConclusioneHEALS scores are a valid and reliable measure of self-reported eHealth literacy among Internet-using chronic disease patients. Practice implicationsProviders can use eHEALS to help identify patients’ eHealth literacy skills.

The Reliability and Validity of the Dating Abuse Perpetration Acts Scale in an Urban, Emergency Department-Based Sample of Male and Female Youth.

Adolescent dating abuse (ADA) is an important public health problem, but existing measurement instruments have limitations. This study assessed the reliability and validity of the Dating Abuse Perpetration Acts Scale (DAPAS), a medium-length instrument (24 items) designed for use with adolescents aged 16 to 21 years. In Phase 1, 466 adolescent pediatric emergency department patients who reported perpetrating at least one act of physical ADA in the last year completed the DAPAS. In the second phase, 33 patients completed the DAPAS; two previously established dating violence measures, the Safe Dates Perpetration Act Scale and the revised Conflict Tactics Scales (CTS2); and a measure of violence, the Self-Report of Offending (SRO). Categorical confirmatory factor analyses indicated that a one-factor model of ADA perpetration best fit the data. Validity analyses showed moderate to high correlations between the DAPAS and both the Safe Dates measure and the CTS2 ( r = .70 and .59) scores, and a small to moderate correlation between the DAPAS and SRO ( r = .43). Sensitivity and specificity analysis of the DAPAS using 10 cut-points revealed an optimal cut-point, which correctly classified 85% of the respondents. The DAPAS showed concurrent validity with the Safe Dates measure and CTS2, as well as discriminant validity in its ability to differentiate perpetrators of violence in general from perpetrators of ADA. The DAPAS scale is a good alternative to the Safe Dates measure and should be considered for use in research studies of ADA.

Perceived discrimination and sexual precursor behaviors in Mexican American preadolescent girls: The role of psychological distress, sexual attitudes, and marianismo beliefs.

This study explored the relation between perceived discrimination and sexual precursor behaviors among 205 Mexican American preadolescent middle school girls. In addition, this study examined whether psychological distress and sexual attitudes mediated and whether marianismo beliefs moderated this relation. A categorical confirmatory factor analysis (CCFA) of the Marianismo Beliefs Scale (MBS) was conducted to test the factor structure with a preadolescent Mexican American population (ages 11-14). A path analysis of analytic models was then performed to examine the hypothesized relations between perceived discrimination, psychological distress, sexual attitudes, marianismo beliefs, and sexual precursor behaviors. Results of the CCFA did not support the original 5-factor structure of the MBS for preadolescent Latina girls. However, a revised version of the MBS indicated an acceptable model fit, and findings from the path analysis indicated that perceived discrimination was both directly and indirectly linked to sexual precursor behaviors via psychological distress. Marianismo was not found to moderate the relation between perceived discrimination and sexual risk behaviors, however certain marianismo pillars were significantly negatively linked with sexual attitudes and precursor behaviors. This study underscores the importance of psychological distress in the perceived discrimination and sexual precursor link as well as the compensatory aspects of marianismo against sexual precursor behaviors in Mexican American preadolescent girls. (PsycINFO Database Record

Comparability of the Patient-Reported Outcomes Measurement Information System Pediatric short form symptom measures across culture: examination between Chinese and American children with cancer.

Patient-Reported Outcomes Measurement Information System (PROMIS) Pediatric forms measure symptoms and function of pediatric patients experiencing chronic disease by using the same measures. Comparability is one of the most important purposes of the PROMIS initiative. This study aimed to test the factorial structures of four symptom measures (i.e., Anxiety, Depression, Fatigue, and Pain Interference) in the original English and the Chinese versions and examine the measurement invariance of the measures across two cultures. Four PROMIS Pediatric measures were used to assess symptoms, respectively, in Chinese (n=232) and American (n=200) children and adolescents (8-17years old) in treatment for cancer or in survivorship. The categorical confirmatory factor analysis (CCFA) model was used to examine factorial structures, and multigroup CCFA was applied to test measurement invariance of these measures between the Chinese and American samples. The CCFA models of the four PROMIS Pediatric symptom measures fit the data well for both the Chinese and American children and adolescents. Minor partial measurement invariance was identified. Factor means and factor variances of the four PROMIS measures were not significantly different between the two populations. Our results provide evidence that the four PROMIS Pediatric symptom measures have valid factorial structures and a statistical property of measurement invariance across American and Chinese children and adolescents with cancer. This means that the items of these measures were interpreted in a conceptually similar manner by two groups. They could be readily used for meaningful cross-cultural comparisons involving pediatric oncology patients in these two countries.

Abstract C16: Validation of a new medical discrimination scale among a diverse population of breast cancer survivors

Abstract Introduction: Experiences of medical discrimination may impact women's decisions regarding breast cancer treatment. However, measures of perceived discrimination generally are not specific to the medical context, and none are specific to breast cancer. The objective of this study was to determine the structure and psychometric properties of a 9-item scale assessing perceived discrimination among a racially/ethnically diverse sample of women receiving breast cancer care. This scale was developed based on existing measures of general discrimination and recent qualitative research with breast cancer survivors of various racial/ethnic backgrounds. Methods: Women ages 20 and older diagnosed with a first invasive primary breast cancer between 2006 and 2009 were sampled from all available case listings in the Greater Bay Area Cancer Registry. 542 of these women completed a telephone interview in English, Spanish, Mandarin, Cantonese, or Tagalog between 2011 and 2013. Women reported how often they perceived 9 types of discriminatory experiences while receiving breast cancer care using a 4-point scale (never, rarely, sometimes, often). Item-level descriptive statistics and reliability analyses were computed. A categorical confirmatory factor analysis (CCFA) was conducted to determine whether the 9 items reflected a single discrimination construct. Logistic regression was used to evaluate differential item functioning (DIF), examining whether the items performed differently by race after controlling for the level of perceived medical discrimination. A summed medical discrimination score was calculated from the items that supported the unidimensional discrimination construct according to the CCFA and did not show evidence of DIF. Correlations between the summed score and single item measures of perceived racial/ethnic discrimination and satisfaction with breast cancer care were used to assess convergent validity. Results: Reports of medical discrimination were fairly low, with no more than a third of any racial/ethnic group of women reporting discrimination on any item. As a result, responses were dichotomized into never vs. ever experiencing discrimination. A CCFA of all 9 items suggested improved model fit could be achieved by adding a correlated error term between two items with similar wording. After removing the item with the smallest factor loading from this item pair, a final CCFA produced adequate model fit (RMSEA=0.06, CFI=0.99, TLI=0.99). Standardized factor loadings ranged from 0.71-0.91. DIF analyses of the remaining 8 items indicated uniform DIF with an appreciable effect size for one item: “had to wait longer than other people to be seen by your health care team” (p<0.001, effect size=0.03). The final medical discrimination scale comprised 7 items and demonstrated good reliability (α=0.84). The mean summed score was 1.24 (SD=1.90; range=0.0-7.0 in total sample). Preliminary evidence of convergent validity was observed: medical discrimination scores were significantly positively correlated with mistreatment while receiving breast cancer care due to race/ethnicity (r=0.39, p<0.0001), and significantly negatively correlated with satisfaction with care (r=-0.42, p<0.0001). Conclusion: A 7-item medical discrimination scale demonstrated unidimensionality, internal reliability, convergent validity, and uniformity of responses by race for women diagnosed with breast cancer. This is the first medical discrimination scale developed for a diverse group of breast cancer patients and may be of interest to researchers and clinicians interested in how perceptions of discrimination in the medical context contribute to disparities in breast cancer care and outcomes. Citation Format: Felisa A. Gonzales, Salma Shariff-Marco, Michelle M. Langer, Bryce B. Reeve, Amani M. Nuru-Jeter, Laura A. Dwyer, Scarlett Lin Gomez. Validation of a new medical discrimination scale among a diverse population of breast cancer survivors. [abstract]. In: Proceedings of the Eighth AACR Conference on The Science of Health Disparities in Racial/Ethnic Minorities and the Medically Underserved; Nov 13-16, 2015; Atlanta, GA. Philadelphia (PA): AACR; Cancer Epidemiol Biomarkers Prev 2016;25(3 Suppl):Abstract nr C16.

Understanding non-linear modeling of measurement invariance in heterogeneous populations

This study examined how a non-linear modeling of ordered categorical variables within multiple-group confirmatory factor analysis supported measurement invariance. A four-item classroom disciplinary climate scale used in cross-cultural framework was empirically investigated. In the first part of the analysis, a separated categorical confirmatory factor analysis was initially applied to account for the complex structure of the relationships between the observed measures in each country. The categorical multiple-group confirmatory factor analysis (MGCFA) was then used to conduct a cross-country examination of full measurement invariance namely the configural, metric, and scalar levels of invariance in the classroom discipline climate measures. The categorical MGCFA modeling supported configural and metric invariances as well as scalar invariance for the latent factor structure of classroom disciplinary climate. This finding implying meaningful cross-country comparisons on the scale means, on the associations of classroom disciplinary climate scale with other scales and on the item-factor latent structure. Application of the categorical modeling appeared to correctly specify the factor structure of the scale, thereby promising the appropriateness of reporting comparisons such as rankings of many groups, and illustrating league tables of different heterogeneous groups. Limitations of the modeling in this study and future suggestions for measurement invariance testing in studies with large numbers of groups are discussed.

Sexual Orientation Self-Concept Ambiguity: Scale Adaptation and Validation.

The current article describes the adaptation of a measure of sexual orientation self-concept ambiguity (SSA) from an existing measure of general self-concept clarity. Latent "trait" scores of SSA reflect the extent to which a person's beliefs about their own sexual orientation are perceived as inconsistent, unreliable, or incongruent. Sexual minority and heterosexual women ( n = 348), ages 18 to 30, completed a cross-sectional survey. Categorical confirmatory factor analysis guided the selection of items to form a 10-item, self-report measure of SSA. In the current report, we also examine (a) reliability of the 10-item scale score, (b) measurement invariance based on respondents' sexual identity status and age group, and (c) correlations with preexisting surveys that purport to measure similar constructs and theoretical correlates. Evidence for internal reliability, measurement invariance (based on respondent sex), and convergent validity was also investigated in an independent, validation sample. The lowest SSA scores were reported by women who self-ascribed an exclusively heterosexual or exclusively lesbian/gay sexual identity, whereas those who reported a bisexual, mostly lesbian/gay, or mostly heterosexual identity, reported relatively higher SSA scores.

Confirmatory Factor Analysis and Measurement Invariance Testing with Ordinal Data: An Application in Revising the Flirting Styles Inventory

Factor analysis (FA) is becoming a common practice in communication research for measurement validation, yet issues associated with FA for ordinal items have not been adequately addressed. As many attitudinal and behavioral measures in communication research consist of ordered-categorical items, this article provides an accessible introduction for applied researchers on categorical confirmatory factor analysis (cat-CFA) and measurement invariance (MI) testing for ordinal data. First, this paper presents conceptual discussions on the classical and the categorical FA models and conditions under which applications of the classical FA can lead to estimation biases. Second, model identification and specification issues with MI for ordinal data are discussed. Third, to demonstrate the techniques, cat-CFA and MI with ordinal data are applied to the revision of an existing self-report Likert-type measure, the Flirting Styles Inventory.

DIF Detection Using Multiple‐Group Categorical CFA With Minimum Free Baseline Approach

The aim of this study is to assess the efficiency of using the multiple‐group categorical confirmatory factor analysis (MCCFA) and the robust chi‐square difference test in differential item functioning (DIF) detection for polytomous items under the minimum free baseline strategy. While testing for DIF items, despite the strong assumption that all but the examined item are set to be DIF‐free, MCCFA with such a constrained baseline approach is commonly used in the literature. The present study relaxes this strong assumption and adopts the minimum free baseline approach where, aside from those parameters constrained for identification purpose, parameters of all but the examined item are allowed to differ among groups. Based on the simulation results, the robust chi‐square difference test statistic with the mean and variance adjustment is shown to be efficient in detecting DIF for polytomous items in terms of the empirical power and Type I error rates. To sum up, MCCFA under the minimum free baseline strategy is useful for DIF detection for polytomous items.

Measurement of Dating Aggression During Middle School: Structure, Measurement Invariance, and Distinction From General Aggression.

Two studies examined the factor structure of a modified version of the Safe Dates dating aggression scale and evaluated whether dating aggression is distinct from general aggression during early adolescence. Analyses were conducted on a derivation sample of 3,894 adolescents from 37 schools (Study 1) and an independent cross-validation sample of 938 middle school youth (Study 2). Categorical confirmatory factor analyses supported a two-factor model (perpetration and victimization) over models differentiating psychological and physical forms of aggression. The model was invariant across time, sex, grade, and season. Study 2 also supported dating aggression as distinct from general aggression. Results supported measuring dating aggression perpetration and victimization as latent constructs represented by ordered categorical indicators that capture item severity and frequency.

A Short Form of the Career Interest Test

The Career Interest Test (CIT) is a 63-item forced-choice instrument designed to measure seven career interests. Although the measure possesses favorable psychometric properties, there have been recent calls for the development of a shortened version of the inventory. The present article reports on research conducted to develop a short form of the CIT. Using archival data from over 180,000 respondents, categorical confirmatory factor analyses were conducted with limited information methods. Items with the highest absolute factor loadings for each career interest were retained for the short form of the inventory, comprising 21 items, and renamed the 21-CIT. The large within-subject correlations between career interest scores on the full CIT and 21-CIT indicate that the short form provides a comparable degree of content coverage for the career interests. We consider implications for career research and practice as well as suggest directions for future research.

The emotional distress of children with cancer in China: an item response analysis of C-Ped-PROMIS Anxiety and Depression short forms.

The pediatric patient-reported outcomes measurement information system (PROMIS) was developed to provide self-reported item banks and short forms for children and adolescents (aged 8-17 years) experiencing a chronic illness to assess their quality of life and symptoms. The pediatric PROMIS short forms have been translated into Chinese and applied in children with cancer in China. This study aimed to describe psychometric properties of the Anxiety and Depression measures. A total of 232 children and adolescents with cancer were recruited in hospital-based inpatient and outpatient clinics. Eight Chinese versions of pediatric PROMIS (C-Ped-PROMIS) measures were administered. Categorical confirmatory factor analysis (CCFA) were conducted to evaluate scale dimensionality and item local dependence of the C-Ped-PROMIS Anxiety and Depression items. Multiple indicator multiple cause models were used to analyze differential item functioning (DIF), item response theory parameters were calculated, and test information against T scores was reported for each measure. The results of the CCFA confirmed that both C-Ped-PROMIS Anxiety and Depression scales appropriately measure the theoretical constructs as designed. No significant DIF was found for the items of the two scales according to age and gender. Both scales have high test reliability as long as their T scores are not on the low or high extremes. The pediatric PROMIS Anxiety and Depression measures were developed to provide efficient and flexible assessment of emotional distress domains. Our results provide evidence that the two C-Ped-PROMIS measures can be readily applied to measure anxious and depressive symptoms in Chinese children with cancer.

Assessing the Straightforwardly-Worded Brief Fear of Negative Evaluation Scale for Differential Item Functioning Across Gender and Ethnicity.

The Brief Fear of Negative Evaluation Scale (BFNE; Leary Personality and Social Psychology Bulletin, 9, 371-375, 1983) assesses fear and worry about receiving negative evaluation from others. Rodebaugh et al. Psychological Assessment, 16, 169-181, (2004) found that the BFNE is composed of a reverse-worded factor (BFNE-R) and straightforwardly-worded factor (BFNE-S). Further, they found the BFNE-S to have better psychometric properties and provide more information than the BFNE-R. Currently there is a lack of research regarding the measurement invariance of the BFNE-S across gender and ethnicity with respect to item thresholds. The present study uses item response theory (IRT) to test the BFNE-S for differential item functioning (DIF) related to gender and ethnicity (White, Asian, and Black). Six data sets consisting of clinical, community, and undergraduate participants were utilized (N=2,109). The factor structure of the BFNE-S was confirmed using categorical confirmatory factor analysis, IRT model assumptions were tested, and the BFNE-S was evaluated for DIF. Item nine demonstrated significant non-uniform DIF between White and Black participants. No other items showed significant uniform or non-uniform DIF across gender or ethnicity. Results suggest the BFNE-S can be used reliably with men and women and Asian and White participants. More research is needed to understand the implications of using the BFNE-S with Black participants.