Factor Score Estimates Research Articles

Two-stage path analysis with definition variables: An alternative framework to account for measurement error.

When estimating path coefficients among psychological constructs measured with error, structural equation modeling (SEM), which simultaneously estimates the measurement and structural parameters, is generally regarded as the gold standard. In practice, however, researchers usually first compute composite scores or factor scores, and use those as observed variables in a path analysis, for purposes of simplifying the model or avoiding model convergence issues. Whereas recent approaches, such as reliability adjustment methods and factor score regression, has been proposed to mitigate the bias induced by ignoring measurement error in composite/factor scores with continuous indicators, those approaches are not yet applicable to models with categorical indicators. In this article, we introduce the two-stage path analysis (2S-PA) with definition variables as a general framework for path modeling to handle categorical indicators, in which estimation of factor scores and path coefficients are separated. It thus allows for different estimation methods in the measurement and the structural path models and easier diagnoses of violations of model assumptions. We conducted three simulation studies, ranging from latent regression to mediation analysis with categorical indicators, and showed that 2S-PA generally produced similar estimates to those using SEM in large samples, but gave better convergence rates, less standard error bias, and better control of Type I error rates in small samples. We illustrate 2S-PA using data from a national data set, and show how researchers can implement it in Mplus and OpenMx. Possible extensions and future directions of 2S-PA are discussed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Thinking thrice about sum scores, and then some more about measurement and analysis

Measurement is fundamental to all research in psychology and should be accorded greater scrutiny than typically occurs. Among other claims, McNeish and Wolf (Thinking twice about sum scores. Behavior Research Methods, 52, 2287-2305) argued that use of sum scores (a) implies that a highly constrained latent variable model underlies items comprising a scale, and (b) may misrepresent or bias relations with other criteria. The central claim by McNeish and Wolf that use of sum scores requires the assumption that a parallel test model underlies item responses is incorrect and without psychometric merit. Instead, if a set of items is unidimensional, estimators of reliability are available even if the factor model underlying the set of items does not have a highly constrained form. Thus, dimensionality of a set of items is the key issue, and whether strict constraints on parameter estimates do or do not hold dictate the appropriate way to estimate reliability. McNeish and Wolf also claimed that more precise forms of scoring, such as estimating factor scores, would be preferable to sum scores. We provide analytic bases for reliability estimation and then provide several demonstrations of reliability estimation and the relative advantages of sum scores and factor scores. We contend that several claims by McNeish and Wolf are questionable and that, as a result, multiple recommendations they made and conclusions they drew are incorrect. The upshot is that, once the dimensional structure of a set of items is verified, sum scores often have a solid psychometric basis and therefore are frequently quite adequate for psychological research.

Supplemental Material for Factor Score Estimation in Multimethod Measurement Designs With Planned Missing Data

Supplemental Material for Factor Score Estimation in Multimethod Measurement Designs With Planned Missing Data

Multigroup CFA and alignment approaches for testing measurement invariance and factor score estimation: Illustration with the schoolwork-related anxiety survey across countries and gender

Within large-scale international studies, the utility of survey scores to yield meaningful comparative data hinges on the degree to which their item parameters demonstrate measurement invariance (MI) across compared groups (e.g., culture). To-date, methodological challenges have restricted the ability to test the measurement invariance of item parameters of these instruments in the presence of many groups (e.g., countries). This study compares multigroup confirmatory factor analysis (MGCFA) and alignment method to investigate the MI of the schoolwork-related anxiety survey across gender groups within the 35 Organisation for Economic Co-operation and Development (OECD) countries (gender × country) of the Programme for International Student Assessment 2015 study. Subsequently, the predictive validity of MGCFA and alignment-based factor scores for subsequent mathematics achievement are examined. Considerations related to invariance testing of noncognitive instruments with many groups are discussed.

Multilevel Analysis of Mediation, Moderation, and Nonlinear Effects in Small Samples, Using Expected a Posteriori Estimates of Factor Scores

ABSTRACT In the analysis of hierarchical data, multilevel structural equation modeling (multilevel SEM) has become the standard in the social sciences. To estimate these models, maximum likelihood (ML) approaches have been applied because they are the default in latent variable software. However, one drawback of ML is that it tends to suffer from estimation problems such as nonconvergence when the sample size is small to moderate, and the results that come from nonconverged solutions are useless in research practice. Nonconvergence is a particularly serious problem when more complex multilevel SEMs are estimated. Therefore, in this article, we show how factor score regression (FSR) can be used to obtain estimates of multilevel mediation, moderation, and nonlinear effects. We conducted two simulation studies to validate our approaches. Our findings were generally promising, which renders FSR an attractive alternative to ML.

Longitudinal change in restricted and repetitive behaviors from 8-36\u2009months

BackgroundRestricted and repetitive behaviors (RRBs) are core features of autism spectrum disorder (ASD) and one of the earliest behavioral signs of ASD. However, RRBs are also present in typically developing (TD) infants, toddlers, and preschool-aged children. Past work suggests that examining change in these behaviors over time is essential to distinguish between normative manifestations of these behaviors and behaviors that denote risk for a neurodevelopmental disorder. One challenge in examining changes in these behaviors over time is that most measures of RRBs have not established longitudinal measurement invariance. The aims of this study were to (1) establish measurement invariance in the Repetitive Behavior Scales for Early Childhood (RBS-EC), a parent-report questionnaire of RRBs, and (2) model developmental change in RRBs from 8 to 36 months.MethodsWe collected RBS-EC responses from parents of TD infants (n = 180) from 8 to 36 months (n = 606 responses, with participants contributing an average of 3-time points). We leverage a novel methodological approach to measurement invariance testing (Bauer, Psychological Models, 22(3), 507–526, 2017), moderated nonlinear factor analysis (MNLFA), to determine whether the RBS-EC was invariant across age and sex. We then generated adjusted factor score estimates for each subscale of the RBS-EC (repetitive motor, self-directed, and higher-order behaviors), and used linear mixed effects models to estimate between- and within-person changes in the RBS-EC over time.ResultsThe RBS-EC showed some non-invariance as a function of age. We were able to adjust for this non-invariance in order to more accurately model changes in the RBS-EC over time. Repetitive motor and self-directed behaviors showed a linear decline from 8 to 36 months, while higher-order behaviors showed a quadratic trajectory such that they began to decline later in development at around 18 months. Using adjusted factor scores as opposed to unadjusted raw mean scores provided a number of benefits, including increased within-person variability and precision.ConclusionsThe RBS-EC is sensitive enough to measure the presence of RRBs in a TD sample, as well as their decline with age. Using factor score estimates of each subscale adjusted for non-invariance allowed us to more precisely estimate change in these behaviors over time.

Best Practices for Binary and Ordinal Data Analyses.

The measurement of many human traits, states, and disorders begins with a set of items on a questionnaire. The response format for these questions is often simply binary (e.g., yes/no) or ordered (e.g., high, medium or low). During data analysis, these items are frequently summed or used to estimate factor scores. In clinical applications, such assessments are often non-normally distributed in the general population because many respondents are unaffected, and therefore asymptomatic. As a result, in many cases these measures violate the statistical assumptions required for subsequent analyses. To reduce the influence of the non-normality and quasi-continuous assessment, variables are frequently recoded into binary (affected-unaffected) or ordinal (mild-moderate-severe) diagnoses. Ordinal data therefore present challenges at multiple levels of analysis. Categorizing continuous variables into ordered categories typically results in a loss of statistical power, which represents an incentive to the data analyst to assume that the data are normally distributed, even when they are not. Despite prior zeitgeists suggesting that, e.g., variables with more than 10 ordered categories may be regarded as continuous and analyzed as if they were, we show via simulation studies that this is not generally the case. In particular, using Pearson product-moment correlations instead of maximum likelihood estimates of polychoric correlations biases the estimated correlations towards zero. This bias is especially severe when a plurality of the observations fall into a single observed category, such as a score of zero. By contrast, estimating the ordinal correlation by maximum likelihood yields no estimation bias, although standard errors are (appropriately) larger. We also illustrate how odds ratios depend critically on the proportion or prevalence of affected individuals in the population, and therefore are sub-optimal for studies where comparisons of association metrics are needed. Finally, we extend these analyses to the classical twin model and demonstrate that treating binary data as continuous will underestimate genetic and common environmental variance components, and overestimate unique environment (residual) variance. These biases increase as prevalence declines. While modeling ordinal data appropriately may be more computationally intensive and time consuming, failing to do so will likely yield biased correlations and biased parameter estimates from modeling them.

Translation and validation of the Short HIV Stigma scale in Brazilian Portuguese

BackgroundHIV-related stigma, or the degree to which people living with HIV endorse negative stereotypes associated with HIV, is associated with poor continuum of care outcomes. We translated the 12-item Short HIV Stigma scale and evaluated its psychometric properties in a Brazilian context with regard to construct validity and reliability.MethodsThe first step included translation, back-translation, evaluation, peer review, and pre-testing of the Short HIV Sigma scale developed by Reinius et al. (Health Qual Life Outcomes 15(1):115, 2017).The second step involved piloting the scale in three convenience samples of adults recruited online through advertisements on different platforms: Grindr (October/2019) and Hornet (February–March/2020), geospatial network apps for sexual encounters for gay, bisexuals and other men who have sex with men, and social media apps (Facebook and WhatsApp, October/2019). The psychometric evaluation included confirmatory factor analysis, differential item functioning using the Multiple-Indicator Multiple-Cause model, and correlations between subscale scores and antiretroviral treatment use and adherence. Reliability was assessed using Cronbach’s alpha, and ordinal alpha and omega from the polychoric correlation matrix.ResultsIn total, 114, 164, and 1824 participants completed the measure items through Grindr, social media, and Hornet, respectively. We confirmed a 4-factor structure with factors for personalized stigma (3 items), disclosure concerns (3 items), concerns with public attitudes (3 items), and negative self-image (3 items). Small differential item functioning with respect to sample was found for one item (“I feel guilty because I have HIV”), which did not substantively influence estimates of latent factor scores. Grindr and Hornet’s participants scored significantly higher than social media participants on all factors except personalized stigma. Higher subscale scores correlated with antiretroviral treatment use among participants from Hornet and with lower treatment adherence in participants from Grindr and Hornet. Reliability as measured by Cronbach’s alpha, ordinal alpha and omega were 0.83, 0.88 and 0.93 for the entire scale.DiscussionThe Brazilian Portuguese version of the Short HIV Stigma scale had satisfactory psychometric properties with present results suggesting that scores from different samples may be compared without concern that measurement differences substantively influence results though further studies with greater representation of women and heterosexual men are warranted.

The Appropriateness of Sum Scores as Estimates of Factor Scores in the Multiple Factor Analysis of Ordered-Categorical Responses

Unit-weight sum scores (UWSSs) are routinely used as estimates of factor scores on the basis of solutions obtained with the nonlinear exploratory factor analysis (EFA) model for ordered-categorical responses. Theoretically, this practice results in a loss of information and accuracy, and is expected to lead to biased estimates. However, the practical relevance of these limitations is far from clear. In this article, we adopt an empirical view and propose indices and procedures (some of them new) for assessing the appropriateness of UWSSs in nonlinear EFA applications. A new automated approach for obtaining UWSSs that maximize fidelity and correlational accuracy is proposed. The appropriateness of UWSSs under different conditions and the behavior of the present proposal in comparison with other more common approaches are assessed with a simulation study. A tutorial for interested practitioners is presented using an illustrative example based on a well-known personality questionnaire. All the procedures proposed in the article have been implemented in a well-known noncommercial EFA program.

The Appropriateness of Sum Scores as Estimates of Factor Scores in the Multiple Factor Analysis of Ordered-Categorical Responses

Unit-weight sum scores (UWSSs) are routinely used as estimates of factor scores on the basis of solutions obtained with the nonlinear exploratory factor analysis (EFA) model for ordered-categorical...

Detection of Health Abnormality Considering Latent Factors Inducing a Disease

Underlying latent factors may cause a person to feel unwell. As the influence of the latent factors increases, the person will become sick. It is difficult to directly measure the influence of latent factors on risk degrees. However, early symptoms of a disease may affect vital signs such as body temperature and blood pressure, which may be a result of the influence of the latent factors. Deep learning is often used to predict the onset of a disease owing to its high accuracy. However, the reliability of this method is limited because of its characteristics of a black-box model. In this study, we propose a new approach to detect health abnormality. We regard the degree of influence of latent factors as the risk of disease and detect health abnormality before the onset of the disease. In our approach, we used a combination of Structural Equation Modeling (SEM) and Hidden Markov Model (HMM). First, in SEM, a domain model was created, and the factor score was estimated based on the relationship between latent factors and the explicit variables influenced by the factors. Thereafter, risk degrees were quantified with HMM using the estimated factor scores, and abnormality was identified in terms of risk degree. Finally, our proposed method was compared with three baselines: PCA (principal component analysis)-based approach, deep learning, and no-degree estimation methods. The average recall of our method was 98.75%, almost the same as the baselines, and false positive rate (FPR) was 0.186%, lower than the baselines. In the five-fold cross-validation comparing with no-degree estimation method, the average accuracy and recall of our method were 99.7% and 98.3% respectively, and FPR was 0.045%, all much better than the baseline. Moreover, our approach can make the process of obtaining the result visible and help detect abnormality sensitively by setting the threshold according to the risk degree, which can contribute to early detection of a disease and improve the reliability of abnormality detection as well.

Physical frailty and decline in general and specific cognitive abilities: the Lothian Birth Cohort 1936

BackgroundPhysical frailty is associated with many adverse outcomes including disability, chronic disease, hospitalisation, institutionalisation and death. It is unclear what impact it might have on the rate of normal cognitive...

Assessing the Quality and Effectiveness of the Factor Score Estimates in Psychometric Factor-Analytic Applications

Abstract. This article proposes an approach, intended for factor-analytic (FA) psychometric applications, which aims to assess the extent to which the FA-derived individual score estimates are accurate and allow the respondents to be consistently ordered and effectively differentiated over the range of trait values that is appropriate given the purposes of the test. Three groups of properties are assessed: (a) fineness, (b) probability, and (c) range, and, within each group, different indices are proposed. Overall, the proposal is comprehensive in that it can be used with (a) different factor score estimates derived from both the linear model and the categorical variable methodology model, and (b) any type of unrestricted or restricted FA solution. All the indices proposed have been implemented in a non-commercial and widely known program for exploratory factor analysis. The usefulness of the proposal is illustrated with a real-data example in the personality domain.

Refining scores based on patient reported outcomes \u2013 statistical and medical perspectives

BackgroundPatient Reported Outcomes (PRO) are gaining more and more importance in the context of clinical trials. The assessment of PRO is frequently performed by questionnaires where the multiple items of a questionnaire are usually pooled within summarizing scores. These scores are used as variables to measure subjective aspects of treatments and diseases. In clinical research, the calculation of these scores is mostly kept very simple, e.g. by a simple summation of item values. In the medical literature, there is hardly any guidance for performing a refinements of questionnaires and for deducing adequate scores. In contrast, in psychometric literature, there are plenty of more sophisticated methods, which overcome typical assumptions made in traditional (sum) scores, however to the prize of more complicated algorithms, which might be difficult to communicate. When faced with the practical task to refine an existing questionnaire, there exist a clear gap of guidance for applied medical researchers. By this article we try to fill this important gap between psychometric theory and medical application by illustrating our methodological choices on the example of a clinical PRO questionnaire.MethodsBased on our experiences with the refinement of the BCTOS, a PRO questionnaire to assess aesthetic and function after breast conserving therapy in breast cancer patients, we present the following general steps that we performed by refining the BCTOS questionnaire and its scores: 1. Refinement of the length of the questionnaire and the (item-factor) structure. 2. Selection of the factor score estimation method. 3. Validation of the refined questionnaire and scores with respect to validity, reliability and structure based on a validation cohort.ResultsOur step-step-step procedure helped us to shorten the current form of the BCTOS and to redefine the factor structure. By this, the compliance of patients can be increased and the interpretation of the results becomes more coherent.ConclusionsWe present a step-by-step procedure to refine an existing medical questionnaire along with its scores illustrated and discussed by the refinement of the BCTOS.Trial registrationDue to the character of the study (no intervention study), no registration was performed.

Conditional Longitudinal Relations of Elementary Literacy Skills to High School Reading Comprehension

The present study explores the longitudinal development between Grade 3 word level reading skills and higher level semantic skills to Grade 10 reading comprehension for 3,157 students. In particular, this work focused on how the developmental relations varied for subsamples of students who are English learners (EL; N = 308), students identified as specific learning disability (SLD; N = 133), and general education (Gen Ed; N = 2,716) students who have no formal classification or diagnoses. Multiple group structural equation modeling showed that the relation between Grade 3 vocabulary and Grade 10 reading comprehension did not vary across three student subgroups when accounting for Grade 3 fluency and that when controlling for students’ vocabulary, the unique effect of oral reading fluency ranged from a standardized effect of γ = .22 to γ = .39 across the three subgroups. Quantile regression using estimated factor scores revealed heterogeneous relations of component skills to reading comprehension across each subgroup.

Can Factor Scores be Used Instead of Total Score and Ability Estimation?

The purpose of this study is to investigate whether factor scores can be used instead of ability estimation and total score. For this purpose, the relationships among total score, ability estimation, and factor scores were investigated. In the research, Turkish subtest data from the Transition from Primary to Secondary Education (TEOG) exam applied in April 2014 were used. Total scores in this study were calculated from the total number of correct answers given by individuals to each item. Ability estimations were obtained from a three-parameter logistic model chosen from among item response theory (IRT) models. The Bartlett method was used for factor score estimation. Thus, the ability estimation, sum, and factor scores of each individual were obtained. When the relationship between these variables was investigated, it was observed that there was a high-level, positive, and statistically significant relationship. In the result section of this study, as variables have a high-level relationship, it was suggested that since variables could be used interchangeably, factor scores should be used. Although the total scores of individuals were equal, there were differences in terms of factor score and ability estimations. Therefore, it was suggested that item response theory assumptions were not met, or factor scores should be used when the sample size is small.

Pain catastrophizing: rumination is a discriminating factor among individuals with different pain characteristic.

The objective of this study was to compare the scores of the Helplessness, Magnification, Rumination, and Catastrophizing factors of the Pain Catastrophizing Scale (PCS) between samples with different pain characteristics. The psychometric properties of the PCS were evaluated in 1,151 Brazilian adults (78.9% female; 38.6 (SD = 10.8) years): 335 had no pain, 390 had been in pain for less than 3 months, 250 had been in recurring pain for more than 3 months, and 176 had been in continuous pain for more than 3 months. Confirmatory factor analysis (CFA) was conducted to verify the fit of the PCS models. Convergent validity and reliability were evaluated. Multi-group analysis was used to estimate the invariance of the factorial model. The global score for the PCS factors was obtained using the regression weight matrix for estimating factor scores from CFA. Analysis of variance was used to compare scores between samples. After excluding three items, the tri-factorial model showed adequate fit. The model parameters were invariant (Δχ2(λ,i,β,Res); p≥0.05). Individuals experiencing pain showed higher scores for catastrophic thoughts. Individuals with pain for less than 3 months had the highest scores for Rumination (p < 0.001). The PCS showed valid, reliable, and invariant results for the sample of Brazilian adults in no pain or with different pain conditions. The PCS adequately discriminated individuals in pain from those without pain. Among those in pain, Rumination was the only discriminating factor.

Simplifying the implementation of modern scale scoring methods with an automated R package: Automated moderated nonlinear factor analysis (aMNLFA)

When generating scores to represent latent constructs, analysts have a choice between applying psychometric approaches that are principled but that can be complicated and time-intensive versus applying simple and fast, but less precise approaches, such as sum or mean scoring. We explain the reasons for preferring modern psychometric approaches: namely, use of unequal item weights and severity parameters, the ability to account for local dependence and differential item functioning, and the use of covariate information to more efficiently estimate factor scores. We describe moderated nonlinear factor analysis (MNLFA), a relatively new, highly flexible approach that allows analysts to develop precise factor score estimates that address limitations of sum score, mean score, and traditional factor analytic approaches to scoring. We then outline the steps involved in using the MNLFA scoring approach and discuss the circumstances in which this approach is preferred. To overcome the difficulty of implementing MNLFA models in practice, we developed an R package, aMNLFA, that automates much of the rule-based scoring process. We illustrate the use of aMNLFA with an empirical example of scoring alcohol involvement in a longitudinal study of 6998 adolescents and compare performance of MNLFA scores with traditional factor analysis and sum scores based on the same set of 12 items. MNLFA scores retain more meaningful variation than other approaches. We conclude with practical guidelines for scoring.

Do screening and a randomized brief intervention at a Level 1 trauma center impact acute stress reactions to prevent later development of posttraumatic stress disorder?

Approximately 20% to 40% of trauma survivors experience posttraumatic stress disorder (PTSD). The American College of Surgeons Committee on Trauma reports that early screening and referral has the potential to improve outcomes and that further study of screening and intervention for PTSD would be beneficial. This prospective randomized study screened hospitalized patients for traumatic stress reactions and assessed the effect of a brief intervention in reducing later development of PTSD. The Primary Care PTSD (PC-PTSD) screen was administered to admitted patients. Patients with symptoms were randomized to an intervention or control group. The brief intervention focused on symptom education and normalization, coping strategies, and utilizing supports. The control group received a 3-minute educational brochure review. Both groups completed in-hospital interviews, then 45- and 90-day telephone interviews. Follow-up collected the PTSD checklist-civilian (PCL-C) assessment and qualitative data on treatment-seeking barriers. The PC-PTSD screen was successful in predicting later PTSD symptoms at both 45 days (β = 0.43, p < 0.001) and 90 days (β = 0.37, p < 0.001) even after accounting for depression. Correlations of the intervention with the PCL-C scores and factor score estimates did not reach statistical significance at either time point (p = 0.827; p = 0.838), indicating that the brief intervention did not decrease PTSD symptoms over time. Of those at or above the PCL-C cutoff at follow-ups, a minority had sought treatment for their symptoms (43.2%). Primary barriers included focusing on their injury or ongoing rehabilitation, financial concerns, or location of residence. The PC-PTSD screen identified patients who later assess positive for PTSD using the PCL-C. The brief intervention did not reduce 45- and 90-day PTSD development. Follow-up interviews revealed lack of treatment infrastructure in the community. It will be important for trauma centers to align with community resources to address the treatment needs of at-risk patients. Prospective randomized controlled trial, level II.

Recovering Predictor–Criterion Relations Using Covariate-Informed Factor Score Estimates

Although it is currently best practice to directly model latent factors whenever feasible, there remain many situations in which this approach is not tractable. Recent advances in covariate-informed factor score estimation can be used to provide manifest scores that are used in second-stage analysis, but these are currently understudied. Here we extend our prior work on factor score recovery to examine the use of factor score estimates as predictors both in the presence and absence of the same covariates that were used in score estimation. Results show that whereas the relation between the factor score estimates and the criterion are typically well recovered, substantial bias and increased variability is evident in the covariate effects themselves. Importantly, using covariate-informed factor score estimates substantially, and often wholly, mitigates these biases. We conclude with implications for future research and recommendations for the use of factor score estimates in practice.