Use Of Total Score Research Articles

Beyond Jacobson and Truax: Estimation of Clinical Significance Trajectories in the Coping Power Intervention Using Measurement Error-Corrected Multilevel Modeling

Coping Power (CP) is a preventive intervention that focuses on reducing child externalizing problems. Although it is typically delivered in a group format (GCP), individually delivered CP (ICP) has produced greater mean reductions in externalizing problems. However, standard analysis of randomized trials loses individual-level information regarding which youth improve, fail to improve, or get worse, whereas clinically significant change (CSC) metrics capture information on individual change. The present study is a secondary analysis of an ICP/GCP trial (N = 360) that examines differences in CSC-based individual-level inferences on externalizing. A novel method for assessing CSC under measurement error-corrected multilevel modeling was used, overcoming three limitations of traditional CSC methods: (a) restriction to two time points, (b) use of total scores, and (c) assumption of constant reliability across time and participants. Because of concerns about Type II errors with all CSC methods, an individual-level effect size metric for CSC was also developed. Based on individualized Cohen’s d estimates, individual-level improvements in externalizing from 4th through 11th grades of d ≥ 0.5 were significantly greater in ICP (73%) versus GCP (45%). Further, GCP saw significantly higher percentages of youth with worsening of externalizing, underscoring concerns about diminished effects for GCP. Half of the sample had improvement that was not statistically significant but exceeded d ≥ 0.5, highlighting the susceptibility to Type II errors of CSC’s results based on statistical significance. An examination of ICP/GCP differences under advanced CSC analysis gives more nuanced information than conventional randomized controlled trial analysis and greater precision in estimating individual-level outcomes than standard CSC methodologies.

Validation of the compassionate engagement and action scales, compassion scale, and Sussex-Oxford compassion scales in a French-Canadian sample.

Compassion towards oneself and towards others has been associated with positive psychological outcomes, however, research is limited by the availability of valid psychometric measures, particularly in languages other than English. The current study translated (English to French) and validated the following measures: the Compassionate Engagement and Action Scales (CEAS), assessing self-compassion (CEAS-SC), compassion to others (CEAS-TO), and compassion from others (CEAS-FROM); the Compassion Scale (CS); and the Sussex-Oxford Compassion Scales for Self (SOCS-S) and Others (SOCS-O). French-speaking participants were recruited online (N = 384) and completed the translated measures as well as questionnaires assessing self-compassion, depression, anxiety, stress, insecure attachment, mindfulness, and well-being. Confirmatory Factor Analysis supports the original factor structures proposed for the CEAS-FROM (two-factor hierarchical), CS (four-factor hierarchical), SOCS-S and SOCS-O (five-factor hierarchical), with alternate factor structures proposed for CEAS-SC (three-factor) and CEAS-TO (two-factor). Results showed good internal consistency and convergent validity for all scales, supporting the use of total scores for the translated measures.

Prevalence of depression and associated symptoms among patients attending primary healthcare facilities: a cross-sectional study in Nepal

BackgroundDepression is a prevalent mental health condition worldwide but there is limited data on its presentation and associated symptoms in primary care settings in low- and middle-income countries like Nepal. This study aims to assess the prevalence of depression, its hallmark and other associated symptoms that meet the Diagnostic and Statistical Manual (DSM-5) criteria in primary healthcare facilities in Nepal. The collected information will be used to determine the content of a mobile app-based clinical guidelines for better detection and management of depression in primary care.MethodsA total of 1,897 adult patients aged 18–91 (63.1% women) attending ten primary healthcare facilities in Jhapa, a district in eastern Nepal, were recruited for the study between August 2, 2021, and March 25, 2022. Trained research assistants conducted face-to-face interviews in private spaces before the consultation with healthcare providers. Depression symptoms, including hallmark symptoms, was assessed using the validated Nepali version of the Patient Health Questionnaire (PHQ-9).ResultsOne in seven (14.5%) individuals attending primary health care facilities in Jhapa met the threshold for depression based on a validated cut-off score ( > = 10) on the PHQ-9. The most commonly reported depressive symptoms were loss of energy and sleep difficulties. Approximately 25.4% of women and 18.9% of men endorsed at least one of the two hallmark symptoms on the PHQ-9. Using a DSM-5 algorithm (at least one hallmark symptom and five or more total symptoms) to score the PHQ-9, 6.3% of women and 4.3% of men met the criteria for depression. The intra-class correlation coefficient for PHQ-9 total scores by health facility as the unit of clustering was 0.01 (95% confidence interval, 0.00-0.04).ConclusionDepression symptoms are common among people attending primary healthcare facilities in Nepal. However, the most common symptoms are not the two hallmark criteria. Use of total scores on a screening tool such as the PHQ-9 risks overestimating the prevalence and generating false positive diagnoses. Compared to using cut off scores on screening tools, training health workers to first screen for hallmark criteria may increase the accuracy of identification and lead to better allocation of treatment resources.

Evaluating the Construct Validity of the Norwegian Version of the Level of Personality Functioning Scale – Brief Form 2.0 in a Large Clinical Sample

The Level of Personality Functioning – Brief Form 2.0 (LPFS-BF 2.0) is a 12-item self-report questionnaire developed to gain a quick impression of the severity of personality pathology according to the DSM-5 Alternative Model for Personality Disorders (AMPD). The current study evaluated the construct validity and reliability of the Norwegian version of the LPFS-BF 2.0 in a large clinical sample (N = 1673). Dimensionality was examined using confirmatory factor analysis and bifactor analysis followed by an analysis of distinctiveness of the subscales using the proportional reduction in mean squared error (PRMSE), and the concurrent validity was examined using correlations with self-report questionnaires and clinical interviews assessing PDs according to section II of the DSM-5. Taking the findings of the dimensionality and concurrent validity results together, we found moderate to good support for the use of total scores for the Norwegian version of the LPFS-BF 2.0. We would advise against the use of subscale scores, since the subscales provided only a small amount of reliable unique variance.

Deconstructing Self-Compassion: How the Continued Use of the Total Score of the Self-Compassion Scale Hinders Studying a Protective Construct Within the Context of Psychopathology and Stress

In a new commentary in Mindfulness, Neff once again tried to defend the use of the Self-Compassion Scale (SCS) total score by arguing that compassionate and uncompassionate self-responding (CS and UCS) are part of a bipolar continuum. In this brief reaction, we refute this notion and also clarify how the continued use of the SCS total score muddies the water of research on self-compassion as a protective variable. We also illustrate how the SCS—by separating CS and UCS—can provide more valid and valuable information on protection and vulnerability within the context of psychopathology and stress than just relying on the total score of the SCS.

Estimating posttraumatic stress disorder severity in the presence of differential item functioning across populations, comorbidities, and interview measures: Introduction to Project Harmony.

Multiple factor analytic and item response theory studies have shown that items/symptoms vary in their relative clinical weights in structured interview measures for posttraumatic stress disorder (PTSD). Despite these findings, the use of total scores, which treat symptoms as though they are equally weighted, predominates in practice, with the consequence of undermining the precision of clinical decision-making. We conducted an integrative data analysis (IDA) study to harmonize PTSD structured interview data (i.e., recoding of items to a common symptom metric) from 25 studies (total N = 2,568). We aimed to identify (a) measurement noninvariance/differential item functioning (MNI/DIF) across multiple populations, psychiatric comorbidities, and interview measures simultaneously and (b) differences in inferences regarding underlying PTSD severity between scale scores estimated using moderated nonlinear factor analysis (MNLFA) and a total score analog model (TSA). Several predictors of MNI/DIF impacted effect size differences in underlying severity across scale scoring methods. Notably, we observed MNI/DIF substantial enough to bias inferences on underlying PTSD severity for two groups: African Americans and incarcerated women. The findings highlight two issues raised elsewhere in the PTSD psychometrics literature: (a) bias in characterizing underlying PTSD severity and individual-level treatment outcomes when the psychometric model underlying total scores fails to fit the data and (b) higher latent severity scores, on average, when using DSM-5 (net of MNI/DIF) criteria, by which multiple factors (e.g., Criterion A discordance across DSM editions, changes to the number/type of symptom clusters, changes to the symptoms themselves) may have impacted severity scoring for some patients.

Diagnosing Student Node Mastery: Impact of Varying Item Response Modeling Approaches

An important feature of learning maps, such as Dynamic Learning Maps and Enhanced Learning Maps, is their ability to accommodate nation-wide specifications of standards, such as the Common Core State Standards, within the map nodes along with relevant instruction. These features are especially useful for remedial instruction, given that accurate diagnosis is available. The year-end achievement tests are potentially useful in this regard. Unfortunately, the current use of total score or area sub-scores are neither sufficiently precise nor sufficiently reliable to diagnose mastery at the node level especially when students vary in their patterns of mastery. The current study examines varying approaches to using the year-end test for diagnosis. Prediction at the item level was obtained using parameters from varying item response theory (IRT) models. The results support using mixture class IRT models predicting mastery in which either items or node scores vary in difficulty for students in different latent classes. Not only did the mixture models fit better but trait score reliability was also maintained for the predictions of node mastery.

Applying Logistic Regression to Detect Differential Item Functioning in Multidimensional Data.

Conventional differential item functioning (DIF) approaches such as logistic regression (LR) often assume unidimensionality of a scale and match participants in the reference and focal groups based on total scores. However, many educational and psychological assessments are multidimensional by design, and a matching variable using total scores that does not reflect the test structure may not be good practice in multidimensional items for DIF detection. We propose the use of all subscores of a scale in LR and compare its performance with alternative matching methods, including the use of total score and individual subscores. We focused on uniform DIF situation in which 250, 500, or 1,000 participants in each group answered 21 items reflecting two dimensions, and the 21st item was the studied item. Five factors were manipulated in the study: (a) the test structure, (b) numbers of cross-loaded items, (c) group differences in latent abilities, (d) the magnitude of DIF, and (e) group sample size. The results showed that, when the studied item measured a single domain, the conventional LR incorporating total scores as a matching variable yielded inflated false positive rates (FPRs) when two groups differed in one latent ability. The situation worsened when one group had a higher ability in one domain and lower ability in another. The LR using a single subscore as the matching variable performed well in terms of FPRs and true positive rates (TPRs) when two groups did not differ in either one latent ability or differed in one latent ability. However, this approach yielded inflated FPRs when two groups differed in two latent abilities. The proposed LR using two subscores yielded well-controlled FPRs across all conditions and yielded the highest TPRs. When the studied item measured two domains, the use of either the total score or two subscores worked well in the control of FPRs and yielded similar TPRs across conditions, whereas the use of a single subscore resulted in inflated FPRs when two groups differed in one or two latent abilities. In conclusion, we recommend the use of multiple subscores to match subjects in DIF detection for multidimensional data.

Construct validity of the Heart Failure Screening Tool (Heart-FaST) to identify heart failure patients at risk of poor self-care: Rasch analysis.

The aim of this study was to psychometrically evaluate the Heart Failure Screening Tool (Heart-FaST) via: (1) examination of internal construct validity; (2) testing of scale function in accordance with design; and (3) recommendation for change/s, if items are not well adjusted, to improve psychometric credential. Self-care is vital to the management of heart failure. The Heart-FaST may provide a prospective assessment of risk, regarding the likelihood that patients with heart failure will engage in self-care. Psychometric validation of the Heart-FaST using Rasch analysis. The Heart-FaST was administered to 135 patients (median age=68, IQR=59-78years; 105 males) enrolled in a multidisciplinary heart failure management program. The Heart-FaST is a nurse-administered tool for screening patients with HF at risk of poor self-care. A Rasch analysis of responses was conducted which tested data against Rasch model expectations, including whether items serve as unbiased, non-redundant indicators of risk and measure a single construct and that rating scales operate as intended. The results showed that data met Rasch model expectations after rescoring or deleting items due to poor discrimination, disordered thresholds, differential item functioning, or response dependence. There was no evidence of multidimensionality which supports the use of total scores from Heart-FaST as indicators of risk. Aggregate scores from this modified screening tool rank heart failure patients according to their "risk of poor self-care" demonstrating that the Heart-FaST items constitute a meaningful scale to identify heart failure patients at risk of poor engagement in heart failure self-care.

Self-Compassion Scale: IRT Psychometric Analysis, Validation, and Factor Structure - Slovak Translation.

The present study verifies the psychometric properties of the Slovak version of the Self-Compassion Scale through item response theory, factor-analysis, validity analyses and norm development. The surveyed sample consisted of 1,181 participants (34% men and 66% women) with a mean age of 30.30 years (SD = 12.40). Two general factors (Self-compassionate responding and Self-uncompassionate responding) were identified, whereas there was no support for a single general factor of the scale and six subscales. The results of the factor analysis were supported by an independent sample of 676 participants. Therefore, the use of total score for the whole scale would be inappropriate. In Slovak language the Self-Compassion Scale should be used in the form of two general subscales (Self-compassionate responding and Self-uncompassionate responding). In line with our theoretical assumptions, we obtained relatively high Spearman’s correlation coefficients between the Self-Compassion Scale and related external variables, demonstrating construct validity for the scale. To sum up, the Slovak translation of The Self-Compassion Scale is a reliable and valid instrument that measures Self-compassionate responding and Self-uncompassionate responding.

General, Health-Specific, and Housing-Specific Self-Efficacy Scales: Preliminary Reliability and Validity Evidence with Homeless or Vulnerably Housed Adults

Self-efficacy (SE) refers to one’s sense of personal competence and is a key element of human agency. Among individuals who are homeless or vulnerably housed, SE has the potential to provide important information about an individual’s ability to seek out and make use of resources and persevere in the face of multiple challenges. SE is understudied as a personal variable in research with homeless samples. Thus, it is important to identify appropriate measures of general and domain-specific SE for this population and evaluate their psychometric properties. Three relevant SE scales are the Generalized Self Efficacy Scale, Perceived Health Competence Scale, and Housing Self-Efficacy Scale. The purposes of this study were to examine the internal structure and score reliability for each of these SE scales, report performance on and intercorrelations among the SE scales, and examine the relationship of demographic variables to SE scale scores, with a sample of adults who were homeless or vulnerably housed. Strict unidimensionality was evaluated using confirmatory factor analysis. Model fit was examined using fit indices and residual polychoric correlation matrices. Essential unidimensionality was evaluated using exploratory factor analysis and the ratio of first to second eigenvalues. Internal consistency estimates of reliability were obtained using ordinal alpha. None of the SE measures were found to be strictly unidimensional but all three measures were found to be essentially unidimensional. This finding supported the use of total scores for each measure. Ordinal alphas ranged from .87 to .93 for the three SE measures and thus were satisfactory. Correlations among the three measures ranged from .36 to .45. Demographic variables showed little relationship to the three SE measures. The study findings provided initial psychometric evidence to support the use of these three SE measures with a homeless or vulnerably housed adult sample.

The Use of Subscores in Higher Education: When Is This Useful?

Assessment in higher education is challenging because teachers face more students, with less contact time as compared to primary and secondary education. Therefore, teachers and management are often interested in efficient ways of giving students diagnostic feedback and providing information on the basis of subscores is one method that is often used in large-scale standardized testing. In this article we discuss some recent psychometric literature that warns against the use of subscores in addition to the use of total scores. We illustrate how the added value of subscores can be evaluated using two college exams: A multiple choice exam and a combined open-ended question and multiple choice exam; these formats are often used in higher education and represent cases in which using subscores may be informative. We discuss the implications of our findings for future classroom evaluation.

Measurement Error Correction Formula for Cluster-Level Group Differences in Cluster Randomized and Observational Studies.

Multilevel modeling (MLM) is frequently used to detect cluster-level group differences in cluster randomized trial and observational studies. Group differences on the outcomes (posttest scores) are detected by controlling for the covariate (pretest scores) as a proxy variable for unobserved factors that predict future attributes. The pretest and posttest scores that are most often used in MLM are total scores. In prior research, there have been concerns regarding measurement error in the use of total scores in using MLM. In this article, using ordinary least squares and an attenuation formula, we derive the measurement error correction formula for cluster-level group difference estimates from MLM in the presence of measurement error in the outcome, the covariate, or both. Examples are provided to illustrate the correction formula in cluster randomized and observational studies using between-cluster reliability coefficients recently developed.

Detecting Intervention Effects in a Cluster-Randomized Design Using Multilevel Structural Equation Modeling for Binary Responses.

Multilevel modeling (MLM) is frequently used to detect group differences, such as an intervention effect in a pre-test-post-test cluster-randomized design. Group differences on the post-test scores are detected by controlling for pre-test scores as a proxy variable for unobserved factors that predict future attributes. The pre-test and post-test scores that are most often used in MLM are summed item responses (or total scores). In prior research, there have been concerns regarding measurement error in the use of total scores in using MLM. To correct for measurement error in the covariate and outcome, a theoretical justification for the use of multilevel structural equation modeling (MSEM) has been established. However, MSEM for binary responses has not been widely applied to detect intervention effects (group differences) in intervention studies. In this article, the use of MSEM for intervention studies is demonstrated and the performance of MSEM is evaluated via a simulation study. Furthermore, the consequences of using MLM instead of MSEM are shown in detecting group differences. Results of the simulation study showed that MSEM performed adequately as the number of clusters, cluster size, and intraclass correlation increased and outperformed MLM for the detection of group differences.

Item response theory analyses of the Cambridge Face Memory Test (CFMT).

We evaluated the psychometric properties of the Cambridge Face Memory Test (CFMT; Duchaine & Nakayama, 2006). First, we assessed the dimensionality of the test with a bifactor exploratory factor analysis (EFA). This EFA analysis revealed a general factor and 3 specific factors clustered by targets of CFMT. However, the 3 specific factors appeared to be minor factors that can be ignored. Second, we fit a unidimensional item response model. This item response model showed that the CFMT items could discriminate individuals at different ability levels and covered a wide range of the ability continuum. We found the CFMT to be particularly precise for a wide range of ability levels. Third, we implemented item response theory (IRT) differential item functioning (DIF) analyses for each gender group and 2 age groups (age ≤ 20 vs. age > 21). This DIF analysis suggested little evidence of consequential differential functioning on the CFMT for these groups, supporting the use of the test to compare older to younger, or male to female, individuals. Fourth, we tested for a gender difference on the latent facial recognition ability with an explanatory item response model. We found a significant but small gender difference on the latent ability for face recognition, which was higher for women than men by 0.184, at age mean 23.2, controlling for linear and quadratic age effects. Finally, we discuss the practical considerations of the use of total scores versus IRT scale scores in applications of the CFMT.

Evaluating the uses of the total score and the domain scores in the Cognitive Abilities Screening Instrument, Chinese Version (CASI C-2.0): results of confirmatory factor analysis

The Cognitive Abilities Screening Instrument (CASI) consists of items which are designed to measure the nine domains of cognitive ability. Its total score is used clinically to represent the overall underlying cognitive ability of the patient. This study aimed to evaluate the justification of the uses of the all-domain-total-score as an overall cognitive measure and the domain scores as measures of designated individual cognitive ability. To justify the use of total score of all the items as an overall measure of cognitive ability, a second-order confirmatory factor analysis was performed to examine whether the items in CASI contributed significantly to a common underlying construct. The uses of domain scores were also examined by inspecting the loadings of the items on their designated domains. The CASI data from 608 patients, 68 normal and 540 with Alzheimer's disease, were analyzed. The goodness-of-fit indices for the second-order factor model were as follows: CFI was 0.912 for WLSMV and 0.977 for WLSM; TLI and RMSEA values were 0.975 and 0.090 respectively. The loadings of the items on the common underlying construct are all salient (>0.3). The loadings of all but the long-term memory items on their respective subdomains were also salient. The items of the CASI C-2.0 were useful not only in profiling the correlated cognitive domain scores, but also in forming an overall measure of the underlying cognitive ability of the patients.

Impact of an increase in grading categories and double reporting on the reliability of breast cancer grade

The present study illustrates the effect of using the consolidated total score (obtained on a 7-point scale by adding the tubular, nuclear and mitotic scores) as well as double reporting on the interobserver reliability of breast cancer grading by the Nottingham method. 50 consecutive breast cancer slides were graded independently by 5 pathologists. The interobserver reliability between the individual pathologists for final grade, total score and grade components was estimated by quadratic weighted kappa (k(w)). Similarly, the inter-observer reliability of double reported scores (obtained by averaging the reported scores of two independent pathologists) for independent observer pairs was estimated. The total scores gave higher reliability figures than the final grade both for individual pathologists and double reported scores. The double reported total scores and grade also gave significantly higher reliability figures than the individual scores and grade (average k(w)=0.65 and 0.78 for the individual and double reported total scores, respectively; 0.61 and 0.66 for the individual and double reported grades).Therefore, use of total scores in addition to grade as well as double reporting of the same significantly increases the interobserver reliability of breast cancer grading.

Injection Drug Use Quality of Life scale (IDUQOL): a validation study.

BackgroundExisting measures of injection drug users' quality of life have focused primarily on health and health-related factors. Clearly, however, quality of life among injection drug users is impacted by a range of unique cultural, socioeconomic, medical, and geographic factors that must also be considered in any measure. The Injection Drug User Quality of Life (IDUQOL) scale was designed to capture the unique and individual circumstances that determine quality of life among injection drug users. The overall purpose of the present study was to examine the validity of inferences made from the IDUQOL by examining the (a) dimensionality, (b) reliability of scores, (c) criterion-related validity evidence, and (d) both convergent and discriminant validity evidence.MethodsAn exploratory factor analysis using principal axis factoring in SPSS 12.0 was conducted to determine whether the use of a total score on the IDUQOL was advisable. Reliability of scores from the IDUQOL was obtained using internal consistency and one-week test-retest reliability estimates. Criterion-related validity evidence was gathered using variables such as stability of housing, sex trade involvement, high-risk injection behaviours, involvement in treatment programs, emergency treatment or overdose over the previous six months, hospitalization and emergency treatment over the subsequent six month period post data collection. Convergent and discriminant validity evidence was gathered using measures of life satisfaction, self-esteem, and social desirability.ResultsThe sample consisted of 241 injection drug users ranging in age from 19 to 61 years. Factor analysis supports the use of a total score. Both internal consistency (alpha = .88) and one-week test-retest reliability (r = .78) for IDUQOL total scores were good. Criterion-related, convergent, and discriminant validity evidence supports the interpretation of IDUQOL total scores as measuring a construct consistent with quality of life.ConclusionThe findings from this study provide initial evidence to support the use of the IDUQOL total score. The results of the study also suggest the IDUQOL could be further strengthened with additional attention to how some IDUQOL domains are described and satisfaction is measured.

Feeling Fat Yet Unconcerned: Self-reported Overweight and the Restraint Scale

Factor analysis was used to reduce the 10-item restraint questionnaire ( Herman, 1978 ) to two underlying common factors, reflecting fluctuations in body weight and actual concern with dieting. Weight History factor scores were higher for those subjects who identified themselves as overweight, and only Dietary Concern scores remained constant. The use of total scores on the restraint questionnaire may not always provide a pure measure of dietary restraint, and alternative approaches are suggested.

A New Look at Treatment Differences

Quite frequently in the behavioral sciences, the raw data obtained from an experimental subject are a series of binary responses. In addition to the example of test item data we could include many questionnaire studies and psychological experiments involving discrimination, perception and the like. If the subjects are also classified on another variable, the design is similar to the split-plot design and is often called a two-factor repeated measures experiment by behavioral researchers. With continuous measurements which fit the normal model the analysis is fairly straightforward. Tests of the assumption of a patterned covariance matrix should be considered but results of Collier, et al., (1967) indicate that moderate departures do not seriously invalidate the usual tests of significance. For situations in which the researcher feels that this approach is unwarranted, a conservative test due to Greenhouse and Geisser (1959) or multivariate procedures described elsewhere e.g., Danford, et al., (1960); Cole and Grizzle (1966) are appropriate. For dichotomous data, however, the literature provides few clues to aid the researcher. Recent non-parametric results of Koch (1969) which are based on ranks, are not directly applicable for dichotomous data. Other than the use of total scores or separate tests of proportions for each response, the present writer knows of no methods which are used with any frequency by behavioral scientists. Robustness of variance ratio procedures suggests that the usual univariate F tests might also be useful with binary data. In fact, when Cochran (1950) introduced the Q statistic, he also presented data which