Psychometric Properties Of Scores Research Articles

Expanding bifactor models of psychological traits to account for multiple sources of measurement error.

Over the last decade, applications of bifactor modeling within clinical settings have increased markedly but typically rely on data collected on single occasions. A shortcoming of such research is that reliability coefficients are likely inflated because key sources of measurement error are inadequately modeled and/or confounded with construct variance. We address these problems using three variations of multi-occasion bifactor models with Bayesian-derived parameter estimates to separate systematic variance into general and group factor effects and measurement error into three subcomponents (transient, specific-factor, and random-response). Collectively, these models produce indices of reliability and validity aligned with both standard confirmatory factor models and generalizability designs that extend interpretations of results to the broader domains from which items and occasions are sampled. We demonstrate how these techniques can provide new insights into psychometric properties of scores using Negative Emotionality domain and facet scales from the newly updated Big Five Inventory (BFI-2; Soto & John, 2017). Overall, the two-occasion congeneric bifactor model provided the best fit to the data and most informative indices for revising measures, examining dimensionality of composite and subscale scores, and evaluating the viability of those scores. We include code in R for analyzing all models in our extended online Supplemental Material. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Psychometric Properties of Scores on the Mental Distress Response Scale among First-Generation College Students: Implications for Retention

First-generation college students” (FGCS) are at risk for suffering from mental health issues, which have direct implication for their retention and academic success. Past investigators consistently find that college students are more likely to discuss mental health issues with their peers than with college personnel. The first step in peer-to-peer mental health support is screening students on campus to gain a baseline understanding of how they respond when encountering a peer in mental distress. The Mental Distress Response Scale (MDRS) is a screening tool for appraising students” responses to a peer in mental distress. Score validation is a crucial step, as FGCS are a unique college student population and the psychometric properties of instrumentation can vary between different populations. Psychometric testing yielded support for the validity of scores on the MDRS with FGCS. Findings have direct implications for aiding student affairs officials” universal screening efforts to support FGCS” mental health and retention.

Psychometric Evaluation of the Depression Anxiety Stress Scales-21 (DASS-21) in Senior Games Athletes

This study examined psychometric properties of scores on the Depression Anxiety Stress Scales-21 (DASS-21) with a nonclinical sample of senior games athletes (N = 383). Information on the internal reliability, item-scale correlations, structural validity, and concurrent validity of the DASS-21 scores is provided to test its applicability to senior games athletes.

Construct Validity of the E-LSRP in a Correctional Sample.

The Expanded-Levenson Self-Report Psychopathy Scale (E-LSRP) was developed by Christian and Sellbom to improve on the psychometric properties of scores on the Levenson Self-Report Psychopathy Scale. The current study investigated the construct validity of scores on the E-LSRP in 393 male inmates. Results provided support for the reliability and construct validity of E-LSRP scores. Specifically, confirmatory factor analysis results demonstrated support for a three-factor model. Additionally, correlation and multiple regression results provided evidence supporting the convergent and discriminant validity of E-LSRP scores against scores on measures assessing psychopathy-related personality traits (e.g., antagonism, disinhibition) and symptoms of internalizing disorders, respectively. Overall, these findings extend those of previous research by establishing that E-LSRP scores demonstrate validity in assessing psychopathy in correctional settings and thus, may be a useful tool for the assessment of psychopathy in these settings.

Psychometric evaluation of the Behavioral Activation for Depression Scale-Short Form in Alzheimer’s caregivers

The present study evaluated the psychometric properties of scores from the Behavioral Activation for Depression Scale Short Form (BADS-SF) in a sample of older age, spousal, Alzheimer’s caregivers participating in an evaluation of Behavioral Activation (BA) therapy compared to an Information Support (IS) group. At baseline assessment, caregivers (N = 170) completed the BADS-SF, which is comprised of two subscales (Activation and Avoidance) that can be summed to produce a total score. Confirmatory factor analysis was used to evaluate structural validity. A two-factor solution fit the data adequately; however, the first item on the scale did not load onto either factor. Internal consistency reliability for the total and subscales scores was poor as measured by Cronbach’s alpha. Construct validity was evidenced by significant expected relationships with depression. Pre- to post-intervention scores did not evidence sensitivity to change. These findings provide some support but raise important concerns about the validity and reliability of BADS-SF scores in a population of older adult caregivers.

The Trauma and Suicide Potential Index–5: Psychometric Evaluation in College Samples

We provide a brief description of the development of the Trauma and Suicide Potential Index–5 (TSPI-5) and report on the psychometric properties of scores from the new instrument. The TSPI-5 is designed to assess an individual’s desire to attempt suicide due to experiencing a trauma. We examined the structure of the instrument in two independent undergraduate samples. Study 1 (N = 415) examined the structure of the TSPI-5 using exploratory structural equation modeling. Study 2 (N = 538) reexamined the instrument structure using the same modeling strategy and also examined other validity estimates. Study 1 demonstrated an adequate fit to the sample data (χ2 [1, N = 415] = 215.99, p < .001, comparative fit index (CFI ) = .969, Tucker-Lewis non-normed fit index (TLI) = .939, root mean squared error of approximation (RMSEA) = .319 [90% confidence interval (CI) = .283, .356], p < .001) and suggested evidence for unidimensionality. Study 2 confirmed the unidimensionality of the TSPI-5 as shown by acceptable fit estimates to the sample data, χ2 (5, N = 538) = 80.45, p < .001, CFI = .996, TLI = .99, RMSEA = .17 (90% CI = .140, .200), p < .001. Scale reliability estimates for the TSPI-5 were good in Study 1 (omega = .94) and Study 2 (omega = .96). The TSPI-5 is a brief unidimensional instrument and its scores demonstrated good reliability and validity for assessing suicide-related behaviors due to experiencing a trauma in undergraduate samples.

Perceptions of the LGBTQ College Campus Climate Scale: Development and Psychometric Evaluation

ABSTRACT This article reports the development and psychometric properties of scores on a new scale designed to assess views of the university/college campus climate concerning LGBTQ students and issues: Perceptions of the LGBTQ College Campus Climate Scale. This 6-item scale includes two subscales: College Response to LGBTQ Students and LGBTQ Stigma. We provide evidence for structural validity (via exploratory and confirmatory factor analyses) and reliability for scores on this new measure. Supporting construct validity, Perceptions of the LGBTQ College Campus Climate full scale and subscales were each positively correlated with experiences of LGBTQ victimization on campus, anxiety, and depression and negatively correlated with satisfaction with college and intention to persist in college. Relationships with mental health and academic outcomes held true, even after controlling for LGBTQ victimization experiences, providing support for incremental validity. Implications for future research and practice are discussed.

Computerized and Traditional Administration of Questionnaires: Psychometric Quality and Completion Time for Measures of Self-Concept

ABSTRACT In two independent studies of questionnaire administration, respondents completed multidimensional self-concept inventories within four randomized research conditions that mirrored the most common administration formats used in practice: paper booklets with and without answer sheets and computer questionnaires with single versus multiple items per screen. Strong differences among conditions emerged for completion time, but not for psychometric properties of scores (means, variances, reliability coefficients, concurrent validity coefficients). Answer sheets increased completion time by 24% to 34%, single-item displays by 20% to 25%, and computerization by 13% to 17%. Completion time was longer for multiple-alternative than for rating-scale items, but relative effects of answer sheets, single-item displays, and computerization remained consistent. We discuss implications for questionnaire construction and administration.

Evaluation of Mediterranean diet adherence scores: a systematic review

ObjectiveThe aim of this review was to evaluate the conceptual suitability, applicability and psychometric properties of scores used internationally to measure adherence to the Mediterranean diet (MD).DesignThis was a systematic...

Spanish Adaptation of the Working Alliance Inventory-Short (WAI-S)

The goal of our research was to adapt the short forms of the Working Alliance Inventory into the Spanish language for patients and therapists (WAI-S-P and WAI-S-T). The psychometric properties of scores from these measurement instruments were evaluated in Spanish outpatients with depressive disorders and their therapists. The scores of both measures exhibited excellent reliability in terms of internal consistency and excellent convergent validity. Their discriminant validity exhibited some limitations, particularly when the scores of both short measures correlated with those of two empathy measures. Regarding predictive validity, the scores of the overall WAI-S-P and WAI-S-T significantly correlated with residual gain scores of patients in the Beck Depression Inventory (BDI) after the tenth psychotherapy session. Additionally, the overall WAI-S-T scores explained a moderate percentage of the variance in patient BDI score changes. These results are satisfactory and similar to those found in the studies that have utilized the respective original versions of WAI-S in English.

Structural Congruence of the Sexual Dependency Inventory—4th Edition

The purpose of the current study was to establish the replicability of the component structure of the Sexual Dependency Inventory (SDI), a broadband, multidimensional measure of problematic sexual behaviors and preoccupations associated with sexual addiction or other sexual disorders, such as the paraphilias. The internal structure of the SDI was previously evaluated in a sexual addiction clinical sample of 626 men and 85 women, but the replicability of the component structure was not evaluated. The present study examined the internal structure of SDI responses in a very large (N > 1,000) clinical sample, and evaluated the replicability of the structure in a second sample of similar size. The structure of the SDI derived was consistent with the previous version (SDI-3.0), was coherent and psychometrically sound, and replicated well in a second sample. Based upon these results, recommendations were made for revisions to establish the Sexual Dependency Inventory—4th Edition (SDI-4.0) Behavior and Preoccupation scales to better match the component structure identified and improved psychometric performance of the measure. Psychometric properties of scores from the revised scales were reported, including estimates of internal consistency reliability from both samples. Clinical implications and suggestions for future research are discussed.

Elaboration, validation and standardization of the five to fifteen (FTF) questionnaire in a Danish population sample.

The five to fifteen (FTF) is a parent questionnaire developed to assess ADHD, its common comorbid conditions and associated problems in children and adolescents. The present study examined (1) the psychometric properties of scores on the new teacher version of the FTF, (2) competing models of the FTF subdomain structure and (3) the psychometric properties and utility of scores on the newly developed FTF impact questions. Parents (n=4258) and teachers (n=1298) of Danish children and adolescents (ages 5 to 17 years), selected using simple random sampling, completed the FTF. In the largest study of the FTF to date, parent and teacher scores had acceptable psychometric properties. The FTF subdomains were organized into six domains labelled cognitive skills, motor/perception, emotion/socialization/behaviour, attention, literacy skills and activity control and analysis of these domains may provide additional information when applying the FTF in the future. The impact questions yielded information above and beyond that provided by symptom count alone and appeared to increase the ability of the FTF to identify at risk children and adolescents.

Psychometric evaluation of the Short Form 36 Health Survey (SF-36) and the World Health Organization Quality of Life Scale Brief Version (WHOQOL-BREF) for patients with schizophrenia.

Quality-of-life (QoL) instruments measure the overall health status of people with schizophrenia, for whom the activities of daily life are often difficult. However, information on the psychometric properties of scores from the Short Form 36 Health Survey (SF-36) and the World Health Organization Quality of Life Scale Brief Version (WHOQOL-BREF), 2 commonly used generic QoL instruments in this population, is limited. Thus, we used a multitrait-multimethod analysis plus confirmatory factor analysis (CFA) to examine their psychometric properties. To test the reliability of their scores, we used methods of absolute reliability (standard error of measurement [SEM] and smallest real difference [SRD]) and relative reliability (i.e., intraclass correlation coefficient [ICC]). We recruited 100 patients with schizophrenia from a psychiatric hospital in southern Taiwan. All participants filled out the SF-36 and the WHOQOL-BREF at baseline and 2 weeks later. The participants' QoL scores were lower than those of the Taiwan general population (ps < .01), and CFA indicated that the constructs of QoL scores for the SF-36 (comparative fit index [CFI] = .918; incremental fit index [IFI] = .919; Tucker-Lewis index [TLI] = .885) and the WHOQOL-BREF (CFI = .967; IFI = .967; TLI = .900) were acceptable. The SEM and SRD analyses suggested that the total scores of the SF-36 (SEM% = 10.03%; SRD% = 27.80%) and of the WHOQOL-BREF (SEM% = 5.55%; SRD% = 15.40%) were reliable. Also, our results demonstrated that the WHOQOL-BREF scores were more reliable and valid than the SF-36 scores for assessing people with schizophrenia. The scores of both questionnaires were valid and reliable and detected different aspects of QOL in the population with schizophrenia.

Occupational Self Efficacy Expectations among Indian Executives: Examining the Psychometric Properties of Occupational Self Efficacy Scale (OSES)

The purpose of the present study was to examine the psychometric properties of scores from occupational self efficacy scale (OSES) by Pethe et al. ( 1999 ). Data were obtained from Indian business executives, a population previously untested with OSES. Three interpretable factors were extracted as a result of exploratory factor analysis instead of six-factor structure proposed by the authors of the scale. Confirmatory factor analysis (CFA) results provided support for superiority of three-factor model over alternative models. All the three factors were found to be internally consistent. Results indicated that Pethe et al.’s scale, with minor modification, showed promise for self-efficacy research.

Rasch analysis of the Spanish version of the Mindful Attention Awareness Scale (MAAS) in a clinical sample

IntroductionThe clinical use of mindfulness has increased recently, and the Mindful Attention Awareness Scale (MAAS) has become one of the most used tools to measure it. The aim of this study was to test the effectiveness of mindfulness training and analyzing the psychometric properties of the MAAS scores in a clinical sample using the Rasch Model. MethodsOne hundred and ninety-nine participants with mood-anxiety clinical symptoms were recruited. The experimental group (n=103) received mindfulness training, and the control group (n=96) a conventional outpatient treatment for the same duration. The pre–post MAAS scores were analyzed to test the effectiveness of training, the psychometric properties of the scores, and differential item functioning (DIF) using the Rating Scale Model (RSM). ResultsMisfit in items 9 and 12, DIF in item 9, and Spanish translation problems in the items 5, 9 and 12 were observed. The repetition of the analysis without these items was decided. Appropriate dimensionality, fit and reliability values were obtained with the short version, MAAS-12. ConclusionsContrary to previous studies, the MAAS was sensitive to treatment-associated change. However, the commonly used MAAS has some translation and metric problems, and should be revised. MAAS-12 is a better scale than MAAS but suffers from construct under-representation. Constructing tools from a coherent theoretical perspective is suggested, so that all mindfulness facets are represented.

Análisis Rasch de la versión española de la escala Mindful Attention Awareness Scale en una muestra clínica

The clinical use of mindfulness has increased recently, and the Mindful Attention Awareness Scale (MAAS) has become one of the most used tools to measure it. The aim of this study was to test the effectiveness of mindfulness training and analyzing the psychometric properties of the MAAS scores in a clinical sample using the Rasch Model.One hundred and ninety-nine participants with mood-anxiety clinical symptoms were recruited. The experimental group (n=103) received mindfulness training, and the control group (n=96) a conventional outpatient treatment for the same duration. The pre-post MAAS scores were analyzed to test the effectiveness of training, the psychometric properties of the scores, and differential item functioning (DIF) using the Rating Scale Model (RSM).Misfit in items 9 and 12, DIF in item 9, and Spanish translation problems in the items 5, 9 and 12 were observed. The repetition of the analysis without these items was decided. Appropriate dimensionality, fit and reliability values were obtained with the short version, MAAS-12.Contrary to previous studies, the MAAS was sensitive to treatment-associated change. However, the commonly used MAAS has some translation and metric problems, and should be revised. MAAS-12 is a better scale than MAAS but suffers from construct under-representation. Constructing tools from a coherent theoretical perspective is suggested, so that all mindfulness facets are represented.

The Statistical and Methodological Acuity of Scholarship Appearing in the International Journal of Listening (1987–2011)

This article investigates the quality of social scientific listening research that reports numerical data to substantiate claims appearing in the International Journal of Listening between 1987 and 2011. Of the 225 published articles, 100 included one or more studies reporting numerical data. We frame our results in terms of eight recommendations to improve future listening scholarship. In particular, the results suggest needed variation in demographics and added attention to psychometric properties of scores. Standards for reporting and inspecting data should also be followed with more regularity, and tests of statistical assumptions along with information about missing data are urged. Effect sizes are rarely included in results, and no studies reported confidence intervals, suggesting overreliance on null hypothesis statistical testing when drawing implications for practice. Lastly, there were some noteworthy misappropriations of statistical techniques that are discussed.

Human Resource Development Climate in India: Examining the Psychometric Properties of HRD Climate Survey Instrument

The purpose of the present study was to examine the psychometric properties of scores from human resource development (HRD) climate survey instrument by Rao and Abraham (1986). Data were obtained from 403 Indian business executives working in both public and private sector business organizations. Six interpretable factors were extracted as a result of exploratory factor analysis instead of three dimensional structure proposed by the authors of the scale. Confirmatory factor analysis (CFA) results provided support for the superiority of six factor model over alternative models. All the six factors were found to be internally consistent. Results indicated that Rao and Abraham’s scale with sound psychometric properties showed promise for HRD research. Establishing the psychometric properties of the instrument will further accelerate the HRD research in India. Importantly, the study has made an attempt to address the lack of research and practice on organizational climate, which is a matter of concern as expressed by several researchers, and revive the interest in climate studies to some extent.

Psychometric Properties of Raw and Scale Scores on Mixed-Format Tests

This paper illustrates that the psychometric properties of scores and scales that are used with mixed-format educational tests can impact the use and interpretation of the scores that are reported to examinees. Psychometric properties that include reliability and conditional standard errors of measurement are considered in this paper. The focus is on mixed-format tests in situations for which raw scores are integer-weighted sums of item scores. Four associated real-data examples include (a) effects of weights associated with each item type on reliability, (b) comparison of psychometric properties of different scale scores, (c) evaluation of the equity property of equating, and (d) comparison of the use of unidimensional and multidimensional procedures for evaluating psychometric properties. Throughout the paper, and especially in the conclusion section, the examples are related to issues associated with test interpretation and test use.

Reynolds adolescent depression scale‐second edition: a reliable and useful instrument

The authors conducted two studies to address issues of the dimensionality, scale reliability, and psychometric properties of scores on the Reynolds Adolescent Depression Scale-Second Edition (RADS-2; Reynolds, 2002) in samples of adolescent psychiatric inpatients. In Study 1 (N=262), they used bifactor analysis to further evaluate the general and specific components of the RADS-2. In Study 2 (N=196), they used confirmatory factor analysis to evaluate the fit of a 1-factor model, the original 4-factor model, a second-order model, and a bifactor model to a new sample data. In both studies, the total RADS-2 and content-specific subscale scores showed acceptable estimates of reliability (i.e., scale reliability estimates >.80). Estimates of concurrent validity were also examined. Scores of the RADS-2 total and content-specific subscale scores were useful in differentiating between the responses of youth with mood disorder diagnoses and those with other primary psychiatric disorder diagnoses. The authors also conducted correlation analyses to identify potential correlates for the total RADS-2 scale and the proposed subscale scores.