Evidence Of Incremental Validity Research Articles

Development, Validation, and Faking-Resistance of an Implicit Measure of Psychopathy in the Workplace

ABSTRACT Researchers have called for faking-resistant measures of psychopathic personality that can be self-administered in high-stakes contexts (e.g. hiring). We developed and validated an implicit measure of psychopathy contextualized in workplace situations. We first detail how the measure is framed, conceptualized, and rooted in psychopathy literature. We then describe the item development process, and Study 1 involves expert review and refining the item list. In Study 2 (N = 396), we examine internal consistency and factor structure for a 22-item version of the measure. In Study 3 (N = 251), we demonstrate test-retest reliability, construct-related validity, and provide initial evidence for criterion-related validity through a two-wave study. Study 4 analyzes the measure using item response theory, based on a sample of 6,746 job seekers, demonstrating effectiveness for measuring high levels of psychopathy. In Study 5 (N = 219) we provide evidence of faking-resistance and criterion-related validity with behavioral (two weeks later) and self-report (one year later) outcomes. Finally, Study 6 provides promising evidence of incremental validity using an organizational sample (N = 615). Overall, scores on this new implicit measure are reliable, with acceptable construct-related, criterion-related, and incremental validity, while also being faking-resistant. Implications for use in workplace settings are discussed.

The Depression Anxiety Stress Scales for Youth (DASS-Y): evidence of validity and cross-national and gender measurement invariance.

The Depression Anxiety Stress Scales for Youth (DASS-Y) is a recently developed measure of unpleasant emotional experiences for children and adolescents; however, the evidence for its validity is still limited. The present study evaluated the evidence for the structural, convergent, and incremental validity of the DASS-Y on a sample of Serbian adolescents aged 15-19 years (N = 421; 48% female, mean age = 16.86). Measurement invariance across gender and two countries (Serbia and Australia) was also tested. The findings supported the bifactor exploratory structural equation modeling (bifactor-ESEM) representation of the DASS-Y, which includes the General factor of negative affect/emotional distress, and three specific factors of Depression, Anxiety, and Stress. The DASS-Y factors showed evidence of incremental validity in predicting global externalizing symptoms and demonstrated convergent validity in relation to measures of global internalizing and externalizing symptoms as well as subjective well-being. The bifactor-ESEM model of the DASS-Y was found to be invariant across gender and two countries. The present study findings highlight the importance of distinguishing between general and specific components of unpleasant emotional states.

Psychometric properties of a Mandarin Chinese version of the Body Appreciation Scale-2 among Chinese adolescents

In the present study, we explored the factor structure and psychometric properties of the Mandarin Chinese BAS-2 among adolescents residing in the Chinese mainland. Exploratory factor analysis in Study 1 (N = 790; 396 girls, 394 boys) supported the unidimensionality of the Mandarin Chinese BAS-2 among Chinese adolescents. Internal consistency reliability was upheld via McDonald’s omega. Convergent validity was supported by its moderate-to-strong relationships with body satisfaction, functionality satisfaction, self-esteem, life satisfaction, positive affect, and negative affect, while its small-to-moderate correlation with social desirability provided somewhat weaker discriminant validity support. Criterion-related validity was upheld by its inverse correlation with eating disorder symptomatology and positive correlation with intuitive eating. It explained unique variance in self-esteem (for girls and boys), eating disorder symptomatology (for girls), and intuitive eating (for boys) beyond age, body satisfaction, and functionality satisfaction, providing incremental validity evidence. A subsample of 134 girls and 114 boys completed the Mandarin Chinese BAS-2 again after three months, and test-retest reliability was upheld. The confirmatory factor analysis in Study 2 (N = 337; 192 girls, 145 boys) replicated the unidimensional structure and supported measurement invariance across gender. Collectively, the present study supported the unidimensionality, reliability, and validity of the Mandarin Chinese BAS-2′s scores among Chinese adolescents.

Development and Validation of the Online Sexual Objectification Experiences Scale Among College Women

In this study, we developed and psychometrically evaluated the Online Sexual Objectification Experiences Scale (OSOES). We collected data from 816 emerging adult college women across two samples. Exploratory and confirmatory factor analyses revealed three factors: Unwanted Sexual Advances (example item: “How often have you received unwanted messages online asking about your sexual experience?”), Negative Appearance Commentary (example item: “How often has another person insulted you by calling you unattractive online?”), and Indirect Sexual Objectification (example item: “How often have you read a sexual comment posted to a photo of another woman's body online?”). OSOES scores were internally consistent and were strongly correlated with scores of self-objectification, body surveillance, and body shame. OSOES subscale scores also provided evidence of incremental validity in predicting self-objectification and objectified body consciousness above and beyond offline interpersonal sexual objectification. We hope that findings from the present study and the OSOES can be used by researchers, clinicians, and educators to understand the frequency of online sexual objectification experiences among women and as a result, work to mitigate harmful outcomes. Additional online materials for this article are available on PWQ's website at http://journals.sagepub.com/doi/suppl/DOI:10.1177/03616843241250295

Workplace Telepressure and Work Rumination: Evidence of Incremental Validity Beyond Workaholism

Workplace Telepressure and Work Rumination: Evidence of Incremental Validity Beyond Workaholism

The development and validation of the experiential avoidance rating scale (EARS): A self-report questionnaire that might actually measure experiential avoidance

Serious concerns have been raised regarding the validity of popular experiential avoidance measures. We developed a new “experiential avoidance rating scale” (EARS) to address some of the psychometric concerns with existing measures. Candidate items were generated according to the original functional contextual definition of experiential avoidance and reviewed by experts and public members for content validity and readability. Items then underwent validation involving n = 3628 participants. Exploratory factor analyses (n = 1069) identified six items with good underlying common variance. Confirmatory factor analyses from college student (n = 1068) and community samples (n = 1413) demonstrated excellent fit. In all samples, the EARS demonstrated evidence of content, factorial, concurrent, convergent, and discriminant validity. The EARS also demonstrated evidence of incremental validity by predicting significant additional variance above-and-beyond neuroticism and extant experiential avoidance measures in depression, anxiety, post-traumatic stress, alcohol use problems, and general mental health measures. Measurement invariance analyses indicated that the EARS demonstrated residual (strict) invariance between cisgender men and women; clinical and non-clinical participants; White Non-Hispanic, White Hispanic, Black, and Asian participants; and heterosexual and bisexual individuals. We suggest that the EARS be further investigated as a brief measure of experiential avoidance in additional studies, particularly with clinical populations. We also discuss areas of future psychometric study and implications.

Affectively effective: Work-related emotional intelligence as a predictor of organizational citizenship.

Efforts to link ability-related emotional intelligence to organizational behavior have resulted in modest findings. The present three studies examine whether a work-contextualized form of emotional intelligence (W-EI) may have greater predictive value, particularly in the organizational citizenship domain. Because W-EI should benefit social relationships within the workplace, positive associations between W-EI and organizational citizenship behavior were hypothesized. This hypothesis was supported in three studies (total N = 462) involving samples of part-time student employees (Study 1), postdoctoral researchers (Study 2), and full-time employees (Study 3). All studies also provided evidence for incremental validity, such as with respect to the Big 5 personality traits, and Study 3 highlighted processes related to workplace engagement (in the form of higher levels of interpersonal job satisfaction and lower levels of burnout). The results demonstrate the importance of W-EI in understanding employee variations in organizational citizenship.

Psychometric properties of a Mandarin Chinese version of the Body Appreciation Scale-2 with residents from Chinese mainland

The current study aimed to translate the Body Appreciation Scale-2 (BAS-2) into Mandarin Chinese and explore its psychometric properties with college students from the Chinese mainland. In Study 1 (N = 840, 415 women, 425 men), an exploratory factor analysis upheld the unidimensional factor structure of the Mandarin Chinese BAS-2, and McDonald’s omega supported the internal consistency reliability of its scores. Evidence of construct validity of the Mandarin Chinese BAS-2 was also accrued via its positive correlations with self-esteem, self-compassion, and body satisfaction as well as negative correlations with BMI, weight discrepancy, negative affect, and body surveillance. Its negative correlation with eating disorder symptomatology and negligible correlation with impression management supported its criterion-related validity and discriminant validity, respectively. Additionally, the Mandarin Chinese BAS-2 contributed unique variance in eating disorder symptomatology and self-esteem beyond the variance explained by BMI and other body satisfaction measures, providing evidence of incremental validity. Study 2 (N = 522, 322 women, 230 men) confirmed its unidimensional factor structure, upheld the internal consistency and stability of its scores over a 3-month period, and supported its measurement invariance between women and men. Taken together, these findings support the cross-cultural validity of the Mandarin Chinese BAS-2 on the Chinse mainland.

BMAT's predictive validity for medical school performance: A retrospective cohort study

BackgroundAlthough used widely, there is limited evidence of the BioMedical Admissions Test's (BMAT) predictive validity and incremental validity over prior educational attainment (PEA). We investigated BMAT's predictive and incremental validity for performance in two undergraduate medical schools, Imperial College School of Medicine (ICSM), UK, and Lee Kong Chian School of Medicine (LKCMedicine), Singapore. Our secondary goal was to compare the evidence collected with published evidence relating to comparable tools.MethodsThis was a retrospective cohort study of four ICSM (1188 students, entering 2010–2013) and three LKCMedicine cohorts (222 students, 2013–2015). We investigated associations between BMAT Section 1 (‘Thinking Skills’), Section 2 (‘Scientific Knowledge and Applications’) and Section 3a (‘Writing Task’) scores, with written and clinical assessment performance across all programme years. Incremental validity was investigated over PEA (A‐levels) in a subset of ICSM students.ResultsWhen BMAT sections were investigated independently, Section 2 scores predicted performance on all written assessments in both institutions with mainly small effect sizes (standardised coefficient ranges: ICSM: 0.08–0.19; LKCMedicine: 0.22–0.36). Section 1 scores predicted Years 5 and 6 written assessment performance at ICSM (0.09–0.14) but nothing at LKCMedicine. Section 3a scores only predicted Year 5 clinical assessment performance at ICSM with a coefficient <0.1. There were no positive associations with standardised coefficients >0.1 between BMAT performance and clinical assessment performance. Multivariable regressions confirmed that Section 2 scores were the most predictive. We found no clear evidence of incremental validity for any BMAT section scores over A‐level grades.DiscussionSchools who wish to assess scientific knowledge independently of A‐levels may find BMAT Section 2 useful. Comparison with previous studies indicates that, overall, BMAT seems less useful than comparable tools. Larger scale studies are needed. Broader questions regarding why institutions adopt certain admissions tests, including those with little evidence, need consideration.

Dimensional Model of Personality Disorder Incorporated Into ICD-11

Dimensional Model of Personality Disorder Incorporated Into ICD-11

Validation of the General Medicine in-Training Examination Using the Professional and Linguistic Assessments Board Examination Among Postgraduate Residents in Japan.

PurposeIn Japan, the General Medicine In-training Examination (GM-ITE) was developed by a non-profit organization in 2012. The GM-ITE aimed to assess the general clinical knowledge among residents and to improve the training programs; however, it has not been sufficiently validated and is not used for high-stake decision-making. This study examined the association between GM-ITE and another test measure, the Professional and Linguistic Assessments Board (PLAB) 1 examination.MethodsNinety-seven residents who completed the GM-ITE in fiscal year 2019 were recruited and took the PLAB 1 examination in Japanese. The association between two tests was assessed using the Pearson product-moment statistics. The discrimination indexes were also assessed for each question.ResultsA total of 91 residents at 17 teaching hospitals were finally included in the analysis, of whom 69 (75.8%) were women and 59 (64.8%) were postgraduate second year residents. All the participants were affiliated with community hospitals. Positive correlations were demonstrated between the GM-ITE and the PLAB scores (r = 0.58, p < 0.001). The correlations between the PLAB score and the scores in GM-ITE categories were as follows: symptomatology/clinical reasoning (r = 0.54, p < 0.001), physical examination/procedure (r = 0.38, p < 0.001), medical interview/professionalism (r = 0.25, p < 0.001), and disease knowledge (r = 0.36, p < 0.001). The mean discrimination index of each question of the GM-ITE (mean ± SD; 0.23 ± 0.15) was higher than that of the PLAB (0.16 ± 0.16; p = 0.004).ConclusionThis study demonstrates incremental validity evidence of the GM-ITE to assess the clinical knowledge acquisition. The results indicate that GM-ITE can be widely used to improve resident education in Japan.

Psychometric properties of an Italian translation of the Functionality Appreciation Scale (FAS)

The Functionality Appreciation Scale (FAS; Alleva et al., 2017) is a widely used instrument for the measurement of individual’s appreciation of their body for what it can do and is capable of doing (i.e., functionality appreciation). Here, we examined the psychometric properties of a novel Italian translation of the FAS. A sample of 950 Italian adults completed the FAS, as well as previously validated measures of body image (body appreciation, body esteem, body surveillance), disordered eating symptoms, and psychological well-being (self-esteem, general distress). Exploratory and confirmatory factor analyses supported a 1-dimensional model of FAS scores, with all 7 items retained. Scores achieved scalar invariance across gender, and the gender difference in latent FAS scores was not significant. FAS scores were found to have adequate internal consistency, test-retest reliability up to three weeks, and convergent and criterion-related validity through significant correlations with all additional constructs. However, evidence of incremental validity was weak, and likely reflected high nomological and conceptual similarity between functionality and body appreciation in this national context. Overall, these results provide strong evidence that scores on the Italian FAS are psychometrically valid, which may aid future development of interventions to promote more positive body image in Italian-speaking samples.

Isolation, self-blame and perceived invalidation in aid personnel: identifying humanitarian-specific distress using the PostAID/Q

ObjectiveHumanitarian-specific psychological distress following deployment can elude detection using contemporary measures of trauma-related stress. This study assesses the unidimensional structure and convergent validity of the Post-deployment Altruistic Identity Disruption Questionnaire (PostAID/Q), an 18-item questionnaire underpinned by the construct Altruistic Identity/Disruption (AI/AID).MethodHumanitarian aid personnel (N=108) completed an online web survey, inclusive of the Moral Injury Questionnaire (MIQ), Posttraumatic Distress Disorder Checklist (PCL-5), Psychological Well-Being Posttraumatic Changes Questionnaire (PWB-PTCQ) and Social Provisions Scale (SPS).ResultsA confirmatory factor analysis suggested a single factor structure providing further support for the conception of AI/AID as a unidimensional construct. Convergent validity was demonstrated through (1) utility for predicting a posttraumatic stress disorder (PTSD) diagnosis assessed by the PCL-5, and (2) moral injury assessed by the MIQ. The PostAID/Q was further moderately and negatively associated with the availability of social support (assessed by the SPS) and lower self-reports of psychological well-being post trauma (assessed by the PWB-PCTQ). Finally, the PostAID/Q demonstrated evidence of incremental validity in predicting humanitarian specific psychological distress over and above the PCL-5. Specifically, the PostAID/Q predicted increased moral injury on the MIQ, and decreased psychological well-being post trauma.ConclusionsThe PostAID/Q can assist in identifying humanitarian specific psychological responses post deployment guiding support for personnel, over and above more traditional measures of posttraumatic stress.

Development and validation of a new adolescent self-report scale to measure loss of interest and pleasure: The Anhedonia Scale for Adolescents.

Anhedonia, the loss of interest and pleasure in previously enjoyable experiences, is a core symptom of depression and a characteristic of other mental health and physical health problems. Most self-report measures of anhedonia has been developed for use with adults and their suitability for adolescents is questionable. In this article, we describe the development and psychometric qualities of a new measure, the Anhedonia Scale for Adolescents (ASA), designed specifically for use with adolescents aged 11-18 years. Items were generated from in-depth qualitative interviews with depressed young people, and then reviewed by an independent group of young people and clinically qualified experts in adolescent mental health. After piloting the new scale (n = 66), we established the structural validity of the measure with two groups of young people using exploratory (n = 1057) and confirmatory (n = 1041) factor analysis. The final scale consisted of 14 items, with 1 general factor and 3 specific factors producing the best fit to the data, (1) Enjoyment, Excitement, and Emotional Flattening (negatively framed); (2) Enthusiasm, Connection, and Purpose (positively framed); and (3) Effort, Motivation, and Drive (negatively framed). The ASA had high test-retest reliability and converged with standardized measures of depression, negative symptoms of schizophrenia, pleasure, and positive affect. Findings from these analyses provided evidence of incremental validity, as the ASA was a stronger predictor of clinical group (high vs. low depressive symptoms) than existing measures used to assess anhedonia. The ASA has potential as a new clinical and research tool to assess adolescent anhedonia. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Adapting “Color Blindness” to South Korean Attitudes Toward Multicultural Minorities: Scale Development

Four studies were conducted to develop and validate the South Korean Attitudes Toward Multicultural Minorities Scale (SKAMMS). Exploratory factor analysis ( n = 336) identified three factors: Unawareness of South Korean Privilege and Discrimination against Multicultural Minorities, Attitudes Against Multiculturalism, and Attitudes Against Advocacy and Policy for Multicultural Minorities. Confirmatory factor analysis ( n = 537) provided cross-validation of the 16-item, three-factor model and identified that a three-factor bifactor model best fit the data. Correlational analyses provided evidence of the discriminant and convergent validities of the SKAMMS. Evidence of incremental validity of the SKAMMS was obtained ( n = 165), where the SKAMMS accounted for additional variance in criterion variables (i.e., cognitive, affective, and behavioral reactions toward racism) above other measures of attitudes toward multicultural minorities. The estimated 2-week test-retest reliabilities ( n = 53) ranged from .67 to .82. We discuss implications for the use of the SKAMMS for practice, advocacy, education/training, and research.

Wellbeing Literacy: Conceptualization, Measurement, and Preliminary Empirical Findings from Students, Parents and School Staff.

Aims: Wellbeing literacy is the intentional use of wellbeing relevant vocabulary, knowledge and language skills to maintain or improve the wellbeing of oneself, others and the world. In this study, we operationalize the human aspects of the concept of wellbeing literacy and empirically test its relationship with wellbeing and illbeing. We also assess its incremental variance in wellbeing and illbeing, after controlling for existing and well-established predictors of these constructs within education settings. Methods: We developed and empirically tested the Wellbeing literacy 6-item (Well-Lit 6) scale to assess the concept of wellbeing literacy in the education context. The scale was developed based on a working definition of wellbeing literacy, in combination with the Australian Curriculum, Assessment and Reporting Authority (ACARA)’s definition of literacy. The Well-Lit 6 was administered via a cross-sectional survey to three Australian samples that comprise different elements of Australian education systems: students (N = 1392), parents (N = 584) and school staff (N = 317). Results: Exploratory factor analysis (EFA) suggested the six items of the Well-Lit 6 form an independent construct, empirically distinguishable from other wellbeing-related constructs (e.g., general wellbeing, resilience, and emotion regulation). Convergent analyses showed wellbeing literacy was positively related to wellbeing and negatively related to illbeing. Incremental validity analyses showed wellbeing literacy predicted variance in wellbeing and illbeing after controlling for participant demographics, resilience, and emotion regulation, showing initial evidence of incremental validity. Conclusions: Our results provide preliminary evidence that wellbeing literacy is a distinct construct from wellbeing and illbeing, and it also demonstrates significant unique variance in these constructs over and above resilience and emotion regulation. The Well-Lit 6 is a useful provisional measure of wellbeing literacy, although we suggest a fruitful avenue for future research is to develop a more comprehensive scale of wellbeing literacy that denotes specific facets of communication, allowing a fuller exploration wellbeing literacy, its components, and their antecedents and consequences. We offer further recommendations for future research and discuss limitations with our approach.

Is God Listening to My Prayers? Initial Validation of a Brief Measure of Perceived Divine Engagement and Disengagement in Response to Prayer

Does God listen and respond to prayers? This project provided initial validation for a brief measure of perceived divine engagement and disengagement in response to prayer. As part of a larger project on religious/spiritual struggles among U.S. undergraduates, we used Sample 1 (n = 400) for exploratory factor analysis and Sample 2 (n = 413) for confirmatory factor analysis and initial validity testing. A two-factor model with four items per factor provided acceptable fit. On average, participants reported more divine engagement than disengagement. They endorsed items about God listening more than those about God responding. Divine engagement showed strong positive associations with religiousness and positive-valence variables involving God. Divine disengagement showed strong positive associations with variables suggesting divine struggle or distance. Importantly, both subscales also showed evidence of incremental validity: Divine engagement predicted positive-valence God variables (e.g., secure attachment, collaborative religious coping, gratitude to God, and awareness of God) even when controlling for religiousness and positive God concepts and attitudes. Divine disengagement predicted more spiritual struggles and more negative-valence and distance-related God variables (divine struggle, anxious and distant attachment, and self-directing religious coping) even when controlling for doubt about God’s existence, negative God images, anger/disappointment toward God, and concern about God’s disapproval. In short, this brief new measure shows promise as a tool to assess beliefs about God’s responsiveness to prayer.

Incremental Validity between the Wartegg and Rorschach tests (R-PAS)

Abstract Incremental validity indicates how much a measure can add prevision to a criterion, more than what can be previewed by other sources of data. In other words, it means how an instrument can complement and aid on information comprehension derived from another. The objective of the study was to verify evidence of incremental validity between the Wartegg and the Rorschach tests (R-PAS). A total of 40 subjects with ages varying between 21 to 70 years participated, divided into two groups, one composed by schizophrenia diagnosis and another, by subjects with a history of psychiatric diseases. Everybody responded to the Rorschach and Wartegg tests. The results indicated predictive capacity among the instruments of 75% for the variable Formal Quality, 98% for Movement and 100% for Content. New studies are suggested about validity evidences with larger samples as well as the analysis of other variables, not explored in this study.

Testing the Psychometric Properties of the Newly Developed ACTive Values Wheel

Acceptance and commitment therapy (ACT) emphasizes values and behavioral commitments to help individuals achieve goals and manage emotions. This study evaluated the preliminary psychometric properties of the ACTive Values Wheel, a smartphone application-based psychometric instrument. ACTive provides a weighted and idiographic index of individuals’ self-reported values-directed behavior. One hundred sixty adults participated on an individual basis to complete the ACTive Values Wheel and questionnaire measures of well-being, psychological distress, psychological flexibility, and valued living to assess the convergent, incremental, discriminant, and criterion-related validity of the ACTive Values Wheel. The ACTive Values Wheel showed evidence of convergent and criterion-related validity through significant positive correlations with existing measures of valued living, emotional well-being, and openness to experience, respectively. Likewise, discriminant validity was evidenced by a nonsignificant correlation with age. However, evidence of incremental validity over and above existing measures of valued living was not found. These findings provide preliminary support for the psychometric properties of the ACTive Values Wheel. Accounting for the methodological limitations of the study, the ACTive Values Wheel shows potential as an accessible and interactive measure of self-reported values-directed behavior that could be extended to other languages and cultural contexts.

Neural and self-reported reward responsiveness are associated with dispositional affectivity and emotion dysregulation in adolescents with evidence for convergent and incremental validity.

Adolescence is a developmental period characterized by heightened reward sensitivity which, in turn, confers risk for pertinent negative outcomes, underscoring the need to better understand biological bases and behavioral correlates of reward responsiveness during this developmental phase. Our goals in the current study were to examine, in a sample of 43 typically developing adolescents (Mage =15.67years; SD=1.01; 32.6% boys), (1) evidence of convergent validity between neural and self-report reward responsiveness, (2) associations between neural reward responsiveness and self-report dispositional affectivity and emotion dysregulation (ED) and (3) evidence of incremental validity of self-report beyond neural reward responsiveness in predicting affectivity and ED. During electroencephalography (EEG), adolescents completed two experimental paradigms probing event-related potential (ERP) indices of reward anticipation and initial responsiveness to reward attainment. Following EEG, they completed self-report measures of reward responsiveness, affectivity, and ED. Findings indicated some evidence of convergent validity between enhanced ERP indices of reward anticipation and initial response to reward and greater reinforcement sensitivity; that ERP indices of both reward responsiveness aspects predicted lower negative affectivity and less ED; and evidence of incremental validity of self-report beyond neural reward responsiveness in predicting outcomes. Results underscore the utility of a multi-method framework in assessing adolescent reward responsiveness and support the relevance of reward responsiveness in explaining individual differences in dispositional affectivity and ED.