Incremental Validity Analyses Research Articles

Development and validation of makeup and sexualized clothing questionnaires

BackgroundBody acceptance programs on college campuses indicated that collegiate women often report feeling pressure to dress in a sexualized manner, and use makeup to enhance beauty. Currently, no quantitative measures exist to assess attitudes and daily behaviors that may arise in response to perceived pressure to wear makeup or dress in a provocative manner. The goal of the current studies was to develop brief self-report questionnaires aimed at assessing makeup and sexualized clothing use and attitudes in young women.MethodsAn exploratory factor analysis in a sample of 403 undergraduate women was used in Study 1 to create items to measure the pressure women feel to wear makeup and sexualized clothing. A confirmatory factor analysis (N = 153) was used in Study 2 to confirm the factor structure found in Study 1. An incremental validity analysis was also conducted in Study 2. Across both studies, participants completed online questionnaires.ResultsIn Study 1, items were developed for two questionnaires to assess perceived pressure to wear makeup and discomfort when not wearing makeup, and perceived pressure to wear sexualized clothing, and body image concerns with regards to sexualized clothing. The exploratory factor analyses revealed Unconfident and Unease scales for the Makeup Questionnaire (MUQ) and Body Dissatisfaction and Pressure scales for the Sexualized Clothing Questionnaire (SCQ). In Study 2, the confirmatory factor analyses confirmed the factor structure for the MUQ and SCQ. The incremental validity analysis revealed that these measures can be used to predict self-objectification and shape and weight concern in women.ConclusionThese studies provide preliminary support for the factor structure of two novel questionnaires aimed at assessing perceived pressure to wear makeup and sexualized clothing.

Resilience or hope? Incremental and convergent validity of the resilience scale for adults (RSA) and the Herth hope scale (HHS) in the prediction of anxiety and depression

BackgroundHope and resilience protect against inner vulnerabilities or harsh life circumstances; they explain individual differences in physical or mental health outcomes under high stress. They have been studied in complementary or competing theoretical frameworks; therefore, the study of measures of hope and resilience should be undertaken prior to explore if they are truly value-added for research. This study investigates the convergent and incremental validity of the Resilience Scale for Adults (RSA) and the Herth Hope Scale (HHS), in the prediction of anxiety and depression (HSCL-25).MethodsParticipants in this community-based sample are 762 adults from 18 to 74 years old. They answered the RSA, HHS, Spanish Language Stressful Life-Events Checklist (SL-SLE), and the Hopkins Symptom Checklist-25 (HSCL-25). Incremental validity analyses combined hierarchical regression and structural equation models (SEM). First, hierarchical regression models were compared based on three criteria (R2Diff.,ΔF, and semi-partial r), then the direct effect of resilience on affective symptoms was compared with the mediated effect of resilience on affective symptoms through hope.ResultsThe hierarchical models showed that (1) hope and resilience account significantly for the variance of affective symptoms above age, sex, and life-stress; (2) Resilience Total score has greater incremental validity than positive scales of HHS Hope; and (3) RSA Total score, HHS Optimism/Spiritual support, Stressful life-events and sex are unique predictors of affective symptoms. The SEM analyses verified a stronger direct effect of resilience in the prediction of affective symptoms above the significant partial mediated effect of resilience through hope. Additionally, results show that age and better educational opportunities were associated with protection (i.e. resilience and hope) and emotional well-being (i.e. affective symptoms and hopelessness). Women showed higher scores in social competences and resources (RSA), interconnectedness and initiative to take action (HHS). However, they have poorer evaluations of own abilities and efficacy (RSA), and higher scores in all the affective symptoms assessed.ConclusionThe RSA has incremental validity above the HHS, however, both the RSA and the HHS are effective, differentiated and complementary measures of protection that are of high relevance for research on psychosocial and emotional well-being.

Predictive Validity of Spatial Ability and Perceptual Speed Tests for Aviator Training

ABSTRACTObjective: To present a multistudy examination of spatial ability and perceptual speed tests as predictors of Air Force aviator academic and flying training outcomes.Background: Whereas verbal and math tests are commonly used in academic and pre-employment selection, spatial ability and perceptual speed testing are less common. Spatial ability and perceptual speed tests have been a mainstay in U.S. Air Force aviation training selection batteries for decades, as part of the process for selecting aviation training candidates.Method and Results: Factor analysis (Study 1) of spatial, perceptual speed, and academic aptitude tests shows a 2-factor solution. Meta-analysis results (Study 2) of the predictive validity of spatial and perceptual speed tests relative to verbal, math, and technical knowledge tests show spatial ability, aviation knowledge, and perceptual speed tests are better significant predictors of aviator flying performance, whereas academic aptitude best predicts classroom performance. Incremental validity analyses (Study 3) of spatial and perceptual speed above academic and technical knowledge for academic and pilot trainee flying performance were performed. Although spatial ability lacks incremental validity over academic and technical aptitude, perceptual speed has incremental validity above academic and technical aptitude for predicting pilot trainee flying performance.Conclusion: Perceptual speed, an aspect of spatial ability, can provide incremental predictive validity beyond academic and technical knowledge measures alone for applied science, technology, engineering, and mathematics fields like military aviation.

Improving measurement of attributional style in schizophrenia; A psychometric evaluation of the Ambiguous Intentions Hostility Questionnaire (AIHQ)

While attributional style is regarded as a core domain of social cognition, questions persist about the psychometric characteristics of measures used to assess it. One widely used assessment of attributional style is the Ambiguous Intentions Hostility Questionnaire (AIHQ). Two limitations of the AIHQ include (1) a possible restricted range resulting from too few and too homogenous item scenarios, and (2) use of rater scores that are cumbersome and time-consuming to score and have unknown incremental validity. The present study evaluated the psychometric properties of the AIHQ while concurrently testing changes aiming to improve the scale, in particular expansion of the number of self-report items and removal of the rater-scored items. One hundred sixty individuals diagnosed with schizophrenia and 58 healthy controls completed the full AIHQ along with measures of symptoms, functioning, and verbal intelligence. The AIHQ – particularly the self-reported blame score – demonstrated adequate internal consistency, test-retest reliability, and distinguished patients from controls. It also was significantly related to clinically-rated hostility and suspiciousness symptoms, and correlated with functional capacity even after controlling for verbal intelligence. Incremental validity analyses suggested that a higher number of self-report items strengthens relationships to outcomes in a manner that justifies this expansion, while rater-scored items had mixed results in providing additional information beyond self-report in the AIHQ.

Is health literacy an example of construct proliferation? A conceptual and empirical evaluation of its redundancy with general cognitive ability

During the last 20years, health literacy has been promoted as an important determinant of individual and group differences in health outcomes. Despite a definition and pattern of associations with health outcomes highly similar to ‘g’ (i.e., the general cognitive ability factor), health literacy has been conceptualized as a distinct construct. This study evaluates the conceptual and empirical distinctiveness of health literacy. A sample of 167 students from a southeastern urban university (117 females and 50 males) between the ages of 18 and 53 (M=21.31, SD=5.61) completed a cognitive ability battery, three health literacy tests, two knowledge tests, and a questionnaire assessing 12 health behaviors and health outcomes. Across 36 tests of criterion-related validity, cognitive ability had an effect in all 36 cases, where the health literacy tests only showed measureable incremental validity in 6 of 36 cases. Factor analysis revealed only three factors defined by the traditional ability tests with the health literacy measures loading on the ability factors as predicted by the content validity analysis. There was no evidence of a health literacy factor. The combined results from a comparative content analysis, an empirical factor analysis, and an incremental validity analysis cast doubt on the uniqueness of a health literacy construct. It is suggested that measures of health literacy are simply domain-specific contextualized measures of basic cognitive abilities. Implications for linking these literatures and increasing our understanding of the influence of intellectual factors on health are discussed.

Examining the Incremental Validity and Relative Importance of Specific Cognitive Abilities in a Training Context

We examine the extent to which g and specific abilities predict learning in a training context by juxtaposing contrasting theories of cognitive ability (i.e., Spearmanian theory and nested-factors theory) and using an analytical strategy (i.e., multidimensional importance analysis) that aligns with underlying theoretical assumptions of both theories. When conducting incremental validity analysis (Spearmanian approach), specific abilities did not add much to the prediction of learning. However, when conducting relative weights analysis (nested-factors approach), specific abilities were the dominant predictors of learning. Results suggest different theoretical and analytical approaches can lead to different results. Results also suggest specific abilities may provide predictive utility beyond g in the training context, but recognizing this utility depends on one’s theoretical assumptions and methodological approach.

Personality Assessment Inventory scores as predictors of misconduct among sex offenders civilly committed as sexually violent predators.

We examined the usefulness of scores on the Personality Assessment Inventory (PAI; Morey, 1991) in predicting treatment program violations among 76 sexual offenders civilly committed as sexually violent predators. Scores on the Borderline Features scale (area under the curve [AUC] = .69, p = .005) and Negative Relationships subscale (BOR-N: AUC = .71, p < .001) were the strongest predictors of misconduct, outperforming scores on scales designed to predict poor treatment amenability and antisocial behavior. Incremental validity analyses indicated that BOR scores made a significant contribution to the prediction of misconduct after controlling for scores on measures of overall self-reported distress (e.g., Mean Clinical Elevation, Negative Impression), which were also predictive of program violations. Overall, our findings point to the potential utility of integrating components of treatment for borderline personality disorder into sex offender treatment.

Development and validation of the Specific Loss of Interest and Pleasure Scale (SLIPS)

BackgroundAnhedonia, or loss of pleasure and/or interest in people or things, is a key component of many forms of psychopathology. Extant anhedonia measures only assess current state or longstanding trait anhedonia, which risks discounting recent changes. Thus, we created the Specific Loss of Interest and Pleasure Scale (SLIPS), to assess recent changes in anhedonia. MethodsFour studies assessed the reliability and validity of the SLIPS through factor and internal consistency analyses, convergent and divergent validity analyses, and incremental validity analyses in individuals reporting non-clinical and clinical levels of psychopathology. ResultsThe SLIPS was internally consistent, converged with existing state measures of anhedonia and depression, and diverged from general measures of affect and trait anticipatory and consummatory pleasure. Furthermore, the SLIPS predicted recent changes in anhedonia independent of existing measures. Moreover, in respondents reporting moderate or severe symptoms of depression, only the SLIPS was associated with recent changes in anhedonia. LimitationsProspective limitations included a resulting one factor solution and the potential that the method of scaling influenced results. However, these limitations were offset by empirical verification and the unique strength of the scaling of the SLIPS. ConclusionsThe SLIPS is the first validated measure that allows for the assessment of recent changes in anhedonia in non-clinical and clinical samples. Results suggest that the SLIPS offers unique clinical utility by discriminating changes in anhedonia within at-risk and clinically depressed groups.

Structural and incremental validity of the Wechsler Adult Intelligence Scale–Fourth Edition with a clinical sample.

Structural and incremental validity of the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV; Wechsler, 2008a) was examined with a sample of 300 individuals referred for evaluation at a university-based clinic. Confirmatory factor analysis indicated that the WAIS-IV structure was best represented by 4 first-order factors as well as a general intelligence factor in a direct hierarchical model. The general intelligence factor accounted for the most common and total variance among the subtests. Incremental validity analyses indicated that the Full Scale IQ (FSIQ) generally accounted for medium to large portions of academic achievement variance. For all measures of academic achievement, the first-order factors combined accounted for significant achievement variance beyond that accounted for by the FSIQ, but individual factor index scores contributed trivial amounts of achievement variance. Implications for interpreting WAIS-IV results are discussed.

Different Fit Perceptions in an Academic Environment

This study examines whether students perceive three different types of fit in an academic environment (i.e., interest–major [I–M] fit, demands–abilities [D–A] fit, and needs–supplies [N–S] fit) and whether these factors predict important academic and well-being criteria using a Chinese student sample. Results from confirmatory factor analyses support the assumption that students differentiate among the three fit perceptions. Incremental validity analysis and relative importance analysis reveal that the three types of fit each have a unique relationship with situational outcomes. Specifically, I–M fit predicts greater proportions of variance in major change intention; D–A fit predicts greater proportions of variance in academic performance; and N–S fit predicts greater proportions of variance in well-being (e.g., academic satisfaction and depression). Practical and theoretical implications of these results are discussed.

The Relationship Between Level of Training and Accuracy of Violence Risk Assessment

Although clinical training programs aspire to develop competency in violence risk assessment, little research has examined whether level of training is associated with the accuracy of clinicians' evaluations of violence potential. This is the first study to compare the accuracy of risk assessments by experienced psychiatrists with those performed by psychiatric residents. It also examined the potential of a structured decision support tool to improve residents' risk assessments. The study used a retrospective case-control design. Medical records were reviewed for 151 patients who assaulted staff at a county hospital and 150 comparison patients. At admission, violence risk assessments had been completed by psychiatric residents (N=38) for 52 patients and by attending psychiatrists (N=41) for 249 patients. Trained research clinicians, who were blind to whether patients later became violent, coded information available at hospital admission by using a structured risk assessment tool-the Historical, Clinical, Risk Management-20 clinical subscale (HCR-20-C). Receiver operating characteristic analyses showed that clinical estimates of violence risk by attending psychiatrists had significantly higher predictive validity than those of psychiatric residents. Risk assessments by attending psychiatrists were moderately accurate (area under the curve [AUC]=.70), whereas assessments by residents were no better than chance (AUC=.52). Incremental validity analyses showed that addition of information from the HCR-20-C had the potential to improve the accuracy of risk assessments by residents to a level (AUC=.67) close to that of attending psychiatrists. Having less training and experience was associated with inaccurate violence risk assessment. Structured methods hold promise for improving training in risk assessment for violence.

Examination of the structural, convergent, and incremental validity of the Reynolds Intellectual Assessment Scales (RIAS) with a clinical sample.

Empirical examination of the Reynolds Intellectual Assessment Scales (RIAS; C. R. Reynolds & R. W. Kamphaus, 2003a) has produced mixed results regarding its internal structure and convergent validity. Various aspects of validity of RIAS scores with a sample (N = 521) of adolescents and adults seeking psychological evaluations at a university-based clinic were examined. Results from exploratory factor analysis indicated only 1 factor, and confirmatory factor analysis (CFA) indicated that the 1-factor model was a good fit and a better fit than the 2-factor model. Hierarchical factor analysis indicated the higher order, general intelligence factor accounted for the largest amount of variance. Correlations with other measures of verbal/crystallized and nonverbal/fluid intelligence were supportive of the convergent validity of the Verbal Intelligence Index but not the Nonverbal Intelligence Index. Joint CFA with these additional measures resulted in a superior fit of the 2-factor model compared with the 1-factor model, although the Odd-Item-Out subtest was found to be a poor measure of nonverbal/fluid intelligence. Incremental validity analyses indicated that the Composite Intelligence Index explained a medium to large portion of academic achievement variance; the NIX and VIX explained a small amount of remaining variance. Implications regarding interpretation of the RIAS when assessing similar individuals are discussed.

Personality constellations in incarcerated psychopathic men.

Advances in the operationalization of psychopathy have led to an increased understanding of the boundaries, structure, and nomological network of this construct, although significant questions remain. The empirical identification of replicable and theoretically meaningful psychopathy subtypes may help to improve the classification and diagnosis of this condition. We conducted a classification study of 91 incarcerated men who met conventional criteria for high levels of psychopathy using the Psychopathy Checklist-Revised. We expanded on the methodology of previous research on psychopathy subtypes by utilizing a comprehensive personality assessment instrument and a prototype matching approach to classification. The analyses revealed a primary (narcissistic) subtype and a secondary (hostile and dysregulated) subtype that were broadly consistent with the previous literature. External validation analyses, statistical controls, and incremental validity analyses provided substantial support for the primary and secondary subtypes.

Predicting recidivism with the Psychopathy Checklist: Are factor score composites really necessary?

In two previous studies on general and violent recidivism (Walters & Heilbrun, 2010; Walters, Knight, Grann, & Dahle, 2008), the summed composite antisocial facet of the Psychopathy Checklist displayed incremental validity relative to the other 3 facets (interpersonal, affective, lifestyle), whereas the other 3 facets generally failed to demonstrate incremental validity relative to the antisocial facet. Because summed composite scores do not account for ordinal item distributions, the 6 Walters et al. (2008) samples were reanalyzed with factor score composites derived from a 4-factor confirmatory factor analysis. The results, however, showed little change from what had been obtained earlier with summed composite scores. Two additional samples not previously included in any incremental validity analyses of the Psychopathy Checklist evidenced a 3-factor structure, with the lifestyle and antisocial facets merged into a single factor. This single factor displayed incremental validity relative to the interpersonal and affective facets, but the reverse was not true regardless of whether summed composite scores or factor score composites were used. A comparison of zero-order correlations from all 8 samples revealed that the antisocial summed composite score predicted significantly better than the summed composite scores for the other 3 facets and that a superordinate factor failed to improve on the performance of either the antisocial summed composite score or the antisocial factor score composite.

Criminal recidivism among juvenile offenders: Testing the incremental and predictive validity of three measures of psychopathic features.

We studied the predictive, comparative, and incremental validity of three measures of psychopathic features (Psychopathy Checklist: Youth Version [PCL:YV]; Antisocial Process Screening Device [APSD]; Childhood Psychopathy Scale [CPS]) vis-à-vis criminal recidivism among 83 delinquent youth within a truly prospective design. Bivariate and multivariate analyses (Cox proportional hazard analyses) showed that of the three measures, the CPS was most consistently related to most types of recidivism in comparison to the other measures. However, incremental validity analyses demonstrated that all of the predictive effects for the measures of psychopathic features disappeared after conceptually relevant covariates (i.e., substance use, conduct disorder, young age, past property crime) were included in multivariate predictive models. Implications for the limits of these measures in applied juvenile justice assessment are discussed.

Incremental Validity Analyses of the Violence Risk Appraisal Guide and the Psychopathy Checklist: Screening Version in a Civil Psychiatric Sample

This study compares two instruments frequently used to assess risk for violence, the Violence Risk Appraisal Guide (VRAG) and the Psychopathy Checklist: Screening Version (PCL:SV), in a large sample of civil psychiatric patients. Despite a strong bivariate relationship with community violence, the VRAG could not improve on the predictive validity of the PCL:SV alone, even though the VRAG includes several ostensible violence risk factors other than psychopathy. Moreover, incremental validity analyses indicated that the remaining VRAG items accounted for little or no variance in violent outcomes once psychopathy scores were controlled. Conversely, the PCL:SV continued to account for considerable variance after controlling for the VRAG. These results reflect the limited validity of the VRAG items in civil psychiatric samples beyond the variance that is explained by the PCL:SV alone.

Risk-Appraisal Versus Self-Report in the Prediction of Criminal Justice Outcomes

Twenty-seven individual pairs of effect sizes from 22 prospective studies employing one or more of the following five risk-appraisal procedures: Historical-Clinical-Risk Scales (HCR–20), Lifestyle Criminality Screening Form (LCSF), Level of Service-Inventory (LSI), Psychopathy Checklist (PCL), Violence Risk Appraisal Guide (VRAG), and one or more self-report measures were subjected to meta-analysis. Although risk-appraisal procedures displayed an advantage over self-report measures in recidivism prediction, the two methods produced comparable results when the meta-analysis was restricted to investigations using content-relevant self-report predictors. Incremental validity analysis of 72 risk-appraisal/self-report contrasts revealed that bothsets of measures accounted for criminal justice outcomes beyond the variance attributable to the alternate method.

The Role of Anxiety on Novices' Rifle Marksmanship Performance

In this study we examined the role of anxiety on novices' performance on the psychomotor task of rifle marksmanship. The stages-of-skill-development model suggests that aptitude influences performance while learning a skill; however, the model does not address the role of affect. Assessments for state anxiety and state worry were administered to 42 novice shooters. Aptitude and state worry predicted shooting performance (r = .67) comparable to more direct psychomotor measures used in prior research. Incremental validity analyses showed that aptitude accounted for 11% of the variance, and state worry explained an additional 34% above and beyond aptitude. The use of noncognitive assessments can broaden the understanding of skilled performance. Although aptitude influences learning a skill, it appears that novice shooters' performance is also influenced by affective states such as anxiety.

An Instrument for Assessing Coping with Temptation: Psychometric Properties of the Alcohol Abuse Coping Response Inventory

In the current study, we examined psychometric properties of The Alcohol Abuse Coping Response Inventory (AACRI), a modification of the Coping with Temptation Inventory for recovering alcoholics. It demonstrated good internal reliability. Three internally consistent factors were identified, one behavioral factor, and two cognitive factors. The measure did not demonstrate adequate concurrent validity. As expected, cognitive coping responses were more predictive of later relapse than behavioral coping responses, and a combination of the two was most predictive of later relapse. In the incremental validity analysis, situational confidence was found to be most predictive of relapse outcome, while the AACRI cognitive coping subscale demonstrated adequate incremental validity. The AACRI was found to have incremental validity over the Coping Behaviours Inventory.

Two methods for studying the incremental validity of a Rorschach variable.

The MMPI can be scored by a clerk. Also, both the number and form level of Rorschach responses can be easily assessed. Other Rorschach variables should be examined for their incremental validity beyond number of responses and form level, or from these variables plus simple MMPI variables. This study applied multiple regression analyses to 2 data sets with reasonable criteria of pathology that were predicted by W. Perry and D. J. Viglione's (1991) Ego Impairment Index considered alone. The index had only slight incremental validity over and above the number of responses and form quality, and even less when the average MMPI elevation and L. R. Goldberg's (1965) formula for predicting psychosis versus neurosis were entered before these Rorschach variables. Another way of assessing incremental validity is through the use of unit weights, that is, adding standardized variables weighted equally rather than optimally. The unit-weighted incremental validity analysis resulted in the same conclusions.