Incremental Predictive Validity Research Articles

Predictive Validity of Tools for Assessing Recidivism Risk in Men Convicted of Sex Offending: Static-99R, Static-2002R and BARR-2002R.

Recidivism risk assessment is crucial for effective case management of men convicted of sex offending. The use of empirical actuarial risk tools has become routine in the field. However, the development of actuarial risk scales for assessing general, violent and/or sexual recidivism in these men is ongoing: The Static-99 led to the Static-99R and the Static-2002R, and the BARR-2002R emerged to assess violent recidivism risk. A study was undertaken to evaluate and compare the inter-rater and predictive validity of the Static-99R, the Static-2002R, and the BARR-2002R in a sample of 328 men convicted of sex offending released from prison in French Belgium. When the instruments were considered integrally, the two versions of the Static-the Static-99R and the Static-2002R-proved better at predicting sexual recidivism and the BARR-2002R was better at predicting violent recidivism. And, the predictive and incremental predictive validity of the factor structure identified by Brouillette-Alarie et al. (2016) was examined. Results proved consistent in that the Youthful stranger aggression and General criminality factors were better at predicting general recidivism and violent non-sexual recidivism while the Persistence/paraphilia factor was better at predicting sexual recidivism.

Comparing the Incremental Predictive Validity of Self-as-Context-Related Measures.

The comparative incremental validity of five self-as-context measures in predicting psychological distress and satisfaction with life, after controlling for relevant demographic variables and other psychological flexibility processes, was evaluated in a college student sample (N = 315). All of the measures except the self-as-context subscale of the Multidimensional Psychological Flexibility Inventory (Rolffs et al., 2018) separately accounted for a significant increase in variability in psychological distress. The centering subscale of the Self-as-Context Scale (Zettle et al., 2018) was the only measure to also display incremental predictive validity in accounting for significant variance in life satisfaction. The conceptual and clinical implications of the findings in the context of study limitations are discussed.

Identifying vulnerable mother-infant dyads: a psychometric evaluation of two observational coding systems using varying interaction periods.

Clinical services require feasible assessments of parent-infant interaction in order to identify dyads requiring parenting intervention. We assessed the reliability and predictive validity of two observational tools and tested whether briefer forms could be identified which retain acceptable psychometric properties over short observation periods. A stratified high-risk community sample of 250 mother-infant dyads from The Wirral Child Health and Development Study completed 7-min play-based interaction at 6-8 months. Film-footage was independently coded by two trained raters using PIIOS and NICHD-SECCYD systems. Incremental predictive validity was assessed from 3, 5 and 7 min observation to attachment outcomes (Strange Situation; 14 months) and infant mental health (BITSEA; 14 and 30 months). Excellent inter-rater reliability was evident at code and subscale level for each tool and observation period. Stability of within-rater agreement was optimal after 5 min observation. ROC analysis confirmed predictive (discriminant) validity (AUCs >0.70) to top decile age 2 mental health outcomes for PIIOS total score and a brief 3-item composite from NICHD-SECCYD (sensitivity, intrusiveness, positive regard; NICHD-3), but not to attachment outcomes. Logistic regression showed dyads rated at-risk for externalizing problems using NICHD-3 were also at significantly higher risk for insecurity at 14 months (OR = 2.7, p = 0.004). PIIOS total and NICHD-3 ratings from 5 min observation are both reliable and valid tools for use in clinical practice. Findings suggest NICHD-3 may have greater utility due to its comparative brevity to train and code, with suitability for use over a broader developmental time frame (3-24 months).

Assessing Implicit Theories in Sexual Offending Using Indirect Measures: Feasibility, Reliability, and Incremental Validity.

This study assessed psychometric qualities of indirect measures assessing Implicit Theories (ITs) of sexual offending: Implicit Association Task (IAT), Implicit Relational Assessment Procedure (IRAP), and Relational Responding Task (RRT). For comparison reasons, aggressive behavior was also assessed. In a male sample from the general population (N = 109), we assessed each measure's (a) feasibility (mean latency, error rate, passing criteria), (b) internal consistency, (c) convergent and discriminant validity, and (d) incremental and predictive validity. Results indicated that no indirect measure met all criteria. Although the IAT was reasonably feasible and reliable in measuring aggression, ITs could not be reliably assessed. The RRT was feasible and somewhat reliable in assessing ITs, whereas the IRAP showed limited feasibility, high task complexity, low reliability, and the presence of a method factor. No measure had incremental predictive validity over the use of self-report measures, although we note that the power to detect such associations was limited. As none of the indirect measures performed satisfactorily on the measured criteria, the use of these measures in clinical practice seems currently unwarranted to assess ITs.

Does Studying Latin in Secondary Education Predict Study Achievement in Academic Higher Education?

AbstractStudying Latin in secondary education is still widespread in Europe and believed to result in cognitive benefits, even beyond the linguistic domain. In this study we explored the relation between such study and later academic achievement in higher education (N = 1,898). First, we demonstrated that Latin students exhibit increased levels of study achievement in higher education, particularly in study programs other than those covering science, technology, engineering, and mathematics (STEM). Second, we explored where the instruction of Latin was a significant predictor in models of academic achievement, explaining incremental variance over 21 other cognitive, attitudinal, and demographic variables. Latin instruction was included as a variable in the prediction models in 42% of the programs (mainly in the non‐STEM ones), but the incremental predictive validity was substantial only in the linguistic programs. Our results highlight how the study of Latin can be a valuable predictor of academic achievement in other study fields.

Personality disorder coverage, prevalence, and convergence: do the DSM-5's two models of personality disorder identify the same patients?

Research on the Alternative DSM-5 Model for Personality Disorders (AMPD) in DSM-5's Section-III has demonstrated acceptable interrater reliability, a largely consistent latent structure, substantial correlations with theoretically and clinically relevant measures, and evidence for incremental concurrent and predictive validity after controlling for DSM-5's Section II categorical personality disorders (PDs). However, the AMPD is not yet widely used clinically. One clinician concern may be caseness - that the new model will diagnose a different set of PD patients from that with which they are familiar. The primary aim of this study is to determine whether this concern is valid, by testing how well the two models converge in terms of prevalence and coverage. Participants were 305 psychiatric outpatients and 302 community residents not currently in mental-health treatment who scored above threshold on the Iowa Personality Disorder Screen (Langbehn et al., ). Participants were administered a semi-structured interview for DSM-5 PD, which was scored for both Section II and III PDs. Convergence across the two PD models was variable for specific PDs, Good when specific PDs were aggregated, and Very Good for 'any PD.' Results provide strong evidence that the AMPD yields the same overall prevalence of PD as the current model and, further, identifies largely the same overall population. It also addresses well-known problems of the current model, is more consistent with the ICD-11 PD model, and provides more complete, individualized characterizations of persons with PD, thereby offering multiple reasons for its implementation in clinical settings.

Driving factors of individual differences in broad retrieval ability: Gr is more than the sum of its parts.

Broad retrieval ability (Gr) posits an essential factor of human cognitive abilities. Previous literature indicates Gr is best modeled as a higher-order factor model with lower-level factors such as ideational fluency (IF), word fluency (WF), expressional fluency (EF), or figural fluency (FF). However, the dimensionality of Gr is not well studied. Furthermore, it remains unclear whether specific retrieval affordances such as differing retrieval time periods (e.g., short vs. long) can be psychometrically separated from more general retrieval affordances. Such a distinction would imply differential associations between specific retrieval, general retrieval, and other cognitive abilities, which, in turn, depict a vital part of explanatory models of individual differences in Gr. To test these assumptions, we conducted a multivariate study (N = 331) and evaluated competing latent variable measurement models for a variety of Gr tests. We then regressed the best measurement model onto working memory capacity, secondary memory, mental speed, and crystallized intelligence in order to evaluate the distinctiveness of Gr. Our results suggest that no specific retrieval affordances with regard to time periods can be distinguished. A higher-order model, with a second-order Gr factor above three first-order factors (IF, WF, EF, and FF) fitted the data best, extending previous literature by increasing construct coverage through the implementation of FF. All covariates show incremental predictive validity, beyond their communality. Summarizing, our results endorse a perspective on Gr as a strong and discriminant factor of cognitive abilities that is not affected by time constraints, and show that Gr is more than a linear combination of its parts. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

An out-of-sample perspective on the assessment of incremental predictive validity

In a psychometric analysis of a new psychological test, we often assess the predictive validity of a new target test over and above a baseline test, known as the incremental predictive validity. Usually, the incremental predictive validity is evaluated using within-sample statistics. Recently, it was argued to use out-of-sample assessment to prevent overfitting and non-replicable findings. In this paper, we elaborate on how to assess incremental predictive validity out-of-sample. In such an approach, we estimate prediction rules in one sample, and evaluate incremental predictive validity in another sample. Using a simulation study, we investigate whether an out-of-sample assessment results in different findings than a within-sample evaluation, taking into account the reliability of the baseline and a target test, and other factors (i.e., sample size). Results show that there is a difference between the in-sample and out-of-sample assessment, especially in small samples. However, the reliability of the two tests has no influence on this difference. In addition, we explore the effects of ridge estimation, ordinary least squares, and SIMEX, three different methods for estimating a prediction rule, on incremental predictive validity. The results show that using SIMEX leads to a bad assessment of incremental predictive validity. Ordinary least squares and ridge estimation result in almost the same incremental predictive validity estimates with a little advantage for ridge regression. In an empirical application, we show how to assess incremental predictive validity in practice and we compare that to the usual assessment.

Longitudinal effects of basic psychological need support on the development of intrinsic motivation and perceived competence in physical education. A multilevel study.

Grounded in self-determination theory, this article deals with a multidimensional measurement of the support of the basic psychological needs and the individual and combined effects on the development of intrinsic motivation and perceived competence in physical education. In addition to the supportive teacher behaviors of autonomy support, competence support and relatedness support, peer relatedness support is examined as an additional factor. A total of 1,047 students from 72 classes from various German-speaking Swiss cantons took part in the study. The hypothesized four factorial structure was analyzed using multilevel confirmatory factor analyses. Longitudinal measurement invariance testing of intrinsic motivation and perceived competence indicates scalar measurement invariance. Multilevel regression analyses were specified to analysis the longitudinal effects on the development of intrinsic motivation and perceived competence, whereby both the effects of the individual factors as well as the adjusted effects under the inclusion of all predictors were examined. Results of multilevel confirmatory factor analysis indicate that the hypothesized four-factor model (e.g., CFI = 0.97; TLI = 0.96; RMSEA = 0.04; SRMR between = 0.11) is to be favored over alternative models at both levels. Regarding the prediction of the development of intrinsic motivation and perceived competence our study underlines the predictive value of basic psychological need support. The models that examine the effects of the individual predictors indicate that the effects are largely consistent with expectations at both the class and individual level. At class level, however, autonomy support appears to be no significant predictor for the development of intrinsic motivation (p < 0.10), but for perceived competence (p < 0.05). Peer relatedness support is a significant predictor for both outcome variables at both levels of analysis. Regarding the simultaneous integration of all predictors, only the effects of peer relatedness support remain significant for both outcome variables. The empirical support of the multidimensionality of the instrument is particularly interesting in the context of the common conceptualization of a unidimensional measurement of autonomy support or a composite factor of psychological need support, whereby only few studies have adequately tested the factorial validity. Although significant effects of supportive teacher behaviors can be demonstrated in the multilevel regression analyses, it is also indicated that the different dimensions lack of incremental predictive validity. Particularly noteworthy is the rarely investigated role of peer relatedness support, which has been shown to be a meaningful predictor, even when supportive teacher behaviors are taken into account.

MOTOR PATHWAYS TO DEMENTIA

Abstract Emerging evidence indicates that non-cognitive manifestations such as gait impairments occur early during dementia. The Motoric Cognitive Risk (MCR) is a pre-dementia syndrome characterized by slow gait and cognitive complaints. MCR is a simple and easily accessible clinical dementia risk assay that does not require cognitive testing. MCR has incremental predictive validity for Alzheimer’s disease and related dementias over its individual cognitive and motoric components. Recent studies from across the world have helped to elucidate the epidemiology, biology, and brain substrates of MCR; paving the way for future interventions to prevent progression to dementia.

Using digital phenotyping to capture depression symptom variability: detecting naturalistic variability in depression symptoms across one year using passively collected wearable movement and sleep data

Major Depressive Disorder (MDD) presents considerable challenges to diagnosis and management due to symptom variability across time. Only recent work has highlighted the clinical implications for interrogating depression symptom variability. Thus, the present work investigates how sociodemographic, comorbidity, movement, and sleep data is associated with long-term depression symptom variability. Participant information included (N = 939) baseline sociodemographic and comorbidity data, longitudinal, passively collected wearable data, and Patient Health Questionnaire-9 (PHQ-9) scores collected over 12 months. An ensemble machine learning approach was used to detect long-term depression symptom variability via: (i) a domain-driven feature selection approach and (ii) an exhaustive feature-inclusion approach. SHapley Additive exPlanations (SHAP) were used to interrogate variable importance and directionality. The composite domain-driven and exhaustive inclusion models were both capable of moderately detecting long-term depression symptom variability (r = 0.33 and r = 0.39, respectively). Our results indicate the incremental predictive validity of sociodemographic, comorbidity, and passively collected wearable movement and sleep data in detecting long-term depression symptom variability.

Finding better raters: The role of observer personality on the validity of observer-reported personality in predicting job performance

Drawing on Funder’s Realistic Accuracy Model, we investigate key sources of variance in the criterion-related validity of observer-reported personality in predicting target job performance. Specifically, this study builds theory on how observer personality impacts (a) access to and (b) processing of personality cues, influencing the extent to which observer-ratings of target personality predict supervisor ratings of target job performance. Results from a multisource field study (n = 301) indicate that observer-ratings of personality have the potential to capture both higher relative and incremental predictive validity over self-ratings. However, the extent to which observer-ratings outperform self-ratings depends upon personality characteristics of the observers, such that predictive validities are highest when observers score high on measures of conscientiousness, openness, and emotional stability.

Assessing professionalism in mental health clinicians: development and validation of a situational judgement test.

Situational judgement test (SJT) scores have been observed to predict actual workplace performance. They are commonly used to assess non-academic attributes as part of selection into many healthcare roles. However, no validated SJT yet exists for recruiting into mental health services. To develop and validate an SJT that can evaluate procedural knowledge of professionalism in applicants to clinical roles in mental health services. SJT item content was generated through interviews and focus groups with 56 professionals, patients and carers related to a large National Health Service mental health trust in England. These subject matter experts informed the content of the final items for the SJT. The SJT was completed by 73 registered nurses and 36 allied health professionals (AHPs). The primary outcome measure was supervisor ratings of professionalism and effectiveness on a relative percentile rating scale and was present for 69 of the participating nurses and AHPs. Personality assessment scores were reported as a secondary outcome. SJT scores statistically significantly predicted ratings of professionalism (β = 0.31, P = 0.01) and effectiveness (β = 0.32, P = 0.01). The scores demonstrated statistically significant incremental predictive validity over the personality assessment scores for predicting supervisor ratings of professionalism (β = 0.26, P = 0.03). These findings demonstrate that a carefully designed SJT can validly assess important personal attributes in clinicians working in mental health services. Such assessments are likely to represent evidence based, cost-effective tools that can support values-based recruitment to mental health service roles.

The Classification of Federally Sentenced Women in Canada: Addition of Gender-Informed Variables to the Custody Rating Scale Contributes Incremental Predictive Validity

The Correctional Service of Canada (CSC) uses the Custody Rating Scale (CRS) for initial security classification; it is gender-neutral. Gender-informed scholars contend that gender-neutral assessments are problematic for use with justice-impacted women, as they exclude factors (e.g., victimization) deemed more relevant for women. Using an archival database with 1,555 federally sentenced women in Canada, we examined the extent that gender-informed indicators could yield incremental predictive validity (predicting institutional misconduct) beyond the CRS. Specifically, gender-informed variables from these domains were tested: mental health, substance misuse, relationship dysfunction, personal/emotional difficulties, parental/family issues, and victimization. Results revealed at least one gender-informed variable from each domain significantly predicted institutional misconducts. Composite gender-informed scales were created from the set of significant gender-informed predictors. Area under the curve (AUC) and hierarchical Cox regression analyses revealed the composite gender-informed scales contributed incremental predictive validity above and beyond the CRS. Although the CRS was predictive, it can be improved by including gender-informed variables.

Protective Factors in Forensic Practice: The Added Value of the SAPROF-Extended Version Pilot in Relation to Aggressive Incidents

Protective factors are now commonly included in comprehensive risk assessment. This study concerns an initial validation of the new SAPROF-Extended Version (SAPROF-EV) pilot, containing modifications and additions to the original SAPROF. For 139 forensic psychiatric inpatients, assessment results with the SAPROF-EV pilot and HCR-20V3 were compared with aggressive incidents. Results show good predictive validity for the SAPROF-EV pilot for all outcomes and incremental predictive validity for both the (modified) original SAPROF and the full SAPROF-EV pilot over the HCR-20V3. For the outcome aggression toward others, the additional SAPROF-EV factors provide incremental predictive validity over the (modified) original SAPROF. In addition, the user feedback from clinicians highlights experienced additional value of the new factors for treatment guidance. Based on the findings from this study, the SAPROF-EV pilot will be adjusted further into an improved and enhanced version of the SAPROF.

Incremental Predictive Validity of the Dualistic Model of Passion for Cannabis Use Among College Undergraduate Students With and Without a Cannabis Use Disorder.

We examined whether the Dualistic Model of Passion (DMP; i.e., obsessive passion [OP] and harmonious passion [HP]) for cannabis use was prospectively associated with cannabis use and use-related outcomes, and with academic performance, relationship attachment style, and social connectedness among college students. We also explored whether the DMP was associated with outcomes when included in a model using established constructs (e.g., coping motives, refusal self-efficacy, cannabis use disorder [CUD] symptoms) as predictors of cannabis use and outcomes. Using a longitudinal cohort design (baseline, 5-month, 10-month [timepoints chosen to better correspond to 9-month academic year]), 513 undergraduate students from two universities who reported using cannabis at least four times in the past month completed a baseline survey (308 meeting criteria for CUD). We used Generalized Estimating Equations to assess longitudinal associations between OP/HP and cannabis use and academic/social outcomes at 5-month and 10-month. At baseline, participants were young adults (Mean age = 20.57, SD = 2.51), 78.8% non-Hispanic, 83.8% White, 55.0% female, and 72.3% heterosexual. Greater HP was not associated with greater past month cannabis use or cannabis-related problems. Greater OP was associated with greater past month cannabis use and more cannabis-related problems. There were no significant passion by time interactions. Greater HP was associated with more anxious attachment. OP was associated with less social connection. This research suggests that the DMP provides novel information about factors associated with cannabis use and use-related consequences, which can aid in our understanding of cannabis use, misuse, and CUD among college students.

Free to be me with you: Development of the Individuality in Couples Questionnaire.

Despite multiple theories and treatment modalities emphasizing the importance of individuality in couple relationships, the field is lacking a reliable and valid measure of this construct. In the present study, we developed the Individuality in Couples (ICQ) questionnaire and demonstrated its strong psychometric properties across two samples of participants in committed intimate relationships (Sample 1 = 580 undergraduates; Sample 2 = 445 community members). The ICQ is comprised of 25 items that can be combined into a reliable total score to measure individuality in the context of couple relationships (i.e., the extent to which someone feels respected by their partner for their individuality and experiences personal autonomy in the relationship). Scores on the ICQ demonstrated high internal consistency, excellent construct replicability, convergent and divergent validity with measures of other relationship dimensions (i.e., intimacy, support, sexual satisfaction, psychological aggression, communication), criterion validity with measures of relationship satisfaction and partner health, and incremental predictive validity for explaining relationship satisfaction and partner well-being when controlling for other relationship dimensions. Results suggest that individuality in couples is largely a unidimensional construct that is distinct from more severe patterns of control and coercion characteristic of psychological aggression. The ICQ holds promise for identifying and promoting dynamics essential for healthy couple relationships. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Goal-Pursuit Tendencies and Retirement Planning: A Time-Lagged and Multi-Dimensional Investigation

Abstract Existing studies regarding retirement planning have suggested that demographic variables, retirement-related characteristics, and work-related factors are closely associated with retirement planning. However, the importance of individual goal-pursuit tendencies within a multi-dimensional framework of retirement planning has been overlooked. As retirement planning is conceptualized as a multi-faceted and goal-directed behavior, the absence of these concepts within the literature presents an important theoretical and empirical gap. Drawing upon the dual-process model of coping, the current study examined the incremental effects of tenacious goal pursuit (TGP) and flexible goal adjustment (FGA) on four dimensions of retirement planning (i.e., financial planning, health planning, interpersonal/leisure planning, and work planning). Data from 377 Chinese older workers at two time points showed that FGA rather than TGP was positively associated with financial, health, and interpersonal/leisure planning. Further, FGA demonstrated incremental predictive validity over TGP. This study contributes to the retirement planning literature by investigating how older adults’ goal-related dispositions may impact their retirement planning in different arenas. Practical implications of our findings, limitations, and future research directions are also discussed.

A Suicide-Specific Diagnosis – The Case For

A Suicide-Specific Diagnosis – The Case For

Understanding the role of emotion regulation strategies in cybervictimization and cyberaggression over time: It is basically your fault!

Adolescents involved in experiences of cybervictimization and cyberbullying are at increased risk of psychological maladjustment and ill-being. However, not all adolescents involved in cyberbullying roles experience similar consequences and cognitive emotion regulation (CER) might be a key factor. Despite growing interest in the role CER strategies play in cyberbullying behaviours, little is known about the predictive utility of these strategies in predicting cybervictimization and cyberbullying over time. Therefore, the aim of this prospective study was to test the incremental predictive validity of specific CER strategies in cybervictimization and cyberbullying in a sample of adolescents. To this end, data were collected in two waves four months apart from a sample of 841 adolescents (466 females) aged 12 to 18 (MageT1 = 13.77, SDT1 = 1.34; MageT2 = 13.71, SDT2 = 1.31). Participants filled out a set of questionnaires measuring cybervictimization, cyberbullying, and CER strategies. Main results showed blaming others as the unique strategy showing incremental predictive value to both cybervictimization and cyberbullying four months later, above and beyond previous cybervictimization and cyberbullying experiences. Overall, the results highlighted the need to account for CER strategies, such as blaming others, in relation to the potential initiation and development of cyberbullying and cybervictimization behaviours. These findings suggest possible future avenues for intervention efforts targeting the maintenance of online aggressive behaviours and victimization across time.