Performance Validity Measures Research Articles

A brief mindfulness-based intervention for stress, pain, emotion and attention regulation in military service members with mild traumatic brain injury.

The primary aim of this study was to conduct an open pilot clinical trial of a brief mindfulness-based intervention for persistent postconcussion symptoms that occur after mild traumatic brain injury in military service members. For many service members, operational tempo and other time constraints may prevent them from completing a standard mindfulness-based stress reduction course. Thus, this study sought to examine the effectiveness of a five-session intervention called mindfulness-based stress, pain, emotion, and attention regulation (MSPEAR). Participants were active duty service members with a history of mild traumatic brain injury (TBI) and persisting postconcussion symptoms, all of whom were recruited from an outpatient TBI rehabilitation program at a military treatment facility. Of the 38 service members that were initially enrolled, 25 completed the 5-session MSPEAR intervention, and 20 returned for a 5-week follow-up evaluation. Questionnaires assessing perceived stress, positive affect, pain interference and catastrophizing, sleep disturbances, perceived behavioral and attention regulation, self-efficacy and satisfaction with life were administered at preintervention, postintervention, and at 5-week follow-up intervals. Neuropsychological testing at preintervention and 5-week follow-up included performance validity measures, attention, working memory, and executive function measures. T-tests were run to compare for questionnaire measures at preintervention (Time 1) to postintervention (Time 2). Repeated analysis of variances were conducted to compare questionnaire and neuropsychological measures at Time 1, Time 2, and at Time 3 which is the 5-week follow-up. Improvements in perceived stress, positive affect, behavioral regulation, metacognition, sleep disturbance, self-efficacy, and satisfaction with life were found immediately after the MSPEAR intervention and were maintained at the 5-week follow-up. Magnification and helplessness aspects of pain catastrophizing improved when comparing preintervention to the 5-week follow-up. Pain interference was not significantly different across study assessment times. Neuropsychological testing revealed improvements in sustained attention, working memory, cognitive flexibility, and inhibitory control when comparing preintervention to the 5-week follow-up assessment. The MSPEAR intervention appears to show promise as a brief and effective therapy for specific postconcussion symptoms after mild traumatic brain injury in military service members. Each of the components of MSPEAR including stress, pain catastrophizing, emotion and attention regulation showed improvements in this study, and bears further investigation in a larger scale, preferably randomized controlled trial in those active duty military service members who experience persisting symptoms after a mild traumatic brain injury.

Ability of the Wisconsin Card-Sorting Test-64 as an embedded measure to identify noncredible neurocognitive performance in mild traumatic brain injury litigants

Objective To investigate the ability of selective measures on the Wisconsin Card Sorting Test-64 (WCST-64) to predict noncredible neurocognitive dysfunction in a large sample of mild traumatic brain injury (mTBI) litigants. Method Participants included 114 adults who underwent a comprehensive neuropsychological examination. Criterion groups were formed based upon their performance on stand-alone measures of cognitive performance validity (PVT). Results Participants failing PVTs performed worse across all WCST-64 dependent variables of interest compared to participants who passed PVTs. Receiver operating curve analysis revealed that only categories completed was a significant predictors of PVT status. Multivariate logistic regression did not add to classification accuracy. Conclusion Consideration of noncredible executive functioning may be warranted in mild traumatic brain injury (mTBI) litigants who complete ≤ 1 category on the WCST-64.

The use of positive deviance approach to improve health service delivery and quality of care: a scoping review

BackgroundQuality has been a persistent challenge in the healthcare system, particularly in resource-limited settings. As a result, the utilization of innovative approaches is required to help countries in their efforts to enhance the quality of healthcare. The positive deviance (PD) approach is an innovative approach that can be utilized to improve healthcare quality. The approach assumes that solutions to problems are already available within the community and identifying and sharing those solutions can help others to resolve existing issues. Therefore, this scoping review aimed to synthesize the evidence regarding the use of the PD approach in healthcare system service delivery and quality improvement programs.MethodsArticles were retrieved from six international databases. The last date for article search was June 02, 2023, and no date restriction was applied. All articles were assessed for inclusion through a title and/or abstract read. Then, articles that passed the title and abstract review were screened by reading their full texts. In case of duplication, only the full-text published articles were retained. A descriptive mapping and evidence synthesis was done to present data with the guide of the Preferred Reporting Items for Systematic Reviews and Meta-analysis extension for Scoping Reviews checklist and the results are presented in text, table, and figure formats.ResultsA total of 125 articles were included in this scoping review. More than half, 66 (52.8%), of the articles were from the United States, 11(8.8%) from multinational studies, 10 (8%) from Canada, 8 (6.4%) from the United Kingdom and the remaining, 30 (24%) are from other nations around the world. The scoping review indicates that several types of study designs can be applied in utilizing the PD approach for healthcare service and quality improvement programs. However, although validated performance measures are utilized to identify positive deviants (PDs) in many of the articles, some of the selection criteria utilized by authors lack clarity and are subject to potential bias. In addition, several limitations have been mentioned in the articles including issues in operationalizing PD, focus on leaders and senior managers and limited staff involvement, bias, lack of comparison, limited setting, and issues in generalizability/transferability of results from prospects perspective. Nevertheless, the limitations identified are potentially manageable and can be contextually resolved depending on the nature of the study. Furthermore, PD has been successfully employed in healthcare service and quality improvement programs including in increasing surgical care quality, hand hygiene practice, and reducing healthcare-associated infections.ConclusionThe scoping review findings have indicated that healthcare systems have been able to enhance quality, reduce errors, and improve patient outcomes by identifying lessons from those who exhibit exceptional practices and implementing successful strategies in their practice. All the outcomes of PD-based research, however, are dependent on the first step of identifying true PDs. Hence, it is critical that PDs are identified using objective and validated measures of performance as failure to identify true PDs can subsequently lead to failure in identifying best practices for learning and dissemination to other contextually similar settings.

Measures of performance validity for Hispanic/Latin American adults: A PRISMA systematic review.

Measures of performance validity for Hispanic/Latin American adults: A PRISMA systematic review.

Detection and Assessment of Point-to-Point Movements During Functional Activities Using Deep Learning and Kinematic Analyses of the Stroke-Affected Wrist.

Stoke is a leading cause of long-term disability, including upper-limb hemiparesis. Frequent, unobtrusive assessment of naturalistic motor performance could enable clinicians to better assess rehabilitation effectiveness and monitor patients' recovery trajectories. We therefore propose and validate a two-phase data analytic pipeline to estimate upper-limb impairment based on the naturalistic performance of activities of daily living (ADLs). Eighteen stroke survivors were equipped with an inertial sensor on the stroke-affected wrist and performed up to four ADLs in a naturalistic manner. Continuous inertial time series were segmented into sliding windows, and a machine-learned model identified windows containing instances of point-to-point (P2P) movements. Using kinematic features extracted from the detected windows, a subsequent model was used to estimate upper-limb motor impairment, as measured by the Fugl-Meyer Assessment (FMA). Both models were evaluated using leave-one-subject-out cross-validation. The P2P movement detection model had an area under the precision-recall curve of 0.72. FMA estimates had a normalized root mean square error of 18.8% with R2=0.72. These promising results support the potential to develop seamless, ecologically valid measures of real-world motor performance.

Supplemental Material for Measures of Performance Validity for Hispanic/Latin American Adults: A PRISMA Systematic Review

Supplemental Material for Measures of Performance Validity for Hispanic/Latin American Adults: A PRISMA Systematic Review

Measuring Situational Cognitive Performance in the Wild: A Psychometric Evaluation of Three Brief Smartphone-Based Test Procedures

Mobile devices provide new opportunities to draw conclusions about cognitive performance in everyday situations. To gain insights into cognitive performance patterns in healthy adult populations, we adapted three established cognitive tests for smartphone use: the Digit Symbol Substitution Task (DSST), Sustained Attention to Response Task (SART), and Psychomotor Vigilance Task (PVT). To increase their feasibility for ambulatory assessment, we identified the minimum measurement durations that provide reliable and valid state measures of cognitive performance. Over 2 weeks, 46 participants performed each test once per day at random times, along with self-reports (e.g., on concentration, mood, and mental demands). The validity and reliability of change are promising for the 30-second PVT and 90-second DSST and SART. The DSST and SART provide fruitful outcomes for ambulatory field studies linked to mood, stress, and mental demands. We provide digital versions of the adapted DSST and SART online for free.

Development of Radiographer Performance Assessment Instruments in the Radiology Installation of Dr. H. Abdul Moeloek Lampung Province

At Abdul Moeloek Regional General Hospital in Lampung Province, evaluating radiographers' performance is crucial for ensuring quality radiology services. The current assessment tools for ongoing professional practice and work behavior lack clear, objective indicators and targets. This research uses research and development (R&D) research methods with a quasi-experimental posttest-only non-equivalent control group design. Data were collected using observation, interviews, and FGD. Expert validation and design improvements are conducted, followed by a trial involving 20 radiographers. Validity is tested using corrected items to total correlation, and reliability is assessed using Cronbach's alpha. Next, expert validation tests and design improvements were carried out by 3 experts, and then a limited trial of 20 RSUDAM radiographers carried out a validity test using corrected items to total correlation. Meanwhile, the reliability test uses Cronbach's alpha. The developed instrument, based on indicators and targets, demonstrates valid and reliable measures of radiographer performance. Expert validation through FGD confirms the instrument's effectiveness. Validity test results indicate significance (> 0.05), and reliability tests show Cronbach's alpha values (> 0.60). Normality tests reveal non-distributed data (p-value < 0.05). The Wilcoxon test, comparing old and new instruments, yields a significant Asymp.sig value (0.000) for the discipline variable, indicating improved performance assessment.

81 An Examination of a Multiple Cutoff Performance Validity Index

Objective:Recent consensus guidelines have advocated for the use of multivariate performance validity assessment on ability-based measures such those used in neuropsychological assessment. Further, previous research has demonstrated that aggregating performance validity indicators may produce superior classification accuracy. The present study builds upon this research by aggregating data from three of the most commonly used performance validity measures (Test of Memory Malingering [TOMM], Rey Fifteen Item Test with recognition trial [FIT plus recognition], and Reliable Digit Span [RDS]) to create a performance validity composite measure in a veteran mild traumatic brain injury (mTBI) population.Participants and Methods:Data of patients evaluated at a VA hospital who had completed the RDS, FIT plus recognition, and TOMM as part of their clinical neuropsychological evaluation were analyzed (n = 20). Two composite performance validity indexes were created: a Single Cutoff Performance Validity Index (SC-PVI), which measures the quantity of failures across performance validity measures (PVMs) by summing the total number of PVM failures, and a Multiple Cutoff Performance Validity Index (MC-PVI) which measures the number of failures as well as degree of failure(s) across measures of performance validity (e.g., a participant would attain a score of 3 if their PVM performance failed to reach a conservative cut point; they would obtain a score of 1 if they met conservative cut point, yet failed to reach a liberal cut point).Results:Only one participant (5%) attained a score of 0 on the SC-PVI (i.e., passing all PVTs using standard cutoffs) and MC-PVI (i.e., passing the most liberal cut points on all three PVMs). Conversely, eight participants (40%) attained a score of 3 on the SC-PVI (i.e., failed all three PVMs) and four participants (20%) attained a score of 9 (i.e., failed the most conservative cut points on all three PVMs). Results showed a significant (p < .001) ordinal association between the two indices (G = .984); however, there was no significant agreement between SC-PVI and MC-PVI models (k = -.087; p = .127).Conclusions:Data revealed discordant findings between the three PVMs utilized. The majority of participants (75%) scored between 2-8 on the MC-PVI, meaning that they did not exceed all liberal cut points or fail all conservative cut points. These “grey area” scores suggest an indeterminate range of performance validity, which cannot be captured by a solitary cut point or neatly classified as pass or fail. The utility of multiple cutoff performance validity models (i.e., aggregating PVMs to consider the severity of failure and number of failures) is that they capture the nuance of these data when determining and discussing the credibility of a profile. Multiple cut point data also highlight how the choice of cutoff influences the outcome of performance validity research and clinical decision making. As such, future research on the classification accuracy of this MC-PVI is needed.

78 Utility of the D-KEFS Color Word Interference Test as a Measure of Performance Validity in Adults Referred for a Psychoeducational Evaluation

Objective:Previous investigations have demonstrated the clinical utility of the Delis-Kaplan Executive Function System (D-KEFS) Color Word Interference Test (CWIT) as an embedded validity indicator in mixed clinical samples and traumatic brain injury. The present study sought to cross-validate previously identified indicators and cutoffs in a sample of adults referred for psychoeducational testing.Participants and Methods:Archival data from 267 students and community members self-referred for a psychoeducational evaluation at a university clinic in the South were analyzed. Referrals included assessment for attention-deficit hyperactivity disorder, specific learning disorder, autism spectrum disorder, or other disorders (e.g., anxiety, depression). Individuals were administered subtests of the D-KEFS including the CWIT and several standalone and embedded performance validity indicators as part of the evaluation. Criterion measures included The b Test, Victoria Symptom Validity Test, Medical Symptom Validity Test, Dot Counting Test, and Reliable Digit Span. Individuals who failed 0 criterion measures were included in the credible group (n = 164) and individuals failing 2 or more criterion measures were included in the non-credible group (n = 31). Because a subset of the sample were seeking external incentives (e.g., accommodations), individuals who failed only 1 of the criterion measures were excluded (n = 72). Indicators of interest included all test conditions examined separately, the inverted Stroop index (i.e., better performance on the interference trial than the word reading or color naming trials), inhibition and inhibition/switching composite, and sum of all conditions.Results:Receiver Operating Characteristics (ROC) curves were significant for all four conditions (p < .001) and the inverted stroop index (p = .032). However, only conditions 2, 3 and 4 met minimal acceptable classification accuracy (AUC = .72 - 81). ROC curves with composite indicators were also significant (p < .001), with all three composite indicators meeting minimal acceptable classification accuracy (AUC = .71- .80). At the previously identified cutoff of age corrected scale score of 6 for all four conditions, specificity was high (.88 -.91), with varying sensitivity (.23 - .45). At the previously identified cutoff of .75 for the inverted stroop index, specificity was high (.87) while sensitivity was low (.19). Composite indicators yielded high specificity (.88 - .99) at previously established cutoffs with sensitivity varying from low to moderate (.19 - .48). Increasing the cutoffs (i.e., requiring higher age corrected scale score to pass) for composite indicators increased sensitivity while still maintaining high specificity. For example, increasing the total score cutoff from 18 to 28 resulted in moderate sensitivity (.26 vs .52) with specificity of .91.Conclusions:While a cutoff of 6 resulted in high specificity for most conditions, the sum of all four conditions exhibited the strongest classification accuracy and appears to be the most robust indicator which is consistent with previous research (Eglit et al., 2019). However, a cutoff of 28 as opposed to 18 may be most appropriate for psychoeducational samples. Overall, the results suggest that the D-KEFS CWIT can function as a measure of performance validity in addition to a measure of processing speed/executive functioning.

64 Neuroimaging Evidence of Neurodegenerative Disease in Former Professional American Football Players Who “Fail” Validity Testing: A Case Series

Objective:Former professional American football players have a high relative risk for neurodegenerative diseases like chronic traumatic encephalopathy (CTE). Interpreting low cognitive test scores in this population occasionally is complicated by performance on validity testing. Neuroimaging biomarkers may help inform whether a neurodegenerative disease is present in these situations. We report three cases of retired professional American football players who completed comprehensive neuropsychological testing, but “failed” performance validity tests, and underwent multimodal neuroimaging (structural MRI, Aß-PET, and tau-PET).Participants and Methods:Three cases were identified from the Focused Neuroimaging for the Neurodegenerative Disease Chronic Traumatic Encephalopathy (FIND-CTE) study, an ongoing multimodal imaging study of retired National Football League players with complaints of progressive cognitive decline conducted at Boston University and the UCSF Memory and Aging Center. Participants were relatively young (age range 55-65), had 16 or more years of education, and two identified as Black/African American. Raw neuropsychological test scores were converted to demographically-adjusted z-scores. Testing included standalone (Test of Memory Malingering; TOMM) and embedded (reliable digit span, RDS) performance validity measures. Validity cutoffs were TOMM Trial 2 < 45 and RDS < 7. Structural MRIs were interpreted by trained neurologists. Aß-PET with Florbetapir was used to quantify cortical Aß deposition as global Centiloids (0 = mean cortical signal for a young, cognitively normal, Aß negative individual in their 20s, 100 = mean cortical signal for a patient with mild-to-moderate Alzheimer’s disease dementia). Tau-PET was performed with MK-6240 and first quantified as standardized uptake value ratio (SUVR) map. The SUVR map was then converted to a w-score map representing signal intensity relative to a sample of demographically-matched healthy controls.Results:All performed in the average range on a word reading-based estimate of premorbid intellect. Contribution of Alzheimer’s disease pathology was ruled out in each case based on Centiloids quantifications < 0. All cases scored below cutoff on TOMM Trial 2 (Case #1=43, Case #2=42, Case #3=19) and Case #3 also scored well below RDS cutoff (2). Each case had multiple cognitive scores below expectations (z < -2.0) most consistently in memory, executive function, processing speed domains. For Case #1, MRI revealed mild atrophy in dorsal fronto-parietal and medial temporal lobe (MTL) regions and mild periventricular white matter disease. Tau-PET showed MTL tau burden modestly elevated relative to controls (regional w-score=0.59, 72nd%ile). For Case #2, MRI revealed cortical atrophy, mild hippocampal atrophy, and a microhemorrhage, with no evidence of meaningful tau-PET signal. For Case #3, MRI showed cortical atrophy and severe white matter disease, and tau-PET revealed significantly elevated MTL tau burden relative to controls (w-score=1.90, 97th%ile) as well as focal high signal in the dorsal frontal lobe (overall frontal region w-score=0.64, 74th%ile).Conclusions:Low scores on performance validity tests complicate the interpretation of the severity of cognitive deficits, but do not negate the presence of true cognitive impairment or an underlying neurodegenerative disease. In the rapidly developing era of biomarkers, neuroimaging tools can supplement neuropsychological testing to help inform whether cognitive or behavioral changes are related to a neurodegenerative disease.

29 Examining the Relationship between Symptom and Performance Validity Measures Across Referral Subtypes

Objective:Performance validity (PVT) and symptom validity tests (SVT) have become standard practice in assessing credibility of neuropsychological profiles and symptom report. While PVTs assess cognitive task engagement, SVTs assess credibility of patient symptom report. Although prior research aimed to conceptualize the relationship between the two validity measure types, it generally focused on SVTs from the Minnesota Multiphasic Personality Inventory (MMPI-2 &RF) and the Structured Inventory of Malingered Symptoms (SIMS; Ord et al., 2021, MMPI-2; Van Dyke et al., 2013). Further studies have demonstrated mixed results, with many studies concluding that symptom and performance validity are separate but related constructs. The current study aimed to assess the relationship between PVTs and SVTs utilizing symptom validity measures from the Personality Assessment Inventory (PAI) across three samples, including neurodevelopmental, psychiatric, and traumatic brain injury groups.Participants and Methods:Participants included 634 individuals consecutively referred for neuropsychological assessment who completed the Test of Memory Malingering (TOMM) and the PAI (mean Age = 41.7, SD = 15.7; mean Education = 13.7, SD = 2.7; 53% female; 89% Caucasian). Participants were divided into three groups based on referral, including neurodevelopmental (mean Age = SD = 10.7; mean Education = 13.4, SD = 2.5; 39% female; 79% Caucasian), psychiatric (mean Age = 44.7, SD = 15.0; mean Education = 13.8, SD = 2.8; 58% female; 90% Caucasian), and traumatic brain injury samples (mean Age = SD = 15.5; mean Education = 13.3, SD = 2.3; 50% female; 91% Caucasian). Four structural equation models (latent variable models) were constructed. The first model was fit across the entire sample while the remaining three were fit for the aforementioned subsamples. TOMM trials modeled the performance validity latent variable while SVTs from the PAI modeled the symptom validity latent variable (Positive Impression Management and Defensiveness Index modeled underreporting; Negative Impression Management, Malingering Index, and Cognitive Bias Scale modeled overreporting).Results:In the full sample model overreporting significantly predicted performance validity (p < 0.001, r = -0.31), indicating higher symptom overreporting related to poorer performance validity while symptom underreporting did not significantly predict performance validity (p = 0.09, r = 0.08). In the neurodevelopmental model overreporting did not significantly predict performance validity (p = 0.44, r = 0.10). Further, symptom underreporting did not significantly predict performance validity (p = 0.40, r = 0.10). Similarly, for the TBI model, overreporting did not significantly predict performance validity (p = 0.82, r = -0.02) and symptom underreporting did not significantly predict performance validity (p = 0.50, r = -0.08). For the psychiatric sample symptom underreporting did not significantly predict performance validity (p = 0.06, r = 0.11); however, symptom overreporting significantly predicted performance validity (p < 0.001, r = - 0.39).Conclusions:The current study expands on prior research comparing the relationship between SVTs and PVTs in neuropsychological evaluation utilizing SVTs from the PAI. Results of the present study suggest the relationship between the SVTs and PVTs varies by referral type and further supports using both PVTs and SVTs in neuropsychological assessment.

84 Using a Combination Score from the Full Rey Complex Figure Test (RCFT) as a Performance Validity Measure (PVT)

Objective:Some RCFT indices are effective Performance Validity Test (PVTs) during neuropsychological evaluations. A combination score that includes the copy score, true positive recognition, and atypical errors has proven to be especially useful (see Lu et al, 2003). However, this score was derived from administration that deviated from protocols outlined by Meyers & Meyers (1995) in that the Recognition trial was administered after the 3-minute delay instead of the 30-minute delay. The current study examined the utility of the RCFT combination score as a performance validity test (PVT) when completing the recognition trial after the 30-minute delay.Participants and Methods:This study utilized archival data from 298 Veterans who presented for a clinical neuropsychological evaluation at a southern Veterans Affairs Medical Center. The evaluation included up to nine PVTs and all trials of the RCFT (per Meyers & Meyers, 1995). Patients were considered credible if all PVT performance fell within normal limits. This resulted in 232 patients in the credible group (Mage = 52.9 years, SDage = 15.2, Medu = SDedu = 2.5, 88% male, 71.2% White, 28.3% Black/African American). Patients were considered non-credible if they failed >2 PVTs. This resulted in 66 patients in the non-credible group (Mage = 51.6, SDage = 13.79, Medu = SDedu = 2.4, 92.4% male, 56.1% White, 43.9% Black/African American). Group assignment was also clinically confirmed. Receiver operating characteristic (ROC) curve analyses were conducted to discriminate between credible and non-credible groups utilizing the established RCFT combination score.Results:RCFT combination scores distinguished groups, with credible participants scoring higher than non-credible participants (F[1, 296]=63.76, p<.001, d=1.11; M = 56.9, SD = 9.3 vs. M = 46.5, SD = 9.5, respectively). A ROC analysis indicated AUC = .800 (95% CI = .73 to .86). When specificity was set at >90%, a cut-score of <46.5 yielded sensitivity at 46.0%. The analogous cut-score from the Lu et al. (2003) study (i.e., <47) was associated with a specificity of 88.7 and sensitivity of 46.0% in the current study.Conclusions:As the Lu et al. (2003) established the combination score of the RCFT with procedures that deviated from the standardized protocol outlined by Meyers and Meyers (1995), clinicians who opted to adhere to Meyers and Meyers’ full protocol may have concerns about using the combination score as a PVT. The current study established a similar cut-off score to what Lu et al., (2003) reported (i.e., <46.5 vs. <47) while following a different administration procedure of the RCFT. Also, the index was moderately sensitive in the current study (i.e., 45.5%) but less so than what Lu et al. reported when using a cut-score that had >90% specificity (i.e., 75.9% sensitivity). This suggests that the index may be robust to deviations in administration procedures. Difference in sensitivity could be related to difference between samples. As the current sample was derived from a clinical, VA setting, current findings extend the generalizability of the index. Future research would benefit exploring if any subgroups would benefit from adjusted cut-scores to reduce the risk of false positive identification.

42 Associations Between Mild Traumatic Brain Injury, Executive Function, and Criminal Justice Involvement among Veterans and Service Members: a LIMBIC-CENC study

Objective:To examine relationships between history of mild traumatic brain injury (mTBI), neuropsychological measures of executive function, and lifetime history of criminal justice (CJ) involvement among combat-exposed Veterans and Service Members (V/SM).Participants and Methods:Participants were combat-exposed V/SM who completed a baseline assessment for the multicenter Long-term Impact of Military-Relevant Brain Injury Consortium - Chronic Effects of Neurotrauma Consortium study (N=1,341) and had adequate engagement/symptom reporting on measures of performance and symptom validity (i.e., Medical Symptom Validity Test and Mild Brain Injury Atypical Symptoms Scale). Neuropsychological battery included the Trail Making Test (A and B), Wechsler Adult Intelligence Scale-IV (WAIS-IV) Digit Span subtest, and the National Institute of Health (NIH) Toolbox Flanker subtest. Lifetime history of brain injury, criminal justice involvement, and demographics were collected. Participants were 87% male, 72% white, with a mean age of 40 years (SD=9.67). Eighty-one percent had at least some college education. Nineteen percent were active duty. Eighty percent of Veterans and 86% of Service Members reported a history of >1 mTBI, and of these 31% and 47% respectively experienced 3+ mTBIs.Results:Three groups were composed based on level of involvement with the CJ system: 1.) No history of arrests or incarcerations (3+ mTBIs: 64%), 2.) A lifetime history of arrest but no felony incarceration (3+ mTBIs: 34%), and 3.) A lifetime history of felony incarceration (3+ mTBIs: 2%). Ordinal regression analyses revealed that performance on a working memory task (Digit Span; b= 0.024, p= .041; OR= 1.024) was significantly associated with increased CJ involvement after adjusting for age, education, service status, and mTBIs. Performance on measures of processing speed (Trails A), set-shifting (Trails B), and inhibition (Flanker) were not significantly associated with CJ involvement. Number of mTBIs was significantly and positively associated with level of CJ involvement in all four models; Digit Span (p= .016), Trails A (p= .016), Trails B (p= .020), and Flanker (p= .008).Conclusions:Performance on most measures of executive functioning was not significantly associated with CJ involvement in this large, representative sample of V/SM who served in combat. Although performance on a working memory task was significantly associated with CJ involvement, the size of the effect was small and the association was in the opposite direction as expected. Number of mTBIs was significantly associated with level of CJ involvement, indicating that sustaining multiple mTBI may be linked to greater risk of CJ involvement. These findings suggest that social and psychological factors beyond executive dysfunction may better explain the relationship between history mTBIs and CJ involvement. Some aspects of military service and veteran status, such as interdisciplinary treatment for brain injury and physical, mental, and psychosocial health needs, may be protective against previously identified risk factors for arrest (e.g., deficits in executive functioning). Contextualizing mTBI within the larger behavioral health profile of V/SM, with emphasis placed on intervention for related co-morbidities, may reduce the impact of previous arrest on wellbeing and/or reduce the risk of future CJ involvement.

26 Improvement in Executive Function Following Goal-Oriented Attentional Self-Regulation Training in Veterans with mTBI

Objective:Goal-Oriented Attentional Self-Regulation (GOALS) is a cognitive rehabilitation training that targets difficulties with executive control functions. Prior research has found that GOALS training improves executive function, functional outcomes, and PTSD symptoms in Veterans with comorbid PTSD and TBI. The objective of this study is to assess the effects of GOALS training versus an active time-matched Behavioral Health Education (BHE) intervention in military Veterans with history of mTBI and executive function difficulties.Participants and Methods:Thirty-four veterans with a history of mild TBI (>6 months post-injury) and cognitive difficulties were randomized to GOALS or BHE group intervention. GOALS is a cognitive rehabilitation intervention that incorporates mindfulness-based attention regulation training, and goal management strategies applied to participant-defined goals. It consists of ten 2-hour sessions administered in small group format, and three individual 1-hour sessions. The BHE group is a matched control condition that mirrors GOALS in terms of duration and level of involvement. Participants completed a comprehensive evaluation consisting of neuropsychological and ecologically valid functional performance measures of complex attention and executive function, before and after GOALS or BHE intervention. Overall neuropsychological attention/executive functioning included measures of working memory (Letter Number Sequencing, Auditory Consonant Trigrams), mental flexibility (Design & Verbal Fluency Switching, Trails B, Stroop Inhibition/Switching), inhibition (Stroop Inhibition), and sustained attention (Digit Vigilance). The Goal Processing Scale (GPS) was used to measure ecologically valid complex functional performance.Results:Post GOALS training, but not post BHE, participants significantly improved on measures of attention /executive function including overall performance (p=.02), working memory (p=.04), sustained attention (p=.03)., and inhibition (p=.01). On measures of complex functional task performance, post GOALS participants significantly improved in the domain of planning (p=.02).Conclusions:GOALS training improved performance on neuropsychological and ecologically valid functional measures of complex attention and executive function in Veterans with mild TBI and cognitive difficulties. The current study highlights how mindfulness-based attention regulation and goalmanagement strategies applied to personal goals has positive effects across various outcomes. A percentage of individuals with mTBI continue to report persistent problems with cognition and daily functioning. Cognitive rehabilitation training such as GOALS may be beneficial for those with persistent difficulties following mTBI, even in chronic phase.

5 The impact of recreational cannabis use on neuropsychological function in epilepsy

Objective:Cannabis is classified as a class B drug in the UK with penalties for possession of up to 5 years in prison, an unlimited fine or both. Nevertheless it is widely available and is the most commonly used drug in the UK with approximately 2.6 million (7.6%) of adults reporting that they sometimes or regularly use it. It is not uncommon for people who present in our epilepsy clinic to report regular use of cannabis; some use it recreationally whilst others report ‘self-medicating’ based on the belief that it has a beneficial impact on their seizures. The aim of this study was to establish the prevalence of cannabis use in people with epilepsy referred for a neuropsychological assessment and to examine the impact of cannabis use on cognitive function in this group.Participants and Methods:All patients who attend for a neuropsychological assessment are routinely asked about illegal drug use in their clinical interview. This information is also captured in the medical and neuropsychiatric assessments they undergo when assessed by the multidisciplinary team. The electronic medical records of 800 consecutive patients who had undergone a neuropsychological assessment between 2019 and 2022 were searched for references to cannabis use. The neuropsychological profiles of patients reporting cannabis use were compared to those seen in the larger series across multiple cognitive domains.Results:Seventy (8.75%) of the patients in the series reported past or present cannabis use. Cannabis users were more likely to be male (p<0.01) and were younger (p<0.01) than those who did not report use. Reading IQ was significantly lower in the cannabis group (p<0.001). Patients who were regularly using cannabis at the time of the neuropsychological assessment did not differ from the rest of the cohort on tests of processing speed, working memory, naming or verbal fluency. There were no differences between the groups in their performance on an embedded measure of performance validity. However the patients who were regularly using cannabis at the time of their neuropsychological assessments scored significantly lower on tests of verbal learning (p<0.05) and reported significantly greater subjective memory problems in everyday life (p=0.02) than the non-cannabis group. The group using cannabis also scored significantly more highly on the depression (p<0.01) and anxiety scales (p=0.02) on the Hospital Anxiety and Depression Scale.Conclusions:The prevalence and patterns of cannabis use in the epilepsy population mirror those seen in the wider population. The impact on regular cannabis use on neuropsychological function appears to be most evident on measures of new learning and subjective measures of memory disturbance. Cannabis use is significantly associated with lower levels of cognitive reserve and elevated levels of anxiety and low mood. Whilst caution must be employed with respect to any direct attribution in these complex clinical presentations, these findings may be helpful in the interpretation of neuropsychological test scores and the planning of interventions, particularly with respect to subjective memory complaints in this group.

85 Use of Embedded Performance Validity Measures Using Verbal Fluency Tests in a Clinical Sample of Military Veterans

Objective:As neuropsychologists aim to collect valid data, maximize the utility of assessments, make effective use of time, and best serve patient populations, measurement of performance validity is considered a critical issue for the field. As effort may vary across an evaluation, including performance validity tests (PVTs) throughout the assessment is important. Incorporating embedded PVTs in addition to free standing PVTs can be particularly useful in this regard. COWAT and animal naming are commonly administered verbal fluency measures. While there have been past investigations into their potential for detecting invalid performance, they are limited, and more research is needed. Perhaps most promising, Sugarman and Axelrod (2015) described a logistic regression derived formula utilizing the combined raw scores of COWAT and animal naming. The current study aimed to investigate the use of embedded PVTs within COWAT and animal naming to provide further support for the use of embedded PVTs in these measures.Participants and Methods:All subjects were from a mixed clinical sample comprising military veterans from two VA Medical Centers in the northeast U.S., who were referred for neuropsychological evaluation. Subjects deemed credible had zero PVT failures. Subjects were considered non-credible performers if they failed at least two out of a possible eight PVTs administered. Subjects who failed one PVT were excluded from the study (n = 53). The final sample consisted of 116 individuals with credible performance (Mean Age = 35.5, SD = 8.8; Mean Edu = 13.6, SD = 2; Mean Est. IQ = 106, SD = 7.9) and 94 individuals with psychometrically determined non-credible performance (Mean Age = 38.5, SD = 9.4; Mean Edu = 113, SD = 2.1; Mean Est. IQ = 101, SD = 8.7). Performance of COWAT and animals in detecting non-credible performances was evaluated through calculation of classification accuracy statistics and use of the logistic regression formulas reported in Sugarman and Axelrod (2015).Results:For COWAT, the optimal cutoff was a raw score of <27 (specificity = 89%; sensitivity = 31%), and a T-score of <35 (specificity = 92%; sensitivity = 31%). For animal naming, optimal cutoffs were <16 for raw score (specificity = 92%, sensitivity = 38%) and <37 for T-score (specificity = 91%; sensitivity = 33%). The logistic regression formula based on raw scores for both COWAT and animal naming was inadequately sensitive at the recommended cutoff in this sample, but a coefficient of > .28 was revealed to be optimal (91% specificity; 42% sensitivity). When the formula for T-scores was used, a coefficient of > .38 was optimal (91% specificity; 28% sensitivity).Conclusions:Results of the current research suggest that PVTs embedded within the commonly administered COWAT and animal naming verbal fluency tests can effectively detect low effort, in concordance with generally accepted standards. A logistic regression formula using raw scores in particular appears to be most effective, consistent with findings reported by Sugarman and Axelrod (2015).

21 A Comparison of the Memory and Non-Memory Based Performance Validity Measures for Detecting Invalid Neuropsychological Test Performance among Individuals with and without Memory Impairment

Objective:Few to no studies have directly compared the relative classification accuracies of the memory-based (Brief Visuospatial Memory Test-Revised Recognition Discrimination [BVMT-R RD] and Rey Auditory Verbal Learning Test Forced Choice [RAVLT FC]) and non-memory based (Reliable Digit Span [RDS] and Stroop Color and Word Test Word Reading trial [SCWT WR]) embedded performance validity tests (PVTs). This study’s main objective was to evaluate their relative classification accuracies head-to-head, as well as examine how their psychometric properties may vary among subgroups with and without genuine memory impairment.Participants and Methods:This cross-sectional study included 293 adult patients who were administered the BVMT-R, WAIS-IV Digit Span, RAVLT and SCWT during outpatient neuropsychological evaluation at a Midwestern academic medical center. The overall sample was 58.0% female, 36.2% non-Hispanic White, 41.3% non-Hispanic Black, 15.7% Hispanic, 4.8% Asian/Pacific Islander, and 2.0% other, with a mean age of 45.7 (SD=15.8) and a mean education of 13.9 years (SD=2.8). Three patients had missing data, resulting in a final sample size of 290. Two hundred thirty-three patients (80%) were classified as having valid neurocognitive performance and 57 (20%) as having invalid neurocognitive performance based on performance across four independent, criterion PVTs (i.e., Test of Malingering Memory Trial 1, Word Choice Test, Dot Counting Test, Medical Symptom Validity Test). Of those with valid neurocognitive performance, 76 (48%) patients were considered as having genuine memory impairment through a memory composite band score (T<37 for (RAVLT Delayed Recall T-score + BVMT-R Delay Recall T-score/2).Results:The average memory composite band score for valid neurocognitive scores was T = 49.63 as compared to T = 27.57 for genuine memory impairment individuals. Receiver operating characteristic [ROC] curve analyses yielded significant areas under the curve (AUCs=.79-.87) for all four validity indices (p’s < .001). When maintaining acceptable specificity (91%-95%), all validity indices demonstrated acceptable yet varied sensitivities (35%-65%). Among the subgroup with genuine memory impairment, ROC curve analyses yielded significantly lower AUCs (.64-.69) for three validity indices (p’s < .001), except RDS (AUC=.644). At acceptable specificity (88%-93%), they yielded significantly lower sensitivities across indices (19%-39%). In the current sample, RAVLT FC and BVMT-R RD had the largest changes in sensitivities, with 19% and 26% sensitivity/90%-92% specificity at optimal cut-scores of <10 and <2, respectively, for individuals with memory impairment, compared to 65% and 61% sensitivity/94% specificity at optimal cut-scores of <13 and <4, respectively, for those without memory impairment.Conclusions:Of the four validity scales, memory-based embedded PVTs yielded higher sensitivities while maintaining acceptable specificity compared to non-memory based embedded PVTs. However, they were also susceptible to the greatest declines in sensitivity among the subgroup with genuine memory impairment. As a result, careful consideration should be given to using memory-based embedded PVTs among individuals with clinically significant memory impairment based on other sources of information (e.g., clinical history, behavioral observation).

89 The Neuropsychology of Complex Homelessness

Objective:Although the cognitive profiles of people experiencing homelessness have been described in the literature, the neuropsychological profile of people experiencing complex homelessness has not been delineated. Complex homelessness is homelessness that continues despite the provision of bricks and mortar solutions. People experiencing complex homelessness often have an array of physical health, mental health, substance use, neurodevelopmental and neurocognitive disorders. The present study aimed to delineate the neuropsychological profile of people experiencing complex homelessness and explore the utility of neuropsychological assessment in supporting this population.Participants and Methods:19 people experiencing complex homelessness in Sydney, Australia, were consecutively referred by specialist homelessness services for neuropsychological assessment. They underwent comprehensive assessment of intelligence, memory and executive functioning and completed questionnaires to screen for the presence of ADHD, PTSD, depression, anxiety and stress. A range of performance validity measures were included. Referrers were asked to complete questionnaires on history of childhood trauma, psychological functioning, drug and alcohol use, functional cognitive abilities, homelessness factors, personality, risk of cognitive impairment and adaptive functioning and to note existing or suspected mental health, neurodevelopmental and neurocognitive disorders. Referrers also completed a post-assessment pathways questionnaires to identify whether the neuropsychological assessment facilitated referral pathways (e.g., for government housing or financial assistance). Clinicians completed a post-assessment diagnosis survey, which was compared to the pre-assessment known or suspected diagnoses. Finally, referrers were asked to complete a satisfaction questionnaire regarding the neuropsychological assessment.Results:Mean (SD) WAIS-IV indexes were VCI = 81.1 (14.5), PRI = 86.1 (10.9), WMI = 80.5 (13.0), PSI = 81.6 (10.2). Mean WMS-IV Flexible (LMVR) indexes were AMI = 68.3 (19.6), VMI = 77.1 (19.3), IMI = 72.7 (17.2), and DMI = 70.5 (17.6). The majority of participants showed unusual differences between WAIS-IV and TOPF-predicted WAIS-IV scores and between WAIS-IV General Ability and WMS-IV Flexible (LMVR) scores. Demographically corrected scores on tests of executive functioning were mostly one or more standard deviations below the mean. The majority of participants screened positive on screening measures of executive dysfunction, PTSD and ADHD and had elevated self-reported psychological distress scores. At least one new diagnosis was made for nine (47%) participants, established diagnoses were confirmed for two (11%) participants, diagnoses were supported for 15 (79%) participants, tentative diagnoses were made for 16 (84%) participants, and five (26%) participants had at least one diagnosis disconfirmed/unsupported. Referrers indicated that the majority of post-assessment pathways were more accessible following the neuropsychological assessment and that they were very satisfied with the neuropsychological assessments overall.Conclusions:This is one of the first studies to delineate the neuropsychological profile of people experiencing complex homelessness using robust psychometric approaches, including performance validity tests. This population experiences a high burden of cognitive impairment and associated substance use, neurodevelopmental and mental health comorbidities. Neuropsychological assessment makes referral pathways more accessible and is valued by referrers of people experiencing complex homelessness.

27 Green’s Word Memory Test (WMT) Immediate Recall as a Screening Tool for Performance Invalidity

Objective:Assessment of performance validity during neuropsychology evaluation is essential to reliably interpret cognitive test scores. Studies (Webber et al., 2018; Wisdom, et al., 2012) have validated the use of abbreviated measures, such as Trial 1 (T1) of the Test of Memory Malingering (TOMM), to detect invalid performance. Only one study (Bauer et al., 2007) known to these authors has examined the utility of Green’s Word Memory Test (WMT) immediate recall (IR) as a screening tool for invalid performance. This study explores WMT IR as an independent indicator of performance validity in a mild TBI (mTBI) veteran population.Participants and Methods:Participants included 211 (Mage = 32.1, SD = 7.4; Medu = 13.1, SD = 1.64; 94.8% male; 67.8% White) OEF/OIF/OND veterans with a history of mTBI who participated in a comprehensive neuropsychological evaluation at one of five participating VA Medical Centers. Performance validity was assessed using validated cut scores from the following measures: WMT IR and delayed recall (DR); TOMM T1; WAIS-IV reliable digit span; CVLT-II forced choice raw score; Wisconsin Card Sorting Test failure to maintain set; and the Rey Memory for Fifteen Items test, combo score. Sensitivity and specificity were calculated for each IR score compared with failure on DR. In addition, sensitivity and specificity were calculated for each WMT IR score compared to failure of at least one additional performance validity measure (excluding DR), two or more measures, and three or more measures, respectively.Results:Results indicated that 46.8% participants failed to meet cut offs for adequate performance validity based on the standard WMT IR cut score (i.e., 82.5%; M = 81.8%, SD = 17.7%); however, 50.2% participants failed to meet criteria based on the standard WMT DR cut score (M = 79.8% SD = 18.6%). A cut score of 82.5% or below on WMT IR correctly identified 82.4% (i.e., sensitivity) of subjects who performed below cut score on DR, with a specificity of 94.2%. Examination of IR cutoffs compared to failure of one or more other PVTs revealed that the standardized cut score of 82.5% or below had a sensitivity of 78.2% and a specificity of 72.4%; whereas, a cut score of 65% or below had a sensitivity of 41% and a specificity of 91.3%. Similarly, examination of IR cutoffs compared to failure of two or more additional PVTs revealed that the cut score of 60% or below had a sensitivity of 45.7% and specificity of 93.1% ; whereas, a cut score of 57.5% or below had a sensitivity of 57.9% and specificity of 90.9% when using failure of three or more PVTs as the criterion.Conclusions:Results indicated that a cut score of 82.5% or below on WMT IR may be sufficient to detect invalid performance when considering WMT DR as criterion. Furthermore, WMT IR alone, with adjustments to cut scores, appears to be a reasonable way to assess symptom validity compared to other PVTs. Sensitivity and specificity of WMT IR scores may have been adversely impacted by lower sensitivity of other PVTs to independently identify invalid performance.