Ecological Validity Of Neuropsychological Tests Research Articles

Beyond cognitive dysfunction: Relevance of ecological validity of neuropsychological tests in multiple sclerosis

In neurological diseases such as multiple sclerosis (MS), a neuropsychological assessment is often requested to assist clinicians in evaluating the role of cognition in a patient’s level of everyday functioning. To be effective in this charge, it is assumed that performance on neuropsychological tests is related to how a person may function in everyday life, and the question is often asked: “Are neuropsychological tests ecologically valid?” In this review, we synthesize the literature that examines the use of neuropsychological tests to assess functioning across a variety of everyday functioning domains in MS (i.e. driving, employment, instrumental activities of daily living (IADLs)). However, we critically examine the usefulness of asking this broad question regarding ecological validity, given the psychometric and conceptual pitfalls it may yield. While many neuropsychological tests may be generally considered “ecologically valid” in MS, it is much more helpful to specify for whom, under what circumstances, and to what degree.

Self-ratings of Everyday Memory Problems in Patients with Acquired Brain Injury - A Tool for Rehabilitation

Introduction: Memory problems are common in everyday life of patients with acquired brain injury (ABI). Some patients with ABI also have problems with self-monitoring/awareness. The ecological validity of neuropsychological tests for everyday life memory problems is questionable. Can self-report instruments supply complementary information? Aims: 1) To document the frequency/impact of self-reported memory problems in a sample of consecutive referrals of ABI patients using PEEM and REEM. 2) To characterize the instruments with respect to psychometrics and internal consistency. 3) To document differences in memory problem patterns for various kinds/localization of brain lesions, and associated anxiety/ depression symptoms. Methods: A descriptive retrospective study of consecutive referrals of ABI patients was performed. Ratings from the Evaluation of Everyday Memory (EEM), in a patient version (PEEM) and a version for relatives/proxies (REEM) were analysed as well as self-ratings of anxiety and depression. Results: The EEM instruments displayed good psychometric characteristics. The mean PEEM score were close to the tenth percentile of healthy controls. PEEM and REEM versions were strongly inter-correlated. Sex, age, and lesion characteristics did not matter much with one exception. Right-hemisphere lesion patients rated their memory problems significantly lower than the proxy, for all other lesions it was vice versa. Anxiety and depression symptoms were associated with memory problems.

Development and Evaluation of an Ecological Task to Assess Executive Functioning Post Childhood TBI: The Children's Cooking Task

AbstractPurpose:Traumatic brain injury (TBI) often leads to executive functions deficits, which may be responsible for severe and longstanding disabilities in everyday activities. Sensitivity and ecological validity of neuropsychological tests of executive functions have been questioned. The aims of this study were to pilot an ecological open-ended assessment of executive functions in children, the ‘Children's Cooking Task’ (CCT), specifically to report its reliability, discriminant validity and concurrent validity.Methods:Twenty-five children with mild (n= 10) or moderate-to-severe TBI (n= 15), and 21 matched controls (aged 8 to 20 years) participated in the study. An open-ended cooking task was designed to test multi-tasking abilities. It required the preparation of two simple recipes using specific instructions. Outcome measures included the number of errors and an overall qualitative analysis of the task. Validating measures of executive functions included the Delis Kaplan Executive Function System, the Six-Part Test and two questionnaires completed by the child's primary care-giver: the Behavior Rating Inventory of Executive Function and the Dysexecutive Questionnaire for Children.Results:Internal consistency of the Children's Cooking Task was high (Cronbach's alpha = .86), as was test–retest reliability (ICC = .89). Children with moderate-to-severe TBI, as well as children with mild TBI made significantly more errors in the Children's Cooking Task in comparison to controls (p< .001). The CCT was correlated with several tests and one questionnaire of executive functioning (Trails, verbal fluency, sorting, 20 questions, Dysexecutive Questionnaire).Discussion and Conclusion:The Children's Cooking Task has good interrater and test–retest reliability, as well as good discriminant and concurrent validity.

Impairmentversusdeficiency in neuropsychological assessment: Implications for ecological validity

Neuropsychological test interpretation has relied on pre- and postmorbid comparisons, as exemplified by the use of demographically adjusted normative data. We argue that, when the assessment goal is to predict real-world functioning, this interpretive method should be supplemented by "absolute" scores. Such scores are derived from comparisons with the general healthy adult population (i.e., demographically unadjusted normative data) and reflect examinees' current ability, that is, the interaction between premorbid and injury/disease-related factors. In support of this view, we found that substantial discrepancies between demographically adjusted and absolute scores were common in a traumatic brain injury sample, especially in participants with certain demographic profiles. Absolute scores predicted selected measures of functional outcome better than demographically adjusted scores and also classified participants' functional status more accurately, to the extent that these scores diverged. In conclusion, the ecological validity of neuropsychological tests may be improved by the consideration of absolute scores. (JINS, 2009, 15, 94-102.).

Assessment of executive functioning in children after TBI with a naturalistic open-ended task: A pilot study

Purpose: Traumatic brain injury (TBI) often leads to executive functions (EF) deficits, resulting in severe longstanding disabilities in daily life activities. The sensitivity and ecological validity of neuropsychological tests have been questioned. The aim of this study was to pilot a novel open-ended naturalistic task and to compare it to other standardized assessments of EF in children post-TBI.Methods: Ten children aged 8–14, with moderate-to-severe TBI, and 18 matched controls participated in the study. The clinical group was assessed using cognitive tests and parent-based questionnaires of EF. An interactive ecological cooking task was designed.Results: Analyses indicated mild-to-moderate executive deficits in the cognitive tests in approximately half of the TBI group. For the experimental cooking task, all quantitative and qualitative variables were significantly impaired for the TBI group compared to the control group and failure in the cooking task was associated with lower scores in cognitive tests of EF. The task was able to discriminate the TBI children from the control group.Conclusion: This pilot study highlights the role of naturalistic assessments, to complement standardized tests in assessing patients’ dysexecutive impairments in complex activities of daily living post-childhood TBI.

The ecological validity of neuropsychological assessment and the role of depressive symptoms in moderate to severe traumatic brain injury

Evaluating the ecological validity of neuropsychological tests has become an increasingly important topic. Previous research suggests that neuropsychological tests have a moderate level of ecological validity when predicting everyday functioning. The presence of depressive symptoms, however, may impact the relationship between neuropsychological tests and real world performance. The current study empirically tests this hypothesis in a sample of 216 participants with moderate to severe traumatic brain injury (TBI) who completed neuropsychological testing, self-report of mood symptoms, and report of everyday functioning six months post-injury. Contrary to some previous research and clinical lore, results indicated that depression was weakly related to neuropsychological test performance, although it was more strongly related to everyday functioning. Neuropsychological test performance was also significantly related to everyday functioning. The ecological validity of the neuropsychological tests together was not impacted by depressive symptoms, when predicting significant other ratings of functional status. However, patient self-report seems somewhat less related to neuropsychological performance in those with significant depressive symptoms. Neuropsychological test performance was equally related to self and other report of everyday functioning in patients without significant depressive symptoms.

Fractionation of the dysexecutive syndrome in a heterogeneous neurological sample: Comparing the Dysexecutive Questionnaire and the Brock Adaptive Functioning Questionnaire

Primary objective: Research on the ecological validity of neuropsychological tests of executive functioning has been limited by disagreement in the number and make-up of the factors underlying the dysexecutive syndrome. The current study sought to determine whether similar components of everyday executive dysfunction are being measured by the Dysexecutive Questionnaire (DEX) and the Brock Adaptive Functioning Questionnaire (BAFQ).Research design: Using the same sample, a factor analysis of the DEX and the BAFQ was conducted to determine whether a similar underlying factor structure for the dysexecutive syndrome would be found across measures.Methods and procedures: The significant others of 46 adults with varying neurological conditions completed the DEX and BAFQ.Main outcomes and results: Behavioural inhibition, goal-directed behaviour/intentionality and executive memory/cognition were found to be key underlying factors of the everyday dysexecutive syndrome identified by both measures. The remaining factors were divergent.Conclusions: Consistent with prior research, this study indicates that the dysexecutive syndrome is multi-factorial. This research further highlights core factors associated with real world executive functioning deficits and demonstrates that the dysexecutive syndrome has distinct sub-components that transcend the specific questionnaire measure used.

The ecological validity of neuropsychological tests: a review of the literature on everyday cognitive skills.

Evaluating the ecological validity of neuropsychological tests has become an increasingly important topic over the past decade. In this paper, we provide a comprehensive review of the research on the ecological validity of neuropsychological tests, as it pertains to everyday cognitive skills. This review is presented in the context of several theoretical issues facing ecological validity research. Overall, the research suggests that many neuropsychological tests have a moderate level of ecological validity when predicting everyday cognitive functioning. The strongest relationships were noted when the outcome measure corresponded to the cognitive domain assessed by the neuropsychological tests. Several other factors that may moderate the degree of ecological validity established for neuropsychological tests are in need of further exploration. These factors include the effects of the population being tested, the approach utilized (verisimilitude vs. veridicality), the person completing the outcome measure (significant other vs. clinician), illness severity, and time from injury until evaluation. In addition, a standard measurement of outcome for each cognitive domain is greatly needed to allow for comparison across studies.

The relationship between neuropsychological performance and daily functioning in individuals with Alzheimer's disease: ecological validity of neuropsychological tests

The current study examined the relationship between neuropsychological test performance and functional status in 42 individuals diagnosed with Alzheimer's disease. A comprehensive battery of cognitive tests was employed in order to assess a wide range of neuropsychological abilities. Functional status was measured through the use of both a performance-based scale of activities of daily living (an expanded version of the Direct Assessment of Functional Status; DAFS, Loewenstein et al., 1989), and by a caregiver/informant-based rating scale (Instrumental Activities of Daily Living; IADL, Lawton & Brody, 1969). Findings suggest that neuropsychological functioning is moderately predictive of functional status. Using multiple regression analyses, neuropsychological variables accounted for 25% of the variance in the IADL and 50% of the variance in the DAFS. Individual domains of both functional measures were also significantly predicted by the neuropsychological variables. The findings provide evidence of a relationship between neuropsychological test performance and ADLs in an Alzheimer disease patient population.

The relationship between neuropsychological performance and daily functioning in individuals with Alzheimer’s disease: ecological validity of neuropsychological tests

The current study examined the relationship between neuropsychological test performance and functional status in 42 individuals diagnosed with Alzheimer’s disease. A comprehensive battery of cognitive tests was employed in order to assess a wide range of neuropsychological abilities. Functional status was measured through the use of both a performance-based scale of activities of daily living (an expanded version of the Direct Assessment of Functional Status; DAFS, Loewenstein et al., 1989), and by a caregiver/informant-based rating scale (Instrumental Activities of Daily Living; IADL, Lawton & Brody, 1969). Findings suggest that neuropsychological functioning is moderately predictive of functional status. Using multiple regression analyses, neuropsychological variables accounted for 25% of the variance in the IADL and 50% of the variance in the DAFS. Individual domains of both functional measures were also significantly predicted by the neuropsychological variables. The findings provide evidence of a relationship between neuropsychological test performance and ADLs in an Alzheimer disease patient population.

Ecological Validity of Neuropsychological Tests: Focus on the California Verbal Learning Test and the Wisconsin Card Sorting Test

To date, few studies have assessed the ecological validity of current measures of memory and executive functioning, particularly the California Verbal Learning Test (CVLT) and the Wisconsin Card Sorting Test (WCST). A battery of tests, including the CVLT and the WCST, was administered to 28 severely head-injured adults who were 1 year or more postinjury. Hierarchical linear regression was performed to assess the relationship between these cognitive tests and work functioning while statistically controlling for physical disability, emotional distress, and memory compensation strategies utilized by the participants. Results suggest that the CVLT was best at predicting performance on the job; however, the CVLT and the WCST were about equally predictive of the type of position held by a participant. Hence, this study demonstrates the ecological validity of measures of memory and executive functioning for predicting work-related skills.

Ecological Validity of Neuropsychological Tests: Focus on the California Verbal Learning Test and the Wisconsin Card Sorting Test

To date, few studies have assessed the ecological validity of current measures of memory and executive functioning, particularly the California Verbal Learning Test (CVLT) and the Wisconsin Card Sorting Test (WCST). A battery of tests, including the CVLT and the WCST, was administered to 28 severely head-injured adults who were 1 year or more postinjury. Hierarchical linear regression was performed to assess the relationship between these cognitive tests and work functioning while statistically controlling for physical disability, emotional distress, and memory compensation strategies utilized by the participants. Results suggest that the CVLT was best at predicting performance on the job; however, the CVLT and the WCST were about equally predictive of the type of position held by a participant. Hence, this study demonstrates the ecological validity of measures of memory and executive functioning for predicting work-related skills.