Predicted Poorer Performance Research Articles

The instinct fallacy: the metacognition of answering and revising during college exams

Students often gauge their performance before and after an exam, usually in the form of rough grade estimates or general feelings. Are these estimates accurate? Should they form the basis for decisions about study time, test-taking strategies, revisions, subject mastery, or even general competence? In two studies, undergraduates took a real multiple-choice exam, described their general beliefs and feelings, tracked their performance for each question, and noted any revisions or possible revisions. Beliefs formed after the exams were poor predictors of performance. In contrast, real-time metacognitive monitoring – measured by confidence ratings for each individual question – accurately predicted performance and were a much better decisional guide. Measuring metacognitive monitoring also allowed us to examine the process of revising an answer. Should a test-taker rely on their first choice or revise in the face of uncertainty? Experience seems to show that first instincts are correct. The decision-making literature calls this the first-instinct fallacy, based on extensive analysis of revisions, and recommends revising more. However, whereas revisions have been analyzed in great detail, previous studies did not analyze the efficacy of sticking with an original choice. We found that both revising and sticking resulted in significantly more correct than incorrect outcomes, with real-time metacognition predicting when each was most appropriate.

Persistence and change in community composition of reef corals through present, past, and future climates.

The reduction in coral cover on many contemporary tropical reefs suggests a different set of coral community assemblages will dominate future reefs. To evaluate the capacity of reef corals to persist over various time scales, we examined coral community dynamics in contemporary, fossil, and simulated future coral reef ecosystems. Based on studies between 1987 and 2012 at two locations in the Caribbean, and between 1981 and 2013 at five locations in the Indo-Pacific, we show that many coral genera declined in abundance, some showed no change in abundance, and a few coral genera increased in abundance. Whether the abundance of a genus declined, increased, or was conserved, was independent of coral family. An analysis of fossil-reef communities in the Caribbean revealed changes in numerical dominance and relative abundances of coral genera, and demonstrated that neither dominance nor taxon was associated with persistence. As coral family was a poor predictor of performance on contemporary reefs, a trait-based, dynamic, multi-patch model was developed to explore the phenotypic basis of ecological performance in a warmer future. Sensitivity analyses revealed that upon exposure to thermal stress, thermal tolerance, growth rate, and longevity were the most important predictors of coral persistence. Together, our results underscore the high variation in the rates and direction of change in coral abundances on contemporary and fossil reefs. Given this variation, it remains possible that coral reefs will be populated by a subset of the present coral fauna in a future that is warmer than the recent past.

Hot executive function in children with Developmental Coordination Disorder: Evidence for heightened sensitivity to immediate reward

Deficits of cool executive function (EF) have been shown in children with motor problems (or Developmental Coordination Disorder – DCD), but little is known of hot EF in this group. Given some evidence of poor self-regulation in DCD, we predicted poorer performance on a measure of hot EF, the Hungry Donkey Task (HDT), relative to typically developing (TD) children. Participants were 14 children with DCD and 22 TD children aged between 6.5 and 12 years. The DCD group performed significantly worse than the TD group on a 100-trial version of the HDT, making more selections from disadvantageous options and less from advantageous ones. Within-group analyses showed that children with DCD had faster responses to disadvantageous options than to advantageous. These results suggest high sensitivity to immediate reward in DCD. This sensitivity may reflect a more generalized deficit in the ability to resist the rewarding aspects of emotionally significant stimuli.

Performance in cognitive and problem-solving tasks in male spotted bowerbirds does not correlate with mating success

Individuals exhibiting a high level of cognitive ability may also exhibit more elaborate traits and so gain higher levels of mating success. This suggests that selection may act on cognitive performance through mate choice. Studies investigating this relationship have tended to focus on single cognitive tasks, or tasks that are closely related to existing natural behaviours, and individuals are frequently tested in captive conditions. This can introduce test artefacts and may tell us more about selection on specific display behaviours that we imagine being particularly cognitively complex, rather than a general cognitive ability. We tested free-living male spotted bowerbirds, Ptilonorhynchus maculatus, that exhibit elaborate sexual displays which appear to be cognitively demanding. We describe a method for testing individuals in the wild, without the need for constraint or captivity. We looked for evidence of a general cognitive ability in males by assaying their performance in a series of novel tasks reflecting their natural bower-building behaviour (bower maintenance) or capturing more abstract measures of cognitive ability (colour and shape discrimination, reversal learning, spatial memory and motor skills). We related performance in these tasks to their mating success. An individual's performance in one task was a relatively poor predictor of performance in any other task. However, an individual's performance across tasks could be summarized by a principal component which explained a level of total variance above which has previously been accepted as evidence of a general cognitive ability. We found no relationships between an individual's overall performance, or performance in any single task, and mating success. Our results highlight the need for further investigation of whether selection on cognition in bowerbirds is exerted through mate choice. We offer this as an example of how classic cognitive tasks can be transferred to the wild, thus overcoming some limitations of captive cognitive testing.

Evaluation of the Structural Determinants of Polymeric Precipitation Inhibitors Using Solvent Shift Methods and Principle Component Analysis

The presence of polymers within solid dose forms, such as solid dispersions, or liquid or semisolid formulations, such as lipid-based formulations, can promote the maintenance of drug supersaturation after dissolution or dispersion/digestion of the vehicle in the gastrointestinal tract. Transiently stable supersaturation delays precipitation, increases thermodynamic activity, and may enhance bioavailability and reduce variability in exposure. In the current study a diverse range of 42 different classes of polymers, with a total of 78 polymers across all classes, grades, and molecular weights were examined, to varying degrees, as potential polymeric precipitation inhibitors (PPIs) using a solvent shift method to initiate supersaturation. To provide a deeper understanding of the molecular determinants of polymer utility the data were also analyzed, along with a range of physicochemical descriptors of the polymers employed, using principle component analysis (PCA). Polymers were selectively tested for their ability to stabilize supersaturation for nine poorly water-soluble model drugs, representing a range of nonelectrolytes, weak acids, and weak bases. In general, the cellulose-based polymers (and in particular hydroxypropylmethyl cellulose, HPMC, and its derivatives) provided robust precipitation inhibition across most of the drugs tested. Subsequent PCA indicate that there is consistent PPI behavior of a given polymer for a given drug type, with clear clustering of the performance of polymers with each of the nonelectrolytes, weak bases, and weak acids. However, there are some exceptions to this, with some specific drug type-polymer interactions also occurring. Polymers containing primary amine functional groups should be avoided as they are prone to enhancing precipitation rates. An inverse relationship was also documented for the number of amide, carboxylic acid, and hydroxyl functional groups; therefore for general good PPI performance the number of these contained within the polymer should be minimized. Molecular weight is a poor predictor of performance, having only a minor influence, and in some cases a higher molecular weight enhances the precipitation process. The importance of ionic interactions to the ability of a PPI to stabilize the supersaturated state was demonstrated by the advantage of choosing a polymer with an opposite charge with respect to the drug. Additionally, when the polymer charge is the same as the supersaturated drug, precipitation is likely to be enhanced. A PCA model based on polymer molecular properties is presented, which has a central oval region where the polymer will general perform well across all three drug types. If the polymer is located outside of this region, then they either show compound-specific inhibition or enhance precipitation. Incomplete separation of the PPI performance based on the molecular properties on the polymers indicates that there are some further molecular properties that might improve the correlation.

Psychometric effects of adding realism to a speech-in-noise test

The speech reception threshold (SRT) is routinely measured in the laboratory to assess speech understanding in noise, but is often reported to be a poor predictor of performance in real world listening situations. The overall goal of this work is to determine whether introducing realistic aspects to speech tests can better capture individual differences and ultimately produce more relevant performance measures. We examined the psychometric effects of (a) transplanting a standard sentence-in-noise test into a simulated reverberant cafeteria environment, and (b) moving from sentence recall to a new ongoing speech comprehension task. Participants included normal hearers and hearing-impaired listeners (who were tested with and without their hearing aids). SRTs in the cafeteria environment were significantly correlated with standard SRTs, but were poorer overall and more sensitive to hearing loss. The comprehension task, despite having very different demands to sentence recall, produced similar SRTs under these conditions. The benefit of hearing aids was weakly correlated across the two listening environments and the two listening tasks. These manipulations promise to be useful for the creation of realistic laboratory tests that are engaging and challenging, yet controlled enough to be useful for psychophysical experiments.

The Influence of Effort on Neuropsychological Performance in Children: Is Performance on the TOMM Indicative of Neuropsychological Ability?

The Test of Memory Malingering (TOMM) is a measure of effort that has traditionally been utilized with adults but is being increasingly used with children, though it is not yet entirely clear what suboptimal TOMM performance means in terms of neuropsychological test scores. The purpose of this study was to investigate whether performance on the TOMM can be used as a predictive marker for neuropsychological performance in children. Participants (N = 75) completed the TOMM, Wechsler Intelligence Scale for Children-Fourth Edition, Wisconsin Card-Sorting Test (WCST), Children's Memory Test, and Delis-Kaplan Executive Function System Trails. Results revealed significant correlations between age, education level, IQ, and many of the neuropsychological measures administered, indicating that as children's TOMM scores increase, so do their cognitive performances. Children were subsequently divided into two groups: optimal effort and suboptimal effort, based on their TOMM Trial 2 scores. Results suggest significant differences in IQ performance and WCST Failure to Maintain Set; however, there were no differences in regard to any other neuropsychological measures administered. It was also found that a larger proportion of the younger children (aged 6 to 10 years old) scored below the TOMM cutoff compared with older children. This study illustrates that although correlations exist, suboptimal effort on the TOMM may not predict poorer performance on a neuropsychological evaluation in children as has been reported in other studies.

Predicting likely student performance in a first year Science, Technology, Society course

To reduce failure rate of first year students, predicting their likely performance would help targeted support. The performance of students in a STS course has been examined using a number of parameters. Student performance correlated with a student’s prior educational performance in secondary school and a student’s attendance at tutorials. In particular, students in the lower half of their secondary school cohort were more likely to fail. A link between lower tutorial attendance rates and failing the STS course is also noted, with lower attendance rates occurring from the first tutorial, so students who miss early tutorials should also receive early intervention. Student age and a student’s family income were poor predictors of student performance. A student’s mark in the course tended to increase as student age increased. The rank score given to students who enter university via an alternate pathway was a poor predictor of performance.

Assessing citizen science data quality: an invasive species case study

An increase in the number of citizen science programs has prompted an examination of their ability to provide data of sufficient quality. We tested the ability of volunteers relative to professionals in identifying invasive plant species, mapping their distributions, and estimating their abundance within plots. We generally found that volunteers perform almost as well as professionals in some areas, but that we should be cautious about data quality in both groups. We analyzed predictors of volunteer success (age, education, experience, science literacy, attitudes) in training-related skills, but these proved to be poor predictors of performance and could not be used as effective eligibility criteria. However, volunteer success with species identification increased with their self-identified comfort level. Based on our case study results, we offer lessons learned and their application to other programs and provide recommendations for future research in this area.

Are self-reported symptoms of executive dysfunction associated with objective executive function performance following mild to moderate traumatic brain injury?

Background and objective: We examined the relationship between self-reported pre- and post-injury changes in executive dysfunction, apathy, disinhibition, and depression, and performance on neuropsychological tests of executive function, attention/processing speed, and memory in relation to mood levels and effort test performance in individuals in the early stages of recovery from mild to moderate traumatic brain injury (TBI). Method: Participants were 71 noncombat military personnel who were in a semiacute stage of recovery (<3 months post injury) from mild to moderate TBI. Pre- and post-TBI behaviors were assessed with the Frontal Systems Behavior Scale (FrSBe; Grace & Malloy, 2001) and correlated with levels of depressive symptoms, effort test performance, and performance on objective measures of attention, executive function, and memory. Results: Self-reported symptoms of executive dysfunction generally failed to predict performance on objective measures of executive function and memory, although they predicted poorer performance on measures of attention/processing speed. Instead, higher levels of depressive symptomatology best predicted poorer performance on measures of executive function and memory. However, the relationship between memory performance and TBI symptoms was no longer significant when effort performance was controlled. Conclusions: Our findings suggest that, among individuals in early recovery from mild to moderate TBI, self-reported depressive symptoms, rather than patients' cognitive complaints, are associated with objective executive function. However, self-reported cognitive complaints may be associated with objectively measured inattention and slow processing speed.

Cochlea‐scaled entropy predicts speech intelligibility when intensity is scaled as linear amplitude or as loudness, but not when scaled logarithmically.

Stilp and Kluender [Proc. Natl. Acad. Sci. 107, 12387–12392 (2010)] reported sentence intelligibility to be well‐predicted by measures of sensory change over time [cochlea‐scaled spectral entropy (CSE)] and not consonants, vowels, or duration of signal replaced. As greater amounts of CSE were replaced by noise, intelligibility worsened. They calculated CSE as Euclidean distances between spectra defined by linear amplitude as a function of equal rectangular bandwidth (ERB). The present experiments compare measures of speech intelligibility as a function of CSE when amplitude is measured on a linear scale as before and when log‐scaled (dB) or loudness‐scaled (sone). Listeners typed all words they understood after hearing sentences with 80‐ and 112‐ms segments replaced by speech‐shaped noise. Log‐scaled (dB) CSE was a poor predictor of performance. Both linear‐scaled and loudness‐scaled measures of CSE predict intelligibility very well, replicating Stilp and Kluender. Linear and loudness scales will be discussed in the context of upward spread of masking and two‐tone suppression as they relate to speech signals. These perceptual findings suggest that conventional use of spectrograms (linear frequency × dB intensity) could systematically misinform researchers concerning properties of speech signals that are most useful to listeners. [Work supported by NIDCD.]

Is Undergraduate Performance Predictive of Postgraduate Performance?

Background: The continuity of undergraduate to postgraduate training suggests that performance in medical school should predict performance later in residency. Purpose: The goal is to determine whether undergraduate performance is predictive of postgraduate performance. Methods: Residency program directors assessed the performance of medical school graduates (Classes 2004–2006) at the end of the 1st postgraduate year. Measures of undergraduate performance were retrieved including grade point averages, clerkship in-training evaluation reports, and the total score on the Medical Council of Canada Part 1 exam. Results: Complete data were available for 242 (81.5%) graduates. Postgraduate performance consisted of two reliable factors (clinical acumen and human sensitivity) that explained 78% of the variance. Correlations between undergraduate and the two postgraduate measures were low (.03–.31). Conclusions: Measures of undergraduate performance appear to be poor predictors of performance in residency that consisted of two primary dimensions (clinical acumen and human sensitivity).

Odor Identification Ability Predicts Executive Function Deficits Following Sleep Deprivation

ABSTRACTOdor identification ability is sensitive to prefrontal lobe dysfunction and preliminary evidence suggests that this capacity may decline with prolonged wakefulness. We hypothesized that declines in odor identification during a single night of sleep loss might, therefore, be predictive of prefrontal lobe executive function deficits following an additional night of sleep deprivation. Change scores between two administrations of the University of Pennsylvania Smell Identification Test (SIT) during 24 hr of sleep deprivation were used to predict performance on the Wisconsin Card Sorting Test (WCST) following 45 hr of wakefulness in 54 healthy adults. Declines in SIT performance predicted poorer performance on the WCST following an additional night of sleep loss. These findings suggest that individual differences in vulnerability to the effects of sleep loss on odor identification ability are predictive of deficits in executive functioning following additional wakefulness. Odor identification ability may provide an unobtrusive method for assessing vulnerability to sleep deprivation.

SPECIFIC PROGRAM FOR INTERVENTION IN INCIPIENT PSYCHOSIS INTRODUCED IN CATALONIA(SPAIN)

Background: Elevated presynaptic striatal dopamine is thought to be a final common effect of multiple specific aetiologies, which in turn gives risk to psychosis through the disruption of incentive salience. In contrast to psychosis, negative symptoms are thought to emerge from the disruption of multiple neurophysiological systems (Howes & Kapur, 2009, Schizophr. Bull., 35, 549-62). We tested this model by examining the relationship of an ecologically valid measure of reward learning with anhedonia and intermediate phenotypes of schizophrenia, while controlling for neuropsychological impairment. Methods: Undergraduates (n=84) completed the stimulus chase task (SCT), which provided a measure of sensitivity to reward, selfreport measures of hedonic capacity, and self-report (Schizotypal Personality Questionnaire [SPQ]) and performance (psychobabble task) measures of intermediate phenotypes of schizophrenia. Motor reaction time, attention, and frontal functions were assessed with a small battery of neuropsychological tests. Results: After controlling for neuropsychological functioning, higher cognitive-perceptual SPQ scores predicted poorer performance on the SCT (r=-0.27, p<0.01). Higher interpersonal SPQ scores also predicted poorer SCT performance (r=-0.26, p<0.01). However, anhedonia and disorganization features were not associated with SCT scores. Discussion: The theoretical link between salience, or the efficacy of reward learning, and psychosis is evident at the level of the intermediate phenotypes of schizophrenia reported by ostensibly healthy individuals. However, an equally strong association of salience with intermediate phenotypes of negative symptoms complicates this picture. The observed associations are not attributable to neuropsychological impairment or anhedonia.

Determining the Number of Patient Charts Necessary for a Reliable Assessment of Practicing Family Physicians’ Performance

In many countries, peer assessment programs based on the examination of patient charts are becoming a standard to assess physician's clinical performance. Although data on validity of the process are acceptable, reliability issues need some improvement. This article addresses the rarely studied aspect of optimal number of patient charts for an acceptable reliable assessment. Fifteen patient charts for each of a group of 20 practicing physicians were independently reviewed by 4 professional peer assessors. Generalizability (G) and decision (D) studies were applied to the data. It appears that as few as 10 patient charts are sufficient for any assessor to obtain a G coefficient of 0.80. Results of the current study suggest the possibility of getting generalizable assessments by peer reviewer with minimal information. These results are not in accordance with the concept of case specificity in which it is claimed that performance on a case is a poor predictor of performance on a different case.

Patients' report on how endometriosis affects health, work, and daily life

The objective of this study was to assess the burden of endometriosis by obtaining patient-reported outcome data describing the experience of living with this disease. Survey data from 107 women with self-reported, surgically diagnosed endometriosis showed that living with this disease may be characterized by physical limitations that disrupt health, work, and daily life.

Older Drivers and Failure to Stop at Red Lights

BackgroundDespite sensational news reports, few studies have quantified the rates of poor driving performance among older drivers and the predictors of poor performance. We determined the rate of running red traffic lights among older drivers and the relationship of failure to stop to measures of vision and cognition.MethodsMultiple measures of vision and cognition were collected at the baseline examination of a population of 1,425 drivers aged 67–87 years in greater Salisbury, Maryland. Each driver had real-time data collected on 5 days of driving performance at baseline and again at 1 year. Failure to stop at a red traffic light was the primary outcome.ResultsOverall, 3.8% of older drivers failed to stop at red traffic lights, with 15% of those who ran the light having failed 10% or more of the traffic lights they encountered. A narrowing of the attentional visual field (AVF; the extent of peripheral vision in which objects are detected while attention is also centrally fixated) was associated with failure to stop at traffic lights at baseline and predictive 1 year later (incidence rate ratio = 1.09 per degree lost, 95% confidence interval = 1.01–1.16). Persons with smaller vertical AVF were more likely to fail to stop. No demographic or vision variable was related to failure to stop.ConclusionsFailure to stop at red lights was a relatively uncommon event in older drivers and associated with reduced ability to pay attention to visual events in the vertical field of vision.

Growth Trajectories and Intellectual Abilities in Young Adulthood: The Helsinki Birth Cohort Study

Slow childhood growth is associated with poorer intellectual ability. The critical periods of growth remain uncertain. Among 2,786 Finnish male military conscripts (1952-1972) born in 1934-1944, the authors tested how specific growth periods from birth to age 20 years predicted verbal, visuospatial, and arithmetic abilities at age 20. Small head circumference at birth predicted poorer verbal, visuospatial, and arithmetic abilities. The latter 2 measures were also associated with lower weight and body mass index (weight (kg)/height (m)(2)) at birth (for a 1-standard-deviation (SD) decrease in test score per SD decrease in body size > or = 0.05, P's < 0.04). Slow linear growth and weight gain between birth and age 6 months, between ages 6 months and 2 years, or both predicted poorer performance on all 3 tests (for a 1-SD decrease in test score per SD decrease in growth > or = 0.05, P's < 0.03). Reduced linear growth between ages 2 and 7 years predicted worse verbal ability, and between age 11 years and conscription it predicted worse performance on all 3 tests. Prenatal brain growth and linear growth up to 2 years after birth form a first critical period for intellectual development. There is a second critical period, specific for verbal development, between ages 2 and 7 years and a third critical period for all 3 tested outcomes during adolescence.

Experience and Expertise: How Do They Relate to Quality and Safety?

Often in clinical practice and research, length of clinical experience is equated with expertise. Clinical experience is generally viewed as an important indicator of clinical competence and is a factor that is used when selecting individuals to serve as mentors in both practice and educational settings. In addition, length of clinical experience is often used as an independent variable when examining the quality of clinical outcomes or other indicators of performance. However, as is often the case, things are not always what they seem to be. And so it is when examining the relationship between experience and expertise. Over three decades of research on expert performance has demonstrated that experience itself is not sufficient to predict superior performance. In fact, experience can actually hinder ability in some circumstances. The ten-year rule has often been a widely accepted minimum amount of experience needed to master highly complex endeavors (Ericsson, Charness, Feltovich, & Hoffman, 2006). However, recent research is now suggesting that experience itself does not guarantee successful performance and the number of years of experience in a particular performance domain is a poor predictor of performance. The traditional view has been that extended experience has resulted in the acquisition of rules to guide performance. As experience increases, these rules are used and eventually experts develop the ability to know intuitively what to do at a given point in time. Extended experience was assumed to be sufficient for individuals to become experts who would continue to demonstrate superior levels of performance (Ericsson, Whyte, & Ward, 2007). Experts tend to be good performers, of course, unless something unpredictable happens. If something happens that changes the rules they typically follow, then performance can be affected. When rules change, experts generally do not perform any better than the rest of us. Consider the unique clinical situation that each patient presents and it then becomes easier to understand how typical can quickly become difficult to define. Highly experienced individuals tend to perform routine tasks almost unconsciously, and they retrieve the information they need quickly, many times without stopping to think or apply rules. Experience in performing a particular task sometimes frees space in the mind for other cognitive pursuits, and this may lead to accidents or unintended behaviors. Experience can also lead to overconfidence in performing routine tasks (Ericsson et al., 2006). None of these are desirable states when performing complex patient care tasks, particularly when there is a high level of uncertainty and instability. Nurses have been observed by researchers using think aloud techniques in which the clinicians talk about the thought processes they are going through during actual nursing practice. This research consistently indicates that the structure and content of the nurses' thinking does not vary between experts and those with less experience (Fisher & Fonteyn, 1995; Greenwood & King, 1996). …

“Hold Test” Use in Environmental Epidemiologic Studies On Behavioral Toxicology: Which Tasks Should Be Used?

ISEE-544 Objective: Neuropsychological testing has become a widely accepted methodology for assessing neurotoxicity in environmental and occupational exposure studies. These tools allow quantification of a continuum of functional effects ranging from subtle deficits to clinical diagnosis; a dose-response pattern can be explored. Neuropsychological performance is a dynamic process requiring a full grasp of all the potential covariates associated with performance. One variable associated with performance on neuropsychological test in adults is premorbid intelligence (an individual's intellectual abilities before exposure). Premorbid abilities are usually estimated through the use of one of more “hold tests,” tasks that are thought to be resistant to neurotoxic effects. The single word reading subtest of the Wide Range Achievement Test-3 (WRAT-R/3) is a widely used hold test in neuropsychological research, but its utility and validity have been questioned (Johnstone et al., 1996; Michelsen et al., 1996). This project assessed its utility with adult neurological and occupational groups. Materials and Methods: A pilot study examining visuospatial functioning in neurological patients (Parkinson's disease or multiple sclerosis diagnosis), lead-exposed workers, and normal controls was conducted using the WRAT-R/3 and 8 visuospatial tasks. Results: Performance on the WRAT reading subtest was a poor predictor of performance on the visuospatial tests in the battery (Spearman correlation coefficients of 0.02–0.23; P=0.14–0.74). The Wechsler Adult Intelligence Scale-III (WAIS-III) Picture Completion subtest, a task that requires identification of missing details in drawings, was significantly correlated with all but one of the visuospatial tests (Spearman correlation coefficient ranged from 0.32 to 0.60; P < 0.001–0.05). This test is generally resistant to the effects of neurotoxicant exposure in adults. Conclusions: Studies using neuropsychological tests to examine the impact of neurotoxic substances should include hold tests for both verbal and nonverbal functioning. The WAIS-III Picture Completion subtest is a strong candidate for measurement of premorbid visuospatial abilities.