Performance-relevant Information Research Articles

"Praise in Public; Criticize in Private": Unwritable Assessment Comments and the Performance Information That Resists Being Written.

Written assessment comments are needed to archive feedback and inform decisions. Regrettably, comments are often impoverished, leaving performance-relevant information undocumented. Research has focused on content and supervisor's ability and motivation to write it but has not sufficiently examined how well the undocumented information lends itself to being written as comments. Because missing information threatens the validity of assessment processes, this study examined the performance information that resists being written. Two sequential data collection methods and multiple elicitation techniques were used to triangulate unwritten assessment comments. Between November 2022 and January 2023, physicians in Canada were recruited by email and social media to describe experiences with wanting to convey assessment information but feeling unable to express it in writing. Fifty supervisors shared examples via survey. From January to May 2023, a subset of 13 participants were then interviewed to further explain what information resisted being written and why it seemed impossible to express in writing and to write comments in response to a video prompt or for their own "unwritable" example. Constructivist grounded theory guided data collection and analysis. Not all performance-relevant information was equally writable. Information resisted being written as assessment comments when it would require an essay to be expressed in writing, belonged in a conversation and not in writing, or was potentially irrelevant and unverifiable. In particular, disclosing sensitive information discussed in a feedback conversation required extensive recoding to protect the learner and supervisor-learner relationship. When documenting performance information as written comments is viewed as an act of disclosure, it becomes clear why supervisors may feel compelled to leave some comments unwritten. Although supervisors can be supported in writing better assessment comments, their failure to write invites a reexamination of expectations for documenting feedback and performance information as written comments on assessment forms.

A transthalamic pathway crucial for perception

Perception is largely supported by cortical processing that involves communication among multiple areas, typically starting with primary sensory cortex and then involving higher order cortices. This communication is served in part by transthalamic (cortico-thalamo-cortical) pathways, which ubiquitously parallel direct corticocortical pathways, but their role in sensory processing has largely remained unexplored. Here, we suggest that transthalamic processing propagates task-relevant information required for correct sensory decisions. Using optogenetics, we specifically inhibited the pathway at its synapse in higher order somatosensory thalamus of mice performing a texture-based discrimination task. We concurrently monitored the cellular effects of inhibition in primary or secondary cortex using two-photon calcium imaging. Inhibition severely impaired performance despite intact direct corticocortical projections, thus challenging the purely corticocentric map of perception. Interestingly, the inhibition did not reduce overall cell responsiveness to texture stimulation in somatosensory cortex, but rather disrupted the texture selectivity of cells, a discriminability that develops over task learning. This discriminability was more disrupted in the secondary than primary somatosensory cortex, emphasizing the feedforward influence of the transthalamic route. Transthalamic pathways may therefore act to deliver performance-relevant information to higher order cortex and are underappreciated hierarchical pathways in perceptual decision-making.

Comparing Entrustment and Competence: AnExploratory Look at Performance-Relevant Information in the Final Year of a Veterinary Program.

Workplace-based assessments and entrustment scales have two primary goals: providing formative information to assist students with future learning; and, determining if and when learners are ready for safe, independent practice. To date, there has not been an evaluation of the relationship between these performance-relevant information pieces in veterinary medicine. This study collected quantitative and qualitative data from a single cohort of final-year students (n = 27) across in-training evaluation reports (ITERs) and entrustment scales in a distributed veterinary hospital environment. Here we compare progression in scoring and performance within and across student, within and across method of assessment, over time. Narrative comments were quantified using the Completed Clinical Evaluation Report Rating (CCERR) instrument to assess quality of written comments. Preliminary evidence suggests that we may be capturing different aspects of performance using these two different methods. Specifically, entrustment scale scores significantly increased over time, while ITER scores did not. Typically, comments on entrustment scale scores were more learner specific, longer, and used more of a coaching voice. Longitudinal evaluation of learner performance is important for learning and demonstration of competence; however, the method of data collection could influence how feedback is structured and how performance is ultimately judged.

Learning how to learn using simulation: Unpacking disguised feedback using a qualitative analysis of doctors’ telephone talk

Introduction: Telephone talk between clinicians represents a substantial workplace activity in postgraduate clinical education, yet junior doctors receive little training in goal-directed, professional telephone communication.Aim: To assess educational needs for telephone talk and develop a simulation-based educational intervention.Methods: Thematic analysis of 17 semi-structured interviews with doctors-in-training from various training levels and specialties.Results: We identified essential elements to incorporate into simulation-based telephone talk, including common challenging situations for junior doctors as well as explicit and informal aspects that promote learning. These elements have implications for both junior doctors and clinical supervisors, including: (a) explicit teaching and feedback practices and (b) informal conversational interruptions and questions. The latter serve as “disguised” feedback, which aligns with recent conceptualizations of feedback as “performance relevant information”.Conclusions: In addition to preparing clinical supervisors to support learning through telephone talk, we propose several potential educational strategies: (a) embedding telephone communication skills throughout simulation activities and (b) developing stand-alone curricular elements to sensitize junior doctors to “disguised” feedback during telephone talk as a mechanism to augment future workplace learning, i.e. 'learning how to learn' through simulation.

Broadening the Scope of Feedback to Promote Its Relevance to Workplace Learning.

The common goal in medical education is to support the health care workforce, both present and future, in becoming and remaining competent professionals. Both during and after medical training, learning takes place in the clinical workplace. Yet, how feedback is defined in medical education and how it is practiced in clinical training situations, combined with a research focus on "what works," limits its potential for learning. This article explores the theoretical background of learning in interaction and current trends in medical education to broaden the scope of feedback and promote its relevance to workplace learning.A new, wider perspective is outlined in which feedback could be redefined as "performance-relevant information" (PRI). PRI can incorporate all information that is deemed relevant to the learner, drawn from interaction in workplace learning and one's interpretation of performance in the clinical workplace. This information can, for example, come from the evaluation of patient outcomes after treatment; observations of role models' performance; evaluations and assessments; exploring feelings of failure or success; and responses of colleagues and peers.PRI draws attention to learning opportunities that better fit the highly social learning of clinical workplaces and current trends in medical education. It supports the interpretation of individual or team performance in terms of relevance to learning. This allows for a comprehensive way of viewing and stimulating workplace learning and the performance of professionals, providing an opportunity to create lifelong learning strategies and potentially improving the care of patients.

AmI ready for it? Students' perceptions of meaningful feedback on entrustable professional activities.

BackgroundReceiving feedback while in the clinical workplace is probably the most frequently voiced desire of students. In clinical learning environments, providing and seeking performance-relevant information is often difficult for both supervisors and students. The use of entrustable professional activities (EPAs) can help to improve student assessment within competency-based education. This study aimed to illustrate what students’ perceptions are of meaningful feedback viewed as conducive in preparing for performing EPA unsupervised.MethodsIn a qualitative multicentre study we explored students’ perceptions on meaningful feedback related to EPAs in the clinical workplace. Focus groups were conducted in three different healthcare institutes. Based on concepts from the literature, the transcripts were coded, iteratively reduced and displayed.ResultsParticipants’ preferences regarding meaningful feedback on EPAs were quite similar, irrespective of their institution or type of clerkship. Participants explicitly mentioned that feedback on EPAs could come from a variety of sources. Feedback must come from a credible, trustworthy supervisor who knows the student well, be delivered in a safe environment and stress both strengths and points for improvement. The feedback should be provided immediately after the observed activity and include instructions for follow-up. Students would appreciate feedback that refers to their ability to act unsupervised.ConclusionThere is abundant literature on how feedback should be provided, and what factors influence how feedback is sought by students. This study showed that students who are training to perform an EPA unsupervised have clear ideas about how, when and from whom feedback should be delivered.Electronic supplementary materialThe online version of this article (doi: 10.1007/s40037-017-0361-1) contains supplementary material, which is available to authorized users.

Agreement of self-other perceptions matters: Analyzing the effectiveness of principal leadership through multi-source assessment

The purpose of this study was to examine the possible effects of the simultaneous inclusion of self and other ratings of principal instructional leadership on teachers’ self-efficacy beliefs. Special attention was given to the case where principal and teacher ratings were incongruent and exhibited a self-other rating disagreement regarding the principals’ effectiveness in instructional leadership. The data used in the analyses were taken from the Organisation for Economic Co-operation and Development’s (OECD's) Teaching and Learning International Survey (TALIS) conducted in 2008 and involved information from 672 principals and 11,323 teachers in four OECD countries located in the broader Asia-Pacific region. The study tested a set of hypotheses based on a multi-source assessment framework for the analysis of leadership performance. Results indicated that principal–teacher incongruence regarding principal instructional leadership was significantly and negatively associated with teacher self-efficacy across all four countries. The findings suggest that multi-source assessment can provide unique performance-relevant information about leadership that would not be captured by traditional single-source ratings alone.

Performance reactivity in a continuous-performance task: Implications for understanding post-error behavior

Although there has been considerable interest in the effects of errors on subsequent performance, relatively few studies have considered the effects of non-error events that contain some performance-relevant information, such as correct performance on critical trials. In the present article, we propose and assess a hypothesis of performance reactivity. In support of this hypothesis, we provide evidence of performance decrements following both incorrect and correct responses but not following performance-irrelevant events. More specifically, in a continuous response task (Sustained Attention to Response Task), we (1) replicate previous findings that errors of commission on rare NOGO trials produce decrements in subsequent performance, and (2) observe that correct withholds to NOGO trials produce decrements in subsequent accuracy relative to task-irrelevant tones. These results corroborate a hypothesis that some error-related effects on subsequent performance are not unique, but are instead a particularly salient version of a more general performance-reactivity effect.

A Tool for Integrating and Communicating Performance-Relevant Information in CO2 Storage Projects: Description and Application to In Salah

Plans for developing a CO2 storage site should be based on transparently assessing all available performance-relevant information. During a storage project planning decisions will need to be taken to reflect evolving understanding due to progressive information acquisition. The decisions and reasons for them must be communicated clearly to stakeholders, to demonstrate progress and build confidence. To help meet these objectives a software tool based on a decision support methodology termed Evidence Support Logic (ESL) was developed during the European Commission (EC)-supported CO2ReMoVe project. The tool was demonstrated by applying it to the In Salah Gas Project in Algeria.

The Impact of Media Coverage and Motivation on Performance-Relevant Information

Representative democracy requires that citizens know the facts about political performance, and if more such facts are available in the media, citizens have more opportunity to acquire them. In view of increasing media choice, such an opportunity may vary from one individual media environment to another. However, it remains unclear how differences in information acquisition correspond with differences in the information available specifically in those sources that citizens choose to use on a routine basis. Drawing on data from a two-wave panel survey and media content analysis, this article examines if greater availability of performance-relevant information in the media facilitates acquisition of such information among those reliant on these media. It also examines when and how strongly this effect differs for people with different levels of learning motivation. Using a multilevel model, we find that citizens are more likely to learn facts about political performance when their preferred sources offer a greater quantity of performance-relevant information. We also find that motivation moderates the influence of availability, such that strongly motivated individuals gain comparatively the most from a greater supply of information. The implications of our findings for representative democracy are considered. [Supplementary material is available for this article. Go to the publisher's online edition of Political Communication for the following free supplemental resource: overview of political information questions.]

Integrating orbitofrontal cortex into prefrontal theory: common processing themes across species and subdivisions.

Currently, many theories highlight either representational memory or rule representation as the hallmark of prefrontal function. Neurophysiological findings in the primate dorsolateral prefrontal cortex indicate that both features may characterize prefrontal processing. Neurons in the dorsolateral prefrontal cortex encode information in working memory, and this information is represented when relevant to the rules governing performance in a task. In this review, we discuss recent reports of encoding in primate and rat orbitofrontal regions indicating that these features also characterize activity in the orbitofrontal subdivision of the prefrontal cortex. These data indicate that (1) neural activity in the orbitofrontal cortex links the current incentive value of reinforcers to cues, rather than representing the physical features of cues or associated reinforcers; (2) this incentive-based information is represented in the orbitofrontal cortex when it is relevant to the rules guiding performance in a task; and (3) incentive information is also represented in the orbitofrontal cortex in working memory during delays when neither the cues nor reinforcers are present. Therefore, although the orbitofrontal cortex appears to be uniquely specialized to process incentive or motivational information, it may be integrated into a more global framework of prefrontal function characterized by representational encoding of performance-relevant information.