Subjective Confidence Research Articles

Forecasts or fortune-telling: When are expert judgements of safety risk valid?

Safety analysis frequently relies on human estimates of the likelihood of specific events. For this purpose, the opinions of experts are given greater weight than the opinions of non-experts. Combinations of individual judgements are given greater weight than judgements made by a lone expert. Various authors advocate specific techniques for eliciting and combining these judgements. All of these factors – the use of experts, the use of multiple opinions, and the use of elicitation and combination techniques – serve to increase subjective confidence in the safety analysis. But is this confidence justified? Do the factors increase the actual validity of the analysis in proportion to the increase in subjective confidence?In this paper, by means of a critical synthesis of evidence from multiple disciplines, we argue that it is plausible that expert judgement deserves special standing, but only for well understood local causal mechanisms. We also conclude that expert judgements can be improved by using appropriate elicitation techniques, including by combining judgement from multiple experts. There is, however, no evidence to suggest that fuzzy, neural network, or any other form of complicated processing of expert judgement has any advantage over simple combination mechanisms.

Effect of Noise-Optimized Monoenergetic Postprocessing on Diagnostic Accuracy for Detecting Incidental Pulmonary Embolism in Portal-Venous Phase Dual-Energy Computed Tomography.

The aim of this study was to evaluate the diagnostic accuracy of virtual monoenergetic images (MEI+) at low kiloelectronvolt levels for the detection of incidental pulmonary embolism (PE) in oncological follow-up portal-venous phase dual-energy (DE-CTpv) staging. Twenty-six patients with incidental PE in DE-CTpv staging, which was confirmed by computed tomography pulmonary angiography (CTPA), were included in this study. In addition, 26 matched control patients who were negative for PE in both DE-CTpv and CTPA were included. All examinations were performed on a third-generation DE-CT system in single-energy (CTPA) and dual-energy mode (DE-CTpv). Subsequently, 2 series of MEI+ data sets were reconstructed at 40 and 55 keV from the DE-CTpv data and compared with CTPA and the linearly blended CTpv images. Diagnostic accuracy and diagnostic confidence were calculated and compared for MEI+ reconstructions and for CTpv images regarding the detection of PE with CTPA as standard of reference. In 3 patients, PE was only detectable in CTPA and in the 40-kV and 55-kV MEI+ reconstructions but not in CTpv images. Diagnostic accuracy increased significantly for both MEI+ series (40 keV; area under the curve [AUC], 0.928; 95% confidence interval [CI], 0.879-0.978; 55 keV; AUC, 0.960; 95% CI, 0.922-0.998) as compared with CTpv (AUC, 0.814; 95% CI, 0.741-0.887; P ≤ 0.004). Diagnostic confidence was rated highest in CTPA (median, 5; range, 3-5) followed by 55-keV MEI+ and 40-keV MEI+ (median, 5; range, 2-5 and median, 5; range, 2-5, respectively) and was lowest for CTpv (median, 4; range, 1-5), with significant differences to CTPA and MEI+ reconstructions (P ≤ 0.001). Monoenergetic image reconstructions from DE-CT data sets at low kiloelectronvolt levels improve diagnostic accuracy for the detection of incidental PE in oncological follow-up DE-CTpv staging, with the highest subjective diagnostic confidence in MEI+ at 55 keV.

Are you early or late?: Temporal error monitoring.

Temporal judgments regarding a target interval typically produce a nearly normally distributed reproduction times centered on the target with substantial variance. This phenomenon indicates that the majority of our temporal judgments are deviations from the target times, which are assumed to originate from the underlying timing uncertainty. Although humans were found to adapt their decisions in response to timing uncertainty, we do not know if they can accurately judge the direction and degree of their temporal errors. In this study, we asked participants to reproduce durations as accurately as possible. After each reproduction, participants were asked to retrospectively rate their confidence in their temporal estimates and to judge if their response time was earlier or later than the target interval. The results revealed that human participants are aware of both the direction and magnitude of their timing errors, pointing at an informationally rich temporal error monitoring ability. We further show that a sequential diffusion process can account for the detection of direction of errors as well as the qualitative features of the relationship of objective temporal errors with subjective confidence ratings and associated response times. (PsycINFO Database Record

Metacognitive ability correlates with hippocampal and prefrontal microstructure

The ability to introspectively evaluate our experiences to form accurate metacognitive beliefs, or insight, is an essential component of decision-making. Previous research suggests individuals vary substantially in their level of insight, and that this variation is related to brain volume and function, particularly in the anterior prefrontal cortex (aPFC). However, the neurobiological mechanisms underlying these effects are unclear, as qualitative, macroscopic measures such as brain volume can be related to a variety of microstructural features. Here we leverage a high-resolution (800µm isotropic) multi-parameter mapping technique in 48 healthy individuals to delineate quantitative markers of in vivo histological features underlying metacognitive ability. Specifically, we examined how neuroimaging markers of local grey matter myelination and iron content relate to insight as measured by a signal-theoretic model of subjective confidence. Our results revealed a pattern of microstructural correlates of perceptual metacognition in the aPFC, precuneus, hippocampus, and visual cortices. In particular, we extend previous volumetric findings to show that right aPFC myeloarchitecture positively relates to metacognitive insight. In contrast, decreased myelination in the left hippocampus correlated with better metacognitive insight. These results highlight the ability of quantitative neuroimaging to reveal novel brain-behaviour correlates and may motivate future research on their environmental and developmental underpinnings.

Modelling Human Behaviours in Disasters from Interviews: Application to Melbourne Bushfires

This paper describes a model for raising awareness of the real (irrational and subjective) behaviours of the population in crisis situations. We analyse residents' statements and police hearings gathered aaer Victo-ria Black Saturday bushfires in to deduce a model of human behaviour based on the distinction between objective (capabilities, danger) and subjective (confidence, risk aversion) attributes, and on individual motivations. We evaluate it against observed behaviour archetypes and statistics, and show its explicative value.

Efficacy of Aquatic Treadmill Training on Gait Symmetry and Balance in Subacute Stroke Patients.

ObjectiveTo determine the efficacy of aquatic treadmill training (ATT) as a new modality for stroke rehabilitation, by assessing changes in gait symmetry, balance function, and subjective balance confidence for the paretic and non-paretic leg in stroke patients.MethodsTwenty-one subacute stroke patients participated in 15 intervention sessions of aquatic treadmill training. The Comfortable 10-Meter Walk Test (CWT), spatiotemporal gait parameters, Berg Balance Scale (BBS), and Activities-specific Balance Confidence scale (ABC) were assessed pre- and post-interventions.ResultsFrom pre- to post-intervention, statistically significant improvements were observed in the CWT (0.471±0.21 to 0.558±0.23, p<0.001), BBS (39.66±8.63 to 43.80±5.21, p<0.001), and ABC (38.39±13.46 to 46.93±12.32, p<0.001). The step-length symmetry (1.017±0.25 to 0.990±0.19, p=0.720) and overall temporal symmetry (1.404±0.36 to 1.314±0.34, p=0.218) showed improvement without statistical significance.ConclusionATT improves the functional aspects of gait, including CWT, BBS and ABC, and spatiotemporal gait symmetry, though without statistical significance. Further studies are required to examine and compare the potential benefits of ATT as a new modality for stroke therapy, with other modalities.

Decisional impulsivity and the associative-limbic subthalamic nucleus in obsessive-compulsive disorder: stimulation and connectivity.

Why do we make hasty decisions for short-term gain? Rapid decision-making with limited accumulation of evidence and delay discounting are forms of decisional impulsivity. The subthalamic nucleus is implicated in inhibitory function but its role in decisional impulsivity is less well-understood. Here we assess decisional impulsivity in subjects with obsessive compulsive disorder who have undergone deep brain stimulation of the limbic and associative subthalamic nucleus. We show that stimulation of the subthalamic nucleus is causally implicated in increasing decisional impulsivity with less accumulation of evidence during probabilistic uncertainty and in enhancing delay discounting. Subthalamic stimulation shifts evidence accumulation in subjects with obsessive-compulsive disorder towards a functional less cautious style closer to that of healthy controls emphasizing its adaptive nature. Thus, subjects with obsessive compulsive disorder on subthalamic stimulation may be less likely to check for evidence (e.g. checking that the stove is on) with no difference in subjective confidence (or doubt). In a separate study, we replicate in humans (154 healthy controls) using resting state functional connectivity, tracing studies conducted in non-human primates dissociating limbic, associative and motor frontal hyper-direct connectivity with anterior and posterior subregions of the subthalamic nucleus. We show lateralization of functional connectivity of bilateral ventral striatum to right anterior ventromedial subthalamic nucleus consistent with previous observations of lateralization of emotionally evoked activity to right ventral subthalamic nucleus. We use a multi-echo sequence with independent components analysis, which has been shown to have enhanced signal-to-noise ratio, thus optimizing visualization of small subcortical structures. These findings in healthy controls converge with the effective contacts in obsessive compulsive disorder patients localized within the anterior and ventral subthalamic nucleus. We further show that evidence accumulation is associated with anterior associative-limbic subthalamic nucleus and right dorsolateral prefrontal functional connectivity in healthy controls, a region implicated in decision-making under uncertainty. Together, our findings highlight specificity of the anterior associative-limbic subthalamic nucleus in decisional impulsivity. Given increasing interest in the potential for subthalamic stimulation in psychiatric disorders and the neuropsychiatric symptoms of Parkinson's disease, these findings have clinical implications for behavioural symptoms and cognitive effects as a function of localization of subthalamic stimulation.

Multivoxel neurofeedback selectively modulates confidence without changing perceptual performance.

A central controversy in metacognition studies concerns whether subjective confidence directly reflects the reliability of perceptual or cognitive processes, as suggested by normative models based on the assumption that neural computations are generally optimal. This view enjoys popularity in the computational and animal literatures, but it has also been suggested that confidence may depend on a late-stage estimation dissociable from perceptual processes. Yet, at least in humans, experimental tools have lacked the power to resolve these issues convincingly. Here, we overcome this difficulty by using the recently developed method of decoded neurofeedback (DecNef) to systematically manipulate multivoxel correlates of confidence in a frontoparietal network. Here we report that bi-directional changes in confidence do not affect perceptual accuracy. Further psychophysical analyses rule out accounts based on simple shifts in reporting strategy. Our results provide clear neuroscientific evidence for the systematic dissociation between confidence and perceptual performance, and thereby challenge current theoretical thinking.

Differentiating attention styles and regulatory aspects of self-reported interoceptive sensibility.

Based on prior research, multiple discriminable dimensions of interoception have been defined: awareness, accuracy and sensibility. Some investigators defined interoceptive awareness as metacognitive awareness of interoceptive accuracy, assessed as correspondence between subjective confidence in and objective accuracy of one's heartbeat detection. However, metacognitive awareness has been understood quite differently: 'a cognitive set in which negative thoughts/feelings are experienced as mental events, rather than as the self' or as 'error awareness'. Interoceptive sensibility, defined as self-reported interoception, distinguishes self-reported interoception from objective interoceptive accuracy, but does not differentiate between anxiety-driven and mindful attention styles towards interoceptive cues, a distinction of key clinical importance: one attention style is associated with somatization and anxiety disorders; the other has been viewed as healthy, adaptive, resilience-enhancing. The self-report Multidimensional Assessment of Interoceptive Awareness was developed to differentiate these attention styles. It has been translated into 16 languages and applied in cross-sectional and longitudinal studies. Findings from these applications suggest that differentiating interoceptive sensibility according to attention style and regulatory aspects (i) provides insights into the psychology of interoceptive awareness, (ii) differentiates between clinically maladaptive and beneficial interoceptive attention, and (iii) helps elucidate therapeutic approaches that claim to provide health benefits by training mindful styles of bodily awareness.This article is part of the themed issue 'Interoception beyond homeostasis: affect, cognition and mental health'.

Can There Be a Burden of the Best Explanation?

Exactly how to understand the different standards of proof has been the object of much controversy in evidence scholarship, however. According to a widely accepted approach, standards of proof should be understood as probability thresholds corresponding to a numerical degree of subjective confidence that the judge or juror must reach with regard to each element of the cause of action in order for a verdict in favor of the party carrying the burden of persuasion to be justified. Probabilistic approaches provide elegant accounts of standards of proof, which becomes clearly formulated numerical thresholds. In recent decades, however, a series of objections to these approaches have been raised against such probabilistic approaches. A group of evidence law scholars has proposed an alternative. According to them, legal fact-finding should involve a determination of the best explanation of the admitted evidence, rather than a determination of whether each element is proven to a specific probability. Under this view of standards, inference to the best explanation (hereinafter “IBE”) becomes the characteristic mode of inference in legal fact-finding. However, the epistemology and philosophy of science literature are filled with attacks and defenses of IBE as an epistemically justified mode of inference. Proponents of the proposal to interpret standards of proof as involving explanatory inferences have rejected familiar objections against IBE. However, their rejection might have been too quick. We have better arguments to address these objections. This point is this Article’s first contribution to strengthening an explanation-based approach to standards of proof. The second contribution this Article aims to make is related to the specific formulations of standards of proof and other evidentiary institutes that should follow from the adoption of an explanatory framework to evidence.

Confidently Biased: Comparisons with Anchors Bias Estimates and Increase Confidence

AbstractAcross a wide variety of situations, exposure to anchors has been shown to bias people's estimates. What is not known, however, is whether externally provided anchors influence the confidence that people have in their estimates. Our studies had two goals. First, we tested whether exposure to anchors influenced people's subjective confidence levels (Studies 1 and 2). These studies revealed that people who made estimates after making comparisons with externally provided anchors tended to be more confident in their estimates than people who did not see anchors. The second goal was to test two explanations as to why anchors increase people's confidence. In Study 3, we tested the explanation that anchors increase confidence because participants thought the anchors provided useful information. In Study 4, we tested the explanation that exposure to anchors causes people to consider a narrower range of plausible values as compared to when not exposed to anchors. Support was found only for the explanation that comparisons with anchors increase confidence because people who are exposed to anchors consider a narrower range of plausible values. Taken together, these studies reveal the powerful influence anchors can have—they not only bias estimates, but also increase people's confidence in their biased estimates. Copyright © 2016 John Wiley &amp; Sons, Ltd.

Intra-individual comparison of CAIPIRINHA VIBE technique with conventional VIBE sequences in contrast-enhanced MRI of focal liver lesions

PurposeTo evaluate the impact of controlled aliasing in parallel imaging results in higher acceleration (CAIPIRINHA) volume interpolated breath-hold examination (VIBE) magnetic resonance imaging (MRI) technique on image quality, reader confidence, and inter-observer agreement for the assessment of focal liver lesions in comparison with the standard VIBE approach. Material and methodsIn this IRB-approved intra-individual comparison study, abdominal arterial and portal-venous contrast-enhanced MRI studies were retrospectively analyzed in 38 patients with malignant liver lesions. Each patient underwent both CAIPIRINHA and conventional VIBE 3T MRI within 3 months, showing stable disease. Images were evaluated using 5-point rating scales by two blinded radiologists with more than 20 and 5 years of experience in MRI, respectively. Readers scored dignity of liver lesions and assessed which liver segments were affected by malignancy (ranging from 1=definitely benign/not affected to 5=definitely malignant/affected by malignancy). Readers also rated overall image quality, sharpness of intrahepatic veins, and diagnostic confidence (ranging from 1=poor to 5=excellent). ResultsReviewers achieved a higher inter-observer reliability using CAIPIRINHA when they reported which liver segments were affected by malignancy compared to traditional VIBE series (κ=0.62 and 0.54, respectively, p<0.05). Similarly, CAIPIRINHA showed a slightly higher inter-rater agreement for the dignity of focal liver lesions versus the standard VIBE images (κ=0.50 and 0.49, respectively, p<0.05). CAIPIRINHA series also scored higher in comparison to standard VIBE sequences (mean scores: image quality, 4.2 and 3.5; sharpness of intrahepatic vessels, 3.8 and 3.2, respectively, p<0.05) for both reviewers and allowed for higher subjective diagnostic confidence (ratings, 3.8 and 3.2, respectively, p<0.05). ConclusionCompared to the standard VIBE approach, CAIPIRINHA VIBE technique provides improved image quality and sharpness of intrahepatic veins, as well as higher diagnostic confidence. Additionally, this technique allows for higher inter-observer agreement when reporting focal liver lesions for both dignity and allocation.

Unexpected arousal modulates the influence of sensory noise on confidence.

Human perception is invariably accompanied by a graded feeling of confidence that guides metacognitive awareness and decision-making. It is often assumed that this arises solely from the feed-forward encoding of the strength or precision of sensory inputs. In contrast, interoceptive inference models suggest that confidence reflects a weighted integration of sensory precision and expectations about internal states, such as arousal. Here we test this hypothesis using a novel psychophysical paradigm, in which unseen disgust-cues induced unexpected, unconscious arousal just before participants discriminated motion signals of variable precision. Across measures of perceptual bias, uncertainty, and physiological arousal we found that arousing disgust cues modulated the encoding of sensory noise. Furthermore, the degree to which trial-by-trial pupil fluctuations encoded this nonlinear interaction correlated with trial level confidence. Our results suggest that unexpected arousal regulates perceptual precision, such that subjective confidence reflects the integration of both external sensory and internal, embodied states.

Predictions Shape Confidence in Right Inferior Frontal Gyrus.

Perceptual decision-making is typically conceived as an integration of bottom-up and top-down influences. However, perceptual decisions are accompanied by a sense of confidence. Confidence is an important facet of perceptual consciousness yet remains poorly understood. Here we implicate right inferior frontal gyrus in constructing confidence from the discrepancy between perceptual judgment and its prior probability. Furthermore, we place right inferior frontal gyrus within an occipitofrontal network, consisting of orbitofrontal cortex and intracalcarine sulcus, which represents and communicates relevant top-down and bottom-up signals. Together, our data reveal a role of frontal regions in the top-down processes enabling perceptual decisions to become available for conscious report.

A signature of neural coding at human perceptual limits.

Simple visual features, such as orientation, are thought to be represented in the spiking of visual neurons using population codes. I show that optimal decoding of such activity predicts characteristic deviations from the normal distribution of errors at low gains. Examining human perception of orientation stimuli, I show that these predicted deviations are present at near-threshold levels of contrast. The findings may provide a neural-level explanation for the appearance of a threshold in perceptual awareness whereby stimuli are categorized as seen or unseen. As well as varying in error magnitude, perceptual judgments differ in certainty about what was observed. I demonstrate that variations in the total spiking activity of a neural population can account for the empirical relationship between subjective confidence and precision. These results establish population coding and decoding as the neural basis of perception and perceptual confidence.

Ongoing Slow Fluctuations in V1 Impact on Visual Perception.

The human brain’s ongoing activity is characterized by intrinsic networks of coherent fluctuations, measured for example with correlated functional magnetic resonance imaging signals. So far, however, the brain processes underlying this ongoing blood oxygenation level dependent (BOLD) signal orchestration and their direct relevance for human behavior are not sufficiently understood. In this study, we address the question of whether and how ongoing BOLD activity within intrinsic occipital networks impacts on conscious visual perception. To this end, backwardly masked targets were presented in participants’ left visual field only, leaving the ipsi-lateral occipital areas entirely free from direct effects of task throughout the experiment. Signal time courses of ipsi-lateral BOLD fluctuations in visual areas V1 and V2 were then used as proxies for the ongoing contra-lateral BOLD activity within the bilateral networks. Magnitude and phase of these fluctuations were compared in trials with and without conscious visual perception, operationalized by means of subjective confidence ratings. Our results show that ipsi-lateral BOLD magnitudes in V1 were significantly higher at times of peak response when the target was perceived consciously. A significant difference between conscious and non-conscious perception with regard to the pre-target phase of an intrinsic-frequency regime suggests that ongoing V1 fluctuations exert a decisive impact on the access to consciousness already before stimulation. Both effects were absent in V2. These results thus support the notion that ongoing slow BOLD activity within intrinsic networks covering V1 represents localized processes that modulate the degree of readiness for the emergence of visual consciousness.

Is fear perception special? Evidence at the level of decision-making and subjective confidence.

Fearful faces are believed to be prioritized in visual perception. However, it is unclear whether the processing of low-level facial features alone can facilitate such prioritization or whether higher-level mechanisms also contribute. We examined potential biases for fearful face perception at the levels of perceptual decision-making and perceptual confidence. We controlled for lower-level visual processing capacity by titrating luminance contrasts of backward masks, and the emotional intensity of fearful, angry and happy faces. Under these conditions, participants showed liberal biases in perceiving a fearful face, in both detection and discrimination tasks. This effect was stronger among individuals with reduced density in dorsolateral prefrontal cortex, a region linked to perceptual decision-making. Moreover, participants reported higher confidence when they accurately perceived a fearful face, suggesting that fearful faces may have privileged access to consciousness. Together, the results suggest that mechanisms in the prefrontal cortex contribute to making fearful face perception special.

Dissociating Perceptual Confidence from Discrimination Accuracy Reveals No Influence of Metacognitive Awareness on Working Memory.

Visual awareness is hypothesized to be intimately related to visual working memory (WM), such that information present in WM is thought to have necessarily been represented consciously. Recent work has challenged this longstanding view by demonstrating that visual stimuli rated by observers as unseen can nevertheless be maintained over a delay period. These experiments have been criticized, however, on the basis that subjective awareness ratings may contain response bias (e.g., an observer may report no awareness when in fact they had partial awareness). We mitigated this issue by investigating WM for visual stimuli that were matched for perceptual discrimination capacity (d′), yet which varied in subjective confidence ratings (so-called relative blindsight). If the degree of initial subjective awareness of a stimulus facilitates later maintenance of that information, WM performance should improve for stimuli encoded with higher confidence. In contrast, we found that WM performance did not benefit from higher visual discrimination confidence. This relationship was observed regardless of WM load (1 or 3). Insofar as metacognitive ratings (e.g., confidence, visibility) reflect visual awareness, these results challenge a strong relationship between conscious perception and WM using a paradigm that controls for discrimination accuracy and is less subject to response bias (since confidence is manipulated within subjects). Methodologically, we replicate prior efforts to induce relative blindsight using similar stimulus displays, providing a general framework for isolating metacognitive awareness in order to examine the function of consciousness.

OC-061 Algorithm and Case-Based Educational Intervention Improves Nursing Staff Confidence in Management of Patients On Direct Oral Anticoagulants (DOACS) in Endoscopy

Introduction The increased use of DOACs in medicine, is a challenge for endoscopy unit staff, faced with new anticoagulants, new national guidance based on low quality evidence, and new renal function tests to interpret. Guidance by Veitch (2016) and a systematic review by Woodhouse (2013) suggest practical management algorithms for DOACs in endoscopy, which we aimed to summarise into an interactive case based teaching session, to improve nursing confidence. Methods We developed an hour long teaching course for all endoscopy nurses in our District General Hospital. Small groups discussed previous experience of DOACs, a brief interactive seminar introduced a locally tailored treatment algorithm based on Veitch (2016) and Woodhouse (2013) (Figure 1). Participants were formatively assessed on identifying DOACs, interpreting renal function and applying a management algorithm to common clinical cases. Subjective confidence on each learning objective was assessed pre- and post-educational intervention, using Likert scales graded 1 (strongly disagree) to 5 (strongly agree), and analysed using a paired T-test. Free text responses were collected and phenomenological analysed. Results 18 endoscopy nursing staff undertook training in two sessions. Mean subjective confidence scores improved significantly, in all learning objectives (Table 1). Eleven free text comments praised the use of case studies and the algorithm, as “real life scenarios help to consolidate my learning”. Others commented this teaching should be “cascaded to referrers”. Conclusion Case based learning activities, using an algorithm approach, amongst nursing staff, significantly increased confidence in managing DOACs in endoscopy. This format is readily reproducible in other endoscopy units, and a useful framework to further educate referrers to endoscopy, and patients taking DOACs. References 1 Woodhouse C, et al. Frontline Gastroenterology 2013;4:213–218. 2 Veitch AM, et al. Gut 2016;65:374–389. Disclosure of Interest None Declared

Standardized approach to training for cataract surgery skill evaluation

To assess interrater reliability in grading cataract surgery performance of ophthalmology residents and attending physicians before and after rater skill training. Metropolitan Chicago, Illinois, USA. Prospective interventional test design. Video versions of a previously validated scoring key for rating cataract surgery were created for participant training. All participants received 2 frame-of-reference training sessions lasting 2hours each. Participants graded 4 videorecorded cataract surgeries before training (pretest) and 4 more after training (posttest). Intraclass correlation coefficients (ICCs), which measured the degree of rater agreement, were calculated before and after training. Participants completed a subjective confidence questionnaire at the beginning and at the end of the study. The study participants included 9 postgraduate year (PGY)-3 residents, 8 PGY-4 residents, and 5 ophthalmology attending physicians from 4 medical centers in metropolitan Chicago. The rater training sessions had the greatest increase in ICCs among PGY-3 residents. The ICC improvement between the PGY-4 residents and attending physicians was uneven. After training, all residents felt more confident about their ability to rate and to perform cataract surgery. Learning to effectively grade performance of cataract surgery by reviewing video clips has the potential to be an important part of ophthalmology residency education. The benefit of such a protocol was greatest in the early stages of residency education and might provide a guideline for quality assessment that accelerates surgical skill development. None of the authors has a financial or proprietary interest in any material or method mentioned.