Verbal Confidence Research Articles

A comparison between numeric confidence ratings and verbal confidence statements.

Is confidence most diagnostic of accuracy when expressed in numbers or when expressed in words? This question bears immense importance in many real-world contexts especially within the confines of eyewitness identification. In an eyewitness identification task, we compared the diagnostic value of numeric confidence across rating scales that varied in grain size (3-point vs. 6-point vs. 21-point vs. 101-point rating scales). We also compared the diagnostic value of numeric confidence to verbal confidence statements using several machine-learning algorithms. We found that fine-grain ratings are more diagnostic of identification accuracy than coarse-grain ratings, which suggests that the former provides a closer correspondence to memory strength than the latter. Moreover, we found that verbal confidence statements capture diagnostic information about the likely accuracy of an identification that numeric confidence ratings do not capture. This suggests that verbal confidence statements and numeric confidence ratings reflect partially independent, nonoverlapping sources of information. These results shed light on the processes that provide diagnostic value to confidence. From an applied standpoint, these results suggest that verbal confidence statements and numeric confidence ratings ought to be collected from eyewitnesses after an identification decision. Collecting both captures more diagnostic information than either can capture in isolation. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Variability in verbal eyewitness confidence

AbstractTypically, an eyewitness' verbal confidence is used to judge the reliability of their lineup identification. Across three experiments (N = 3976), we examined eyewitnesses' own words confidence in their lineup decision. For identification decisions (n = 1099), we identified 781 quantitatively unique responses representing 132 qualitatively unique statements that could be categorized into low, medium, and high confidence. For rejectors (n = 781), we identified 599 quantitatively unique responses representing 143 qualitatively unique responses that could be categorized into low, medium, and high confidence. Most participants provided a verbal phrase (e.g., pretty sure) but a significant proportion—34.19% of identifiers and 29.05% of rejectors—provided numbers (e.g., 80%). The present data highlight the variability in how confidence is expressed. The criminal justice system would benefit from guidance for interpreting verbal confidence. We provide a picture of eyewitnesses' verbal confidence as a first step.

Interpreting eyewitness confidence: Numeric, verbal, and graded verbal scales

AbstractIn empirical research, eyewitnesses typically report their confidence numerically (e.g., “I'm 90% sure”). In contrast, in the field, lineup administrators typically ask witnesses to explain their confidence verbally, in the witness' own words (e.g., “I'm quite sure”). Across three studies, we explored how evaluators assess verbal confidence statements: both freely reported and reported using a graded verbal scale. Results showed wide variability in the interpretation of both kinds of confidence statements. Even when evaluating seemingly very strong statements of verbal confidence (e.g., “completely certain”) participants did not necessarily translate these statements into the strongest levels of numeric confidence. Variability in the interpretation of verbal confidence was particularly pronounced for low confidence statements. Moreover, participants preferred to report their confidence numerically rather than verbally. These results indicate the importance of documenting confidence verbatim at the time of the lineup so that the meaning of the witness' original confidence statement is preserved.

Unskilled, underperforming, or unaware? Testing three accounts of individual differences in metacognitive monitoring

Many studies show that competence (e.g., skill, expertise, natural ability) influences individuals' capabilities of monitoring their item-level performance. However, debate persists about how best to explain these individual differences in metacognition. The competence-based account ascribes differences in monitoring to individuals' objective ability level, arguing that the same skills necessary to perform a task are required to effectively monitor performance. The performance-based account attributes differences in monitoring to changes in overall task performance – no individual differences in competence required. Finally, the metacognitive awareness account proposes that alignment between an individuals' self-assessed and objective ability leads to differences in monitoring. In this study, 603 participants completed a self-assessment of face recognition ability, a lineup identification task, and an objective assessment of face recognition ability. We manipulated the number of encoding repetitions and delay between encoding and test to produce varying levels of task performance across objective face recognition ability. Following each lineup decision, participants provided both a numeric confidence rating and a written expression of verbal confidence. We transformed verbal confidence into a quantitative value using machine learning techniques. When matched on overall identification accuracy, objectively stronger face recognizers used numeric and verbal confidence that a) better discriminates between correct and filler lineup identifications than weaker recognizers, and b) shows better calibration to accuracy. Participants with greater self-assessed ability used higher levels of confidence, irrespective of trial accuracy. These results support the competence-based account.

Self-presentation in medicine: How language patterns reflect physician impression management goals and affect perceptions

This paper evaluated how physicians' communication patterns reflect their self-presentation goals and link to patient perceptions. Specifically, in a large field study (N = 54,420 profiles from HealthGrades.com), physician self-descriptions were analyzed linguistically through automated means, with evidence suggesting those who were more self-focused and confident tended to have higher patient ratings online than those who were less self-focused and confident. Physicians who discussed their expertise and compassion for patients were also rated more favorably. A within-subjects experiment in Study 2 (N = 500) also demonstrated that linguistic self-presentation patterns can affect patient perceptions. Participants who read physician profiles with high rates of self-references and verbal confidence rated doctors as warmer and more competent than those who read physician profiles with low rates of self-references and certainty. Together, words are indicators of physicians’ self-presentation strategies and can change patient evaluations. Theoretical contributions for self-presentation and psychology of language research are discussed.

Improving the interpretation of verbal eyewitness confidence statements by distinguishing perceptions of certainty from those of accuracy.

When an eyewitness makes an identification from a lineup, police are also instructed to collect a verbal expression of confidence. This recommendation hinges on the assumption that evaluators will perceive confidence in the manner the witness intended. However, research has consistently shown that these interpretations can be biased by accompanying contextual information. For example, statements that reference facial features (e.g., "I'm very sure. I remember his eyes.") are perceived as less confident than when the statement is presented alone ("I'm very sure.") (featural justification effect). Additionally, perceptions of witness confidence are altered when the witness's identification (mis-)matches the police suspect in a lineup (prior knowledge). We find that the same underlying mechanism explains the bias induced by both featural justification (Experiments 1 and 2) and prior knowledge (Experiment 3) manipulations. Evaluators conflate their own beliefs about the accuracy of an identification with the witness's intended level of confidence. A simple warning that highlights the differences between confidence and accuracy eliminates the featural justification effect, but is less effective for mitigating the influence of prior knowledge. The key takeaway from this paper is that distinguishing perceptions of certainty from those of accuracy improves the interpretation of verbal confidence statements. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Communicating uncertainty in national security intelligence: Expert and nonexpert interpretations of and preferences for verbal and numeric formats.

Organizations in several domains including national security intelligence communicate judgments under uncertainty using verbal probabilities (e.g., likely) instead of numeric probabilities (e.g., 75% chance), despite research indicating that the former have variable meanings across individuals. In the intelligence domain, uncertainty is also communicated using terms such as low, moderate, or high to describe the analyst's confidence level. However, little research has examined how intelligence professionals interpret these terms and whether they prefer them to numeric uncertainty quantifiers. In two experiments (N=481 and 624, respectively), uncertainty communication preferences of expert (n =41 intelligence analysts in Experiment 1) and nonexpert intelligence consumers were elicited. We examined which format participants judged to be more informative and simpler to process. We further tested whether participants treated verbal probability and confidence terms as independent constructs and whether participants provided coherent numeric probability translations of verbal probabilities. Results showed that although most nonexperts favored the numeric format, experts were about equally split, and most participants in both samples regarded the numeric format as more informative. Experts and nonexperts consistently conflated probability and confidence. For instance, confidence intervals inferred from verbal confidence terms had a greater effect on the location of the estimate than the width of the estimate, contrary to normative expectation. Approximately one-fourth of experts and over one-half of nonexperts provided incoherent numeric probability translations for the terms likely and unlikely when the elicitation of best estimates and lower and upper bounds were briefly spaced by intervening tasks.

The language of accurate and inaccurate eyewitnesses.

This article uses machine-learning techniques to examine people's use of verbal expressions of confidence. Across the field of academic psychology, it is often assumed that such statements reflect the same underlying information as numeric confidence ratings. We show that verbal confidence is not redundant with numeric confidence but instead contributes unique diagnostic value in predicting the accuracy of a response. We use eyewitness confidence in a lineup identification as our model paradigm. There is potentially great applied value in developing a machine-learning algorithm that can predict eyewitness identification accuracy, such as by reducing false convictions. To this end, we applied a machine-learning methodology to investigate the natural language of accurate and inaccurate eyewitnesses. This method revealed that verbal confidence statements provide rich diagnostic information about the likely accuracy of eyewitness identifications. Moreover, verbal confidence statements provide unique diagnostic information that traditional indicators of identification accuracy such as numeric confidence ratings and response times do not provide. However, the diagnostic value of an eyewitness confidence statement depends in part on the face recognition ability of the eyewitness: The natural language of strong face recognizers is more diagnostic than the natural language of weak face recognizers. These results are theoretically interesting but, from an applied perspective, this machine-learning methodology may prove useful to those in the criminal justice system who must evaluate eyewitnesses' verbal confidence statements. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Eyewitness confidence and mock juror decisions of guilt: A meta-analytic review.

We investigated the impact of eyewitness confidence on the following dependent variables: (a) guilty or not-guilty verdict; (b) judgments of guilt as measured on a scale; and (c) mock jurors' perception of the accuracy of an eyewitness's identification. In addition, we examined two potential moderators of the effects of eyewitness confidence: (a) whether the eyewitness expressed confidence at trial versus during the initial lineup identification and (b) whether the eyewitness provided a numerical versus a verbal statement of confidence. We expected all analyses to reveal that highly confident eyewitnesses are more persuasive to mock jurors than are eyewitnesses with lower confidence (Hypothesis 1). We expected eyewitness confidence at trial (relative to at identification) to be more persuasive to mock jurors (Hypothesis 2). We expected numerical expressions of confidence to be more persuasive to mock jurors than verbal confidence expressions (Hypothesis 3). We conducted a meta-analysis of 35 studies from 20 published papers and seven theses or dissertations to quantify the effect of eyewitness confidence on juror judgments and investigated the influence of two primary moderator variables, time of confidence and format of confidence expression. All analyses revealed an effect of eyewitness confidence on mock juror decisions (gs = .21-.36). Our moderator analysis showed that the timing of the confidence statement (identification vs. trial) did not affect the influence of eyewitness confidence on mock jurors' judgments of guilt or accuracy. The influence of eyewitness confidence was not moderated by verbal versus numerical expressions of confidence. Although eyewitness confidence is persuasive to mock jurors, the size of this effect is modest. Moreover, verbal and numerical expressions of confidence have similar persuasive effects, and mock jurors do not appear to be sensitive to the likely difference in evidentiary strength of eyewitness confidence expressed at the initial identification versus at trial. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Eyewitnesses' free-report verbal confidence statements are diagnostic of accuracy.

We assessed recent policy recommendations to collect eyewitnesses' confidence statements in witnesses' own words as opposed to numerically. We conducted an experiment to test whether eyewitnesses' free-report verbal confidence statements are as diagnostic of eyewitness accuracy as their numeric confidence statements and whether the diagnostic utility of eyewitnesses' verbal and numeric confidence statements varies across witnessing conditions. We hypothesized that eyewitnesses' verbal and numeric confidence statements are both significantly associated with identification accuracy among choosers and that their diagnostic utility holds across varying witnessing conditions. In the first phase of the experiment, eyewitnesses (N = 4,795 MTurkers; 48.8% female; 50.8% male; .3% other; age M = 36.9) viewed a videotaped mock-crime and made an identification decision from a culprit-present or culprit-absent lineup. We manipulated witnessing conditions at encoding and retrieval to obtain varied levels of memory performance. In the second phase of the experiment, evaluators (N = 456 MTurkers; 35.5% female; 62.7% male .4% other; age M = 36.5) translated witnesses' verbal confidence statements to a numeric estimate and we used calibration and confidence-accuracy characteristic analyses to compare the diagnosticity of witnesses' verbal and numeric confidence statements across the two levels of memory performance. Witnesses' verbal and numeric confidence statements were significantly and nondifferentially diagnostic of eyewitness accuracy for both choosers and nonchoosers, and their diagnostic utility held across variations in witnessing conditions. These findings suggest the applied utility of collecting either verbal or numeric confidence statements from eyewitnesses immediately following an identification decision. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Supplemental Material for Eyewitnesses’ Free-Report Verbal Confidence Statements Are Diagnostic of Accuracy

Supplemental Material for Eyewitnesses’ Free-Report Verbal Confidence Statements Are Diagnostic of Accuracy

Judging memory: strong verbal confidence and lineup context influence inferences about eyewitnesses

ABSTRACT Recent research demonstrated that people readily integrate contextual information when interpreting eyewitnesses’ verbal expressions of confidence. Identifications justified by memory for a facial feature that appeared not to discriminate between lineup members reduced mock jurors’ judgments of perceived eyewitness confidence, but seemingly discriminative justifications were rated similarly to statements mentioning confidence alone. This suggests that participants may assume confident witnesses already have a good basis for their identification. In Experiment 1, participants rated the extent to which various factors influenced witness confidence. In Experiment 2, participants made judgments about the presumed witnessing and retrieval conditions (e.g. amount of attention to the perpetrator) for statements that varied with respect to the degree of witness confidence, type of justification and feature discriminability of the suspect. Results suggest that people believe that confident identifications are the product of relatively favorable conditions, although these inferences are influenced by the lineup context. These beliefs have the potential to impact how jurors assess the credibility of eyewitnesses. Furthermore, our findings have implications for ways of reducing bias when jurors evaluate eyewitness identification evidence.

The Confidence-Accuracy Relationship Using Scale Versus Other Methods of Assessing Confidence

Historically, researchers have collected eyewitness identification confidence using scales; however, in practice, eyewitnesses are more commonly asked for a verbal statement. In Experiment 1, participants viewed a simultaneous lineup and provided confidence in their own words, by explaining why they made their decision or by selecting from statements made by real eyewitnesses, and then provided a scale rating (0–100%) or provided only the scale rating. In Experiment 2, participants viewed a sequential lineup and provided confidence in their own words followed by the scale rating or provided only the scale rating. Confidence predicted identification accuracy in all conditions, although verbal statements were highly variable and challenging to interpret. For example, only when scale-based confidence was high (80%+) did interpretation of the verbal confidence statement reliably align with scale-based confidence. These data highlight the complexity of verbal confidence statements and the need to establish meaningful boundaries for interpreting verbal confidence statements.

Prior knowledge influences interpretations of eyewitness confidence statements: ‘The witness picked the suspect, they must be 100% sure%

ABSTRACTWhen an eyewitness identifies a suspect from a lineup, it is important to know how certain they are about the decision. Even though eyewitnesses are likely to express certainty with words, past research shows that verbal confidence statements (e.g. ‘I’m pretty sure’) are prone to systematic misinterpretation. Until now, no one has examined how an evaluator's prior knowledge, such as which lineup member is the police suspect, influences their interpretation of eyewitness confidence about a lineup identification. Experiments 1 and 3 show that participants perceived the identical statement of confidence as meaning a higher and lower level of certainty, respectively, when the eyewitness's selection either matched or mismatched the police's suspect. Experiment 2 shows that these effects generally persist when the bias manipulation is manipulated between-subjects. Finally, Experiment 3 finds that clarifying the witness's statement with numeric information (e.g. I’m 80% sure) does not eliminate the influence of biasing information.

Understanding metacognitive confidence: Insights from judgment-of-learning justifications

This study employed the delayed judgment-of-learning (JOL) paradigm to investigate the content of metacognitive judgments; after studying cue-target word-pairs, participants predicted their ability to remember targets on a future memory test (cued recognition in Experiments 1 and 2 and cued recall in Experiment 3). In Experiment 1 and the confidence JOL group of Experiment 3, participants used a commonly employed 6-point numeric confidence JOL scale (0–20–40–60–80–100%). In Experiment 2 and the binary JOL group of Experiment 3 participants first made a binary yes/no JOL prediction followed by a 3-point verbal confidence judgment (sure-maybe-guess). In all experiments, on a subset of trials, participants gave a written justification of why they gave that specific JOL response. We used natural language processing techniques (latent semantic analysis and word frequency [n-gram] analysis) to characterize the content of the written justifications and to capture what types of evidence evaluation uniquely separate one JOL response type from others. We also used a machine learning classification algorithm (support vector machine [SVM]) to quantify the extent to which any two JOL responses differed from each other. We found that: (i) participants can justify and explain their JOLs; (ii) these justifications reference cue familiarity and target accessibility and so are particularly consistent with the two-stage metacognitive model; and (iii) JOL confidence judgements do not correspond to yes/no responses in the manner typically assumed within the literature (i.e. 0–40% interpreted as no predictions).

Context influences interpretation of eyewitness confidence statements.

When an eyewitness makes an identification from a lineup, he or she is asked to provide a confidence statement to help jurors assess credibility. However, these are verbal statements and people must rely on metacognitive processes to correctly interpret them. Recently, Dodson and Dobolyi (2015) argued that a person's interpretation of a witness's verbal confidence is influenced by the diagnosticity of the features used to justify his or her identification. We tested this hypothesis in 2 experiments. Experiment 1 found that, relative to a confidence-only control, participants reduced their ratings of confidence when statements were justified using a facial feature that was shared by lineup members, but not when the feature was unique to the member chosen from the lineup. In Experiment 2, we found that participants integrated across the dimensions of witness confidence and accuracy, even when asked to make separate ratings. Altogether, the results suggest that people readily incorporate contextual information into their interpretations of witnesses' verbal expressions of confidence and evaluations of accuracy. (PsycINFO Database Record

Intercultural interactions of mono-cultural, mono-lingual local students in small group learning activities: a Bourdieusian analysis

This research examines the understandings and experiences of mono-cultural, mono-lingual local students in relation to intercultural interactions within small group learning activities at university. Bourdieu’s concepts of field, habitus and capital are employed to illuminate a number of barriers to intercultural interaction. Using qualitative analysis of interviews with local students, the study revealed intercultural interactions to be co-shaped by multiple co-existing, interdependent and often competing structural and contextual elements, as well as the personal dispositions that students bring to the classroom. The field tends to privilege the capital held by local students (knowledge of small group work, verbal confidence, language), and imposes structural limits on interactions. While students' habitus mediates this effect, the overall result discourages interactions between local students and those from different cultural backgrounds. Implications for universities are discussed.

Gambling on the unconscious: A comparison of wagering and confidence ratings as measures of awareness in an artificial grammar task

We explore three methods for measuring the conscious status of knowledge using the artificial grammar learning paradigm. We show wagering is no more sensitive to conscious knowledge than simple verbal confidence reports but is affected by risk aversion. When people wager rather than give verbal confidence they are less ready to indicate high confidence. We introduce a “no-loss gambling” method which is insensitive to risk aversion. We show that when people are just as ready to bet on a genuine random process as their own classification decisions, their classifications are still above baseline, indicating knowledge participants are not aware of having. Our results have methodological implications for any study investigating whether people are aware of knowing.

Suspect/Foil Identification in Actual Crimes and in the Laboratory: A Reality Monitoring Analysis.

Four reality monitoring variables were used to discriminate suspect from foil identifications in 183 actual criminal cases. Four hundred sixty-one identification attempts based on five and six-person lineups were analyzed. These identification attempts resulted in 238 suspect identifications and 68 foil identifications. Confidence, automatic processing, eliminative processing and feature use comprised the set of reality monitoring variables. Thirty-five verbal confidence phrases taken from police reports were assigned numerical values on a 10-point confidence scale. Automatic processing identifications were those that occurred "immediately" or "without hesitation." Eliminative processing identifications occurred when witnesses compared or eliminated persons in the lineups. Confidence, automatic processing and eliminative processing were significant predictors, but feature use was not. Confidence was the most effective discriminator. In cases that involved substantial evidence extrinsic to the identification 43% of the suspect identifications were made with high confidence, whereas only 10% of the foil identifications were made with high confidence. The results of a laboratory study using the same predictors generally paralleled the archival results. Forensic implications are discussed.

When 90% confidence intervals are 50% certain: on the credibility of credible intervals

AbstractEstimated confidence intervals for general knowledge items are usually too narrow. We report five experiments showing that people have much less confidence in these intervals than dictated by the assigned level of confidence. For instance, 90% intervals can be associated with an estimated confidence of 50% or less (and still lower hit rates). Moreover, interval width appears to remain stable over a wide range of instructions (high and low numeric and verbal confidence levels). This leads to a high degree of overconfidence for 90% intervals, but less for 50% intervals or for free choice intervals (without an assigned degree of confidence). To increase interval width one may have to ask exclusion rather than inclusion questions, for instance by soliciting ‘improbable’ upper and lower values (Experiment 4), or by asking separate ‘more than’ and ‘less than’ questions (Experiment 5). We conclude that interval width and degree of confidence have different determinants, and cannot be regarded as equivalent ways of expressing uncertainty. Copyright © 2005 John Wiley & Sons, Ltd.