Confidence-accuracy Relationship Research Articles

Decision Strategies and the Confidence–Accuracy Relationship in Face Recognition

ABSTRACTThe decision strategy used to select a choice set from an array of alternative options is known to affect the composition of the final choice set. Specifically, individuals incorporate more answers into their choice set when it is created by eliminating implausible items than when the set is created through the inclusion of plausible options. This difference is accounted for in a decision framework that posits a general reluctance to change the status quo (i.e., actively include or exclude an item). We extended this work to investigate, not only the decisions themselves, but also metacognitive judgments (i.e., confidence in the accuracy of the choice set). In two face recognition experiments, we tested the impact of decision strategy (Experiment 1) and confidence judgment strategy (Experiment 2) on the confidence–accuracy relationship. In Experiment 1, participants completed two blocks of recognition trials, one under inclusion (marking previously seen faces) and one under elimination (marking previously unseen faces) instructions. We observed superior resolution (i.e., discrimination between correct and incorrect) for inclusion trials, but only when they were completed prior to use of the elimination strategy. In Experiment 2, all participants completed face recognition trials under inclusion instructions, but we manipulated the strategy used to assess confidence. Again, we observed a significant impact of strategy on confidence–accuracy resolution. Thus, we observed that both the strategy employed to reach a decision and that employed to assess confidence affected the confidence–accuracy relationship. We discuss theoretical and applied (particularly for eyewitness identification and multiple‐choice testing) implications. Copyright © 2012 John Wiley & Sons, Ltd.

The self-consistency model of subjective confidence.

How do people monitor the correctness of their answers? A self-consistency model is proposed for the process underlying confidence judgments and their accuracy. In answering a 2-alternative question, participants are assumed to retrieve a sample of representations of the question and base their confidence on the consistency with which the chosen answer is supported across representations. Confidence is modeled by analogy to the calculation of statistical level of confidence (SLC) in testing hypotheses about a population and represents the participant's assessment of the likelihood that a new sample will yield the same choice. Assuming that participants draw representations from a commonly shared item-specific population of representations, predictions were derived regarding the function relating confidence to inter-participant consensus and intra-participant consistency for the more preferred (majority) and the less preferred (minority) choices. The predicted pattern was confirmed for several different tasks. The confidence-accuracy relationship was shown to be a by-product of the consistency-correctness relationship: It is positive because the answers that are consistently chosen are generally correct, but negative when the wrong answers tend to be favored. The overconfidence bias stems from the reliability-validity discrepancy: Confidence monitors reliability (or self-consistency), but its accuracy is evaluated in calibration studies against correctness. Simulation and empirical results suggest that response speed is a frugal cue for self-consistency, and its validity depends on the validity of self-consistency in predicting performance. Another mnemonic cue-accessibility, which is the overall amount of information that comes to mind-makes an added, independent contribution. Self-consistency and accessibility may correspond to the 2 parameters that affect SLC: sample variance and sample size.

Confidence–Accuracy Calibration with General Knowledge and Eyewitness Memory Cued Recall Questions

SummarySummary: The confidence–accuracy relationship has primarily been studied through recognition tests and correlation analysis. However, cued recall is more ecological from a forensic perspective. Moreover, there may be more informative ways of analysing the confidence–accuracy relationship than correlations. In the present study, participants viewed a video of a bank robbery and were asked cued recall questions covering general knowledge and the video itself. Confidence ratings were collected, and correlations, calibration and discrimination measures were calculated. All measures indicated a strong confidence–accuracy relationship that was better for general knowledge than eyewitness memory questions. However, there were no differences in confidence ratings for correct answers, suggesting that the differences could be limited to the evaluation of incorrect answers. We concluded that confidence may be a good marker for accuracy with cued recall, but that further research using ecological tests and more informative data analysis techniques is needed. Copyright © 2011 John Wiley & Sons, Ltd.

Subjective confidence in perceptual judgments: A test of the self-consistency model.

Two questions about subjective confidence in perceptual judgments are examined: the bases for these judgments and the reasons for their accuracy. Confidence in perceptual judgments has been claimed to rest on qualitatively different processes than confidence in memory tasks. However, predictions from a self-consistency model (SCM), which had been confirmed for general-information questions (Koriat, 2010) and social attitudes (Koriat & Adiv, 2010), are shown to hold true also for perceptual judgments. In SCM, confidence is modeled by the procedure for assessment of statistical level of confidence: For a 2-alternative, forced-choice item, confidence is based on the consistency with which the choice is favored across a sample of representations of the item, and acts as a monitor of the likelihood that a new sample will yield the same choice. Assuming that these representations are drawn from commonly shared populations of representations associated with each item, predictions regarding the basis of confidence were confirmed by results concerning the functions relating confidence and choice latency to interparticipant consensus and to intraparticipant consistency for majority and minority choices. With regard to the confidence-accuracy (C/A) relationship, the consensuality principle, documented for general-knowledge tasks (Koriat, 2008a), was replicated for perceptual judgments: Confidence correlated with the consensuality of the choice rather than with its correctness, suggesting that the C/A correlation is due to the relationship between confidence and self-consistency and is positive only as long as the correct choices are the consistently made choices. SCM provides a general model for the basis and accuracy of confidence judgments across different domains.

Different confidence–accuracy relationships for feature-based and familiarity-based memories.

Participants studied naturalistic pictures presented for varying brief durations and then received a recognition test on which they indicated whether each picture was old or new and rated their confidence. In 1 experiment they indicated whether each "old"/"new" response was based on memory for a specific feature in the picture or instead on the picture's general familiarity; in another experiment, we defined pictures that tended to elicit feature versus familiarity responses. Thus, feature/familiarity was a dependent variable in 1 experiment and an independent variable in the other. In both experiments feature-based responses were more accurate than those that were familiarity based, and confidence and accuracy increased with duration for both response types. However, when confidence was controlled for, mean accuracy was higher for familiarity-based than for feature-based responses. The theoretical implication is that confidence and accuracy arise from different underlying information. The applied implication is that confidence differences should not be taken as implying accuracy differences when the phenomenal basis of the memory reports differ.

Supplemental Material for Different Confidence–Accuracy Relationships for Feature-Based and Familiarity-Based Memories

Supplemental Material for Different Confidence–Accuracy Relationships for Feature-Based and Familiarity-Based Memories

Response latency as a predictor of the accuracy of children's reports.

Researchers have explored various diagnostic cues to the accuracy of information provided by child eyewitnesses. Previous studies indicated that children's confidence in their reports predicts the relative accuracy of these reports, and that the confidence-accuracy relationship generally improves as children grow older. In this study, we examined the added contribution of response latency to the prediction of children's accuracy over and above that of confidence ratings. In Experiments 1 and 2, 2nd and 5th graders studied picture-event pairs and were tested using forced-choice, 2-alternative, or 5-alternative questions. In Experiment 3, children watched a slideshow depicting a story and were tested by 5-alternative questions about story details. The children indicated their confidence in each response, and response latency was measured. The results of all experiments suggested that children in both age groups relied on response latency as a cue for confidence, and this reliance contributed to the success with which they monitored the accuracy of their reports. When the test format was easy (Experiment 1), 2nd graders were as accurate as 5th graders in monitoring the accuracy of their answers, and the latency of their responses was no less predictive of accuracy. When the task was more difficult, age differences emerged. Nevertheless, in all experiments and for both age groups, response latency was found to have added value for predicting accuracy over and above that of confidence. Theoretical and practical implications of these findings for predicting the accuracy of children's reports are discussed.

The effect of retention interval on the confidence–accuracy relationship for eyewitness identification.

Recent research using a calibration approach indicates that eyewitness confidence assessments obtained immediately after a positive identification decision provide a useful guide as to the likely accuracy of the identification. This study extended research on the boundary conditions of the confidence-accuracy (CA) relationship by varying the retention interval between encoding and identification test. Participants (N = 1,063) viewed one of five different targets in a community setting and attempted an identification from an 8-person target-present or -absent lineup either immediately or several weeks later. Compared to the immediate condition, the delay condition produced greater overconfidence and lower diagnosticity. However, for choosers at both retention intervals there was a meaningful CA relationship and diagnosticity was much stronger at high than low confidence levels.

Neural basis for recognition confidence in younger and older adults.

Although several studies have examined the neural basis for age-related changes in objective memory performance, less is known about how the process of memory monitoring changes with aging. The authors used functional magnetic resonance imaging to examine retrospective confidence in memory performance in aging. During low confidence, both younger and older adults showed behavioral evidence that they were guessing during recognition and that they were aware they were guessing when making confidence judgments. Similarly, both younger and older adults showed increased neural activity during low- compared to high-confidence responses in the lateral prefrontal cortex, anterior cingulate cortex, and left intraparietal sulcus. In contrast, older adults showed more high-confidence errors than younger adults. Younger adults showed greater activity for high compared to low confidence in medial temporal lobe structures, but older adults did not show this pattern. Taken together, these findings may suggest that impairments in the confidence-accuracy relationship for memory in older adults, which are often driven by high-confidence errors, may be primarily related to altered neural signals associated with greater activity for high-confidence responses.

The benefits of knowing what you know (and what you don’t): How calibration affects credibility

People tend to believe, and take advice from, informants who are highly confident. However, people use more than a mere “confidence heuristic.” We believe that confidence is influential because—in the absence of other information—people assume it is a valid cue to an informant’s likelihood of being correct. However, when people get evidence about an informant’s calibration (i.e., her confidence–accuracy relationship) they override reliance on confidence or accuracy alone. Two experiments in which participants choose between two opposing witnesses to a car accident show that neither confidence nor accuracy alone explains judgments of credibility; rather, whether a person is seen as credible ultimately depends on whether the person demonstrates good calibration. Credibility depends on whether sources were justified in believing what they believed.

The confidence-accuracy relationship in eyewitness identification: Effects of lineup instructions, foil similarity, and target-absent base rates.

Discriminating accurate from mistaken eyewitness identifications is a major issue facing criminal justice systems. This study examined whether eyewitness confidence assists such decisions under a variety of conditions using a confidence-accuracy (CA) calibration approach. Participants (N = 1,200) viewed a simulated crime and attempted 2 separate identifications from 8-person target-present or target-absent lineups. Confidence and accuracy were calibrated for choosers (but not nonchoosers) for both targets under all conditions. Lower overconfidence was associated with higher diagnosticity, lower target-absent base rates, and shorter identification latencies. Although researchers agree that courtroom expressions of confidence are uninformative, our findings indicate that confidence assessments obtained immediately after a positive identification can provide a useful guide for investigators about the likely accuracy of an identification.

Positive versus negative face recognition decisions: confidence, accuracy, and response latency

A large amount of eyewitness identification and face recognition research has investigated the confidence-accuracy (CA) relationship. One consistent finding is that positive recognition decisions (or choosers) demonstrate superior CA calibration to negative recognition decisions (or non-choosers). This experiment tested whether an explanation of this difference, based on the information available for confidence judgements, accounted for the pattern of CA calibration in positive and negative face recognition decisions. CA calibration for positive and negative decisions was compared for both item and associative recognition judgements. Significantly greater resolution was observed for positive decisions in both the item and associative conditions. Similarly, for both judgement types, positive decisions evidenced a stronger response latency-accuracy relationship than negative decisions. Implications for diagnosing the accuracy of eyewitness identification are discussed. Copyright © 2005 John Wiley & Sons, Ltd. Language: en

The Confidence-Accuracy and Decision Latency-Accuracy Relationships in Children's Eyewitness Identification

This study examined the eyewitness confidence-accuracy and decision latency-accuracy relations in a sample of children (N = 240) and a control group of adolescents (N = 159). Participants witnessed a video of a simulated crime and later attempted an identification from an eight-person photospread line-up on a computer screen under target-present conditions. Both identification confidence and decision latency were measured. Although the confidence-accuracy correlations were similarly modest for both samples, the subjective certainty-accuracy relation clearly distinguished children from adolescents, with children displaying pronounced over-confidence. For the decision latency-accuracy relation, children were not distinguished from adolescents, although there was evidence that children's decisions were more impulsive. The results suggest that, in contrast with patterns emerging in research with adults, neither confidence nor decision latency has promise for distinguishing accurate from inaccurate identification responses in children.

Common-sense beliefs, recognition and the identification of familiar and unfamiliar speakers from verbal and non-linguistic vocalizations

Listeners first listened to tape-recorded speech samples of words and non-linguistic vocalizations to decide whether each came from a familiar or an unfamiliar speaker. If a vocalization was judged as being uttered by a familiar speaker, listeners were asked to identify, if possible, the individual by name. Listeners were instructed not to guess. Confidence-accuracy scores in judgements and identification also were assessed. A separate group of participants attempted to predict the accuracy of performance of the listeners for each of the utterances made by the familiar and unfamiliar speakers. Tokens of some utterances, namely the words hello and help me and the sounds of laughter and clearing the throat, allowed equally good performance on the familiar/unfamiliar decision whether their source was a familiar or an unfamiliar speaker. For other vocalizations (moan, cough, grunt and sigh) listeners were better at detecting unfamiliar speakers than at recognizing familiar speakers. Those utterances judged to be made by familiar speakers led to correct identifications approximately 50 percent of the time on hearing the words hello and help me, and laughter. All other utterances led to less than 30 percent correct speaker identifications. For those unfamiliar speakers who were falsely recognized as familiar, listeners falsely identified by name between 22 and 30 percent of the vocalizations. Few of the confidence-accuracy relationships were significant. Laypersons’ common-sense beliefs for speaker identification were generally unrealistic.

The influence of courtroom questioning style on actual and perceived eyewitness confidence and accuracy

Purpose. Little research has been conducted on the effects of courtroom examination/questioning styles on witness confidence and accuracy. Two studies were therefore conducted, one investigating the effects of examination style on witness confidence and accuracy, the other investigating observers/jurors perceptions of witness confidence and accuracy.Method. In Study 1, after observing a video event, 60 witnesses were individually interviewed about the event according to one of three conditions: (1) simple questioning style, (2) lawyerese questioning style (containing leading and suppositional phrases), and (3) lawyerese with negative feedback style. In study 2, 60 observers/ jurors observed a good and a poor witness under examination by one of the three questioning styles. Measures of the perceived fairness of the examination were also taken in study 2.Results. In the main, significant results were found only for question items classed as difficult to remember. The lawyerese style appeared to have an adverse affect on confidence‐accuracy relationships. Adding subtle negative feedback reduced the problem, but at the price of reduced overall accuracy. Observers (jurors) also seemed to be most affected by observing the negative feedback style; judging the witness overall to be less accurate. An unexpected result was that, regardless of questioning style, presenting the testimony of the least confident witness first appeared to spuriously boost confidence and thereby perceived accuracy, in that witness's testimony. No significant effects were found for perceived fairness.Conclusions. In general, these results lend some support to those who have asserted that the lawyerese style of questioning may be unwise.

Inter‐racial contact and the own‐race bias for face recognition in South Africa and England

AbstractOwn‐race bias, where people are more accurate recognizing faces of people from their own race than other races, can lead to misidentification and, in some cases, innocent people being convicted. This bias was explored in South Africa and England, using Black and White participants. People were shown several photographs of Black and White faces and were later asked if they had seen these faces (and several fillers). In addition, participants were given a questionnaire about inter‐racial contact. Cross‐race identification accuracy for Black participants was positively correlated with self‐reported inter‐racial contact. The confidence–accuracy relationship was strongest when making own‐race judgements. Copyright © 2003 John Wiley & Sons, Ltd.

The confidence-accuracy relationship in eyewitness identification: The effects of reflection and disconfirmation on correlation and calibration.

The confidence-accuracy relationship in eyewitness identification: The effects of reflection and disconfirmation on correlation and calibration.

The confidence-accuracy relationship in eyewitness identification: The effects of reflection and disconfirmation on correlation and calibration.

Participants viewed a simulated crime and attempted an identification from an 8-person target-present or target-absent lineup. The authors examined identification confidence-accuracy relations, contrasting a control condition (n = 310) with 2 manipulations designed to improve confidence scaling. Before indicating confidence, participants reflected on encoding and identification test conditions (n = 316) or suggested hypotheses about why their identification decision might have been wrong (n = 318). Confidence-accuracy correlations were weak and did not differ across conditions. However, for positive identifications, confidence and accuracy were well calibrated in the experimental conditions, although not in the control condition; similar patterns were observed for lineup rejections. Explanations for calibration differences in terms of discrimination difficulty, (mis)match between encoding and test stimuli, and the availability of confidence cues were advanced.

Selecting accurate statements from the cognitive interview using confidence ratings.

Participants viewed a videotape of a simulated murder, and their recall (and confidence) was tested 1 week later with the cognitive interview. Results indicated that (a) the subset of statements assigned high confidence was more accurate than the full set of statements; (b) the accuracy benefit was limited to information that forensic experts considered relevant to an investigation, whereas peripheral information showed the opposite pattern; (c) the confidence-accuracy relationship was higher for relevant than for peripheral information; (d) the focused-retrieval phase was associated with a greater proportion of peripheral and a lesser proportion of relevant information than the other phases; and (e) only about 50% of the relevant information was elicited, and most of this was elicited in Phase 1.

Confidence–accuracy relations for real and suggested events

Participants completed a multiple item, multiple event version of a traditional misinformation procedure, as well as a battery of individual difference measures. The relation between memory accuracy and selfreported confidence was assessed through the comparison of items involving misinformation and items not involving misinformation. Selected individual differences in the confidence–accuracy relation were also examined for items that did and did not involve false post-event information. Results indicated significant differences between the measure of the confidence–accuracy relationship for misinformation and non-misinformation items. Several significant, although weak, individual difference correlates of the confidence–accuracy relations were also found.