Confidence Judgments Research Articles

The Naïve Utility Calculus as a unified, quantitative framework for action understanding

The human ability to reason about the causes behind other people’ behavior is critical for navigating the social world. Recent empirical research with both children and adults suggests that this ability is structured around an assumption that other agents act to maximize some notion of subjective utility. In this paper, we present a formal theory of this Naïve Utility Calculus as a probabilistic generative model, which highlights the role of cost and reward tradeoffs in a Bayesian framework for action-understanding. Our model predicts with quantitative accuracy how people infer agents’ subjective costs and rewards based on their observable actions. By distinguishing between desires, goals, and intentions, the model extends to complex action scenarios unfolding over space and time in scenes with multiple objects and multiple action episodes. We contrast our account with simpler model variants and a set of special-case heuristics across a wide range of action-understanding tasks: inferring costs and rewards, making confidence judgments about relative costs and rewards, combining inferences from multiple events, predicting future behavior, inferring knowledge or ignorance, and reasoning about social goals. Our work sheds light on the basic representations and computations that structure our everyday ability to make sense of and navigate the social world.

Resilience of perceptual metacognition in a dual-task paradigm.

When people do multiple tasks at the same time, it is often found that their performance is worse relative to when they do those same tasks in isolation. However, one aspect that has received little empirical attention is whether the ability to monitor and evaluate one's task performance is also affected by multitasking. How does dual-tasking affect metacognition and its relation to performance? We investigated this question through the use of a visual dual-task paradigm with confidence judgments. Participants categorized both the color and the motion direction of moving dots, and then rated their confidence in both responses. Across four experiments, participants (N = 87) exhibited a clear dual-task cost at the perceptual level, but no cost at the metacognitive level. We discuss this resilience of metacognition to multitasking costs, and examine how our results fit onto current models of perceptual metacognition.

Breast Cancer knowledge and perceived barriers to help seeking among pre-clinical and clinical female medical students of King Edward Medical University, Lahore: a cross-sectional study

BackgroundBreast cancer is the most common form of cancer in women and second most common cancer after lung cancer. The prognosis of breast cancer depends on early detection and intervention which in turn relies upon awareness. Health workers in all communities are responsible for educating the population at risk. This study evaluates the knowledge regarding breast cancer, personal judgment of confidence, perceived barriers to help seeking and breast self-examination practices among Pakistani female medical students and studies the impact of clinical training on the studied variables.MethodsDescriptive cross-sectional study design using self-administered validated questionnaires (BCAM-Breast Cancer Awareness Measure designed by Cancer Research UK) was employed. Female medical students enrolled in clinical and pre-clinical classes of King Edward Medical University, Lahore were targeted and questioned regarding symptoms, risk factors and their practices regarding breast cancer. Possible barriers to seeking help were also studied.ResultThe mean number of symptoms correctly identified was 6.7 ± 3.2 (5.5 ± 3.2 for pre-clinical and 7.8 ± 2.9 for clinical students) and for risk factors it was 4.3 ± 2.1 (3.7 ± 2.1 for pre-clinical and 4.7 ± 2.2 for clinical students). The difference in the level of perception of two groups was found to be significant (p < 0.001 for symptoms and p < 0.001 for risk factors). 38.7% of the subjects responded that they check their breasts rarely, 33.1% were fairly confident while 8.6% were very confident about detecting a change in their breast, 50.0% never noticed a change in their breast, and 77.4% will contact a doctor within a week or less of finding a change in their breast. Confidence about detecting a change significantly improved (p < 0.001) after the start of clinical training.ConclusionThis study suggests that clinical training may have improved knowledge of female medical students regarding breast cancer; yet the knowledge related to the symptoms and risk factors of breast cancer and frequency of breast self-examination of female medical students is less than anticipated.

Confidence in Context: Perceived Accuracy of Quantitative Estimates Decreases With Repeated Trials

Across seven studies (combined N = 5,484), we demonstrated that confidence in one’s judgments decreases over a series of quantitative estimates. This finding was robust to various methods of confidence elicitation, the presence of incentives, and different estimation topics (Studies 1, 2, and 4). Our results also stand in contrast to participant expectations (Study 3). The phenomenon does not appear to be driven by fatigue, lack of effort, or various explanations based on incorporating uncertainty from prior judgments into subsequent ones. Our findings suggest that rather than evaluating confidence in isolation, participants evaluate confidence in reference to their stated confidence on earlier judgments. We theorize that confidence in earlier judgments increases in hindsight because of biased forgetting of disconfirming evidence. As a result, confidence in subsequent judgments appears to be comparatively lower (preregistered Studies 5–7). We discuss the implications for confidence research and consumer, organizational, and policy decision-making.

Predictors of Adults’ Metacognitive Monitoring Ability: The Roles of Task and Item Characteristics

Learners’ monitoring judgments during reading are based on a variety of cues, and the roles of task features in promoting and constraining judgment accuracy are beginning to be understood. This work examined task and item characteristics influencing adults’ monitoring of performance during reading and study tasks. In Study 1a, adults (N = 242) read an expository text, provided text ratings, completed comprehension items, and provided confidence judgments during testing. Readers’ text complexity ratings predicted their item-level confidence, while their ratings of interest and cohesion predicted their bias scores. In Study 1b, the roles of linguistic features in predicting item-level monitoring estimates were examined using the comprehension items (item n = 23) from Study 1a. Selected item features, including length, ease, density, concreteness, and meaningfulness, were examined. Item ease significantly predicted item-level bias, while ease, density, concreteness, and meaningfulness predicted item-level confidence. Study 2 (N = 68) applied similar procedures to examine the effect of item feedback on the contributions of item features to learners’ monitoring. Participants were randomly assigned to one of two item feedback conditions. Item density emerged as a significant predictor of item-level confidence, but only for those who first received item-by-item correctness feedback followed by massed feedback. Implications for future research are discussed.

How the “wisdom of the inner crowd” can boost accuracy of confidence judgments.

How the “wisdom of the inner crowd” can boost accuracy of confidence judgments.

What does physiological synchrony reveal about metacognitive experiences and group performance?

AbstractThere is a growing body of research on physiological synchrony (PS) in Collaborative Problem Solving (CPS). However, the current literature presents inconclusive findings about the way in which PS is reflected in cognitive and affective group processes and performance. In light of this, this study investigates the relationship between PS and metacognitive experiences (ie, judgement of confidence, task interest, task difficulty, mental effort and emotional valence) that are manifested during CPS. In addition, the study explores the association between PS and group performance. The participants were 77 university students who worked together on a computer‐based CPS simulation in groups of three. Participants’ electrodermal activity (EDA) was recorded as they worked on the simulation and metacognitive experiences were measured with situated self‐reports. A Multidimensional Recurrence Quantification Analysis was used to calculate the PS among the collaborators. The results show a positive relationship between continuous PS episodes and groups’ collective mental effort. No relationship was found between PS and judgement of confidence, task interest, task difficulty or emotional valence. The relationship between PS and group performance was also non‐significant. The current work addresses several challenges in utilising multimodal data analytics in CPS research and discusses future research directions.

Emotion recognition and confidence ratings predicted by vocal stimulus type and prosodic parameters

Human speech expresses emotional meaning not only through semantics, but also through certain attributes of the voice, such as pitch or loudness. In investigations of vocal emotion recognition, there is considerable variability in the types of stimuli and procedures used to examine their influence on emotion recognition. In addition, accurate metacognition was argued to promote correct and confident interpretations in emotion recognition tasks. Nevertheless, such associations have rarely been studied previously. We addressed this gap by examining the impact of vocal stimulus type and prosodic speech attributes on emotion recognition and a person’s confidence in a given response. We analysed a total of 1038 emotional expressions according to a baseline set of 13 prosodic acoustic parameters. Results showed that these parameters provided sufficient discrimination between expressions of emotional categories to permit accurate statistical classification. Emotion recognition and confidence judgments were found to depend on stimulus material as they could be reliably predicted by different constellations of acoustic features. Finally, results indicated that listeners’ accuracy and confidence judgements were significantly higher for affect bursts than speech-embedded stimuli and that the correct classification of emotional expressions elicited increased confidence judgements. Together, these findings show that vocal stimulus type and prosodic attributes of speech strongly influence emotion recognition and listeners’ confidence in these given responses.

Eyewitness confidence malleability: Misinformation as post-identification feedback.

Objective: Feedback from lineup administrators about identification accuracy significantly impacts witness confidence. In the current studies, we investigated the effect of post-identification feedback given 1-week after an initial, pristine lineup. We tested 2 kinds of feedback: typical feedback (i.e., about identification accuracy) and misinformation feedback. Misinformation feedback came in the form of suggestive questioning that falsely suggested the participant was either more or less confident in their initial identification than they actually reported. Hypotheses: We hypothesized both confirming misinformation and typical feedback would significantly inflate witness confidence relative to no feedback controls while disconfirming misinformation and typical feedback would deflate witness confidence relative to controls. Method: Across 2 studies, participants (N = 907), recruited via Amazon Mechanical Turk, watched a mock crime video, made an identification, and reported their confidence under unbiased lineup conditions. One week later, they received either confirming or disconfirming misinformation or typical feedback. They then provided a retrospective confidence judgment. Results: Misinformation feedback caused significant confidence change. Participants given false feedback that they were more confident in their initial identification than they reported later recalled greater initial confidence. Even when pristine identification conditions were used, typical confirming feedback caused participants to later remember greater confidence than they initially reported at the time of the lineup. Even in the absence of any feedback, control participants showed significant confidence inflation over time. Conclusion: These results highlight the need for lineup administrators to both ask for and document verbatim witness confidence at the time of the initial identification. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Confidence in the Real World.

Humans often assign confidence to multioption decisions, but most computational research only uses two-alternative tasks. In a new study, Li and Ma begin to reveal the mechanisms of confidence generation in multialternative tasks. This research should inspire further experiments on how humans assign confidence judgments in real-world situations.

S65. THE ‘STRESS-SENSITIVE SELF’ IN PSYCHOSIS: A PILOT STUDY AMONG HEALTHY INDIVIDUALS OF A CHALLENGE (STRESS) TEST OF SELF-RELATED NEUROCOGNITIVE CAPACITIES

BackgroundDespite attempts to predict which clinical high risk (CHR) individuals will convert to schizophrenia (SCZ), current neurocognitive tests have yielded only modest results. Stress plays a pivotal role in the inception of psychotic symptoms. Accordingly, we have recently speculated that modest predictive success may be an outcome of the discrepancy between the neutral environment in which testing occurs and the stressful environments in which SCZ outbursts. According to this view, prediction of conversion to psychosis requires ‘stress’ tests analogous to those used in general medicine, such as examining cardiac activity under physical exertion. The purpose of this pilot study was to test this notion by developing and validating a ‘Stress-sensitive Self’ neurocognitive test. The focus on the self was chosen because disturbances of the basic self, such as impaired sense of agency (SoA) are central deficits in SCZ. We examined four hypotheses: (1) subjective ratings of stress will be associated with physiological markers of stress, (2) stress will impair judgments and metacognition of SoA, (3) the effect of stress on SoA will be related to the level of attenuated psychotic symptoms (APS), and (4) based on accumulating evidence relating cardiac signals to the bodily self, accuracy in SoA will be related to cardiac deceleration.MethodsTo test these hypotheses 14 healthy controls (HC) performed the ‘Virtual Hand’ (VH) task under two conditions; a neutral block and following a psychosocial stress induction in a ‘stress’ block. In the task they perform a movement with their hand, that is hidden from them, while on a monitor a VH is presented that performs the same movement. The movement is either presented synchronously or with temporal delays. Participants judge whether the VH’s movement is identical to their actual movement (i.e. agency question) and rate their confidence (i.e. metacognition question). Heart rate (HR) was continuously recorded. Presence and distress associated with APS were assessed via the brief version of the Prodromal Questionnaire (PQ-B).ResultsConsistent with our first hypothesis, HR variability captured subjective ratings of stress, and the two were strongly correlated (r = -.637, p = .014). Inconsistent with our second hypothesis, stress did not significantly alter sensitivity (d’) in recognition of the VH (F(1,12) < 1, p = .34). Moreover, it significantly improved, rather than impaired, metacognitive monitoring (i.e., the association between confidence judgments and the actual correctness of the answers) (F(1,12) = 10.64, p =.007, η²p = 0.47). Consistent with our third hypothesis, the beneficial effect of stress on metacognitive monitoring was smaller for HCs with high levels of APS, albeit not significantly so (r = -.28, p = .26). Finally, HR deceleration during VH presentation was significantly predictive of both SoA accuracy (F(1,12) = 13.3, p =.003, η²p = 0.53), and confidence in SoA judgments (F(1,12) = 10.64, p =.002, η²p = 0.52).DiscussionThe results of this pilot study provide mixed yet promising preliminary support for the notion of a ‘Stress-sensitive Self’ test’. More specifically, they suggest that mild stress improves metacognitive monitoring judgments of self-agency, whereas an opposite trend was observed for healthy participants that reported higher scores of APS. In addition, they provide preliminary evidence for a pre-reflexive implicit physiological mechanism supporting the basic self, a mechanism that may be disturbed in SCZ. Ongoing studies are examining CHR’s performance on the ‘Stress-sensitive Self’ test and replicating the current results in a larger sample of HCs.

Confident or familiar? The role of familiarity ratings in adults’ confidence judgments when estimating fraction magnitudes

Understanding fraction magnitudes is especially important in daily life, but fraction reasoning is quite difficult. To accurately reason about fraction magnitudes, adults need to monitor what they know and what they do not know. However, little is known about which cues adults use to monitor fraction performance. Across two studies, we examined adults’ trial-by-trial fraction estimates, confidence judgments, and ratings of fraction familiarity. Adults were more confident when their estimates were more precise as well as when estimating fractions they rated as more familiar. However, adults judged their confidence in estimating fraction magnitudes, in part, based on their familiarity with each fraction. The role familiarity cues play in judgments of confidence with fractions suggests that people may be less likely to check for errors when reasoning about highly-familiar fractions.

Nonlinear neural network dynamics accounts for human confidence in a sequence of perceptual decisions

Electrophysiological recordings during perceptual decision tasks in monkeys suggest that the degree of confidence in a decision is based on a simple neural signal produced by the neural decision process. Attractor neural networks provide an appropriate biophysical modeling framework, and account for the experimental results very well. However, it remains unclear whether attractor neural networks can account for confidence reports in humans. We present the results from an experiment in which participants are asked to perform an orientation discrimination task, followed by a confidence judgment. Here we show that an attractor neural network model quantitatively reproduces, for each participant, the relations between accuracy, response times and confidence. We show that the attractor neural network also accounts for confidence-specific sequential effects observed in the experiment (participants are faster on trials following high confidence trials). Remarkably, this is obtained as an inevitable outcome of the network dynamics, without any feedback specific to the previous decision (that would result in, e.g., a change in the model parameters before the onset of the next trial). Our results thus suggest that a metacognitive process such as confidence in one’s decision is linked to the intrinsically nonlinear dynamics of the decision-making neural network.

Interplay of metacognitive experiences and performance in collaborative problem solving

Metacognitive experiences are the feelings and judgments that emerge in relation to an ongoing learning task. Much of the work on metacognitive experiences has studied these constructs piecemeal and mostly in individual learning settings. Thus, little is known about how metacognitive experiences co-occur in social learning settings. In light of this, we investigated the relationships between metacognitive experiences and their impact on perceived and objective task performance in a collaborative problem solving (CPS) simulation. Seventy-seven higher education students participated in the study. Metacognitive experiences (judgment of confidence, mental effort, task difficulty, task interest, and emotional valence) were measured with self-reports at multiple time points during CPS. A path analysis was conducted to investigate the relationship between metacognitive experiences and perceived performance. A generalized estimating equation was used to observe the relationships between metacognitive experiences and objective group CPS performance. Overall, the findings indicate complex relationships among the metacognitive experiences and performance in CPS and further highlight the social characteristics of metacognition.

Priors and payoffs in confidence judgments.

Priors and payoffs are known to affect perceptual decision-making, but little is understood about how they influence confidence judgments. For optimal perceptual decision-making, both priors and payoffs should be considered when selecting a response. However, for confidence to reflect the probability of being correct in a perceptual decision, priors should affect confidence but payoffs should not. To experimentally test whether human observers follow this normative behavior for natural confidence judgments, we conducted an orientation-discrimination task with varied priors and payoffs that probed both perceptual and metacognitive decision-making. The placement of discrimination and confidence criteria were examined according to several plausible Signal Detection Theory models. In the normative model, observers use the optimal discrimination criterion (i.e., the criterion that maximizes expected gain) and confidence criteria that shift with the discrimination criterion that maximizes accuracy (i.e., are not affected by payoffs). No observer was consistent with this model, with the majority exhibiting non-normative confidence behavior. One subset of observers ignored both priors and payoffs for confidence, always fixing the confidence criteria around the neutral discrimination criterion. The other group of observers incorrectly incorporated payoffs into their confidence by always shifting their confidence criteria with the same gains-maximizing criterion used for discrimination. Such metacognitive mistakes could have negative consequences outside the laboratory setting, particularly when priors or payoffs are not matched for all the possible decision alternatives.

Effects of induced levels of prior knowledge on monitoring accuracy and performance when learning from self-regulated problem solving.

Both accurate monitoring and adequate control are crucial for effective self-regulation when learning from problem-solving tasks. Prior research has shown that self-regulated learning is especially harmful for low prior knowledge students, given their difficulties with accurate monitoring and control decisions. Although many studies have indicated that prior knowledge facilitates monitoring accuracy, none of these studies experimentally manipulated students’ prior knowledge. The purpose of the current study was to investigate whether experimentally induced levels of prior knowledge affect the accuracy of metacognitive judgments (i.e., ease-of-learning judgments [EOLs], and retrospective confidence judgments [RCJs]) when learning from problem-solving tasks, and whether this in turn moderates the effect of learner control on performance. To manipulate prior knowledge, we compared conditions in which undergraduate students (N = 165) watched either a video about Mendel’s biography (low prior knowledge) or a video about Mendel’s laws of heredity (high prior knowledge). After that, we presented students with a set of problem-solving tasks and manipulated the type of learner control between conditions (i.e., full control vs. restricted control over the selection of tasks). Results indicated that students with high induced prior knowledge were more accurate in their EOLs and RCJs, and achieved higher performance than students with low prior knowledge. However, monitoring accuracy did not moderate the effects of different types of learner control on performance, even though students used their RCJs as a basis for the task selections. These results suggest that monitoring accuracy is a prerequisite for successful self-regulated problem solving, but it is not sufficient. (PsycInfo Database Record (c) 2020 APA, all rights reserved)

The Impact of Cooperation and Competition on Metacognitive Monitoring in Classroom Context

Metacognitive monitoring skills are crucial for middle school students to improve academic performance and promote self-regulation. The current study examined the effect of social interaction on metacognitive monitoring training assessed by calibration accuracy measures and performance on comprehension items. Sixth-grade students (N = 84) assigned to competitive, cooperative, and individual conditions completed multiple-choice comprehension items and item-level confidence judgments for expository and narrative reading passages in eight sessions distributed over six weeks. Results indicated no improvement in monitoring accuracy due to classroom conditions. In contrast, overconfidence increased for students in the competitive condition across the intervention. However, students in the cooperative condition demonstrated higher comprehension on the final training passage relative to students in the other two conditions. In addition, text interest was positively related to students’ judgment accuracy in the cooperative and individual conditions but not in the competitive condition. This study underscores the importance of social context with regard to metacognitive processes and performance outcomes during classroom instruction.

Response-Related Signals Increase Confidence But Not Metacognitive Performance.

Confidence judgments are a central tool in metacognition research. In a typical task, participants first perform perceptual (first-order) decisions and then rate their confidence in these decisions. The relationship between confidence and first-order accuracy is taken as a measure of metacognitive performance. Confidence is often assumed to stem from decision-monitoring processes alone, but processes that co-occur with the first-order decision may also play a role in confidence formation. In fact, some recent studies have revealed that directly manipulating motor regions in the brain, or the time of first-order decisions relative to second-order decisions, affects confidence judgments. This finding suggests that confidence could be informed by a readout of reaction times in addition to decision-monitoring processes. To test this possibility, we assessed the contribution of response-related signals to confidence and, in particular, to metacognitive performance (i.e., a measure of the adequacy of these confidence judgments). In human volunteers, we measured the effect of making an overt (vs covert) decision, as well as the effect of pairing an action to the stimulus about which the first-order decision is made. Against our expectations, we found no differences in overall confidence or metacognitive performance when first-order responses were covert as opposed to overt. Further, actions paired to visual stimuli presented led to higher confidence ratings, but did not affect metacognitive performance. These results suggest that confidence ratings do not always incorporate motor information.

The role of meta‐cognitive cues on the comprehension of proficient and poor readers

We explored whether performance differences exist between proficient and poor readers on implicit text information. Next, we explored whether indices of meta‐cognitive monitoring predicted reading performance. Finally, we examined whether poor and proficient readers exhibited distinct meta‐cognitive profiles with respect to reading comprehension ability. Chilean undergraduate students (N = 146) completed a task on inconsistency detection within texts and a standardised reading comprehension performance measure, which we used, along with confidence in performance judgements, to calculate meta‐cognitive monitoring accuracy. Our results confirmed that proficient readers outperformed poor readers on nearly all measures of interest, except global retrospective meta‐cognitive monitoring judgements, and that proficient readers performed significantly better on items related to implicit information of texts than poor readers. Additionally, when combined in a single group, number of inconsistencies correctly detected and repaired and accurate global evaluation of learning judgements significantly predicted reading performance whereas retrospective global and local meta‐cognitive monitoring judgements did not. Of special significance to our investigation, when separated in two groups, poor and proficient readers exhibited unique meta‐cognitive profiles. Proficient and poor readers employ different meta‐cognitive strategies, and poor readers benefit more from strategies than proficient readers.

Confidence controls perceptual evidence accumulation

Perceptual decisions are accompanied by feelings of confidence that reflect the likelihood that the decision was correct. Here we aim to clarify the relationship between perception and confidence by studying the same perceptual task across three different confidence contexts. Human observers were asked to categorize the source of sequentially presented visual stimuli. Each additional stimulus provided evidence for making more accurate perceptual decisions, and better confidence judgements. We show that observers’ ability to set appropriate evidence accumulation bounds for perceptual decisions is strongly predictive of their ability to make accurate confidence judgements. When observers were not permitted to control their exposure to evidence, they imposed covert bounds on their perceptual decisions but not on their confidence decisions. This partial dissociation between decision processes is reflected in behaviour and pupil dilation. Together, these findings suggest a confidence-regulated accumulation-to-bound process that controls perceptual decision-making even in the absence of explicit speed-accuracy trade-offs.