Confidence Judgments Research Articles

Epistemic modality in American presidential discourse

This research examines the exploitation of epistemic modality markers in political discourse. This research inspects modal auxiliaries and semi-modals use in four speeches addressed by Obama at the annual United Nations General Assembly during his second period as a President of America. Applying both qualitative and quantitative methods, this research is in an attempt to accomplish the whole investigation dealing with explorative and quantification of epistemic modality in Obama’s political discourse. The data were taken from Obama’s speeches from 2013 to 2016 at the United Nations General Assembly. The findings demonstrate that 471 modality markers were found in Obama’s speeches and he frequently delivered epistemic probability with 189 cases or 40.2% modal auxiliaries and semi-modals. Epistemic certainty and possibility were found in 125 cases or 26.6% and in 157 cases or 33.2%. The highest degree of epistemic modality, epistemic certainty, is expressed by employing must, have to, need to, cannot, could not, and may not. Modal should, will, would, be going to, and ought to express epistemic probability and modal can, could, may, might, and be able to are exploited to express the lowest degree of epistemic modality, epistemic possibility. The higher epistemic modality markers involved in a proposition indicate the higher confidence of evaluation and judgment asserted based on the speaker’s knowledge, belief, and evidence. In contrast, the lower epistemic modality markers found indicate lower confidence in the evaluation and judgment of the proposition.

The Relation Between Perceived Mental Effort, Monitoring Judgments, and Learning Outcomes: A Meta-Analysis

Accurately monitoring one’s learning processes during self-regulated learning depends on using the right cues, one of which could be perceived mental effort. A meta-analysis by Baars et al. (2020) found a negative association between mental effort and monitoring judgments (r = -.35), suggesting that the amount of mental effort experienced during a learning task is usually negatively correlated with learners’ perception of learning. However, it is unclear how monitoring judgments and perceptions of mental effort relate to learning outcomes. To examine if perceived mental effort is a diagnostic cue for learning outcomes, and whether monitoring judgments mediate this relationship, we employed a meta-analytic structural equation model. Results indicated a negative, moderate association between perceived mental effort and monitoring judgments (β = -.19), a positive, large association between monitoring judgments and learning outcomes (β = .29), and a negative, moderate indirect association between perceived mental effort and learning outcomes (β = -.05), which was mediated by monitoring judgments. Our subgroup analysis did not reveal any significant differences across moderators potentially due to the limited number of studies included per moderator category. Findings suggest that when learners perceive higher levels of mental effort, they exhibit lower learning (confidence) judgments, which relates to lower actual learning outcomes. Thus, learners seem to use perceived mental effort as a cue to judge their learning while perceived mental effort only indirectly relates to actual learning outcomes.

Confidence judgments interfere with perceptual decision making

Determining one’s confidence in a decision is a vital part of decision-making. Traditionally, psychological experiments have assessed a person’s confidence by eliciting confidence judgments. The notion that such judgments can be elicited without impacting the accuracy of the decision has recently been challenged by several studies which have shown reactivity effects—either an increase or decrease in decision accuracy when confidence judgments are elicited. Evidence for the direction of reactivity effects has, however, been decidedly mixed. Here, we report three studies designed to specifically make reactivity effects more prominent by eliciting confidence judgment contemporaneously with perceptual decisions. We show that confidence judgments elicited contemporaneously produce an impairment in decision accuracy, this suggests that confidence judgments may rely on a partially distinct set of cues/evidence than the primary perceptual decision and, additionally, challenges the continued use of confidence ratings as an unobtrusive measure of metacognition.

Role of Metacognitive Confidence Judgments in Curiosity: Different Effects of Confidence on Curiosity Across Epistemic and Perceptual Domains.

Previous research suggests that curiosity is sometimes induced by novel information one has no relevant knowledge about, but it is sometimes induced by new information about something that one is familiar with and has prior knowledge about. However, the conditions under which novelty or familiarity triggers curiosity remain unclear. Using metacognitive confidence judgments as a proxy to quantify the amount of knowledge, this study evaluates the relationship between the amount of relevant knowledge and curiosity. We reviewed previous studies on the relationship between subjective curiosity and confidence and reanalyzed existing large-sample data. The findings indicate that the relationship between curiosity and confidence differs depending on the nature of the stimuli: epistemic versus perceptual. Regarding perceptual stimuli, curiosity is stronger when individuals have lower confidence levels. By contrast, for epistemic stimuli, curiosity is stronger when individuals have higher confidence levels. These results suggest that curiosity triggered by perceptual stimuli is based on perceived novelty, whereas that triggered by epistemic stimuli is based on familiarity with prior knowledge.

ADHD and Film Watching: Unveiling the Impact on Attention, Confidence, and Cinematic Experience

ADHD, a neurodevelopmental condition impacting millions globally, is characterized by symptoms such as inattention, hyperactivity, and impulsivity. While existing research has investigated ADHD in various contexts, this study sheds light on how individuals with ADHD perceive and engage with cinematic narratives, specifically examining overconfidence. Individuals overestimate their abilities and impulse activities on this cognitive bias, like skipping through videos. By examining the mental and emotional processes involved in comprehension among those with ADHD, this paper seeks to understand how overconfidence influences their interpretation of cinematic stories and characters. The literature review discusses the intricate connections between ADHD, attention deficits, impulsivity, and self-perception. ADHD patients tend to have deficits in attention span and self-perception, indicating a Positive Illusory Bias (PIB) where individuals overestimate their abilities. For the study, 300 participants were exposed to a video and assessed using the CAARS scale to measure ADHD symptomatology levels, perceived likeability of characters, confidence in judgments, and time spent watching the video. Results indicate a negative correlation between ADHD symptomatology and video watch time, with higher ADHD levels associated with earlier self-truncation. Mediation analysis suggests that participants' confidence in judgments mediates this relationship, with higher ADHD symptomatology linked to greater confidence and decreased video watch durations. The findings contribute to understanding how individuals with ADHD perceive and interact with cinematic storytelling, emphasizing the need for empathy and awareness in accommodating their unique perspectives and tendencies.

Knowing that you know that you know? An extreme-confidence heuristic can lead to above-chance discrimination of metacognitive performance.

In daily life, we can not only estimate confidence in our inferences ('I'm sure I failed that exam'), but can also estimate whether those feelings of confidence are good predictors of decision accuracy ('I feel sure I failed, but my feeling is probably wrong; I probably passed'). In the lab, by using simple perceptual tasks and collecting trial-by-trial confidence ratings visual metacognition research has repeatedly shown that participants can successfully predict the accuracy of their perceptual choices. Can participants also successfully evaluate 'confidence in confidence' in these tasks? This is the question addressed in this study. Participants performed a simple, two-interval forced choice numerosity task framed as an exam. Confidence judgements were collected in the form of a 'predicted exam grade'. Finally, we collected 'meta-metacognitive' reports in a two-interval forced-choice design: trials were presented in pairs, and participants had to select that in which they thought their confidence (predicted grade) best matched their accuracy (actual grade), effectively minimizing their quadratic scoring rule (QSR) score. Participants successfully selected trials on which their metacognition was better when metacognitive performance was quantified using area under the type 2 ROC (AUROC2) but not when using the 'gold-standard' measure m-ratio. However, further analyses suggested that participants selected trials on which AUROC2 is lower in part via an extreme-confidence heuristic, rather than through explicit evaluation of metacognitive inferences: when restricting analyses to trials on which participants gave the same confidence rating AUROC2 no longer differed as a function of selection, and likewise when we excluded trials on which extreme confidence ratings were given. Together, our results show that participants are able to make effective metacognitive discriminations on their visual confidence ratings, but that explicit 'meta-metacognitive' processes may not be required.

What the eyes, confidence, and partner's identity can tell about change of mind.

Perceptual confidence reflects the ability to evaluate the evidence that supports perceptual decisions. It is thought to play a critical role in guiding decision-making. However, only a few empirical studies have actually investigated the function of perceptual confidence. To address this issue, we designed a perceptual task in which participants provided a confidence judgment on the accuracy of their perceptual decision. Then, they viewed the response of a machine or human partner, and they were instructed to decide whether to keep or change their initial response. We observed that confidence predicted participants' changes of mind more than task difficulty and perceptual accuracy. Additionally, interacting with a machine, compared to a human, decreased confidence and increased participants tendency to change their initial decision, suggesting that both confidence and changes of mind are influenced by contextual factors, such as the identity of a partner. Finally, variations in confidence judgments but not change of mind were correlated with pre-response pupil dynamics, indicating that arousal changes are linked to confidence computations. This study contributes to our understanding of the factors influencing confidence and changes of mind and also evaluates the possibility of using pupil dynamics as a proxy of confidence.

Exploring Motivators for Trust in the Dichotomy of Human—AI Trust Dynamics

This study analyses the dimensions of trust in artificial intelligence (AI), focusing on why a significant portion of the UK population demonstrates a higher level of trust in AI compared to humans. Conducted through a mixed-methods approach, this study gathered 894 responses, with 451 meeting the criteria for analysis. It utilised a combination of a six-step Likert-scale survey and open-ended questions to explore the psychological, sociocultural, and technological facets of trust. The analysis was underpinned by structural equation modelling (SEM) and correlation techniques. The results unveil a strong predilection for trusting AI, mainly due to its perceived impartiality and accuracy, which participants likened to conventional computing systems. This preference starkly contrasts with the scepticism towards human reliability, which is influenced by the perception of inherent self-interest and dishonesty in humans, further exacerbated by a general distrust in media narratives. Additionally, this study highlights a significant correlation between distrust in AI and an unwavering confidence in human judgment, illustrating a dichotomy in trust orientations. This investigation illuminates the complex dynamics of trust in the era of digital technology, making a significant contribution to the ongoing discourse on AI’s societal integration and underscoring vital considerations for future AI development and policymaking.

Biased memory retrieval in the service of shared reality with an audience: The role of cognitive accessibility.

After communicators have tuned a message about a target person's behaviors to their audience's attitude, their recall of the target's behaviors is often evaluatively consistent with their audience's attitude. This audience-congruent recall bias has been explained as the result of the communicators' creation of a shared reality with the audience, which helps communicators to achieve epistemic needs for confident judgments and knowledge. Drawing on the "Relevance Of A Representation" (ROAR) model of cognitive accessibility from motivational truth relevance, we argue that shared reality increases the accessibility of information consistent (vs. inconsistent) with the audience's attitude. We tested this prediction with a novel reaction time task in three experiments employing the saying-is-believing paradigm. Faster reactions to audience-consistent (vs. audience-inconsistent) information were found for trait information but not for behavioral information. Thus, an audience-congruent accessibility bias emerged at the level at which impressions and judgments of other persons are typically organized. Consistent with a shared-reality account, the audience-consistent accessibility bias correlated with experienced shared reality with the audience about the target person and with epistemic trust in the audience. These findings support the view that the creation of shared reality with an audience triggers a basic cognitive mechanism that facilitates the retrieval of audience-congruent (vs. audience-incongruent) trait information about a target person. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The developmental path of metacognition from toddlerhood to early childhood and its influence on later memory performance.

Recently, several studies have suggested that metacognition emerges early in infancy and toddlerhood. However, to date, the developmental trajectory of these early metacognitive monitoring and control processes and their influence on children's later memory functioning remains poorly understood. The aim of this study was to longitudinally document the development of metacognition between the ages of 2.5 and 4.5 years and to examine the link between these early metacognitive skills and later memory performance. To do so, 69 children initially aged 29-33 months old (NT0 = 69; 32 females; Mage = 32.3 months; SD = 1.6) were tested at three time points (12-month intervals) with a recognition memory paradigm designed to assess both metacognitive monitoring, through retrospective confidence judgment, and metacognitive control, through a cue selection task (i.e., children had the opportunity to ask for a cue to help them change their memory decision). In addition, at the last session, an episodic memory task (story recall) was also administered. Our results revealed an improvement in monitoring and control processes between 2.5 and 4.5 years with above-chance performance from around age 3.5. Mixed-effects modeling also indicated that metacognitive monitoring at ages 2.5 and 4.5, but not-unexpectedly-metacognitive control, was related to children's memory performance at age 4.5. Overall, our results provide evidence to enhance our understanding of the developmental course of metacognition from toddlerhood to early childhood and suggest that metacognitive processes are involved in memory performance much earlier than had previously been shown. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Primary school students’ awareness of their monitoring and regulation judgment accuracy

BackgroundImproving students' monitoring and regulation judgment accuracy is necessary for improving the effectiveness of self-regulated learning, but might not be sufficient: Students presumably also need to feel confident about the accuracy of their judgments to act upon them. However, little is known about students’ awareness of their monitoring judgment accuracy, and awareness of their regulation judgment accuracy has not yet been investigated. AimsWe investigated (1) primary school students’ awareness of their monitoring and regulation judgment accuracy in mathematics and (2) whether self-scoring, which is known to improve monitoring/regulation accuracy, would also improve awareness of their regulation judgment accuracy. Sample(s)Primary school students (9-10 year-olds) from 34 classes (N = 564). MethodsStudents completed problem-solving tasks twice (parallel versions) on two different days and made monitoring/regulation judgments, rated their confidence in the accuracy of those judgments, self-scored their work, and again made (confidence) judgments, on both occasions. If an increase/decrease in judgment accuracy from day 1 to day 2 would be accompanied by an increase/decrease in their confidence in the accuracy of their judgements, students show accuracy awareness. ResultsStudents' judgment accuracy did not predict their confidence in the accuracy of their judgments, indicating that students were not aware of their monitoring/regulation accuracy. Self-scoring improved students’ awareness of their regulation judgment accuracy for students whose regulation judgment accuracy increased or stayed maximally accurate after self-scoring, but not for students whose regulation judgment accuracy decreased or stayed equally inaccurate after self-scoring. ConclusionsPrimary school students were not aware of their monitoring/regulation judgment accuracy. Self-scoring improved the awareness of their regulation judgment accuracy for some students.

Confidence in Covid-19 models

Epidemiological models of the transmission of SARS-CoV-2 played an important role in guiding the decisions of policy-makers during the pandemic. Such models provide output projections, in the form of time -series of infections, hospitalisations, and deaths, under various different parameter and scenario assumptions. In this paper I caution against handling these outputs uncritically: raw model-outputs should not be presented as direct projections in contexts where modelling results are required to support policy -decisions. I argue that model uncertainty should be handled and communicated transparently. Drawing on methods used by climate scientists in the fifth IPCC report I suggest that this can be done by: attaching confidence judgements to projections based on model results; being transparent about how multi-model ensembles are supposed to deal with such uncertainty; and using expert judgement to ‘translate’ model-outputs into projections about the actual world. In a slogan: tell me what you think (and why), not (just) what your models say. I then diffuse the worry that this approach infects model-based policy advice with some undesirably subjective elements, and explore how my discussion fares if one thinks the role of a scientific advisor is to prompt action, rather than communicate information.

Clinicians’ approach to predicting post-cardiac arrest outcomes for patients enrolled in a United States clinical trial

PurposePerceived poor prognosis can lead to withdrawal of life-sustaining therapies (WLST) in patients who might otherwise recover. We characterized clinicians’ approach to post-arrest prognostication in a multicenter clinical trial. MethodsSemi-structured interviews were conducted with clinicians who treated a comatose post-cardiac arrest patient enrolled in the Influence of Cooling Duration on Efficacy in Cardiac Arrest Patients (ICECAP) trial (NCT04217551). Two authors independently analyzed each interview using inductive and deductive coding. The clinician reported how they arrived at a prognosis for the specific patient. We summarized the frequency with which clinicians reported using objective diagnostics to formulate their prognosis, and compared the reported approaches to established guidelines. Each respondent provided demographic information and described local neuroprognostication practices. ResultsWe interviewed 30 clinicians at 19 US hospitals. Most claimed adherence to local hospital neuroprognostication protocols (n = 19). Prognostication led to WLST for perceived poor neurological prognosis in 15/30 patients, of whom most showed inconsistencies with guidelines or trial recommendations, respectively. In 10/15 WLST cases, clinicians reported relying on multimodal testing. A prevalent theme was the use of “clinical gestalt,” defined as prognosticating based on a patient's overall appearance or a subjective impression in the absence of objective data. Many clinicians (21/30) reported using clinical gestalt for initial prognostication, with 9/21 expressing high confidence initially. ConclusionClinicians in our study state they follow neuroprognostication guidelines in general but often do not do so in actual practice. They reported clinical gestalt frequently informed early, highly confident prognostic judgments, and few objective tests changed initial impressions. Subjective prognostication may undermine well-designed trials.

The illusory certainty: Information repetition and impressions of truth enhance subjective confidence in validity judgments independently of the factual truth

People not only judge repeatedly perceived information as more likely being true (the so-called truth effect) they also tend to be more confident after judging the validity of repeated information. These phenomena are assumed to be caused by a higher subjective feeling of ease (i.e., fluency) when processing repeated (vs. new) information. Based on the suggestion that a higher number of coherent mental activations is promoting a fluency experience, we argue that besides repetition an already existing information network, that is (nonspecific) prior knowledge, can enhance fluency. Following this argumentation, information repetition as well as the act of judging incoming information as being true (vs. false) should feed into subjective confidence – independently of the factual truth (when judging under uncertainty). To test this, we reanalyzed two published data sets and conducted a new study. In total, participants (N = 247) gave 29,490 truth judgments and corresponding ratings of subjective confidence while attending two judgement phases (i.e., 10 min and 1 week after the exposure phase in each experiment). Results showed that (a) repetition (in 3 of 3 data sets) and (b) impressions of truth (in 2 of 3 data sets) were systematically related to higher subjective confidence. Moreover, we found (c) a significant positive interaction between repetition and impressions of truth after both intervals in all data sets. Our analyses further underline the moderating effect of time: Influences of repetition significantly decreased with increasing time interval. Notably, the factual truth did not systematically affect any of the above reported effects.

Metacognitive accuracy predicts self-reported quality of life following traumatic brain injury

ABSTRACT Objective Metacognition and quality of life (QoL) are both adversely affected by traumatic brain injury (TBI), but the relation between them is not fully understood. As such, the purpose of this study was to determine the degree to which metacognitive accuracy predicts QoL in individuals with TBI. Methods Eighteen participants with moderate-to-severe TBI completed a stimulus-response task requiring the discrimination of emotions depicted in pictures of faces and then provided a retrospective confidence judgment after each response. Metacognitive accuracy was calculated using participants’ response accuracy and confidence judgment accuracy. Participants also completed the Quality of Life After Brain Injury (QOLIBRI) questionnaire to assess QoL in various areas of functioning. Results Performance of a linear regression analysis revealed that higher metacognitive accuracy significantly predicted lower overall QoL. Additionally, higher metacognitive accuracy significantly predicted lower QoL related to cognition and physical limitations. Conclusion The study results provide evidence of an inverse relation between metacognitive performance and QoL following TBI. Metacognitive changes associated with TBI and their relation to QoL have several clinical implications for TBI rehabilitation.

Understanding the social-emotional components of our "number sense": insights from a novel non-symbolic numerical comparison task.

A large body of work has identified a core sense of number supported by the Approximate Number System (ANS) that is present in infancy and across species. Although it is commonly assumed that the ANS directly processes perceptual input and is relatively independent from affective factors, some evidence points at a correlation between ANS performance and math anxiety. However, the evidence is mixed. We tested whether giving participants active control in completing a numerical task would change the relationship between math anxiety on performance. Adult participants (N = 103) completed a novel four-alternative-forced-choice non-symbolic numerical comparison task. In a repeated-measures design, participants either passively viewed different dot arrays or actively chose to view each array (i.e., active information-seeking) before deciding on the largest quantity. Participants also provided confidence judgments during the passive version of the task. We replicated the ratio-dependent signature in participants' accuracy in both the passive and active versions of the task using this novel paradigm, as well as in trial-level confidence judgments and information-seeking behavior. Participants' self-reported math anxiety significantly correlated with their accuracy on the passive version of the task. Critically, the correlation disappeared in the active version of the task. Gender also emerged as a predictor of confidence judgments and a moderator of the effect of task on overall accuracy and the effect of active information seeking on accuracy in the active version of the task. Exploratory analysis of estimated Weber Fraction suggests that these results may be driven by auxiliary factors instead of changes in ANS acuity. These findings have implications for understanding the relationship between math anxiety and performance on numerical tasks.

The autocorrelated Bayesian sampler: A rational process for probability judgments, estimates, confidence intervals, choices, confidence judgments, and response times.

Normative models of decision-making that optimally transform noisy (sensory) information into categorical decisions qualitatively mismatch human behavior. Indeed, leading computational models have only achieved high empirical corroboration by adding task-specific assumptions that deviate from normative principles. In response, we offer a Bayesian approach that implicitly produces a posterior distribution of possible answers (hypotheses) in response to sensory information. But we assume that the brain has no direct access to this posterior, but can only sample hypotheses according to their posterior probabilities. Accordingly, we argue that the primary problem of normative concern in decision-making is integrating stochastic hypotheses, rather than stochastic sensory information, to make categorical decisions. This implies that human response variability arises mainly from posterior sampling rather than sensory noise. Because human hypothesis generation is serially correlated, hypothesis samples will be autocorrelated. Guided by this new problem formulation, we develop a new process, the Autocorrelated Bayesian Sampler (ABS), which grounds autocorrelated hypothesis generation in a sophisticated sampling algorithm. The ABS provides a single mechanism that qualitatively explains many empirical effects of probability judgments, estimates, confidence intervals, choice, confidence judgments, response times, and their relationships. Our analysis demonstrates the unifying power of a perspective shift in the exploration of normative models. It also exemplifies the proposal that the "Bayesian brain" operates using samples not probabilities, and that variability in human behavior may primarily reflect computational rather than sensory noise. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Sensitivity to speech-relevant features in hallucination-prone individuals

As hallucinations occur in the absence of an external stimulus, they constitute an intriguing model for how percepts are generated and for how perception can fail. This study explores whether hallucination proneness is linked to an altered perception of speech-related acoustic features. Involving 320 healthy adults with varying predispositions for hallucinations, participants evaluated ambiguous sound textures for their speech-likeness. Psychophysical reverse correlation revealed that higher hallucination proneness was associated with reduced weighting of speech-typical low-frequency acoustic energy. Temporal modulation discrimination capabilities were unrelated to hallucination proneness in a subset of 41 participants. Confidence judgments in individual trials were influenced by both acoustic evidence and individual hallucination proneness and schizotypy scores. Overall, these findings suggest that hallucination-prone individuals exhibit qualitative and quantitative changes in their perception of speech-relevant modulations, supporting the notion of altered perceptual priors and differential weighting of sensory evidence.

Short-term retention of popular music in older adults: Support for a plasticity theory of implicit music knowledge acquisition

Based on our Plasticity Theory of Implicit Music Knowledge Acquisition (PTIMKA), we tested the hypothesis that adolescence is a sensitive period for acquiring musical information and lifelong musical grammar. Older adults (N = 27, mean age = 65.7 years; SD = 6.7) identified artist, title, and year of popularity and rated their familiarity for short excerpts of 36 songs popular between 1962 and 2021. Knowledge and familiarity were greater for music popular during the participants’ adolescence. A subsequent surprise retention task required participants to choose which of 2 excerpts had been presented in the first task; for each of 36 trials, targets and foils represented the same era of popularity. Retention (dꞌ) scores and confidence in retention judgment were higher for the songs popular during adolescence, even though targets of all eras had just been presented in the previous 15 minutes. It is argued that popular music styles congruent with an adolescence-established grammar were more accurately encoded than styles violating this grammar, as would songs popular before or after adolescence. Prior data from younger adults showing trajectories opposite to those for older adults as a function of decade of popularity are further consistent with this interpretation and with PTIMKA. (Work supported by NSERC)

When competence and confidence are at odds: a cross-country examination of the Dunning–Kruger effect

Research has shown that some students who underperform in mathematics overestimate their performance, while others who excel in mathematics underestimate it. Looking at this mismatch of performance and confidence judgement—the Dunning–Kruger effect (DKE)—the current study investigates how well students’ confidence judgement and item-specific mathematics competence relate with each other and whether such a relationship differs across six European countries (i.e., Norway, Sweden, Finland, Estonia, Serbia and Portugal). We also examine whether perceived competence, mathematics identity, gender, socioeconomic status and immigration background predict this mismatch and whether these demographic factors function differently between the examined countries. The results show that the DKE could be found across grades three and four in all six countries. However, there are country-specific patterns regarding the relationship between performance, mathematics identity and perceived competence; the DKE; and how different demographic variables predict its occurrences in particular subpopulations.