Confidence Judgments Research Articles

Retrospective judgments of confidence in a complex span task

Although the study of metamemory monitoring originated in predictions for simple span tasks, the study of metacognition for working memory (WM) has been somewhat neglected in comparison with long-term memory. We aimed to fill this gap by exploring the ability to self-assess WM operations. Thirty-four participants performed 16 series of complex span tasks and rated their confidence in a verbal recall paradigm. We manipulated the cognitive load based on the TBRS model in order to analyze the role of attentional resources on both WM and metacognitive evaluations. As expected, we found that recall is affected by cognitive load and we found standard serial position effects. Interestingly, metacognitive evaluations followed the same pattern, and measures of metacognitive sensitivity suggest that participants are able to make item-by-item retrospective judgments reflective of their performance. We discuss how these results contribute to our understanding of metacognitive access to newly-formed WM contents.

Comparative Learning

AbstractThis article concerns the diachronic rationality norms for comparative confidence judgments, that is, judgments of the form “I am at least as confident in $p$ as I am in $q$.” Specifically, it identifies, characterizes, and evaluates an intuitively compelling learning rule called comparative conditionalization that specifies how agents should revise their comparative confidence judgments in the face of novel evidence.

Differences is students' performance, (mis)calibration, and confidence judgments depending on their preferred sitting position in class

Students who perform better academically are generally more accurate in assessing their performance. In other words, they are better calibrated, showing smaller discrepancy between their predicted and actual performance. Research into contextual variables related to performance has shown that students sitting position may be related to their performance in class. In the current study we attempted to bring several lines of research together and examine whether there are differences in academic performance, calibration, and confidence judgments among students who choose various sitting positions in the classroom.130 college students (M = 21.52, SD = 1.59; 103 female) attending courses at the University of Jaen in Spain participated. Results revealed that students' calibration differed depending on their preferred sitting position. We found that students in middle rows were optimally calibrated, students in the front rows overestimated their performance, and those in the back rows tended to underestimate their performance. Furthermore, students in the front rows were the most confident in their estimates, whereas students in the middle and back rows were less confident, suggesting the manifestation of the Dunning-Kruger effect. Future directions for contextual variables contributing to students' optimal calibration are discussed.

Ontology of guided development of scientific abilities of young people. Part 1: basic concepts and models

The article attempts to analyze new trends in work with gifted youth as a new form of systemic, organizational and scientifically oriented independent activity of a motivated schoolchild or student. This activity, firstly, is aimed at satisfying the cognitive, intellectual, as well as other needs of the student and the effective development of their creative potential; secondly, it is associated with the consistent solution of research problems with not foreknown results, increasing novelty, relevance and complexity; thirdly, it is supported by a specially organized unified developing scientific and educational environment within the framework of its educational organization, region, industry, country; and finally, it involves a scientifically based unified assessment of both the developing effect and the scientific significance of the current results of this activity . The article discusses the basic concepts characterizing this activity and illustrating their examples and mathematical models. These include the author's structuring of the functions and levels of research activity, a multifaceted system of criteria for evaluating the scientific and developmental results of research work performed by young researchers, a functional profile, a functional rating and a professional functional rating of a young researcher, calculated based on the assessment results of their research work. For the first time, a mathematical model for the formation of acceptable options for research activities is proposed, which allows forming the optimal developing methodological structure of planned research. One of the unique points of the proposed approach is the widespread use of ordinal rating scales with the transformation of the results of their application into quantitative estimates based on the method of confident judgments of decision makers. The introduced concepts are formed into a graphical scheme of the ontology of guided development of scientific abilities of young people.

Metacognition and sense of agency

Intelligent agents need to understand how they can change the world, and how they cannot change it, in order to make rational decisions for their forthcoming actions, and to adapt to their current environment. Previous research on the sense of agency, based largely on subjective ratings, failed to dissociate the sensitivity of sense of agency (i.e., the extent to which individual sense of agency tracks actual instrumental control over external events) from judgment criteria (i.e., the extent to which individuals self-attribute agency independent of their actual influence over external events). Furthermore, few studies have examined whether individuals have metacognitive access to the internal processes underlying the sense of agency. We developed a novel two-alternative-forced choice (2FAC) control detection task, in which participants identified which of two visual objects was more strongly controlled by their voluntary movement. The actual level of control over the target object was manipulated by adjusting the proportion of its motion that was driven by the participant's movement, compared to the proportion driven by a pre-recorded movement by another agent, using a staircase to hold 2AFC control detection accuracy at 70%. Participants identified which of the two visual objects they controlled, and also made a binary confidence judgment regarding their control detection judgment. We calculated a bias-free measure of first-order sensitivity (d’) for detection control at any given level of participant's own movement. The proportion of pre-recorded movements determined by the stairecase could then be used as an index of control detection ability. We identified two distinct processes underlying first-order detection of control: one based on instantaneous sensory predictions for the current movement, and one based on detection of a regular motor-visual relation across a series of movements. Further, we found large individual differences across 40 particpants in metacognitive sensitivity (meta-d’) even though first-order sensitivity of control detection was well controlled. Using structural equation modelling (SEM), we showed that metacognition was negatively correlated with the predictive process component of detection of control. This result is inconsistent with previous hypotheses that detection of control relies on metacognitive monitoring of a predictive circuit. Instead, it suggests that predictive mechanisms that compute sense of agency may operate unconsciously.

Evaluating false positive rates of standard and hierarchical measures of metacognitive accuracy

A key aspect of metacognition is metacognitive accuracy, i.e., the degree to which confidence judgments differentiate between correct and incorrect trials. To quantify metacognitive accuracy, researchers are faced with an increasing number of different methods. The present study investigated false positive rates associated with various measures of metacognitive accuracy by hierarchical resampling from the confidence database to accurately represent the statistical properties of confidence judgements. We found that most measures based on the computation of summary-statistics separately for each participant and subsequent group-level analysis performed adequately in terms of false positive rate, including gamma correlations, meta-d′, and the area under type 2 ROC curves. Meta-d′/d′ is associated with a false positive rate even below 5%, but log-transformed meta-d′/d′ performs adequately. The false positive rate of HMeta-d depends on the study design and on prior specification: For group designs, the false positive rate is above 5% when independent priors are placed on both groups, but the false positive rate is adequate when a prior was placed on the difference between groups. For continuous predictor variables, default priors resulted in a false positive rate below 5%, but the false positive rate was not distinguishable from 5% when close-to-flat priors were used. Logistic mixed model regression analysis is associated with dramatically inflated false positive rates when random slopes are omitted from model specification. In general, we argue that no measure of metacognitive accuracy should be used unless the false positive rate has been demonstrated to be adequate.

P15.12.A BIUX2X2

Abstract BACKGROUND Brain metastases occur in 20-40% of cancer patients. In connection with the development of radiosurgical treatment, the problem of differential diagnosis of continued tumor growth and post-radiation changes becomes relevant. The main method of radiosurgical treatment of secondary lesions is the Gamma-Knife. Positron emission tomography with 11C-Meth and 18F-FET is an important method in monitoring the treatment of patients with secondary brain tumors. MATERIAL AND METHODS We examined 73 patients with brain metastases. Of these, 36.36% were patients with breast cancer, 39.39% were patients with lung cancer, 9.00% were patients with melanoma, 6.06% were patients with kidney cancer, 9.09% were other nosology, which corresponds to the data on the frequency of metastasis to the brain of various types of cancers. All patients underwent MRI with standard sequences (T1, T2, FLAIR, T1+C) and PET-CT with 18F-FET. PET-CT was performed according to the standard protocol: scanning 20 minutes after the INTRODUCTION of the radiopharmaceutical, scanning time 20 minutes. The evaluation of PET images was carried out by a semi-quantitative method - TBR max. RESULTS The mean age of patients with breast cancer was 54.5 ± 10.4, which significantly differs from the mean age of patients with lung cancer (65.9 ± 8.7) and the mean age of patients with other primary cancers (62.8 ± 8, 8) (Kruskal-Wallis test: H(2, N=66)=13.99052 p=.0009). Differences in TBR max and the number of metastases in these groups are not significant, as well as, according to the Kaplan-Meier analysis, differences in the duration of the relapse-free period. However, differences in the duration of the relapse-free period in the groups after a single ( Median, Q1; Q3) (12.9, 10.5; 24.4) or multiple (7.5, 3.5; 10.5) radiosurgical treatment are significant ( Gehan's Wilcoxon Test, Test statistic = 2.063905 p = .03903). Based on the analysis of the data obtained, it was found that the threshold value TBR max >=2.8 in 100% of cases indicates progression, TBR max <=1.7 indicates a stable course. Whereas TBR max in the range from 1.7 to 2.8 (in 33 people from the examined group, i.e. in 50% of all cases) may indicate both progression and post-radiation changes, as well as confirmed positive dynamics. CONCLUSION Thus, we confirmed that the numerical value of TBR max is a relative indicator. And when interpreting PET data, we cannot rely only on the TBR max value in the range from 1.7 to 2.8 for a confident judgment about the genesis of increased RP uptake in tumors. In the absence of other markers of the development of progression (for example, the appearance of new pathological foci), to assess the dynamics and confidently identify post-radiation changes, a repeat PET study should be prescribed after 2-3 months with a preliminary MRI with contrast enhancement.

Confidence in visual detection, familiarity and recollection judgments is preserved in schizophrenia spectrum disorder

An effective way to quantify metacognitive performance is to ask participants to estimate their confidence in the accuracy of their response during a cognitive task. A recent meta-analysis1 raised the issue that most assessments of metacognitive performance in schizophrenia spectrum disorders may be confounded with cognitive deficits, which are known to be present in this population. Therefore, it remains unclear whether the reported metacognitive deficits are metacognitive in nature or rather inherited from cognitive deficits. Arbitrating between these two possibilities requires equating task performance between experimental groups. Here, we aimed to characterize metacognitive performance among individuals with schizophrenia across three tasks (visual detection, familiarity, recollection) using a within-subject design while controlling experimentally for intra-individual task performance and statistically for between-subject task performance. In line with our hypotheses, we found no metacognitive deficit for visual detection and familiarity judgments. While we expected metacognition for recollection to be specifically impaired among individuals with schizophrenia, we found evidence in favor of an absence of a deficit in that domain also. We found no specific metacognitive deficit in schizophrenia spectrum disorder in the visual or memory domain. The clinical relevance of our findings is discussed in light of a hierarchical framework of metacognition.

The timing of confidence computations in human prefrontal cortex

Knowing when confidence computations take place is critical for building a mechanistic understanding of the neural and computational bases of metacognition. Yet, even though a substantial amount of research has focused on revealing the neural correlates and computations underlying human confidence judgments, very little is known about the timing of confidence computations. To understand when confidence is computed, we delivered single pulses of transcranial magnetic stimulation (TMS) at different times after stimulus presentation while subjects judged the orientation of a briefly presented visual stimulus and provided a confidence rating. TMS was delivered to either the right dorsolateral prefrontal cortex (DLPFC) in the experimental group or to vertex in the control group. We found that TMS to right DLPFC, but not to vertex, led to increased confidence in the absence of changes to accuracy or metacognitive efficiency. Critically, equivalent levels of confidence increase occurred for TMS delivered between 200 and 500 msec after stimulus presentation. These results suggest that confidence computations occur during a broad window that begins before the perceptual decision has been fully made and thus provide important constraints for theories of confidence generation.

From local to global estimations of confidence in perceptual decisions.

Perceptual confidence has been an important topic recently. However, one key limitation in current approaches is that most studies have focused on confidence judgments made for single decisions. In three experiments, we investigate how these local confidence judgments relate and contribute to global confidence judgments, by which observers summarize their performance over a series of perceptual decisions. We report two main results. First, we find that participants exhibit more overconfidence in their local than in their global judgments of performance, an observation mirroring the aggregation effect in knowledge-based decisions. We further show that this effect is specific to confidence judgments and does not reflect a calculation bias. Second, we document a novel effect by which participants' global confidence is larger for sets which are more heterogeneous in terms of difficulty, even when actual performance is controlled for. Surprisingly, we find that this effect of variability also occurs at the level of local confidence judgments, in a manner that fully explains the effect at the global level. Overall, our results indicate that global confidence is based on local confidence, although these two processes can be partially dissociated. We discuss possible theoretical accounts to relate and empirical investigations of how observers develop and use a global sense of perceptual confidence. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A COMPARATIVE ANALYSIS OF PRIMARY & SECONDARY SCHOOL TEACHERS IN REFERENCE TO THEIR SELF CONFIDENCE

In the present study, an attempt has been made to compare the self confidence of primary and secondary school teachers of Bareilly District on the basis of 10 areas :-I- Physical Confidence, II - Technological Confidence, III - Social Confidence, IV - Psychological Confidence, V - Judgment Confidence, VI - Readiness Confidence, VII - Environment Confidence, VIII - Stage confidence, IX - Status confidence, X - Peer Independence Confidence. This study has been conducted on a sample of 114 primary and secondary school teachers selected randomly from schools affiliated to C.B.S.E. & U.P. Board. The main finding of the study exhibited significant difference between male and female, primary and secondary school teachers of C.B.S.E & U.P. Board. Male teachers were found to posses more physical confident as compared to that of female teachers. Further, significant difference in physical confidence was also found between male and female teachers of primary and secondary schools. No significant difference was found in other dimensions such as:-Technological Confidence, Social Confidence, Psychological Confidence, Judgment Confidence, Readiness Confidence, Environment Confidence, Stage Confidence, Status Confidence and Peer Independence Confidence.

The relationship between students' gender and their confidence in the correctness of their solutions to complex and difficult mathematics problems

It is well established that men more frequently exhibit over-confidence than women in mathematics. Less is known about the relationship between gender and problem-specific confidence judgments and whether this relationship depends on problem difficulty. To investigate the relationship between gender, problem difficulty, and problem-specific confidence judgments, including how calibrated and under/over-confident men and women are, we examine data from 349 women and 279 men who were instructed to solve 13 difficult and complex mathematics problems and report their confidence in the correctness of their solutions. We find that men were more confident than women and that the gender gap in confidence decreases with increasing problem difficulty. Women were better calibrated than men and the gender gap in calibration increased with problem difficulty. As problem difficulty increased, women, but not men, transitioned from being under- to over-confident.

Pre-assessment judgement framework for judging nursing students’ performance in clinical placements: A qualitative case study

AimThis study aims to better understand and articulate the pre-assessment judgement processes commonly used by experienced clinical facilitators when assessing nursing students undertaking clinical placement. BackgroundIn the Australian context, clinical facilitators are registered nurses who primarily educate, monitor, support and assess groups of nursing students on clinical placements without carrying a patient load.The duties and scope of clinical facilitators may differ across international and institutional contexts. However, the core concepts of this paper will be relevant despite these differences as the importance of facilitators’ confidence in making pre-assessment judgements of individual nursing student performance while on placement is universally acknowledged.Nursing students are often assessed on their provision of safe practice, patient task-orientated outcomes and professional behaviour. Clearly articulating performance judgements prior to formal assessment is vital to ensure progressive learning of students. Literature reports that many clinical facilitators lack confidence in the art of making performance judgements and call for targeted professional training and support in the clinical assessment of nursing students. To better understand and address this problem, clinical facilitators need a shared understanding of how individual nursing students’ pre-assessment performance judgements are reached during placement experiences. DesignA qualitative case study was used, with data collected via semi-structured interviews. Fifteen Australian clinical facilitators participated, each with over six months of experience. MethodsInterview transcripts were analysed through an interpretive-constructivist paradigm. Thematic analysis revealed themes that were then deductively described through the application of the Cognitive Continuum Theory. ResultsSix modes of pre-assessment judgement emerged from the data synthesis process: 1) Recognising patterns, 2) Acknowledging uncertainty, 3) Understanding key players, 4) Verifying or refuting the information, 5) Benchmarking performance and 6) Contextualising information. Each mode is validated through the deductive application of the Cognitive Continuum theory. ConclusionsUnderstanding how experienced clinical facilitators make pre-assessment performance judgements has the potential to increase confidence in performance judgement decisions. In turn, confidence in judgements will increase clinical facilitator’s capacity to give nursing students feedback that can be explained and justified. The pre-assessment judgement framework also provides a preliminary model for teaching the art of reaching accurate performance judgements to clinical educators in disciplines beyond nursing.

The influence of irrelevant emotionally negative stimuli on early and late retrospective metacognitive judgements.

It is well established that negative emotions influence a range of cognitive processes. How these emotions influence the metacognitive judgement individuals make about their own performance and whether this influence is similar depending on the conditions under which metacognition is assessed, however, is far less understood. The primary aim of this study was to determine whether exposure to emotional stimuli could influence metacognitive judgements made under short or long time constraints. A total sample of 144 young adults (aged 18-35 years) was recruited and asked to complete an arithmetic strategy selection task under emotional or neutral condition. Following each strategy selection trial, participants also provided a retrospective confidence judgement (RCJ). Both strategy selection and RCJ were collected under short or long time constraints (1,500 vs. 2,500 ms for strategy selection and 800 vs. 1,500 ms for RCJ). In addition to replicating previous findings showing lower rates of better strategy selection under negative emotions compared with neutral condition, an effect of negative stimuli on the accuracy of participants' confidence judgements was found, but only if participants had a short time limit to make their second-level evaluation. Such findings are consistent with the hypothesis that exposure to emotional stimuli disturbs early, but not late metacognitive processes and have important implications to further our understanding of the role of emotions on metacognition.

Metacognitive awareness in the sound-induced flash illusion.

Hundreds (if not thousands) of multisensory studies provide evidence that the human brain can integrate temporally and spatially discrepant stimuli from distinct modalities into a singular event. This process of multisensory integration is usually portrayed in the scientific literature as contributing to our integrated, coherent perceptual reality. However, missing from this account is an answer to a simple question: how do confidence judgements compare between multisensory information that is integrated across multiple sources, and multisensory information that comes from a single, congruent source in the environment? In this paper, we use the sound-induced flash illusion to investigate if confidence judgements are similar across multisensory conditions when the numbers of auditory and visual events are the same, and the numbers of auditory and visual events are different. Results showed that congruent audiovisual stimuli produced higher confidence than incongruent audiovisual stimuli, even when the perceptual report was matched across the two conditions. Integrating these behavioural findings with recent neuroimaging and theoretical work, we discuss the role that prefrontal cortex may play in metacognition, multisensory causal inference and sensory source monitoring in general. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Does Using None-of-the-Above (NOTA) Hurt Students' Confidence?

Students claim that multiple-choice questions can be tricky, particularly those with competitive incorrect choices or choices like none-of-the-above (NOTA). Additionally, assessment researchers suggest that using NOTA is problematic for assessment. In experiments conducted online (with trivia questions) and in the classroom (with course-related questions), I investigated the effects of including NOTA as a multiple-choice choice alternative on students' confidence and performance. In four experiments, participants answered two types of questions: basic multiple-choice questions (basic condition) and equivalent questions in which one incorrect choice was replaced with NOTA (NOTA condition). Immediately after answering each question, participants rated their confidence in their answer to that question (item-by-item confidence). At the end of the experiments, participants made aggregate confidence judgments for the two types of questions and provided additional comments about the use of NOTA as an alternative. Surprisingly, I found no significant differences in item-by-item confidence or performance between the two conditions in any of the experiments. However, across all four experiments, when making aggregate judgments, participants provided lower confidence estimates in the NOTA condition than in the basic condition. Although people often report that NOTA questions hurt their confidence, the present results suggest that they might not-at least not on a question-by-question basis.

Metacognition in the audiovisual McGurk illusion: perceptual and causal confidence.

Almost all decisions in everyday life rely on multiple sensory inputs that can come from common or independent causes. These situations invoke perceptual uncertainty about environmental properties and the signals' causal structure. Using the audiovisual McGurk illusion, this study investigated how observers formed perceptual and causal confidence judgements in information integration tasks under causal uncertainty. Observers were presented with spoken syllables, their corresponding articulatory lip movements or their congruent and McGurk combinations (e.g. auditory B/P with visual G/K). Observers reported their perceived auditory syllable, the causal structure and confidence for each judgement. Observers were more accurate and confident on congruent than unisensory trials. Their perceptual and causal confidence were tightly related over trials as predicted by the interactive nature of perceptual and causal inference. Further, observers assigned comparable perceptual and causal confidence to veridical 'G/K' percepts on audiovisual congruent trials and their causal and perceptual metamers onMcGurk trials (i.e. illusory 'G/K' percepts). Thus, observers metacognitively evaluate the integrated audiovisual percept with limited access to the conflicting unisensory stimulus components on McGurk trials. Collectively, our results suggest that observers form meaningful perceptual and causal confidence judgements about multisensory scenes that are qualitatively consistent with principles of Bayesian causal inference. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Confidence in reality monitoring judgments.

Confidence in reality monitoring judgments.

Depression, Race-Based Bias, and Judgments of Veracity

Abstract: Introduction: The depressive realism hypothesis suggests that depressed individuals have a more realistic perception than nondepressed. Most studies depict the effects of depressive realism on self-perceptions. However, some lie detection studies suggest the effects of depressive realism on the perception of others. Our study investigated the hypothesis that individuals with subclinical depression levels (dysphoric) show a heightened accuracy in lie detection and a lower truth bias. Furthermore, we expected these effects to be stronger in positive statements. Moreover, we expected that nondysphoric but not dysphoric individuals would show a heightened truth bias for Black targets. We also expected to find the effects of depressive realism in judgmental confidence and in self-evaluations of performance. Methods: 472 participants classified 16 video statements as truth or lie. We tested all hypotheses using three depression measures: IPIP-300 Depression Subscale, the PHQ-9, and the CES-D. Results: In contrast to our hypothesis, we found no heightened accuracy in the veracity judgments of dysphoric individuals. Truth bias was higher in dysphoric participants than in nondysphoric. There was an interaction of valence with PHQ-9 but not with IPIP or CES-D. Nondysphoric but not dysphoric participants had a higher truth bias for Black targets. Furthermore, dysphoric individuals had lower judgmental confidence and lower but not more accurate self-evaluation values. Conclusion: We discuss the results and future directions.

Who am I really? The ephemerality of the self-schema following vmPFC damage

We studied the role of the ventromedial prefrontal cortex (vmPFC) in supporting the self-schema, by asking vmPFC patients, along with healthy and brain-damaged controls, to judge the degree to which they (or another person) were likely to engage in a series of activities, and how confident they were in their responses. Critically, participants provided their judgments on two separate occasions, a week apart. Our underlying assumption was that a strong self-schema would lead to confident and stable self-related judgments. We observed that control groups exhibited higher across-session consistency for self-related compared to other-related judgments, while this self-advantage was absent in vmPFC patients. In addition, regression analyses showed that in control groups the level of confidence associated with a specific (self- or other-related) judgment predicted the stability of that judgment across sessions. In contrast, vmPFC patients’ confidence and rating consistency were aligned only for other-related judgments. By contrast, self-related judgments changed across sessions regardless of the confidence level with which they were initially endorsed. These findings indicate that the vmPFC is crucial to maintaining the self-schema and supporting the reliable retrieval of self-related information.