Confidence Judgments Research Articles

Inference and Search on Graph-Structured Spaces

How do people learn functions on structured spaces? And how do they use this knowledge to guide their search for rewards in situations where the number of options is large? We study human behavior on structures with graph-correlated values and propose a Bayesian model of function learning to describe and predict their behavior. Across two experiments, one assessing function learning and one assessing the search for rewards, we find that our model captures human predictions and sampling behavior better than several alternatives, generates human-like learning curves, and also captures participants’ confidence judgements. Our results extend past models of human function learning and reward learning to more complex, graph-structured domains.

Retrospective metamemory monitoring of semantic memory in community-dwelling older adults with subjective cognitive decline and mild cognitive impairment

ABSTRACT In neurodegenerative conditions, better memory/cognitive awareness, indexed by greater “metamemory monitoring accuracy”, is linked to stronger cognitive remediation outcomes. Differences in metamemory monitoring accuracy in predementia conditions, which could inform treatment effectiveness, have not been systematically investigated. We utilized a retrospective confidence judgment (RCJ) task for general knowledge recognition in community-dwelling older adults: 106 cognitively healthy (HC), 68 subjective cognitive decline (SCD) despite intact neuropsychological function, 14 amnestic mild cognitive impairment (aMCI), and 31 non-amnestic mild cognitive impairment (naMCI). Participants gave confidence ratings after making recognition responses to general knowledge questions. Recognition accuracy, confidence levels, and absolute and relative RCJ accuracy (i.e., metamemory monitoring accuracy) were analysed. Compared to HC and SCD, absolute RCJ accuracy was significantly poorer in both MCI groups but relative RCJ accuracy was significantly poorer in naMCI, but not aMCI. This novel result may be driven by lower confidence for correct recognition responses in naMCI and suggests that poorer RCJ accuracy in naMCI may be attributable to poorer performance monitoring. We discuss results in relation to the possibility that individuals in distinct preclinical dementia conditions, who have different levels of memory/cognitive awareness, may differentially benefit from cognitive remediation strategies tailored to their levels of memory/cognitive awareness.

Effects of motor response pattern on serial dependence in visual confidence

Visual confidence -- the evaluation of one’s own performance in a visual task -- demonstrates serial dependence when the visual task is performed over a series of trials. Despite the converging evidence for such serial effects on visual confidence, the underlying mechanisms remain unclear. The present study investigated the effects of motor response pattern on serial dependence in visual confidence using a novel paradigm. In each trial, participants performed a left-right, direction-discrimination task on a random-dot motion pattern. Simultaneously, they rated their confidence by pressing one of the 8 keys (2 directions x 4 points). Task difficulty was manipulated by varying motion coherence, which had been calibrated to individual participant’s direction discriminability using adaptive staircase. In the main experiment, we sequenced trials based on their difficulty levels, such that a (target) medium-difficulty trial was preceded by either easy or difficult trials. In Experiment 1, we found that confidence rating in target trials was higher when they were preceded by easy trials than by hard trials, despite matched task performance of the target trials between the two conditions. This suggests that confidence judgments can be biased in a series of tasks. In Experiment 2, we employed the same paradigm but instructed participants to respond either using the keyboard as in Experiment 1 or using the mouse to indicate confidence on one of two (left and right) on-screen, continuous sliders. We found the same serial confidence bias only when mouse was used in both preceding and target trials. Interestingly, using the meta-d’ analysis, we found that metacognitive sensitivity was higher when participants switched from mouse (preceding) to keyboard (target). In other words, repeatedly responding in the same motor pattern could impair metacognitive sensitivity. This suggests that the continuity in motor response pattern may contribute in producing serial effects on visual confidence.

Perceiving motion in the world during pursuit eye movements: Directional and confidence judgements favour a re-calibration model

Perceiving motion in the world during pursuit eye movements: Directional and confidence judgements favour a re-calibration model

Crossmodal metaperception: Visual and tactile confidence share a common scale

Abstract Perceptual decisions are typically accompanied by a subjective sense of (un)certainty. There is robust evidence that observers have access to a reliable estimate of their own uncertainty and can judge the validity of their perceptual decisions. However, there is still a debate to what extent these meta-perceptual judgements underly a common mechanism that can monitor perceptual decisions across different sensory modalities. It has been suggested that perceptual confidence can be evaluated on an abstract scale that is not only task-independent but also modality-independent. We aimed to scrutinize these findings by measuring visual contrast and tactile vibration discrimination thresholds in a confidence forced-choice task. A total of 56 participants took part in our study. We determined thresholds for trials in which perceptual decisions were chosen as confident and for those that were declined as confident. Confidence comparisons were made between perceptual decisions either within the visual and tactile modality, respectively, or across both modalities. Furthermore, we assessed executive functions to explore a possible link between cognitive control and meta-perceptual capacities. We found that perceptual performance was a good predictor of confidence judgments and that the threshold modulation was similarly pronounced in both modalities. Most importantly, participants compared their perceptual confidence across visual and tactile decisions with the same precision as within the same modality. Cognitive control capacities were not related to meta-perceptual performance. In conclusion, our findings corroborate that perceptual uncertainty can be accessed on an abstract scale, allowing for confidence comparisons across sensory modalities.

Formation of global self-beliefs in the human brain

Humans create metacognitive beliefs about their performance across many levels of abstraction-from local confidence in individual decisions to global estimates of our skills and abilities. Despite a rich literature on the neural basis of local confidence judgements, how global self-performance estimates (SPEs) are constructed remains unknown. Using functional magnetic resonance imaging, we scanned human subjects while they performed several short blocks of tasks and reported on which task they think they performed best, providing a behavioral proxy for global SPEs. In a frontoparietal network sensitive to fluctuations in local confidence, we found that activity within ventromedial prefrontal cortex and precuneus was additionally modulated by global SPEs. In contrast, activity in ventral striatum was associated with subjects' global SPEs irrespective of fluctuations in local confidence, and predicted the extent to which global SPEs tracked objective task difficulty across individuals. Our findings reveal neural representations of global SPEs that go beyond the tracking of local confidence, and lay the groundwork for understanding how a formation of global self-beliefs may go awry in conditions characterized by distorted self-evaluation.

Method of informational risk range evaluation in decision making

Looks into evaluation of information provision probability from different sources, based on use of linguistic variables. Formation of functions appurtenant for its unclear variables provides for adoption of decisions by the decision maker, in conditions of nonprobabilistic equivocation. The development of market relations in Ukraine increases the independence and responsibility of enterprises in justifying and making management decisions that ensure their effective, competitive activities. As a result of the analysis, it is determined that the condition of economic facilities can be described and determined by the decision-maker, in the presence of the necessary information. The confidence of the decision-maker in the information received is different and the decisions made have a correspondingly different level of information risk. It is important to substantiate the procedure for assessing the numerical extent of information risk in decision-making based on the information obtained in conditions of uncertainty. The use of a linguistic variable in the processing of expert data presented in the form of a matrix of binary relations of values of the membership function, which allowed to move to further processing of knowledge to support decision-making in the management of industrial, commercial, financial and other activities. As a mathematical model for estimating the numerical measure of information risk when making decisions based on the information obtained in conditions of non-stochastic uncertainty, a model has been developed to model natural language uncertainties, which differs from existing ones by formalizing knowledge taking into account uncertainty of input information. Making such a clear decision in a fuzzy environment has appropriate values of effectiveness and risk. The paper proposes all the functions and accessories of indicators of both quantitative nature and qualitative nature to bring their values in the field of definition to one scale. Then the indicator of the effectiveness of decision-making will be a measure of the clarity of the cross-section of fuzzy subsets, which correspond to the introduced indicators of information risk. The condition of economic facilities can be described and determined by the decision-maker, if the necessary information is available. Decision-making on thenumerical measure of information risk must be determined by a set of basic indicators (criteria), which can be both quantitative and qualitative in nature. Predictive values of indicators should be determined in conditions of nonstochastic uncertainty. In this case, the indicators of a quantitative nature can be determined by fuzzy triangular numbers, which implement a high level of confidence in the subjective judgments of experts. Indicators of qualitative nature should be presented in linguistic variables. The values of the indicators of qualitative nature that are predicted must be considered for all fuzzy variable terms-sets of linguistic variables introduced into consideration. For any fuzzy variable, the introduction to the consideration of a clear set of values as carriers of the α-level of its membership function allows to reduce to a single interpretation of the predicted values of indicators of quantitative and qualitative nature in terms of non-stochastic uncertainty.

Childhood growth in math and reading differentially predicts adolescent non-ability-based confidence: An examination in the SECCYD

Non-ability-based confidence is one of the most pervasive human psychological biases. It is a part of a family of confidence judgments, including overconfidence and metacognitive calibration accuracy, defined by a discrepancy between self-perception of ability and actual ability. Across many domains, most people exhibit some degree of miscalibration in their confidence. Some people may be overconfident and others are underconfident. Despite the prevalence of non-ability-based confidence, relatively little research has investigated how non-ability-based confidence develops and why some people are more or less confident than others despite sharing the same level of ability. We use a longitudinal dataset to explore the childhood predictors of adolescent non-ability-based confidence. Achievement growth in math and reading in childhood was modeled and used to predict adolescent non-ability-based confidence in math and reading. Results show that the initial level of achievement predicts lower non-ability-based confidence in math. On the other hand, a faster rate of achievement growth across childhood predicts greater non-ability-based confidence in reading. These results highlight how previous experiences inform people's self-perceptions over and above their true abilities. Discussion focuses on the factors that shape non-ability-based confidence over the lifespan and the limitations of the current findings.

Discrete confidence levels revealed by sequential decisions.

Humans can meaningfully express their confidence about uncertain events. Normatively, these beliefs should correspond to Bayesian probabilities. However, it is unclear whether the normative theory provides an accurate description of the human sense of confidence, partly because the self-report measures used in most studies hinder quantitative comparison with normative predictions. To measure confidence objectively, we developed a dual-decision task in which the correctness of a first decision determines the correct answer of a second decision, thus mimicking real-life situations in which confidence guides future choices. While participants were able to use confidence to improve performance, they fell short of the ideal Bayesian strategy. Instead, behaviour was better explained by a model with a few discrete confidence levels. These findings question the descriptive validity of normative accounts, and suggest that confidence judgments might be based on point estimates of the relevant variables, rather than on their full probability distributions.

Social Judgments of Digitally Manipulated Stuttered Speech: Cognitive Heuristics Drive Implicit and Explicit Bias.

Purpose People who stutter are susceptible to discrimination, stemming from negative stereotypes and social misattributions. There has been a recent push to evaluate the underlying explicit and implicit cognitive mechanisms associated with social judgments, moving away from only evaluating explicit social bias about people who stutter. The purpose of the current study was to evaluate how listeners change their implicit and explicit social (mis)attributions after hearing a people who stutter produce disfluent speech. Method The current project was an adaptation of the Byrd et al. (2017) study to evaluate listener implicit/explicit social judgments of stuttered speech across five categories (i.e., confidence, friendliness, intelligence, distractibility, and extroversion) before and after a stuttering self-disclosure. This was done by implementing a modified version of the Ferguson et al. (2019) computer mouse-tracking paradigm. Results Consistent with previous findings, participants made more explicit positive social judgments of confidence, friendliness, extroversion, and intelligence after a stuttering self-disclosure, but the distractedness category was resistant to change. Also consistent with previous findings, participants experienced a higher degree of cognitive competition (i.e., higher area under the curve) shortly after self-disclosure, which lessened over time. Conclusions Explicit and implicit biases exist, but self-disclosure significantly impacts the cognitive system of listeners. Specifically, self-disclosure may reduce explicit bias through experience and explicit belief updating, but when cognitive heuristics are strong, implicit bias may be slower to change.

Impaired recollection-based episodic memory as a cognitive endophenotype in schizophrenia

ABSTRACT Introduction Patients with schizophrenia show impaired recollection but largely preserved familiarity-based episodic memory. This study was done to clarify the endophenotypic nature of recollection and familiarity-based episodic memory in schizophrenia and the role of emotional valence of memoranda and degree of recall confidence in it. Method Twenty-five patients with schizophrenia, one unaffected sibling of each patient, and twenty-three healthy controls completed two tasks assessing recollection and familiarity-based processes in episodic memory. In the first task, participants were asked to remember positive, negative, and neutral emotional valence words in a remember–know paradigm. In the second task, in addition to recollection and familiarity-based responses, participants were asked to make confidence judgments about their responses. Results Patients with schizophrenia and their first-degree relatives (FDRs) performed poorly on recollection but not familiarity-based responses, compared to healthy controls; performance of first-degree relatives was in between and significantly different from that of both patients and controls. The differences in recollection and familiarity-based responses across the three groups were not moderated by recall confidence judgments or emotional valence of memoranda. Furthermore, there was no correlation between recollection-based memory impairments and duration or severity of illness or current medication exposure. Conclusions Impaired recollection-based memory constitutes a potential cognitive endophenotype in schizophrenia. Furthermore, selective impairment of recollection-based, but sparing of familiarity-based, memory in patients and their FDRs supports the distinct nature of recollection and familiarity-based episodic memories.

Similarities and differences in spatial and non-spatial cognitive maps.

Learning and generalization in spatial domains is often thought to rely on a "cognitive map", representing relationships between spatial locations. Recent research suggests that this same neural machinery is also recruited for reasoning about more abstract, conceptual forms of knowledge. Yet, to what extent do spatial and conceptual reasoning share common computational principles, and what are the implications for behavior? Using a within-subject design we studied how participants used spatial or conceptual distances to generalize and search for correlated rewards in successive multi-armed bandit tasks. Participant behavior indicated sensitivity to both spatial and conceptual distance, and was best captured using a Bayesian model of generalization that formalized distance-dependent generalization and uncertainty-guided exploration as a Gaussian Process regression with a radial basis function kernel. The same Gaussian Process model best captured human search decisions and judgments in both domains, and could simulate realistic learning curves, where we found equivalent levels of generalization in spatial and conceptual tasks. At the same time, we also find characteristic differences between domains. Relative to the spatial domain, participants showed reduced levels of uncertainty-directed exploration and increased levels of random exploration in the conceptual domain. Participants also displayed a one-directional transfer effect, where experience in the spatial task boosted performance in the conceptual task, but not vice versa. While confidence judgments indicated that participants were sensitive to the uncertainty of their knowledge in both tasks, they did not or could not leverage their estimates of uncertainty to guide exploration in the conceptual task. These results support the notion that value-guided learning and generalization recruit cognitive-map dependent computational mechanisms in spatial and conceptual domains. Yet both behavioral and model-based analyses suggest domain specific differences in how these representations map onto actions.

Monitoring the Own Spatial Thinking in Second Grade of Primary Education in a Spanish School: Preliminary Study Analyzing Gender Differences

Previous studies on metacognitive performance have explored children’s abilities during primary school (7–11 years) in abstract and mathematical reasoning tasks. However, there have been no studies evaluating the metamemory processes with spatial tasks in primary school children, and even more generally, only a few studies have explored spatial metacognition in adults. Taking as a preliminary study a Spanish school, the present work explores the validity of the confidence judgment model when thinking about one’s own performance in a spatial test, for boys and girls in Second Year of Primary Education (mean age of 7 years). A total of 18 boys and 15 girls applied a 4-point scale to evaluate, item by item, the confidence of their responses in the Spatial aptitude test “E” of the EFAI-1 (Factorial Assessment of Intellectual Abilities to mentally process visual stimuli). Accessibility and Accuracy Indexes were calculated for each item of the spatial task. The effect of gender was analyzed too. The tasks were administered in small groups; at the end examiners interviewed each participant, performing the confidence judgment task, item by item, of the EFAI-1 previously answered. The results (analyses carried out by SPSS) showed a high mean confidence (3 mean points out of a maximum of 4), without finding any significant differences either in the spatial performance or in the mean confidence rating between boys and girls. A significant relationship between confidence judgments and spatial task performance accuracy was found. The relationship between confidence judgments and spatial performance cannot be confirmed. The procedure adapted for testing spatial judgments about the own responses has been useful for showing the well calibrated perception about performance at this stage. The implications of the results of this exploratory study and the potential of the application of the procedure to promote thought about one’s own spatial performance and the development of strategies that modulate the effective approach of this type of spatial tasks are discussed within an educational approach.

Is there a G factor for metacognition? Correlations in retrospective metacognitive sensitivity across tasks.

Is metacognition a general resource shared across domains? Previous research has documented consistent biases in judgments across tasks. In contrast, there is debate regarding the domain generality or the domain specificity of the ability to discriminate between correct and incorrect answers (metacognitive sensitivity) because most previous work has documented nonsignificant correlations across domains. However, such null findings may be due to low statistical power and differences in task structure or performance, thereby masking a latent domain generality in metacognition. We examined across-domain correlations in confidence level and sensitivity in a large sample (N = 181). Participants performed 4 2-alternative forced-choice tasks (episodic memory, semantic memory, executive function, and visual perception) with trial-by-trial confidence judgments. We found significant correlations in average confidence level across tasks. By applying a hierarchical Bayesian model to estimate cross-task covariance, we found five out 6 cross-task correlations in metacognitive efficiency (meta-d′/d′) were significant, even for pairs of tasks in which first-order performance was not correlated. This suggests that at least some components of metacognitive efficiency in retrospective confidence are domain general.

Understanding and supporting Chinese middle Schoolers’ monitoring accuracy in mathematics

While many studies of monitoring accuracy have been conducted with college students, less is known about middle school students’ monitoring accuracy, especially in Asian countries. Prior research also found discrepancies in students’ monitoring accuracy between Western and Asian cultures. To understand and support Chinese middle school students’ monitoring accuracy in mathematics, we implemented a three-week long monitoring intervention with 7th grade students in Southwestern China. Students were divided into three conditions: a control condition, a confidence rating condition (CR), and a confidence rating combined with monitoring instructions condition (CR + MI). Findings indicated that Chinese middle school students were slightly underconfident, yet quite accurate when monitoring during task completion in mathematics. Specifically, although students in the CR and CR + MI conditions did not show significant increases in mathematics performance, students in the control condition significantly decreased their performance overtime. In terms of monitoring, students’ confidence bias increased overtime in the CR + MI and control conditions while students in the CR condition did not show significant change. Students also demonstrated increased self-reported metacognitive awareness across the three conditions. We also asked students to provide justifications for their confidence judgments. Findings indicated that students tended to consider a single factor when judging their performance during a mathematics task. Regression models of students’ justifications indicated potential suggestions for future interventions. Implications and future directions are discussed.

Eye spy a liar: assessing the utility of eye fixations and confidence judgments for detecting concealed recognition of faces, scenes and objects

BackgroundIn criminal investigations, uncooperative witnesses might deny knowing a perpetrator, the location of a murder scene or knowledge of a weapon. We sought to identify markers of recognition in eye fixations and confidence judgments whilst participants told the truth and lied about recognising faces (Experiment 1) and scenes and objects (Experiment 2) that varied in familiarity. To detect recognition we calculated effect size differences in markers of recognition between familiar and unfamiliar items that varied in familiarity (personally familiar, newly learned).ResultsIn Experiment 1, recognition of personally familiar faces was reliably detected across multiple fixation markers (e.g. fewer fixations, fewer interest areas viewed, fewer return fixations) during honest and concealed recognition. In Experiment 2, recognition of personally familiar non-face items (scenes and objects) was detected solely by fewer fixations during honest and concealed recognition; differences in other fixation measures were not consistent. In both experiments, fewer fixations exposed concealed recognition of newly learned faces, scenes and objects, but the same pattern was not observed during honest recognition. Confidence ratings were higher for recognition of personally familiar faces than for unfamiliar faces.ConclusionsRobust memories of personally familiar faces were detected in patterns of fixations and confidence ratings, irrespective of task demands required to conceal recognition. Crucially, we demonstrate that newly learned faces should not be used as a proxy for real-world familiarity, and that conclusions should not be generalised across different types of familiarity or stimulus class.

Supplemental Material for The Confident Conservative: Ideological Differences in Judgment and Decision-Making Confidence

Supplemental Material for The Confident Conservative: Ideological Differences in Judgment and Decision-Making Confidence

Timescales of Evidence Evaluation for Decision Making and Associated Confidence Judgments Are Adapted to Task Demands.

Decision making often involves choosing actions based on relevant evidence. This can benefit from focussing evidence evaluation on the timescale of greatest relevance based on the situation. Here, we use an auditory change detection task to determine how people adjust their timescale of evidence evaluation depending on task demands for detecting changes in their environment and assessing their internal confidence in those decisions. We confirm previous results that people adopt shorter timescales of evidence evaluation for detecting changes in contexts with shorter signal durations, while bolstering those results with model-free analyses not previously used and extending the results to the auditory domain. We also extend these results to show that in contexts with shorter signal durations, people also adopt correspondingly shorter timescales of evidence evaluation for assessing confidence in their decision about detecting a change. These results provide important insights into adaptability and flexible control of evidence evaluation for decision making.

Communicating Assurance Using Practitioner-Customized Procedures: An Experiment and Emerging Research Opportunities

SUMMARYTraditionally, financial and nonfinancial information assurance standards have specified either “high” assurance based on “sufficient evidence” or “moderate” assurance based on analytical procedures and inquiries. Recently, in response to rapidly growing nonfinancial assurance demand, the IAASB extended the possible range of limited assurance by allowing practitioner-customized procedure descriptions and assuming diverse users can “appreciate” the varying reliability achieved. To test the validity of this policy change, we examine report users' confidence judgments for a GHG emissions assurance report using combinations of report attributes: critical practitioner-customized procedure descriptions, conclusion frame, and engagement label. We find that, consistent with an “assurance communication gap,” including or explicitly excluding a practitioner-customized procedure deemed essential-for-reasonable assurance does not significantly affect users' confidence judgments. However, we find that both positive conclusion frames and reasonable assurance engagement labels incrementally enhance confidence judgments. We outline research and practice opportunities related to emerging policy prescriptions involving practitioner-customized procedures.

Performance monitoring for sensorimotor confidence: A visuomotor tracking study

To best interact with the external world, humans are often required to consider the quality of their actions. Sometimes the environment furnishes rewards or punishments to signal action efficacy. However, when such feedback is absent or only partial, we must rely on internally generated signals to evaluate our performance (i.e., metacognition). Yet, very little is known about how humans form such judgements of sensorimotor confidence. Do they monitor their actual performance or do they rely on cues to sensorimotor uncertainty? We investigated sensorimotor metacognition in two visuomotor tracking experiments, where participants followed an unpredictably moving dot cloud with a mouse cursor as it followed a random horizontal trajectory. Their goal was to infer the underlying target generating the dots, track it for several seconds, and then report their confidence in their tracking as better or worse than their average. In Experiment 1, we manipulated task difficulty with two methods: varying the size of the dot cloud and varying the stability of the target's velocity. In Experiment 2, the stimulus statistics were fixed and duration of the stimulus presentation was varied. We found similar levels of metacognitive sensitivity in all experiments, which was evidence against the cue-based strategy. The temporal analysis of metacognitive sensitivity revealed a recency effect, where error later in the trial had a greater influence on the sensorimotor confidence, consistent with a performance-monitoring strategy. From these results, we conclude that humans predominantly monitored their tracking performance, albeit inefficiently, to build a sense of sensorimotor confidence.