HMeta-d: hierarchical Bayesian estimation of metacognitive efficiency from confidence ratings.

Stephen M Fleming

doi:10.1093/nc/nix007

Abstract

Metacognition refers to the ability to reflect on and monitor one’s cognitive processes, such as perception, memory and decision-making. Metacognition is often assessed in the lab by whether an observer’s confidence ratings are predictive of objective success, but simple correlations between performance and confidence are susceptible to undesirable influences such as response biases. Recently, an alternative approach to measuring metacognition has been developed (Maniscalco and Lau 2012) that characterizes metacognitive sensitivity (meta-d') by assuming a generative model of confidence within the framework of signal detection theory. However, current estimation routines require an abundance of confidence rating data to recover robust parameters, and only provide point estimates of meta-d’. In contrast, hierarchical Bayesian estimation methods provide opportunities to enhance statistical power, incorporate uncertainty in group-level parameter estimates and avoid edge-correction confounds. Here I introduce such a method for estimating metacognitive efficiency (meta-d’/d’) from confidence ratings and demonstrate its application for assessing group differences. A tutorial is provided on both the meta-d’ model and the preparation of behavioural data for model fitting. Through numerical simulations I show that a hierarchical approach outperforms alternative fitting methods in situations where limited data are available, such as when quantifying metacognition in patient populations. In addition, the model may be flexibly expanded to estimate parameters encoding other influences on metacognitive efficiency. MATLAB software and documentation for implementing hierarchical meta-d’ estimation (HMeta-d) can be downloaded at https://github.com/smfleming/HMeta-d.

Highlights

Metacognition is defined as ‘knowledge of one’s own cognitive processes’ (Flavell 1979)
Metacognition is often assessed by whether an observer’s confidence ratings are predictive of objective success, but simple correlations between performance and confidence are susceptible to undesirable influences such as response biases
I conduct parameter recovery simulations to compare hierarchical Bayesian and standard estimation routines. These results show that, when data are limited, the new hierarchical Bayesian estimation of meta-d’ (HMeta-d) method outperforms traditional fitting procedures and provides appropriate control over false positives

Summary

Introduction

Metacognition is defined as ‘knowledge of one’s own cognitive processes’ (Flavell 1979). A blindsight patient may perform a task (e.g. discriminating the location of a stimulus) at a reasonably high level in the otherwise blind field, and yet lack insight as to whether they have performed accurately on any given trial (Persaud et al 2007). It is plausible that a joint lack of metacognition and conscious visual experience are both consequences of disruptions to higher-order representations (Lau and Rosenthal 2011; Ko and Lau 2012). While there are clearly other drivers of confidence in one’s task performance aside from sensory certainty (such as response requirements; Pouget et al 2016; Denison 2017), understanding the mechanisms supporting metacognition may shed light on the putative underpinnings of conscious experience. Understanding the relationship between metacognition and perceptual and cognitive processes has broader application in work on judgment and decision-making (Lichtenstein et al 1982), developmental psychology (Weil et al 2013; Goupil et al 2016), social psychology (Heatherton 2011)

Objectives

Methods

Results

Discussion

Conclusion