estimateMetaI {statConfR}R Documentation

Estimate Measures of Metacognition from Information Theory

Description

estimateMetaI estimates meta-I, an information-theoretic measure of metacognitive sensitivity proposed by Dayan (2023), as well as similar derived measures, including meta-I_{1}^{r} and Meta-I_{2}^{r}. These are different normalizations of meta-I:

All measures can be calculated with a bias-reduced variant for which the observed frequencies are taken as underlying probability distribution to estimate the sampling bias. The estimated bias is then subtracted from the initial measures. This approach uses Monte-Carlo simulations and is therefore not deterministic (values can vary from one evaluation of the function to the next). However, this is a simple way to reduce the bias inherent in these measures.

Usage

estimateMetaI(data, bias_reduction = TRUE)

Arguments

data

a data.frame where each row is one trial, containing following variables:

  • participant (some group ID, most often a participant identifier; the meta-I measures are estimated for each subset of data determined by the different values of this column),

  • stimulus (stimulus category in a binary choice task, should be a factor with two levels, otherwise it will be transformed to a factor with a warning),

  • rating (discrete confidence judgments, should be a factor with levels ordered from lowest confidence to highest confidence; otherwise will be transformed to factor with a warning),

  • correct (encoding whether the response was correct; should be 0 for incorrect responses and 1 for correct responses)

bias_reduction

logical. Whether to apply the bias reduction or not. If runtime is too long, consider setting this to FALSE (default: TRUE).

Details

It is assumed that a classifier (possibly a human being performing a discrimination task) or an algorithmic classifier in a classification application, makes a binary prediction R about a true state of the world S and gives a confidence rating C. Meta-I is defined as the mutual information between the confidence and accuracy and is calculated as the transmitted information minus the minimal information given the accuracy,

meta-I = I(S; R, C) - I(S; R).

This is equivalent to Dayan's formulation where meta-I is the information that confidence transmits about the correctness of a response,

meta-I = I(S = R; C).

Meta-I is expressed in bits, i.e. the log base is 2). The other measures are different normalizations of meta-I and are unitless. It should be noted that Dayan (2023) pointed out that a liberal or conservative use of the confidence levels will affected the mutual information and thus influence meta-I.

Value

a data.frame with one row for each subject and the following columns:

Author(s)

Sascha Meyen, saschameyen@gmail.com

References

Dayan, P. (2023). Metacognitive Information Theory. Open Mind, 7, 392–411. doi:10.1162/opmi_a_00091

Examples

# 1. Select two subjects from the masked orientation discrimination experiment
data <- subset(MaskOri, participant %in% c(1:2))
head(data)

# 2. Calculate meta-I measures with bias reduction (this may take 10 s per subject)

metaIMeasures <- estimateMetaI(data)


# 3. Calculate meta-I measures for all participants without bias reduction (much faster)
metaIMeasures <- estimateMetaI(MaskOri, bias_reduction = FALSE)
metaIMeasures

[Package statConfR version 0.2.0 Index]