ExtremisationWAgg {aggreCAT}R Documentation

Aggregation Method: ExtremisationWAgg

Description

Calculate beta-transformed arithmetic means of best estimates.

Usage

ExtremisationWAgg(
  expert_judgements,
  type = "BetaArMean",
  name = NULL,
  alpha = 6,
  beta = 6,
  cutoff_lower = NULL,
  cutoff_upper = NULL,
  placeholder = FALSE,
  percent_toggle = FALSE,
  round_2_filter = TRUE
)

Arguments

expert_judgements

A dataframe in the format of data_ratings.

type

One of "BetaArMean" or "BetaArMean2".

name

Name for aggregation method. Defaults to type unless specified.

alpha

parameter for the 'shape1' argument in the stats::pbeta function (defaults to 6)

beta

parameter for the 'shape2' argument in the stats::pbeta function (defaults to 6)

cutoff_lower

Lower bound of middle region without extremisation in "BetaArMean2" aggregation types.

cutoff_upper

Upper bound of middle region without extremisation in "BetaArMean2" aggregation types.

placeholder

Toggle the output of the aggregation method to impute placeholder data.

percent_toggle

Change the values to probabilities. Default is FALSE.

round_2_filter

Note that the IDEA protocol results in both a Round 1 and Round 2 set of probabilities for each claim. Unless otherwise specified, we will assume that the final Round 2 responses (after discussion) are being referred to.

Details

This method takes the average of best estimates and transforms it using the cumulative distribution function of a beta distribution.

type may be one of the following:

BetaArMean: Beta transformation applied across the entire range of calculated confidence scores.

\[\hat{p}_c\left( \text{BetaArMean} \right) = H_{\alpha \beta}\left(\frac{1}{N} \sum_{i=1}^N B_{i,c} \right),\]

where \(H_{\alpha \beta}\) is the cumulative distribution function of the beta distribution with parameters \(\alpha\) and \(\beta\), which default to 6 in the function.

The justification for equal parameters (the 'shape1' and 'shape2' arguments in the stats::pbeta function) are outlined in Satopää et al (2014) and the references therein (note that the method outlined in that paper is called a beta-transformed linear opinion pool). To decide on the default shape value of 6, we explored the data_ratings dataset with random subsets of 5 assessments per claim, which we expect to have for most of the claims assessed by repliCATS.

BetaArMean2: Beta transformation applied only to calculated confidence scores that are outside a specified middle range. The premise being that we don't extremise "fence-sitter" confidence scores.

\[\hat{p}_c\left( \text{BetaArMean2} \right) = \begin{cases} \displaystyle H_{\alpha \beta}\left(\frac{1}{N} \sum_{i=1}^N B_{i,c} \right), \text{ for } \frac{1}{N} \sum_{i=1}^N B_{i,c} < \textit{cutoff\_lower} \cr \displaystyle \frac{1}{N} \sum_{i=1}^N B_{i,c}, \text{ for } \textit{cutoff\_lower} \leq \frac{1}{N} \sum_{i=1}^N B_{i,c} \leq \textit{cutoff\_upper} \cr \displaystyle H_{\alpha \beta}\left(\frac{1}{N} \sum_{i=1}^N B_{i,c} \right), \text{ for } \frac{1}{N} \sum_{i=1}^N B_{i,c} > \textit{cutoff\_upper} \cr \end{cases}\]

Value

A tibble of confidence scores cs for each paper_id.

Examples

ExtremisationWAgg(data_ratings)


[Package aggreCAT version 1.0.0 Index]