ExtremisationWAgg {aggreCAT} | R Documentation |
Aggregation Method: ExtremisationWAgg
Description
Calculate beta-transformed arithmetic means of best estimates.
Usage
ExtremisationWAgg(
expert_judgements,
type = "BetaArMean",
name = NULL,
alpha = 6,
beta = 6,
cutoff_lower = NULL,
cutoff_upper = NULL,
placeholder = FALSE,
percent_toggle = FALSE,
round_2_filter = TRUE
)
Arguments
expert_judgements |
A dataframe in the format of data_ratings. |
type |
One of |
name |
Name for aggregation method. Defaults to |
alpha |
parameter for the 'shape1' argument in the |
beta |
parameter for the 'shape2' argument in the |
cutoff_lower |
Lower bound of middle region without extremisation in |
cutoff_upper |
Upper bound of middle region without extremisation in |
placeholder |
Toggle the output of the aggregation method to impute placeholder data. |
percent_toggle |
Change the values to probabilities. Default is |
round_2_filter |
Note that the IDEA protocol results in both a Round 1 and Round 2 set of probabilities for each claim. Unless otherwise specified, we will assume that the final Round 2 responses (after discussion) are being referred to. |
Details
This method takes the average of best estimates and transforms it using the cumulative distribution function of a beta distribution.
type
may be one of the following:
BetaArMean: Beta transformation applied across the entire range of calculated confidence scores.
\[\hat{p}_c\left( \text{BetaArMean} \right) = H_{\alpha \beta}\left(\frac{1}{N} \sum_{i=1}^N B_{i,c} \right),\]where \(H_{\alpha \beta}\) is the cumulative distribution function of the beta distribution with parameters \(\alpha\) and \(\beta\), which default to 6 in the function.
The justification for equal parameters (the 'shape1' and 'shape2' arguments in the stats::pbeta
function)
are outlined in Satopää et al (2014) and the references therein (note that the method outlined in that paper
is called a beta-transformed linear opinion pool).
To decide on the default shape value of 6
, we explored the data_ratings
dataset with random subsets of 5 assessments per claim,
which we expect to have for most of the claims assessed by repliCATS.
BetaArMean2: Beta transformation applied only to calculated confidence scores that are outside a specified middle range. The premise being that we don't extremise "fence-sitter" confidence scores.
\[\hat{p}_c\left( \text{BetaArMean2} \right) = \begin{cases} \displaystyle H_{\alpha \beta}\left(\frac{1}{N} \sum_{i=1}^N B_{i,c} \right), \text{ for } \frac{1}{N} \sum_{i=1}^N B_{i,c} < \textit{cutoff\_lower} \cr \displaystyle \frac{1}{N} \sum_{i=1}^N B_{i,c}, \text{ for } \textit{cutoff\_lower} \leq \frac{1}{N} \sum_{i=1}^N B_{i,c} \leq \textit{cutoff\_upper} \cr \displaystyle H_{\alpha \beta}\left(\frac{1}{N} \sum_{i=1}^N B_{i,c} \right), \text{ for } \frac{1}{N} \sum_{i=1}^N B_{i,c} > \textit{cutoff\_upper} \cr \end{cases}\]Value
A tibble of confidence scores cs
for each paper_id
.
Examples
ExtremisationWAgg(data_ratings)