BayesianWAgg {aggreCAT} | R Documentation |
Aggregation Method: BayesianWAgg
Description
Bayesian aggregation methods with either uninformative or informative prior distributions.
JAGS Install
For instructions on installing JAGS onto your system visit https://gist.github.com/dennisprangle/e26923fae7477566510757ab3341f54c
Usage
BayesianWAgg(
expert_judgements,
type = "BayTriVar",
priors = NULL,
name = NULL,
placeholder = FALSE,
percent_toggle = FALSE,
round_2_filter = TRUE
)
Arguments
expert_judgements |
A dataframe in the format of data_ratings. |
type |
One of |
priors |
(Optional) A dataframe of priors in the format of data_supp_priors, required for |
name |
Name for aggregation method. Defaults to |
placeholder |
Toggle the output of the aggregation method to impute placeholder data. |
percent_toggle |
Change the values to probabilities. Default is |
round_2_filter |
Note that the IDEA protocol results in both a Round 1 and Round 2 set of probabilities for each claim. Unless otherwise specified, we will assume that the final Round 2 responses (after discussion) are being referred to. |
Details
type
may be one of the following:
BayTriVar: The Bayesian Triple-Variability Method, fit with JAGS.
Three kinds of variability around best estimates are considered:
generic claim variability: variation across individuals within a claim
generic participant variability: variation within an individual across claims
claim - participant specific uncertainty (operationalised by bounds): informed by interval widths given by individual \(i\) for claim \(c\).
The model takes the log odds transformed individual best estimates as input (data), uses a normal likelihood function and derives a posterior distribution for the probability of replication.
\[log( \frac{B_{i,c}}{1-B_{i,c}}) \sim N(\mu_c, \sigma_{i,c}),\]where \(\mu_c\) denotes the mean estimated probability of replication for claim \(c\), and \(\sigma_{i,c}\) denotes the standard deviation of the estimated probability of replication for claim \(c\) and individual \(i\) (on the logit scale). Parameter \(\sigma_{i,c}\) is calculated as: \[\sigma_{i,c} = (U_{i,c} - L_{i,c} + 0.01) \times \sqrt{\sigma_i^2+\sigma_c^2}\] with \(\sigma_i\) denoting the standard deviation of estimated probabilities of replication for individual \(i\) and \(\sigma_c\) denoting the standard deviation of the estimated probability of replication for claim \(c\).
The uninformative priors for specifying this Bayesian model are \(\mu_c \sim N(0,\ 3)\), \(\sigma_i \sim U(0,\ 10)\) and \(\sigma_c \sim U(0,\ 10)\). After obtaining the median of the posterior distribution of \(\mu_c\), we can back transform to obtain \(\hat{p}_c\):
\[\hat{p}_c\left( BayTriVar \right) = \frac{e^{\mu_c}}{1+e^{\mu_c}}\]BayPRIORsAgg: Priors derived from predictive models, updated with best estimates.
This method uses Bayesian updating to update a prior probability of
replication estimated from a predictive model with an aggregate of the individuals’ best
estimates for any given claim. Methodology is the same as type
"BayTriVar"
except an
informative prior is used for \(\mu_c\). Conceptually the parameters of the prior
distribution of \(\mu_c\) are informed by the PRIORS model (Gould et al. 2021)
which is a multilevel logistic regression model that predicts the probability of
replication using attributes of the original study. However, any model providing predictions of
the probability of replication can be used to generate the required priors.
Value
A tibble of confidence scores cs
for each paper_id
.
Warning
Both BayTriVar
and BayPRIORsAgg
methods require a minimum of two claims for which judgements are supplied to expert_judgements
. This is due to the mathematical definition of these aggregators: BayesianWAgg
calculates the variance in best estimates across multiple claims as well as the variance in best estimates across claims per individual. Thus when only one claim is provided in expert_judgements
, the variance is 0, hence more than one claim is required for the successful execution of both Bayesian methods.
Examples
## Not run: BayesianWAgg(data_ratings)