ShiftingWAgg {aggreCAT} | R Documentation |
Aggregation Method: ShiftingWAgg
Description
Weighted by judgements that shift the most after discussion
Usage
ShiftingWAgg(
expert_judgements,
type = "ShiftWAgg",
name = NULL,
placeholder = FALSE,
percent_toggle = FALSE
)
Arguments
expert_judgements |
A dataframe in the format of data_ratings. |
type |
One of |
name |
Name for aggregation method. Defaults to |
placeholder |
Toggle the output of the aggregation method to impute placeholder data. |
percent_toggle |
Change the values to probabilities. Default is |
Details
When judgements are elicited using the IDEA protocol (or any other protocol that allows experts to revisit their original estimates), the second round of estimates may differ from the original first set of estimates an expert provides. Greater changes between rounds will be given greater weight.
type
may be one of the following:
ShiftWAgg: Takes into account the shift in all three estimates
Considers shifts across lower, \(L_{i,c}\), and upper, \(U_{i,c}\), confidence limits, and the best estimate, \(B_{i,c}\). More emphasis is placed on changes in the best estimate such that:
\[w\_Shift_{i,c} = |B1_{i,c} - B_{i,c}| + \frac{|L1_{i,c} - L_{i,c}|+|U1_{i,c} - U_{i,c}|}{2},\]where \(L1_{i,c}, B1_{i,c},U1_{i,c}\) are the first round lower, best and upper estimates (prior to discussion) and \(L_{i,c}, B_{i,c},U1_{i,c}\) are the individual’s revised second round estimates (after discussion).
\[\hat{p}_c(ShiftWAgg) = \sum_{i=1}^N \tilde{w}\_Shift_{i,c}B_{i,c}\]BestShiftWAgg: Weighted according to shifts in best estimates alone
Taking into account the fact that the scales best estimates are measured on are bounded, we can calculate shifts relative to the largest possible shift.
\[w\_BestShift_{i,c}= \begin{cases} \frac{|B1_{i,c} - B_{i,c}|}{B1_{i,c}}, \begin{aligned} \displaystyle &\ for\ (B1_{i,c} > 0.5\ and\ B_{i,c} \leq 0.5) \cr \displaystyle &\ or\ B_{i,c} < B1_{i,c} \leq 0.5\ or\ B1_{i,c} > B_{i,c} > 0.5 \end{aligned} \cr \frac{|B1_{i,c} - B_{i,c}|}{1- B1_{i,c}}, \begin{aligned} \displaystyle &\ for\ (B1_{i,c} < 0.5\ and\ B_{i,c} \geq 0.5) \cr \displaystyle &\ or\ B1_{i,c} < B_{i,c} < 0.5\ or\ B_{i,c} > B1_{i,c} > 0.5. \end{aligned} \end{cases}\] \[\hat{p}_c(BestShiftWAgg) = \sum_{i=1}^N \tilde{w}\_BestShift_{i,c}B_{i,c}\]IntShiftWAgg: Weighted by shifts in interval widths alone.
Individuals whose interval widths narrow between rounds are given more weight.
\[w\_IntShift_{i,c} = \frac{1}{(U_{i,c}-L_{i,c})-(U1_{i,c}-L1_{i,c})+1}\] \[\hat{p}_c(IntShiftWAgg) = \sum_{i=1}^N \tilde{w}\_IntShift_{i,c}B_{i,c}\]DistShiftWAgg: Weighted by whether best estimates become more extreme (closer to 0 or 1) between rounds.
\[w\_DistShift_{i,c} = 1 - (\min (B_{i,c}, 1-B_{i,c}) - \min (B1_{i,c}, 1-B1_{i,c}))\] \[\hat{p}_c(DistShiftWAgg) = \sum_{i=1}^N \tilde{w}\_DistShift_{i,c}B_{i,c}\]DistIntShiftWAgg: Rewards both narrowing of intervals and shifting towards the certainty limits between rounds.
We simply multiply the weights calculated in the "DistShiftWAgg" and "IntShiftWAgg" methods.
\[w\_DistIntShift_{i,c} = \tilde{w}\_IntShift_{i,c} \cdot \tilde{w}\_DistShift_{i,c}\] \[\hat{p}_c(DistIntShiftWAgg) = \sum_{i=1}^N \tilde{w}\_DistIntShift_{i,c}B_{i,c}\]Value
A tibble of confidence scores cs
for each paper_id
.
Examples
ShiftingWAgg(data_ratings)