func_epsilon {binaryRL} | R Documentation |
Function: Exploration Strategy
Description
The exploration strategy parameters are
threshold
, epsilon
, and lambda
.
-
Epsilon-first strategy: Used when only
threshold
is set. Subjects choose randomly for trials less thanthreshold
and by value for trials greater than 'threshold
. -
Epsilon-greedy strategy: Used if
threshold
is default (1) andepsilon
is set. Subjects explore with probabilityepsilon
throughout the experiment. -
Epsilon-decreasing strategy: Used if
threshold
is default (1), andlambda
is set. In this strategy, the probability of random choice (exploration) decreases as trials increase. The parameterlambda
controls the rate at which this probability declines with each trial.
Usage
func_epsilon(
i,
L_freq,
R_freq,
L_pick,
R_pick,
L_value,
R_value,
var1 = NA,
var2 = NA,
threshold = 1,
epsilon = NA,
lambda = NA,
alpha,
beta
)
Arguments
i |
The current row number. |
L_freq |
The frequency of left option appearance |
R_freq |
The frequency of right option appearance |
L_pick |
The number of times left option was picked |
R_pick |
The number of times left option was picked |
L_value |
The value of the left option |
R_value |
The value of the right option |
var1 |
[character] Column name of extra variable 1. If your model uses more than just reward and expected value, and you need other information, such as whether the choice frame is Gain or Loss, then you can input the 'Frame' column as var1 into the model.
|
var2 |
[character] Column name of extra variable 2. If one additional variable, var1, does not meet your needs, you can add another additional variable, var2, into your model.
|
threshold |
[integer]
Controls the initial exploration phase in the epsilon-first strategy.
This is the number of early trials where the subject makes purely random
choices, as they haven't yet learned the options' values. For example,
|
epsilon |
[numeric] A parameter used in the epsilon-greedy exploration strategy. It defines the probability of making a completely random choice, as opposed to choosing based on the relative values of the left and right options. For example, if 'epsilon = 0.1', the subject has a 10 choice and a 90 relevant when 'threshold' is at its default value (1) and 'lambda' is not set.
|
lambda |
[vector] A numeric value that controls the decay rate of exploration probability in the epsilon-decreasing strategy. A higher 'lambda' value means the probability of random choice will decrease more rapidly as the number of trials increases.
|
alpha |
[vector] Extra parameters that may be used in functions. |
beta |
[vector] Extra parameters that may be used in functions. |
Value
A numeric value, either 0 or 1. 0 indicates no exploration (choice based on value), and 1 indicates exploration (random choice) for that trial.
Note
When customizing these functions, please ensure that you do not modify the arguments. Instead, only modify the 'if-else' statements or the internal logic to adapt the function to your needs.
Examples
## Not run:
func_epsilon <- function(
# Trial number
i,
# Number of times this option has appeared
L_freq,
R_freq,
# Number of times this option has been chosen
L_pick,
R_pick,
# Current value of this option
L_value,
R_value,
# Extra variables
var1 = NA,
var2 = NA,
# Free Parameters
threshold = 1,
epsilon = NA,
lambda = NA,
# Extra parameters
alpha,
beta
){
# Epsilon-First: random choosing before a certain trial number
if (i <= threshold) {
try <- 1
} else if (i > threshold & is.na(epsilon) & is.na(lambda)) {
try <- 0
# Epsilon-Greedy: random choosing throughout the experiment with probability epsilon
} else if (i > threshold & !(is.na(epsilon)) & is.na(lambda)){
try <- sample(
c(1, 0),
prob = c(epsilon, 1 - epsilon),
size = 1
)
# Epsilon-Decreasing: probability of random choosing decreases as trials increase
} else if (i > threshold & is.na(epsilon) & !(is.na(lambda))) {
try <- sample(
c(1, 0),
prob = c(
1 / (1 + lambda * i),
lambda * i / (1 + lambda * i)
),
size = 1
)
}
else {
try <- "ERROR"
}
return(try)
}
## End(Not run)