eval_treatment_equality {fairmetrics} | R Documentation |
Examine Treatment Equality of a Model
Description
This function evaluates Treatment Equality, a fairness criterion that assesses whether the ratio of false negatives to false positives is similar across groups (e.g., based on gender or race). Treatment Equality ensures that the model does not disproportionately favor or disadvantage any group in terms of the relative frequency of missed detections (false negatives) versus false alarms (false positives).
Usage
eval_treatment_equality(
data,
outcome,
group,
probs,
cutoff = 0.5,
confint = TRUE,
alpha = 0.05,
bootstraps = 2500,
digits = 2,
message = TRUE
)
Arguments
data |
Data frame containing the outcome, predicted outcome, and sensitive attribute |
outcome |
Name of the outcome variable |
group |
Name of the sensitive attribute |
probs |
Predicted probabilities |
cutoff |
Cutoff value for the predicted probabilities |
confint |
Logical indicating whether to calculate confidence intervals |
alpha |
The 1 - significance level for the confidence interval, default is 0.05 |
bootstraps |
Number of bootstraps to use for confidence intervals |
digits |
Number of digits to round the results to, default is 2 |
message |
Logical; if TRUE (default), prints a textual summary of the
fairness evaluation. Only works if |
Value
A list containing the following elements:
False Negative / False Positive ratio for Group 1
False Negative / False Positive ratio for Group 2
Difference in False Negative / False Positive ratio
Ratio in False Negative / False Positive ratio If confidence intervals are computed (
confint = TRUE
):A vector of length 2 containing the lower and upper bounds of the 95% confidence interval for the difference in False Negative / False Positive ratio
A vector of length 2 containing the lower and upper bounds of the 95% confidence interval for the ratio in False Negative / False Positive ratio
See Also
eval_acc_parity
, eval_bs_parity
, eval_pos_pred_parity
, eval_neg_pred_parity
Examples
library(fairmetrics)
library(dplyr)
library(magrittr)
library(randomForest)
# Data for tests
data("mimic_preprocessed")
set.seed(123)
train_data <- mimic_preprocessed %>%
dplyr::filter(dplyr::row_number() <= 700)
# Fit a random forest model
rf_model <- randomForest::randomForest(factor(day_28_flg) ~ ., data = train_data, ntree = 1000)
# Test the model on the remaining data
test_data <- mimic_preprocessed %>%
dplyr::mutate(gender = ifelse(gender_num == 1, "Male", "Female")) %>%
dplyr::filter(dplyr::row_number() > 700)
test_data$pred <- predict(rf_model, newdata = test_data, type = "prob")[, 2]
# Fairness evaluation
# We will use sex as the sensitive attribute and day_28_flg as the outcome.
# Evaluate Treatment Equality
eval_treatment_equality(
data = test_data,
outcome = "day_28_flg",
group = "gender",
probs = "pred",
cutoff = 0.41,
confint = TRUE,
alpha = 0.05,
bootstraps = 2500,
digits = 2,
message = FALSE
)