parameter_sensitivity_analysis {topolow} | R Documentation |
Parameter Sensitivity Analysis
Description
Analyzes the sensitivity of model performance (MAE) to changes in a parameter. Uses binning to identify the minimum MAE across parameter ranges and calculates thresholds for acceptable parameter values.
Usage
parameter_sensitivity_analysis(
param,
samples,
bins = 30,
mae_col = "Holdout_MAE",
threshold_pct = 5,
min_samples = 1
)
Arguments
param |
Character name of parameter to analyze |
samples |
Data frame containing parameter samples and performance metrics |
bins |
Integer number of bins for parameter range (default: 40) |
mae_col |
Character name of column containing MAE values (default: "Holdout_MAE") |
threshold_pct |
Numeric percentage above minimum for threshold calculation (default: 5) |
min_samples |
Integer minimum number of samples required in a bin (default: 1) |
Details
The function performs these steps:
Cleans the input data using MAD-based outlier detection
Bins the parameter values into equal-width bins
Calculates the minimum MAE within each bin. Analogous to "poorman's likelihood" approach, minimum MAE within each bin is an empirical estimate of the performance surface at this parameter value when other parameters are at their optimal values.
Identifies a threshold of acceptable performance (default: Topolow min. +5% MAE)
Returns an object for visualization and further analysis
Value
Object of class "parameter_sensitivity" containing:
param_values |
Vector of parameter bin midpoints |
min_mae |
Vector of minimum MAE values per bin |
param_name |
Name of analyzed parameter |
threshold |
Threshold value (default: Topolow min. +5%) |
min_value |
Minimum MAE value across all bins |
sample_counts |
Number of samples per bin |