find_permuted_perf_metric {mikropml} | R Documentation |
Get permuted performance metric difference for a single feature (or group of features)
Description
Requires the future.apply
package
Usage
find_permuted_perf_metric(
test_data,
trained_model,
outcome_colname,
perf_metric_function,
perf_metric_name,
class_probs,
feat,
test_perf_value,
nperms = 100,
alpha = 0.05,
progbar = NULL
)
Arguments
test_data |
Held out test data: dataframe of outcome and features. |
trained_model |
Trained model from |
outcome_colname |
Column name as a string of the outcome variable
(default |
perf_metric_function |
Function to calculate the performance metric to
be used for cross-validation and test performance. Some functions are
provided by caret (see |
perf_metric_name |
The column name from the output of the function
provided to perf_metric_function that is to be used as the performance metric.
Defaults: binary classification = |
class_probs |
Whether to use class probabilities (TRUE for categorical outcomes, FALSE for numeric outcomes). |
feat |
feature or group of correlated features to permute. |
test_perf_value |
value of the true performance metric on the held-out test data. |
nperms |
number of permutations to perform (default: |
alpha |
alpha level for the confidence interval
(default: |
progbar |
optional progress bar (default: |
Value
vector of mean permuted performance and mean difference between test and permuted performance (test minus permuted performance)
Author(s)
Begüm Topçuoğlu, topcuoglu.begum@gmail.com
Zena Lapp, zenalapp@umich.edu
Kelly Sovacool, sovacool@umich.edu