validate {dgpsi} | R Documentation |
Validate a constructed GP, DGP, or linked (D)GP emulator
Description
This function calculates Leave-One-Out (LOO) cross validation or Out-Of-Sample (OOS) validation statistics for a constructed GP, DGP, or linked (D)GP emulator.
Usage
validate(
object,
x_test,
y_test,
method,
sample_size,
verb,
M,
force,
cores,
...
)
## S3 method for class 'gp'
validate(
object,
x_test = NULL,
y_test = NULL,
method = NULL,
sample_size = 50,
verb = TRUE,
M = 50,
force = FALSE,
cores = 1,
...
)
## S3 method for class 'dgp'
validate(
object,
x_test = NULL,
y_test = NULL,
method = NULL,
sample_size = 50,
verb = TRUE,
M = 50,
force = FALSE,
cores = 1,
...
)
## S3 method for class 'lgp'
validate(
object,
x_test = NULL,
y_test = NULL,
method = NULL,
sample_size = 50,
verb = TRUE,
M = 50,
force = FALSE,
cores = 1,
...
)
Arguments
object |
can be one of the following:
|
x_test |
OOS testing input data:
|
y_test |
the OOS output data corresponding to
|
method |
|
sample_size |
the number of samples to draw for each given imputation if |
verb |
a bool indicating if trace information for validation should be printed during function execution.
Defaults to |
M |
|
force |
a bool indicating whether to force LOO or OOS re-evaluation when the |
cores |
the number of processes to be used for validation. If set to |
... |
N/A. |
Details
See further examples and tutorials at https://mingdeyu.github.io/dgpsi-R/.
Value
If
object
is an instance of thegp
class, an updatedobject
is returned with an additional slot calledloo
(for LOO cross validation) oroos
(for OOS validation) that contains:two slots called
x_train
(orx_test
) andy_train
(ory_test
) that contain the validation data points for LOO (or OOS).a column matrix called
mean
, ifmethod = "mean_var"
, ormedian
, ifmethod = "sampling"
, that contains the predictive means or medians of the GP emulator at validation positions.three column matrices called
std
,lower
, andupper
that contain the predictive standard deviations and credible intervals of the GP emulator at validation positions. Ifmethod = "mean_var"
, the upper and lower bounds of a credible interval are two standard deviations above and below the predictive mean. Ifmethod = "sampling"
, the upper and lower bounds of a credible interval are 2.5th and 97.5th percentiles.a numeric value called
rmse
that contains the root mean/median squared error of the GP emulator.a numeric value called
nrmse
that contains the (max-min) normalized root mean/median squared error of the GP emulator. The max-min normalization uses the maximum and minimum values of the validation outputs contained iny_train
(ory_test
).-
an integer called
M
that contains the size of the conditioning set used for the Vecchia approximation, if used, for emulator validation. an integer called
sample_size
that contains the number of samples used for validation ifmethod = "sampling"
.
The rows of matrices (
mean
,median
,std
,lower
, andupper
) correspond to the validation positions.If
object
is an instance of thedgp
class, an updatedobject
is returned with an additional slot calledloo
(for LOO cross validation) oroos
(for OOS validation) that contains:two slots called
x_train
(orx_test
) andy_train
(ory_test
) that contain the validation data points for LOO (or OOS).a matrix called
mean
, ifmethod = "mean_var"
, ormedian
, ifmethod = "sampling"
, that contains the predictive means or medians of the DGP emulator at validation positions.three matrices called
std
,lower
, andupper
that contain the predictive standard deviations and credible intervals of the DGP emulator at validation positions. Ifmethod = "mean_var"
, the upper and lower bounds of a credible interval are two standard deviations above and below the predictive mean. Ifmethod = "sampling"
, the upper and lower bounds of a credible interval are 2.5th and 97.5th percentiles.a vector called
rmse
that contains the root mean/median squared errors of the DGP emulator across different output dimensions.a vector called
nrmse
that contains the (max-min) normalized root mean/median squared errors of the DGP emulator across different output dimensions. The max-min normalization uses the maximum and minimum values of the validation outputs contained iny_train
(ory_test
).-
an integer called
M
that contains size of the conditioning set used for the Vecchia approximation, if used, for emulator validation. an integer called
sample_size
that contains the number of samples used for validation ifmethod = "sampling"
.
The rows and columns of matrices (
mean
,median
,std
,lower
, andupper
) correspond to the validation positions and DGP emulator output dimensions, respectively.-
If
object
is an instance of thedgp
class with a categorical likelihood, an updatedobject
is returned with an additional slot calledloo
(for LOO cross validation) oroos
(for OOS validation) that contains:two slots called
x_train
(orx_test
) andy_train
(ory_test
) that contain the validation data points for LOO (or OOS).a matrix called
label
that contains predictive samples of labels from the DGP emulator at validation positions. The matrix has its rows corresponding to validation positions and columns corresponding to samples of labels.a list called
probability
that contains predictive samples of probabilities for each class from the DGP emulator at validation positions. The element in the list is a matrix that has its rows corresponding to validation positions and columns corresponding to samples of probabilities.a scalar called
log_loss
that represents the average log loss of the predicted labels in the DGP emulator across all validation positions. Log loss measures the accuracy of probabilistic predictions, with lower values indicating better classification performance.log_loss
ranges from0
to positive infinity, where a value closer to0
suggests more confident and accurate predictions.an integer called
M
that contains size of the conditioning set used for the Vecchia approximation, if used, in emulator validation.an integer called
sample_size
that contains the number of samples used for validation.
If
object
is an instance of thelgp
class, an updatedobject
is returned with an additional slot calledoos
(for OOS validation) that contains:two slots called
x_test
andy_test
that contain the validation data points for OOS.a list called
mean
, ifmethod = "mean_var"
, ormedian
, ifmethod = "sampling"
, that contains the predictive means or medians of the linked (D)GP emulator at validation positions.three lists called
std
,lower
, andupper
that contain the predictive standard deviations and credible intervals of the linked (D)GP emulator at validation positions. Ifmethod = "mean_var"
, the upper and lower bounds of a credible interval are two standard deviations above and below the predictive mean. Ifmethod = "sampling"
, the upper and lower bounds of a credible interval are 2.5th and 97.5th percentiles.a list called
rmse
that contains the root mean/median squared errors of the linked (D)GP emulator.a list called
nrmse
that contains the (max-min) normalized root mean/median squared errors of the linked (D)GP emulator. The max-min normalization uses the maximum and minimum values of the validation outputs contained iny_test
.-
an integer called
M
that contains size of the conditioning set used for the Vecchia approximation, if used, in emulator validation. an integer called
sample_size
that contains the number of samples used for validation ifmethod = "sampling"
.
Each element in
mean
,median
,std
,lower
,upper
,rmse
, andnrmse
corresponds to a (D)GP emulator in the final layer of the linked (D)GP emulator.
Note
When both
x_test
andy_test
areNULL
, LOO cross validation will be implemented. Otherwise, OOS validation will be implemented. LOO validation is only applicable to a GP or DGP emulator (i.e.,object
is an instance of thegp
ordgp
class). If a linked (D)GP emulator (i.e.,object
is an instance of thelgp
class) is provided,x_test
andy_test
must also be provided for OOS validation.
Examples
## Not run:
# See gp(), dgp(), or lgp() for an example.
## End(Not run)