tsbacktest.tsissm.autospec {tsissm}R Documentation

Walk Forward Model Backtest

Description

Generates an expanding window walk forward backtest.

Usage

## S3 method for class 'tsissm.autospec'
tsbacktest(
  object,
  start = floor(length(object$y)/2),
  end = length(object$y),
  h = 1,
  estimate_every = 1,
  rolling = FALSE,
  weights_scheme = c("AIC", "BIC", "U"),
  weights = NULL,
  seed = NULL,
  trace = FALSE,
  ...
)

## S3 method for class 'tsissm.spec'
tsbacktest(
  object,
  start = floor(length(object$target$y_orig)/2),
  end = length(object$target$y_orig),
  h = 1,
  estimate_every = 1,
  rolling = FALSE,
  seed = NULL,
  trace = FALSE,
  ...
)

Arguments

object

an object of class “tsissm.spec” “tsissm.autospec”.

start

numeric data index from which to start the backtest.

end

numeric data index on which to end the backtest. The backtest will end 1 period before that date in order to have at least 1 out of sample value to compare against.

h

forecast horizon. As the expanding window approaches the “end”, the horizon will automatically shrink to the number of available out of sample periods.

estimate_every

number of periods at which the model is re-estimated and new predictions are generated (defaults to 1).

rolling

this indicates whether forecasts are made only on the estimation date (FALSE) or whether to filter the data 1 period at a time and forecast from the filtered data (TRUE).

weights_scheme

the weighting scheme to use when using ensembling (see note).

weights

a vector of fixed user supplied weights of length top_n when choosing “U” in the “weights_scheme”.

seed

an value specifying if and how the random number generator should be initialized (‘seeded’). Either NULL or an integer that will be used in a call to set.seed before simulating the response vectors.

trace

whether to show the progress bar. The user is expected to have set up appropriate handlers for this using the “progressr” package.

...

not used.

Value

A list with the following data.tables:

Note

The function can use parallel functionality as long as the user has set up a plan using the future package. Model ensembling is used when the input object is of class “tsissm.autospec” and top_n is greeter than 1. The following weighting schemes are available:

U:

User supplied fixed weights.

AIC:

Models are weighted based on their Akaike Information Criterion (AIC), favoring models with lower AIC values. The weights are computed as:

w_i = \frac{\exp(-0.5 \times \Delta_i)}{\sum_{j} \exp(-0.5 \times \Delta_j)}

where

\Delta_i = AIC_i - \min(AIC)

BIC:

Similar to AIC-based weighting but uses the Bayesian Information Criterion (BIC) instead, which penalizes model complexity more strongly.

The final ensemble prediction is computed as:

\hat{y}_{\text{ensemble}} = \sum_{i} w_i \hat{y}_i

where w_i are the model weights and \hat{y}_i are the individual model predictions. AIC-based weighting is preferred when models have different complexities but are estimated on the same dataset, while BIC-based weighting may be better suited for large sample sizes due to its stronger penalty on complexity.

See Also

[tsensemble()]


[Package tsissm version 1.0.1 Index]