Conditional error spending functions

Introduction

We describe conditional error spending functions for group sequential designs. These functions are used to calculate conditional error spending boundaries for group sequential designs using spending functions proposed by Xi and Gallo (2019). Note that for all spending functions, for \(t>1\) we define the spending function as if \(t = 1\). For \(\gamma \in [0, 1]\), we define

\[ z_\gamma = \Phi^{-1}(1 - \gamma) \]

where \(\Phi\) is the standard normal cumulative distribution function.

There are 3 spending functions proposed by Xi and Gallo (2019), which we will refer to as Method 1 (sfXG1()), Method 2 (sfXG2()), and Method 3 (sfXG3()). When there is a single interim analysis, conditional error from Method 1 is almost exactly the same as the parameter \(\gamma\) but it allows a narrower range of \(\gamma\). Method 2 provides less accurate approximation but allows a wider range of \(\gamma\). Method 3 was proposed to approximate Pocock bounds with equal bounds on the Z-scale. We replicate spending function bounds of Xi and Gallo (2019) along with corresponding conditional error computations below. We also compare results to other commonly used spending functions.

Implementation in gsDesign

library(gsDesign)
library(tibble)
library(dplyr)
library(gt)

Method 1

For \(\gamma \in [0.5, 1)\), the spending function is defined as

\[ \alpha_\gamma(t) = 2 - 2\times \Phi\left(\frac{z_{\alpha/2} - z_\gamma\sqrt{1-t}}{\sqrt t} \right). \]

Recalling the range \(\gamma \in [0.5, 1)\), we plot this spending function for \(\gamma = 0.5, 0.6, 0.75, 0.9\).

pts <- seq(0, 1.2, 0.01)
pal <- palette()
plot(
  pts,
  sfXG1(0.025, pts, 0.5)$spend,
  type = "l", col = pal[1],
  xlab = "t", ylab = "Spending", main = "Xi-Gallo, Method 1"
)
lines(pts, sfXG1(0.025, pts, 0.6)$spend, col = pal[2])
lines(pts, sfXG1(0.025, pts, 0.75)$spend, col = pal[3])
lines(pts, sfXG1(0.025, pts, 0.9)$spend, col = pal[4])
legend(
  "topleft",
  legend = c("gamma=0.5", "gamma=0.6", "gamma=0.75", "gamma=0.9"),
  col = pal[1:4],
  lty = 1
)

Method 2

For \(\gamma \in [1 - \Phi(z_{\alpha/2}/2), 1)\), the spending function for Method 2 is defined as

\[ \alpha_\gamma(t)= 2 - 2\times \Phi\left(\frac{z_{\alpha/2} - z_\gamma(1-t)}{\sqrt t} \right) \]

For \(\alpha=0.025\), we restrict \(\gamma\) to [0.131, 1) and plot the spending function for \(\gamma = 0.14, 0.25, 0.5, 0.75, 0.9\).

plot(
  pts,
  sfXG2(0.025, pts, 0.14)$spend,
  type = "l", col = pal[1],
  xlab = "t", ylab = "Spending", main = "Xi-Gallo, Method 2"
)
lines(pts, sfXG2(0.025, pts, 0.25)$spend, col = pal[2])
lines(pts, sfXG2(0.025, pts, 0.5)$spend, col = pal[3])
lines(pts, sfXG2(0.025, pts, 0.75)$spend, col = pal[4])
lines(pts, sfXG2(0.025, pts, 0.9)$spend, col = pal[5])
legend(
  "topleft",
  legend = c("gamma=0.14", "gamma=0.25", "gamma=0.5", "gamma=0.75", "gamma=0.9"),
  col = pal[1:5],
  lty = 1
)

Method 3

For \(\gamma \in (\alpha/2, 1)\)

\[ \alpha_\gamma(t)= 2 - 2\times \Phi\left(\frac{z_{\alpha/2} - z_\gamma(1-\sqrt t)}{\sqrt t} \right). \]

For \(\alpha=0.025\), we restrict \(\gamma\) to \((0.0125, 1)\) and plot the spending function for \(\gamma = 0.013, 0.02, 0.05, 0.1, 0.25, 0.5, 0.75, 0.9\).

plot(
  pts,
  sfXG3(0.025, pts, 0.013)$spend,
  type = "l", col = pal[1],
  xlab = "t", ylab = "Spending", main = "Xi-Gallo, Method 3"
)
lines(pts, sfXG3(0.025, pts, 0.02)$spend, col = pal[2])
lines(pts, sfXG3(0.025, pts, 0.05)$spend, col = pal[3])
lines(pts, sfXG3(0.025, pts, 0.1)$spend, col = pal[4])
lines(pts, sfXG3(0.025, pts, 0.25)$spend, col = pal[5])
lines(pts, sfXG3(0.025, pts, 0.5)$spend, col = pal[6])
lines(pts, sfXG3(0.025, pts, 0.75)$spend, col = pal[7])
lines(pts, sfXG3(0.025, pts, 0.9)$spend, col = pal[8])
legend(
  "bottomright",
  legend = c(
    "gamma=0.013", "gamma=0.02", "gamma=0.05", "gamma=0.1",
    "gamma=0.25", "gamma=0.5", "gamma=0.75", "gamma=0.9"
  ),
  col = pal[1:8],
  lty = 1
)

Replicating published examples

We replicate spending function bounds of Xi and Gallo (2019) along with corresponding conditional error computations. We have two utility functions. Transposing a tibble and a custom function to compute conditional error.

# Custom function to transpose while preserving names
# From https://stackoverflow.com/questions/42790219/how-do-i-transpose-a-tibble-in-r
transpose_df <- function(df) {
  t_df <- data.table::transpose(df)
  colnames(t_df) <- rownames(df)
  rownames(t_df) <- colnames(df)
  t_df <- t_df %>%
    tibble::rownames_to_column(.data = .) %>%
    tibble::as_tibble(.)
  return(t_df)
}
ce <- function(x) {
  k <- x$k
  ce <- c(gsCPz(z = x$upper$bound[1:(k - 1)], i = 1:(k - 1), x = x, theta = 0), NA)
  t <- x$timing
  ce_simple <- c(pnorm((last(x$upper$bound) - x$upper$bound[1:(k - 1)] * sqrt(t[1:(k - 1)])) / sqrt(1 - t[1:(k - 1)]),
    lower.tail = FALSE
  ), NA)
  Analysis <- 1:k
  y <- tibble(
    # Analysis = Analysis,
    Z = x$upper$bound,
    "CE simple" = ce_simple,
    CE = ce
  )
  return(y)
}

Method 1

The conditional error spending functions of Xi and Gallo (2019) for Method 1 and Method 2 are designed to derive interim efficacy bounds with conditional error approximately equal to the spending function parameter \(\gamma\). The conditional probability of crossing the final bound \(u_K\) given an interim result \(Z_k=u_k\) at analysis \(k<K\) under the assumption of no treatment effect is

\[ p_0(Z_K\ge u_K|Z_k=u_k) = 1 - \Phi\left(\frac{u_K - u_k\sqrt t}{\sqrt{1-t}}\right). \]

Conditional rejection probabilities accounting for all future analyses as well as under any assumed treatment effect are explained further in the gsDesign technical manual. We will see that where there are future interim analyses below, the conditional error for crossing at least one future efficacy bound is substantially greater than the simple conditional error ignoring future interims.

We will compare the Method 1 conditional error spending functions of Xi and Gallo (2019) with O’Brien-Fleming bounds (sfu = "OF"), exponential spending (sfu = sfExponential), and the Lan-DeMets spending function to approximate O’Brien-Fleming bounds (sfu = sfLDOF). The O’Brien-Fleming bounds are specifically known to the 0.5 (simple) conditional error as seen in the table below. The exponential spending function provides the closest approximation of O’Brien-Fleming bounds with a parameter of 0.76; this was suggested previously by Anderson and Clark (2010). The other Method 1 spending functions generally have higher than the targeted simple conditional error; thus, if you wish to use a particular conditional error at bounds, it may be better to see if a smaller \(\gamma\) than the targeted conditional error provides a better match.

xOF <- gsDesign(k = 4, test.type = 1, sfu = "OF")
xLDOF <- gsDesign(k = 4, test.type = 1, sfu = sfLDOF)
xExp <- gsDesign(k = 4, test.type = 1, sfu = sfExponential, sfupar = 0.76)
x1.8 <- gsDesign(k = 4, test.type = 1, sfu = sfXG1, sfupar = 0.8)
x1.7 <- gsDesign(k = 4, test.type = 1, sfu = sfXG1, sfupar = 0.7)
x1.6 <- gsDesign(k = 4, test.type = 1, sfu = sfXG1, sfupar = 0.6)
x1.5 <- gsDesign(k = 4, test.type = 1, sfu = sfXG1, sfupar = 0.5)
xx <- rbind(
  transpose_df(ce(xOF)) %>% mutate(gamma = "O'Brien-Fleming"),
  transpose_df(ce(xExp)) %>% mutate(gamma = "Exponential, nu=0.76 to Approximate O'Brien-Fleming"),
  transpose_df(ce(xLDOF)) %>% mutate(gamma = "Lan-DeMets to Approximate O'Brien-Fleming"),
  transpose_df(ce(x1.5)) %>% mutate(gamma = "gamma = 0.5"),
  transpose_df(ce(x1.6)) %>% mutate(gamma = "gamma = 0.6"),
  transpose_df(ce(x1.7)) %>% mutate(gamma = "gamma = 0.7"),
  transpose_df(ce(x1.8)) %>% mutate(gamma = "gamma = 0.8")
)
xx %>%
  gt(groupname_col = "gamma") %>%
  tab_spanner(label = "Analysis", columns = 2:5) %>%
  fmt_number(columns = 2:5, decimals = 3) %>%
  tab_options(data_row.padding = px(1)) %>%
  tab_header(
    title = "Xi-Gallo, Method 1 Spending Function",
    subtitle = "Conditional Error Spending Functions"
  ) %>%
  tab_footnote(
    footnote = "Conditional Error not accounting for future interim bounds.",
    locations = cells_stub(rows = seq(2, 20, 3))
  ) %>%
  tab_footnote(
    footnote = "CE = Conditional Error accounting for all analyses.",
    locations = cells_stub(rows = seq(3, 21, 3))
  )
Xi-Gallo, Method 1 Spending Function
Conditional Error Spending Functions
Analysis
1 2 3 4
O'Brien-Fleming
Z 4.049 2.863 2.337 2.024
1 CE simple 0.500 0.500 0.500 NA
2 CE 0.687 0.625 0.500 NA
Exponential, nu=0.76 to Approximate O'Brien-Fleming
Z 4.052 2.890 2.346 2.020
1 CE simple 0.502 0.513 0.509 NA
2 CE 0.682 0.636 0.509 NA
Lan-DeMets to Approximate O'Brien-Fleming
Z 4.333 2.963 2.359 2.014
1 CE simple 0.570 0.546 0.523 NA
2 CE 0.747 0.668 0.523 NA
gamma = 0.5
Z 4.333 2.963 2.359 2.014
1 CE simple 0.570 0.546 0.523 NA
2 CE 0.747 0.668 0.523 NA
gamma = 0.6
Z 4.784 3.230 2.508 1.983
1 CE simple 0.682 0.665 0.647 NA
2 CE 0.804 0.749 0.647 NA
gamma = 0.7
Z 5.265 3.514 2.671 1.969
1 CE simple 0.778 0.767 0.754 NA
2 CE 0.858 0.821 0.754 NA
gamma = 0.8
Z 5.826 3.845 2.863 1.963
1 CE simple 0.864 0.857 0.849 NA
2 CE 0.908 0.887 0.849 NA
1 Conditional Error not accounting for future interim bounds.
2 CE = Conditional Error accounting for all analyses.

Method 2

Method 2 provides a wider range of \(\gamma\) values targeting conditional error at bounds. Again, choice of \(\gamma\) to get the targeted conditional error may be worth some evaluation.

x1.8 <- gsDesign(k = 4, test.type = 1, sfu = sfXG2, sfupar = 0.8)
x1.7 <- gsDesign(k = 4, test.type = 1, sfu = sfXG2, sfupar = 0.7)
x1.6 <- gsDesign(k = 4, test.type = 1, sfu = sfXG2, sfupar = 0.6)
x1.5 <- gsDesign(k = 4, test.type = 1, sfu = sfXG2, sfupar = 0.5)
x1.4 <- gsDesign(k = 4, test.type = 1, sfu = sfXG2, sfupar = 0.4)
x1.3 <- gsDesign(k = 4, test.type = 1, sfu = sfXG2, sfupar = 0.3)
x1.2 <- gsDesign(k = 4, test.type = 1, sfu = sfXG2, sfupar = 0.2)
xx <- rbind(
  transpose_df(ce(x1.2)) %>% mutate(gamma = "gamma = 0.2"),
  transpose_df(ce(x1.3)) %>% mutate(gamma = "gamma = 0.3"),
  transpose_df(ce(x1.4)) %>% mutate(gamma = "gamma = 0.4"),
  transpose_df(ce(x1.5)) %>% mutate(gamma = "gamma = 0.5"),
  transpose_df(ce(x1.6)) %>% mutate(gamma = "gamma = 0.6"),
  transpose_df(ce(x1.7)) %>% mutate(gamma = "gamma = 0.7"),
  transpose_df(ce(x1.8)) %>% mutate(gamma = "gamma = 0.8")
)
xx %>%
  gt(groupname_col = "gamma") %>%
  tab_spanner(label = "Analysis", columns = 2:5) %>%
  fmt_number(columns = 2:5, decimals = 3) %>%
  tab_options(data_row.padding = px(1)) %>%
  tab_footnote(
    footnote = "Conditional Error not accounting for future interim bounds.",
    locations = cells_stub(rows = seq(2, 20, 3))
  ) %>%
  tab_footnote(
    footnote = "CE = Conditional Error accounting for all analyses.",
    locations = cells_stub(rows = seq(3, 21, 3))
  ) %>%
  tab_header(
    title = "Xi-Gallo, Method 2 Spending Function",
    subtitle = "Conditional Error Spending Functions"
  )
Xi-Gallo, Method 2 Spending Function
Conditional Error Spending Functions
Analysis
1 2 3 4
gamma = 0.2
Z 3.016 2.350 2.208 2.224
1 CE simple 0.204 0.213 0.267 NA
2 CE 0.475 0.368 0.267 NA
gamma = 0.3
Z 3.516 2.574 2.239 2.097
1 CE simple 0.348 0.348 0.376 NA
2 CE 0.591 0.498 0.376 NA
gamma = 0.4
Z 3.940 2.774 2.295 2.044
1 CE simple 0.466 0.454 0.455 NA
2 CE 0.677 0.592 0.455 NA
gamma = 0.5
Z 4.333 2.963 2.359 2.014
1 CE simple 0.570 0.546 0.523 NA
2 CE 0.747 0.668 0.523 NA
gamma = 0.6
Z 4.724 3.152 2.429 1.995
1 CE simple 0.664 0.629 0.586 NA
2 CE 0.807 0.734 0.586 NA
gamma = 0.7
Z 5.141 3.353 2.509 1.982
1 CE simple 0.751 0.709 0.648 NA
2 CE 0.861 0.795 0.648 NA
gamma = 0.8
Z 5.627 3.588 2.604 1.973
1 CE simple 0.834 0.788 0.714 NA
2 CE 0.909 0.853 0.714 NA
1 Conditional Error not accounting for future interim bounds.
2 CE = Conditional Error accounting for all analyses.

Method 3

Method 3 spending functions are designed to approximate Pocock bounds with equal bounds on the Z-scale. Two common approximations used for this is the Hwang, Shih, and De Cani (1990) spending function with \(\gamma = 1\)

\[ \alpha_{HSD}(t, \gamma) = \alpha\frac{1-e^{-\gamma t}}{1 - e^{-\gamma}}. \]

and the Lan and DeMets (1983) spending function to approximate Pocock bounds

\[ \alpha_{LDP}(t) = \alpha \log(1 +(e -1)t). \]

We compare these methods in the following table. While the \(\gamma = 0.025\) conditional error spending bounds are close to the targeted Pocock bounds, the \(\gamma = 0.05\) conditional error spending bound is substantially higher than the targeted value at the first interim. The traditional Lan-DeMets and Hwang-Shih-DeCani approximations are quite good approximations of the Pocock bounds. The one number not reproduced from Xi and Gallo (2019) is the conditional error at the first analysis for \(\gamma = 0.05\); while here we have computed 0.132, in the paper this value was 0.133. There are differences in the computation algorithms that may account for this difference. The method used in gsDesign is from Chapter 19 of Jennison and Turnbull (2000), specifically designed for numerical integration for group sequential trials. The method used in Xi and Gallo (2019) is a more general method approximating multivariate normal probabilities.

xPocock <- gsDesign(k = 4, test.type = 1, sfu = "Pocock")
xLDPocock <- gsDesign(k = 4, test.type = 1, sfu = sfLDPocock)
xHSD1 <- gsDesign(k = 4, test.type = 1, sfu = sfHSD, sfupar = 1)
x3.025 <- gsDesign(k = 4, test.type = 1, sfu = sfXG3, sfupar = 0.025)
x3.05 <- gsDesign(k = 4, test.type = 1, sfu = sfXG3, sfupar = 0.05)
xx <- rbind(
  transpose_df(ce(xPocock)) %>% mutate(gamma = "Pocock"),
  transpose_df(ce(xLDPocock)) %>% mutate(gamma = "Lan-DeMets to Approximate Pocock"),
  transpose_df(ce(xHSD1)) %>% mutate(gamma = "Hwang-Shih-DeCani, gamma = 1"),
  transpose_df(ce(x3.025)) %>% mutate(gamma = "gamma = 0.025"),
  transpose_df(ce(x3.05)) %>% mutate(gamma = "gamma = 0.05 ")
)
xx %>%
  gt(groupname_col = "gamma") %>%
  tab_spanner(label = "Analysis", columns = 2:5) %>%
  fmt_number(columns = 2:5, decimals = 3) %>%
  tab_options(data_row.padding = px(1)) %>%
  tab_footnote(
    footnote = "Conditional Error not accounting for future interim bounds.",
    locations = cells_stub(rows = seq(2, 11, 3))
  ) %>%
  tab_footnote(
    footnote = "CE = Conditional Error accounting for all analyses.",
    locations = cells_stub(rows = seq(3, 12, 3))
  ) %>%
  tab_header(
    title = "Xi-Gallo, Method 3 Spending Function",
    subtitle = "Conditional Error Spending Functions"
  )
Xi-Gallo, Method 3 Spending Function
Conditional Error Spending Functions
Analysis
1 2 3 4
Pocock
Z 2.361 2.361 2.361 2.361
1 CE simple 0.086 0.164 0.263 NA
2 CE 0.228 0.283 0.263 NA
Lan-DeMets to Approximate Pocock
Z 2.368 2.368 2.358 2.350
1 CE simple 0.089 0.170 0.269 NA
2 CE 0.230 0.289 0.269 NA
Hwang-Shih-DeCani, gamma = 1
Z 2.376 2.357 2.350 2.357
1 CE simple 0.088 0.164 0.260 NA
2 CE 0.235 0.286 0.260 NA
gamma = 0.025
Z 2.269 2.339 2.422 2.483
1 CE simple 0.060 0.120 0.220 NA
2 CE 0.196 0.230 0.220 NA
gamma = 0.05
Z 2.609 2.330 2.281 2.270
CE simple 0.132 0.189 0.278 NA
CE 0.328 0.318 0.278 NA
1 Conditional Error not accounting for future interim bounds.
2 CE = Conditional Error accounting for all analyses.

Summary

Xi and Gallo (2019) proposed conditional error spending functions for group sequential designs are implemented in the gsDesign package. When there is a single interim analysis, conditional error from Method 1 is almost exactly the same as the parameter \(\gamma\) but it allows a narrower range of \(\gamma\). Method 2 provides less accurate approximation but allows a wider range of \(\gamma\). Using \(\gamma = 0.5\) replicates the Lan-DeMets spending function to approximate O’Brien-Fleming bounds. An exponential spending function provides a possibly better approximation of O’Brien-Fleming bounds. Simply selecting a smaller \(\gamma\) than the targeted conditional error may provide a better match for the targeted bounds. While Method 3 provides a reasonable approximation of a Pocock bound with equal bounds on the Z-scale, its stated objective, traditional approximations of Pocock bounds with the Lan-DeMets and Hwang-Shih-DeCani spending functions may be slightly better.

Results duplicated findings from the original paper.

References

Anderson, K. M., and J. Clark. 2010. “Group Sequential Design: Selecting Appropriate Bounds.” Journal of Clinical Oncology 28 (15_suppl): e13159–59.
Hwang, Irving K, Weichung J Shih, and John S De Cani. 1990. “Group Sequential Designs Using a Family of Type i Error Probability Spending Functions.” Statistics in Medicine 9 (12): 1439–45.
Jennison, Christopher, and Bruce W. Turnbull. 2000. Group Sequential Methods with Applications to Clinical Trials. Boca Raton, FL: Chapman; Hall/CRC.
Lan, K. K. G., and David L. DeMets. 1983. “Discrete Sequential Boundaries for Clinical Trials.” Biometrika 70: 659–63.
Xi, Dong, and Paul Gallo. 2019. “An Additive Boundary for Group Sequential Designs with Connection to Conditional Error.” Statistics in Medicine 38 (23): 4656–69.