1 Purpose

This vignette is used to introduce futility and it’s two versions: binding and non-binding.

2 Futility boundaries

In a sequential meta-analysis it is possible to stop early based on evidence for benefit of treatment or harm of treatment. But it might also be of interest to stop the sequential meta-analysis early for evidence of futility; when the probability of rejecting the null hypothesis is sufficiently low [1]. Futility boundaries are computed in RTSA to be able to stop a sequential meta-analysis for futility.

The futility boundaries in RTSA allows to stop for futility while not compromising the control of type-I-errors. There are two kinds of futility boundaries to choose from: binding and non-binding futility boundaries [2]. We will in this section introduce these two kinds of futility boundaries. After the introduction will we compare the two. The original TSA software only included non-binding futility boundaries [3].

2.1 Binding futility

We provide two examples of binding futility before explaining what is means to have binding futility in detail. See Figure 1 for one- and two-sided designs with no futility or with binding futility.

library(RTSA)
bound_none <- boundaries(timing = c(0.5,0.75, 1), alpha  = 0.025, beta = 0.1, 
                    side = 1, futility = "none", es_alpha = "esOF")
bound_bind <- boundaries(timing = c(0.5,0.75, 1), alpha  = 0.025, beta = 0.1, 
                    side = 1, futility = "binding", es_alpha = "esOF",
                    es_beta = "esOF")
bound_none2 <- boundaries(timing = c(0.5,0.75, 1), alpha  = 0.05, beta = 0.1, 
                    side = 2, futility = "none", es_alpha = "esOF")
bound_bind2 <- boundaries(timing = c(0.5,0.75, 1), alpha  = 0.05, beta = 0.1, 
                    side = 2, futility = "binding", es_alpha = "esOF",
                    es_beta = "esOF")
## 
## Attaching package: 'gridExtra'
## The following object is masked from 'package:dplyr':
## 
##     combine
Boundaries for benefit and harm (red lines) and boundaries for futility (blue lines) are shown on the four plots.

Figure 2.1: Boundaries for benefit and harm (red lines) and boundaries for futility (blue lines) are shown on the four plots.

To control for type-I-errors, it is assumed that one will stop the sequential meta-analysis if any of the alpha boundaries are crossed. When using binding futility boundaries, it will also be assumed that the meta-analysis stops when entering the futility area. If the meta-analysis does not stop when reaching the futility area but continues will the hypothesis testing of the null hypothesis be too optimistic (more than the set level of type-I-error). In the examples above, we have a too optimistic test. We can see this based on the boundaries printed below. The first print is the one-sided test with no futility. Here the boundaries are larger than 1.96 due to the multiple testing.

bound_none
## Boundaries for a 1-sided design with a type-I-error of 0.025 and type-II-error of 0.1.
## Futility is set to: none. Alpha-spending function: esOF.
## Beta-spending function: .
## 
## Timing and boundaries:
##  SMA_Timing Upper
##       0.509 2.963
##       0.764 2.359
##       1.018 2.014

The boundaries of our design with the binding futility will however have some boundaries less than 1.96. It is not true for all designs with binding futility to have boundaries below 1.96. The boundaries depends on the choice of spending function and timing of the interim analyses. But the example here shows that under binding futility we can encounter a boundary less than 1.96 (here 1.93). The reason is that if we have binding futility, the chance of a type-I-error becomes smaller as we will stop for futility. If the null hypothesis is true, the futility boundaries will catch it with a specific probability and hence reducing the risk of a false-positive, a type-I-error. This affects the boundaries as shown below.

bound_bind
## Boundaries for a 1-sided design with a type-I-error of 0.025 and type-II-error of 0.1.
## Futility is set to: binding. Alpha-spending function: esOF.
## Beta-spending function: esOF.
## 
## Timing and boundaries:
##  SMA_Timing Upper FutLower
##       0.527 2.963    0.299
##       0.790 2.359    1.251
##       1.053 1.963    1.963

So in this specific design, if we continue the meta-analysis without respecting the futility stopping boundaries, we will on average have a type-I-error larger than 5%. This means that scenarios should as the one on the left plot in Figure 2 must not occur under binding futility if one wants to state that the type-I-error is controlled.

Boundaries for benefit and harm (red lines) and boundaries for futility (blue lines) are shown on the 2 plots.

Figure 2.2: Boundaries for benefit and harm (red lines) and boundaries for futility (blue lines) are shown on the 2 plots.

2.2 Non-binding futility

To allow for crossing of the futility boundary without deciding to stop and also without compromising the type-I-error, one can use non-binding futility boundaries. We look into two none-binding designs:

bound_nbind <- boundaries(timing = c(0.5,0.75, 1), alpha  = 0.025, beta = 0.1, 
                    side = 1, futility = "non-binding", es_alpha = "esOF",
                    es_beta = "esOF")
bound_nbind2 <- boundaries(timing = c(0.5,0.75, 1), alpha  = 0.05, beta = 0.1, 
                    side = 2, futility = "non-binding", es_alpha = "esOF",
                    es_beta = "esOF")

The boundaries are shown on Figure 3.