How to compute confidence intervals for power estimates?

Summary

The Mediana R package uses a simulation-based method to estimate statistical power in clinical trials. Since the power level is estimate, it is natural to compute a confidence interval around the resulting estimate.

A simple example will be used to illustrate how to compute confidence intervals of this kind. We will consider the case study used in the FAQ page that focused on the calculation the required sample size in a trial. Please refer to the case study before reading the rest of this section.

Simulation results

To facilitate the review of simulation results produced by the CSE function in this case study, the estimated power values are presented in the following table.

Sample size scenario	Sample size per arm	Marginal power (%)
1	55	84.7
2	60	87.7
3	65	90.0
4	70	92.1
5	75	93.7

Confidence intervals

The results presented above were generated using 10,000 simulation runs. The estimated power is simply the result of estimating a proportion, therefore, the corresponding confidence interval can be derived using the following normal approximation:

\[\hat{p} \pm z_{\alpha} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}\]

where $\hat{p}$ is the estimated power, $z_{\alpha}$ is the $1-\alpha/2$ quantile of the standard normal distribution and $n$ is the number of simulation runs.

The following function can be called to compute a confidence interval for the estimated power from the simulation results returned by the CSE function.

############################################################################################################################
# Function: PowerConfidenceInterval
# Argument: CSE (object returned by the CSE function) and covprob (coverage probability)
# Description: Compute binomial confidence interval based on normal approximation.
PowerConfidenceInterval = function(CSE, covprob) {

  # Error check
  if (class(CSE) != "CSE") stop("PowerConfidenceInterval: a CSE object must be used in the CSE argument.")
  results = CSE$simulation.results
  n.sims = CSE$sim.parameters$n.sims

  if (covprob <= 0 | covprob >=1) stop("PowerConfidenceInterval: covprob parameter must lies between 0 and 1.")

  q = qnorm(1-(1-covprob)/2)
  power = results$result


  ci_l = pmax(0, round(power - q * sqrt(power* (1 - power) / n.sims), nchar(n.sims)))
  ci_u = pmin(1, round(power + q * sqrt(power* (1 - power) / n.sims), nchar(n.sims)))

  CSE$simulation.results = cbind(results,
                                 ci_l = ci_l,
                                 ci_u = ci_u)
  return(CSE)

}
# End of PowerConfidenceInterval

This function requires two arguments, the first one (CSE) is the object generated by the CSE function and the second one (ci) is the level of the confidence interval which must lie between 0 and 1.

It is shown below how to invoke this function to compute a confidence interval for the estimated power in this case study.

# Compute 95% confidence interval
case.study1.results = PowerConfidenceInterval(CSE = case.study1.results,
                                              ci = 0.95)
summary(case.study1.results)

A summary of the results is presented below.

Sample size scenario	Sample size per arm	Marginal power (%)	95% confidence interval (%)
1	55	84.7	[83.95;85.37]
2	60	87.7	[87.09;88.37]
3	65	90.0	[89.47;90.65]
4	70	92.1	[91.52;92.58]
5	75	93.7	[93.18;94.14]

Download

Click on the icons below to download the R code used in this case study: