Sample size and power

How many patients should a trial enrol? You decide that before the study starts, not after the data come in. A study that is too small can miss a real treatment effect and waste every participant's time. A study that is too large spends money and keeps people on placebo longer than needed. A sample size calculation turns a vague plan into a number you can defend, and most ethics committees and funders now ask to see one. This lesson works through the two commonest designs, comparing two means and comparing two proportions, and shows how to run each in R and Python.

Two kinds of error

A significance test can go wrong in two directions. You can reject the null hypothesis when it is true, or fail to reject it when it is false. Statisticians name these the type I error and the type II error.

The chance of a type I error is the significance level . Set it to 0.05 and you accept a 5% risk of calling a difference real when none exists. The chance of a type II error is : the probability of missing an effect that is genuinely there. The quantity you plan around is power, the chance of detecting a true effect of the size you care about.

A study with 90% power has a 10% chance of missing the target effect. Common choices are 80% or 90% power, with two-sided. Higher power costs more patients.

Four linked quantities

Sample size does not stand alone. It is tied to three other numbers, and the four move together:

• Sample size, the per group you are solving for.
• Effect size, the difference you want to be able to detect.
• Significance level , the type I error you will tolerate.
• Power, the chance of finding that effect if it is real.

Fix any three and the fourth is determined. At the planning stage you usually fix the effect size, , and power, then solve for . When the number of available patients is capped, you instead fix and the rest, then solve for power to see whether the study is worth running. The functions power.t.test in R and solve_power in Python work either way: leave one argument out and they return it.

Sample size for comparing two means

Picture a two-arm trial measuring a continuous outcome, such as birth weight or blood pressure. Two things drive the required size. The difference you want to detect, , and the spread of the outcome, the standard deviation . A smaller difference is harder to see against the noise, so it needs more people. A noisier outcome needs more people too. The per-group formula makes both effects explicit.

Here is the one-sided normal value for the power (1.28 for 90%, 1.64 for 95%) and is the two-sided value for the significance level (1.96 at 5%, 2.58 at 1%). The difference sits in the denominator squared, so halving the difference you chase roughly quadruples the sample. In practice you let software apply the t-distribution version, which adds a few patients over the normal formula.

Worked example 1: birth weight in a supplementation trial

An antenatal nutrition trial in rural Sabah will randomise pregnant women to a supplement or usual care. The team wants to detect a 0.25 kg increase in mean birth weight. Past records put the standard deviation of birth weight near 0.4 kg in both arms. They want 95% power and a strict 1% significance level. Plug in and solve for the per-group size.

delta <- 0.25
sd    <- 0.4

res <- # call power.t.test() leaving n out

res <- power.t.test(delta = delta, sd = sd, sig.level = 0.01, power = 0.95)
n   <- ceiling(res$n)
n
# 93  (per group, so about 186 in total)

# verify: feed n back in and read the power
power.t.test(n = n, delta = delta, sd = sd, sig.level = 0.01)$power
# about 0.952

import math
import pandas as pd
from statsmodels.stats.power import TTestIndPower
delta, sd = 0.25, 0.4
analysis = TTestIndPower()

n = # solve_power for nobs1

es = delta / sd
n  = math.ceil(analysis.solve_power(effect_size=es, alpha=0.01, power=0.95,
                                    ratio=1, alternative="two-sided"))
# verify: read the power back at this n
power = analysis.power(effect_size=es, nobs1=n, alpha=0.01,
                       ratio=1, alternative="two-sided")
print(pd.Series({"n_per_group": n, "total": 2 * n, "power": round(power, 3)}))
# n_per_group 93, total 186, power about 0.952

Both tools return about 93 women per group, near 186 in total. The verify step is the habit to keep: put the chosen back in and confirm the power lands where you asked.

How the difference and the variability move n

The formula says a smaller target difference inflates fast. Watching the numbers makes it concrete. Hold the standard deviation at 0.4 kg, the significance at 5%, and the power at 90%, then shrink the difference you want to detect.

sd     <- 0.4
deltas <- c(0.40, 0.25, 0.15)

n <- # ceiling of power.t.test n for each delta

n <- sapply(deltas, function(d)
       ceiling(power.t.test(delta = d, sd = sd, sig.level = 0.05, power = 0.90)$n))
data.frame(difference = deltas, n_per_group = n)
#   difference n_per_group
# 1       0.40          23
# 2       0.25          55
# 3       0.15         151

import numpy as np
import pandas as pd
from statsmodels.stats.power import TTestIndPower
sd = 0.4
deltas = [0.40, 0.25, 0.15]
analysis = TTestIndPower()

n = # ceiling of solve_power for each delta

n = [int(np.ceil(analysis.solve_power(effect_size=d / sd, alpha=0.05,
                                      power=0.90, ratio=1)))
     for d in deltas]
print(pd.DataFrame({"difference": deltas, "n_per_group": n}))
#    difference  n_per_group
# 0        0.40           23
# 1        0.25           55
# 2        0.15          151

Cutting the target difference from 0.40 kg to 0.15 kg pushes the per-group size from about 23 to about 151, roughly sixfold. Chasing a tiny effect, or measuring a noisy outcome, is expensive. This is why teams spend effort tightening measurement and choosing an effect size that is both real and worth detecting.

Sample size for comparing two proportions

When the outcome is yes or no (recovered or not, progressed or not), you compare two proportions, and . The per-group formula has the same shape: a numerator built from the two error values and the outcome variability, over the squared difference you want to detect.

Here is the average of the two proportions. As with means, the difference in the denominator governs the size: a small drop in event rate needs a large trial.

Worked example 2: a dengue fluid-management trial

A trial across Klinik Kesihatan sites compares a structured fluid protocol against standard care for dengue inpatients. Under standard care about 30% progress to warning signs (). The team hopes the new protocol cuts that to 20% (). They want 90% power at 5% two-sided significance.

p1 <- 0.30
p2 <- 0.20

res <- # call power.prop.test() leaving n out

res <- power.prop.test(p1 = p1, p2 = p2, sig.level = 0.05, power = 0.90)
n   <- ceiling(res$n)
n
# 392  (per group, about 784 in total)

# verify: read the power back at this n
power.prop.test(n = n, p1 = p1, p2 = p2, sig.level = 0.05)$power
# about 0.901

import math
import pandas as pd
from statsmodels.stats.proportion import proportion_effectsize
from statsmodels.stats.power import NormalIndPower
p1, p2 = 0.30, 0.20
analysis = NormalIndPower()

n = # solve_power using proportion_effectsize

h = proportion_effectsize(p1, p2)          # Cohen's h, the proportion effect size
n = math.ceil(analysis.solve_power(effect_size=h, alpha=0.05, power=0.90,
                                   ratio=1, alternative="two-sided"))
power = analysis.power(effect_size=h, nobs1=n, alpha=0.05,
                       ratio=1, alternative="two-sided")
print(pd.Series({"effect_h": round(h, 3), "n_per_group": n,
                 "total": 2 * n, "power": round(power, 3)}))
# effect_h 0.232, n_per_group about 391, total about 782, power about 0.901

R reports about 392 per group and Python about 391. The one-patient gap is the two tools using slightly different normal approximations, not an error in either. Round up and plan for roughly 392 per arm, near 784 in total before any dropout adjustment.

One-sided tests, and inflating for dropout

Two adjustments come up on almost every real calculation.

1. One-sided versus two-sided. A two-sided test asks whether the arms differ in either direction. A one-sided test asks only whether the new treatment beats the control. A one-sided test needs fewer patients for the same power, but reviewers are sceptical of it, because it cannot flag harm. Use two-sided by default and justify one-sided only when the opposite direction is genuinely irrelevant.
2. Loss to follow-up. The calculated is the number you need to analyse, not the number to enrol. Some participants withdraw or go missing. If you expect a fraction to drop out, enrol so the analysed sample still hits the target.

n_per_group <- 93
dropout     <- 0.15

n_enrol <- # inflate n_per_group for the dropout

n_enrol <- ceiling(n_per_group / (1 - dropout))
n_enrol
# 110  (enrol this many per group to analyse 93)

two_sided <- ceiling(power.t.test(delta = 0.25, sd = 0.4,
                                  sig.level = 0.05, power = 0.90)$n)
one_sided <- ceiling(power.t.test(delta = 0.25, sd = 0.4,
                                  sig.level = 0.05, power = 0.90,
                                  alternative = "one.sided")$n)
c(two_sided = two_sided, one_sided = one_sided)
# two_sided 55   one_sided about 44

import math
import pandas as pd
from statsmodels.stats.power import TTestIndPower
n_per_group, dropout = 93, 0.15
analysis = TTestIndPower()

n_enrol = # inflate n_per_group for the dropout

n_enrol = math.ceil(n_per_group / (1 - dropout))
two_sided = math.ceil(analysis.solve_power(effect_size=0.625, alpha=0.05,
                                           power=0.90, ratio=1,
                                           alternative="two-sided"))
one_sided = math.ceil(analysis.solve_power(effect_size=0.625, alpha=0.05,
                                           power=0.90, ratio=1,
                                           alternative="larger"))
print(pd.Series({"n_enrol_after_dropout": n_enrol,
                 "two_sided": two_sided, "one_sided": one_sided}))
# n_enrol_after_dropout 110, two_sided 55, one_sided about 45

The 15% dropout assumption turns a target of 93 analysed into 110 enrolled per arm. The one-sided test trims the count, which is exactly why it draws scrutiny: it buys power by ignoring one direction.

Common mistakes

• Forgetting the formula gives per-group n. For two means or two proportions, power.t.test and power.prop.test return the size of each arm. The total study is twice that. Reporting the per-group number as the whole study halves the planned recruitment.
• Powering on a difference you wish for, not one you expect. Picking an optimistic effect size shrinks on paper, then the real, smaller effect goes undetected. Base the difference on prior data or the smallest effect that would change practice.
• Reporting observed (post-hoc) power to defend a null result. "Our power was only 30%, so the negative finding is unsurprising" is circular. Observed power just restates the p-value. Use the confidence interval to show what the study could rule out.
• Skipping the dropout adjustment. Enrolling exactly the analysed leaves the study short once people withdraw, quietly dropping the real power below target.
• Quoting one scenario as if it were exact. The inputs are guesses. Run the calculation across a range of effect sizes and standard deviations, and report the spread, not a single magic number.

Tips

• State all four quantities in the protocol: the effect size, , the power, and the resulting per group. A reviewer should be able to reproduce your number.
• For a proportion outcome, the variability is largest near . If you are unsure of the control rate, a value closer to 0.5 gives a safe, slightly conservative .
• Compute a small table of against plausible effect sizes before you commit. It shows the funder the trade-off and protects you if recruitment is tight.
• Add the dropout inflation last, after the core calculation, so the analysed target stays visible: enrol to analyse .
• Default to two-sided tests. Reach for one-sided only when harm in the other direction is truly not of interest, and expect to defend it.