Bayesian statistics

A new targeted therapy reaches a small phase-2 trial at a Klang Valley oncology centre. Nine of twelve patients respond. The frequentist estimate is a 75% response rate, but the confidence interval is wide and the result ignores everything already known about drugs in this class. Bayesian statistics folds that outside knowledge in. You start with a prior belief about the response rate, let the trial update it, and end with a posterior you can read as a direct probability statement about the rate itself.

This lesson builds the Bayesian idea on the proportions and binomial machinery from Part E. We use a conjugate Beta-Binomial model, which keeps every step in closed form, so a posterior and a credible interval come straight from base R or scipy with no special sampler.

Prior, likelihood, posterior

Bayesian inference works with three pieces. The prior is your belief about the unknown parameter before this study, written as a probability distribution. The likelihood is what the data say: the probability of the observed counts for each possible value of the parameter. The posterior is your updated belief after seeing the data. Bayes' theorem is the rule that combines them.

Read the proportional sign as "rescale so the area is 1." The posterior is the prior reweighted by how well each parameter value explains the data. Where the data are strong, the likelihood dominates and the prior fades. Where the data are thin, the prior still carries weight.

Bayesian against frequentist

The split is about what probability describes. A frequentist treats the response rate as a fixed unknown and puts probability on the data. A p-value is the chance of data this extreme if the null were true. A confidence interval is a procedure that covers the fixed rate 95% of the time across repeated studies. A Bayesian puts probability on the parameter itself and can say "there is a 95% probability the rate lies in this range."

The credible interval has the plain-language meaning people often, and wrongly, attach to a confidence interval. That direct reading is the practical pull of the Bayesian approach.

The conjugate Beta-Binomial model

Our unknown is a response rate between 0 and 1. The natural prior for a probability is the Beta distribution, written . You can read its two numbers as prior pseudo-counts: imagined responders and imagined non-responders, with prior mean . A larger means a more confident prior.

The data are responders out of , a binomial likelihood. The Beta is conjugate to the binomial, which means the posterior is again a Beta with updated counts. Add the observed responders to and the non-responders to .

The posterior mean follows directly, and the credible interval is read straight off the Beta quantiles with qbeta in R or beta.ppf in Python.

Worked example 1: a small response-rate trial

Take the trial above: responders out of . Start with a weakly-informative prior , which is flat on 0 to 1 and says every rate is equally plausible before the data.

1. Update the counts: posterior .
2. Posterior mean: , close to the raw 0.75 because the flat prior barely pulls.
3. 95% credible interval: the 2.5% and 97.5% quantiles of , which run from 0.462 to 0.909.

You can now say there is a 95% probability the true response rate lies between about 46% and 91%, given this flat prior and these twelve patients.

a <- 1
b <- 1
r <- 9
n <- 12

post_a <- # fill in
post_b <- # fill in

post_a <- a + r
post_b <- b + n - r
post_mean <- post_a / (post_a + post_b)
ci <- qbeta(c(.025, .975), post_a, post_b)
grid <- seq(0, 1, length.out = 200)
plot(grid, dbeta(grid, post_a, post_b), type = "l",
     xlab = "Response rate", ylab = "Density")
lines(grid, dbeta(grid, a, b), lty = 2)   # prior, dashed
round(c(mean = post_mean, lower = ci[1], upper = ci[2]), 4)
#   mean  lower  upper
# 0.7143 0.4619 0.9091

import numpy as np
from scipy.stats import beta
import matplotlib.pyplot as plt
a, b, r, n = 1, 1, 9, 12

post_a = # fill in
post_b = # fill in

post_a, post_b = a + r, b + n - r
post_mean = post_a / (post_a + post_b)
ci = beta.ppf([0.025, 0.975], post_a, post_b)
grid = np.linspace(0, 1, 200)
plt.plot(grid, beta.pdf(grid, post_a, post_b), label="posterior")
plt.plot(grid, beta.pdf(grid, a, b), "--", label="prior")
plt.xlabel("Response rate"); plt.ylabel("Density"); plt.legend()
plt.show()
print(round(post_mean, 4), np.round(ci, 4))
# 0.7143 [0.4619 0.9091]

The prior matters, then it washes out

A flat prior is rarely all you know. Drugs in this class have historically responded near 30%, so a colleague proposes a informative skeptical prior : prior mean 0.30, worth 20 pseudo-patients. With the same twelve patients the posterior becomes , mean 0.469. The skeptical prior pulls the estimate down from 0.71 to 0.47, because twelve patients cannot overturn a prior worth twenty.

That pull is not permanent. As data accumulate the likelihood swamps the prior and the choice stops mattering. Run a far larger trial at the same 75% rate, 450 responders in 600, and the two priors give posterior means of 0.749 and 0.735. The gap of 0.25 at twelve patients has shrunk to about 0.01. This is the washout: with enough data a Bayesian and a frequentist analysis converge, and a weakly-informative prior leaves almost no fingerprint.

Worked example 2: a stronger prior, and the washout

Compare the two priors on the same data, then repeat with fifty times the data to watch the difference fade.

post_mean <- function(a, b, r, n) (a + r) / (a + b + n)

small_weak <- # fill in with post_mean()

round(c(small_weak = post_mean(1,  1,   9,  12),
        small_info = post_mean(6, 14,   9,  12),
        big_weak   = post_mean(1,  1, 450, 600),
        big_info   = post_mean(6, 14, 450, 600)), 4)
# small_weak small_info   big_weak   big_info
#     0.7143     0.4688     0.7492     0.7355
round(qbeta(c(.025, .975), 6 + 9, 14 + 12 - 9), 4)   # informative, n = 12
# 0.3015 0.6397

import numpy as np
from scipy.stats import beta
def post_mean(a, b, r, n):
    return (a + r) / (a + b + n)

small_weak = # fill in with post_mean()

for label, args in [("small_weak", (1,  1,   9,  12)),
                    ("small_info", (6, 14,   9,  12)),
                    ("big_weak",   (1,  1, 450, 600)),
                    ("big_info",   (6, 14, 450, 600))]:
    print(label, round(post_mean(*args), 4))
# small_weak 0.7143
# small_info 0.4688
# big_weak 0.7492
# big_info 0.7355
print(np.round(beta.ppf([0.025, 0.975], 6 + 9, 14 + 12 - 9), 4))  # informative, n = 12
# [0.3015 0.6397]

When Bayesian methods earn their place

The Bayesian machinery is not always worth the extra assumptions, but three situations in medical statistics suit it well.

• Small samples. A twelve-patient phase-2 study gives a frequentist interval too wide to act on. A defensible prior from earlier trials sharpens the estimate without waiting for data you do not have.
• Real prior evidence. When earlier trials, registries, or a known mechanism carry information, a prior puts it to work instead of throwing it away.
• Decision-making. A posterior answers the question a clinician actually asks: "What is the probability this drug beats standard care?" A p-value cannot, because it treats the effect as fixed.

For larger studies with no strong prior, the two approaches agree and the simpler frequentist analysis is fine. Conjugate models like the Beta-Binomial cover one parameter cleanly. Richer models need simulation methods such as Markov chain Monte Carlo, which sample the posterior when no closed form exists, but the idea of prior, likelihood, and posterior carries straight over.

Common mistakes

• Reading a credible interval as a confidence interval. They answer different questions. The credible interval is a probability statement about the parameter; the confidence interval is a property of the procedure across repeated studies. Quote each with its correct meaning.
• Hiding the prior. A posterior is only as credible as the prior behind it. Report the prior and, for a small sample, show the result under a weakly-informative prior so readers see its influence.
• Calling a Beta(1, 1) prior "no prior." Flat is still a choice. On other scales, such as the log odds, a uniform prior on the probability is not flat, and a prior that looks harmless can sway a tiny sample.
• Updating the counts the wrong way. Responders add to and non-responders to . Adding to instead of inflates the rate. Check that the posterior counts sum to .
• Treating the posterior mean as the whole story. With a skewed Beta the mean, median, and mode differ. For a small trial report the credible interval, not just the point.

Tips

• Sanity-check a Beta prior by its pseudo-count . Compare it to your real sample size: if it is larger, the prior will lead, so make sure that is what you intend.
• Get the credible interval straight from the quantile function: qbeta(c(.025, .975), a + r, b + n - r) in R, beta.ppf([.025, .975], a + r, b + n - r) in Python. No normal approximation is needed, and the interval stays inside 0 to 1.
• Plot the prior and posterior on one set of axes. Seeing the curve shift, and tighten, makes the update concrete in a way the numbers alone do not.
• When you expect pushback on a prior, run a weakly-informative prior alongside the informative one and report both. Agreement is reassuring; a large gap tells you the data are too thin to settle the question.
• For two-arm trials, the same conjugate trick gives a Beta posterior for each arm, and you can read the probability that one rate beats the other directly from samples of the two posteriors.