Two proportions: risk ratio, odds ratio, risk difference, and confidence intervals

You have two groups and a yes/no outcome, and you want one number that says how much the exposure changed the outcome. There are three honest answers. They measure different things, and they each come with their own confidence interval and their own hypothesis test. This lesson works all three on one dataset, then shows where the test machinery quietly differs from the interval machinery. That gap is the part most courses skip, and it is the part that trips people up.

Here is the running example. A cohort of 400 arthritis patients was followed for one year. 200 took an NSAID (a non-steroidal anti-inflammatory drug) and 200 did not. The outcome is a gastrointestinal bleed, written GI bleed. The counts came out as below.

The 2x2 table and three measures of effect

By convention the exposed group is the top row and the outcome event is the left column. We label the four inner cells with letters so the formulae stay short. Call them reading across the rows: exposed with the event, exposed without, unexposed with the event, unexposed without. Some textbooks write the event counts as and and the non-event counts as and . Same cells, different letters.

The risk in a group is the number who bled over the group total. For NSAID users it is , so 15 percent. For non-users it is , so 6 percent. Write these two numbers down first, because every measure below is built from them. From the same table you can read three measures of effect: the risk difference, the risk ratio, and the odds ratio. They answer different questions, so they are not interchangeable.

Risk difference

The risk difference (RD) is the plain subtraction. It says, on the absolute scale, how much the exposure raised the chance of the event.

Here . Taking the NSAID raised the absolute risk of a bleed by 9 percentage points. The absolute scale is what a patient actually feels, so do not skip it in favour of the ratio.

Flip the risk difference over and you get the number needed to harm (NNH): how many people you expose before you cause one extra bad event. It is , here , so round up to 12. Treat about 12 patients with the NSAID for a year and you expect one extra bleed that would not have happened otherwise. When the exposure is protective the same arithmetic gives the number needed to treat (NNT): how many you treat to prevent one event. Same formula, opposite sign of effect.

An NNH or NNT means nothing without its time frame. Ours is one extra bleed per 12 patients per year of NSAID use. Quote the follow-up period every time, or the number cannot be read.

Confidence interval for the risk difference

A point estimate without an interval is half an answer. The risk difference can sit anywhere on the real line, so it uses the normal approximation directly. Its standard error combines the two individual standard errors, one per group, each estimated from that group's own risk.

Plug in the numbers: . The 95 percent interval is the estimate plus or minus 1.96 standard errors, which gives . We are 95 percent confident the NSAID raises the absolute bleed risk by between about 3.1 and 14.9 percentage points. The interval excludes 0, so the difference is significant at the 5 percent level. R's prop.test applies a continuity correction by default and returns a slightly wider interval, about (0.026, 0.154). The key fact about this standard error: it is built from the two separate risks. Hold on to that, because the hypothesis test uses a different one.

Testing the difference: the pooled standard error

To test whether the two risks differ at all, you use a z-test. Here is the subtlety. The null hypothesis says the two groups share one common true risk. If that is true, it is wrong to estimate the standard error from the two separate sample risks, because under the null there is only one risk to estimate. So you pool the groups: combine all the events over all the people to get a single proportion , and build the standard error from that.

The z statistic is the risk difference over this pooled standard error.

Our pooled proportion is . The pooled standard error is , which is close to but not equal to the 0.0303 from the interval. The z is , giving a two-sided p-value near 0.003. There is clear evidence the risks differ. Square that z and you get the Pearson chi-squared statistic for the same table, which is why prop.test prints a chi-squared value. The z-test and the chi-squared test are the same test. When any expected cell count is small (say under 5), add a continuity correction or switch to Fisher's exact test (covered in the next lesson).

Risk ratio

The risk ratio (RR), also called relative risk, divides one risk by the other. It says how many times more likely the event is in the exposed group.

Here . NSAID users were 2.5 times as likely to bleed. An RR of 1 means no association, above 1 means raised risk, below 1 means protection, and the further from 1 the stronger the association. A risk ratio is always positive. The ratio reads the same whatever the baseline rate, which is why epidemiologists reach for it: 2.5 times the risk carries the same meaning whether the baseline is 6 percent or 0.6 percent. The risk difference would not. Double the follow-up time and the risk difference roughly doubles, while the risk ratio stays put.

Risk ratio: confidence interval and test on the log scale

Ratios cannot use the plain estimate-plus-or-minus-standard-error recipe. That recipe can hand back a negative lower limit, which is impossible for a quantity that is always positive. The fix is to work on the log scale, where the sampling distribution is roughly symmetric, then exponentiate the two limits back. The standard error of the log risk ratio comes from the delta method, a technique for getting the standard error of a transformed quantity from the original. For the log risk ratio it is:

This is the same quantity as the textbook form , just written with risks instead of counts. Then build the interval on the log scale and exponentiate.

For our table, and . The log interval is , and exponentiating gives a 95 percent CI for the risk ratio of about 1.32 to 4.74. To test that the RR equals 1, note that this is the same as the log RR being 0, so the z statistic is , p near 0.005. The interval excludes 1 and the test rejects, as they should agree.

Odds ratio and the cross-product

The odds of a bleed in a group is the number who bled divided by the number who did not. For NSAID users that is ; for non-users it is . The odds ratio (OR) divides one by the other, and after the algebra cancels it reduces to the cross-product of the table.

Plug the cells in: . Multiply the two cells on the main diagonal, divide by the product of the other two. That is why it is called the cross-product ratio. An OR of 1 means no association, the minimum is 0, and there is no upper bound. Unlike the risk ratio, the odds ratio is not capped by the baseline risk, which matters in the next section.

Odds ratio: confidence interval and test on the log scale

The odds ratio uses exactly the same log machine as the risk ratio. Only the standard error formula changes. For the log odds ratio it is Woolf's formula, which uses all four cells.

For our table, and . The log interval is ; exponentiating gives a 95 percent CI of about 1.37 to 5.57. The test that the OR equals 1 uses , p near 0.005. This z-test gives the same answer as a simple logistic regression of the outcome on the exposure, which is one reason the odds ratio dominates modelling.

When the OR approximates the RR, and when it exaggerates

Our OR is 2.76 while the RR is 2.5. The odds ratio is always further from 1 than the risk ratio, and the gap grows as the outcome gets more common. The reason is simple: when an event is rare, the odds of it (events over non-events) are close to the risk of it (events over total), so OR and RR nearly match. A bleed risk of 6 to 15 percent is not rare, so the two have drifted apart.

The trap is reading an odds ratio out loud as if it were a risk ratio. Saying "NSAID users are 2.76 times as likely to bleed" is wrong; that is the odds ratio, and the likelihood multiplier is 2.5. For a common outcome the error is large. If the unexposed risk were 0.5 and the OR were 2, the actual RR would only be 1.33, because a risk cannot exceed 1 and so the risk ratio is constrained while the odds ratio is not. Report an OR as an OR.

Why case-control studies use the OR, not the RR

In a cohort study like ours you follow fixed groups forward and count events, so you can measure risk in each group directly. A case-control study runs backwards. You recruit people who already have the disease (cases) and a sample of people who do not (controls), then look back at exposure. Because you decide how many controls to enrol, the proportion of diseased people in your sample is an artefact of your sampling design, not the real risk. Risk, risk difference, and risk ratio cannot be estimated.

The odds ratio survives this. The cross-product has a symmetry: the odds ratio for disease (odds of disease in exposed versus unexposed) equals the odds ratio for exposure (odds of exposure in cases versus controls). So it returns the same answer whether you sampled on disease or on exposure. That invariance is exactly what a case-control design needs, and it is the historical reason the OR became the default measure for binary outcomes in epidemiology, even when the outcome is rare and the RR would have been easier to read.

Building the 2x2 and running it in code

The first block computes the three effect measures straight from the table. Run it, then read the printed RR, RD, and OR back against the numbers above.

# exposure in rows, outcome in columns
m <- matrix(c(30, 170, 12, 188), nrow = 2, byrow = TRUE)
rownames(m) <- c("NSAID", "Non-user")
colnames(m) <- c("Bleed", "NoBleed")

r1 <- m[1,1] / sum(m[1,])
r0 <- m[2,1] / sum(m[2,])
# fill in RR, RD, OR

r1 <- m[1,1] / sum(m[1,])   # 0.15
r0 <- m[2,1] / sum(m[2,])   # 0.06
RR <- r1 / r0                            # 2.5
RD <- r1 - r0                            # 0.09
OR <- (m[1,1]*m[2,2]) / (m[1,2]*m[2,1])  # 2.7647
c(r1 = r1, r0 = r0, RR = RR, RD = RD, OR = OR)

import numpy as np
# rows = exposure, cols = outcome (bleed, no bleed)
m = np.array([[30, 170],
              [12, 188]])

r1 = m[0,0] / m[0].sum()
r0 = m[1,0] / m[1].sum()
# fill in RR, RD, OR

r1 = m[0,0] / m[0].sum()   # 0.15
r0 = m[1,0] / m[1].sum()   # 0.06
RR = r1 / r0                            # 2.5
RD = r1 - r0                            # 0.09
OR = (m[0,0]*m[1,1]) / (m[0,1]*m[1,0])  # 2.7647
print(dict(r1=r1, r0=r0, RR=RR, RD=RD, OR=OR))

The second block puts the pooled-versus-unpooled point into code. The risk-difference interval comes from prop.test, which uses the unpooled standard error (with a continuity correction). The z-test built by hand uses the pooled standard error. Run both and compare the two standard errors.

a <- 30; b <- 170; cc <- 12; d <- 188
n1 <- a + b; n0 <- cc + d
r1 <- a/n1; r0 <- cc/n0

# interval (unpooled SE, via prop.test)
prop.test(c(a, cc), c(n1, n0))$conf.int
# now build the pooled-SE z by hand

prop.test(c(a, cc), c(n1, n0))$conf.int   # ~ (0.026, 0.154), unpooled

se_unpooled <- sqrt(r1*(1-r1)/n1 + r0*(1-r0)/n0)   # 0.0303, used for the CI

p <- (a + cc) / (n1 + n0)                           # pooled proportion 0.105
se_pooled <- sqrt(p*(1-p)*(1/n1 + 1/n0))            # 0.0307, used for the test
z <- (r1 - r0) / se_pooled                          # 2.94
c(se_unpooled = se_unpooled, se_pooled = se_pooled, z = z,
  p_value = 2*(1 - pnorm(abs(z))))

import numpy as np
from scipy.stats import norm
a, b, c, d = 30, 170, 12, 188
n1, n0 = a + b, c + d
r1, r0 = a/n1, c/n0

# build se_unpooled (CI) and se_pooled (test)
# then form the interval and the z

se_unpooled = np.sqrt(r1*(1-r1)/n1 + r0*(1-r0)/n0)   # 0.0303, for the CI
rd_ci = (r1 - r0) + np.array([-1, 1]) * 1.96 * se_unpooled  # ~ (0.031, 0.149)

p = (a + c) / (n1 + n0)                               # pooled 0.105
se_pooled = np.sqrt(p*(1-p)*(1/n1 + 1/n0))            # 0.0307, for the test
z = (r1 - r0) / se_pooled                             # 2.94
pval = 2 * (1 - norm.cdf(abs(z)))
print(rd_ci, se_unpooled, se_pooled, z, pval)

The third block builds the log-method confidence intervals and tests for the risk ratio and the odds ratio, computed by hand so every piece is visible.

a <- 30; b <- 170; cc <- 12; d <- 188
n1 <- a + b; n0 <- cc + d
RR <- (a/n1) / (cc/n0); OR <- (a*d) / (b*cc)
z975 <- qnorm(0.975)   # 1.96

# se_lnRR uses counts; se_lnOR is Woolf (all four cells)

se_lnRR <- sqrt(1/a - 1/n1 + 1/cc - 1/n0)   # 0.327
rr_ci <- exp(log(RR) + c(-1, 1) * z975 * se_lnRR)   # ~ (1.32, 4.74)
z_rr <- log(RR) / se_lnRR                            # 2.81

se_lnOR <- sqrt(1/a + 1/b + 1/cc + 1/d)     # Woolf, 0.358
or_ci <- exp(log(OR) + c(-1, 1) * z975 * se_lnOR)   # ~ (1.37, 5.57)
z_or <- log(OR) / se_lnOR                            # 2.84
list(rr_ci = rr_ci, z_rr = z_rr, or_ci = or_ci, z_or = z_or)

import numpy as np
from scipy.stats import norm
a, b, c, d = 30, 170, 12, 188
n1, n0 = a + b, c + d
RR = (a/n1) / (c/n0); OR = (a*d) / (b*c)
z975 = norm.ppf(0.975)   # 1.96

# se_lnRR uses counts; se_lnOR is Woolf (all four cells)

se_lnRR = np.sqrt(1/a - 1/n1 + 1/c - 1/n0)   # 0.327
rr_ci = np.exp(np.log(RR) + np.array([-1, 1]) * z975 * se_lnRR)  # (1.32, 4.74)
z_rr = np.log(RR) / se_lnRR                                       # 2.81

se_lnOR = np.sqrt(1/a + 1/b + 1/c + 1/d)     # Woolf, 0.358
or_ci = np.exp(np.log(OR) + np.array([-1, 1]) * z975 * se_lnOR)   # (1.37, 5.57)
z_or = np.log(OR) / se_lnOR                                       # 2.84
print(rr_ci, z_rr, or_ci, z_or)

Worked example: a smoking and lung-cancer cohort

Now take a second dataset where the outcome is more lopsided, to watch the OR pull away from the RR. A cohort study followed 1,000 smokers and 2,000 non-smokers for a year and counted lung-cancer diagnoses.

Work the four measures in order.

1. Risks. , .
2. Risk difference. ; NNH per year.
3. Risk ratio. . Smokers had nine times the risk.
4. Odds ratio. .

The OR (9.79) is noticeably larger than the RR (9.0). At a 9 percent risk in the exposed group, the odds and the risk are no longer interchangeable, so the cross-product runs ahead. If you reported "smokers had a 9.79-fold higher risk" you would overstate the effect. The risk is multiplied by 9, not 9.79.

For the odds ratio interval, Woolf's standard error is , and . The log interval is , so exponentiating gives a 95 percent CI of about 5.99 to 16.0. For comparison the risk-ratio interval comes out at roughly 5.58 to 14.5. Both exclude 1 by a wide margin, so the smoking association is strong however you measure it. The block below computes the measures and draws the two risks side by side.

a <- 90; b <- 910; cc <- 20; d <- 1980
n1 <- a + b; n0 <- cc + d
r1 <- a/n1; r0 <- cc/n0

RR <- r1 / r0; OR <- (a*d) / (b*cc)
c(r1 = r1, r0 = r0, RR = RR, OR = OR)
# then add the OR confidence interval and a barplot

RR <- r1 / r0                  # 9.0
OR <- (a*d) / (b*cc)           # 9.79
se_lnOR <- sqrt(1/a + 1/b + 1/cc + 1/d)
or_ci <- exp(log(OR) + c(-1, 1) * qnorm(0.975) * se_lnOR)  # ~ (5.99, 16.0)
print(or_ci)

barplot(c(Smoker = r1, `Non-smoker` = r0), ylim = c(0, 0.1),
        ylab = "Risk of lung cancer",
        col = c("#B22222", "#4682B4"),
        main = "Lung-cancer risk by smoking status")

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm
a, b, c, d = 90, 910, 20, 1980
n1, n0 = a + b, c + d
r1, r0 = a/n1, c/n0

RR = r1 / r0; OR = (a*d) / (b*c)
print(r1, r0, RR, OR)
# then add the OR confidence interval and a bar chart

RR = r1 / r0                  # 9.0
OR = (a*d) / (b*c)            # 9.79
se_lnOR = np.sqrt(1/a + 1/b + 1/c + 1/d)
or_ci = np.exp(np.log(OR) + np.array([-1, 1]) * norm.ppf(0.975) * se_lnOR)
print(or_ci)                  # ~ (5.99, 16.0)

plt.bar(["Smoker", "Non-smoker"], [r1, r0], color=["#B22222", "#4682B4"])
plt.ylim(0, 0.1)
plt.ylabel("Risk of lung cancer")
plt.title("Lung-cancer risk by smoking status")
plt.show()

# reference only, not runnable in this lesson
library(janitor)
library(gtsummary)
library(epiR)

patients %>%
  tabyl(nsaid_use, gi_bleed) %>%      # build the 2x2 from raw rows
  adorn_totals(c("row", "col"))

patients %>%
  tbl_summary(by = gi_bleed, include = nsaid_use) %>%
  add_p()                             # chi-squared / Fisher p-value

# RR, OR and risk difference with CIs in one call
epi.2by2(table(patients$nsaid_use, patients$gi_bleed), method = "cohort.count")

Common mistakes

• Calling an odds ratio a risk multiplier. Our NSAID OR is 2.76 but the risk only multiplied by 2.5; the smoking OR is 9.79 but the risk multiplied by 9. For common outcomes the OR runs ahead of the RR. Always name which measure you are quoting.
• Using the interval's standard error for the test. The risk-difference CI uses the two separate risks (unpooled); the z-test uses the pooled proportion under the null. They are different formulas. Mixing them gives a wrong p-value.
• Building a normal CI for a ratio. Plain estimate plus or minus 1.96 standard errors on an RR or OR can return a negative lower limit. Always log, build the interval, then exponentiate.
• Reporting only the relative effect. A doubling of a tiny risk is still a tiny risk. Pair every RR or OR with the risk difference or the NNH so the absolute size is visible.
• Computing risk in a case-control study. The disease proportion there is set by your sampling, so RR and RD are meaningless. Report the OR.
• Quoting an NNH or NNT with no time frame. "One extra bleed per 12 patients" is unreadable until you add "per year".