Confounding and stratification

You compare an exposure against an outcome, you get an odds ratio, and it looks convincing. Before you believe it, ask one question: is something else different between the exposed and the unexposed that also drives the outcome? If yes, your unadjusted estimate is part exposure effect and part the effect of that other thing. That other thing is a confounder, and this lesson is about spotting it, removing it by stratification, and combining the strata into one adjusted estimate with the Mantel-Haenszel method. It assumes you can already read a 2x2 table, an odds ratio, and a chi-squared P-value.

What a confounder is

A confounder is a third variable that gets tangled into an exposure-outcome comparison and distorts it. Write the exposure as E, the outcome (disease) as D, and the candidate confounder as C. For C to confound the E to D association it must meet three conditions at once.

• It is associated with the exposure. C is distributed differently in the exposed and the unexposed groups.
• It affects the outcome on its own. C is a risk factor for D even among people who are not exposed.
• It does not sit on the causal path from E to D. E must not cause C which then causes D.

The third condition is the one people forget. If smoking during pregnancy lowers a baby's birth weight, and you are studying whether low income lowers birth weight, smoking is on the path (low income leads to more smoking leads to lower weight). Adjusting for it would wrongly erase part of the income effect you set out to measure. A confounder distorts; a mediator transmits. They are not the same, and you adjust only for the first.

How confounding distorts the crude estimate

Take long-term NSAID use as the exposure and an upper gastrointestinal bleed as the outcome, in 800 arthritis patients followed for a year. The crude table ignores everything else.

The crude odds ratio is (50 × 470) / (250 × 30) = 23500 / 7500 = 3.13. NSAID users look to have more than triple the odds of a bleed. Now split the same 800 patients by age, because older patients both use NSAIDs more (for arthritis) and bleed more (for separate reasons). Age is associated with the exposure and is a risk factor for the outcome, so it is a candidate confounder.

Within the young the odds ratio is (10 × 380) / (90 × 20) = 2.11. Within the old it is (40 × 90) / (160 × 10) = 2.25. Both strata sit near 2.1, yet the crude estimate was 3.13. The gap is confounding. The exposed group is loaded with older patients (200 of 300 NSAID users are 60+, against 100 of 500 non-users), and older patients bleed more whatever they take. That extra age risk leaked into the crude odds ratio and inflated it.

Stratify, then combine: the Mantel-Haenszel odds ratio

Stratification means splitting the data into subgroups (strata) that share one value of the confounder, so that within a stratum the confounder cannot vary and cannot distort. You compute a separate odds ratio in each stratum.

Here and are the diseased and healthy counts among the exposed in stratum i, and , the same among the unexposed. If the strata agree, you want one summary number that controls for C. The naive move is to average the stratum odds ratios. The better move is a weighted average that trusts the bigger strata more.

The Mantel-Haenszel method picks the weight , where is the stratum total. That weight is exactly the denominator of the stratum odds ratio, so the messy weighted average collapses into a clean ratio of two sums.

For the NSAID data, work out Q and R term by term. Young: and . Old: and . So Q = 7.6 + 12.0 = 19.6 and R = 3.6 + 5.33 = 8.93. The adjusted odds ratio is 19.6 / 8.93 = 2.19. The age-driven inflation is gone, and the summary lands between the two stratum values, as a weighted average must.

# each 2x2: rows = NSAID use, cols = bleed / no bleed
young <- matrix(c(10, 20, 90, 380), nrow = 2)
old   <- matrix(c(40, 10, 160, 90), nrow = 2)
arr   <- array(c(young, old), dim = c(2, 2, 2))

mantelhaen.test(arr, correct = FALSE)

mantelhaen.test(arr, correct = FALSE)
# common odds ratio 2.19, 95% CI 1.27 to 3.80
# Mantel-Haenszel X-squared = 8.24, df = 1, p = 0.0041
(50 * 470) / (250 * 30)   # crude, ignoring age: 3.13, inflated

import numpy as np, pandas as pd
from statsmodels.stats.contingency_tables import StratifiedTable
young = np.array([[10, 90], [20, 380]])   # rows: NSAID / non, cols: bleed / no
old   = np.array([[40, 160], [10, 90]])
st = StratifiedTable([young, old])

print(round(st.oddsratio_pooled, 3))   # Mantel-Haenszel OR

print(round(st.oddsratio_pooled, 3))                 # 2.194
print(np.round(st.oddsratio_pooled_confint(), 3))    # 1.27 .. 3.80
print(st.test_null_odds(correction=False))           # chi2 8.24, p 0.0041
print((50 * 470) / (250 * 30))                       # crude: 3.13, inflated

Is the adjustment real? The Mantel-Haenszel chi-squared

An adjusted odds ratio of 2.19 still needs a test against the null that the true value is 1. The Mantel-Haenszel chi-squared compares the exposed cases you observed against the number expected under no association, pooled across strata, and divides the squared gap by its variance.

The expected count in each stratum is and the variance is , both built from the margins alone. Young: , . Old: , . Observed exposed cases O = 10 + 40 = 50, expected E = 6.0 + 33.33 = 39.33, variance V = 13.81. So on 1 degree of freedom, P = 0.0041. The test keeps 1 degree of freedom no matter how many strata you pool. The 95% confidence interval for the odds ratio, 1.27 to 3.80, excludes 1 and agrees. After adjusting for age, NSAID use is associated with bleeding.

ors <- c(Young = 2.11, Old = 2.25, Crude = 3.13, "MH adj" = 2.19)

barplot(ors, ylab = "Odds ratio")

cols <- c("grey70", "grey70", "firebrick", "steelblue")
barplot(ors, col = cols, ylim = c(0, 3.5), ylab = "Odds ratio",
        main = "Crude is inflated; MH sits with the strata")
abline(h = 1, lty = 2)               # the null
abline(h = 2.19, lty = 3, col = "steelblue")

import numpy as np
import matplotlib.pyplot as plt
labels = ["Young", "Old", "Crude", "MH adj"]
ors = [2.11, 2.25, 3.13, 2.19]

plt.bar(labels, ors)
plt.ylabel("Odds ratio")
plt.show()

cols = ["grey", "grey", "firebrick", "steelblue"]
plt.bar(labels, ors, color=cols)
plt.axhline(1, ls="--", color="black")     # the null
plt.axhline(2.19, ls=":", color="steelblue")
plt.ylabel("Odds ratio")
plt.title("Crude is inflated; MH sits with the strata")
plt.show()

Effect modification is not confounding

Pooling with Mantel-Haenszel rests on one assumption: the true odds ratio is the same in every stratum, and the strata differ only by sampling noise. When the exposure genuinely acts differently across levels of C, that assumption fails. Then C is an effect modifier, the effect is said to show interaction or heterogeneity, and a single summary hides the real story rather than cleaning it up.

Consider oral contraceptive (OC) use and venous thromboembolism (VTE), split by smoking. The numbers below are illustrative.

Among non-smokers the odds ratio is (20 × 480) / (480 × 20) = 1.0, with a 95% interval of about 0.53 to 1.88. Among smokers it is (60 × 185) / (140 × 15) = 5.29, interval about 2.88 to 9.70. Those intervals do not overlap. OC use does little on its own but multiplies the clotting risk in smokers. The Mantel-Haenszel summary here is 2.52, a number that describes neither group. Reporting it alone would be the wrong call.

Test the assumption before you pool. The chi-squared test of heterogeneity asks whether the stratum odds ratios scatter further from the pooled value than noise allows. One common form weights each squared log-ratio gap by its precision.

Here and . For these data the statistic is 13.86 on 1 degree of freedom, P = 0.0002. That is strong evidence the OC effect really differs by smoking. So you do not report 2.52. You report each stratum: roughly no effect in non-smokers, a five-fold effect in smokers.

nonsmk <- matrix(c(20, 20, 480, 480), nrow = 2)   # OR = 1.0
smk    <- matrix(c(60, 15, 140, 185), nrow = 2)   # OR = 5.29
arr2   <- array(c(nonsmk, smk), dim = c(2, 2, 2))

mantelhaen.test(arr2, correct = FALSE)$estimate   # pooled OR

mantelhaen.test(arr2, correct = FALSE)$estimate    # 2.52 -- describes neither group
or  <- c((20*480)/(480*20), (60*185)/(140*15))     # 1.0 and 5.29
v   <- c(1/20+1/480+1/20+1/480, 1/60+1/140+1/15+1/185)
w   <- 1 / v
lp  <- sum(w * log(or)) / sum(w)
chi <- sum(w * (log(or) - lp)^2)
c(chi = chi, p = pchisq(chi, df = 1, lower.tail = FALSE))  # 13.86, p 0.0002

import numpy as np, pandas as pd
from statsmodels.stats.contingency_tables import StratifiedTable
nonsmk = np.array([[20, 480], [20, 480]])   # OR 1.0
smk    = np.array([[60, 140], [15, 185]])   # OR 5.29
st2 = StratifiedTable([nonsmk, smk])

print(round(st2.oddsratio_pooled, 2))   # pooled OR -- check homogeneity first

print(round(st2.oddsratio_pooled, 2))      # 2.52 -- describes neither group
print(st2.test_equal_odds())               # Breslow-Day test, p < 0.001
print((20*480)/(480*20), (60*185)/(140*15))   # report each: 1.0 and 5.29

When stratification runs out

Stratification is exact, transparent, and easy to defend, which is why it is the right first tool. It also has a hard ceiling. The trouble is the number of strata. Control for age (4 bands), sex (2), and diabetes (2) at once and you already have 4 × 2 × 2 = 16 strata. Add another confounder and the count multiplies again. Spread a few hundred patients across that many tables and most cells go to zero or one. The Mantel-Haenszel odds ratio survives sparse strata, but the stratum-specific estimates and the heterogeneity test fall apart, and you can no longer adjust for a continuous confounder like exact age without chopping it into bands and losing information.

That ceiling is the reason logistic regression exists. A regression model adjusts for several confounders at once, keeps continuous variables continuous, and lets you test interaction terms directly, all without slicing the data into ever-smaller tables. The Mantel-Haenszel odds ratio and a logistic model with one binary exposure and one stratifying variable give nearly the same adjusted estimate, so regression is the natural extension once one or two confounders become several. You meet it in the logistic-regression module next.

Common mistakes

• Adjusting for a mediator. If the exposure causes C and C causes the outcome, C is on the causal path. Stratifying on it removes part of the real effect. Confirm C is not a consequence of E before you adjust.
• Pooling without testing homogeneity. A Mantel-Haenszel summary across heterogeneous strata is a number that fits no one, like the 2.52 above. Run the heterogeneity or Breslow-Day test, and scan the stratum intervals, before you report a pooled value.
• Reading a similar crude and adjusted estimate as proof of no confounding. Two confounders can pull in opposite directions and cancel in the crude table. Check each candidate, not just the bottom line.
• Feeding percentages into the calculation. The Mantel-Haenszel sums need the raw cell counts in every stratum. Proportions throw away the stratum sizes that drive the weights.
• Calling the heterogeneity test negative because P was large. That test has low power. A large P is weak evidence of uniformity, not proof of it. Trust the pattern of estimates, not the P alone.

Tips

• Decide your confounders from subject knowledge and a causal diagram before you touch the data, not by hunting for whichever variable shifts the estimate.
• Report the crude and the adjusted estimate together. The size of the move is itself the evidence of how much confounding there was.
• Use age bands narrow enough that risk is roughly flat within a band. Five-year bands are a common default; ten-year bands can leave residual confounding by age.
• When you stratify on two confounders, build the full cross-classified strata (every age-by-sex cell), not one variable at a time, or you have not controlled for both jointly.
• Once you need three or more confounders, or a continuous one, move to logistic regression rather than forcing more strata. Quote the Mantel-Haenszel result as a sanity check against the model.