Chi-squared: 2x2 tables, larger tables, trend, and exact tests

You have counted people into a table. Rows are an exposure, columns are an outcome, and every cell holds a raw count. The question is always the same: could this table have come from chance, or do the row and column variables move together? The test answers it. This lesson takes the test past the basic 2x2: the continuity correction, when the test stops being valid, Fisher's exact test for small counts, larger r x c tables, and a sharper test for an exposure that has a natural order. It assumes you already know how to read a 2x2 table and a P-value.

The null is independence, and that fixes the expected counts

The null hypothesis of the test is independence: the outcome does not depend on the exposure. If that were true, every row would carry the same outcome split, equal to the overall split. That single idea is what generates the expected counts, so they are not a separate assumption you bolt on. They are forced by the margins.

Take the NSAID and gastrointestinal bleed study. You follow 400 arthritis patients for a year. Exposure is long-term NSAID use, outcome is an upper GI bleed.

Across both arms, 42 of 400 patients bled, a rate of 0.105. Under independence you apply that one rate to each arm: 0.105 × 200 gives 21 expected bleeds per arm. The general rule reads the margins directly. The expected count in cell i, j is its row total times its column total, divided by the grand total.

For the NSAID bleed cell: (200 × 42) / 400 = 21. The four expected counts come out 21, 179, 21, 179. They reproduce the same row and column totals as the data. That is the point: the expected table is the one table that keeps your margins fixed while forcing both rows to agree.

The statistic, the correction, and degrees of freedom

The statistic sums a standardised gap over every cell: observed minus expected, squared, over expected.

Each NSAID bleed cell sits 9 away from its expected 21. Summing the four terms gives = 8.62. The number of independent cells you could change before the margins pin the rest is the degrees of freedom. For an r x c table that is the rows minus one times the columns minus one.

A 2x2 table has 1 degree of freedom. On 1 df, 8.62 gives a P-value of 0.0033. That is strong evidence against independence: a bleed and NSAID use are associated.

The distribution is continuous, but counts are discrete, so the approximation runs a little anti-conservative on a 2x2. Yates' continuity correction trims each gap by half before squaring, which shrinks the statistic.

For the NSAID table the corrected value is 7.69, P = 0.0056. The conclusion does not change here. The correction only matters near the margin of significance or when the total is small. There is no continuity correction for tables larger than 2x2, and none for the regression methods you meet later.

# rows = exposure, columns = outcome
m <- matrix(c(30, 170,
              12, 188), nrow = 2, byrow = TRUE)
rownames(m) <- c("NSAID", "No NSAID")
colnames(m) <- c("Bleed", "No bleed")

chisq.test(m, correct = FALSE)   # uncorrected, matches the hand sum

chisq.test(m, correct = FALSE)            # X-squared = 8.62, df = 1, p = 0.0033
chisq.test(m)                             # Yates default: 7.69, p = 0.0056
chisq.test(m, correct = FALSE)$expected   # every cell: 21 and 179
fisher.test(m)                            # exact test, no large-sample approximation

import numpy as np
from scipy.stats import chi2_contingency, fisher_exact
m = np.array([[30, 170], [12, 188]])

chi2, p, dof, exp = chi2_contingency(m, correction=False)
print(f"Pearson chi2={chi2:.3f}  dof={dof}  p={p:.4f}")

chi2, p, dof, exp = chi2_contingency(m, correction=False)
print(f"Pearson chi2={chi2:.3f}  dof={dof}  p={p:.4f}")   # 8.62, 1, 0.0033
chi2y, py, _, _ = chi2_contingency(m, correction=True)
print(f"Yates   chi2={chi2y:.3f}  p={py:.4f}")             # 7.69, 0.0056
print("expected:\n", np.round(exp, 1))                    # 21 and 179
orr, pf = fisher_exact(m)
print(f"Fisher  OR={orr:.3f}  p={pf:.4f}")

Validity, and Fisher's exact test for small counts

The P-value is an approximation, and it only holds when the expected counts are not too small. The working rule for a 2x2 table is that no expected cell count should fall below about 5. Note the word expected, not observed. You judge validity from the E values, not the O values. A common refinement: the test is fine when the overall total exceeds 40 whatever the expected counts; between 20 and 40 it is fine only if every expected count is at least 5; below 20 use the exact test.

Fisher's test sums the probabilities of all tables at least as extreme as yours that keep the same row and column totals. For larger tables Fisher generalises too, but it gets heavy, and the standard fix for a sparse big table is to combine sparse categories that make clinical sense rather than reach for an exact test.

Larger r x c tables

The same machinery runs on any contingency table. Suppose the NSAID question is asked by dose, and the outcome is still a GI bleed. Three ordered dose groups give a 3x2 table.

The overall bleed rate is 51/500 = 0.102. The expected bleeds are 0.102 × group size: 20.4 in the none group, 15.3 in each of low and high, with the no-bleed cells filling the rest of each row. Apply to all six cells, sum , and you get = 9.37. Degrees of freedom are (3−1)(2−1) = 2, which gives P = 0.0092. The dose groups differ in their bleed rate. Apply the test to counts only. Never feed it the percentages: the 6.0%, 10.0%, 16.0% column would give a meaningless statistic.

m3 <- matrix(c(12, 188,
               15, 135,
               24, 126), nrow = 3, byrow = TRUE)
rownames(m3) <- c("None", "Low", "High")
colnames(m3) <- c("Bleed", "No bleed")

chisq.test(m3, correct = FALSE)

chisq.test(m3, correct = FALSE)            # X-squared = 9.37, df = 2, p = 0.0092
chisq.test(m3, correct = FALSE)$expected   # 20.4 / 179.6, 15.3 / 134.7, 15.3 / 134.7

import numpy as np
from scipy.stats import chi2_contingency
m3 = np.array([[12, 188], [15, 135], [24, 126]])

chi2, p, dof, exp = chi2_contingency(m3, correction=False)
print(f"chi2={chi2:.2f}  dof={dof}  p={p:.4f}")

chi2, p, dof, exp = chi2_contingency(m3, correction=False)
print(f"chi2={chi2:.2f}  dof={dof}  p={p:.4f}")   # 9.37, 2, 0.0092
print("expected:\n", np.round(exp, 1))

Ordered exposure: the chi-squared test for trend

The general 3x2 test treats none, low, and high as three unordered labels. It would give the same answer if you shuffled the rows. But the doses have a natural order, and the bleed rate climbs steadily across them: 6.0%, then 10.0%, then 16.0%. A test that uses that order is more sensitive than one that ignores it.

The chi-squared test for trend assigns a score to each ordered group, usually 0, 1, 2, then tests whether the outcome rate rises or falls linearly across the scores. It spends its evidence on one question, a single straight-line trend, so it carries 1 degree of freedom instead of 2. The statistic divides a trend quantity U by its variance V.

For the dose data the trend statistic is 9.24, P = 0.0024. Compare that with the general test, 9.37 on 2 df, P = 0.0092. Almost all of the general statistic, 9.24 of 9.37, sits in the linear trend, and concentrating it into 1 df sharpens the P-value. The leftover, 9.37 − 9.24 = 0.13 on (c−2) = 1 df, tests for departure from a straight line, and here it is tiny: the rise is well described as linear. Reach for the trend test whenever your exposure groups are ordered, like dose, age band, or stage.

events <- c(12, 15, 24)    # bleeds: none, low, high
totals <- c(200, 150, 150) # group sizes

prop.trend.test(events, totals)

prop.trend.test(events, totals)                       # X2 = 9.24, df = 1, p = 0.0024
prop.trend.test(events, totals, score = c(0, 1, 2))   # explicit scores, same result

import numpy as np
from scipy.stats import chi2 as chi2dist
events = np.array([12, 15, 24])
totals = np.array([200, 150, 150])
x = np.array([0, 1, 2])        # ordered scores

O = events.sum(); N = totals.sum()
U = (events * x).sum() - (O / N) * (totals * x).sum()
V = (O * (N - O) / (N**2 * (N - 1))) * (N * (totals * x**2).sum() - (totals * x).sum()**2)
print(U, V)

O = events.sum(); N = totals.sum()
U = (events * x).sum() - (O / N) * (totals * x).sum()
V = (O * (N - O) / (N**2 * (N - 1))) * (N * (totals * x**2).sum() - (totals * x).sum()**2)
chi2_trend = U**2 / V
p = chi2dist.sf(chi2_trend, df=1)
print(f"trend chi2={chi2_trend:.2f}  df=1  p={p:.4f}")   # 9.24, 0.0024

Statistical significance is not clinical significance

A small P-value says the association is unlikely to be chance. It says nothing about whether the effect is large enough to matter. The two come apart in both directions. In the 2x2, P = 0.0033 came with a 9 percentage point jump in bleed risk, so the effect is both real and clinically meaningful. But in a study of 50,000 patients a 0.3 point difference can reach P = 0.001 and still change no decision. The reverse also happens: a genuinely important difference in a small trial can miss significance because the sample was too small to detect it. So read the P-value alongside the effect size and its confidence interval. The test tells you whether, not how much.

Common mistakes

• Feeding percentages or proportions into the test. needs raw counts (30, 12), never 15% and 6%. Percentages destroy the sample-size information the test depends on.
• Judging validity from observed counts. The expected ≥ 5 rule is about expected cell counts. An observed zero can be perfectly valid if its expected count is 8.
• Using Yates or Fisher on a table larger than 2x2. The continuity correction and the simple exact test are 2x2 tools. For a sparse larger table, combine categories that make clinical sense instead.
• Running the general r x c test on an ordered exposure. It throws away the ordering and loses power against a dose-response. Use the trend test for ordered groups.
• Reading a small P-value as a big effect. Significance scales with sample size. Always pair the P-value with the effect size and its confidence interval.

Tips

• Print the expected counts every time with chisq.test(m)$expected. It is your validity check and a sanity check on the table at once.
• On a 2x2, let the software default to Yates in R, or pass correction=True in SciPy, unless you are deliberately matching a hand calculation or a z-test.
• If a larger table has sparse cells, merge adjacent categories that are clinically alike, then re-run. Document the merge.
• For an ordered exposure, plot the outcome proportion against the score first. A roughly straight climb justifies the trend test and its single degree of freedom.