Linking analysis to study design

You have learned methods one outcome at a time: means in the comparing-means module, proportions and odds in binary outcomes, incidence rates in the rates-and-Poisson module, time-to-event in survival analysis. This lesson puts them on one map. A real study asks a plainer question than any single method: I ran this design and I measured this kind of outcome, so what do I compute? The design decides what you are allowed to estimate. The outcome type decides which test reports it. Match the analysis to the study you actually ran, never to the study you wish you had run.

The four designs in brief

Four designs cover most of clinical research, and they differ in one thing: how people are sampled. That sampling rule is what fixes the measure you can report.

A cross-sectional survey of dengue antibody in a Selangor district tells you what fraction carry antibody today, a prevalence. It cannot tell you the rate at which people are newly infected, because it never watches anyone over time. A cohort that follows uninfected residents forward does measure that incidence. A case-control study starts from people who already have the disease and looks backward at exposure, so it can compare exposure between cases and controls but cannot say how common the disease is. A trial adds randomization on top of a cohort-style follow-up, which is what lets it claim cause rather than association.

Matching the measure to the design

Each design hands you one measure naturally, and the reason is always the sampling.

• Risk and the risk ratio come from a cohort or a trial, because you start with people free of the outcome and watch who develops it. The denominator is real people at risk.
• The rate ratio comes from person-time data, where follow-up varies between people. You count events and divide by person-years, as in the rates-and-Poisson module. A cohort with staggered entry or losses reports a rate, not a plain risk.
• The odds ratio is what a case-control study can give. Sampling on outcome fixes the number of cases and controls by your own choice, so any risk or prevalence you compute is an artefact of that choice. The odds ratio survives this because it does not depend on the case-to-control ratio. When the disease is rare, that odds ratio approximates the risk ratio.
• Prevalence and the prevalence ratio come from a cross-sectional survey, the snapshot measure.

Outcome type and design together pick the method

Once the design has told you the measure, the outcome type names the test. Numerical outcomes use means and the t-test or linear regression. Binary outcomes use the chi-squared test and logistic regression from the binary-outcomes and logistic-regression modules. Person-time rates use a Poisson-based test. Time-to-event outcomes use the log-rank test and Cox regression from survival analysis. The table below is the whole course read as one grid.

Two ways to handle confounding: design or analysis

Confounding, from the confounding module, has two lines of defence. The first is built into the design before any data arrive. The second is applied in the analysis afterwards.

1. Design tools. Randomization in a trial balances known and unknown confounders in expectation, which is why a well-run trial needs little adjustment. Restriction recruits only one level of a confounder, say only non-smokers, so it cannot confound. Matching forces cases and controls (or exposed and unexposed) to share the confounder distribution.
2. Analysis tools. Stratification splits the data by the confounder and pools with Mantel-Haenszel, giving one adjusted estimate. Regression (logistic, Poisson, Cox) adjusts for several confounders at once, including continuous ones.

Design handles only the confounders you anticipated. Randomization is the exception, because it handles the ones you never measured. Analysis can only adjust for confounders you recorded, so an unmeasured confounder is beyond rescue at the analysis stage.

Stratified analysis or a regression model?

Both control confounding, so the choice is about transparency against flexibility. Reach for a stratified Mantel-Haenszel analysis when you have one or two categorical confounders and you want a table that shows each stratum, which also lets you eyeball effect modification. Reach for a regression model when confounders are many, when one is continuous (age in years rather than age bands), or when the strata get so thin that the stratified estimate becomes unstable. One rule holds firm: a matched case-control study must be analysed as matched, with conditional logistic regression or McNemar's test, never as if the pairs were independent.

Worked example 1: a case-control study, odds ratio and chi-squared

Betel quid chewing (sireh and pinang) is common in parts of Malaysia and the wider region and is linked to oral cancer. A hospital team recruits 120 oral-cancer cases and 120 cancer-free controls, then asks each about chewing history. The design samples on outcome, so the measure is the odds ratio and the test is chi-squared.

The odds ratio is the cross-product, . Chewers have four times the odds of oral cancer in this sample. Do not read 80/120 as a risk: that 0.67 is fixed by the team choosing 120 cases.

tab <- matrix(c(80, 40, 40, 80), nrow = 2, byrow = TRUE,
              dimnames = list(exposure = c("Chewer", "Non-chewer"),
                              status   = c("Case", "Control")))

or <- # cross-product ratio
chisq.test(tab, correct = FALSE)

or <- (tab[1, 1] * tab[2, 2]) / (tab[1, 2] * tab[2, 1])
ct <- chisq.test(tab, correct = FALSE)
round(c(OR = or, chisq = unname(ct$statistic), p = ct$p.value), 4)
#      OR   chisq       p
#  4.0000 26.6667  0.0000

import numpy as np
from scipy.stats import chi2_contingency
tab = np.array([[80, 40],
                [40, 80]])

or_ = # cross-product ratio

or_ = (tab[0, 0] * tab[1, 1]) / (tab[0, 1] * tab[1, 0])
chi2, p, dof, _ = chi2_contingency(tab, correction=False)
print(round(or_, 4), round(chi2, 4), round(p, 6))
# 4.0 26.6667 0.0

Worked example 2: a cohort with person-time, the rate ratio

A Malaysian HIV clinic follows two groups for incident tuberculosis: people living with HIV and HIV-negative attendees. Follow-up differs between people, so the denominator is person-years and the measure is the rate ratio, tested with a Poisson method (the rates-and-Poisson module).

1. HIV-positive group: 30 TB cases in 500 person-years, a rate of 60 per 1,000 person-years.
2. HIV-negative group: 20 TB cases in 2,000 person-years, a rate of 10 per 1,000 person-years.
3. Rate ratio: .

HIV carries six times the TB incidence rate here. A plain proportion would have been wrong, because the two groups contributed markedly different amounts of follow-up time.

d  <- c(30, 20)        # incident TB cases
pt <- c(500, 2000)     # person-years

rr <- # ratio of the two rates
poisson.test(d, pt)

rate <- d / pt
rr   <- rate[1] / rate[2]
pt_test <- poisson.test(d, pt)
round(c(rate1 = rate[1], rate2 = rate[2], RR = rr,
        p = pt_test$p.value), 5)
#   rate1   rate2      RR       p
# 0.06000 0.01000 6.00000 0.00000

import numpy as np
from scipy.stats import binomtest
d  = np.array([30, 20])       # incident TB cases
pt = np.array([500, 2000])    # person-years

rr = # ratio of the two rates

rate = d / pt
rr   = rate[0] / rate[1]
# conditional on total events, cases in group 1 ~ Binomial(N, pt1 / total pt)
res = binomtest(int(d[0]), int(d.sum()), pt[0] / pt.sum())
print(round(rate[0], 5), round(rate[1], 5), round(rr, 4), round(res.pvalue, 6))
# 0.06 0.01 6.0 0.0

Common mistakes

• Quoting a risk or risk ratio from a case-control study. Sampling on outcome fixes the case-to-control ratio, so any absolute risk reflects your recruiting, not the population. Report the odds ratio, and note it approximates the risk ratio only when the disease is rare.
• Reading incidence off a cross-sectional survey. A snapshot measures prevalence, which blends incidence with how long disease lasts. For the rate at which new cases arise, you need a cohort with person-time.
• Significance-testing the baseline table of a trial. Randomization guarantees any baseline imbalance is chance, so a p-value tests a hypothesis already known to be true. Describe a few strong predictors instead and adjust only if one matters.
• Analysing a matched case-control study as unmatched. Splitting the matched pairs discards the design and biases the odds ratio toward the null. Use conditional logistic regression or McNemar's test.
• Collapsing person-time data into a plain proportion. When follow-up varies between people, dividing events by headcount ignores time at risk. Count person-years and use a rate.

Tips

• Write the design and the outcome type at the top of the analysis plan before choosing any function. Those two facts name the cell in the grid and the test follows.
• For a rare disease, state plainly that the case-control odds ratio is standing in for the risk ratio, so readers interpret it correctly.
• Use a stratified Mantel-Haenszel analysis when one categorical confounder needs adjusting and you want a transparent per-stratum table. Move to regression once confounders are many or continuous.
• For person-time outcomes, record events and person-years per group, then a Poisson-based test, rather than reducing the data to a single proportion.
• No analysis trick repairs a design that cannot estimate the measure you need. Note the limitation honestly and build it into the next study.