Variables, types, and the question behind the data

Every dataset is a stack of answers to questions you have not written down yet. Before you run a single test, two questions decide everything that follows: who are these rows meant to stand for, and what kind of thing is each column. A blood pressure in mmHg, a yes-or-no on insulin, and a clinic name in Johor are three different species of number and word. Treat them the same and your analysis breaks in quiet ways. This lesson sets up the habit the rest of the course rests on. Name the population, then classify every variable by type. The type tells you which summary and which test you are allowed to use.

Population and sample

You almost never measure everyone you care about. You measure a sample, and you use it to say something about a larger population. A diabetes nurse at a Klinik Kesihatan in Selangor records HbA1c for the 200 patients who came in this month. She does not care about those 200 people as a closed list. She wants to know how well diabetes is controlled among the patients this clinic serves, including the ones who did not show up and the ones who will register next year.

Here is the reason the whole field exists. Take a fresh sample of 200 patients from the same clinic and you get a different mean HbA1c. Take a third sample, different again. The numbers move around even though the underlying population has not changed. This movement is called sampling variation, and it is the source of every confidence interval and p-value you will compute later. If samples never varied, you could read the population straight off one of them and statistics would be arithmetic. Because they do vary, you need a way to separate a real signal from the wobble. That is the job of the methods in Parts B through D.

Numerical variables

A numerical (quantitative) variable is a measured or counted number where the size carries meaning. It splits into two kinds. A continuous variable sits on a smooth scale and can in principle take any value in a range: weight in kg, HbA1c percentage, systolic blood pressure, gestational age in weeks. Between any two values there is always another. A discrete variable counts whole things and steps from one integer to the next: number of clinic visits in a year, number of dengue episodes, parity (how many times a woman has given birth). You cannot have 2.7 admissions.

The split matters because counts often pile up at small numbers and at zero, so the bell-shaped methods that suit a continuous measurement can mislead on a count. Keep the distinction in mind even when a count looks large enough to treat as continuous.

Categorical variables

A categorical (qualitative) variable records which group an individual falls into. The value is a label, not a quantity. Three sub-types matter, and they behave differently.

• Binary (dichotomous): exactly two categories. Sex recorded as female or male, smoker yes or no, HIV test positive or negative, alive at six months or not. Binary outcomes get their own part of this course because so much of medicine is yes-or-no.
• Nominal: three or more categories with no natural order. State of residence (Selangor, Johor, Kedah), ABO blood group, type of antibiotic. You can count how many fall in each, but ranking them makes no sense.
• Ordinal (ordered categorical): categories that follow a natural order, but the gaps between them are not equal or known. Pain scored as mild, moderate, severe. Blood pressure control graded poor, fair, good. Cancer stage I to IV. The order is real, so "worse than" has meaning, but the step from mild to moderate is not guaranteed to equal the step from moderate to severe.

Rates as a third type

Some outcomes are neither a fixed number nor a single category, because they unfold over time. A rate counts events per amount of person-time: cases per 1,000 person-years, or episodes of diarrhoea per child per year. A rate of divides the number of events by the total time that people were observed and at risk. Rates arise in follow-up studies where people enter and leave at different times, so a plain proportion will not do. They get a dedicated part of the course; for now, just recognise that "events per person-time" is its own type.

Why the type decides the method

This is the spine of everything ahead, so hold onto it. The type of the outcome variable, more than anything else, picks the summary you report and the test you run. Summarise a continuous variable with a mean and standard deviation, or a median; summarise a binary one with a proportion or risk; summarise a rate with events per person-time. Compare two groups on a continuous outcome with a t-test, on a binary outcome with a chi-squared test or logistic regression, on a rate with methods built for person-time. Pick a method that does not match the type and the output may still print a number, which is the dangerous part. It will just be the wrong number.

Derived variables

The columns you analyse are often built from the ones you recorded. There are four common moves.

• Calculated. Age at diagnosis from date of birth and date of visit. Body mass index from weight and height. Estimated glomerular filtration rate from creatinine, age, and sex. The result is usually a new continuous variable.
• Categorized from a continuous variable. Cut a continuous measurement into bands. Age into ten-year groups, income into quintiles. The cut points can be data-specific (quintiles depend on your sample) so they may not transfer to another study.
• Threshold or cutoff. A special case of categorizing using an agreed clinical cut point, not one chosen from your data. Birth weight below 2,500 g becomes low-birth-weight yes or no. HbA1c at or above 6.5% flags diabetes. Because the threshold is fixed in advance, it means the same thing across studies.
• Transformed. Replace a value with a function of it, most often the logarithm, to make a skewed variable behave for a given method. Antibody titres, parasite counts, and concentrations are routinely log-transformed. The variable stays continuous; only its scale changes.

Outcome versus exposure

One more split shapes every analysis. The outcome variable is the thing whose variation you are trying to explain: did the baby have low birth weight, what was the HbA1c, did the patient survive. The exposure variable is a factor you think might influence the outcome: did the mother smoke, how many weeks of breastfeeding, which treatment arm. Other names you will meet are response and explanatory, dependent and independent, y-variable and x-variable. In a trial the exposure is the treatment group; in a case-control study the outcome is case-or-control status and the exposures are called risk factors.

This framing drives the analysis because the type of the outcome chooses the method, while the exposures are what you compare across. "Does smoking during pregnancy raise the risk of low birth weight?" has a binary outcome (low birth weight yes or no) and a binary exposure (smoked yes or no). That combination points straight at risk ratios and the binary-outcome toolkit. Name the outcome and the exposure first, classify their types, and the right method narrows down fast.

Storing each type correctly in R and pandas

Software needs to know the type too, or it will guess. Store a measurement as a number, a nominal label as a factor (R) or category (pandas), and an ordinal label as an ordered factor or ordered category so that "good" outranks "poor". Get this right at load time and the correct summaries and tests follow with less effort later.

age    <- c(54, 61, 47, 58)          # continuous numeric
visits <- c(3L, 5L, 2L, 4L)          # discrete count (integer)
sex    <- factor(c("F", "M", "M", "F"))               # nominal / binary
bp     <- ordered(c("Good", "Poor", "Fair", "Good"),
                  levels = c("Poor", "Fair", "Good")) # ordinal

str(age)
# inspect visits, sex, bp the same way

str(age); str(visits); str(sex); str(bp)
levels(bp)                 # "Poor" "Fair" "Good", low to high
bp > "Fair"                # ordered comparison: TRUE where control beats Fair
# class(age) numeric, class(visits) integer,
# class(sex) factor, class(bp) ordered factor

import pandas as pd
age    = pd.Series([54, 61, 47, 58], dtype="float64")   # continuous
visits = pd.Series([3, 5, 2, 4], dtype="int64")         # discrete count
sex    = pd.Series(["F", "M", "M", "F"], dtype="category")  # nominal / binary
bp     = pd.Categorical(["Good", "Poor", "Fair", "Good"],
                        categories=["Poor", "Fair", "Good"],
                        ordered=True)                   # ordinal

print(age.dtype)
# inspect visits, sex, bp the same way

print(age.dtype, visits.dtype, sex.dtype)   # float64 int64 category
print(bp)                                   # shows order Poor < Fair < Good
print(bp > "Fair")                          # ordered comparison works
# [True False False True]

A procedure for any new dataset

Run the same five steps on every dataset before you analyse it.

1. List every column in the dataset.
2. For each column, decide whether the value is a measurement or a label. Measurements are numerical; labels are categorical.
3. If numerical, ask whether it varies on a smooth scale (continuous) or counts whole things (discrete).
4. If categorical, ask whether the categories have a natural order (ordinal) or none (nominal, or binary when there are exactly two).
5. Name the outcome variable and the exposure variable before you open the analysis.

Worked example 1: classify a clinic dataset and store it right

A diabetes registry at a Klinik Kesihatan holds these columns: patient id, age in years, number of visits this year, sex, state of residence, blood-pressure control graded poor/fair/good, and whether the patient is on insulin. Before any analysis, label each column.

If the question is "is poor blood-pressure control more common in patients on insulin?", then on_insulin and bp_control are the variables of interest: on_insulin is the exposure, bp_control is the outcome. Now build the table with each column stored as its right type.

clinic <- data.frame(
  id         = c("P01", "P02", "P03", "P04", "P05"),
  age        = c(54, 61, 47, 58, 66),
  visits     = c(3L, 5L, 2L, 4L, 6L),
  sex        = factor(c("F", "M", "M", "F", "M")),
  state      = factor(c("Selangor", "Johor", "Selangor", "Kedah", "Johor")),
  bp_control = ordered(c("Good", "Poor", "Fair", "Good", "Poor"),
                       levels = c("Poor", "Fair", "Good")),
  on_insulin = factor(c("No", "Yes", "No", "No", "Yes")),
  stringsAsFactors = FALSE
)

str(clinic)
# then tabulate bp_control by on_insulin

str(clinic)
summary(clinic$bp_control)            # counts per ordered level
table(clinic$on_insulin, clinic$bp_control)   # exposure by outcome

import pandas as pd
clinic = pd.DataFrame({
    "id":         ["P01", "P02", "P03", "P04", "P05"],
    "age":        [54, 61, 47, 58, 66],
    "visits":     pd.array([3, 5, 2, 4, 6], dtype="int64"),
    "sex":        pd.Categorical(["F", "M", "M", "F", "M"]),
    "state":      pd.Categorical(["Selangor", "Johor", "Selangor", "Kedah", "Johor"]),
    "bp_control": pd.Categorical(["Good", "Poor", "Fair", "Good", "Poor"],
                                 categories=["Poor", "Fair", "Good"], ordered=True),
    "on_insulin": pd.Categorical(["No", "Yes", "No", "No", "Yes"]),
})

print(clinic.dtypes)
# then cross-tabulate bp_control by on_insulin

print(clinic.dtypes)                       # age int64, visits int64, rest category
print(clinic["bp_control"].value_counts())
print(pd.crosstab(clinic["on_insulin"], clinic["bp_control"]))  # exposure by outcome

Worked example 2: the cost of categorizing a continuous variable

Suppose HbA1c is linked to a downstream risk score, and you are deciding whether to analyse HbA1c as the continuous measurement or as a binary "uncontrolled" flag (at or above 7%). Categorizing looks tidy, but it weakens the association you can detect. Measure the cost directly by comparing how strongly each version tracks the risk score.

hba1c <- c(5.2, 6.9, 7.1, 8.4, 9.7, 6.5, 7.6, 10.2)
risk  <- c(10.4, 13.9, 14.0, 17.1, 19.2, 13.1, 15.0, 20.6)
uncontrolled <- ifelse(hba1c >= 7, 1, 0)  # 1 = at or above 7%, 0 = below

cor(hba1c, risk)
# now correlate the binary version with risk

round(cor(hba1c, risk), 3)          # continuous predictor
round(cor(uncontrolled, risk), 3)   # binary version, weaker
# the continuous variable tracks risk much more closely
# than the 7% split, which collapses the spread within each group

import numpy as np
hba1c = np.array([5.2, 6.9, 7.1, 8.4, 9.7, 6.5, 7.6, 10.2])
risk  = np.array([10.4, 13.9, 14.0, 17.1, 19.2, 13.1, 15.0, 20.6])
uncontrolled = (hba1c >= 7).astype(int)  # 1 = at or above 7%, 0 = below

print(np.corrcoef(hba1c, risk)[0, 1])
# now correlate the binary version with risk

print(round(np.corrcoef(hba1c, risk)[0, 1], 3))      # continuous predictor
print(round(np.corrcoef(uncontrolled, risk)[0, 1], 3)) # binary version, weaker
# the continuous variable tracks risk much more closely
# than the 7% split, which throws away within-group spread

The continuous correlation is close to 1, while the binary version drops well below it. That gap is the information you discard by cutting. In a real study, the same loss shows up as a wider confidence interval and a weaker, harder-to-detect effect. Categorize for a summary table if it helps a reader, but feed the continuous variable to the analysis.

Common mistakes

• Letting an ID column act like a number. Patient id or hospital code "3" is a label, not a quantity. Averaging it, or letting software read it as numeric, produces nonsense like a mean ward of 2.1. Store identifiers and codes as text or as a factor.
• Averaging an ordinal variable. Coding pain as 1, 2, 3 and reporting a mean of 2.4 assumes the gap from mild to moderate equals the gap from moderate to severe, which you have not measured. Summarise ordinal data with counts and a median category, or use methods built for ordered categories.
• Categorizing continuous data by reflex. Cutting age or HbA1c into bands before the analysis discards detail and power. Worked example 2 shows the correlation falling once you binarize. Band variables for a table, keep them continuous for the model.
• Confusing the sample with the population. A clean summary of the 200 patients you measured describes only them. The reason you compute a standard error or a confidence interval is to carry that summary, with its uncertainty, back to the population the sample came from.
• Not deciding which variable is the outcome. If you have not named the outcome and the exposure, you cannot pick a method, because the type of the outcome is what chooses it. Settle that before you open the analysis.

Tips

• When a new dataset lands, write a one-line type label next to every column before anything else: continuous, discrete, binary, nominal, ordinal, rate. That table is your map to every later choice of summary and test.
• Set types at load time. In R use factor() and ordered(); in pandas use dtype="category" and pd.Categorical(..., ordered=True). Fixing types early stops silent miscoding downstream.
• Spell out the level order for ordinal variables (poor, fair, good) rather than trusting the default, which is alphabetical. Alphabetical order would put "fair" before "good" before "poor", which is wrong.
• Name the outcome and the exposure in one plain sentence before you start: "does X (exposure) affect Y (outcome)?" The type of Y then narrows the method for you.
• Keep clinical thresholds (low birth weight at 2,500 g, diabetes at HbA1c 6.5%) for reporting, since they mean the same across studies, but prefer the underlying continuous variable inside the analysis.