Sensitivity analyses
How robust is your estimate? Vary the assumptions, and probe for unmeasured confounding
Under development.
Register data lacks important lifestyle variables (smoking, BMI, alcohol, diet). So reviewers will almost always ask: how robust is your result? A sensitivity analysis repeats the main analysis under a different but plausible assumption and checks whether the conclusion holds. If it holds, the result stands stronger; if it collapses, you know a particular assumption is carrying the result, and you should be honest about that.
Two principles: pre-specify your sensitivity analyses in the analysis plan (otherwise they look like fishing for a nice number), and report them alongside the main estimate, not instead of it.
The examples use generic variable names. You rarely need new packages: a sensitivity analysis is usually your main model run again with one changed choice.
Robustness checks: change one choice at a time
Most sensitivity analyses consist of changing one choice at a time and seeing whether the conclusion moves:
| What you vary | Why | Example |
|---|---|---|
| Exposure definition | Are you capturing real use or just one incidental purchase? | Require 2 filled prescriptions instead of 1 |
| Outcome definition | Strict vs. broad capture of the outcome | Only primary diagnosis vs. also secondary diagnoses |
| Adjustment set | Does a single uncertain covariate carry the result? | Minimal vs. full adjustment; with/without a variable you are unsure is a confounder or a mediator |
| Restriction / lag time | Reduce reverse causation (the disease was there before the exposure) | Exclude events in the first 6-12 months after index (induction period) |
| Missing data | Does the estimate hold without the imputation assumptions? | Complete-case vs. multiple imputation (see Missing data) |
| Censoring / follow-up | Is dropout informative? | Alternative censoring rule, or IPCW (weighting against selective dropout, see IP weighting) |
The conclusion need not be numerically identical across the analyses - it just has to point the same way. A hazard ratio moving from 1.8 to 1.6 is robust; one shifting from 1.8 to 0.9 means the changed choice is carrying the result.
Probe for unmeasured confounding
The checks above vary your choices. But register data also lacks variables you simply do not have (smoking, BMI, etc.). Two design tricks, both standard in Danish register research, reveal or reduce that kind of unmeasured confounding.
Negative control outcome
Idea: pick an extra outcome the exposure cannot possibly cause, but which is hit by the same hidden confounders as your real outcome. Run your main model on it. If you still find an “effect”, that is a warning flag: the groups are systematically different (residual confounding, i.e. confounding left after adjustment), and your main result is probably coloured by it too. If the estimate instead sits close to 1, it strengthens your main analysis.
Pedagogical example (healthy-vaccinee). Studies often find that influenza-vaccinated older adults have markedly lower all-cause mortality. But if you look at mortality outside the influenza season - a negative control outcome, because the vaccine cannot protect against influenza when there is no influenza - they are still “protected”. That reveals the vaccinated are healthier and more resourceful to begin with (healthy-vaccinee bias), not that the vaccine itself halves mortality.
library(survival) # coxph(), Surv()
# Main analysis: the effect on the REAL outcome
coxph(Surv(time_days, outcome) ~ exposure + age + sex, data = df)
# Negative control outcome: the EXACT same model, but an outcome the exposure cannot affect
# (here e.g.: death outside the influenza season). A clear "effect" here = a confounding warning.
coxph(Surv(time_days, neg_control_outcome) ~ exposure + age + sex, data = df)Active comparator (new user)
Idea: instead of comparing drug users with non-users (who are wildly different - they do not even have the indication), compare them with new users of an alternative drug for the same condition. Both groups then share the indication, so confounding by indication is sharply reduced. “New user” means both start treatment at index, so you do not mix newly started and long-term users (prevalent-user bias).
Pedagogical example. You study whether GLP-1 receptor agonists (a second-line drug for type 2 diabetes) affect the risk of an outcome. If you compare against non-users, you are really comparing people with and without treatment-requiring diabetes - two wildly different groups, where the disease itself confounds the picture. If you instead compare against new users of DPP-4 inhibitors (another second-line drug for the same condition), the groups are far more alike on indication and disease severity, and the estimate is less coloured by why people are on treatment.
Active comparator is about how you build the cohort (Phase 10) - see the new-user and risk-set logic in Comparison cohort.
Self-controlled designs (where the person is their own control and all time-invariant confounding disappears) are a related, powerful trick in register research. See Self-controlled and family-based designs.
Remember: anything leaving DST must go through output control - no small cells, only aggregated results. See Phase 14 - Export and repatriation.
- Hernán & Robins, Causal Inference: What If - the causal foundation under confounding and sensitivity analyses (free PDF).
- Lund, Richardson & Stürmer, “The Active Comparator, New User Study Design in Pharmacoepidemiology”, Curr Epidemiol Rep 2015 - active comparator and new-user design.
- Lipsitch, Tchetgen Tchetgen & Cohen, “Negative controls: a tool for detecting confounding and bias in observational studies”, Epidemiology 2010 - an introduction to negative controls (uses exactly the influenza vaccine example above).