A model with 94% accuracy confidently told a hospital system that patients who received a particular treatment were more likely to die. The model was right. The treatment was preferentially given to the sickest patients. Adjust for severity, and the treatment saved lives. The model was predicting mortality from confounded observational data — and without anyone asking whether the prediction reflected a causal relationship, it nearly became a clinical decision rule.
This is the causal inference problem, and it shows up everywhere data scientists use observational data to inform decisions. The argument here is simple: predictive accuracy and causal validity are distinct properties, and treating them as interchangeable is one of the most expensive mistakes in applied data science. As machine learning expands deeper into consequential domains — hiring, lending, clinical care, policy — getting this distinction wrong at scale has real costs.
What prediction gets right and what it doesn't
Prediction and causal inference are both legitimate statistical goals, but they answer different questions.
A predictive model asks: given what I observe about X, what is my best guess about Y? It doesn't care why the relationship holds. It cares that the relationship is stable enough to be useful. If patients with a particular biomarker tend to develop a condition, a predictive model exploits that correlation — regardless of whether the biomarker causes the condition, is caused by it, or is simply correlated through a shared upstream factor.
A causal model asks: if I intervene on X — actually change it — what happens to Y? This is the question that matters for decisions. You can't use a purely predictive model to answer it, because interventions break the correlational structure the model was trained on. When you give a treatment to a new population, the distribution of confounders changes. The model's learned associations no longer hold.
Pearl and Mackenzie's The Book of Why frames this as the difference between seeing and doing. Observational data tells you what the world looks like when you passively observe it. Causal inference asks what the world would look like if you reached in and changed something. These are different questions and they require different methods.
The three frameworks
The field of causal inference has converged on three major frameworks, each with different strengths. A 2025 survey in the Journal of Data Science provides a useful overview; here's what practitioners need to know.
The potential outcomes framework (associated with Rubin and Imbens) defines causal effects in terms of counterfactuals. The average treatment effect is the difference between what would happen if a unit received treatment versus if it didn't — for the same unit, at the same time. Since we can never observe both simultaneously (the fundamental problem of causal inference), estimation requires assumptions about how the treatment was assigned and whether all relevant confounders have been measured.
Structural equation models (associated with economists like Heckman) represent causal relationships as equations. If Y = f(X, U) where U is unobserved, estimation requires finding instruments or natural experiments that affect X but don't directly affect Y. This is the foundation of most policy evaluation work in economics.
Directed acyclic graphs (DAGs), associated with Pearl, offer a visual and formal language for representing causal assumptions. A DAG lets you identify which variables to adjust for (and which not to — adjusting for colliders can introduce rather than remove bias), and whether a causal effect is identifiable from observational data at all given your assumed graph structure.
The frameworks are not competing: they often lead to the same estimands and can be combined. What they share is a commitment to making causal assumptions explicit — which is precisely what purely predictive machine learning avoids.
Where machine learning fits in
This is not an argument against machine learning. It's an argument for using it correctly within a causal framework.
ML excels at high-dimensional nuisance estimation — fitting flexible models for propensity scores, outcome surfaces, and heterogeneous treatment effects in ways that classical parametric methods can't. Stefan Wager's work at Stanford and the broader literature on causal forests demonstrate that combining ML flexibility with causal identification strategies outperforms either alone.
The 2025 review in WIREs Computational Statistics on combining observational and experimental estimates makes this concrete: even when you have experimental data, machine learning methods for effect heterogeneity can extract more information about who benefits than classical estimators. And when experiments are unavailable, modern methods — double machine learning, targeted learning, causal forests — can make observational analysis substantially more robust.
What ML cannot do is substitute for causal identification. Flexible function approximation doesn't solve omitted variable bias. A neural network trained on confounded data learns the confounded relationship with greater precision.
The practical checklist
Before deploying any model that will inform an intervention — a treatment, a policy, a product change, a hiring decision — these questions need answers:
Is this model being used predictively or causally? If the goal is to flag high-risk individuals for outreach, a predictive model may be appropriate. If the goal is to decide what intervention to apply, you're in causal territory.
What are the confounders? List every variable that affects both your treatment/exposure and your outcome. Which of these are measured? Which aren't? Unmeasured confounding is the central threat to observational causal inference, and pretending it doesn't exist doesn't make it go away.
Would a randomized experiment give a different answer? If yes, and if the model is being used to make decisions, you have a problem. Consider whether a natural experiment, an instrumental variable, or a regression discontinuity exists that could identify the effect more credibly.
What happens when the model is deployed? Deploying a model changes the data-generating process. Credit scoring models change who applies for credit. Risk stratification models change who gets treated. If the training distribution shifts under intervention — and it usually does — the model's predictions break down.
The stakes
The integration of ML with causal methods is one of the more productive developments in the field over the past five years. Researchers in healthcare, economics, and social science are increasingly building pipelines that use ML for nuisance estimation while maintaining rigorous identification strategies for the causal quantities they care about.
What hasn't kept pace is practice outside research settings. Industry data science teams regularly train models on observational data, evaluate them on predictive metrics, and deploy them to inform decisions — without ever asking whether the estimated relationship is causal. For low-stakes predictions, this is probably fine. For decisions that affect people's lives, it isn't.
The good news is that the tools exist. The frameworks are well-developed. The software — DoWhy, EconML, and the broader PyWhy ecosystem — has made causal methods accessible to practitioners who don't hold a PhD in econometrics. The gap is mostly conceptual: knowing when you're asking a causal question and reaching for the right tools when you are.
The hospital model that nearly became a clinical decision rule wasn't the product of bad intentions or incompetent analysts. It was the product of a team that optimized for predictive accuracy without asking what the prediction was for. That question — what is this model for, and does this approach actually answer it — is worth asking before deployment, not after.