Two books once kept me company through long nights, mostly by putting me to sleep within ten pages: Metaphors We Live By and A Brief History of Paradox. Books like that are strange: read them without thinking and you are lost; read them with thinking and you get a headache. Annoyingly, they are also interesting.
Many breakthroughs in basic science come from paradoxes or counterexamples. The theoretical kind usually means the theory itself is in trouble and needs a new explanation for what observation or experiment shows. Other paradoxes are illusions: nothing is actually contradictory, but a one-sided explanation makes it look that way. Magic belongs here. A trick can display something that seems to violate common sense, yet once you know the method it is just habit and expectation producing the wrong interpretation. A real paradox is semantic. This sentence is false is the classic example. If it is false, it should be true; if it is true, it describes itself as false. Self-reference leaves no way out. The third crisis in mathematics was patched in a similar spirit by drawing boundaries around semantic interpretation—a loophole inside logic itself.
Simpson's paradox is closer to the illusion. In its bare form, there is a grouping that makes
$\frac{A_1+B_1}{C_1+D_1} > \frac{A_2+B_2}{C_2+D_2}$
while
$\frac{A_1}{C_1}<\frac{A_2}{C_2}$ and $\frac{B_1}{D_1}<\frac{B_2}{D_2}$.
At first glance it feels absurd, but mathematically such numbers are easy to construct. If you cannot sleep, do not count sheep; build a Simpson setup—it works faster.
$\frac{1+3}{5+4} = \frac{4}{9} > \frac{4}{10} = \frac{2+2}{8+2}$
$\frac{1}{5}<\frac{1}{4}$ and $\frac{3}{4}<\frac{1}{1}$
There is no mathematical paradox, because $\frac{A}{B}+\frac{C}{D} \neq \frac{A+B}{C+D}$. The comparison between $\frac{A}{B}$ and $\frac{C}{D}$ does not carry over to the comparison of combined fractions.
Give the numbers a context and everything changes. Multiply the example by ten. A chemical exposure group has 90 people, 40 diseased; the control group has 100 people, 40 diseased. The researcher concludes that the exposed group has a higher incidence. But in the exposed group, 50 men include 10 diseased, while in the control group, 80 men include 20 diseased: the male control group has the higher rate. Among women, 40 exposed include 30 diseased; 20 controls include 20 diseased: again the exposed group is lower.
The same data can make an environmental scientist say this is a pathogenic pollutant that must be banned, while a drug company says it is a preventive medicine effective in both sexes. Both readings can be publishable in good journals, and both can be defensible. Their meanings are opposite. The body also tends to return to normal: some illnesses recover in seven days with medicine and in a week without it. When an effect is not strong enough to be obvious and only appears after repeated slicing of the data, Simpson's paradox is likely to be waiting.
This is awkward for many new drugs and common supplements that sit just above the line of not killing anyone, with placebo and belief doing much of the work. Most supplements are harmless but useless, roughly food with a different price tag. A person with a chronic illness already has good days and bad days. Take a supplement and an illusion forms: a good day means it worked; a bad day means you did not take enough, or forgot. The supplement gets the credit, and you get the blame. With that kind of faith, what could not succeed?
I have a long history with placebo. In primary school there was an annual fitness test. I was bad at running, so I started scheming. The test did not include a drug test, so why not use a stimulant? The problem was that my home had no stimulants. After a frantic search I found a tin of coffee. I had never drunk coffee, but I figured it counted as a stimulant. I made a bowl, took it to the test, found it sweet and quite drinkable, and my performance improved noticeably. Nearly a year later, before the next test, I remembered it. This time I read the label more carefully: it said 'coffee mate,' and those two characters were not easy to recognize. I had been drinking non-dairy creamer with milk powder. My performance collapsed. Years later I finally drank real coffee. Honestly, it was not as good as the coffee mate.
When I told my parents, they were puzzled too: nobody in the family drank coffee, so why was there a tin of coffee mate? The answer was not hard to guess. It was a New Year gift that had been passed around among relatives. They probably bought it thinking it was coffee; the packaging looked foreign and it was much cheaper than real coffee. What improved my performance was the belief that I had taken a stimulant, not the stimulant itself. When psychological factors dominate, the drug's effect becomes a matter of metaphysics. One caveat: do not try tricks in a fitness test. Human potential is far greater than any external stimulant, and regular training is the right path. Otherwise, what luck gains, strength will take back—regression again.
Now back to Simpson's paradox. We have seen that comparing an overall ratio with subgroup ratios is mathematically meaningless; every pattern can appear. Once the numbers carry meaning, however, different fields can produce opposite conclusions from different standpoints, each internally coherent. In pure calculation the comparison is pointless, but with context a real question appears: should that chemical be promoted or banned? How do we answer?
This pushes us into causal inference. We have to model the relations among chemical, sex, and disease—a three-body problem. There are 23 possible directed relationships among three variables: one with no edges, six with a single pairwise relation, eight with two pairwise relations, and eight with three pairwise relations. Professional knowledge or semantics can narrow the candidates. Sex is innate, so models in which chemical determines sex can be excluded. Disease cannot determine sex or chemical either. That leaves six models:
- no edges: chemical, sex, disease independent
- one edge: chemical←sex, disease; chemical, sex→disease; sex, chemical→disease
- two edges: chemical←sex→disease; sex→chemical→disease
- three edges: chemical←sex→disease←chemical
If sex really affects both chemical exposure and disease, then the net effect after controlling for sex has to be considered. If sex affects only chemical exposure and not disease—say women use more cosmetics and therefore have higher exposure—then sex need not be controlled to examine chemical effects on disease. If sex does not affect chemical exposure either, the chemical-disease relation can also be examined directly. Those are the models in which chemical and disease are related. If chemical and disease are unrelated, or even sex and disease are unrelated—say this is a bacterial infection—then the study had no causal support from the start, and its results are numerology. But such possibilities should not be dismissed by common sense alone. If neither common sense nor expertise can rule them out, each has to be tested.
In the three models where chemical and disease are connected, the first thing to check is whether sex and disease are related. Once that is settled, it becomes clear whether sex should be used as a grouping variable. At that point Simpson's paradox is practically resolved—by the semantics and expertise we supply. But expertise can be wrong, and semantics can be ambiguous. Those are the roots of the paradox.
The original Simpson's paradox uses sex, college, and admission rate. To solve it, one would again list all 23 possibilities and exclude the ones that are semantically or practically impossible, such as college determining sex or admission rate determining sex. Fewer possibilities can be excluded here than in my chemical example, so more models remain to be tested. Five involve sex and admission. Consider two of the models with three pairwise relations:
- sex→college←admission←sex
- sex→college→admission←sex
Whether college determines admission rate or admission rate determines college is not easy to say. The two may not determine each other at all; the school may control both, for example by setting college size and a minimum admission rate. Then it is no longer a three-body problem, because another variable, school, has appeared:
- sex→college←school→admission←sex
Now the causal graph contains a collider. If you need to control for college, you open a backdoor and cannot correctly estimate the effect of sex. In that case you should not control any variable and should look directly at the relation. But the real world is not that simple. School is often unobserved, so its relation to college is hard to assume. If college can influence school decisions, then college still has to be controlled. Whether Simpson's paradox has a solution depends entirely on the causal relations that actually exist.
Judea Pearl once wrote a technical report on Simpson's paradox that presented an infinite hierarchy of causal graphs. He called it a Simpson's paradox machine. In that graph, if you control confounders and colliders in sequence, the need to control or not control variables alternates again and again. To solve the paradox, the important thing is to solve the causal graph behind it. Under different contexts, logics, or graphs, the decision to control other variables can differ completely. In the end we can only accept a correct answer relative to a context. The absolute mathematical question has no meaning.
So Simpson's paradox hints that different self-consistent logics can yield more than one correct answer, and correctness depends on assumptions about the underlying causal model. Those assumptions need not be unique. The good news is that in natural science many assumptions can be confirmed or refuted by experiment. The bad news is that ordinary language and semantics may be inherently unable to describe causality clearly. Numbers that can be quantified lack physical meaning; numbers with physical meaning cannot be quantified. We see the truth inside a model but cannot verify the model itself. This is the realism versus anti-realism debate in philosophy of science—also an excellent sleep aid.
Maybe what we call solving a paradox is only introducing a model that hides the discussion that produces it. That would show that the paradox has no final solution and that natural language has holes. As for different disciplines reporting results according to their own interests, there is not much to worry about. These weak effects mostly produce a pile of supplements and overpriced products, and they are basically harmless.