The Base Rate Fallacy: Why 99% Accurate Tests Can Fool You

Imagine you’ve just taken a medical test for a rare disease, and the result is positive… With 99% accuracy, you’d expect you’re almost certainly sick, right? Not quite! This is where the base rate fallacy sneaks in: ignoring how rare the disease actually is (the base rate) can make you wildly overestimate your odds. In this post, we’ll unpack why high accuracy doesn’t always mean high likelihood, and how reframing problems with natural frequencies can clear things up.

A brief history

In 1763, the Presbyterian minister Thomas Bayes posthumously published his groundbreaking theorem for updating probabilities in light of new evidence. Though his work flew under the radar for decades, Bayes’s ideas quietly laid the foundation for understanding how prior odds (base rates) matter. Fast forward to the 1970s, and psychologists Daniel Kahneman and Amos Tversky were running experiments showing that people regularly ignore base rates when evaluating likelihood—giving us the name we now use: the base rate fallacy.

Where you’ll see this in real life

Medical screening: Doctors know that even a highly accurate test can yield more false positives than true positives when the disease is rare. Courtrooms: Jurors can be misled if they hear a “99% chance the DNA matches” without knowing how common that DNA profile is in the population. Spam filters: Email systems balance false positives (blocking good mail) and false negatives (letting spam through) by taking overall spam rates into account. Airport security: Machines flag a percentage of harmless items as threats—if you fly often, you might notice frequent secondary checks simply because scanners err on the side of caution.

A common misconception

Just because a test is 99% accurate doesn’t mean a positive result is 99% chance you have the disease. The trick is to include the base rate. Imagine 10,000 people, 1 in 1,000 has the disease (10 people). A 99% accurate test catches 9 of the 10 sick people but also flags 1% of the 9,990 healthy folks—around 100 false positives. That means out of 109 positives, only 9 are true cases (about 8%). Seeing the numbers in a tree diagram or as natural frequencies makes the real odds much clearer.