Whadda The Chances

It’s a beautiful November day in Dallas, with not a cloud in the sky.  What are the chances of seeing a man with an open umbrella standing by the side of the street?  And what are the chances he’d be exactly where John F. Kennedy’s motorcade is passing just as the president is shot?

Hmmm.

The “umbrella man” has been a staple of conspiracy theorists for decades now (not to mention the Grassy Knoll, multiple shooters, fabricated X-rays and more), and now the New York Times weighs in with an “Op Doc,” a short video opinion piece by filmmaker Errol Morris to debunk it.   In it, he notes that a person came forward in 1978, saying that he had carried that umbrella on that fateful day in 1963 as a protest against Kennedy’s father.  “This is just wacky enough that it has to be true,” says Errol in the video.

OK, so maybe I wouldn’t want to base a huge investigative story on deciding that something was so improbable it had to be true – and it certainly isn’t going to satisfy those who believe otherwise.

But it is true that probability is at the heart of what much of journalism is.  We look for the unusual (Man Bites Dog!) or seek to uncover hidden causes (Cancer Cases Surge After New Drug is Introduced!) – and that’s based fundamentally on how likely or unlikely those occurrences are.

If men bit dogs routinely, it wouldn’t be a story.  If cancer cases have been on an uptrend anyway, then it’s hard to tie that to the new drug.  If lots of people regularly carried open umbrellas on sunny days, it wouldn’t seem suspicious.

And so journalists really ought to know a lot more about probability and how it works.  But by and large we don’t.

That’s not surprising – most people don’t.  It’s not an intuitive branch of mathematics, even for dyed-in-the-wool gamblers, who have money at stake.  But not knowing how to calculate odds can put journalists’ reputations – or at least story accuracy – at stake.

Consider a fund manager with a 15-year track record of beating the market.  How likely is it that he did that by chance?  With that kind of record, he must be some kind of investing genius, right?  And it’s true – the chances that any specific person would beat the market 15 years in a row purely by luck are infinitesimal.  But in a population of tens of thousands of fund managers, it’s actually quite probable that someone could beat the market for that period of time – even if they’re all picking stocks by throwing darts.

Imagine you’re tossing a huge number of coins a huge number of times.  It’s unlikely that any specific coin will come up heads 15 times in a row – but given thousands of coins, there’s an excellent chance that at least one will.   So maybe that fund manager isn’t so smart – he’s just lucky.  Or he could be an investing Einstein.  The point is that his track record actually gives you very little information about that.

How many similar events does it take to make a trend?  What constitutes a suspicious coincidence?  It really comes down to understanding the numbers, and how probability works.

If you’re walking along a chilly New York street, and a man goes past wearing shorts and a tee shirt, it may seem odd, but it could just be one of those things.  New York is a big city, and people wear all sorts of things – trust me.  So it may be a low-probability event, but one that falls within a normal distribution of events in a town like NY. But what if, at that moment, a prominent politician is assassinated across the street?  All of a sudden, the oddity of the inappropriately-dressed man – which earlier just seemed odd – now seems deeply suspicious.  (Conspiracy theorists will doubtless call him “Shorts Man” for decades.)

But the point is that a political assassination is an even lower-probability event – and so anything connected to it, such as a man with an umbrella, dressed in shorts, or any other unusual activity, is by definition also a very low-probability event.  So the question to ask isn’t, “what’s the chance that a man carrying an open umbrella on a sunny day is at the precise spot where the president is assassinated?” but more “what’s the chance that a man will be carrying an open umbrella on a sunny day in Dallas?”

(I realize this isn’t quite the best example – in the JFK case, the man says he was deliberately there as a protest, so it’s not really a random event.  But you get the point.  I hope.)

Part of the problem in understanding probability is that it runs directly in the face of a basic human desire to find a narrative – and meaning – in events, even when there is none.   It’s what drives us to see basketball players having a “hot hand” when they sink multiple baskets in a row, despite all scientific evidence to show that no such thing  exist.

Consider the famous “conjunction fallacy” demonstrated in this experiment by social scientists Amos Tversky and Daniel Kahneman.

Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.

Which is more probable?

1. Linda is a bank teller.
2. Linda is a bank teller and is active in the feminist movement.

You may have answered 2.  But of course, all feminist bank tellers are also bank tellers – so the class of people in category 1 is, by definition, bigger than the class of people in category 2.  And that makes it more probable – obviously – that Linda is a bank teller rather than any specific kind of bank teller.   But answer 2 fits into a narrative more neatly, and so we tend to choose it.

Kahneman’s latest book, Thinking, Fast and Slow, is full of gems like this about the kinds of mental biases and errors we make.  I’m only halfway through, and will write more about it in a bit, but so far it’s been a fascinating read.   So too is A Mathematician Reads the Newspaper, by mathematician John Allen Paulos.  Check them out.

Meanwhile, check the math in any story you do.  More importantly, check the mathematical and probability assumptions behind any story you do.   Chances are it’ll help you get it right.