Posted by: structureofnews | July 12, 2021

Of Human Frailty, Part Deux

Just riffing off my recent riffing off on Noise, the new book by Daniel Kahneman, Olivier Sibony and Cass Sunstein: OK, so people are flawed, illogical and inconsistent, and that leads to injustice – even when the people making the decisions aren’t biased.

(Of course it’s worse when they are.)

And we as journalists should expose injustice in systems whenever we find it – no matter whether the cause is.

But more importantly, we also can and should delve into how those systems could be better. As well as think about how these ideas could apply to our own decision making and our own organizations – such as when we hire or promote, since that’s a place of intersection with systems, journalism and diversity.

So how does one reduce “noise” and inconsistency in decision making?

The book lists a series of practices that help, mostly involving slowing us down from our very human habits of jumping to conclusions, relying on intuition, or building a narrative to support a decision we’re predisposed to make, regardless of the facts. And – more controversially – turning more to algorithms to help us drive more consistency.

These aren’t hard things to do – they’re just a pain, and they take away some of what we think of as our essential humanity: Our ability to make exceptions, look beyond the facts, go with our gut. And sometimes those are good instincts – and many times they’re not.

Among the suggestions: Break judgments up into individual tasks, so we’re less inclined to make holistic decisions, overriding inconvenient facts. Get multiple, independent opinions, and aggregate them into a wisdom-of-crowds judgment. Think statistically, not in terms of narrative or neat causal stories. Resist premature intuitions. Try to make judgments on just the issues that are relevant.

Take hiring: We often based those decisions on interviews and discussions with a hiring panel – possibly one of the worst ways we can find the best candidate. Instead, where there are specific skills we’re looking for that can be quasi-objectively judged, let’s make applicants take tests that measure those skills. And anonymize the entries, so we’re not swayed but who someone is, or how the look, or what school they went to.

If you’re trying to hire a copy editor, have candidates take a copy-editing test. And scrub their names from the submissions. Have the hiring panel grade the results – independently. Then compare notes. If you’re looking for a reporter, get anonymized reporting memos from the applicants.

That’s not to say that you shouldn’t also interview the candidates. But the test may throw up results you weren’t expecting.

Look at the experience of US orchestras, which I wrote about some time ago:

Consider that in 1970 women comprised less than less than 10% of major orchestras in the US and fewer than 20% of new hires.  As Mahzarin (Banaji) recounts in her book, back then auditions for new members were conducted in front of a team of seasoned musicians, often from that orchestra.  You’d expect that they had well-trained ears, able to select the best candidates.  And they largely picked men.

But then an interesting thing happened when they started auditioning candidates behind a curtain, and taking pains not to let the panel know if it was a man or a woman was playing.  More women won spots.

And remember – they weren’t trying to hire women. They just wanted to hire the best musicians. But apparently, when you can see the people who are playing, your eyes deceive your ears.

I once went through a highly structured interview process, where we – the hiring panel – stuck rigorously to a set of questions for all the candidates designed to elicit information about the skills we were seeking, scoring each answer independently, and then aggregating the scores before we started discussing the candidates. I hated it. It took all the fun out of the process. But then again, the process isn’t intended to be fun for us; it’s supposed to help us find the best candidate. And I think it did. But it takes discipline.

And, of course, there’s bringing even more non-human discipline into systems, via algorithms, or rules.

To be sure, many people – myself included – worry about the influence algorithms already have over our lives. Should we give them even more power, especially when we know how flawed many are?

Kahneman, Sibony and Sunstein make a case that we should at least compare the flaws of algorithms to the flaws of humans – and try to marry the best of both worlds.

Perhaps most importantly, noise-reduction strategies may have unacceptable downsides.  Many concerns about algorithms are overblown, but some are legitimate. Algorithms may produce stupid mistakes that a human would never make, and therefore lose credibility even if they also succeed in preventing many errors that humans do make. They may be biased by poor design or by training on inadequate data.  Their facelessness may inspire distrust.

If we can use algorithms – ideally unbiased ones – to at least help human decision-making, maybe we’ll reduce some of the noise in the systems.

Regardless of how we do it, Kahneman et all make a good case for why we should both work harder at reducing noise, and make more noise about the noise in our systems.

In many cases, variability in such decisions is clearly driven by biases, including identifiable cognitive biases and certain forms of discrimination.  What that is so, people tend to find the situation intolerable, and the law may be invoked as a corrective, requiring new and different practices.  Organizations all over the world see bias as a villain.  They are right.  They do not see noise that way.  They should.


  1. […] tests are a great way of shielding us from irrelevant information that colors our views.  I’ve written about this before, but when American orchestras moved to having “blind auditions,” the number of women hired […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: