Posted by: structureofnews | May 20, 2018

Humans In The Loop


“My dog can play checkers.”

“That’s amazing!”

“Not really – he’s not very good.  I beat him three times out of five.”

OK, it’s a bad joke.  But it does kind of make a point about the world of artificial intelligence, big data and technology – it is amazing how far they’ve come in the relatively short time they’ve been in existence, but we also need to remember their limitations, and not believe all the hype.

At least that’s one of the major takeaways from Artificial Unintelligence, by NYU journalism professor Meredith Broussard, which I just finished on the flight from New York to Gdynia, Poland.  (Don’t ask.).  Like Weapons of Math Destruction, by Cathy O’Neil (another book I finished on a plane), this one also focuses on the misuse of technology and misplaced faith in algorithms and artificial intelligence’s ability to solve the world’s problems.

There’s much to recommend about having a high level of skepticism about the promise of AI, not just in terms of how well any particular system works, but also what assumptions and data are fed into it.  As an industry we haven’t really covered algorithmic accountability particularly well, and it’s a critical gap in fulfilling our public service mission of informing the world about things that matter to society.

And yet it’s important to also recognize how far machines have come, and what capabilities we can harness to do better journalism (even if sometimes there are questions about exactly how far they’ve come).

And one way to better harness machines is to not completely depend on them, as Meredith notes, but instead to build “human-in-the-loop” systems:

It’s also important not to expect that the technology will take care of the edge cases.  Effectively, human-centered design requires the engineer to acknowledge that sometimes, you’ll have to finish the job by hand if you want it done.

There’s a name for systems like this that include humans: humans-in-the-loop-systems.

Exactly.  That’s what Lynx Insight, our new capability at Reuters, is built around – marrying the best of what machines do with the best of what humans do.  (Apologies for the shameless plug.)  And it’s similar to Bailiwick, a tool that Meredith has built:

I put together a plan for a new artificial intelligence engine to detect campaign finance fraud and investigate privacy.  It’s a human-in-the-loop system that automates the process of discovering new investigative story ideas.

There’s much more in the book, from her scary experiences in self-driving cars to a healthy distrust of hackathons to a lovely anecdote about how her mother redo her flowerbeds to adapt to the technology of the day – riding lawnmowers.  It’s a great example of how machines aren’t perfect – in this case, they don’t cut right-angled flowerbeds very well – and how sometimes we just adapt to them – in this case, making curved flowerbeds.

But probably the best example in the book has her walking readers through the process of building a machine-learning system to predict who would survive the Titanic – and not surprisingly, it’s the people who paid more for their tickets and hence got on the few lifeboats first.  That’s all true, but it takes humans to draw the right conclusions from that data and finding.

Meanwhile, out in the world, these numbers have consequences.  It would be unwise to conclude from this data that people who pay more would have a greater chance of surviving a maritime disaster.  Nevertheless, a corporate executive could easily argue that it would be statistically legitimate to conclude this.

And hence charge lower insurance rates to those folks.

All in all, she brings a nice, clear-eyed view of technology and what it can and can’t do.  As we move into a world increasingly powered by AI, it’s important to steer that narrow path between a healthy skepticism and blind faith.






  1. […] we can use algorithms – ideally unbiased ones – to at least help human decision-making, maybe we’ll reduce some of the noise in the […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: