Posted by: structureofnews | March 17, 2018

Automating Insights


So I’ve written a lot about the “cybernetic newsroom” and how news organizations should focusing on marrying the best capabilities of humans and machines to improve journalism – rather than trying to make poor copies of each other.  It’s certainly been very rewarding to riff on a subject close to me.

But it’s much nicer to make it a reality.

(Warning: shameless self-referential plug!) Last week, my colleague Padraic Cassidy unveiled Lynx Insight, our new automation tool, at a session at NICAR in Chicago – finally taking the wraps off a big project that we’ve been working on for more than a year.  It’s has gotten loads of press, (here, here, here, here and here are some of the examples), and marks a big step forward on this front.

So what is it, exactly?  Glad you asked.  On one level, it’s a three-layered system that ingests data, analyzes it to find patterns and anomalies, and then turns them into sentences and paragraphs for reporters to use.  On another level, it’s a tool that helps journalists by automatically trawling through mounds of data, looking for insights that it can present to them – whether as leads to chase down, or simply lines they can use in a story.  Or which they can ignore.

The key idea here – and how Lynx Insight differs from most other story automations – is that we don’t want it to write whole stories.  What we want it to do is analyze data, because that’s what we think machines are good at.  That analysis gets turned into sentences, because that’s what humans are good at evaluating, and then they can decide what to do with those insights.  Basically, it marries machine analysis with human judgement.

Why did we approach it this way? For one thing, it’s faster to build something that isn’t trying to duplicate an entire, complex story – an insight that Padraic came to very early in the process.  (Although, to be fair, we also do automate whole stories, but mostly in the interest of publishing stories about economic and corporate statistics at speed). To be sure, companies like Automated Insights and Narrative Science have done great work going down that path of complete stories, but we were trying to do something else – how to supercharge the work of 2,500 journalists.  It also leverages one of the core advantages Reuters has – access to a ton of data.

And that data isn’t just numbers, although there is a lot of that.  There are company names, descriptions, locations of headquarters, best-selling products, and so on, that we can turn into sentences that also help journalists work faster – bringing efficiency as well as insight to the table.

Plus, we’re not stuck with just the financial data, although that’s what we’re starting with.  There’s sports data, legal data and so on that we can hook into Lynx Insight down the road.

And beyond adding data sets, there are multiple paths we’re looking at to enhance the system.  We’ll keep developing its analytic capabilities, of course, as well as its language generation systems.  We intend to hook it up to another system that flags when there are interesting patterns in stock behavior, so it’ll generate key parts of a story and alert a journalist to it.  We could build in a plug-in that turns out graphics related to the insights and stories.  And probably key is figuring out the best user interface to drive use: In an ideal world, journalists wouldn’t even know they were using the system, because it would detect what they were working on and offer up insights on the fly.  (Not unlike – and I hate to use this analogy – “Clippy” in Microsoft Word.  And now I’m really sorry I said that.)

Further down the road, Lynx Insight opens up new possibilities for story and product types as well: For example, by marrying human-written parts of a market report with a machine-generated personalized paragraph or two about how a reader’s portfolio had done.  And creating it on demand.

So going from:

It’s 4 pm.  The market closed up 2%.  To:

It’s 4 pm. The market closed up 2% but your portfolio fell 1%.  To:

It’s 3.22 pm.  The market is currently up 2% but your portfolio is down 1%. To:

It’s 3.22 pm.  The market is currently up 2%, but your portfolio is down 1%.  If you hadn’t sold IBM last week, you idiot, you’d be up up 3%.  (Showing the value of doing perturbation analysis as well.  Although we may leave out the “you idiot” part.)

The point being, this lets us rethink what a story is, or how we serve readers.  So it’s a big step for us, and we hope, for where machines can help journalism more broadly.


  1. […] whether in text or some other format.  That’s what we were building when I was at Reuters with Lynx Insight.  Language generation was the least interesting part of it; what we wanted was smart pattern […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: