Posted by: structureofnews | February 15, 2013

Judgment Call

News judgment: We all kind of know want it means; we talk about it regularly; and we usually describe it as a key skill that journalists need to have.  It certainly plays a big role in what we spend our time doing – and not doing.  And, at some level, it involves our sense of how important and how well-read a story will be.

But what about writing 52,000 stories that aren’t intended to be read by a big audience?  What kind of news judgment is required there?  And what’s the definition of news judgment in an age where we’re writing not only for big audiences but also – via automation – very small audiences?

Those questions come to mind reading Scott Klein’s posting on ProPublica about their tie-up with Narrative Science to dredge through a ton of data to create a horde of stories.

I’ve written about machine-generated stories before, and touched on the advantages it can confer, not least scale, speed, volume, cost and personalization.  But I confess I’ve never thought deeply about what it meant in news judgment terms.

And that’s not a small issue.

Of course, news judgment is one of those tricky concepts to begin with.  As Jeff Jarvis notes, when it comes to public datasets the role of journalists is often to add value rather than simply republishing them (plus, it’s not a great business model anyway).

I think this will be our primary job description going forward: adding value to flows of information that can now exist without our mediation. We should add value in many ways: contributing context, explanation, caveats (how the information can be out of date or flawed), education (how to verify the information), in some cases editing (the value The Times and Guardian added to Wikileaks data was not just distribution but also redaction of necessary secrets), and especially and always reporting… That’s where the need for journalism and its future lies.

Certainly journalists have always applied a set of filters to everything they do, choosing what to devote time to, and to devote once-valuable newsprint and airtime to.

That’s made us focus on stories that have the broadest impact – which, at its noblest, is about covering issues of public interest that affects society most.  Which makes sense. After all, no one wants to write stories that only affect a couple of people down the road, if the same amount of work can yield stories that change policy for a city or a government.  It’s not simply noble intentions that drive this, of course; there are simple economic considerations at play as well.  If you have finite staff (and I haven’t heard yet about any infinitely-staffed news orgs), then you want every one of the team to be doing things that affect as many people (or readers) as possible.

But what does that mean in a world where we can not only publish as much as want, but where the theoretical cost of publishing stories that are of interest to a single person are nearly nil?  Where a key part of our value will likely be in designing systems for publishing rather than actually choosing which stories to run – not unlike the difference between designing specific pages for a newspaper vs. designing templates for websites?

In world like that – and we’re very close to that world – shouldn’t we rethink what at least one type of news judgment means?  Sure, we want to have a huge impact on the world.  But don’t we want to have an impact on individuals and microcommunities as well?   If cost and resources aren’t an issue, do we care if a “story” is read by one person or a million?

Certainly Scott and the folks at ProPublica grappled with the issues – read his excellent post – ranging from what variables to include to what points each story should highlight to how to actually check and edit a vast number of stories.

It isn’t practical to read 52,000 narratives. Also, Narrative Science’s systems are more complex than simple boilerplate with interpolated variables. Editing one narrative does not mean you’ve edited them all. In addition to recasting whole paragraphs, their systems generate a variety of sentences to express the same kind of data, so that reading the narratives for several schools would seem more natural and not automated. So edits that made sense in one case ended up not working in other cases, and sentences that seemed correct given one set of circumstances seemed wrong in others – often subtly.

All of this speaks to a quickly-evolving set of practices and judgments journalists will be coming up with on the fly as we dive more deeply into this world.    For one thing, it will mean living with a new level of uncertainty – we can’t get human eyes on every story we publish (any more than we can inspect every data point we put into a visualization), and that will likely raise debates about standards of verification.  But it’ll also involve rethinking what judgments we apply to what we consider a “story.”

That’s not a bad thing at all; it’s part of defining our role in a digital age and how we think we bring value to society and our audience.


  1. I am tempted to say that if costs (monetary or time) are almost zero, then shouldn’t journalists aim to produce stories which may well be read by just one reader. Perhaps, with this model, journalists and journalism can be all things to all men… there is still a vital role for conventional news judgement, however: where the resource does cost, it is placed appropriately and proportionately at the stories which have biggest impact/readership/public interest.
    The beauty of the model above is there appears little to no trade-off — it is not an either/or scenario, it is cover the stories of micro interest AND those of broader, far-ranging interest.

    • Ossian,

      As you say, it’s not entirely costless; we have to decide what things we want to build templates for, after all. (Eg, do we want to set up to cover the Olympics, or the Oscars?) But once we invest in it and set it up, the marginal costs of creating every additional story – even if sent to just one person – is close to zero. Which really does change the nature of how we figure out what’s worth covering, in a way. Is it better to do one big story that has broad impact on a lot of people, but where there’s little data at an individual/local level, or should we work on a story that perhaps doesn’t have broad implications but is meaningful individually to many people?


  2. A pre-digital example of bending news judgment to report for a very select audience was provided by Peter Jay when he was economics editor of The Times. When a sub-editor said he found one of his articles difficult to follow, Jay grandly replied: “I only wrote this piece for three people – the editor of The Times, the Governor of the Bank of England, and the Chancellor of the Exchequer.”

  3. […] coming into the mainstream, but also because it throws up potentially huge questions about how news judgments can be embedded into algorithms, and how newsrooms might have to evolve to allow that to happen as […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


%d bloggers like this: