News judgment: We all kind of know want it means; we talk about it regularly; and we usually describe it as a key skill that journalists need to have. It certainly plays a big role in what we spend our time doing – and not doing. And, at some level, it involves our sense of how important and how well-read a story will be.
But what about writing 52,000 stories that aren’t intended to be read by a big audience? What kind of news judgment is required there? And what’s the definition of news judgment in an age where we’re writing not only for big audiences but also – via automation – very small audiences?
I’ve written about machine-generated stories before, and touched on the advantages it can confer, not least scale, speed, volume, cost and personalization. But I confess I’ve never thought deeply about what it meant in news judgment terms.
And that’s not a small issue.
Of course, news judgment is one of those tricky concepts to begin with. As Jeff Jarvis notes, when it comes to public datasets the role of journalists is often to add value rather than simply republishing them (plus, it’s not a great business model anyway).
I think this will be our primary job description going forward: adding value to flows of information that can now exist without our mediation. We should add value in many ways: contributing context, explanation, caveats (how the information can be out of date or flawed), education (how to verify the information), in some cases editing (the value The Times and Guardian added to Wikileaks data was not just distribution but also redaction of necessary secrets), and especially and always reporting… That’s where the need for journalism and its future lies.
Certainly journalists have always applied a set of filters to everything they do, choosing what to devote time to, and to devote once-valuable newsprint and airtime to.
That’s made us focus on stories that have the broadest impact – which, at its noblest, is about covering issues of public interest that affects society most. Which makes sense. After all, no one wants to write stories that only affect a couple of people down the road, if the same amount of work can yield stories that change policy for a city or a government. It’s not simply noble intentions that drive this, of course; there are simple economic considerations at play as well. If you have finite staff (and I haven’t heard yet about any infinitely-staffed news orgs), then you want every one of the team to be doing things that affect as many people (or readers) as possible.
But what does that mean in a world where we can not only publish as much as want, but where the theoretical cost of publishing stories that are of interest to a single person are nearly nil? Where a key part of our value will likely be in designing systems for publishing rather than actually choosing which stories to run – not unlike the difference between designing specific pages for a newspaper vs. designing templates for websites?
In world like that – and we’re very close to that world – shouldn’t we rethink what at least one type of news judgment means? Sure, we want to have a huge impact on the world. But don’t we want to have an impact on individuals and microcommunities as well? If cost and resources aren’t an issue, do we care if a “story” is read by one person or a million?
Certainly Scott and the folks at ProPublica grappled with the issues – read his excellent post – ranging from what variables to include to what points each story should highlight to how to actually check and edit a vast number of stories.
It isn’t practical to read 52,000 narratives. Also, Narrative Science’s systems are more complex than simple boilerplate with interpolated variables. Editing one narrative does not mean you’ve edited them all. In addition to recasting whole paragraphs, their systems generate a variety of sentences to express the same kind of data, so that reading the narratives for several schools would seem more natural and not automated. So edits that made sense in one case ended up not working in other cases, and sentences that seemed correct given one set of circumstances seemed wrong in others – often subtly.
All of this speaks to a quickly-evolving set of practices and judgments journalists will be coming up with on the fly as we dive more deeply into this world. For one thing, it will mean living with a new level of uncertainty – we can’t get human eyes on every story we publish (any more than we can inspect every data point we put into a visualization), and that will likely raise debates about standards of verification. But it’ll also involve rethinking what judgments we apply to what we consider a “story.”
That’s not a bad thing at all; it’s part of defining our role in a digital age and how we think we bring value to society and our audience.