Aaah – another site about The Future of Journalism.

A dull one.  Without the  invective and ideology about free vs. paid, pajama-clad bloggers vs. stick-in-the-mud mainstream media curmudgeons, and Utopian visions of crowdsourced news vs. dark fears about falling standards you can find elsewhere.  It has words like taxonomy and persistent content in it; discusses business models and revenue streams in dull, accountant-like language; and tries to dissect the sparkling prose journalists turn out into tiny bytes of data.

But there is a purpose here, and it’s based around the idea that we as journalists haven’t really thought about how people are changing the way they access information, or how we need to fundamentally rethink the way we carry out journalism and the kinds of – for want of a better word – products we turn out for them.

There’s much hand-wringing over the loss of the traditional business model of news, it’s true.  Perhaps too much.  And this site will contribute its share.  But hopefully it’ll also explore some of the less-explored questions about where the profession goes in a digital age.   And lay out some of the thinking behind one concrete idea that might help move the business forward: Something I’m calling Structured Journalism.

So, welcome – and I hope you find this interesting.

(An update: I first wrote those words seven years ago, and it’s amazing how some of those passionately argued debates – free vs. paid! – have basically gone away.  Which is great.  So I could and should rewrite this intro.  But the second paragraph remains just as valid. Plus, I’m pretty lazy. )

Just riffing off my recent riffing off on Noise, the new book by Daniel Kahneman, Olivier Sibony and Cass Sunstein: OK, so people are flawed, illogical and inconsistent, and that leads to injustice – even when the people making the decisions aren’t biased.

(Of course it’s worse when they are.)

And we as journalists should expose injustice in systems whenever we find it – no matter whether the cause is.

But more importantly, we also can and should delve into how those systems could be better. As well as think about how these ideas could apply to our own decision making and our own organizations – such as when we hire or promote, since that’s a place of intersection with systems, journalism and diversity.

So how does one reduce “noise” and inconsistency in decision making?

The book lists a series of practices that help, mostly involving slowing us down from our very human habits of jumping to conclusions, relying on intuition, or building a narrative to support a decision we’re predisposed to make, regardless of the facts. And – more controversially – turning more to algorithms to help us drive more consistency.

These aren’t hard things to do – they’re just a pain, and they take away some of what we think of as our essential humanity: Our ability to make exceptions, look beyond the facts, go with our gut. And sometimes those are good instincts – and many times they’re not.

Among the suggestions: Break judgments up into individual tasks, so we’re less inclined to make holistic decisions, overriding inconvenient facts. Get multiple, independent opinions, and aggregate them into a wisdom-of-crowds judgment. Think statistically, not in terms of narrative or neat causal stories. Resist premature intuitions. Try to make judgments on just the issues that are relevant.

Take hiring: We often based those decisions on interviews and discussions with a hiring panel – possibly one of the worst ways we can find the best candidate. Instead, where there are specific skills we’re looking for that can be quasi-objectively judged, let’s make applicants take tests that measure those skills. And anonymize the entries, so we’re not swayed but who someone is, or how the look, or what school they went to.

If you’re trying to hire a copy editor, have candidates take a copy-editing test. And scrub their names from the submissions. Have the hiring panel grade the results – independently. Then compare notes. If you’re looking for a reporter, get anonymized reporting memos from the applicants.

That’s not to say that you shouldn’t also interview the candidates. But the test may throw up results you weren’t expecting.

Look at the experience of US orchestras, which I wrote about some time ago:

Consider that in 1970 women comprised less than less than 10% of major orchestras in the US and fewer than 20% of new hires.  As Mahzarin (Banaji) recounts in her book, back then auditions for new members were conducted in front of a team of seasoned musicians, often from that orchestra.  You’d expect that they had well-trained ears, able to select the best candidates.  And they largely picked men.

But then an interesting thing happened when they started auditioning candidates behind a curtain, and taking pains not to let the panel know if it was a man or a woman was playing.  More women won spots.

Whose fault is it when something bad happens? Who do we hold to account when we find injustice?

It’s a natural – and good – impulse journalists have, to look for wrongdoers when we see wrongdoing, to identify bad actors and uncover bad motives. And long may that continue.

But what happens when injustice isn’t anyone’s fault; if it’s – essentially – everyone’s fault?

That’s at least one of the takeaways from Daniel Kahneman’s interesting new book, Noise, written with Olivier Sibony and Cass Sustein. It’s about how, beyond bias, discrimination and prejudice, simple randomness – caused by human frailty (“A flaw in human judgment”) – can also lead to injustice and inequality. And it offers a lesson for journalists: We should look for and expose systems that are failing, regardless of whether there’s systemic bias in them.

The book offers example after example of how what should be consistent judgements made on the merits of an argument often aren’t, even when the people making those calls are trying hard to be impartial and fair.

A study of thousands of juvenile court decisions found that when the local football teams loses a game on the weekend, the judges make harsher decisions on Monday (and, to a lesser extent, for the rest of the week.) Black defendants disproportionately bear the brunt of the increased harshness. A different study looked at 1.5 million judicial decisions over three decades and similarly found that judges are more severe on days that follow the local city’s football team than they are on days that follow a win.

Or this:

A review of 207,000 immigration court decisions over four years found a significant effect of daily temperature variations; when it is hot outside, people are less likely to get asylum.

The fundamental thesis of the book – which, admittedly, can be a little hard to get through – is that we don’t spend enough time looking at “noise” in our systems: the factors that produce wide variations in what should be much more standardized decisions. That’s not to say that bias and systemic bias isn’t an issue, and certainly journalists expend a huge amount of effort to find cases of discrimination, both blatant and subtle. But we don’t focus as much on the noise that can cause as much injustice in outcomes. If immigration cases aren’t judged purely on the merits, if the results are somewhat random, that’s just as bad an outcome as if cases are regularly stacked against a type of applicant or group.

Just the facts, ma’am – Sgt. Joe Friday, Dragnet

Not really. All stories need framing and narrative for the facts to speak, for audiences to engage, and for insight to emerge. We call that editing. But in an age of visual, interactive and multimedia journalism, maybe we should call it designing.

There’s a nice piece in the New Yorker (largely a review of a book, A History of Data Visualization & Graphic Communication) that makes a point that I love to make: That how information is presented – designed, for want of a better word – isn’t just an add-on to other forms of journalism that can make a story better; it’s as much an independent form of narrative that, properly used, can bring completely different insights to users.

It can even literally make the difference between life and death.

The piece starts with the now-famous chart of the space shuttle’s O-ring failures in frigid temperatures, and how poor presentation of that data may have misled engineers to believe the components were robust enough to hold up in cold conditions, and hence green-light the tragic launch. They had all the data; they just didn’t see it clearly. And that’s because it wasn’t presented correctly. As the story notes:

A decade later, Edward Tufte, the great maven of data visualization, used the Challenger teleconference as a potent example of the wrong way to display quantitative evidence. The right graph, he pointed out, would have shown the truth at a glance.

The flip side of that, as I wrote some time ago, was how information presented correctly can bring instant insight:

Probably the best-known example of the value of mapping is John Snow’s famous map (above) tracking cholera outbreaks in London, showing that an outbreak in the mid-1800s could be traced to a contaminated pump.   Without the map, you’re faced with a long list of addresses where outbreaks occurred; with the map, you can immediately see where there’s a real concentration of cases and hence where to investigate.

To be sure, those are examples of single information graphics rather than the all-bells-and-whistles multimedia interactive extravaganzas that we’ve come to expect from any major journalism project. But the core point remains the same: It isn’t simply how words are edited, pictures cropped, or charts plotted; it’s also how they are integrated, how the narrative flows, and how the audience experiences them that brings the understanding, engagement and insight to the fore. It’s how the information is designed.

So it’s been a while. OK, a very long while. OK, a very very long while.

But since this post is about the value of old stuff, perhaps it’s appropriate it’s been three years or so since I wrote a real new post. (Although the real reason for not writing for a while is, as I’ve noted here, that I’ve been kinda preoccupied.)

In any case, I was struck by this interesting article at Digital Content Next that asks the thought-provoking question: What if the value of news isn’t what’s new? It cites the example of the Daily Nation, Kenya’s largest national paper, which is starting a subscription service – not for current news, but for anything older than seven days old. In other words, the “new news” is free; it’s the “old news” you have to pay for. As the piece notes:

So, can an approach like this work? Can you effectively sell a “news” subscription where the content you’re charging for isn’t, well, new?

Spoiler alert: Yes.

Or: Probably yes. Or: Possibly yes, if… And we’ll get into what those ifs are, and some reasons more people aren’t trying this strategy. And why, perhaps, they should.

But back to the article, which goes on to cite examples like National Geographic, Esquire and others that have managed to monetize their archives. The Daily Nation may well be the first – I admit, I haven’t done exhaustive research on this – to bank its entire subscription strategy on selling, well, old news.

It’s not as crazy as it sounds. True, some readers/users really value getting information first – and certainly I work at a news organization which is largely built around speed, sometimes to the tune of milliseconds. But let’s set Reuters and the like apart for a moment, and focus on the vast majority of other newsrooms. A lot of news is widely reported, and if the information is behind a paywall at one site, I can probably get the gist of what I need to know from another site. True, I may really like one news organization’s take on a particular event, but in many cases there’s not necessarily enough differentiation to make me take out my credit card.

Sure, there are exclusive stories, features and deep investigative work, but most news sites pin their value on news.

But speed is only one metric that readers value. As I noted a decade ago:

All those events mattered in some way or another, but knowing about them minutes – or even hours – ahead of others isn’t critical to most of us. What often matters more is the thoughtful, considered analysis of events, or perhaps the exploratory database/interactive that lets us understand the information on our own terms, or the insightful commentary piece a couple of days later.

And that’s where archives – or previous reporting, or well-structured data – can really bring new, and sometimes instant additional value to an audience. Very few events happen in a vacuum; if there’s a plane crash, how many others happened at that location, or with that airline, or with that type of aircraft? If a politician makes a claim, who else has made the same assertion, who has debunked it, what other claims has he or she made? And so on.

Yes, a good journalist can dig that all up in the heat of coverage, and get it into a well-written and nuanced story. But it’s tough to do on deadline. More importantly, much that information probably already exists in the site’s old stories; why not just let readers find it – or better yet, let it find readers?

So it’s been a while (a long while!) since I’ve written here – not because there isn’t much to say about changes and developments in technology, journalism and news products; there’s in fact probably too much to dive into, analyze and brainstorm around.  I just haven’t had much time. 

One reason is this: 

I’ve been busy with a different type of change and development.  Specifically, about me.

I’m transgender, and have as of today, transitioned into what I know to be my true identity.  There’s not much else to say, although there’s a bit on the About section.

Thanks for visiting, and I hope one of these days to get back to writing about how journalism is or should be evolving.

It takes courage to grow up and become who you really are – e. e. cummings

Just a riff on a recent NYT magazine piece about the debate around “autonomous weapons,” or machines that can make decisions about who and when to kill.  Spoiler alert: There’s no consensus about them.  Actually, not even close to being a consensus. Which is probably a good thing.

That said, it’s a good entryway to revisit the notion that we as an industry/profession could be doing a better job covering the multiple algorithms that now govern our lives, even if they aren’t literally designed to kill us.

Algorithms influence what news and information we see, how financial markets behave, where police put their resources, whether we can get loans and at what price, and much more.  And beyond that, they have to power – as do other automations – to reshape how we build and structure our world, beyond replacing humans.

As the NYT piece notes about the debates about autonomous weapons:

This argument parallels the controversial case for self-driving cars. In both instances, sensor-rich machines navigate a complex environment without the fatigue, distractions and other human fallibilities that can lead to fatal mistakes. Yet both arguments discount the emergent behaviors that can come from increasingly intelligent machines interpreting the world differently from humans.

Precisely.  Autonomous machines can and will go beyond replacing humans and potentially fundamentally change our world – not necessarily for better or worse,

Remember the old joke about the 1960s British Thunderbirds puppet TV series?  “The show’s good but the acting is kind of wooden?”  OK, so you probably didn’t.  Anyway, the joke was that the characters were played by puppets, so they were a little wooden… oh, never mind.

Fast forward to today, and Chinese news agency Xinhua has just unveiled a news anchor that’s – well, not a puppet, but certainly not human.  As Xinhua notes:

The news anchor, based on the latest AI technology, has a male image with a voice, facial expressions and actions of a real person. “He” learns from live broadcasting videos by himself and can read texts as naturally as a professional news anchor.

The AI news anchor was jointly developed by Xinhua News Agency, the official state-run media outlet of China, and Chinese search engine company Sogou.com.

The new newsperson got a lot of coverage – here, here, and here for example, but the reviews haven’t been kind.  The BBC quoted an Oxford professor:

The presenter struggled to appear completely natural, said Michael Wooldridge at the University of Oxford.

It was stuck somewhat in the “uncanny valley” – a term used to describe human-like robots and avatars which seem subtly unrealistic.

“It’s quite difficult to watch for more than a few minutes. It’s very flat, very single-paced, it’s not got rhythm, pace or emphasis,” Prof Wooldridge told the BBC.

He also pointed out that human news presenters have traditionally – in many cases – become highly trusted public figures.

“If you’re just looking at animation you’ve completely lost that connection to an anchor,” he added.

And India’s Scroll piled on:

Although the virtual anchor’s features are based on a real-life Xinhua host, Zhang Zhao, his voice remains robotic and detached.

But in some ways, everyone is asking the wrong question.  Xinhua

Things I like: Journalism.  Broadway.  Lin-Manuel Miranda.

So if you can get all three together, what’s not to like?

I stumbled on 21 Chump Street, a  This American Life project by accident, listening to the radio one morning.  It’s a musical dramatization of a 2012 story the program did on a number of drug busts in Florida high schools, based on the work of undercover police officers who posed as students.  The musical is pretty faithful to the facts, handles the nuance of the story and the he-said, she-said nature of the story well – and it works well as a musical too.  Not surprising, since it was written and narrated by a pre-Hamilton Lin-Manuel Miranda.

So it was just fun to watch.  But it also flags the need for more experimentation in how we get audiences to engage with the information we unearth and the stories we tell.  True, great writing can pull people through thousands of words. gripping documentaries can keep viewers coming back through multiple episodes, and well-designed interactive presentations can make audiences care about subjects they wouldn’t have otherwise explored.

But there are a lot more possible paths to explore, from plays and pop-up newsrooms to games to old-fashioned storybooks (even from journalists in jail! #FreeWaLoneKyawSoeOo).

I realize many of these may not scale well.  Or be financially sustainable.  (And there’s only so many Lin-Manuels out there.)

But arguably, engagement is one of the biggest challenges we face –

So it’s been a long time since I posted anything here.  I did think about it a bunch of times (honest!) but there’s been more than enough other things going on – from having two colleagues unjustly jailed in Myanmar to a multi-billion deal that nets Reuters a 30-year, $325 million-a-year contract to supply news – that this blog just hasn’t been a priority.

Still, there’s a lot happening in the world of technology and news, and even in structured journalism.  Like this nice, short piece in the New York Times on Sunday, riffing on the promise and limitations of machine learning.  And which reminds us, again, that we need to cover the algorithms that govern our lives in a much better way.

It’s mostly about how textgenrnn, a machine-learning algorithm that imitates text, has come up with a number of creative, funny and quirky new Halloween costume names – Sexy Minecraft Person or Piglet Crayon, anyone? – after being fed a series of costume names.  That’s pretty impressive, given that the system wasn’t given any information about words, grammar or spelling – it basically iterated towards things that made sense (sort of) to a human.

Which speaks to some of the power of such algorithms – their ability, in many ways, to come to “reasonable” results with a minimum of human intervention.  And – in a much more troubling way – with a minimum of human understanding as well.  As the piece notes:

Even when we can peer inside the neural network’s virtual brain and examine its virtual neurons, the rules it learns for its prediction-making are usually very hard to interpret.

…

…for the most part these algorithms are black boxes, producing predictions without explanation.

The key inputs for such systems are the training set of data – the pool of information that the system should be able to emulate – and the goals we set for the algorithm.  But the training data can be biased, leading to algorithms that turn out biased results (as Cathy O’Neil’s Weapons of Math Destruction nicely describes), and machines can take orders somewhat too literally, optimizing for conditions their creators never intended.

“My dog can play checkers.”

“That’s amazing!”

“Not really – he’s not very good.  I beat him three times out of five.”

OK, it’s a bad joke.  But it does kind of make a point about the world of artificial intelligence, big data and technology – it is amazing how far they’ve come in the relatively short time they’ve been in existence, but we also need to remember their limitations, and not believe all the hype.

At least that’s one of the major takeaways from Artificial Unintelligence, by NYU journalism professor Meredith Broussard, which I just finished on the flight from New York to Gdynia, Poland.  (Don’t ask.).  Like Weapons of Math Destruction, by Cathy O’Neil (another book I finished on a plane), this one also focuses on the misuse of technology and misplaced faith in algorithms and artificial intelligence’s ability to solve the world’s problems.

There’s much to recommend about having a high level of skepticism about the promise of AI, not just in terms of how well any particular system works, but also what assumptions and data are fed into it.  As an industry we haven’t really covered algorithmic accountability particularly well, and it’s a critical gap in fulfilling our public service mission of informing the world about things that matter to society.

And yet it’s important to also recognize how far machines have come, and what capabilities we can harness to do better journalism (even if sometimes there are questions about exactly how far they’ve come).

And one way to better harness machines is to not completely depend on them, as Meredith notes, but instead to build “human-in-the-loop” systems: