Aaah – another site about The Future of Journalism.

A dull one.  Without the  invective and ideology about free vs. paid, pajama-clad bloggers vs. stick-in-the-mud mainstream media curmudgeons, and Utopian visions of crowdsourced news vs. dark fears about falling standards you can find elsewhere.  It has words like taxonomy and persistent content in it; discusses business models and revenue streams in dull, accountant-like language; and tries to dissect the sparkling prose journalists turn out into tiny bytes of data.

But there is a purpose here, and it’s based around the idea that we as journalists haven’t really thought about how people are changing they’re accessing information, or about how we need to fundamentally rethink the way we carry out journalism and the kinds of – for want of a better word – products we turn out for them.

There’s much hand-wringing over the loss of the traditional business model of news, it’s true.  Perhaps too much.  And this site will contribute its share.  But hopefully it’ll also explore some of the less-explored questions about where the profession goes in a digital age.   And lay out some of the thinking behind one concrete idea that might help move the business forward: Something I’m calling Structured Journalism.

So, welcome – and I hope you find this interesting.

Just a quick post to congratulate colleagues Jason Szep and Andrew R.C. Marshall for winning a Pulitzer for International Reporting, Reuters’ first in a text category, for their great coverage of the Rohingya Muslim minority in Myanmar.  Great work that had a real impact and helped save lives. 

Awarded to Jason Szep and Andrew R.C. Marshallof Reuters for their courageous reports on the violent persecution of the Rohingya, a Muslim minority in Myanmar that, in efforts to flee the country, often falls victim to predatory human-trafficking networks.

Also: Megan Twohey is a finalist in the Investigative category “for her exposure of an underground Internet marketplace where parents could bypass social welfare regulations and get rid of children they had adopted overseas but no longer wanted, the stories triggering governmental action to curb the practice.“

And the amazing Goran Tomasevic is also a finalist for his series of photographs covering combal in Syria.

A really nice day to be in the Reuters newsroom.

If machines could write stories as well as humans, what would you have them do? Write faster, more cheaply, and on more topics than humans can, almost certainly.  But what else?

That wasn’t the core part of Thursday’s panel discussion on “computational storytelling” organized by Nick Diakopoulos of the Tow Center for Digital Journalism at Columbia Journalism School, and moderated by yours truly, but it threw up a number of smart, interesting ideas that get us beyond doing what we’ve always done and into truly taking advantage of technology to help fulfil journalism’s mission.

Among the ideas: Different versions of the same story for different people; hypothetical stories to explore different theories, and “perturbation analysis” to understand how solid a story’s conclusions might be.

But first, some background on the panel, which featured three top-notch computer scientists:  Jichen Zhu of Drexel, Mark Riedl of Georgia Tech and Larry Birnbaum of Northwestern U and Narrative Science.  We spent nearly two hours exploring a wide range of questions about machine-generated narratives, from biases inherent in algorithms, data and interaction design to the importance of the provenance of data in a world where stories are created purely from data to our inherent (if unjustified) trust of narrative and anthropomorphic avatars.

You can watch it – telling insights, corny jokes and all – here.  It’s pretty interesting, even if I say so myself.

For example, we touched on the subject of how there’s some research to suggest that people respond better to avatars than to impersonal interfaces.  It may make sense, if that’s the case, to try and create more human-like user interactions for news sites.  But does it also mean that Amazon would sell more stuff if recommendations came to us in the form

So I’m off to moderate a panel discussion on computational storytelling at the Tow Center for Digital Journalism tonight.

It should be interesting, not just because the automated production of stories from data is rapidly coming into the mainstream, but also because it throws up potentially huge questions about how news judgments can be embedded into algorithms, and how newsrooms might have to evolve to allow that to happen as part of their workflow.

And beyond that, what are the uses of such capabilities, beyond replacing or adding to what humans currently do – ie, write stories based on judgments of newsworthiness for a broad audience?  Narrative Science and others have shown the value in expanding coverage to events such as high school sports that in the past would never have merited a mention – in effect serving microaudiences at a reasonable cost. But having the capability (in theory, at least) to create, on the fly and cheaply, news stories that speak to audiences of as small as one, opens up a raft of other potential uses, including much more personalization and the creation of “news-on-demand.”  And that opens up questions about what’s newsworthy as well.

Nor should such discussions be limited to stories; after all, storytelling comes in many different forms, including data visualizations and news games.

All in all, it should be a a great discussion. The panelists are Larry Birnbaum, of Northwestern University and Narrative Science, Mark Riedl from Georgia Tech, and Jichen Zhu of Drexel University.

So it’s been a week now since returning home from the NICAR conference in Baltimore, and I still can’t get over the size of the event: 1,000 attendees, up 400 from last year’s meeting, which was already a record, and three times as many as IRE was struggling to get to show up not that many years ago.

Which is great news on so many levels, even if the crush of people there made it awfully hard to fight through to the bar to get a drink.  Not that I noticed the conference was any more sober than normal.  But I digress.

It’s great because the conference is one place pulling together the disparate strands of “classic” computer-assisted reporters, journalist-technologists, design and user interface experts and computer scientists – and we keep to get together more if we’re to play a leading role in how the industry develops.  And it’s clear – at least based on a couple of observations in Baltimore – that we’ve got a long ways to go before we all understand each others’ language.

Back in the day when printing presses reigned supreme, newsrooms could focus on getting and writing the news; the hugely important intricacies of print runs, color quality, delivery routes and subscription fulfillment were someone else’s problem, and probably quite rightly so. But these days, the means of production is digital, the internet delivers your product, and not only can anyone start up a news site at will, the form and format of news itself is rapidly evolving. So if we don’t take a broader view of what it means to gather, analyze, produce and deliver news – well, someone else will.

But that takes a fairly big change of mindset, and we’re not there by a long shot. At the level of story or project, it’s still

As Bill Adair notes at Poynter today, four of us – Bill, Laura and Chris Amico and myself – locked ourselves in a windowless conference room at Reuters last week, sustained only by cold pizza, and spent a day discussing… well, what exactly?

It’s not that we didn’t know what we were talking about.  Let me rephrase that.  We knew we wanted to talk about projects like Politifact, Homicide Watch and Connected China, and how to evangelize the kind of journalism they do, but there was also a question about what exactly to call this kind of journalism.  To be sure, it wasn’t a big part of the conversation, but it was helpful to actually discuss the name.

Bill, of course, winds up calling it “structured journalism” in his piece – which works for me, not least because it means I don’t have to rename this site.  But there were other ideas, including “narrative data” from Laura, “frameworks for reporting” from Chris, and “new forms of storytelling” from Bill, all of which have lots to recommend them.

But before delving into that, it’s probably worth asking what we actually mean when we talk about this kind of journalism – which is in fact the way Laura started off our meeting.

I’m not sure we settled on a firm definition, but we did define a chunk of characteristics common to these kinds of projects: They atomize the information collected in the daily course of journalism

Ah, Baltimore.  Home to Avon Barksdale, Jimmy McNulty and the Wire.  Or, if you’re there this weekend, several hundred inebriated journalists who have wrangled approval for travel and lodging from their bosses on the basis that they’ll be learning about the latest and greatest advancements in data and computational journalism.

At least that’s my excuse for heading there, and I’m sticking to it.

So apparently this year’s CAR conference is going to be bigger than last year’s, and last year’s already hit a record in attendance. Which is either a sign that data journalism is really coming into the mainstream, or that it’s everyone’s flavor of the month.  Or both. Either way, it’s a good thing for the industry if more people think it’s worth understanding more about data.  And/or understanding more about journalism, and how the two fields mix.

Because the other thing that’s been happening at the conference is that there are a lot more data scientists and non-journalists showing up; they bring skills and experiences that we don’t have, but we offer them our skills and ways of looking at things that they don’t have – from a public interest perspective to reporting techniques to understanding narrative structure.

In a world where anyone can practice journalism – and where collaboration is increasingly important – it’s important to try and spread the ideals of good journalism to people who are interested in the process of gathering, analyzing and publishing facts, wherever they work and whatever they call themselves.

At least that’s my excuse for heavy socializing at the bar, and I’m sticking to it.

Much has been written about the departure of Ezra Klein, of Wonkblog fame, from the Washington Post to a new “Project X” at Vox Media; some coverage has focused on the rise of journalists’ brands in their own right, while other pieces have looked at the business viability of such a venture.

But the one that interested me the most was Benjamin Wallace’s piece in New York magazine that discussed Klein’s philosophy of news and the problem he wanted to try to solve.

Klein’s theory of the news grew out of his frustration with the industry’s relentless presentism, with the fact that, because media organizations prioritize what’s new (that’s why it’s called news), an article about the latest development in Syria’s civil war would likely not mention the single most important fact necessary to understand what is happening: the historical enmity between Alawites and Sunnis. There is little allowance made for readers coming to a story late and an assumption that anyone who’s been following a story over time will remember all the relevant contextual information.

You have to love the word ‘presentism’, and it’s a great one to steal; certainly Wonkblog has done its share to get beyond the day’s news and focus on things that Klein believes matters to readers, and to examine them in a lively, but detailed way.  It’s not designed in the way that Homicide Watch, Politifact or Connected China are – which is to say, with an underlying data structure that aggregates daily updates into more persistent types of content – but it hits all the themes of structured journalism, from the rethinking of what news and the atomic structure of news should be, to the new ways people come to information, to the idea that we haven’t begun to really take advantage of the longevity of content in the digital age.  As the New York piece notes:

The answer, as Klein sees it, lies in the handling of what he calls “persistent content,” the more static information that makes the new stuff make sense. And here, he believes, the Internet has untapped potential. Traditional media organizations have taken advantage of the Internet’s speed but not its longevity.

…

People who think about digital journalism distinguish between what they call unchanging “stock content” and ephemeral “flow content.” Klein believes that distinction is unhelpfully stark. “We’re interested in ending the ‘versus’ there,” he says. “We believe there are rivers and lakes of content that work together.”

Absolutely.  The goal shouldn’t be to run two news organizations, one “fast” and one “in-depth” that don’t speak to each other; it should be to figure out how to more effectively turn the first into the building blocks of the second. 

There was a great piece in the New York Times magazine a couple of months ago – in fact, just about when I got too busy to write very regularly here, hence the last couple-of-months hiatus – about the world’s best poker-playing machine.  It’s so good that, despite the complexities of the game – from bluffing and knowing when to raise and fold and so on – it’s pretty much unbeatable by any human.  

And no one – not even its designer – actually knows how it works.  And increasingly, that’s the kind of machines we’ll be dealing with, for good and bad.

That’s because it’s based on a machine-learning system that modified itself over the course of millions of hands.  As Michael Kaplan notes in the piece, when investor Gregg Giuffria wanted to test the quality of the game, he had a problem.

…because these had been developed through self-training and not created by humans, there was no source code — the computer instructions written out by programmers — to analyze. You couldn’t track the logic behind the system’s actions. “We had to take a black-box approach,” says Bob Honeycutt, Giuffria’s lead engineer on the project. They had to look at the results without being able to know how they were produced. Honeycutt customized math-based programs that look for probabilities to play poker against (engineer) Fredrik Dahl’s neural nets. Honeycutt’s software lost. The neural nets showed no patterns or anomalies.

Machine learning isn’t new, of course.  But its use is increasing – even in journalism – even as the role of machines and algorithms in our lives are increasing.

The technology behind Dahl’s game has the potential to do a lot more than simply taking money from casino customers. Dahl could see it being adapted to make credibility assessments, like deciding who should get a loan, for example, by analyzing applicants in comparison with databases of borrowers who repaid their loans and those who did not.

So what does it mean if we’re not able to fully unpick and understand how some of those assessments are made?  There are already real questions about how transparent the systems are that provide us with credit ratings, assess teachers or price goods based on our location and browser history; but at least in theory someone knew how they operated.  What if no one does?

Who do you think you are?

I was looking at an interesting book, Whistling Vivaldi: How Stereotypes Affect Us and What We Can Do, by Claude Steele last night, and was struck by some experiments he describes that show how wildly variable test results can be, depending on how they’re described to the people who are taking them.  And that should be a reminder about really understanding all the biases and factors that go into creating data.

White students at Princeton who who were told their scores on a game of miniature golf reflected their natural athletic ability did much worse then those who were just told to play a round; Black students in the same experiment showed no such variation.  On the other hand, when the Black subjects were told the game was a test of “sports strategic intelligence,” they did much worse that those who weren’t told anything.

What’s going on here?  Steele makes the case that we internalize stereotypes – so White students are unconsciously living up to the gross generalization that they don’t have tremendous natural physical ability, and the Black students are similarly hobbled by an internalized view that they’re not great at strategic thinking.

It’s an interesting theory – and I confess I haven’t finished the book, so I don’t know much more than that – but the experimental results are remarkable.  He cites another study where women do better on a math test when they’re told it doesn’t reflect gender differences than when they simply take the test – again, evidence that our internal perceptions of ourselves, and about what we are and aren’t supposed to be good at, affect our actual performance.

Which is really fascinating stuff, and throws up all sorts of questions of identity, stereotype and self-stigmatization.  But what has this got to do with data journalism?

Great news: Laura and Chris Amico, the team behind the smart, award-winning – revolutionary – Homicide Watch site, are branching out into a new area.  And they’re bringing the notions of structured journalism with them.

In a post at Nieman Labs, Laura talks about how they’ll be partnering with Boston public radio station WBUR to launch a vertical focused on K-12 education in Massachusetts. And not just any vertical: One with structured data, collected by journalists, at its heart – adding Learning Lab, as they’re calling it, to the small roster of news sites such as Homicide Watch, Politifact, and Connected China that are experimenting with this form of journalism.

The expansion of the narrative data concept from violent crime to education is a significant one, and Learning Lab offers Chris and me an opportunity to further test our theories about innovative news structures, this time within an existing news organization. Our goal is to turn daily reporting into structured, reusable data. This means we will be able to see how any given reform initiative fits into the larger picture of education in the Commonwealth, and it means we can ask questions of our data. It means we won’t lose information, and we’ll be able to provide the public with access to source documents, data, and tools to increase visibility and transparency of specific reform efforts and experiments.

It’s early days yet, but Laura and Chris will now get to marry their experience from Homicide Watch with more resources, on a topic that’s of huge interest – and potentially dripping with data – to a large community. Education matters to lots of parents – and certainly a fair number of students – but they generally come to the topic in multiple different ways, and at different times.  Everyone cares about the same broad issues of funding, teachers’ pay and working conditions, presumably; but someone trying to figure out what school to send their child to has far different interests from someone who wants to know if or why the school their son is in is underperforming state averages; and so on.

Which should provide an excellent test of how well a large audience that’s as engaged with deep dives into information as it is with breaking news and investigative reports will respond to a structured journalism-type site. It’ll certainly be a very different experience from most other news sites, as Laura points out (and thanks for the shout-out, Laura):

Homicide Watch D.C. was a project in what Reg Chua calls “structured journalism.” This, I truly believe, is what set Homicide Watch D.C. apart from all other murder blogs, homicide maps, and similar projects.

I’m sure Laura and Chris will be just as successful with Learning Lab – not just with creating structures that serve their audience better, but using those structures to do better reporting.  Which will help serve their audience even better.  Now that’s a virtuous cycle.

And as we build we will work to find ways to make individual reporters more effective in the work they are already doing. We believe the right tools make good reporters more powerful. And powerful reporters are at the heart of informed, engaged and active communities. This is the space where Homicide Watch succeeds. And it is where Learning Lab will thrive, too.