There was a great piece in the New York Times magazine a couple of months ago – in fact, just about when I got too busy to write very regularly here, hence the last couple-of-months hiatus – about the world’s best poker-playing machine. It’s so good that, despite the complexities of the game – from bluffing and knowing when to raise and fold and so on – it’s pretty much unbeatable by any human.
And no one – not even its designer – actually knows how it works. And increasingly, that’s the kind of machines we’ll be dealing with, for good and bad.
That’s because it’s based on a machine-learning system that modified itself over the course of millions of hands. As Michael Kaplan notes in the piece, when investor Gregg Giuffria wanted to test the quality of the game, he had a problem.
…because these had been developed through self-training and not created by humans, there was no source code — the computer instructions written out by programmers — to analyze. You couldn’t track the logic behind the system’s actions. “We had to take a black-box approach,” says Bob Honeycutt, Giuffria’s lead engineer on the project. They had to look at the results without being able to know how they were produced. Honeycutt customized math-based programs that look for probabilities to play poker against (engineer) Fredrik Dahl’s neural nets. Honeycutt’s software lost. The neural nets showed no patterns or anomalies.
Machine learning isn’t new, of course. But its use is increasing – even in journalism – even as the role of machines and algorithms in our lives are increasing.
The technology behind Dahl’s game has the potential to do a lot more than simply taking money from casino customers. Dahl could see it being adapted to make credibility assessments, like deciding who should get a loan, for example, by analyzing applicants in comparison with databases of borrowers who repaid their loans and those who did not.
So what does it mean if we’re not able to fully unpick and understand how some of those assessments are made? There are already real questions about how transparent the systems are that provide us with credit ratings, assess teachers or price goods based on our location and browser history; but at least in theory someone knew how they operated. What if no one does?
Of course, there are huge advantages to embracing machine learning as a way of much more quickly improving computing over very complex systems. But its rise as an important way decisions are made raises important questions for journalism as it tries to explain the world and hold decision-makers accountable.
Even without the rise of machine learning, we should be covering the algorithms that manage our lives more closely, a point that Nick Diakapolous makes.
Who writes them? Who selects them? How well do they work? How fair are they – and what does fairness mean in a world of much more granular personalization, anyway? How should they be regulated? And how can we cover them? And how can we cover them effectively if their workings are a mystery – a black box – to us?
I hope to dive more into these topics in the next couple of months. Meanwhile, Happy New Year. This isn’t – yet – the year Skynet takes over. I hope.