Pedro Domingos:
"The Master Algorithm" (2015)

(Copyright © 2018 Piero Scaruffi | Terms of use )
It is always fascinating to review a book (especially books on A.I.) a few years after it was published. I am writing this review in 2019, four years after the book came out. Let me start with saying that Domingos wrote an excellent book, for many reasons, so this won't sound just criticism or satire. But it is impressive how even the greatest experts can totally miss what is going to happen and what really matters.

From the outset, Domingos warns us that we are increasingly surrounded by algorithms. But he conflates technology and sociology without realizing it. The trend to have our lives run according to algorithms predates computers, and depends only in part on computers. The ones who enforce traffic laws are not computers (yet): they are highway cops, parking cops and so forth. The ones who enforce rules in the national parks of the USA are rangers, not computers. And so on. We (humans) increasingly introduce and enforce rules and regulations for just about everything. If you enter a restaurant in the USA, you are supposed to wait for a host to sit you and bring you the menu, and then wait patiently for the waiter to come and take your order, and so on: there is an algorithm even for eating at a restaurant. It is difficult to mention one activity that we can perform without being forced (by the law or by local customs) to follow an algorithm. As computers became widespread, it was natural to program those algorithms inside computers, but it is misleading to blame the computers for the algorithms that surround us: there were human beings "running" them before they started running on computers. This is the "vast algorithmic bureaucracy" that i mentioned in my book "Intelligence is not Artificial". So yes, we are surrounded by algorithms, but the reason is important: we humans like to structure our society, and there seems no limit to how much we're willing to structure it. In a sense, every society is becoming a dictatorship, with less and less freedom allowed, except that in democracies there is no dictator who is curtailing that freedom - it is the democracy itself that does it. But i digress.

The second starting point of Domingos' book is that machine learning is something special that happened in our age. With all due respect for the many friends who work in A.I., machine learning is simply the new form of statistics. Statistics analyzes data and tries to make predictions. Machine learning is simply an evolution of statistics. The real factor is the availability of a lot more data, thousands of times more data, and computers are important because no human could analyze so many data in real time. But it is misleading to claim that today's machine learning is significantly different from the old machine learning that ran good old-fashioned statistical formulas. Presenting it as the next step in the evolution of life (a` la Kurzweil) is ridiculous: machine learning is not particularly "intelligent" and very often incredibly stupid. See for example Mehran Sahami's presentation at Stanford (at an event that i chaired). Machine learning in the age of big data displays all the old problems of statistics, simply increased by an order of magnitude.

Domingos' book examined five kinds of machine learning. The one that became popular the following year (thanks mainly to Google's massive P.R. campaign in its favor) was the connectionist one (neural networks, deep learning). At the time it was impressive that a machine could "learn" to recognize faces and play the game of go. A few years later it is painfully self-evident that the machine does not learn like us. How would you call someone who needs to see thousands of bananas before it can recognize a banana? I don't think you would use the word "intelligent". Any child and pretty much any animal learns to recognize things after seeing them just once or twice: small data, not big data. The machine learning that we have today requires colossal amounts of data, big data, very big data. Imagine if you had to show a child thousands of bananas before finally the child can tell a banana from an apple. It is also painfully self-evident that these machine have no common sense. Our daily lives is not based on being able to perform big-data analysis on millions of data but on using the very few data available in intelligent ways. We make mistakes, lots of mistakes, but they are mostly reasonable mistakes, like thinking that the passer-by is our friend Joe when in fact he's someone else. Machines can make colossal mistakes, mistakes that make no sense, like mistaking a black person for a gorilla, or a traffic sign for a refrigerator (a famous mistake made by a Google neural network). In other words, the most successful form of machine learning, the connectionist one, builds machines that have virtually none of the fundamental attributes of intelligent beings. They have little in common with real brains. And that's also what neuroscientists tell us: the brain is infinitely more complex than these machines.

In 2015 Domingos optimistically wrote that a self-driving car learn by itself. In Silicon Valley, after so many accidents and at least three deaths, we now call it "the self-crashing car". And, by the way, there is no real self-driving car as they all run around with a human copilot or radio-controlled. And even the experimental ones that run in closed environments are not "learning to drive" but "learning to drive the same route over and over again". The real algorithm to watch is not the (laughable) algorithm of the self-driving car but the algorithm of the vast algorithmic bureaucracy: that bureaucracy will eventually forbid humans from driving and decide that only self-driving cars can drive, and that will solve the problem because cars will then be able to coordinate each other the same way that trains coordinate each other (no need for "machine learning", just old-fashioned machine communication that can be implemented in a few lines of software).

His relatively optimistic view of the machine-learning algorithms of 2015 sounds naive today. He mentions the recommendation algorithms of Amazon and Netflix (that he calls "matchmakers"), but today these algorithms feel like curses given the biases and distortions that seem inherent to them. He mentions Obama's presidential campaign of 2008, that used a "big data" approach, but one year after the publication of this book Trump won the presidential elections thanks to old-fashioned smearing campaigns, bullying, lies and a little help from Russian hackers. It turns out that machine-learning algorithms can easily be fooled, and can easily be used not to "derive new knowledge" but to spread disinformation.

Domingos was justifiably proud that machine learning was being applied to help biologists, but so far, despite the millions of data that are becoming available every month, machine learning has produced very little of value; and, again, scientists have to be careful not to be completely misled by the machine-learning algorithms that they use.

Domingos writes that the future of machine learning could be the "master algorithm" that combines the best of all existing algorithms. This is certainly possible. But then he thinks that this master algorithm could "derive all the knowledge in the world". And this should make philosophers cringe. Knowledge is not out there in the world. Knowledge is created by humans. For example, Michelangelo created the Sistine Chapel, and Kafka "The Castle", and TS Eliot "The Waste Land", and Leibniz the monadology, and i have written my version of the history of rock music. There is a lot out there that we want to know that is not created by human but it becomes knowledge only when humans incorporate it in some theoretical system. This doesn't mean that a machine cannot learn new knowledge but that the existing machine-learning methods don't do that. It is not just a matter of improving them. I am not even sure that it means that a machine can "derive" the knowledge of, say, a new artistic movement. A new artistic movement is created by the critics, historians, museums and art galleries. Machine learning as it is today can only analyze data, lots of data, and try to find patterns. These patterns are sometimes useful to scientists, and sometimes are simply misleading. Just like old-fashioned statistics.

Domingos' point in this book is that machine-learning algorithms can be very simple and very powerful at the same time. A clear example is evolution, which relies on a simple algorithm (essentially the one discovered by Darwin) and it has created all the life that we see on Earth (Domingos forgets to mention that the vast majority of species has gone extinct and that there is no guarantee that life will still exist a century from now, let alone for eternity). Another example is Bayes' theorem, a simple formula that is powerful for predicting the future (but any statistician can tell you that Bayes' theorem left unchecked can lead to gross misunderstandings and distortions, and in fact it has never correctly predicted the winner of the soccer world cup, something that most of us have achieved more than once). Armed with such evidence, Domingos argues that: "All knowledge - past, present and future - can be derived from data by a single, universal learning algorithm" and he calls it "the master algorithm". The first problem with this statement is that i literally don't know what it means. He gives us no definition of knowledge so we don't know if his "knowledge" includes the winners of the world cup and the next big thing in the visual arts, or just scientific knowledge of the kind that Physics and Biology study. The second problem is about the definition of "algorithm" itself: a typical algorithm can be viewed as a set of algorithms because it is a sequence of steps, and each step is an algorithm in itself. If you put together many algorithms, you get an algorithm, not something else. Any "app" on your smartphone is such an algorithm, made of many algorithms. So his "single, universal learning algorithm" is probably made of many algorithms, and most likely these algorithms interact. If we took all the computer programs and smartphone apps ever written and combines them into an algorithm, we would get pretty close to having "a single, universal learning algorithm" that does everything that we do with computers and phones; and it would include countless statistical algorithms that "learn" things from what we do. Whether this algorithm would derive "all knowledge - past, present and future" depends, again, on the definition of knowledge. The third problem is, of course, that i think this master algorithm is pretty useless. It's annoying enough to have 20 apps on my smartphone and the algorithm that Google and Microsoft devised for their email programs are awful enough: the last thing that we want is more complications. I vote for replacing today's slow complicated bulky apps with a few simple useful applications that can run on cheap computers and phones using little memory and little battery.

Domingos probably has in mind a "master algorithm" that can unlock all the secrets of the universe, i.e. all of the Physics, Biology, Neuroscience, etc that still have no solution. That's a different definition of "knowledge" from mine. There is no limit to scientific knowledge. Explain something and someone will ask "Why is that?" Explain that, and someone else will ask "Why is that?" Luckily this can go on forever, and most likely will go on forever, or at least until we self-destroy. So Domingos is probably thinking of a master algorithm that can be a useful tool to quickly solve all the pratical problems that we still have, from cancer to space travel.

Towards the ends he proposes his own "master algorithm", a program called Alchemy developed in his lab and based on the "Markov logic networks" that he invented in 2006. He humbly admits that it is just a tentative idea. Domingos sees the master algorithm as leading to a phase transition (rather than to Kurzweil's semi-religious idea of the exponential Singularity) and thinks that we will coevolve with it. At the same time, Domingos rules out the possibility that this master algorithm can take over the world (and, for example, kill us all).

Domingos' book is stimulating and very well written. It occasionally risks falling into Kurzweil-ian sensationalism and tech-spiritualism, but never becomes obnoxious. And it is very accurate, unlike most popular books on A.I. (I would just caution him from accepting Wikipedia's versions of the facts: John Holland was certainly the first PhD in computer science in Michigan, a small state of the USA, but not in the world - David Wheeler, for example, graduated eight years earlier).

As i said at the beginning, for me the most interesting thing about reading this book was the difference that four years can make in a young science like "machine learning".

TM, ®, Copyright © 2019 Piero Scaruffi