(These are excerpts from my book "Intelligence is not Artificial")
Artificial General Intelligence and the No Free Lunch Theorems
In 1995 and 1996 computer scientist David Wolpert at the Santa Fe Institute proved some of the most famous theorems in computer science, the "No Free Lunch" theorems, that apply to search ("No Free Lunch Theorems for Search", 1995), supervised learning ("The Lack of a prior Distinctions between Learning Algorithms", 1996), and optimization ("No Free Lunch Theorems for Optimization", December 1996). He proved that no learning algorithm can possibly excel at learning everything: for every pattern that a learner masters, there is another pattern that the same learner cannot master. Shortly afterwards, Joseph Culberson at the University of Alberta in Canada published an "algorithmic view" of the No Free Lunch theorems that related them to complexity theory ("On the Futility of Blind Search", 1996). These theorems are clearly an obstacle towards the dream of a universal learning algorithm, towards an artificial intelligence that can be "general". Of course, Wolpert proved his theorems under some assumptions (in particular, they considered only discrete functions) which may or may not apply depending on which machine learning method you choose. Anne Auger and Olivier Teytaud at INRIA in France (the National Institute for Research in Informatics and Automation) proved that the theorems do not hold in continous domains ("Continuous Lunches are Free", 2007); but then Michael Vose of the University of Tennessee and others proved that they do hold in arbitrary domains ("Reinterpreting No Free Lunch", 2009).
The debate has continued, and its impact on the possibility of an artificial general intelligence is unresolved.
As Erik Hoel of Columbia University wrote in 2017: "Superintelligence is a free lunch, and there are no free lunches". He believes that evolution is all about learning new skills while forgetting others, so that new forms of intelligence are being continuously created, but none is "general". Whenever a mutation makes an organism fit in an environment, it also makes it less fit in other environments. And perhaps, at the individual level: as we get more intelligent in doing something, we also get more stupid in doing something else.
We humans (creatures to which the "no free lunch" theorem doesn't seem to apply) can safely conclude that learning without knowledge is terribly difficult, and our innate knowledge comes from evolution.
Back to the Table of Contents
Purchase "Intelligence is not Artificial")