(These are excerpts from my book "Intelligence is not Artificial")
Mathematical Teaser: If you Really Have to... Phase Transitions in Intelligence
An exponential increase (especially if it is only about computational speed) is not enough to demonstrate that a qualitative change will ever take place. If you build exponentially faster cars, you will eventually get a car that can travel close to the speed of light, but it would still be a car, not an elephant.
Should machines ever reach a superhuman level of intelligence, one would imagine that this would entail a sequence of phase transitions, e.g. from mere arithmetic calculation to pattern recognition to higher and higher forms of mental faculties.
A better paradigm to analyze the behavior of large-scale computation was suggested by Bernardo Huberman and Tad Hogg at Xerox PARC, one year before Hans Moravec's "Mind Children" came out: studying which phase transitions can computational systems undergo as they become bigger and more complex ("Phase Transitions in Artificial Intelligence Systems", 1987).
This paradigm echoes similar ideas proposed by scholars who studied the human mind. For example, the Canadian neuropsychologist Merlin Donald, in his book "Origins of the Modern Mind" (1991), argued that the modern mind of symbolic thought arose from a non-symbolic form of intelligence through gradual absorption of new representational systems. The four "phase transitions" envisioned by Donald roughly correspond to the stages of cognitive growth in children that were studied by the Swiss psychologist Jean Piaget (his classic "The Language and Thought of the Child" of 1923) and by the Russian psychologist Lev Vygotsky (his classic "Thought and Language" of 1934): children follow a path of "phase transitions" from non-symbolic to full-fledged symbolic thinking.
Cognitive faculties are not fixed at birth but evolve during the lifetime of the individual, and the evolution is not smooth but due to quantum jumps in cognitive skills. Precisely, the development of children's intellect proceeds from simple mental arrangements to progressively more complex ones not by gradual evolution but by sudden rearrangements of mental operations that produce qualitatively new forms of thought. First a child lives a "literal" sensorymotor life, then the child begins to deal with internal symbols, then the child learns to perform internal manipulations on symbols that represent real objects, and, finally, the child's mental life extends to abstract objects. Piaget's four transition phases start with a stage in which the dominant factor is perception, which is irreversible, and end with a stage in which the dominant factor is thought, which is reversible.
Jerome Bruner at Harvard reached a similar conclusion: intellectual abilities develop in three stages ("Studies in Cognitive Growth", 1966). Maybe we should study the phase transitions of a computational system and build the equivalent of Piaget's and Vygotsky's epistemological theories, and from that fusion of Physics, Psychology and Computational Mathematics we may learn something about the kind of "intelligences" that are possible beyond ours.
There is a vast literature in the "stages" of human cognitive development, which sometimes varies from Piaget's original formulation. In more recent times one finding that struck me was the study "Beliefs about Beliefs" (1983) by psychologists Heinz Wimmer (University of Salzburg) and Josef Perner (University of Sussex). Three-year-olds tend to fail "false-belief" problems which become easy to solve just one year later: that's an impressive "phase transition" in human cognition. The problem has to do with guessing what a person thinks, not what with actually happened: what happened is obvious. A person leaves an object in a place and then walks out of the room. A second person walks in and moves the object to another place. The children are asked to predict where the first person will look for the object when she returns to the room. Three-year-olds tend to answer that the person will look for the object where the second person moved it. Four-year-olds correctly answer that the person will look for the object where she left it. What happens between age three and age four is not known, but obviously the brain undergoes a "phase transition" such that it can now correctly evaluate false-belief problems. Alison Gopnik at UC Berkeley, pioneer of the "child-scientist" theory, has argued that the learning of 2- to 4-year-old children can be modeled with Bayesian inference networks ("A Theory of Causal Learning in Children", 2004), the kind of reasoning that deep learning uses.
The philosopher Hubert Dreyfus wrote the book "What Computers can't do" (1979) to criticize expert systems, but indirectly provided another perspective on the phase transitions of human intelligence. He broke down human acquisition of performance into five stages. First, we are born novices: we simply follow the rules (an instructor, a manual). The moves of novices are not secure and not fluid, although they can be technically correct. Sometimes applying a rule is plain silly, but the novice will still do so because he doesn't know better. Eventually, after practicing for a while, we become advanced beginners. At this stage we are capable of modifying rules based on the situation. Our behavior is still driven by rules but it doesn't look as awkward. Competent humans, the next stage of experience, follow rules but in a very fluid manner. Their rules are also adaptable: the competent human knows that she can modify the rules. In fact, she will feel guilty if something goes wrong, even if she followed the proper rules. Proficient performers do not even follow rules anymore: they act by reflex. The fact that they have encountered similar situations many times before matters more than the original rules. Experts, the final stage, do not even remember the rules. Sometimes if they have to articulate them they can't even figure them out. They just act based on their expertise and their intuition. They are often not even aware of what they are doing. An expert driver does not realize that she is shifting gears and at which point she is shifting gears. She just shifts gear when it's appropriate to. An expert has synthesized experience in an unconscious behavior that reacts instantaneously to a complex situation. What the expert knows cannot be decomposed in rules.
A failure usually results in degradation: an expert driver does not even remember the rules for starting a car, but, if she can't start the car, she gradually walks down the ladder from expert to merely competent all the way down to novice, and, if nothing in her experience helps, she will finally pick up the driver's manual to figure out why the car won't start.
Neural networks belong to the class of complex systems, which are characterized by nonlinear dynamics.
In 1978 Jack Cowan (who was now at the University of Chicago) showed that neural networks have a mathematical structure that closely resembles that of quantum field theory (unpublished until "Stochastic Neurodynamics", 1991). Neurons can be in three, not two, states: quiescent, stimulated, and refractory (a period during which the neuron would not react to a new stimulation). Cowan even showed that the neurodynamics of a three-state neuron is described by mathematical matrices that are similar to the ones used in 1964 by the physicist Murray Gell-Mann to describe quarks. The algebra of state transitions in a neural network is eerily reminiscent of the algebra of quantum chromodynamics. Cowan, working with the mathematician Bard Ermentrout of University of Pittsburgh, also applied bifurcation theory to the analysis of neural field equations ("Large Scale Spatially Organized Activity in Neural Nets", 1978, but published only in 1980).
Elizabeth Gardner at the University of Edinburgh, not coincidentally an expert in spin glass theory, applied statistical mechanics to neural networks but she died of cancer a few weeks before her two papers were published (notably "The Phase Space of Interactions in Neural Networks Models", 1988). Michael Biehl at the Institute for Theoretical Physics of Wuerzburg in Germany ("Statistical Mechanics of Unsupervised Structure Recognition", 1994) has studied, in general, phase transitions in networks (whether the World-wide Web, ecological nets, social nets, cellular nets, linguistic nets or neural nets).
Both physics and neural networks study systems with many degree of freedoms: physics studies many-body interactions, neural networks processes data in high dimensions. Physics uses a trick called "renormalization" to handle complex systems with many degrees of freedom. Neural networks use the approximation tricks of deep learning. The connection between the two fields has been mainly explored by physicists such as Pankaj Mehta of Boston University and David Schwab of Northwestern University ("An Exact Mapping between the Variational Renormalization Group and Deep Learning", 2014).
"You must be the change you wish to see in the world" (Mahatma Gandhi).
Back to the Table of Contents
Purchase "Intelligence is not Artificial")