(These are excerpts from my book "Intelligence is not Artificial")
Footnote: Neural Networks and the Math of Glass
Hopfield based his model on the physics of spin glasses. He wasn't the first one to make the connection between neurons and spins. The mathematician Neville Temperley at the Atomic Weapons Research Establishment in Britain was perhaps the first to realize that a network of neurons behaves like a network of spins ("Memory - The Analogy with Ferromagnetic Hysteresis", 1955). Almost twenty years later the idea was resurrected by Bill Little at Stanford University (the physicist who in 1964 had shown that superconductivity can happen at room temperature), who published a study about the analogy between spin systems and both short-term and long-term memory, between the firing or not firing of a neuron and the up and down spin states of an atom ("The Existence of Persistent States in the Brain", 1974).
In 1975 the physicists Sam Edwards and Philip Anderson at Cambridge University had solved the problem of the disordered states of the so-called "spin glass", the ultimate disordered system (Theory of Spin Glasses", 1975) and the Italian physicist Giorgio Parisi had found a more general solution ("Mean Field Theory for Spin Glasses", 1980). Hopfield extended their theory of spin glasses to neural networks, a fact even better clarified by the Israeli physicists Daniel Amit, Hanoch Gutfreund and Haim Sompolinsky at the Hebrew University in Israel ("Storing Infinite Numbers of Patterns in a Spin-Glass Model of Neural Networks", 1985).
While known since ancient times, glass is one of the most mysterious materials. In fact, physicist can't even decide whether it is a liquid or a solid. Even when it is shaped in smooth and elegant structures, glass is an example of disordered material. Glass is stuck, or "quenched," in a low-temperature disordered state. Physics knows well how to study disordered states caused by high temperatures, when atoms move frantically; but glass is disordered at low temperatures. The atoms of a glass are distributed at random locations but standing still. When water becomes ice, it undergoes a phase transition. What is even more puzzling about glass is that a liquid becomes glass without undergoing a phase transition: glass is, in a sense, a supercooled liquid. A glass is a liquid that gets more and more viscous while it is being cooled until it eventually stops flowing. Spin glasses are materials that exhibit the same kind of "quenched" disorder, the opposite of "annealed" disorder. (The "glass" in the name is a misnomer: they are called that way only because of the analogy with the disorder of glass). They are systems far from equilibrium whose study has provided much of the mathematics used to study complex systems, for example the methods to solve combinatorial optimization problems.
Any real-world system is disordered, or, better, has a component of disorder. If we "quench" a system, the state is instantaneously changed to a permanent state far from equilibrium with the environment (imagine dipping an incandescent metal into a bucket of ice). If we "anneal" the system, the state is changed gradually so that the system is almost in equilibrium with its environment at all times (e.g., if we cool the hot metal slowly). Quenched disorder is frozen in time, annealed disorder evolves in time (e.g., it can reverse itself). Quenched disorder is much harder to model mathematically than annealed disorder. Nonetheless, the applications are intriguing, from protein folding to neural networks.
For example, Arkadiusz Jedrzejewski and Katarzyna Sznajd-Weron at Wroclaw University in Poland described how the two main psychological theories of person's behavior are theories of disorder: respectively, the theory that the situation is more important to determine a person's behavior is a theory of annealed disorder whereas the theory the personality traits are more important is a theory of quenched disorder ("Person-Situation Debate Revisited", 2017). The "person-situation debate" in social psychology pitted the trait theorists, such as Hans Eysenck at King's College London, who wrote "Dimensions of Personality" (1947), and Raymond Cattell at the University of Illinois, who wrote "Personality" (1950), and one of the first psychologists to use a computer, the Illiac (operational in 1952), against the situationists, such as Walter Mischel at Stanford, whose book "Personality and Assessment" (1968) started the whole riot, and Richard Nisbett at the University of Michigan ("The Trait Construct in Lay and Professional Psychology", 1980).
Needless to say, a spin glass is not a brain: there is virtually nothing in our brain that is similar to a spin glass. Alas, there is little that is symmetric and mathematically elegant inside our brain.
But i digress...
Back to the Table of Contents
Purchase "Intelligence is not Artificial")