Inquire about purchasing the book | Table of Contents | Annotated Bibliography | Class on Nature of Mind

**These are excerpts and elaborations from my book "The Nature of Consciousness"**

In 1943 the US physiologist
and mathematician Warren McCulloch, in cooperation with Walter Pitts, wrote a seminal paper (“A Logical Calculus of the Ideas Immanent in
Nervous Activity”) that laid down the foundations for a computational theory of
the brain. McCulloch transformed the neuron into a
mathematical entity by assuming that it can only be in one of two possible
states (formally equivalent to the zero and the one of computer bits). These
"binary" neurons have a fixed threshold below which they never fire.
They are connected to other binary neurons through connections (or
"synapses") that can be either “inhibitory” or “excitatory”: the
former bring signals that keep the neuron from firing, the latter bring signals
that want the neuron to fire. All binary neurons integrate their input signals
at discrete intervals of time, rather than continuously. The model is therefore very elementary: if
no inhibitory synapse is active and the sum of all excitatory synapses is
greater than the threshold, then the neuron fires; otherwise it doesn’t. This
represents a rather rough approximation of the brain, but it can do for the
purpose of mathematical simulation. Next, McCulloch and Pitts proved an important theorem:
that a network of binary neurons is fully equivalent to a Universal Turing Machine, i.e., that any finite
logical proposition can be realized by such a network, i.e., that every
computer program can be implemented as a network of binary neurons. Two most
unlikely worlds as that of Neurophysiology and of Mathematics had been linked. It took a few years for the
technology to catch up with the theory. Finally, at the end of the 1950s, a few
neural machines were constructed. Frank Rosenblatt's "Perceptron" (1957), Oliver Selfridge's
"Pandemonium" (1958), Bernard Widrow's and Marcian Hoff's “Adaline”
(1960) introduced the basic concepts for building a neural network. For
simplicity purposes a neural network can be structured in layers of neurons,
the neurons of each layer firing at the same time after the neurons of the
previous layer have fired. The input pattern is fed to the input layer, whose
neurons trigger neurons in the second layer, and so forth till neurons in the
output layer are triggered at last, and a result is produced. Each neuron in a
layer can be connected to many neurons in the previous and following layer. In
practice, most implementations had only three layers: the input layer, an
intermediary layer and the output layer. After a little while, each
layer has "learned" something, but at a different level of
abstraction. In general, the layering of neurons plays a specific role. For
example, the wider the intermediate layer, the faster but less accurate the
process of categorization, and viceversa. In many cases, learning is
directed by feedback. "Supervised learning" is a way to send feedback
to the neural network by changing synaptic strengths so as to reflect the
error, or the difference between what the output is and what it should have
been; whereas in "unsupervised" learning mode the network is able to
learn categories by itself, without any external help. Unsupervised learning became feasible after
the introduction of algorithms such as Teuvo Kohonen's Self-Organized Maps
(1982), Geoffrey Hinton's and Terry Sejnowski's Boltzmann Machine (1983), Stephen
Grossberg's Adaptive Resonance Theory (1987) although the idea dates back to
British mathematician Albert Uttley’s “Informon” (“The informon: A network for
adaptive pattern recognition”, 1970). Whether supervised or not, a
neural network can be said to have learned a new concept when the weights of
the connections converge towards a stable configuration. Back to the beginning of the chapter "Connectionism and Neural Machines" | Back to the index of all chapters |