
By: Albertus Magnus
Re: As they lay dying...

As They Lay Dying

Near the end, artificial neural networks become creative

Not too many personal computers are known to hallucinate. But the one
belonging to Steven Thaler has been doing so, off and on, for the past couple
of years. The physicist, at McDonnell Douglas in St. Louis, has been
exploring what happens as an artificial neural network breaks down. But
rather than allowing the network to peter out into oblivion, Thaler has a
second network observe the last gasps of its dying sibling. Some of those
near-death experiences, it turns out, are novel solutions to the problem the
net was designed to solve. Thaler says he has found a kind of creativity
machine that can function more quickly and efficiently than traditional
computer programs can.

An artificial neural network is software written to mimic the function and
organization of biological neurons. The system consists of units
(representing neurons) connected by links (standing in for dendrites and
axons). Like the brain, an artificial network can learn: the programmer
presents it with training patterns, which it learns by adjusting the
strengths, or weights, of the links. Many researchers use these networks to
model brain function and, by destroying part of the net, to mimic disorders
such as dyslexia.

Thaler took that concept one step further: he killed his networks. As the
links between units were randomly severed over time, the net produced not
only gibberish but also some of its original training patterns. For instance,
a neural net taught to act as an "or" logic gate would often begin spitting
out its trained patterns of 0 and 1 (yes or no) in addition to nonsense (that
is, other numbers).

Nothing mystical is going on. Thaler's explanation is that in a fully
functioning network, all the weighted links to a given unit are about the
same in magnitude but opposite in sign. The sum of several weights to the
unit, therefore, is often zero. Without any input, the unit might not notice
the loss of those links, because it might not have been receiving any signals
from them anyway. A few surviving units are often enough to generate coherent
output. Indeed, Thaler used his earlier work to model human near-death
visions, suggesting that the reported imagery may have some mathematical
basis rather than being purely biochemical.

Thaler soon began experimenting with more sophisticated nets and found that
the output contained some unusual juxtapositions of learned patterns and
balderdash. To see if those combinations would be useful or esthetically
pleasing, he drafted a second neural net to sort through the output and
record the most interesting products.

By keeping the dying network partially alive, Thaler has been able to
generate many kinds of novelties. For instance, after feeding 30 years' worth
of top-10 musical tunes to the networks and letting them run for a few days,
Thaler created 11,000 songs--which he has copyrighted. "This diabolical plot
will make me the most prolific songwriter of all time," he jokes. From
photographs of Thaler's own body movements, another net generated dances.
More serious applications included searches for ultrahard materials and for
plausible automobile designs.

But what can Thaler's net offer that more traditional programs cannot?
"That's the big question," notes Andy Clark, who studies philosophy and
neural science at Washington University. The network would have to be
compared with classic creativity programs such as EURISKO, Clark observes,
which established a benchmark. That algorithm, developed along more
traditional programming lines in the 1980s by Douglas B. Lenat and his
colleagues at Stanford University, defeated all other programs in various
games by coming up with unorthodox solutions. In a military competition, for
example, it sank its own disabled ships to improve the overall
maneuverability of its fleet.

Nevertheless, EURISKO requires a human to update its heuristics, whereas
Thaler's system functions automatically, so dying neural nets may have an
advantage in some applications. Thaler also believes his software has
philosophical implications. "I am claiming this is a model of consciousness,"
he asserts. "The images are triggered by internal noise--the network
manufactures experiences from stored experiences."

But whether the net emulates the creative mind is debatable. "Creativity
isn't a thing in itself," notes mathematical biologist Stephen Grossberg of
Boston University. If the network were truly a model of consciousness, it
would have to explain something about a particular function of the
brain--such as its ability to tune in to only one conversation at a cocktail
party. "It may be telling us something about hallucination," Clark echoes,
"but creativity seems to be a long way away."--Philip Yam


SCIENTIFIC AMERICAN  May 1995  Volume 272  Number 5  Page 24

... Is language the adequate expression of all realities? - Nietzsche
