Neural Networks for Physicists 5

On August 23-26, 1995, the Theoretical Physics Institute at the University of Minnesota hosted NNP5, the fifth edition of the conference "Neural Networks for Physicists," sponsored by the Theoretical Physics Institute, the Cray Research Foundation, and the University of Minnesota Supercomputer Institute.

The format and size of the workshop have remained the same over the years. This year, 26 talks were given by speakers from the United States and Europe to an audience of about 45 participants. Neural Networks for Physicists is a small, informal conference aimed at fostering interaction and Socratic learning. By all appearances the goal was successfully met.

The introductory talk, by Professor Charles Anderson, Washington University, St. Louis, proposed that neuronal outputs, far from being binary-like (either firing or not firing), encode a complete probability distribution function, and that some neurons play a key role by modulating the strength of the connection between two others. This multiplicative function might be realized simply through coincidence detection of incoming pulses in the dendrite. Present experiments only measure the average firing rate, i.e., the mean of the above probability density function, for one cell at a time. Simultaneous measurements on several interacting cells would shed invaluable light on control and learning dynamics. Meanwhile, simple models appear sufficient to describe muscular, motor and heartbeat control.

Another topic, central to the workshop again this year, was vision. While computer scientists polish their pattern recognition engines, aiming at a classification of objects based on a single three-dimensional image, independent of rotation and translation, biologists are investigating all aspects of the visual system. Measuring the response of a single nerve cell to visual stimuli reveals several surprises. First of all, this response is not time-invariant, but can be "modulated," for instance, by the orientation of the head; this effect can be obtained effectively by postulating the presence of neurons which shift perception from one side to another, as advocated by Charles Anderson. However, the existence of such three-body interactions in the brain is still controversial. Second, each cell is tuned to respond best to a global pattern rather than a single bright spot. Remarkably, this is also the kind of response pattern which emerges in an artificial neural network optimized for sparseness of coding on natural images. If very few firing neurons should represent a natural scene, then each of them should be sensitive to a wavelet-like pattern. Data is then further processed in the brain to match concepts. It was emphasized that the two goals of discriminating between slightly different images, and of generalizing a concept to a similar image, are mutually conflicting and require different kinds of learning. Finally, eye-motion control was looked at anew by Dr. Sebastian Seung of Bell Labs. He pointed out that it is not the saccadic motion of the eye which is hard to explain, but rather its stability between saccades. He argued convincingly that the neurons controlling such motion must form a dynamical system with an attractor of marginal stability. A saccade briefly disturbs the system away from one point on the attractor, then it quickly relaxes to another such point.

A cautionary note for biological modelers was sounded by Prof. James Anderson of Brown University. His viewpoint is somewhat reminiscent of Anthony Damasio's in his recent book Descartes' Error. He stressed that the brain is anything but a machine for symbolic concept manipulation. Each abstract concept is indelibly tied to sensory input (e.g., the magnitude of a number is attached to an impression of size), which participates in intuition and reasoning. He also emphasized that accuracy, of memory retrieval for instance, should not be the goal of a neural model. The human brain is rather error-prone, and accuracy is sacrificed in favor of flexibility. A similar proposal came from a physicist. Adapting the highly successful idea of self-organized criticality to a new area, he offered the thesis that synaptic weights in the brain evolve dynamically to a state of maximum plasticity: they are maximally affected by a small change in the training input.

Instead of applying to biology a concept borrowed from physics, a series of talks described the success of the reverse approach: applying "genetic" algorithms to optimization and control problems. These Monte Carlo algorithms are "genetic" because new states are typically obtained from a pool of older ones by "mutations," "cross-breeding," and "natural selection." These states can represent rules which control a complex dynamical system, producing a so-called evolutionary strategy. As a spectacular example of such, NNP5 participants were able to witness the improvement of a jet-fighter automatic pilot in virtual dogfights.

Finally, another aspect of Artificial Neural Networks which was thoroughly discussed was their properties and use as adaptive, universal approximators. They often compare favorably with other data-fitting methods: if one accepts the high cost of training them, they appear rather accurate and robust. Several applications of Artificial Neural Networks were described in which a lot of information needed to be squeezed out of scanty and/or noisy data, for example, polluting properties of various chemicals from laboratory experiments on mice, heart transplant success versus weight difference between donor and receiver, and organic chemistry reaction rate prediction. One application of particular interest to all was described by Phil Hotchkiss, of Neural Dynamics, Minneapolis. He described, with exceptional openness, how his software DYNASTY uses neural networks, in association with principal-component analysis and genetic algorithms, to predict stock market values eight days in advance from a carefully selected, small set of indicators. However, he pointed out that such prediction is the easy part of what he does. Assigning a confidence level to that prediction and making investment decisions which maximize profit, constitute the difficult part, which he did not describe in further detail!

As the field of Neural Networks matures, it branches off into several subfields which have more overlap with other disciplines (biology or data-fitting) than with each other. Many NNP5 participants, specialists in one subfield, were exposed to subjects they knew very little about. But the high quality of the talks, the general curiosity, and good food ensured the success of the workshop again this year.

The reader who would like more information on the content of NNP5 presentations may consult the following research report:

UMSI 95/187 September 1995
Proceedings of the Neural
Networks for Physicists V
Conference
Theoretical Physics Institute (editor)

This document contains a copy of the transparencies used by all NNP5 speakers, and is available for consultation by sending a request to: