What Are Neural Networks, and What Can They Contribute to Psychology?
Daniel S. Levine, University of Texas at Arlington
The "Chunnel" across the English Channel between Britain and France was built from both ends, with laser technology utilized to make them meet in the middle. Similarly, if one wants to understand the biological basis of human behavior, one needs to go back and forth between the biology of the brain and behavioral functions, and try to get them to "meet in the middle."
If the biology of the brain is analogous to "Britain," and the psychology of behavior, emotion, and cognition to "France," then the growing interdisciplinary field of neural networks is one of the "tunnels" between them. Neural networks are mathematical and computer models that are composed of simulated brain regions and connections between them. At the same time, the networks are designed with the goal of achieving with computer simulations some results that can be interpreted as analogous to some set of behavioral or neural or psychological data. One example of such data is Pavlovian conditioning (the type of learning that occurs when dogs are trained to salivate at the sound of the bell by repeated presentation of the bell followed by meat powder). Another is a card sorting test used by clinical neuropsychologists to detect frontal lobe damage.
We who construct these models sometimes work "top down" from observed human or animal behavior. At other times we work "bottom up" from the physiology of neurons (nerve cells) comprising the brain. Like tunnel builders, we start at one or another end (either with the psychological results or with what we know about physiology and anatomy) and then refine our model to make it fit better with the other end. In the process we go back and forth until we have reached a certain stage of understanding.
What do we mean, though, by simulating the learning of, say, a bell-to-food association using mathematical equations solved on a computer? It may sound like comparing two things that are entirely different. But mathematics has been used for centuries as a descriptive language for everything in nature. Most people are by now used to using mathematics to describe physical variables ó numbers of atoms, positions of objects, electrical charges, and so forth. By contract, psychological variables, such as strength of a hunger drive or memory of a bell, donít seem terribly "mathematical." But they are part of nature, so we expect that scientists will eventually find precisely describable biological effects that approximate these drives or memories, even if we can never fit them exactly. In the meantime, we can use our networks and equations not as exact fits to data, but as metaphors for psychological effects that are partly measurable.
So what exactly is a "neural network"? The work of David Rumelhart and James McClelland,1 and its clinical application by Jonathan Cohen2, have been so widely read by psychologists and clinicians that the term is often restricted to mean Rumelhart and McClellandís specific three-layer structure. But networks of that type are only a fraction of the literature in the field, and not dominant. There is no universally recognized definition of the term. The closest is probably the one developed in 1988 by a team of neural network experts from a study commissioned by the United States Department of Defense:3
a neural network is a system composed of many simple processing elements operating in parallel whose function is determined by network structure, connection strengths, and the processing performed at computing elements or nodes. ... Neural network architectures are inspired by the architecture of biological nervous systems, which use many simple processing elements operating in parallel...
What is meant by the nodes or elements in this definition? In the work of Warren McCulloch, one of the fieldís pioneers in the 1940s, nodes were conceived to be analogous to single neurons (brain cells). But as the field developed, scientists more often conceptualized the nodes as large groups of neurons or as regions of the brain. This change occurred for several reasons. First, the number of neurons in the human brain is extremely large, of the order of a trillion, so a neuron-by-neuron "wiring diagram" would be impractical. Second, experiments from neurophysiology laboratories have suggested that the electrical patterns of single neurons and the biochemistry of the connections (called synapses) between neurons arenít very regular in their organization. But if some of the irregularities at the levels of single neurons and synapses are averaged out across large groups or brain regions, regular connection patterns emerge that are important for behavior and for mental functions.
Sometimes, a node in a model neural network represents not a known brain area but the brain's encoding of a particular concept ó for example, "the letter A," "the hunger drive," "the rule that classifies cards by color." Some biological purists object to that, since using current knowledge they canít localize such a node to electrical patterns in a particular brain region. But this is just the "tunnel" being built from the psychological side of the problem. If you want to construct a computer model of a complex behavior, such as classifying cards, you first need to break it down into simpler behaviors, such as perceiving features like color or shape of the designs on the cards. So some of these neural network nodes may represent the "subfunctions" necessary to understand a more complex, larger mental function.
Now what does a typical piece of "tunnel" built from the neurobiological side look like? The process by which neurons transmit signals to other neurons is described in many textbooks,4 and only a sketchy description is given here. Each neuron consists of three major parts. These are the cell body, which is a central area containing the nucleus which all biological cells, including neurons, possess; the axon, which is a long then filament projecting out from the cell body; and the dendrites, which are a large number (in the thousands for each neuron) of smaller branches going into the cell body. The typical neuron receives electrical signals from other neurons at the dendrites, then processes these signals at the cell body. If their combined strength is large enough, it is translated into another signal that travels down the axon to one of the synapses it makes with other neurons. At the synapse, the mechanism changes and processes involving chemical transmitters take over.
But what is the electrical signal? It consists of a temporary change in the voltage across a membrane which covers the cell, all along the dendrites, cell body, and axon. This is done by means of movement across the membrane of some electrically charged atoms or ions ó sodium, potassium, and chloride ions. By processes that arenít yet fully understood, release of a chemical transmitter substance from synapses leading to a neuron can either raise or lower the probability of this exchange of electrically charged atoms across the membrane. The changed voltage, by other processes that are also not completely understood, in turn causes the release of a certain amount of chemical transmitter from some synapses going from that neuron to other neurons.
In most model neural networks that perform behaviors, the nodes or units are interpreted as groups of a large number of neurons, maybe several thousand. At this stage of development of the models, there isnít enough precise knowledge to assign each of these nodes to a specific, measurable brain area (even though, at times, a rough general location in the brain and/or neuron type is indicated). For this reason, the details of electrical signals and electrically charged atoms donít appear in the equations for typical computer networks. Instead, the electrical signals are averaged out into variables called the activity of each node. The most common biological interpretation of "activity" is of the current average frequency of electrical signals for a group of neurons in some area over some window of time. Some readers are uncomfortable with using an abstract notion such as activity which canít be fully defined yet in real-world terms. But scientists in many fields have often used abstract constructs like this to gain understanding of the real physical world by studying a simplified, idealized version of the world.
Connections between neurons in the brain can either be excitatory (tending to increase the probability of an action potential) or inhibitory (tending to decrease the probability of an action potential). Likewise, a connection between nodes in a neural network is excitatory if a signal produced by activity in one node tends to increase activity in the other node. The connection is inhibitory if a signal produced by activity in one node tends to decrease activity in the other node. Both excitation and inhibition perform cognitive functions in artificial as well as biological neural networks. Excitation is important for creating associations between concepts (e.g., for Pavlovís dogs, hearing the bell "excites" the thought of food). It also plays a key role in causing either an emotional drive, or a reasoned plan, to stimulate action. Inhibition is important for making decisions between alternatives, because a person or animal needs to do one thing and not do another thing. It can cause us, for example, to engage in one behavior and not engage in a competing behavior. Likewise, inhibition can make us attend to one part of a perceived stimulus but not attend to another part. Inhibition is also important for controlling the intensity of brain activity, that is, for keeping excitatory signals from overwhelming the network with epileptic-like discharges.
There is a great deal of variety in the mathematical rules for neural networks. Most of them involve changes over time in the activities of interacting nodes and, often, in the strengths of connections (sometimes called connection weights) between nodes. Connection weights are idealized biochemical variables, just as node activities are idealized electrical variables. They are thought to represent amounts of chemical transmitter substances or properties of certain molecules on the cell membranes that bind with the transmitters and so cause the receiving neuronís electrical properties to change. By now there is considerable neurophysiological evidence that strengths of many synapses between pairs of neurons change when both neurons are repeatedly electrically active at the same time. Psychologists interested in learning (including Sigmund Freud) suggested the idea of changes at synapses long before neurophysiologists observed it. This kind of flexibility of connection strengths seems to be required not only for learning but for the brainís overall function of mediating between the rest of the body and the outside environment.
For the purposes of this article, the technical details of neural networks are unnecessary. What is important is that they represent the dynamics of interactions between nodes that are identified either with regions of the brain or types of neurons within given brain regions; with representations of particular mental objects such as percepts, actions, memories, emotions, plans, or concepts; or with both simultaneously.
Like all mathematical models of real-world events, neural network models of the brain can be considered caricatures of what they model. That is, a model doesnít represent everything about the system itís reproducing, only those features needed to understand particular system behaviors. But just as caricatures in cartoons can bring alive some tendencies of the characters they portray, caricatures in neural network models can yield some valuable intuitions for what types of brain structures are likely to produce certain behaviors. The two "take-home messages" about neural networks should be:
(1) Neural networks are a metaphor for the fact that all of mental life is dynamically interrelated. Perceptions, categorizations, beliefs, emotions, plans, and actions canít be separated from each other, but instead form what some religions call "an interconnected web." Results from experimental psychology show, for example, that cognitive ambiguity can lead to emotional discomfort, and that emotional biases can influence how categories are chosen.
(2) Some specific neural network architectures can function as useful metaphors for specific human attitude tendencies. In a later article, for example, I will introduce a neural network that serves as a metaphor for the human tendency to get stuck in entrenched, unrewarding behaviors. Another network serves as a metaphor for jumping between polar emotional opposites (such as love and hate). But there is also a neural network metaphor for the creative process that encourages self-actualization!
Current Uses of Neural Networks in Neuroscience and Psychology
Neural networks are designed to simulate various mental functions, wherever those functions appear. As a result, in addition to their biological and psychological applications, neural networks have found wide usage since the mid-1980s in industrial and engineering applications that call for some form of "intelligent" functioning. These include, for example, visual pattern processing, speech signal processing, robotics, manufacturing analysis, financial forecasting, and many other applications.5
The applications of neural networks to understanding actual human mental processes have lagged behind the industrial applications. But the late 1990s have seen rapid growth of biological and behavioral models. This growth is happening in part because more neurobiologists and psychologists than ever before have access to high-speed personal computers. It is also spurred by many advances in experimental neuroscience, such as positron emission tomography (PET) and magnetic resonance imaging (MRI) scanning, which are enabling a more complete (though not yet perfect) account of the actual metabolism of brain tissue during the performance of cognitive tasks. The assumption is that those areas that are most active, in terms of metabolism and blood flow, are the parts of the brain being most used in the current task. All these advances are making it seem possible for brain science to be given a solid theoretical framework. In fact, many psychology and neuroscience laboratories are hiring or collaborating with researchers whose expertise is primarily theoretical rather than, or as well as, experimental. These researchers often combine knowledge of relevant neuroscientific or psychological literature with training in computer science, engineering, mathematics, or physics.
Four books have appeared since 1996 on neural network models relating to mental illness 6. In 1995, the first two major conferences took place ó one in London and one in College Park, Maryland ó on neural network models of mental and cognitive disorders, such as schizophrenia, epilepsy, Alzheimerís disease, depression, stroke, and aphasia. These conferences drew attendance from many practicing psychiatrists and neurologists as well as academic scientists. There have also been symposia on neural networks at conferences such as annual meetings of the American Psychological Society and American Psychological Association. There are also at least four ongoing annual international conferences devoted to neural networks themselves; some of these meetings include both sessions devoted to neuroscience and other sessions devoted to industrial applications. In addition, one of these conferences had between 1994 and 1996 a session devoted to studying and modeling consciousness. In the United States, western Europe, and Japan, government support of neural network research has begun to be integrated with support for neuroscience.
The best known early successes of neural network modeling were in the field of perceiving, classifying, and categorizing sensory patterns. This gave insights into how the brainís perceptual systems, particularly its visual system, work. These pattern classification networks also have a wide variety of applications to the performance of "intelligent" functions by computers. Among these are medical diagnosis, wherein the visual display of a particular organ for someone with a disease is different from the same display without the disease. It has also been used to classify radar signals as coming from different emitters or handprinted numerals in zip codes as being specific digits. Neural network modeling of the psychological process of conditioning has also been an active field of research since the 1970s.
More recently, this same methodology has come closer to understanding the most complex human cognitive processes and their characteristic breakdowns with brain damage or mental illness. There have been models of the effects of frontal lobe damage on the ordered planning of behaviors. There have also been preliminary models, for example, of Alzheimerís disease, one type of dyslexia, and a qualitative neural network theory of manic-depressive illness. Also, there has been at least one computational model of the disruption of motor behavior by Parkinson's disease.
Neural network models of complex cognitive processes are often developed by breaking these processes into simpler subprocesses for the purposes of analysis. For example, neural networks have been used to model of categorization of sensory patterns such as those in letters of the alphabet. Handwritten characters such as might appear on a postal envelope, and at times are written imprecisely, are categorized as to what letter they are closest to. In order to model that categorization process, we need models of at least two subprocesses. One of these subprocesses is learning, because the representation of Roman letters in the brain isnít hard-wired: the same neural structures are equally capable, for example, of learning Japanese, Hindi, Hebrew, or Russian letters. Another subprocess is deciding between two alternatives. For example, a sloppily written letter might look somewhere in between an "E" and an "F," so we need to be able to decide which letter is more likely, enhance our mental image of that letter, and suppress our image of the other letter.
The neural network modelers David Rumelhart and James McClelland called this type of analysis an exploration of the microstructure of cognition. Another group of modelers, headed by Stephen Grossberg, has applied such analysis to a range of cognitive and behavioral processes including categorization, conditioning, visual perception, word recognition, and speech encoding.7 The computational theories of Grossberg and his colleagues suggest that similar types of subprocesses are components in all these different things that our brains do. For example, the same principles of associative learning and perceptual decision are used both to model the process of categorization and also, in a different form, to model the role of selective attention in conditioning. Also, a book in press by the modeler John Taylor develops a series of interrelated neural network theories for many areas of the brain involved in high-level cognitive processes.8 These include the organization of planned behavior sequences ó based on a combination of rational analysis and emotional preferences ó and the way memory is involved in consciousness.
What all these neural modelers have done is develop a "tool kit" consisting of different parts, or subblocks, of neural networks, that can be used repeatedly and in different combinations. Is this, as I believe, roughly the way our brains are really constructed? That would mean that just as a few base substances account for all of our rich genetic code, a few types of characteristic neural connections repeat in many if not all parts of the nervous system, from the spinal cord and midbrain reflex centers up to the frontal lobes and other association areas of the cortex.
If these authors are on the right track, their methodology can be universal when it comes to mental processes. They hint that the same kinds of neural structures that handle relatively simple processes like visual perception and conditioning can also, in different combinations and with greater complexity, handle much more complex processes like reason, inference, and decision making. These same processes can yield plausible theories of behavioral, cognitive, and affective disorders and their treatment. Ultimately, sufficient understanding of mental processes can even lead to theories of self-actualization, ethical behavior, and how we decide right from wrong.
I am sometimes asked what a neural network, or computational, approach can add to our understanding of human psychology over and above what can be gained by just thinking intelligently about mental processes. My answer is that approaching problems via a new discipline doesnít automatically lead to a different viewpoint. Neural networks donít change our view of the brain and behavior dramatically. They merely help us tackle problems of human behavior using a systems approach. This means that each of our personalities, like any other complex system, is seen as a web of different subsystems (in this case emotion, cognition, reason, memory, perception, motor action, et cetera), all influencing each other dynamically but each somewhat autonomous. In such a web, it makes no sense to say that some parts of our personality structures are "better" or more "basic" than other parts. Our neural networks are studied through the mathematical theory of dynamical systems (also sometimes known as chaos theory), which applies to a wide range of other types of physical and social systems.
There are no gimmicks here, and the reader may ask "So whatís new?" But the dynamical systems approach turns out to have surprising implications for human relations.9 Rather than imposing a mathematical strait-jacket on the human personality, as many people fear, dynamical systems suggest that people have far more potential than they are exhibiting in one given moment and one given context. This suggest standards of intrapersonal and interpersonal behavior that are within reach for almost everyone. So for people willing to break new ground, scientific approaches will provide hope.
1. Rumelhart, David & McClelland, James L., Editors. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol 1 and 2, Cambridge, MA: MIT Press, 1986. Their essential ideas were developed earlier in Paul Werbosí 1974 Harvard dissertation, Beyond regression: New tools for prediction and analysis in the behavioral sciences.
2. See, for example, Cohen, Jonathan & Servan_Schreiber, David, Context, cortex and dopamine: A connectionist approach to behavior and biology in schizophrenia. Psychological Review 1992, 99, 45_77.
3. DARPA Neural Network Study (1988). Alexandria, VA: AFCEA International Press, p. 60.
4. Kandel, Eric & Schwartz, J. H. (Eds.). Principles of Neural Science. New York: Elsevier, 1985. Shepherd, Gordon. Neurobiology. New York: Oxford University Press, 1983.
5. Arbib, Michael (Ed.). The Handbook of Brain Theory and Neural Networks. Cambridge, MA: MIT Press, 1995. Miller, Thomas, Sutton, Richard, & Werbos, Paul (Eds.). Neural Networks for Control. Cambridge, MA: MIT Press, 1990.
6. Parks, Randolph, Levine, Daniel, & Long, Debra (Eds.). Fundamentals of Neural Network Modeling: Neuropsychology and Cognitive Neuroscience. Cambridge, MA: MIT Press, 1998. Reggia, James, Ruppin, Eytan, & Berndt, Rita (Eds.). Neural Network Modeling of Brain Disorders. Singapore: World Scientific, 1996. Stein, Daniel (Ed.). Neural Networks and Psychopathology. Cambridge, UK: Cambridge University Press, 1998. Reggia, James, Ruppin, Eytan, & Glanzman, Dennis (Eds.), Disorders of Brain, Behavior, and Cognition: The Neurocomputational Perspective. Amsterdam: Elsevier, 1999.
7. Grossberg, Stephen. Neural Networks and Natural Intelligence. Cambridge, MA: MIT Press, 1988.
8. Taylor, John. The Race for Consciousness. Cambridge, MA: MIT Press, in press.
9. Levine, Daniel, Common Sense and Common Nonsense, submitted for publication. Levine, Daniel, What can academic psychology contribute to psychotherapy? Psychline, Vol. 1 (No. 2), 18-19, 1996. Levine, Daniel, Cognitive dissonance, halo effects, and the self-esteem trap. Psychline, Vol. 2 (No. 3), 25-26, 1998.