The cerebral cortex has no direct experience of the visual world. Rather, our perceptual experience is constructed from streams of action potentials in the 3 million optic nerve fibers that project from the two retinas to the central nervous system. After a relay synapse in the visual portion of the thalamus, visual information reaches a hierarchically organized network of more than 30 visual areas within the occipital, temporal, and parietal lobes of the cerebral cortex. Ultimately, neural processing within these cortical areas results in visual perception and multiple forms of visually based cognition such as attention, short-term memory, and decision making.
We are interested in several questions: How is information about the external world encoded in streams of action potentials? What is the functional significance of neurons in the visual cortex that respond selectively to stimulus orientation, direction of motion, stereo disparity, or color? How is the information carried by cortical neurons interpreted, or "read out," so as to inform behavioral decisions? How are decisions influenced by internal brain states related, for example, to the animal's anticipation of rewards?
To address such issues, we conduct simultaneous behavioral and physiological experiments in rhesus monkeys trained to perform carefully selected visual discrimination tasks. By recording the activity of cortical neurons while monkeys perform such tasks, we gain insights into the relationship between neural activity and the monkey's perceptual judgments about what is seen. Further insight is obtained by manipulating the difficulty of the task and correlating changes in the animal's performance with changes in the activity of cortical neurons. Such observations lead to hypotheses concerning the role of specific cortical circuits in the monkey's behavior, which are then tested by using electrical microstimulation or pharmacological inactivation techniques to modify neural activity in those circuits. Computer modeling facilitates the development of rigorous, testable hypotheses concerning the relationship of neural activity to behavior.
A major focus of our research concerns the activity of directionally selective neurons in the visual cortex and their relationship to behavioral performance on a direction discrimination task. Directionally selective neurons respond optimally to visual stimuli moving in a particular direction—the "preferred" direction—and respond not at all to motion in the opposite—"null"—direction. In the middle temporal visual area (MT) of extrastriate cortex, directionally selective neurons are organized in an intricate system of "columns" such that the 1,000 or so neurons in a particular column share the same preferred direction while neighboring columns prefer other directions. A complete set of columns encodes all possible directions of motion, and this columnar machinery is repeated many times within MT so that all directions can be encoded across all locations in visual space. This elegant neural architecture seems specialized for analyzing motion information, and our working hypothesis is that neural activity in MT is a major source of signals underlying the ability to perceive and judge motion direction.
We tested this hypothesis directly (in research supported by the National Eye Institute) by using electrical microstimulation to excite individual columns of MT neurons artificially while the monkey performed a direction discrimination task. For example, we passed trains of small stimulating pulses into an "up" column while the monkey performed an "up versus down" discrimination. To our surprise, stimulation of single MT columns affected performance substantially, biasing the monkey to choose the direction of motion signaled by the stimulated column. These effects occurred even when the downward motion in the visual stimulus itself was sufficiently strong to elicit 90 percent downward judgments under control (nonstimulated) conditions. These results demonstrate conclusively that specific circuits of directionally selective neurons are tightly linked to a specific class of perceptual judgments. An important corollary for systems neuroscience is that physiological properties measured at the single-neuron level indeed provide important clues about the functional role of those neurons in visual perception.
A second research focus in our laboratory is to use perceptual discrimination tasks to explore a fundamental problem in cognitive neuroscience: the neural basis of a simple "decision process." Decision processes, or decision "rules," are ubiquitous in models of sensory and cognitive behavior: they are the critical "cognitive element" that links perception to action. We now know a great deal about the neural circuitry that provides sensory information underlying performance on our direction discrimination task (i.e., area MT), and we also know a great deal about the neural circuitry underlying the animal's motor response (an eye movement), which reveals the perceptual decision. Only recently, however, have we begun to acquire insight into the actual neural circuits and computations that are the decision process itself.
By recording electrical signals from brain structures that are intermediate between strictly sensory and motor structures, we have identified at least two brain areas—the lateral intraparietal region of the parietal lobe (LIP) and the superior colliculus—that appear to participate in the decision process. Both structures contain high-level neurons whose discharge predicts the animal's decisions almost perfectly, both for correct judgments and for errors. Furthermore, these neurons begin their predictive discharge at a very early point in the behavioral trial, as though they are integrating (in the mathematical sense) the visual motion signals from MT to form a signal appropriate for guiding the operant eye movement. We are testing this hypothesis more rigorously. If successful, mechanistic studies of this nature will be a significant new development in the attempt to bring cognitive operations under physiological scrutiny.
In the study of decision making, psychophysicists and sensory physiologists (such as myself) traditionally emphasize the effects of sensory stimuli on the outcome of the decision process. Psychologists and economists, however, have long known that decision making is influenced not only by the sensory stimulus but also by an organism's prior experience or beliefs concerning the "value" of the alternative choices, expressed in terms of likely rewards (or aversive consequences) for any particular behavioral choice. Because we have discovered brain structures that appear to be involved in decision making (described above), we are now in a position to explore the influence of reward anticipation, or value calculation, on the putative decision-making circuitry.
In an exciting new research direction, we have designed a decision-making task in which monkeys are encouraged to calculate the relative probability of gaining a reward for two possible choices. We have found that these valuations, which resemble valuations that animals make during foraging behavior in the wild, exert profound effects on decision making in our behavioral task, and neural correlates of the animal's subjective valuation exist in area LIP of the parietal lobe—one of the areas implicated in decision making in our earlier studies. This finding opens the door to an extensive series of studies that will explore how value is calculated within the brain and becomes associated with particular choices or motor acts. These studies are highly relevant to understanding normal reward mechanisms and choice behavior, as well as the pathologies of choice and reward that underlie addictive behavior.
The core issues we are pursuing are among the most fundamental in neuroscience: How is information processed within the brain, how does this processing result in organized, purposeful behavior, and how do patterns of past rewards exert their influence upon ongoing behavior? The visual system currently provides the best platform for investigating such issues, but the intellectual thrust of the research extends far beyond visual science, engaging issues that are central to virtually all mental function. If these principles can be elucidated in the visual system, rapid application to other systems will almost certainly follow. The combination of behavioral, electrophysiological, and computational approaches provides a realistic basis for investigation of elementary cognitive functions such as perception, decision making, motivation, and reward.
