HomeResearchNeural Mechanisms Underlying Visually Based Cognition

Our Scientists

Neural Mechanisms Underlying Visually Based Cognition

Research Summary

William Newsome's research focuses on the neural mechanisms underlying visually based decision making and related issues in cognitive neuroscience. He seeks to understand how higher mammals acquire sensory information about the world, how that information is processed within the brain, and how behavioral responses to that information are organized. New experimental and conceptual tools are creating a paradigm shift in the field. The analysis of single-neuron activity remains critical, but single-neuron activity is most comprehensible in the context of collective neural states that are revealed through dynamical systems analysis.

My lab is interested in fundamental questions concerning the neural basis of cognition: How is information encoded in the streams of action potentials emitted by single neurons? How does the activity of large populations of neurons represent the external world and internal states of the organism? How do the high-level behavioral goals of an organism influence the low-level processing of sensory information? What are the neural mechanisms underlying the formation of decisions, the key cognitive processes that link perception to action?

As a field, we are in the early stages of a paradigm shift in how such questions are addressed. For decades we have faced two fundamental barriers to progress. First, although it is a truism that cognitive processes must depend upon the coordinated activity of large populations of neurons, the traditional neurophysiological approach is to record from one cell at a time during repeated behavioral sessions, hoping to stitch together post hoc a picture of population activity. This simple approach has been more successful than we had any right to expect, but it is fundamentally incapable of capturing complex interactions among large populations of cells in real time. Second, single neurons are noisy signaling devices. Thus, neurophysiologists typically average electrophysiological data across multiple trials, time locked to some measurable external event (onset of a stimulus or execution of a movement) to reveal the "signals" carried by a particular neuron. Most cognitive states, however, are not precisely time locked to external events, nor are they repeated with sufficient frequency and reliability to be readily detected by averaging techniques. To get to the next level, therefore, we must devise new experimental and analytic approaches that allow detection and dynamic tracking of neural population states during spontaneous, internally generated cognitive processes.

Exciting new developments, both technical and conceptual, are now enabling us to make novel advances in detecting and analyzing higher organizational states of the nervous system. We and other labs are using multi-electrode arrays to obtain simultaneous recordings from hundreds of neurons in behaving animals, providing the essential rich substrate for dynamical analysis of neural populations. Critically, we are at the same time devising new data analysis techniques whose main thrust is to discover low-dimensional axes of variation in neural population activity that are relevant to the ongoing behavior of the animal. Finally, we are analyzing the resulting low-dimensional data in the conceptual framework of dynamical systems.

The central idea underlying dynamical systems analysis is that the behavior of complex systems is best described and understood in terms of high-level states and transitions between states (dynamics) whose relationship to the behavior of individual components may be very indirect. Critically, the high-level states of the system are not directly observable, but are latent and must therefore be inferred from the coordinated activity of the components. For example, to reverse engineer a digital computer, one would of necessity measure dynamic patterns of signals across many transistors. Ultimate success, however, would involve inference of the succinct, unobserved rules—the software—that govern the behavior of the system. Physical scientists and engineers have long employed the tools of dynamical systems analysis to study the behavior of complex physical systems, and a few pioneers have applied this approach to the nervous system as well (Gilles Laurent [Max Planck Institute for Brain Research] and Krishna Shenoy [Stanford University] have been inspirational to me personally). But systematic analysis has not been possible without new experimental techniques for population recording and new analytic techniques for effective dimensionality reduction.

As a first step toward dynamical systems analysis of cognition, we have tackled the problem of contextually sensitive decision making. If I am in my house and the phone rings, I answer it; if the phone rings in your house, however, I do not answer it. The same stimulus enters the nervous system, but different decisions and motor outputs ensue. The key variable that "gates" the flow of information from input to output is behavioral context.

To investigate the neural mechanisms underlying context sensitivity and gating of information flow within the cortex, we devised a context-sensitive decision-making task that is performed by animals in the laboratory. Frontal cortex recordings during this task reveal a bewildering mixture of signals at the single-neuron level: signals related to the animal's choice, the behavioral context, and different sensory inputs are expressed in nearly random proportions from neuron to neuron. Remarkably, however, orderly representation emerges at the level of the neural population when analyzed within a dynamical systems framework. Furthermore, a new computational model reproduces key aspects of the neural data and suggests a novel dynamical mechanism for gating and integration of information within the cortex. We are formulating predictions of the model that can be tested in future experiments.

This type of research program is a new and promising way forward for systems and cognitive neuroscience. Using dynamical systems analysis, we can detect high-level states and system variables that govern the behavior of the entire system, thus rendering the activity of individual components of the system comprehensible. As a traditional single-neuron physiologist, I find myself in a curious but most interesting situation. The physiological responses of single neurons remain critical—single-neuron activity is the necessary substrate on which dynamical systems analysis is based. Yet the meaning of single-neuron activity is revealed most beautifully within the collective context of the population and the high-level rules that drive population activity. Though it is too early to know for sure, I suspect that this general approach to understanding the neural mechanisms of cognition (pioneered by others) will be extraordinarily productive in the future—sufficiently so to comprise a paradigm shift for the field.

Supported by the Howard Hughes Medical Institute.

As of April 28, 2016

Scientist Profile

Stanford University