Joseph B. Rayman, Ph.D.
postdoctoral fellow
Columbia University

Tony, AK, CA

I am watching the fascinating HHMI 2008 Holiday Lecture " Making Your Mind". Our experiences have components of our various senses. If each sensory component is a neural circuit, would there be some form of indexing to link the various circuits? Could you recommend an introductory book/article on the subject?

Joseph B. Rayman
HHMI lab associate,
postdoctoral fellow,
Columbia University
The student asks a fascinating question. Indeed, one of the major challenges of neuroscience is to understand the physiological basis of multisensory integration. In other words, how does the brain collect information about our environment using the five basic senses and then combine this information in a way that will lead to an appropriate and efficient behavioral response? While it is clear that the brain is capable of synthesizing information from distinct sensory modalities such as vision and hearing, the short answer is that we really do not have a good understanding of how this process works. In fact, because of the immense complexity of this problem and the associated technical limitations, scientists generally focus on one sensory modality at a time. Nevertheless, progress is being made at the level of multisensory integration.

We know the brain is organized in a manner that allows hierarchical processing of sensory information. For example, when we look at a physical object, different pathways within the part of the brain known as the primary visual cortex process basic information about its shape and color. This information is then integrated in the visual unimodal association cortex, one of several higher-order regions of the brain that receive and process sensory inputs from a single sensory modality. There is also a somatosensory unimodal association cortex that is specific for our sense of touch, while the auditory unimodal association cortex is specific for hearing, and so forth. In turn, neurons in the various unimodal association cortices project to multimodal association cortices, which serve to integrate sensory information from more than one sensory modality. In other words, sensory information is processed at increasingly sophisticated levels by different regions of the brain that are highly interconnected with one another.

There is also clear evidence that multisensory integration occurs at the level of individual cells. For example, there are specific types of neurons that respond to stimuli from more than one sense. The response of these multisensory neurons, which are found throughout the brain, will vary depending on the intensity and timing of the sensory stimuli being presented. Furthermore, the response of a multisensory neuron to two different sensory modalities will differ from the response to either alone. These features are hallmarks of sensory integration, a synergistic process that allows much more information to be conveyed by the combination of multiple sensory modalities than by individual sensory modes alone.

Given the complex nature of this subject, I recommend starting with a good introductory neuroscience textbook such as Principles of Neural Science by Eric Kandel and colleagues. A particularly ambitious student might also wish search the PubMed database for review articles on the subject of multisensory integration.

08/13/11 13:30