Merging of the senses

When we are constantly and simultaneously bombarded with various types of sensory inputs, how do we make decisions or motor outputs to deal with the world in a meaningful manner? This is a central issue in multisensory integration. One example of such type of processing is eating, as placing food in one’s mouth provides taste, olfactory and somatosensory (texture, temperature) information ( Verhagen and Engelen, 2006 , Simon et al., 2006 ). Indeed, virtually all combinations of the senses are multimodal ( Verhagen and Engelen, 2006 ). Whatever the combination of modalities, information from widely separated areas is rapidly integrated to give a motor response. In the case of taste the response to swallow or reject is yielded. One possibility that our brains have evolved to deal with multisensory stimuli could be due to the combined response reveals more about the nature of the external event. Our grand challenge is to know how information from different modalities that originally seem to be processed as unimodal events is combined during development to produce a coherent percept. This is known as the “ binding problem” ( Singer, 2001 ) and is important to all multisensory processing.

Many issues regarding multisensory processing are found in a very recent and excellent review by Stein and Stanford ( Stein and Stranford, 2008 ), and given the space limitations, I will not repeat them but rather give my opinion as to what is missing and what experiments would lead to a better understanding.

It has been proposed that the entire neocortex is multisensory ( Ghazanfar and Schroeder, 2006 ) and, although there has been some controversy about this proposal, it would be useful to have an atlas of the multisensory specificity of all subcortical and cortical areas.

In electrophysiological studies of multisensory integration most recordings have been from single units, and confined to a single area. No doubt these recordings have lead to many advances and the formulation of rules that govern multisensory processing (e. g. superadditivity ( Stein and Stranford, 2008 ), but it would be more informative if ensembles of neurons could be recorded in different areas while animals are being tested. This could be done either with implanted arrays of electrodes or with optical imaging ( Nicolelis, 2007 ; Gradinaru, 2007 ). In this manner we will learn how different neurons in the same and different areas interact (synchronously or asynchronously) at different time scales. Such details can be use to determine which area signified the event (say congruent or non-congruent), even on a single trial ( Nicolelis, 2007 ). We will also understand how the temporal structure of the neuronal dynamics provides information as to the behavioal task. The new technique involving the reversible use of light-induced activation or inactivation ( Gradinaru, 2007 ) of neuronal ensembles in a given area will enable researchers to directly determine how the input of one modality affects the other modalitities and to determine whether multisensory information is feedforward and/or arises from top-down influences. This method may also permit researchers to begin to understand how animals determine saliency of cues to the extent as to which sensory modality yields most information. With these advances in animal research together with fMRI, EEG and evoked potential experiments in humans much progress will be made towards achieving our grand challenge – namely, to understand how the many facets of the sensory world around us becomes one coherent percept in our brain.

References

Verhagen, J. V., Engelen, L. (2006). The neurocognitive bases of human multimodal food perception: Sensory integration. Neuroscience and Biobehavioral Reviews 30: 613–650

Simon, S. A., de Araujo, I. E., Gutierrez, R., Nicolelis, M. A. (2006). The neural mechanisms of gustation: a distributed processing code. Nature Reviews Neurosci. 7, 890–901.

Stein, B. E., Stranford, T. R. (2008). Multisensory integration: current issues from the perspective of the single neuron. Nature Reviews Neuroscience 9, 255–266.

Singer, W. (2001). Consciousness and the binding problem. Ann. N. Y. Acad. Sci. 929, 123–146.

Ghazanfar, A. A. and Schroeder, C. E. (2006). Is neocortex essentially multisensory? Trends Cogn. Sci. 10, 278–285.

Nicolelis, M. A. L. (2007). Methods for Neural Ensemble Recordings. (Frontiers in Neuroscience, Series) CRC Press Inc; 2Rev Ed edition, 296 pages.

Gradinaru, V., Thompson, K. R., Zhang, F., Mogri, M., Kay, K., Schneider, M. B., Deisseroth, K. (2007). Targeting and readout strategies for fast optical neural control in vitro and in vivo. J. Neuroscience. 27, 14231–14238.