MIMe · The multisensory dimension of memory, from single neuron to neural network. A multiscale electrophysiological approach to reveal the mechanism of face-voice association for person identity recognition

The multisensory dimension of memory, from single neuron to neural network.A multiscale electrophysiological approach to reveal the mechanism of face-voice association forperson identity recognition

Multisensory integration is a fundamental aptitude of our brain: it links the information provided by the senses. This process allows to build (e.g. in memory) a multimodal object as a whole from its different sensory features; like a face and a voice that are bound to form person identity. Multisensory integration is also crucial to take a decision: our senses provide us complementary information that we combine optimally when we are making a choice. For instance, walking in the wood at dusk, we may decide to change our path if, while hearing a faint howl, we are starting to distinguish a barely visible wolf from the rising mist. Currently, two major unknowns reside in how our brain constructs multimodal association and how our brain implements the interplay between multisensory integration, memory and decision making. A recent stream of research in neuroscience proposed that the transfer of information in brain circuitry relies on neuronal oscillations. This premise is based on the observation that ongoing oscillations reflect variations in neuronal activity. Thus, when two groups of neurons present an optimal alignment of their phases of excitability, they are more likely to exchange information. While it remains to be generalized, this framework provides a neuronal mechanism by which the brain may associate sensory inputs to create a multimodal construct. In our research project, the first aim was directed at assessing if multimodal association relies on the phase synchronization between distant neuronal. So far, perceptual decision making has been studied mainly in the context of one modality at a time and has been described as a chain of processing steps from perception to the realization of an action. First, a sensory signal is encoded in the related sensory cortex. Thereafter, this inherently noisy sensory signal is accumulated over time in associative regions to form a decision. Lastly, if a decision criterion is reached, an appropriate motor response is triggered. Thus, formally perceptual decision making can be divided into sensory encoding and decision formation stages. Given these two stages, the second aim of our project was to evaluate if multisensory integration takes place: either during sensory encoding only (i.e. before a supramodal decision formation step), or during decision formation (which would be fostered by information originated from the different senses), or both during sensory encoding and decision formation. The aims of the project were twofold. (i) To reveal the neuronal mechanism underlying multimodal association within brain networks. (ii) To answer the question whether multisensory integration is accomplished before and/or during perceptual decision making (i.e. during sensory encoding and decision formation).

Data: CORDIS, © European Union