Menu
MapConnectionsTopicsBulgariaThe Night (K-TRIO)Marie Skłodowska-CurieFor studentsFor teachersMethodologySurprise me
H2020Individual fellowship2015–2019

MULTISSOUND · Multisensory-Based Spatial Sound

Horizon 2020 — Marie Skłodowska-Curie Actions

Duration
2015-11-01 → 2019-06-12
EU contribution
€191,326
Participants
1
Scheme
MSCA-IF

Lines connect the coordinator with its partners.

Results in brief

Multisensory-Based Spatial Sound

The worlds of multisensory research and multimedia technologies have originated and, for the most part, still remain largely independent. The first exists mostly within the field of experimental psychology, or psychophysics, while the second one is mostly studied and developed by engineers. This project aimed at implementing this perfect multidisciplinary marriage in the field of acoustics. A series of psychophysical experiments were carried out at Aalto School of Engineering, in the Spatial Sound Group, aiming at obtaining detailed guidelines on how to reproduce sound for the best audiovisual experience. The ultimate goal was to reach every consumer, namely in television, cinema, gaming, computer streaming and teleconferencing applications. This project observed how acoustic events spatialized inside and outside the field of view, congruent and incongruently, could affect the perception of visual events. It was found that sounds can affect negatively the perception of visual events in audiovisual reproductions. In one experiment, it was found that subjects' gaze moved more when the soundtrack was more immersive, when compared to stereo reproductions. The ability to count visual events was also affected by different sound conditions. In another experiment, it was found that sounds do not need to be presented in the same place as light events, but they should happen within the same region. Visual lateralization was maximally affected by sounds occurring in a region slightly wider than the visual event region. These findings were replicated in an immersive environment, with a movie reproduction. It was found that greater spatialization of sound sources led to slightly more dispersion of visual attention. Audiovisual demos were created to demonstrate the effect. Preliminary solutions were proposed and tested.

Data: CORDIS, © European Union

Project objective

The worlds of multisensory research and multimedia technologies have originated and, for the most part, still remain largely independent. The first exists mostly within the field of experimental psychology, or psychophysics, while the second one is mostly studied and developed by engineers. This project aims at implementing this perfect multidisciplinary marriage in the field of acoustics. A series of psychophysical experiments will be carried out at Aalto School of Engineering, in the Spatial Sound Group, aiming at obtaining detailed guidelines on how to reproduce sound for the best audiovisual experience. The results will be further applied to a new sound spatialization algorithm designed specifically to enhance human audiovisual experience, while reducing computational complexity where it is not needed. The ultimate goal is to reach every consumer, namely in television, cinema, gaming, computer streaming and teleconferencing applications. Direct actions to reach the general community, as well as the industry, will ensure maximum practical impact. I am a young researcher, in psychoacoustics with strong background in multisensory interactions, as demonstrated by numerous publications and projects. My supervisor is a highly regarded scientist and developer of spatial sound algorithms that are implemented worldwide. I am from Portugal, the host is Finnish; I am a psychologist, and the research group is composed of engineers. This extremely innovative project, complementary at so many levels, will create a perfect combination for my personal scientific development and open great career opportunities, while bringing immediate institutional benefits. The topic is framed within one of the most profitable business areas in the world, Europe being no exception. This project will bring a very significant change to that field.

Original text from CORDIS.

Participants

  • AALTO KORKEAKOULUSAATIO SR · EspooCoordinatorFinland

Links

Data: CORDIS, © European Union