CROWDMERSE · CAPTURING AND REPRODUCTION OF CROWDED ACOUSTIC ENVIRONMENTS FOR IMMERSIVE APPLICATIONS

CAPTURING AND REPRODUCTION OF CROWDED ACOUSTIC ENVIRONMENTS FOR IMMERSIVE APPLICATIONS

"The central objective of CROWDMERSE is to enable the real-time capture, processing, and reproduction of an immersive audio experience to the user of any event of interest, focusing on ""crowded"" acoustic events. Acoustic scenes with large audiences, such as sports events or music concerts, present significant challenges for immersive reproduction which have not been so far adequately considered in the signal processing community. These challenges include the need to spatially capture and reproduce numerous sound sources, such as the applause and yells of the spectators, in addition to the “main” sounds of the event (ball kicks, musical instruments, etc.) In order to spatially sample large outdoors venues that are of central interest in this project, the acoustic sensors should be low-cost, small-sized, and easy to deploy supporting wireless operation, as in Wireless Acoustic Sensor Networks (WASNs). There are several applications of the immersive reproduction of crowded acoustic events. An application of considerable commercial interest is sporting events: by placing a microphone sensor in a football stadium within the fans’ bleachers, it would be possible to capture and reproduce the experience of ""being there"" to any TV or tablet user sitting in his/her living room. Going one step further, imagine a stadium populated by dozens of microphone sensors, all feeding audio signals to a central server that allows a producer, or even the end-user, to select the desired audio feed heard. Users could be offered a visual representation of the stadium, and interactively navigate the sonic landscape. Another application space of similar nature is coverage of large music concerts. Our final objective is to derive the technology that enables the real-time delivery of such crowded acoustic events, so that the designed platform can be readily integrated with today’s High-Definition TV (HDTV) programs. CROWDMERSE focuses on spatial capturing of the sound information using multiple closely placed microphones forming a microphone array (a node of the deployed WASN) in order to offer immersive reproduction of the soundscape to the end-user in real-time. The audio content is rendered via a home entertainment system equipped with multiple loudspeakers, accompanying the audio-visual content of a typical HDTV broadcast."

Data: CORDIS, © European Union