H2020Индивидуална стипендия2016–2018

AVISSO · Audiovisual Speech Segmentation and Oscillations

„Хоризонт 2020“ — Действия „Мария Склодовска-Кюри“

Период: 2016-05-02 → 2018-05-01
Финансиране от ЕС: 165 599 €
Участници: 1
Схема: MSCA-IF-EF-ST

Линиите свързват координатора с партньорите.

Накратко на български

Мозъкът обработва речта, като съчетава звука с ритъма на жестовете и мимиките на говорещия. Разбирането на този процес може да помогне при рехабилитацията на хора с нарушения в комуникацията, като например след инсулт или при болест на Паркинсон.

Този кратък обзор е генериран от изкуствен интелект

Кратко обяснение, генерирано от езиков модел по текста на CORDIS. Оригиналът е по-долу.

Резултати накратко

Audiovisual Speech Segmentation and Oscillations

Speech is often multimodal and listeners need to integrate visual prosody conveyed by a speaker’s body expressions together with prosody, which appears to involve temporal coordination. Listeners may rely on the integration of temporal prosodic structure conveying information at delta frequency (1-3Hz) in both modalities to segment inputs and facilitate audiovisual speech processing. How the brain translates naturally AV rhythms to facilitate speech segmentation was the central aim of this proposal. The project investigated the oscillatory correlates of audiovisual speech relying on the temporal integration of body movements associated with suprasegmental features in speech (audiovisual prosody). Besides investigating the neural basis of AV speech processing, this project relates to a wider range of relevant areas (e.g. communication), and its possible outcomes can influence the rehabilitation of speech dysfunctions in clinical populations suffering from temporal processing deficits (e.g. Parkinson disease, stroke). The overall objectives of the project were to establish how the precise temporal alignment between visual and auditory rhythmic features conveyed by prosody, and its neural integration, contribute to successful AV speech processing. I adopted a multimodal approach to investigate the oscillatory neural correlates supporting prosodic features integration at delta (WP1), and the potential implication on the temporal-processing network (i.e. pre Supplementary Motor Area, Basal ganglia, and cerebellum) supporting prosodic-based multimodal speech integration (WP2).

Текст от CORDIS, на английски · Данни: CORDIS, © Европейски съюз

Цел на проекта

Daily social tasks (e.g. meetings, shopping) rely on human communication with speech. Auditory only studies established that continuous speech segmentation relies on the temporal integration of rhythmic acoustic features occurring at delta-theta rates in the signal, via a possible mechanism of entrainment coupled with beta activity. In terms of neural correlates, the timing network (including the basal ganglia, supplementary motor area and cerebellum) may actively contribute to the temporal integration of auditory signal during continuous speech segmentation. However, speech is often multimodal and listeners need to integrate speaker’s body movements aligned to rhythmic acoustic features. Consequently, AV speech may generate different rhythmic patterns than auditory only speech. How the brain translates naturally concomitant AV rhythms to facilitate speech segmentation clearly remains to be addressed. The goal of this project is to investigate the oscillatory patterns and neural correlates of AV rhythms resulting from the temporal integration of body movements and salient acoustic features in speech. In a first EEG experiment, I will compare the modulations of delta-theta entrainment and coupling with beta activity, depending on the relationship between visual and auditory information (congruent, incongruent and auditory only). Secondly, in an fMRI version of the experiment, I will explore the contribution of the timing network during AV speech segmentation, and its different patterns of activations across speech conditions. Finally, to test whether the brain areas revealed by fMRI have a necessary role for the successful AV speech segmentation, I will run the same fMRI experiment with Parkinson’s disease patients with dysfunctional basal ganglia. Looking at the timing network patterns of activations will also reveal if correlated body information eventually help PD to compensate basal ganglia deficit with greater contribution of alternative path (i.e. cerebellum).

Оригинален текст от CORDIS (на английски).

Участници

UNIVERSITEIT MAASTRICHT · MaastrichtКоординаторНидерландия