ProsoDeep · Deep understanding and modelling of the hierarchical structure of Prosody

Deep understanding and modelling of the hierarchical structure of Prosody

Speech prosody is a multidimensional phenomenon comprising intonation, energy, and rhythm. It is the carrier of both linguistic information, e.g. sentence structure, focus and contrast, lexical stress; as well as paralinguistic information, e.g. gender, age, personality, and emotions. In recent years, prosodic research has considerably enlarged the spectrum of its properties and functions. In contrast, prosodic models able to map signals to functions or vice-versa are rare: comprehensive models of rhythm and intonation have difficulties coping with this expanding dimensionality, and machine learning techniques still have difficulties with offering structuring principles. This issue is of increasing importance to society as speech enabled applications see widespread deployment in our everyday lives. Specifically, we have seen a proliferation of virtual assistants and virtual call operators whose primary mode of communication with the user is speech. Even though these systems handle well the linguistic content of the speech signal, i.e. the spoken words, they struggle understanding information embedded in the prosody, i.e. the meaning behind these words. Thus, a model is needed that will enable these systems to disentangle and decode the various information embedded in speech prosody. The prime objective of the ProsoDeep project was to develop a state-of-the-art Deep Prosody Model that would provide a deeper understanding of the hierarchical encoding of information through the language of prosody. A secondary objective was the recording of a prosodically rich Database that can be used to analyse the interaction of multiple linguistic functions in the way they are communicated through prosody. The ProsoDeep project has achieved both of these objectives.

Data: CORDIS, © European Union