LISTEN · Hands-free Voice-enabled Interface to Web Applications for Smart Home Environments

Hands-free Voice-enabled Interface to Web Applications for Smart Home Environments

In this project, our goal has been to design a complete (software and hardware solution) voice-enabled interface specifically designed for Internet applications for the smart home environment, for controlling specific automations of the smart home, but also for providing access to the web for specific tasks, such as access to emails, social networking platforms, calendar, etc. A clear distinction compared to currently popular systems for voice-based Internet access is the Wireless Acoustic Sensor Network (WASN) approach, that supports a truly seamless operation of the voice interface. The proposed design was based on a distributed operation of several acoustic sensors, able to localise the speakers and enhance the speech capture for their particular locations, and enhance and transmit the speech signal to the speech transcription engine. LISTEN’s sensor network features low-cost hardware and stand-alone real-time operation due to a custom MEMS (micro-electro-mechanical systems) microphones design. The acoustic front-end was jointly developed and optimised with the automatic speech recognition (ASR) system. A central motivation behind forming the LISTEN consortium and objectives was to form a much-needed interdisciplinary team and research plan, so as to achieve substantial progress beyond state-of-the-art in hands-free ASR systems. The objectives of the LISTEN project have been: • Objective 1: To develop a large-vocabulary speech recognition system for the smart home, for multiple domains (e.g., command/control of the smart home functionalities, web access, web search, email / message dictation, calendar and to-do lists, social networks, etc.), and for multiple languages. Focus has been on English, Greek, Italian, and German to demonstrate the system’s easy adaptation to further languages. • Objective 2: To develop a front-end for robust speech capture in a distributed fashion, performing in real-time localisation of multiple speakers and accordingly enhancing the speech capture for these specific locations. The front-end design includes the hardware design of a low-cost distributed acoustic sensor network, where each sensor is equipped with an on-board processor for the signal localisation and enhancement to be performed in a collaborative manner. • Objective 3: To create a prototype and evaluate it in practical situations, including in the Ambient Intelligence Facility of the coordinator (FORTH). To widely disseminate the project results and maximise the potential societal and commercial impact of our activities and the developed platform.

Data: CORDIS, © European Union