MARIE · Multimodal Activity Recognition for Interactive Environments
FP7 — People (Marie Curie Actions)
- Duration
- 2008-07-01 → 2010-06-30
- EU contribution
- €168,257
- Participants
- 1
- Scheme
- MC-IEF
Lines connect the coordinator with its partners.
Results in brief
Multimodal activity recognition for interactive environments
Your eyes are the closest thing you have to a window on your mind. While we cannot easily discern what you are thinking, we can get some idea about what you are looking at. A major contribution of this work has been to extend this idea towards using your eyes as a means of finding out what you are doing: of recognising your activity. We introduced the idea that using eye movements alone we can collect sufficient information to automatically recognise certain activities. We developed a complete recognition methodology: from devising features based on an analysis on the fundamentals of eye movement; selecting those features most useful for each activity; to the classification of movement sequences with varying complexity. We published two complete multi-subject datasets of eye movement for future use by other researchers. This work, in partnership with ETH Zurich, has directly spawned a new EPSRC-funded project which will continue where MARIE left off. With already three major publications in the first two years, another two in progress, and a growing number of early-stage researchers working on the idea (there are now four researchers directly working on the topic), eye-based activity recognition is an important topic with a huge potential for future success. Evaluation is an essential component of any scientific research field. Activity recognition is no different. For many years the field continued with severe drawbacks in the way researchers measured and evaluated their work. Some of these drawbacks were identified in the researcher's earlier PhD work. Through the MARIE project we introduced a complete system of evaluation - results comparison, error scoring, metric calculation - and demonstrated these on widely available activity recognition datasets. We demonstrated the utility of our approach in comparison to the standard methods. This work has recently been accepted for publication in a forthcoming ACM journal, and has already drawn a number of interested users from the activity recognition research community. In a parallel work through collaboration with over 30 of the leading researchers in activity recognition, we drew together a consensus on how activity recognition can best proceed in future. A publication summarising this work is now under way. This has a strong the potential to influence the entire research field in future.
Data: CORDIS, © European Union
Project objective
With the increasingly ubiquitous use of computing devices in our environment, there is a clear need for new methods of human computer interaction. In order to support day-to-day human activity seamlessly, it is essential that computing systems have a sense of the user’s situation or context. As a consequence, activity recognition has become a central research challenge toward ambient intelligence. The proposed work aims to make three distinct contributions to the study of activity recognition: First, this project will introduce the use of eye movement patterns as a novel sensing modality for wearable activity recognition. Almost everything most of us do is guided by the use of our vision system., and Consequently, by studying what our eyes are doing, clues can be gathered as to what it is that we are doing, or intend to do. Beyond established gaze tracking, this work will look at the general patterns our eyes make during certain tasks and in certain situations, as context for interaction. Secondly, we aim to study robust activity recognition in a realistic, everyday setting. In a novel approach, we plan to exploit the synergy of information from wearable sensors - on user actions - with that of ambient sensors – on the environment being manipulated - and demonstrate this through recognition of a number of everyday activities. Thirdly, we will investigate how activity recognition can be used to infer user attention. Eye gaze is a good indicator of this (and has been used in the past), but this is not always a convenient modality to use. In this work we plan to assess the use of activity, in particular locomotion, recognition as an indicator of user attention. A final aspect of the proposal will be in the methodology applied to implementing and evaluating the various recognition problems encountered in this work – advancing the fellow's fundamental work on performance evaluation for activity recognition.
Original text from CORDIS.
Participants
- UNIVERSITY OF LANCASTER · LANCASTERCoordinatorUnited Kingdom
Links
Data: CORDIS, © European Union