GETUI · Gestures in Tangible User Interfaces

Gestures in Tangible User Interfaces

Coordinated by the Luxembourg Institute of Science and Technology (LIST) and funded by the EU programme Horizon2020, the project GETUI applies user studies to investigate the use of gestures on Tangible User Interfaces (TUIs) in the context of technology-based assessment (TBA) of collaborative and complex problem solving skills. TUIs provide tangible representations to digital information allowing users to control data by means of physical artifacts/objects. The connection between gesture studies and TUI-based collaboration and assessment is very close, as gesturing is a natural form of communication, and communication plays a crucial role in collaborative problem solving activities. Collaborative problem solving is defined as the capacity to recognise the perspective of other people in a group, participate, contribute knowledge, recognise the need for contributions, and build knowledge and understanding as a member in a collaborative setting. Most of the research in technology-based assessment of collaborative problem solving skills has dealt with improving the assessment of traditional skills. However, the focus should shift to so-called “21st Century skills”: complex problem solving (CPS), creativity, critical thinking, learning to learn, and decision-making. ICT can provide both the solution for assessment of these skills, as well as the problem, as there is a lack of scientific and practical knowledge about, on one hand, adapting existing assessment models and, on the other hand, creating appropriate, authentic problem solving situations. The main objective of GETUI is to explore the gestural performance of users while interacting on a TUI with physical objects in a collaborative problem solving task and investigate its impact on 21st century skills. The physical objects are round objects, usually made of cardboard or laser-printed. Our TUI is a product from Multitaction and is realized as a tangible tabletop (75x120 cm), provides visual feedback in real-time, and displays the effects with pictures and animations. Empirical user studies with 60 participants from public schools in Luxembourg assess collaborative complex problem solving and reasoning skills and result in a multimodal corpus of speech and gesture. This corpus will be annotated and thus, a gesture taxonomy will be defined. Gestures are categorized into physical or free hand gestures (mainly pointing or iconic) and manipulative (rotating, tracing the objects on the TUI). The former concern human-human interaction, while the latter human-computer interaction. Last but not least, the outcomes of the study will provide input on which Interaction Design aspects of TUIs are effective for measuring collaborative problem solving and contribute to the development of assessment models and methods.

Текст от CORDIS, на английски · Данни: CORDIS, © Европейски съюз