FASTDEFORM · Real-time understanding of dexterous deformable object manipulation with bio-inspired hybrid hardware architectures
FP7 — People (Marie Curie Actions)
- Duration
- 2012-07-15 → 2014-07-14
- EU contribution
- €168,896
- Participants
- 1
- Scheme
- MC-IEF
Lines connect the coordinator with its partners.
Results in brief
Real-time understanding of dexterous deformable object manipulation with bio-inspired hybrid hardware architectures
This project aims to close the perception-action loop in dexterous manipulation scenarios that involve rigid and non-rigid objects with arbitrary physical characteristics. This requires removing a sensory/semantic communication bottleneck that currently exists between low-level vision such as dense motion and depth perception on the one hand, and abstract scene understanding and reasoning on the other. We have achieved this through the development of novel methods for building and maintaining a detailed 3D scene representation, the complexity of which is dynamically adapted based on sensory information extracted from the scene in real-time. All these methods were specifically designed to exploit the massive parallelism provided by Graphics Processing Units and realized on a real-time hybrid platform. In accordance with biological vision systems, the sensory/semantic bottleneck is removed using visual attention methods that focus communication on scene items of interest and at an abstraction level that is supported by the sensory and prior information. On a number of benchmark datasets specifically designed for this project (and made publicly available) the system has been shown to outperform state-of-the-art methods. Highly robust and real-time operation has been shown on (1) the manipulation of a large number of rigid objects with accurate tracking of occlusions and dynamics, (2) visual servoing guided by accurate joint multi-object and manipulator tracking with inaccurate calibration and camera motion, and (3) complex origami style articulated cardboard manipulation with frequent occlusions. A number of videos demonstrating the real-time system in all these scenarios are available at: www.youtube.com/user/karlpauwels This project has shown that a tight integration and real-time interaction between visual simulation and visual perception, both realized on the same hardware, can significantly enhance state-of-the-art scene understanding in terms of speed, robustness, and accuracy. The results of this project can provide robotic systems with the ability to perceive and understand complex manipulations of non-rigid objects in closed perception/action loops. This can enable such systems to operate outside the highly-constrained industrial settings they are currently restricted to, opening up a wide-variety of applications in manufacturing, medicine, elderly support and general domestic applications. For more results, publications, and contact details see www.karlpauwels.com
Data: CORDIS, © European Union
Project objective
The ability to perceive and understand dexterous object manipulation from visual information, irrespective of an object’s physical characteristics, is critical for next generation robotic systems. It enables fluent interaction outside the highly constrained environments of today’s applications and allows robots to enter domestic settings and natural environments. Many approaches have been proposed to this end, but to enable real-time performance, all of them need to severely restrict model detail, degrees of freedom of articulated objects, and/or deformation complexity. The real-time aspect is critical since the perception of deformable objects under manipulation seems to be a dynamic process, in which specific actions may be efficiently used as active exploration primitives towards model abstraction. This requires closed-loop perception-action cycles. In this project, the candidate will develop a theoretical communication framework that removes a critical obstacle in current systems: the sensory/semantic communication bottleneck. In accordance with biological vision systems, the feedback and feedforward communication between dense low level sensory and compact high level semantic representations will be shaped using visual attention methods. An internal model representation will be developed that supports varying degrees of invariance to an object’s physical characteristics, and the action state space will be reduced by considering objects together with their manipulators. The candidate will also realize the communication framework as a real-time hybrid architecture by combining conventional (CPUs) with graphics processing units (GPUs).By providing multiple relevant contributions across the spectrum of the FP7 objectives in terms of its potential to advance robotic manufacturing, medical image processing, and computing paradigms, this project will enable the candidate to maintain and enhance his position at the forefront of advances in this field.""
Original text from CORDIS.
Participants
- UNIVERSIDAD DE GRANADA · GranadaCoordinatorSpain
Links
Data: CORDIS, © European Union