H2020Индивидуална стипендия2018–2020

VisualGrasping · Visually guided grasping and its effects on visual representations

„Хоризонт 2020“ — Действия „Мария Склодовска-Кюри“

Период: 2018-04-02 → 2020-07-02
Финансиране от ЕС: 159 461 €
Участници: 1
Схема: MSCA-IF-EF-ST

Линиите свързват координатора с партньорите.

Накратко на български

Зрението и начинът, по който мозъкът планира хващането на предмети, например чаша или химикал, са в центъра на анализа. Разбирането на този процес помага да се разбере как хората взаимодействат с околната среда и защо роботите често се провалят в подобни задачи.

Този кратък обзор е генериран от изкуствен интелект

Кратко обяснение, генерирано от езиков модел по текста на CORDIS. Оригиналът е по-долу.

Резултати накратко

Visually guided grasping and its effects on visual representations

In everyday life, we effortlessly grasp and pick up objects without much thought. However, the ease with which we do this belies its computational complexity (see Fig 1, reprinted with permission from [Klein, Maiello et al; 2020]). To pick something up, our brains must work out which locations on the object will lead to stable, comfortable grasps so that we can perform a desired action, such as taking a sip from a cup or writing with a pen. Most potential grasps would actually be unsuccessful, for example, requiring the thumb and forefinger to cross over, or grasping the object too far from its center, so that it slips under its own weight once we try to pick it up. Somehow, the brain has to work out which out of all possible grasps are actually going to succeed. Despite this, we rarely drop objects or find ourselves unable to complete an action because we are holding the object the wrong way. Vision helps us select, plan, and execute actions, including grasping, that allow us to interact with our environment. To do so, the visual system needs to reconstruct the 3D shape and layout of objects in our surroundings from ambiguous 2D retinal images—a mathematically under-constrained task. Understanding how we use vision to pick up and interact with objects effectively is thus one of the most important challenges in behavioral science. Even state-of-the-art robotic AIs can fail to visually identify effective grasps nearly 20% of the time [Levine et al, 2018]. The main objectives of the project were thus to understand how humans use vision to plan grasping, and to investigate the visual representations along the human dorsal visual stream that are thought to play an important role in grasp planning. We found that humans combine visual information about object 3D shape, orientation and material composition to identify optimal grasping locations across different objects. Furthermore, we were able to characterize the neural computations that the brain employs to reconstruct the 3D shape of objects. Klein LK, Maiello G, Paulun VC, Fleming RW (2020) Predicting precision grip grasp locations on three-dimensional objects. PLOS Computational Biology 16(8): e1008081. https://doi.org/10.1371/journal.pcbi.1008081

Текст от CORDIS, на английски · Данни: CORDIS, © Европейски съюз

Цел на проекта

I ask how vision guides grasping, and conversely, how learning to grasp objects constrains visual processing. Grasping an object feels effortless, yet the computations underlying grasp planning are nontrivial and there is an extensive literature describing the multifaceted features of visually guided grasping. I aim to bind this fragmented body of knowledge into a unified framework for understanding how humans visually select grasps. To do so I will use motion-tracking hardware (already in place at the University of Giessen) to measure and model human grasping patterns to 3D objects. I will rely on Dr. Fleming’s unique expertise with physical simulation to simulate human grasping with objects varying in shape and material. Joining behavioral measurements with computer simulations will provide a powerful data- and theory-driven approach to fully map out the space of human grasping behavior. The complementary goal of this proposal is to understand how grasping constrains visual processing of object shape and material. I plan to tackle this goal by building a computational model of visual processing for grasp planning. Both Dr. Fleming and I have previous experience with computational modelling of visual function. I will exploit powerful machine learning techniques to infer what kinds of visual representations are necessary for grasp planning. I will train Deep Neural Nets (for which the hardware and software is already in place and in use by the Fleming lab) using extensive physics simulations. Dissecting the learned network architecture and comparing the network’s performance to human behavior will tell us what information about shapes, material, and objects the human visual system encodes to plan motor actions. In short, with this research I aim to determine how processing within the human visual system is shaped by and guides hand motor action.

Оригинален текст от CORDIS (на английски).

Участници

JUSTUS-LIEBIG-UNIVERSITAET GIESSEN · GIESSENКоординаторГермания