EYE-HAND CONTROL · Eye-hand coordination and control: Neural network identification and experimental characterization of the brain's visual-motor reference frame transformation
FP6 — Marie Curie Actions (Human Resources and Mobility)
- Duration
- 2005-01-01 → 2007-12-31
- EU contribution
- €196,927
- Participants
- 2
- Scheme
- OIF
Lines connect the coordinator with its partners.
Results in brief
Final Activity Report Summary - EYE-HAND CONTROL (Eye-hand coordination and control: Neural network identification and ... characterization of the brain's visual-motor reference frame transformation)
Approximately half of the human brain is devoted to processing visual information for generating behaviour and one of the most common forms of behaviour is motor actions, e.g. moving our arms to reach out for a cup of coffee. Therefore, the present research project investigated how visual information was transformed into accurate arm movements. More specifically, visual information is encoded relative to the line of sight, i.e. in a retinal frame of reference, whereas a motor command for the arm must be specified in a reference frame relative to the shoulder, i.e. the insertion point of the arm in the upper body. As such, accurate motor planning requires a geometrical transformation between a visual and effector-related reference frame. In this project, a complete mathematical and experimental framework was developed for the first time to describe this highly non-linear three-dimensional visuomotor reference frame transformation in its entire complexity. In a first step, a formulaic mathematical framework was developed, modelling the complete geometry of the eye-to-head-to-arm transformation. We used Clifford's dual quaternion formalism as used in robotics to describe the involved three-dimensional geometry. This model made quantitative predictions on the precision and accuracy of reaching movements to visual targets in cases for which non-visual signals about eye and head orientation were not taken into account by the brain. These predictions were then tested experimentally in a second step. As a result, reaching errors made by human subjects were compatible with the hypothesis that the human brain used an internal model of the complete body geometry for reach planning. This computation was costly, as reflected by the increased reaching variability under large eye or head positions. Once we demonstrated that such a three-dimensional visuomotor conversion was indeed performed by the brain, an artificial neural network was trained and analysed in order to uncover a possible neural mechanism implementing reference frame conversions in distributed processing. It was shown that individual neurons in such a network performed partial, fixed reference frame transformations through their input-output relationship and that those partial transformations were then combined in a gain-weighted fashion to produce the overall transformation at the population level. In addition, we demonstrated that using different physiological techniques to identify the reference frame of neurons could result in different reference frames within the same neuron. This major advance called for a more careful interpretation of the results that were obtained across different methods. In an attempt to uncover the neural substrates of the real brain that implemented this three-dimensional reference frame transformation, high temporal resolution functional brain activity was recorded during a memory-guided pointing task using Magnetoencephalography (MEG). We could show that the visual to effector-centred transformation occurred in a network of occipital-parietal areas as soon as 300 ms after target presentation. The transformation between an extrinsic, higher-level spatial, and intrinsic, e.g. muscle-related, motor plan occurred in pre-motor and motor areas around the time of movement execution. This study identified for the first time the spatial and temporal aspects of the three-dimensional reference frame transformation for arm movements. Finally, the current theory about reference frame transformations was extended to velocity signals. This was important because static position signals and dynamic velocity information were processed in anatomically different brain structures that separated before any reference frame transformation could occur. We demonstrated in model-based experiments that smooth pursuit eye movements, whose main input was velocity, were geometrically accurate. Thus, velocity signals also underwent a three-dimensional reference frame transformation and this study opened up a whole new field of research.
Data: CORDIS, © European Union
Project objective
Eye-hand coordination is the very basis of human behaviour and thus a major issue in Neuroscience. No matter what we do with our hands, vision provides essential information about the movement goal to the motor system. However, in order to produce accurate movements, the brain must take the actual configuration of the eyes with respect to the head and body into account. Indeed, the hand has to move with respect to the arm and the shoulder whereas the target is projected onto the retina. The difference between these 'reference frames' is largely governed by the non-linear mathematics of rotations and the brain has thus to solve this problem to send a precise movement command to the involved muscles. Here, we propose to establish a fundamental framework to exp lain how the brain performs this reference frame transformation problem in eye-hand coordination. We will apply a multidisciplinary approach using mathematical and medical research tools to provide an overall understanding of how the system works. Therefore, we will for the first time establish an algorithmic and a physiologically realistic neural network model to get insight into the computational nature of the underlying brain areas. Resulting hypotheses and predictions will be tested experimentally by performing behavioural studies of eye-hand coordination on healthy human subjects. We will also directly identify the involved neural structures using functional Magnetic Resonance Imaging (fMRI) techniques. The results we expect will have a high impact in t he neural control of movement field because it describes how movement is created accurately. Furthermore, we will provide mathematical and physiological models for the understanding of neural diseases and future theoretical lesion studies. In addition, this first insight into the basic functional properties of the cortex will offer the unique knowledge to the prosthetics research community needed to develop neural prosthetic devices.
Original text from CORDIS.
Participants
Links
Data: CORDIS, © European Union