Accidents and military conflicts make the design of intelligent prostheses for amputees
urgent. Their control still remains difficult, and the cost is prohibitive. Prostheses’
democratisation and enrichment are important societal issues. Our project seeks
to make a step forward in these directions.
The project proposes to develop multimodal computerized technologies for an efficient
control of neuroprostheses by amputees in ecological environment i.e. at home. It will
enable an easy interaction with the environment and socialisation.
Analysis of visual information by machine learning, from body-worn video cameras and
eye-tracker, will be integrated with EMG signals from muscles. An HW/SW prototype with
a robotic arm will be developed and tested on volunteers amputees.
The project will advance the state-of-the-art in fusion of multimodal information
(Electromyography [EMG], vision), real-time recognition of natural objects in 2D/3Dframework.
Recognition of natural objects in an ecological environment is an open and strongly researched
problem in computer vision. The project advances the state-of-the-art developing efficient
tools on the basis of Deep Convolutional Neural Networks. The real-life scenarios of
interaction of amputees with their ecological
environment imply the necessity of adaptation of the recognition approach.
All vision tasks have to respond to real-time
constraints which cannot be satisfied only by hardware implementation. Fast search methods have
to be developed. The inclusion of computer vision in prostheses control is a new trend and very
early works are limited to the wrist orientation and artificial hand opening/closing. The approach
will include elbow and shoulder movement analysis thus covering a much wider range of
amputations and injuries and advancing control schemes of articulated robotic arms. The prototype
will be developed on POPPY platform. This will enable personalizing of prostheses. Furthermore,
the POPPY OpenSource environment will allow a rapid prototyping with available OpenSource sensors.
All these components and Software suites will be made available in OpenSource as well. The integration
of machine vision, EMG analysis and POPPY control components will be performed on a Software/Hardware
platform with distributed and wearable computing. The prototype will be tested in scenarios of gradually
increasing complexity including participation of volunteers amputees. All ethical issues will be addressed
accordingly to national regulations.
The multidisciplinary consortium comprises IT, sensorimotor neuroscience and medical researchers from
specialized rehabilitation hospital, and amputees’ association. The project is coordinated by INCIA