Sensory fusion with EMG and DVS/DAVIS sensors

As the traditional single sensor information acquisition system is becoming more and more difficult to meet the complex practical applications, the fusion application of multisensory have been made a great deal of research. In this workgroup, we propose to fuse EMG and DAVIS signals to classify the human upper limb movements more accurately. The final goal of the WG is to build a mobile app to run the fusion and recognition in real time on a portable platform.
For the purpose of the WG we need to collect a new dataset consisting of finger movements recorded by EMG sensors and DAVIS camera. The dataset can be processed using two approaches:

1) EMG and DAVIS signal will be processed separately by feature extraction, and the two outputs will be fused at the classification level in a mobile app. This scenario can be divided into multiple tasks:
a) EMG feature extraction (spiking and/or non-spiking algorithms)
b) event and or frame-based image feature extraction
c) Feature fusion and training the classifier offline (e.g. SVM/LSTM)
d) App implementation of the trained network

2) EMG and DAVIS signals will be fused in a spiking recurrent neural network and an event-based learning algorithm will be used to train the readout phase for gesture recognition. This scenario can be either implemented on simulation (Brian2) or a neuromorphic chip (Dynap-se).
Disclaimer we do not see scenario N.2 as being portable to a mobile app as yet.

Go to group wiki


Day Time Location
Wed, 24.04.2019 15:30 - 16:00 Sala Panorama
Wed, 24.04.2019 19:00 - 20:00 Sala Panorama
Thu, 25.04.2019 14:00 - 15:00 Sala Panorama
Fri, 26.04.2019 21:30 - 22:30 Sala Arcate
Mon, 29.04.2019 19:00 - 20:00 Sala Panorama


Elisa Donati
Melika Payvand
Gemma Taverni


Enea Ceolini
Erika Covi
Elisa Donati
Jakub Fil
Charlotte Frenkel
Giacomo Indiveri
Lyes Khacef
Sepp Kollmorgen
Alexander Kugele
Marco Monforte
Melika Payvand
Adam Perrett
Nicoletta Risi
Marcus Schüffny
Baris Serhan
Jonathan Tapson
Jingyue Zhao