Multi-Sensing Data Fusion for Human Activity Recognition based on Neuromorphic Computing

Abstract

This paper proposes a multi-sensing Human Activity Recognition framework, which uses Neuromorphic computing to processing from Sensors and Radars of different type signals for data fusion and classification. At this point, Inertial Measurement Unit sensors and Universal Software-defined Radio Peripheral, and Radar devices are used to collect human activities signals separately. The feature extraction and selection process the sensors signal to dimension reduction without time factor by design an attention mechanism. And then, following Expectation-Maximization calculation to achieve a binary feature pattern that fits the discrete Hopfield neural network input. Depend on the Neuromorphic computing of associative memory function and similarity calculation to the neurons’ feedback output. It finally achieves human activity recognition with one-shot learning. There are explores multi-sensing human activity recognition between limited dataset and ensures accuracy without dropping. The technique can be extended to include more hardware signal processing to the system.