On human behavior recognition with deep learning and IR spectral signal restoration technologies in a natural classroom

Abstract

Human behavior recognition in a nature environment needs a special effort in accuracy human pose estimation. Infrared spectral signal has illustrated key role in the fields of human behavior recognition. But the slow spectral imaging data (band overlap and random noise) has limited its applications. In this work, we develop a deep learning network architecture to extract the human body parts and links them as a skeleton. To extract the human features precisely, we develop a rapid blind restoration model with linear canonical transform (LCT) regularization to recover the weak infrared spectral signals. Firstly, we apply the LCT transform on the weak IR spectral signal and high-resolution one to analyze their essential difference. Then we reveal that the sparsity distributions of the weak IR spectra imaging data is sparser than the high-resolution one. Inspired by those findings, an infrared spectral restoration method is developed to constraint the sparsity distribution of the observed IR spectra by L0-norm. Experimental results illustrate that the LCT based IR spectrum restoration method can well save IR spectral band structures and remove the existed Poisson noises for human behavior tracking. Furthermore, the recovered IR spectral signals are also applied to estimate the human pose on a nature classroom.