Synthesis of Micro-Doppler Signatures of Human Activities From Different Aspect Angles Using Generative Adversarial Networks

Ibrahim Alnujaim,Shobha Sundar Ram,Youngwook Kim,Daegun Oh

doi:10.1109/access.2021.3068075

Abstract

In this paper, we propose to produce synthesized micro-Doppler signatures from different aspect angles through conditional generative adversarial networks (cGANs). Micro-Doppler signatures of non-rigid human body motions vary considerably as a function of the radar's aspect angle. Because the direction of the human motion can be arbitrary, a large volume of training data across diverse aspects is needed for practical target activity classification through machine learning. As measurements can require significant monetary and labor costs, the synthesis of micro-Doppler signatures can be an alternate solution. Therefore, we investigate the feasibility of data augmentation through synthesizing micro-Doppler signatures of human activities from diverse radar aspect angles with input data from a single aspect angle. For the training data, the micro-Doppler radar signatures of 12 human activities are generated from different angles ranging from 0 to 315 degrees, at 45-degree increments, through simulations. For each angle, cGANs are trained to synthesize the micro-Doppler signatures for that specific angle given micro-Doppler signatures from another angle. The output of each model is evaluated by calculating mean-square errors and structural similarity indexes between the synthesized micro-Doppler signatures and the ground-truth ones obtained from simulations. We test three different scenarios, and report the respective results.

Highlights

Radar based analysis of human activities is increasingly applied in defense, surveillance, and health care scenarios for its capability to operate 24/7, in through-object scenarios, under poor weather conditions, and in situations where privacy is a concern
We explore the feasibility of using radar spectrograms gathered from a single aspect angle of human activity to synthesize spectrograms from other angles using conditional Generative adversarial networks (GAN), a variant of the general GAN used in previous micro-Doppler radar literature [12], [13]
The contributions of this paper are as follows; i) we proposed to synthesize micro-Doppler signatures seen from different aspect angles, ii) image-to-image translation through deep learning has been applied to micro-Doppler signatures

Summary

INTRODUCTION

Radar based analysis of human activities is increasingly applied in defense, surveillance, and health care scenarios for its capability to operate 24/7, in through-object scenarios, under poor weather conditions, and in situations where privacy is a concern. Deep learning algorithms, known as some of the best for classification, require large data sets of micro-Doppler signatures, captured across diverse radar aspects, to exhibit stable classification accuracy for arbitrary human motions. I. Alnujaim et al.: Synthesis of Micro-Doppler Signatures of Human Activities From Different Aspect Angles Using GANs features in radar imagery such as spectrograms or range-Doppler diagrams for target recognition [9], while DRNN analyzes time-varying radar features and identifies temporal patterns in them [10]. From an information point of view, multistatic radars offer a second important advantage over single-aspect angle data obtained from a monostatic radar.

HUMAN MICRO-DOPPLER SIMULATION

CONCLUSION