Radar micro-Doppler Signatures Research Articles

HRSpecNET: A Deep Learning-Based High-Resolution Radar Micro-Doppler Signature Reconstruction for Improved HAR Classification

HRSpecNET: A Deep Learning-Based High-Resolution Radar Micro-Doppler Signature Reconstruction for Improved HAR Classification

Whitening-Aided Learning from Radar Micro-Doppler Signatures for Human Activity Recognition.

Deep learning architectures are being increasingly adopted for human activity recognition using radar technology. A majority of these architectures are based on convolutional neural networks (CNNs) and accept radar micro-Doppler signatures as input. The state-of-the-art CNN-based models employ batch normalization (BN) to optimize network training and improve generalization. In this paper, we present whitening-aided CNN models for classifying human activities with radar sensors. We replace BN layers in a CNN model with whitening layers, which is shown to improve the model's accuracy by not only centering and scaling activations, similar to BN, but also decorrelating them. We also exploit the rotational freedom afforded by whitening matrices to align the whitened activations in the latent space with the corresponding activity classes. Using real data measurements of six different activities, we show that whitening provides superior performance over BN in terms of classification accuracy for a CNN-based classifier. This demonstrates the potential of whitening-aided CNN models to provide enhanced human activity recognition with radar sensors.

Model-Based Data Augmentation Applied to Deep Learning Networks for Classification of Micro-Doppler Signatures Using FMCW Radar

Deep neural networks (DNNs) have become a relevant subject in the classification of radio frequency signals and remote sensing data. A primary challenge is a tradeoff between obtaining data that are suitable for DNN training and the effort that making experimental measurements requires. Hence, the quality and quantity of data used for the training and testing of models are crucial for effective classifier development. The training dataset should cover a wide range of cases that synthesize the actual scenarios being classified. This work proposes a novel data augmentation method based on a deterministic model to generate a simulated dataset of radar micro-Doppler signatures suitable for unmanned aerial vehicle (UAV) target classification, without requiring measurement data. It is shown that the DNN trained using the properly generated model-based data offers improved classification accuracy performance. Results are presented for a two-class classification of the number of UAV motors using a 77-GHz frequency-modulated continuous-wave (FMCW) automotive radar system. The effectiveness of the proposed methodology is proven: a classification accuracy of 78.68% is achieved using a convolutional neural network (CNN) trained using the synthetic dataset, while an accuracy of 66.18% is achieved by using a typical signal processing data augmentation method on a limited measured dataset.

Exploring Radar Micro-Doppler Signatures for Recognition of Drone Types

In this study, we examine the use of micro-Doppler signals produced by different blades (i.e., puller and lifting blades) to aid in radar-based target recognition of small drones. We categorize small drones into three types based on their blade types: fixed-wing drones with only puller blades, multi-rotor drones with only lifting blades, and hybrid vertical take-off and landing (VTOL) fixed-wing drones with both lifting and puller blades. We quantify the radar signatures of the three drones using statistical measures, such as signal-to-noise ratio (SNR), signal-to-clutter ratio (SCR), Doppler speed, Doppler frequency difference (DFD), and Doppler magnitude ratio (DMR). Our findings show that the micro-Doppler signals of lifting blades in all three drone types were stronger than those of puller blades. Specifically, the DFD and DMR values of pusher blades were below 100 Hz and 0.3, respectively, which were much smaller than the 200 Hz and 0.8 values for lifting blades. The micro-Doppler signals of the puller blades were weaker and more stable than those of the lifting blades. Our study demonstrates the potential of using micro-Doppler signatures modulated by different blades for improving drone detection and the identification of drone types by drone detection radar.

A novel feature extraction method of human‐vehicle classification based on low resolution radar

In this paper, a novel method using target movement characteristic is proposed to classify human and vehicle based on low resolution radar system. Many studies have demonstrated the ability of radar micro-Doppler signature for ground moving targets classification. However, the micro-Doppler signals are not always observed in practical application. Considering the motion characteristic of human and vehicle, a multi-frame joint scheme is proposed, in which three suitable features are extracted and the Random Forest classifier is used to identify three kinds of moving targets. The experiment results with satisfied classification accuracy prove that the proposed method is capable of human-vehicle classification.

Person Identification With Low Training Sample Based on Micro-Doppler Signatures Separation

Due to the ability to handle low-light environments, poor weather conditions and privacy protection, person identification based on radar micro-Doppler (m-D) signatures has emerged as a research interest. Moreover, as it is always difficult to construct large-scale radar data sets, we propose an approach to recognize a person’s identity with limited training sample. In which, the fine m-D signal processing technology is first used to obtain high-quality m-D data spectrograms. A three-layer convolutional principal component analysis network with a dimension optimization architecture (CPCAN-3) is then designed to learn the highly discriminative features and address the identification problem. The model has few network parameters, is easy to train, and has low dependence on a large number of training samples. Different daily activities data are captured in an indoor environment to evaluate the performance of proposed method, and the state-of-the-art algorithms are chosen for comparison. The experimental results show that the proposed scheme performs better performance than the others on small data sets. Especially when the motion of “running” is adopted to identify persons, the model achieves 98% accuracy on the identification of ten people.

Pedestrian and Animal Recognition Using Doppler Radar Signature and Deep Learning.

Pedestrian occurrences in images and videos must be accurately recognized in a number of applications that may improve the quality of human life. Radar can be used to identify pedestrians. When distinct portions of an object move in front of a radar, micro-Doppler signals are produced that may be utilized to identify the object. Using a deep-learning network and time–frequency analysis, we offer a method for classifying pedestrians and animals based on their micro-Doppler radar signature features. Based on these signatures, we employed a convolutional neural network (CNN) to recognize pedestrians and animals. The proposed approach was evaluated on the MAFAT Radar Challenge dataset. Encouraging results were obtained, with an AUC (Area Under Curve) value of 0.95 on the public test set and over 0.85 on the final (private) test set. The proposed DNN architecture, in contrast to more common shallow CNN architectures, is one of the first attempts to use such an approach in the domain of radar data. The use of the synthetic radar data, which greatly improved the final result, is the other novel aspect of our work.

Person identification using a new CNN-based method and radar gait micro-Doppler signatures

Abstract The use of gait micro-Doppler signatures to identify a person is a hot topic of research. In this paper, we present a new CNN-based method called Multi-Scale CNN (MS-CNN) to obtain features at multiple scales. It extracts shallow features at low-level multi-scale blocks by using multiple kernels at the same time, then extracts deep features and fuses multi-branch embedding features at high-level multi-branch blocks. Experimental results reveal that our method outperforms other commonly used CNN algorithms in terms of accuracy, allowing it to be used for personal identification.

Semisupervised Human Activity Recognition With Radar Micro-Doppler Signatures

Human activity recognition (HAR) plays a vital role in many applications, such as surveillance, in-home monitoring, and health care. Portable radar sensor has been increasingly used in HAR systems in combination with deep learning (DL). However, it is both difficult and time-consuming to obtain a large-scale radar dataset with reliable labels. Insufficient labeled data often limit the generalization of DL models. As a result, the performance of DL models will drop when being applied to a new scenario. In this sense, only labeling a small portion of data in the large-scale radar dataset is more feasible. In this article, we propose a semisupervised transfer learning (TL) algorithm, “ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">joint domain and semantic transfer learning</i> ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">JDS-TL</i> ),” for radar-based HAR, which is composed of two modules: unsupervised domain adaptation (DA) and supervised semantic transfer. By employing a sparsely labeled dataset to train the HAR model, the proposed method alleviates the need of labeling a significantly large number of radar signals. We adopt a public radar micro-Doppler spectrogram dataset including six human activities to evaluate <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">JDS-TL</i> . Experiments show that the proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">JDS-TL</i> is able to recognize the six activities with an average accuracy of 87.6% when there are only 10% instances labeled in the training dataset. Ablation analysis also demonstrates the efficiency of the DA and the semantic transfer modules.

Multiscenario Open-Set Gait Recognition Based on Radar Micro-Doppler Signatures

Unlike face recognition, gait recognition does not require the cooperation of human targets, making it an attractive research area in both academia and industry. While there has been extensive research in recent years on the feasibility of employing radar micro-Doppler signatures for gait recognition, little research has been conducted on how to deal with unknown target detection in micro-Doppler-based gait recognition problems, referred to as ’open-set’ gait recognition. In this case, it is probable that the unknown samples will resemble known samples, hence increasing the likelihood of misclassification. In this paper, we focus on the problem of multi-scenario open-set gait recognition and present a new open-set classifier that minimizes misclassification in open-set recognition. We create a compact representation space to reduce false negatives with a supervised contrastive learning method and propose an ensemble-based out-of-distribution (OOD) detection module so that false positives can be alleviated at both the pixel and semantic levels. We perform the radar measurements in eight different walking manners and construct a micro-Doppler-based multi-scenario gait dataset for further evaluate the proposed method’s performance. The results reveal that the proposed method outperforms the previous state-of-the-art open-set models, demonstrating its great potential for solving the radar-based multi-scenario open-set gait recognition problem. This research would facilitate gait recognition in complex scenarios and make non-contact sensing applicable to industrial security.

DecompNet: Deep Context Dependent Decomposition Network for Micro-Doppler Signature of Walking Human

In recent years, human activity recognition using radar micro-Doppler signature has been the focus of many researches. Echo signal from body limbs is the origin of motion parameters estimation. To this end, decomposition of this composite signal into components corresponding to each limb is necessary and rather complex. In this paper, a novel method for body micro-Doppler signal decomposition is proposed using a deep convolutional neural network designated DecompNet which uses a context window on the spectral input stream to enhance the decomposition performance. This network is able to decompose 9 components of body limbs. The proposed network, is able to perform signal decomposition and denoising simultaneously, even in low SNR conditions. So, it outperforms the other similar methods both in terms of decomposition ability and robustness to observation noise. Simulation results show that DecompNet can outperform mathematical transform-based methods and also keep working in the SNRs as low as −15dB.

Open-Set Human Identification Based on Gait Radar Micro-Doppler Signatures

Human identification based on gait radar micro-Doppler signatures has received significant attention in recent years. In real world applications, a recognition system should have the ability to reject unknown identities as well as correctly recognize known identities, i.e. the open-set identification. In this paper, we propose a novel framework to tackle this challenging problem. We base our approach on a probabilistic discriminant model built on a deep discriminative representation network (DDRN). Specifically, we first train a DDRN with the cosine margin (CM) loss to map gait samples into an embedding space where the learned features belonging to the same identity are much closer together and those of different identities are further apart. Then, in the learned embedding space, the class-inclusion probability (CIP) model associated with each known identity can be constructed based on the statistical Extreme Value Theory (EVT) to bound each other’s support region, which can then be used to estimate the class-belongingness probabilities of a probe sample. Finally, we threshold on these probabilities to determine whether it belongs to one of the known identities or an unknown class. Experiments on a radar measured gait dataset show the effectiveness of our approach.

Simulation framework for activity recognition and benchmarking in different radar geometries

Radar micro-Doppler signatures have been proposed for human monitoring and activity classification for surveillance and outdoor security, as well as for ambient assisted living in healthcare-related applications. A known issue is the performance reduction when the target is moving tangentially to the line of sight of the radar. Multiple techniques have been proposed to address this, such as multistatic radar and to some extent, interferometric (IF) radar. A simulator is presented to generate synthetic data representative of eight radar systems (monostatic, circular multistatic and in-line multistatic [IM] and IF) to quantify classification performances as a function of aspect angles and deployment geometries. This simulator allows an unbiased performance evaluation of different radar systems. Six human activities are considered with signatures originating from motion-captured data of 14 different subjects. The classification performances are analysed as a function of aspect angles ranging from 0° to 90° per activity and overall. It demonstrates that IF configurations are more robust than IM configurations. However, IM performs better at angles below 55° before IF configurations take over.

Human Motion Recognition With Limited Radar Micro-Doppler Signatures

The performance of deep learning (DL) algorithms for radar-based human motion recognition (HMR) is hindered by the diversity and volume of the available training data. In this article, to tackle the issue of insufficient training data for HMR, we propose an instance-based transfer learning (ITL) method with limited radar micro-Doppler (MD) signatures, alleviating the burden of collecting and annotating a large number of radar samples. ITL is a unique algorithm that consists of three interconnected parts, including DL model pretraining, correlated source data selection, and adaptive collaborative fine-tuning (FT). Any of the three components cannot be excluded; otherwise, the performance of the entire algorithm decreases. The experiments with a radar data set of six human motions show that ITL achieves state-of-the-art performance for HMR with limited training samples, outperforming several existing transfer learning approaches. Especially, when there are only 100 samples per person per class, ITL yields an F1 score of 96.7%. Last but not least, ITL is more generalized to human motion differences. Though adapted to recognize the persons’ motions in a small-scale target data set, ITL can also classify the persons’ motion data used for pretraining, achieving up to 11.0% F1 score enhancement over the conventional FT method.

Doppler Radar for the Extraction of Biomechanical Parameters in Gait Analysis.

The applicability of Doppler radar for gait analysis is investigated by quantitatively comparing the measured biomechanical parameters to those obtained using motion capturing and ground reaction forces. Nineteen individuals walked on a treadmill at two different speeds, where a radar system was positioned in front of or behind the subject. The right knee angle was confined by an adjustable orthosis in five different degrees. Eleven gait parameters are extracted from radar micro-Doppler signatures. Here, new methods for obtaining the velocities of individual lower limb joints are proposed. Further, a new method to extract individual leg flight times from radar data is introduced. Based on radar data, five spatiotemporal parameters related to rhythm and pace could reliably be extracted. Further, for most of the considered conditions, three kinematic parameters could accurately be measured. The radar-based stance and flight time measurements rely on the correct detection of the time instant of maximal knee velocity during the gait cycle. This time instant is reliably detected when the radar has a back view, but is underestimated when the radar is positioned in front of the subject. The results validate the applicability of Doppler radar to accurately measure a variety of medically relevant gait parameters. Radar has the potential to unobtrusively diagnose changes in gait, e.g., to design training in prevention and rehabilitation. As contact-less and privacy-preserving sensor, radar presents a viable technology to supplement existing gait analysis tools for long-term in-home examinations.

The accuracy and predictability of micro Doppler radar signature projection algorithm measuring functional movement in NCAA athletes

BackgroundDevelopment of accessible cost-effective technology to objectively, reliably, and accurately predict musculoskeletal injury risk could aid the effort to prevent chronic pain and disability. Recent work on micro-Doppler radar suggests it merits investigation towards these goals. The micro-Doppler signals that are created can infer differences in gross movements such as walking versus crawling in military settings where direct vision is not possible. Unique micro-Doppler signals may be able to identify more subtle movement patterns which would not be easily seen by the human eye. Research QuestionCan micro Doppler radar predictably and accurately identify subtle differences in movement conditions? MethodsThis is a cross sectional study recruiting NCAA athletes to jump in front of the micro-Doppler radar barefoot, with shoes, and shoes with a heel lift. The micro-Doppler radar signature projection algorithm was developed to determine whether the radar is able to distinguish the three distinct movement patterns. ResultsConfusion matrices were used to visualize the performance of the support-vector machine at the 80/20 test/train split correctly classifying barefoot subjects, shoes and heel lift, and shoes correctly at 0° with respect to the radar 90.9 %, 86.7 %, and 89.5 % of the time, respectively. At 90° with respect to the radar, it was successful 94.1 %, 100 %, and 80 % of the time, respectively. ConclusionThis study suggests that the micro-Doppler radar signature projection algorithm is highly accurate and able to predict subtle differences in movement that are not readily observed with conventional motion capture systems. Future studies are needed to better understand if micro-Doppler signals can identify pathologic movement patterns or movement that is associated with increased risk of injury.

Temporal Convolutional Neural Networks for Radar Micro-Doppler Based Gait Recognition.

The capability of sensors to identify individuals in a specific scenario is a topic of high relevance for sensitive sectors such as public security. A traditional approach involves cameras; however, camera-based surveillance systems lack discretion and have high computational and storing requirements in order to perform human identification. Moreover, they are strongly influenced by external factors (e.g., light and weather). This paper proposes an approach based on a temporal convolutional deep neural networks classifier applied to radar micro-Doppler signatures in order to identify individuals. Both sensor and processing requirements ensure a low size weight and power profile, enabling large scale deployment of discrete human identification systems. The proposed approach is assessed on real data concerning 106 individuals. The results show good accuracy of the classifier (the best obtained accuracy is 0.89 with an F1-score of 0.885) and improved performance when compared to other standard approaches.

Synthesis of Micro-Doppler Signatures of Human Activities From Different Aspect Angles Using Generative Adversarial Networks

In this paper, we propose to produce synthesized micro-Doppler signatures from different aspect angles through conditional generative adversarial networks (cGANs). Micro-Doppler signatures of non-rigid human body motions vary considerably as a function of the radar's aspect angle. Because the direction of the human motion can be arbitrary, a large volume of training data across diverse aspects is needed for practical target activity classification through machine learning. As measurements can require significant monetary and labor costs, the synthesis of micro-Doppler signatures can be an alternate solution. Therefore, we investigate the feasibility of data augmentation through synthesizing micro-Doppler signatures of human activities from diverse radar aspect angles with input data from a single aspect angle. For the training data, the micro-Doppler radar signatures of 12 human activities are generated from different angles ranging from 0 to 315 degrees, at 45-degree increments, through simulations. For each angle, cGANs are trained to synthesize the micro-Doppler signatures for that specific angle given micro-Doppler signatures from another angle. The output of each model is evaluated by calculating mean-square errors and structural similarity indexes between the synthesized micro-Doppler signatures and the ground-truth ones obtained from simulations. We test three different scenarios, and report the respective results.

Theoretical and Experimental Analysis of Radar Micro-Doppler Signature Modulated by Rotating Blades of Drones

Radar signatures contributed by rotating blades of drones usually refer to the kinematical micro-Doppler phenomenon on spectrograms. Here, theoretical analysis indicates that when sampling time is short enough that this signature becomes instant micro-Doppler signatures with scattering characteristics, appearing as “blade flash” in the time domain and rotor blade modulation (RBM) feature in the spectrum. In an anechoic chamber, we track this instant micro-Doppler signature by using a dynamic measurement system composed of a network analyzer and a high-speed camera to collect high-resolution range profile of a consumer drone, DJI Phantom 3 Vision within S-band, X-band, and Ku-band. The experimental results demonstrate that rotating blades do cause signal fluctuation of “blade flash” and register the RBM feature in the spectrum, and the modulation effects are weak within S-band, strong within X-band, and stronger within Ku-band. This instant micro-Doppler with scattering characteristic establishes a mapping between radar signature and the rotation structure component of the drone and enables us to distinguish a drone from other nonblades objects in the air, especially with little radar observation time.

Dop‐NET: a micro‐Doppler radar data challenge

Radar sensors have a new growing application area of dynamic hand gesture recognition. Traditionally radar systems are considered to be very large, complex and focused on detecting targets at long ranges. With modern electronics and signal processing it is now possible to create small compact RF sensors that can sense subtle movements over short ranges. For such applications, access to comprehensive databases of signatures is critical to enable the effective training of classification algorithms and to provide a common baseline for benchmarking purposes. This Letter introduces the Dop-NET radar micro-Doppler database and data challenge to the radar and machine learning communities. Dop-NET is a database of radar micro-Doppler signatures that are shareable and distributed with the purpose of improving micro-Doppler classification techniques. A continuous wave 24 GHz radar module is used to capture the first contributions to the Dop-NET database and classification results based on discriminating these hand gestures as shown.