micro-Doppler Signatures Research Articles

Continuous Human Motion Recognition Using Micro-Doppler Signatures in the Scenario With Micro Motion Interference

The application of micro-Doppler-based continuous human motion recognition (HMR) is greatly hindered by non-target micro motion interference, due to the deformation of micro-Doppler signatures of target human motion caused by such interference. In this paper, we propose a novel continuous HMR method using micro-Doppler signatures that can work in the scenario with non-target micro motion interference. Specifically, a signal preprocessing architecture is designed, where the empirical mode decomposition is employed to remove the interference in radar raw signal and the multiwindow time-frequency representation is used to generate the time-frequency distribution (TFD) with high concentration. Moreover, a tailored network, that integrates multiscale squeeze-and-excitation network for feature sequence extraction, stacked bidirectional long short-term memory for sequence labeling and connectionist temporal classification algorithm for label transcription, is employed to recognize continuous human motion from TFD. The experimental results show that the proposed method outperforms existing methods in terms of recognition accuracy and generalization.

Accurate Person Identification Based on Combined Sit-to-Stand and Stand-to-Sit Movements Measured Using Doppler Radars

This article demonstrates the identification of 10 persons with 99% accuracy achieved by combining micro-Doppler signatures of sit-to-stand and stand-to-sit movements. Data from these movements are measured using two radars installed above and behind the person. Images of Doppler spectrograms generated using the measured data are combined and input to a convolutional neural network. Experimental results show the significantly better accuracy of the proposed method compared with conventional methods that do not perform data combination. The accuracy of identifying 10 participants having similar ages and physical features was 96–99%, despite the relatively small training set (number of training samples: 50–90 Doppler radar images per person). These results suggest that combining sit-to-stand and stand-to-sit movements provides sufficient information for accurate person identification and such information can be remotely acquired using Doppler radar measurements.

An Investigation of Rotary Drone HERM Line Spectrum under Manoeuvering Conditions.

Detecting and identifying drones is of great interest due to the proliferation of highly manoeuverable drones with on-board sensors of increasing sensing capabilities. In this paper, we investigate the use of radars for tackling this problem. In particular, we focus on the problem of detecting rotary drones and distinguishing between single-propeller and multi-propeller drones using a micro-Doppler analysis. Two different radars were used, an ultra wideband (UWB) continuous wave (CW) C-band radar and an automotive frequency modulated continuous wave (FMCW) W-band radar, to collect micro-Doppler signatures of the drones. By taking a closer look at HElicopter Rotor Modulation (HERM) lines, the spool and chopping lines are identified for the first time in the context of drones to determine the number of propeller blades. Furthermore, a new multi-frequency analysis method using HERM lines is developed, which allows the detection of propeller rotation rates (spool and chopping frequencies) of single and multi-propeller drones. Therefore, the presented method is a promising technique to aid in the classification of drones.

Active Shooter Detection in Multiple-Person Scenario Using RF-Based Machine Vision

Emerging applications of radio frequency (RF) vision sensors for security and gesture recognition primarily target single individual scenarios which restricts potential applications. In this article, we present the design of a cyber-physical framework that analyzes RF micro-Doppler signatures for individual anomaly detection, such as a hidden rifle among multiple individuals. RF avoids certain limitations of video surveillance, such as recognizing concealed objects and privacy concerns. Current RF-based approaches for human activity detection or gesture recognition usually consider single individual scenarios, and the features extracted for such scenarios are not applicable for multi-person cases. From a machine learning perspective, the RF sensor spectrogram images are conducible for training using deep convolutional neural networks. However, generating a large labeled training dataset with an exhaustive variety of multi-person scenarios is extremely time consuming and nearly impossible due to the wide range of combinations possible. We present approaches for multi-person spectrogram generation based on individual person spectrograms that can augment the training dataset and increase the accuracy of prediction. Our results show that the spectrogram generated by RF sensors can be harnessed by artificial intelligence algorithms to detect anomalies such as a concealed weapon for single and multiple people scenarios. The proposed system can aid as a standalone tool, or be complemented by video surveillance for anomaly detection, in scenarios involving single or multiple individuals.

Time‐based multi‐component irregular FM micro‐Doppler signals decomposition via STVMD

The decomposition of multi-component micro-Doppler signals overlapping in the time-frequency (T-F) domain is critical and challenging, especially in the case of irregular instantaneous Doppler frequency. The authors propose a novel time-based decomposition method called the short-time variational mode decomposition (STVMD) to analyse the irregular (FM) micro-Doppler signals, and present an optimal model combined with T-F transformation. Then, considering the STVMD may fail to extract the instantaneous frequency (IF) of overlapped components, an improved STVMD algorithm is put forward. Since the dependence of the STVMD algorithm on the initial value, they adopt the Kalman filtering to implement IF tracking and regrouping under the global constraint, further accelerating the convergence of the algorithm. Furthermore, due to the mode aliasing at the intersection point, they adopt a degenerate STVMD model to decompose the signals with known centre frequencies, which can be viewed as a Wiener filter. With the two steps, the improved STVMD algorithm can effectively solve the decomposition of T-F overlapping irregular FM micro-Doppler signals. Compared with the peak ridge technique and the ridge path regrouping and intrinsic chirp component decomposition (RPRG + ICCD), the proposed method shows the effectiveness and adaptability even for irregular FM signals with large T-F spectrum amplitude fluctuation in the low signal-to-noise ratio environment.

Theoretical and Experimental Analysis of Radar Micro-Doppler Signature Modulated by Rotating Blades of Drones

Radar signatures contributed by rotating blades of drones usually refer to the kinematical micro-Doppler phenomenon on spectrograms. Here, theoretical analysis indicates that when sampling time is short enough that this signature becomes instant micro-Doppler signatures with scattering characteristics, appearing as “blade flash” in the time domain and rotor blade modulation (RBM) feature in the spectrum. In an anechoic chamber, we track this instant micro-Doppler signature by using a dynamic measurement system composed of a network analyzer and a high-speed camera to collect high-resolution range profile of a consumer drone, DJI Phantom 3 Vision within S-band, X-band, and Ku-band. The experimental results demonstrate that rotating blades do cause signal fluctuation of “blade flash” and register the RBM feature in the spectrum, and the modulation effects are weak within S-band, strong within X-band, and stronger within Ku-band. This instant micro-Doppler with scattering characteristic establishes a mapping between radar signature and the rotation structure component of the drone and enables us to distinguish a drone from other nonblades objects in the air, especially with little radar observation time.

EM-Sign: A Non-Contact Recognition Method Based on 24 GHz Doppler Radar for Continuous Signs and Dialogues

We studied continuous sign language recognition using Doppler radar sensors. Four signs in Chinese sign language and American sign language were captured and extracted by complex empirical mode decomposition (CEMD) to obtain spectrograms. Image sharpening was used to enhance the micro-Doppler signatures of the signs. To classify the different signs, we utilized an improved Yolov3-tiny network by replacing the framework with ResNet and fine-tuned the network in advance. This method can remove the epentheses from the training process. Experimental results revealed that the proposed method can surpass the state-of-the-art sign language recognition methods in continuous sign recognition with a precision of 0.924, a recall of 0.993, an F1-measure of 0.957 and a mean average precision (mAP) of 0.99. In addition, dialogue recognition in three daily conversation scenarios was performed and evaluated. The average word error rate (WER) was 0.235, 10% lower than in of other works. Our work provides an alternative form of sign language recognition and a new approach to simplify the training process and achieve a better continuous sign language recognition effect.

Multiperson Continuous Tracking and Identification From mm-Wave Micro-Doppler Signatures

In this work, we investigate the use of backscattered mm-wave radio signals for the joint tracking and recognition of identities of humans as they move within indoor environments. We build a system that effectively works with multiple persons concurrently sharing and freely moving within the same indoor space. This leads to a complicated setting, which requires one to deal with the randomness and complexity of the resulting (composite) backscattered signal. The proposed system combines several processing steps: at first, the signal is filtered to remove artifacts, reflections, and random noise that do not originate from humans. Hence, a density-based classification algorithm is executed to separate the Doppler signatures of different users. The final blocks are trajectory tracking and user identification, respectively, based on Kalman filters and deep neural networks. Our results demonstrate that the integration of the last-mentioned processing stages is critical toward achieving robustness and accuracy in multiuser settings. Our technique is tested both on a single-target public data set, for which it outperforms state-of-the-art methods, and on our own measurements, obtained with a 77 GHz radar on multiple subjects simultaneously moving in two different indoor environments. The system works in an online fashion, permitting the continuous identification of multiple subjects with accuracies up to 98%, e.g., with four subjects sharing the same physical space, and with a small accuracy reduction when tested with unseen data from a challenging real-life scenario that was not part of the model learning phase.

Human identification based on natural gait micro‐Doppler signatures using deep transfer learning

Gait-based human identification aims to identify individuals by their walking style. In this study, the authors investigate the use of micro-Doppler (m-D) signatures retrieved from a frequency-modulated continuous-wave radar sensor to identify individuals based on their natural gait characteristics. The gait dataset of 20 persons has been collected in an indoor environment where each subject was allowed to walk naturally and freely, which is absolutely more realistic and challenging than most existing works based on limited walking behaviour. Then, they perform identification using a transfer learned ResNet-50, which was fine-tuned on the gait m-D dataset based on the deep transfer learning technique. Through experiments, they first determined the optimal observation window length of m-D samples, and with this input, they achieved an average identification accuracy of 96.7% on the test set for 20 subjects, which highly outperforms the state-of-the-art methods. The presented work provides prospects in developing a solution to automatically identify persons based on gait characteristics using a simple and cost-efficient radar device.

Extraction of Doppler signature of micro‐to‐macro rotations/motions using continuous wave radar‐assisted measurement system

Mechanical–structural vibration and/or rotational parts of the targets will induce micro-Doppler frequency in addition to the main Doppler components. In the today's military era, detections, measurements and decision making have to be made after the thorough analysis of main and micro-Doppler signatures of the targets to get their full profile particularly for defence applications. In addition, few of the low radar cross-section targets can be detected only by extracting and processing the micro-Doppler signatures corresponding to the rotations of their propellant rotor blades. Therefore, experimental studies to measure the micro-to-macro rotation/motion generated Doppler frequency and performing its associated measurements become significant. The authors built a C-band (5.3 GHz) continuous wave radar and used it to measure the Doppler frequency generated by micro-to-macro rotations/motions. The detection and measurement accuracy of the developed radar is assessed by series of different open-environment experimental case studies: revolution per minute measurement of rotating blades, separation of multiple rotating blades, oscillation per minute measurement of a swinging pendulum, detection of approaching/receding motion and the Doppler signature extraction of walking/jogging/cycling person. All these measurement values are validated against the standard master instrument readings and theoretical calculations. Finally, the limitations of this system and required near-future research works to enhance its performance are listed.

Improved Drone Classification Using Polarimetric Merged-Doppler Images

We propose a drone classification method for polarimetric radar, based on convolutional neural network (CNN) and image processing methods. The proposed method improves drone classification accuracy when the micro-Doppler signature is very weak by the aspect angle. To utilize received polarimetric signal, we propose a novel image structure for three-channel image classification CNN. To reduce the size of data from four different polarization while securing high classification accuracy, an image processing method and structure are introduced. The data set is prepared for a three type of drone, with a polarimetric Ku-band frequency modulated continuous wave (FMCW) radar system. Proposed method is tested and verified in an anechoic chamber environment for fast evaluation. A famous CNN structure, GoogLeNet, is used to evaluate the effect of the proposed radar preprocessing. The result showed that the proposed method improved the accuracy from 89.9% to 99.8%, compared with single polarized micro-Doppler image. We compared the result from the proposed method with conventional polarimetric radar image structure and achieved similar accuracy while having half of full polarimetric data.

Radar recognition of multiple micro‐drones based on their micro‐Doppler signatures via dictionary learning

Most existing work on radar classification of micro-drones assumes that the received signal is wholly reflected from a single micro-drone. However, when there are multiple micro-drones in the observed scene, the superimposition of their micro-Doppler signatures increases the classification difficulty. In particular, it is even more challenging to determine if a specific type of micro-drones exists. In this study, a method for recognition of multiple micro-drones based on their micro-Doppler signatures via dictionary learning is introduced. First, the dictionary is learnt for each type of micro-drone by using the K-SVD algorithm on cadence-velocity diagrams (CVD) of training samples. The CVD is obtained by computing the Fourier transform of the time series of a complex time-frequency spectrogram. Subsequently, the sparse representation of the CVD of multiple micro-drones is obtained by the orthogonal matching pursuit algorithm with the learnt dictionary. Finally, a threshold detector is applied to the sparse solution in order to extract the components of multiple micro-drones. Experimental results using measured data, which are collected from hovering drones by a continuous-wave radar in an indoor environment, show that this dictionary-learning-based method achieves a recognition performance of 93% when half of the measured data are used for training.

Multi-Person Recognition Using Separated Micro-Doppler Signatures

It is challenging to recognize individuals when they move in the radar field of view due to the superimposition of micro-Doppler signatures. This paper presents a multi-person recognition approach by separating micro-Doppler signatures of multiple persons into their individual components. The preliminary separation can be obtained by their range difference in a high resolution radar. A multi-task learning network is designed for both the fine separation of micro-Doppler signatures and the personnel recognition. A frequency modulated continuous waveform (FMCW) radar that operates at 77 GHz for automotive applications is used in experiments. The proposed deep-neural-network-based approach gives a convincing result in the test.

A RPCA-Based ISAR Imaging Method for Micromotion Targets.

Micro-Doppler generated by the micromotion of a target contaminates the inverse synthetic aperture radar (ISAR) image heavily. To acquire a clear ISAR image, removing the Micro-Doppler is an indispensable task. By exploiting the sparsity of the ISAR image and the low-rank of Micro-Doppler signal in the Range-Doppler (RD) domain, a novel Micro-Doppler removal method based on the robust principal component analysis (RPCA) framework is proposed. We formulate the model of sparse ISAR imaging for micromotion target in the framework of RPCA. Then, the imaging problem is decomposed into iterations between the sub-problem of sparse imaging and Micro-Doppler extraction. The alternative direction method of multipliers (ADMM) approach is utilized to seek for the solution of each sub-problem. Furthermore, to improve the computational efficiency and numerical robustness in the Micro-Doppler extraction, an SVD-free method is presented to further lessen the calculative burden. Experimental results with simulated data validate the effectiveness of the proposed method.

Human Activity Classification Based on Micro-Doppler Signatures by Multiscale and Multitask Fourier Convolutional Neural Network

Human detection and activity classification has recently become a key technology in many applications, e.g., human computer interaction and surveillance for public and industrial security. In this work, we propose a novel end-to-end deep learning-based framework called the Fourier convolutional neural network (F-Convents) to tackle this problem. The input of F-ConvNet consists of raw frames of radar data. It is fed to a new layer called the Fourier layer, which transforms the raw radar signal into a domain optimized for the classification task. A novel weight initialization method tailored for the Fourier layer is also proposed. Moreover, we use dilated convolutions to further improve both performance and efficiency. To achieve better convergence and accuracy, a multi-scale and multi-task loss consisting of cross-entropy and triplet loss with a novel training paradigm called dynamic training is proposed. Experimental results show that F-ConvNet surpasses state-of-the-art methods by 3% in terms of classification accuracy.

A Harmonics Interferometric Doppler Sensor With a Neon Lamp Detector

A cost-effective harmonic interferometric Doppler sensor has been developed to obtain both amplitude and phase of the micro-Doppler signatures of moving objects. A phase chopper is used to generate the harmonics of the interferometry signal. For cost-effectiveness, a Gunn diode source is used for signal generation and a neon lamp is employed as a broadband detector. An advanced signal processing algorithm has been developed to extract both amplitude and phase Doppler signatures. A prototype has been built to demonstrate the effectiveness of the harmonic interferometric Doppler sensor at 35 GHz, with Doppler objects such as human hand and metal objects. In addition, the broadband detection capability of the neon lamp allows the Doppler sensor to work up to the millimeter-wave regime and potentially beyond the terahertz (THz) wave regime. Due to the ruggedness of the neon lamp detector, it is expected that the Doppler sensor has important applications of sensing and wireless communication in harsh environment such as high-temperature, high-power and radioactive conditions.

Micro Details Make for Macro Applications [Book/Software Reviews

This book is divided into three basic areas: overview, mathematics, and applications. The analysis of micro-Doppler rapidly examines the mathematics that describe motion: fortunately, the author offers a web link to download MATLAB code for many examples. The author also provides many clear and detailed explanations with interesting examples of the use of micro-Doppler signals. This second edition of the book contains three new chapters, mostly on applications, along with updates to the final two chapters. As would be expected, the first chapter of the book covers the basics of Doppler signals and micro-Doppler. Naturally, because objects are moving and changing, there is a tradeoff between the frequency change in micro-Doppler and the time in which it occurs. All of this makes signal processing interesting and brings up timefrequency transforms. The next three chapters are the mathematical core of the book, with discussions of Euler angles, quaternions, rigid body motion (like drone rotor blades), and nonrigid body motion (such as people and animals). All of the math provides the foundation for Chapters 5, 6, and 8, which look at applications and the signatures of motion and statistics. Chapter 7 complements all of the application information, discussing electronic system block Alfy Riddle. The final chapter provides a summary and a look toward future progress. This book is very appealing for anyone and is particularly useful for someone developing a product or working in this field.

Dop‐NET: a micro‐Doppler radar data challenge

Radar sensors have a new growing application area of dynamic hand gesture recognition. Traditionally radar systems are considered to be very large, complex and focused on detecting targets at long ranges. With modern electronics and signal processing it is now possible to create small compact RF sensors that can sense subtle movements over short ranges. For such applications, access to comprehensive databases of signatures is critical to enable the effective training of classification algorithms and to provide a common baseline for benchmarking purposes. This Letter introduces the Dop-NET radar micro-Doppler database and data challenge to the radar and machine learning communities. Dop-NET is a database of radar micro-Doppler signatures that are shareable and distributed with the purpose of improving micro-Doppler classification techniques. A continuous wave 24 GHz radar module is used to capture the first contributions to the Dop-NET database and classification results based on discriminating these hand gestures as shown.

Micro-Doppler based target classification in ground surveillance radar systems

In this paper, we investigate effect of different Time-Frequency (TF) analysis of micro-Doppler signature for robust and efficient automatic target classification in next-generation radar systems. Indeed, we provide a novel mathematical baseband model for the non-stationary reflected signal of the targets and explore how Short Time Fourier Transform (STFT) and General Linear Chirplet Transform (GLCT) are useful in categorization of different ground moving targets, including walking person, animal, light vehicles and drone. To simulate the micro-Doppler signature, we will introduce a simple, but sufficiently realistic signal model. Simulation and results which are provided for various scenarios, illustrate the importance of appropriate TF representation in target classification.

Quantitative Evaluation of Channel Micro-Doppler Capacity for MIMO UWB Radar Human Activity Signals Based on Time–Frequency Signatures

A novel quantitative method to evaluate channel micro-Doppler capacity of multiple-input and multiple-output (MIMO) system is proposed here. The method is valid for ultrawideband (UWB) MIMO radar human activity systems based on time–frequency signatures, and the quality measure will be noted as a relative signal to noise ratio (RSNR). The method quantifies these signatures and evaluates the relative superiority or inferiority of these MIMO channels. Examples of micro-Doppler signature ( $\mu $ Ds) characteristics of human activities in a channel will be considered and compared to that of all other channels. First, the MIMO UWB radar human activity signal is modeled, and its corresponding time–frequency ( $T$ – $F$ ) characteristics are analyzed to justify the rationality of using the new RSNR metric. Second, the method is evaluated using experimental data and the capability of distinguishing the $\mu \text{D}$ capacity differences among channels is demonstrated. This new method clearly and accurately shows much better visible $\mu \text{D}$ evaluation performance than that of the conventional signal to noise ratio in time domain ( ${\mathrm {SNR}}_{t}$ ). Moreover, this evaluation method can still work well, even for signals with low signal to noise ratio (SNR) down to −4-dB level. Therefore, it can be successfully used to select the superior channels and eliminate any inferior channels or provide confidence coefficients for the collected multiple channel data of human activities. This method should lead to a significant reduction of the inferior channels’ influence on further MIMO-based classification or imaging of human activities.