MMW Radar Research Articles

Towards a Low-Cost Solution for Gait Analysis Using Millimeter Wave Sensor and Machine Learning.

Human Activity Recognition (HAR) that includes gait analysis may be useful for various rehabilitation and telemonitoring applications. Current gait analysis methods, such as wearables or cameras, have privacy and operational constraints, especially when used with older adults. Millimeter-Wave (MMW) radar is a promising solution for gait applications because of its low-cost, better privacy, and resilience to ambient light and climate conditions. This paper presents a novel human gait analysis method that combines the micro-Doppler spectrogram and skeletal pose estimation using MMW radar for HAR. In our approach, we used the Texas Instruments IWR6843ISK-ODS MMW radar to obtain the micro-Doppler spectrogram and point clouds for 19 human joints. We developed a multilayer Convolutional Neural Network (CNN) to recognize and classify five different gait patterns with an accuracy of 95.7 to 98.8% using MMW radar data. During training of the CNN algorithm, we used the extracted 3D coordinates of 25 joints using the Kinect V2 sensor and compared them with the point clouds data to improve the estimation. Finally, we performed a real-time simulation to observe the point cloud behavior for different activities and validated our system against the ground truth values. The proposed method demonstrates the ability to distinguish between different human activities to obtain clinically relevant gait information.

Review on Millimeter-Wave Radar and Camera Fusion Technology

Cameras allow for highly accurate identification of targets. However, it is difficult to obtain spatial position and velocity information about a target by relying solely on images. The millimeter-wave radar (MMW radar) sensor itself easily acquires spatial position and velocity information of the target but cannot identify the shape of the target. MMW radar and camera, as two sensors with complementary strengths, have been heavily researched in intelligent transportation. This article examines and reviews domestic and international research techniques for the definition, process, and data correlation of MMW radar and camera fusion. This article describes the structure and hierarchy of MMW radar and camera fusion, it also presents its fusion process, including spatio-temporal alignment, sensor calibration, and data information correlation methods. The data fusion algorithms from MMW radar and camera are described separately from traditional fusion algorithms and deep learning based algorithms, and their advantages and disadvantages are briefly evaluated.

Adaptive Multi-Pedestrian Tracking by Multi-Sensor: Track-to-Track Fusion Using Monocular 3D Detection and MMW Radar

Accurate and reliable tracking of multi-pedestrian is of great importance for autonomous driving, human-robot interaction and video surveillance. Since different scenarios have different best-performing sensors, sensor fusion perception plans are believed to have complementary modalities and be capable of handling situations which are challenging for single sensor. In this paper, we propose a novel track-to-track fusion strategy for multi-pedestrian tracking by using a millimeter-wave (MMW) radar and a monocular camera. Pedestrians are firstly tracked by each sensor according to the sensor characteristic. Specifically, the 3D monocular pedestrian detections are obtained by a convolutional neural network (CNN). The trajectory is formed by the tracking-by-detection approach, combined with Bayesian estimation. The measurement noise of the 3D monocular detection is modeled by a detection uncertainty value obtained from the same CNN, as an approach to estimate the pedestrian state more accurately. The MMW radar utilizes the track-before-detection method due to the sparseness of the radar features. Afterwards, the pedestrian trajectories are obtained by the proposed track-to-track fusion strategy, which can work adaptively under challenging weather conditions, low-illumination conditions and clutter scenarios. A group of tests are carried out to validate our pedestrian tracking strategy. Tracking trajectories and optimal sub-pattern assignment (OSPA) metric demonstrate the accuracy and robustness of the proposed multi-sensor multi-pedestrian tracking system.

Learning-Based Split Unfolding Framework for 3-D mmW Radar Sparse Imaging

The application of the compressed sensing (CS) method in the radar field enables the radar imaging system to satisfy both low data cost and high reconstruction quality, however, it is accompanied by enormous iterative operations and difficult adjustments of parameters. In this paper, we propose a learning-based split unfolding framework, dubbed as split iterative sparse reconstruction network (SISR-Net), for near-field 3-D millimeter-wave (mmW) radar sparse imaging. Firstly, a sparse reconstruction algorithm, i.e., SISRA, is proposed to theoretically guide the structure of the imaging framework. Subsequently, by combining the model-based CS method and data-driven deep learning method, SISR-Net is constructed by SISRA to produce 3-D mmW radar images efficiently with excellent explainability and generalization ability. Joint the radar-imaging kernel, echo-generation kernel, and the split Bregman method, the efficiency and stability of SISR-Net are guaranteed, all parameters are layer-varied and learned steadily by end-to-end training to improve the convergence and robustness of the imaging network. Simulated data and the echo from a high-resolution mmW radar dataset 3DRIED, are used to train and test the SISR-Net based on the Adam optimizer. For both simulation and extensive 3-D mmW radar measured experiments, the proposed SISR-Net outperforms other state-of-the-art imaging methods in terms of imaging accuracy and generalization ability.

Wave Height Estimation Based on the Phase Time Series of Millimeter-Wave Radar

Estimating ocean wave height is vital for coastal activities. Herein, we propose a method for the estimation of wave height using a phase time series of the frequency-modulated continuous-wave (FMCW) millimeter-wave (MMW) radar. We simulated a one-dimensional wavy ocean surface under different wind speeds and the electromagnetic echo of MMW radar on the ocean surface. The simulation results show that the unwrapping phase time series-based wave heights are highly consistent with the actual wave heights. This shows that the phase time series of the FMCW MMW radar is effective for estimating wave height, and the FMCW MMW radar can be used to monitor ocean states.

Lightweight FISTA-Inspired Sparse Reconstruction Network for mmW 3-D Holography

Integrating compressed sensing (CS) with millimeter-wave (mmW) holography has shown great potential to achieve lightweight onboard hardware, low sampling ratio, and high-speed sensing. However, conventional CS-driven algorithms are always limited by nontrivial adjusting of parameters and excessive computational cost caused by plenty of iterations. To address this problem, we propose a lightweight model-based deep learning framework (LFIST-Net) for mmW 3-D holography, by combining the interpretability of fast iterative shrinkage-thresholding algorithm (FISTA) and tuning-free merit of data-driven deep neural network. First, the single-frequency (SF) holographic imaging technique is integrated into FISTA, which serves as the sensing kernels, to avoid large-scale matrix multiplications. Subsequently, the kernel-based FISTA (KFISTA) is mapped into layer-fixed and parameter-learnable LFIST-Net, whose weights are relaxed to be layer-varied. The updating of key parameters in LFIST-Net, including step sizes, thresholds, and momentum coefficients, are regularized by soft-plus function to ensure the non-negativity and monotonicity. As for 3-D holography implementation, the “1-D + 2-D” scheme is adopted, where the matched filtering (MF) and well-trained LFIST-Net are used for range focusing and reconstructions of azimuth slices. Without losing efficiency, the range-focused subechoes are processed parallelly in 3-D cube form. Experiments, including both simulated and measured tests based on a commercial mmW radar, prove that LFIST-Net is capable of reconstructing the imaging scene precisely. In particular, in near-field mmW 3-D holography tests, both numerical and visual results demonstrate LFIST-Net yields compelling reconstruction performance while maintaining high computational speed compared with MF-based, conventional CS-driven, and network-based methods.

Person Identification With Millimeter-Wave Radar in Realistic Smart Home Scenarios

Compared with visual sensors that have light dependence and privacy intrusion issues, non-intrusion millimeter-wave (mmW) radars are more suitable for the daily person identification. In a realistic home scenario, there are new challenges that are not taken into account in the existing research. This letter attempts to address these issues such as multipath interference, complex walking process, and recognition robustness in smart home scenarios and designs a lightweight multi-branch convolutional neural network (CNN) with an Inception-Pool module and a Residual-Pool module to learn and classify gait Doppler features. The experimental results in a home living room scenario indicate that the designed mmW radar person identification system can achieve accurate and robust real-time identification performance.

Continuous blood pressure monitoring with MMW radar sensor

Continuous blood pressure monitoring with MMW radar sensor

3DRIED: A High-Resolution 3-D Millimeter-Wave Radar Dataset Dedicated to Imaging and Evaluation

Millimeter-wave (MMW) 3-D imaging technology is becoming a research hotspot in the field of safety inspection, intelligent driving, etc., due to its all-day, all-weather, high-resolution and non-destruction feature. Unfortunately, due to the lack of a complete 3-D MMW radar dataset, many urgent theories and algorithms (e.g., imaging, detection, classification, clustering, filtering, and others) cannot be fully verified. To solve this problem, this paper develops an MMW 3-D imaging system and releases a high-resolution 3-D MMW radar dataset for imaging and evaluation, named as 3DRIED. The dataset contains two different types of data patterns, which are the raw echo data and the imaging results, respectively, wherein 81 high-quality raw echo data are presented mainly for near-field safety inspection. These targets cover dangerous metal objects such as knives and guns. Free environments and concealed environments are considered in experiments. Visualization results are presented with corresponding 2-D and 3-D images; the pixels of the 3-D images are 512×512×6. In particular, the presented 3DRIED is generated by the W-band MMW radar with a center frequency of 79GHz, and the theoretical 3-D resolution reaches 2.8 mm × 2.8 mm × 3.75 cm. Notably, 3DRIED has 5 advantages: (1) 3-D raw data and imaging results; (2) high-resolution; (3) different targets; (4) applicability for evaluation and analysis of different post processing. Moreover, the numerical evaluation of high-resolution images with different types of 3-D imaging algorithms, such as range migration algorithm (RMA), compressed sensing algorithm (CSA) and deep neural networks, can be used as baselines. Experimental results reveal that the dataset can be utilized to verify and evaluate the aforementioned algorithms, demonstrating the benefits of the proposed dataset.

Machine Learning-Based Target Classification for MMW Radar in Autonomous Driving

Millimeter-wave (MMW) radar sensors are considered key components of autonomous vehicles. Because of the performance degeneration of cameras and lidars under inclement weather conditions, robust autonomy must rely on radar sensors to perform target detection and classification. Unlike basic target classifier methods in literature that make use of target velocity, the proposed approach is far more comprehensive and can be applied to targets with zero-Doppler. Depending on the radar type and target range, this paper presents four target classification models based on four different types of radar data: statistical RCS, distributed (time-domain) RCS, range-azimuth angle radar images and 3D radar images. The classification models are implemented by machine learning approaches artificial neural network (ANN) and convolutional neural network (CNN) with a comprehensive simulated dataset. Good classification accuracies are demonstrated, and the proposed model is validated with measured data. Different radar target classification approaches are compared, which clearly reveals the trade-off between classification performance and system complexity. The proposed radar target classification methods can be applied effectively to both static and dynamic targets, at near or far ranges, using traditional or imaging radars, resulting in improved safety for autonomous vehicles in a wide variety of complex environments.

Wideband Millimeter-Wave Flat Chaos Generation With Controllable Power Spectrum Using Optical Time Lens

We propose and numerically demonstrate the generation of wideband millimeter-wave (mmW) flat chaos with controllable power spectrum by injection of chaotic signal from external cavity semiconductor laser (ECSL) into optical time lens with noise phase modulation. Simulation results indicate consistent elimination of the ECSL relaxation oscillation frequency domination over the RF spectrum for large scale parameters of the optical time lens module and a wideband flat chaos, whose efficient bandwidth rapidly increases with the bandwidth of the noise signal driving the phase modulator and phase modulation index. Besides, we show that the time delay signature suppression can be concurrently achieved for moderate values of the noise bandwidth and phase modulation index. The proposed wideband mmW flat chaos exhibits great potential applications for ultrahigh-speed chaos communications, mmW radars and macroscopic mmW noise source required for mmW research and design.

A Local Environment Model Based on Multi-Sensor Perception for Intelligent Vehicles

Accurate perception of the driving environment is a key technology for intelligent vehicles. Given some critical problems such as low robustness, low detection precision, difficulty in actual deployment, we propose a local environment model (LEM) based on multi-sensor fusion technology through Lidar, millimeter-wave (MMW) radar, camera, and ultrasonic radar. The local environment model mainly consists of the drivable area and the dynamic target list. The drivable area is extracted by the ground gradient threshold algorithm. Based on it, we propose an effective trim algorithm to make the drivable area model more practical. Furthermore, low-cost ultrasonic radars are deployed to compensate for the blind area of Lidar. The dynamic target list is established by local tracking and global tracking in the forward area. Kalman filter and converted measurement Kalman filter (CMKF) are adopted in the local tracking of Lidar, camera, and MMW radar. In the global tracking, the global nearest neighbor (GNN) algorithm is used for data association and the optimal distributed estimation fusion (ODEF) algorithm is used for sensor fusion. To improve the robustness of tracking, we use an assignment method to better exploit sensor performance. Finally, the vehicle experiment is carried out in the campus environment. Experimental results indicate that the proposed algorithm can avoid the false detection of the drivable area and realize real-time multi-target dynamic tracking. Therefore, the robustness and accuracy of the local environment model is verified.

A 2-D Radon Transformation for Enhancing the Detection and Imaging of Embedded Defects in Layered Composite Structures Using Millimeter-Wave System

This paper aims to tackle the challenge of detecting the presence of embedded defects in layered composite structures using imaging techniques including a 2D Radon transformation. The proposed technique aims at enhancing the detection and imaging of embedded defects in layered composite structures that is captured using a millimeterwave (MMW) radar system. A 94 GHz MMW system is used in the near field to detect defects such as disbond and delamination embedded in complex glass fiber composite structures. In order to simplify the detection in glass fibers, measurement data attained using a MMW radar are plotted as images using MATLAB. A stable detection and identification of these defects from the collected images is proposed using a robust two-stage algorithm. The Weiner filter-based deconvolution technique is used for the reflectivity deconstruction in the image and a two-dimensional Radon transformation called back-projection is used to as an additional filtering step enhance the detection of the embedded defects. The proposed method allows defects that are embedded 24mm below the surface of the scanning probe to be detected and identified in terms of location. Our results show potential in the proposed method to be employed in accurately imaging defects in composites that are used in the aerospace and civil infrastructure industries.

Detection of Low RCS Supersonic Flying Targets with a High-Resolution MMW Radar.

In this study, the detection of a low radar cross-section (RCS) target moving at a very high speed using a high-resolution millimeter-wave radar is presented. This real-time detection is based on the transmission of a continuous wave and heterodyning of the received signal reflected from the moving target. This type of detection enables one to extract the object’s movement characteristics, such as velocity and position, while in motion and also to extract its physical characteristics. In this paper, we describe the detection of a fired bullet using a radar operating at an extremely high-frequency band. This allowed us to employ a low sampling rate which enabled the use of inexpensive and straightforward equipment, including the use of small antennas that allow velocity detection at high resolution and with low atmospheric absorption.

Millimeter-wave radar image analysis for the traffic sensing

Millimeter-wave(MMW) radar sensing is one of the most promising technologies to provide safe navigation for autonomous vehicles due to its expected high-resolution imaging capability However, driverless cars have higher request for different environment and light conditions. Therefore, millimetre-wave imaging is of paramount importance for complex load scenario. In this paper, we have built models of pavement pits and bulges and analysed their with differences ways of antennas. A comparison of the imaging performance of experimental systems operating at a MMW radar and a Lidar is presented with the analysis of features for initial image interpretation Experimental images of the complex road surface are made by a 94GHz frequency-modulated continuous-wave (FMCW) radar technique with 3mm wavelength.

An Anti-FOD Method Based on CA-CM-CFAR for MMW Radar in Complex Clutter Background.

Traditional constant false alarm rate (CFAR) methods have shown their potential for foreign object debris (FOD) indication. However, the performance of these methods would deteriorate under the complex clutter background in airport scenes. This paper presents a threshold-improved approach based on the cell-averaging clutter-map (CA-CM-) CFAR and tests it on a millimeter-wave (MMW) radar system. Clutter cases are first classified with variability indexes (VIs). In homogeneous background, the threshold is calculated by the student-t-distributed test statistic; under the discontinuous clutter conditions, the threshold is modified according to current VI conditions, in order to address the performance decrease caused by extended clutter edges. Experimental results verify that the chosen targets can be indicated by the t-distributed threshold in homogeneous background. Moreover, effective detection of the obscured targets could also be achieved with significant detectability improvement at extended clutter edges.

Visibility Detection Based on the Recognition of the Preceding Vehicle’s Taillight Signals

This paper proposes a method for visibility detection based on the recognition of the preceding vehicle’s taillight signals using in-vehicle camera and millimeter-wave (mm-W) radar. First, we design two methods of vehicle identification. One is to use Haar-like features and an AdaBoost algorithm to train the vehicle classifier, which is mainly used to identify vehicles without turning on the taillights. The other is to identify vehicles with taillights on by means of taillight segmentation. The two identification methods are combined with a Discriminative Scale Space Tracker (DSST) to track the vehicle in the image acquired by vehicle camera and to measure anthropic visibility with mm-W radar. In addition, we drove a test vehicle on a foggy highway and collected experimental data through in-vehicle camera and mm-W radar. The experimenter observed the movement of the vehicle in front until it disappeared from the field of vision and recorded the distance of the vehicle in front measured by radar at that time as human visibility, which was also used as the ground truth to verify the accuracy of the proposed visibility detection method. The experimental results show that the visibility measured by the proposed algorithm is essentially consistent with the visibility obtained by human eyes, that is, the visibility of vehicles with no taillights, clearance lamp, emergency flasher, or fog lamp tends to rise, with an average accuracy of 88%, 91%, 90%, and 95%, respectively. In contrast to the traditional visibility measurement, this method mainly measures the maximum distance that the driver can observe when the front vehicle is not turned on or different taillights are turned on.

Enhanced Multi-Channel Feature Synthesis for Hand Gesture Recognition Based on CNN With a Channel and Spatial Attention Mechanism

Millimeter-wave (MMW) radar hand gesture recognition technology is becoming important in many electronic device control applications. Currently, most existing approaches utilize the radical and micro-Doppler features from single-channel MMW radar, which ignores the different importance of the information contained in the micro-Doppler feature background or target areas. In this paper, we propose an algorithm for hand gesture recognition jointly using multi-channel signatures. The algorithm blends the information of both micro-Doppler features and instantaneous angles (azimuth and elevation) to accomplish hand gesture recognition performed with the convolutional neural network (CNN). To have a better features fusion and make CNN focus on the most important target signal regions and suppress the unnecessary noise areas, we apply the channel and spatial attention-based feature refinement modules. We also employ gesture movement mechanism-based data augmentation for more effective training to alleviate potential overfitting. Extensive experiments demonstrate the effectiveness and superiorities of the proposed algorithm. This method achieves a correct classification rate of 96.61%, approximately 5% higher than that of the single-channel-based recognition strategy as measured based on MMW radar datasets.

Developing an On-Road Object Detection System Using Monovision and Radar Fusion

In this study, a millimeter-wave (MMW) radar and an onboard camera are used to develop a sensor fusion algorithm for a forward collision warning system. This study proposed integrating an MMW radar and camera to compensate for the deficiencies caused by relying on a single sensor and to improve frontal object detection rates. Density-based spatial clustering of applications with noise and particle filter algorithms are used in the radar-based object detection system to remove non-object noise and track the target object. Meanwhile, the two-stage vision recognition system can detect and recognize the objects in front of a vehicle. The detected objects include pedestrians, motorcycles, and cars. The spatial alignment uses a radial basis function neural network to learn the conversion relationship between the distance information of the MMW radar and the coordinate information in the image. Then a neural network is utilized for object matching. The sensor with a higher confidence index is selected as the system output. Finally, three kinds of scenario conditions (daytime, nighttime, and rainy-day) were designed to test the performance of the proposed method. The detection rates and the false alarm rates of proposed system were approximately 90.5% and 0.6%, respectively.

Research on Target Detection Based on Distributed Track Fusion for Intelligent Vehicles.

Accurate target detection is the basis of normal driving for intelligent vehicles. However, the sensors currently used for target detection have types of defects at the perception level, which can be compensated by sensor fusion technology. In this paper, the application of sensor fusion technology in intelligent vehicle target detection is studied with a millimeter-wave (MMW) radar and a camera. The target level fusion hierarchy is adopted, and the fusion algorithm is divided into two tracking processing modules and one fusion center module based on the distributed structure. The measurement information output by two sensors enters the tracking processing module, and after processing by a multi-target tracking algorithm, the local tracks are generated and transmitted to the fusion center module. In the fusion center module, a two-level association structure is designed based on regional collision association and weighted track association. The association between two sensors’ local tracks is completed, and a non-reset federated filter is used to estimate the state of the fusion tracks. The experimental results indicate that the proposed algorithm can complete a tracks association between the MMW radar and camera, and the fusion track state estimation method has an excellent performance.