Mm-wave Radar Research Articles

BmmW: A DNN-based joint BLE and mmWave radar system for accurate 3D localization with goal-oriented communication

Bluetooth Low Energy (BLE) has emerged as one of the reference technologies for the development of indoor localization systems, due to its increasing ubiquity, low-cost hardware, and to the introduction of direction-finding enhancements improving its ranging performance. However, the intrinsic narrowband nature of BLE makes this technology susceptible to multipath and channel interference. As a result, it is still challenging to achieve decimetre-level localization accuracy, which is necessary when developing location-based services for smart factories and workspaces. To address this challenge, we present BmmW, an indoor localization system that augments the ranging estimates obtained with BLE5.1’s constant tone extension feature with mmWave radar measurements to provide 3D localization of a mobile tag with decimetre-level accuracy. Specifically, BmmW embeds a deep neural network (DNN) that is jointly trained with both BLE and mmWave measurements, practically leveraging the strengths of both technologies. In fact, mmWave radars can locate objects and people with decimetre-level accuracy, but their effectiveness in monitoring stationary targets and multiple objects is limited, and they also suffer from a fast signal attenuation limiting the usable range to a few meters. We evaluate BmmW’s performance experimentally, and show that its joint DNN training scheme allows to track mobile tags with a mean 3D localization accuracy of 10 cm when combining angle-of-arrival BLE measurements with mmWave radar data. We further assess two variations of BmmW: BmmW-Lite and BmmW-Lite+, both tailored for single-antenna BLE devices, which eliminates the necessity for bulky and expensive multi-antenna arrays and represents a cost-effective solution that is easy to integrate into compact IoT devices. In contrast to classic BmmW (which utilizes angle-of-arrival info), BmmW-Lite uses raw in-phase/quadrature (I/Q) measurements, and achieves a mean localization accuracy of 36 cm, thus facilitating precise object tracking in indoor environments even when using budget-friendly single-antenna BLE devices. BmmW-Lite+ extends BmmW-Lite by allowing the localization task to be transferred from the edge to the cloud due to device memory and power constraints. To this end, BmmW-Lite+ employs a goal-oriented communication paradigm that compresses initial BLE features into a more compact semantic format at the edge device, which allows to minimize the amount of data that needs to be sent to the cloud. Our experimental results show that BmmW-Lite+ can compress raw BLE features by up to 12% of their initial size (hence allowing to save network bandwidth and minimize energy consumption), with negligible impact on the localization accuracy.

VirTeach: mmWave Radar Point-Cloud-Based Pose Estimation With Virtual Data as a Teacher

VirTeach: mmWave Radar Point-Cloud-Based Pose Estimation With Virtual Data as a Teacher

Simultaneous Authentication of Multiple Users Using a Single mmWave Radar

Simultaneous Authentication of Multiple Users Using a Single mmWave Radar

Beamforming for Sensing: Hybrid Beamforming based on Transmitter-Receiver Collaboration for Millimeter-Wave Sensing

Previous mmWave sensing solutions assumed good signal quality. Ensuring an unblocked or strengthened LoS path is challenging. Therefore, finding an NLoS path is crucial to enhancing perceived signal quality. This paper proposes Trebsen, a Transmitter-REceiver collaboration-based Beamforming scheme SENsing using commercial mmWave radars. Specifically, we define the hybrid beamforming problem as an optimization challenge involving beamforming angle search based on transmitter-receiver collaboration. We derive a comprehensive expression for parameter optimization by modeling the signal attenuation variations resulting from the propagation path. To comprehensively assess the perception signal quality, we design a novel metric perceived signal-to-interference-plus-noise ratio (PSINR), combining the carrier signal and baseband signal to quantify the fine-grained sensing motion signal quality. Considering the high time cost of traversing or randomly searching methods, we employ a search method based on deep reinforcement learning to quickly explore optimal beamforming angles at both transmitter and receiver. We implement Trebsen and evaluate its performance in a fine-grained sensing application (i.e., heartbeat). Experimental results show that Trebsen significantly enhances heartbeat sensing performance in blocked or misaligned LoS scenes. Comparing non-beamforming, Trebsen demonstrates a reduction of 23.6% in HR error and 27.47% in IBI error. Moreover, comparing random search, Trebsen exhibits a 90% increase in search speed.

Advancing Cycling Safety: On-Bike Alert System Utilizing Multi-Layer Radar Point Cloud Clustering for Coarse Object Classification.

Cyclists are considered to be vulnerable road users (VRUs) and need protection from potential collisions with cars and other vehicles induced by unsafe driving, dangerous road conditions, or weak cycling infrastructure. Integrating mmWave radars into cycling safety measures presents an efficient solution to this problem given their compact size, low power consumption, and low cost compared to other sensors. This paper introduces an mmWave radar-based bike safety system designed to offer real-time alerts to cyclists. The system consists of a low-power radar sensor affixed to the bicycle, connected to a micro-controller, and delivering a preliminary classification of detected obstacles. An efficient two-level clustering based on the accumulation of radar point clouds from multiple frames with a temporal projection from previous frames into the current frame is proposed. The clustering is followed by a coarse classification algorithm in which we use relevant features extracted from the resulting clusters. An annotated RadBike dataset composed of radar point cloud data synchronized with RGB camera images is developed to evaluate our system. The two-level clustering outperforms the DBSCAN algorithm, achieving a v-measure score of 0.91, compared to 0.88 with classical DBSCAN. Different classifiers, including decision trees, random forests, support vector machines (SVMs), and AdaBoost, have been assessed, with an overall accuracy of 87% for the three main object classes: four-wheeled, two-wheeled, and others. The system has the ability to improve rider safety on the road and substantially reduce the frequency of incidents involving cyclists.

End-to-End Target Liveness Detection via mmWave Radar and Vision Fusion for Autonomous Vehicles

The successful operation of autonomous vehicles hinges on their ability to accurately identify objects in their vicinity, particularly living targets such as bikers and pedestrians. However, visual interference inherent in real-world environments, such as omnipresent billboards, poses substantial challenges to extant vision-based detection technologies. These visual interference exhibit similar visual attributes to living targets, leading to erroneous identification. We address this problem by harnessing the capabilities of mmWave radar, a vital sensor in autonomous vehicles, in combination with vision technology, thereby contributing a unique solution for liveness target detection. We propose a methodology that extracts features from the mmWave radar signal to achieve end-to-end liveness target detection by integrating the mmWave radar and vision technology. This proposed methodology is implemented and evaluated on the commodity mmWave radar IWR6843ISK-ODS and vision sensor Logitech camera. Our extensive evaluation reveals that the proposed method accomplishes liveness target detection with a mean average precision of 98.1%, surpassing the performance of existing studies.

HomeOSD: Appliance Operating-Status Detection Using mmWave Radar.

Within the context of a smart home, detecting the operating status of appliances in the environment plays a pivotal role, estimating power consumption, issuing overuse reminders, and identifying faults. The traditional contact-based approaches require equipment updates such as incorporating smart sockets or high-precision electric meters. Non-constant approaches involve the use of technologies like laser and Ultra-Wideband (UWB) radar. The former can only monitor one appliance at a time, and the latter is unable to detect appliances with extremely tiny vibrations and tends to be susceptible to interference from human activities. To address these challenges, we introduce HomeOSD, an advanced appliance status-detection system that uses mmWave radar. This innovative solution simultaneously tracks multiple appliances without human activity interference by measuring their extremely tiny vibrations. To reduce interference from other moving objects, like people, we introduce a Vibration-Intensity Metric based on periodic signal characteristics. We present the Adaptive Weighted Minimum Distance Classifier (AWMDC) to counteract appliance vibration fluctuations. Finally, we develop a system using a common mmWave radar and carry out real-world experiments to evaluate HomeOSD's performance. The detection accuracy is 95.58%, and the promising results demonstrate the feasibility and reliability of our proposed system.

Vital Sign Monitoring in Dynamic Environment via mmWave Radar and Camera Fusion

Contact-free vital sign monitoring, which uses wireless signals for recognizing human vital signs (i.e, breath and heartbeat), is an attractive solution to health and security. However, the subject's body movement and the change in actual environments can result in inaccurate frequency estimation of heartbeat and respiratory. In this paper, we propose a robust mmWave radar and camera fusion system for monitoring vital signs, which can perform consistently well in dynamic scenarios, e.g., when some people move around the subject to be tracked, or a subject waves his/her arms and marches on the spot. Three major processing modules are developed in the system, to enable robust sensing. Firstly, we utilize a camera to assist a mmWave radar to accurately localize the subjects of interest. Secondly, we exploit the calculated subject position to form transmitting and receiving beamformers, which can improve the reflected power from the targets and weaken the impact of dynamic interference. Thirdly, we propose a weighted multi-channel Variational Mode Decomposition (WMC-VMD) algorithm to separate the weak vital sign signals from the dynamic ones due to subject's body movement. Experimental results show that, the 90 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${^{th}}$</tex-math></inline-formula> percentile errors in respiration rate (RR) and heartbeat rate (HR) are less than 0.5 RPM (respirations per minute) and 6 BPM (beats per minute), respectively.

A mmWave Radar SLAM Method in Subterranean Tunnel for Low Visibility and Degradation

A mmWave Radar SLAM Method in Subterranean Tunnel for Low Visibility and Degradation

Smart Streetlight Energy Saving System Based on mmWave Radar

Streetlights serve as fundamental infrastructure to meet the lighting needs of people on every road. However, their extensive deployment often results in unnecessary energy waste, with many streetlights maintaining high brightness despite minimal usage during the night. This study aims to develop a smart energy-efficient streetlight system that automatically adjusts lighting levels based on the absence of vehicles and pedestrians, detected after a 3-minute countdown. Specifically, the study utilizes mmWave radar to collect point cloud data, which undergoes denoising through Doppler, DBSCAN, and XYZ techniques. Additionally, the mmWave radar assists in training an LSTM model to identify pedestrian pathways. The implementation of the proposed system significantly reduces energy consumption and annual costs by automatically dimming or turning off streetlights in areas with minimal pedestrian activity during nighttime.

HUMAN ACTIVITY DETECTION USING MMWAVE SENSOR

Human Activity Recognition (HAR) is becoming increasingly useful for applications such as well-being monitoring and personalizing smart spaces. Traditional methods for HAR often require wearable devices or cameras. The former is not feasible for every environment and the latter has strong privacy concerns. The mmWave radar has been shown to be a promising alternative as it does not imply the same privacy concerns and does not require the users to have wearable devices. In this paper we have used a low- cost mmWave radar to generate micro-Doppler spectrograms to ultimately classify different activities. For this, multiple classifiers and methods of spectrogram filtering have been examined. Finally, a Time-Distributed Convolutional Neural Network in conjunction with a Bi-Directional Long Short-Term Memory has attained an average accuracy of 99.62% on a dataset of 5 activities, involving 2 participants.

Characterization of Automotive Paint Permittivity for mmWave Radar Applications

Characterization of Automotive Paint Permittivity for mmWave Radar Applications

MmDetect: YOLO-Based Processing of mm-Wave Radar Data for Detecting Moving People

mmDetect: YOLO-Based Processing of mm-Wave Radar Data for Detecting Moving People

4D mmWave Radar for Autonomous Driving Perception: A Comprehensive Survey

4D mmWave Radar for Autonomous Driving Perception: A Comprehensive Survey

Mm-CasGAN: A cascaded adversarial neural framework for mmWave radar point cloud enhancement

Handling and interpreting sparse 3D point clouds, especially from mmWave radar, presents unique challenges due to the inherent data sparsity and the vast domain difference compared to denser point clouds like those from LiDAR. In this paper, we introduce a novel cascaded generative adversarial network (GAN) approach to bridge this domain gap. The core principle is to progressively refine the radar-based point cloud through a series of GANs, each targeting a higher resolution. By leveraging multi-level features and a hybrid loss function that combines adversarial, geometric, and consistency components, our method ensures a smooth transition from the sparse radar representation to a high-resolution LiDAR-like point cloud. Our cascaded approach operates at a patch level, and the integrated loss function ensures that the generated points not only resemble the target domain but also maintain geometric and structural fidelity. Real-life dataset consisting mostly of moving pedestrians were collected using a system made of Radar, LiDAR, and RGB Camera. Through an extensive experiment on the collected real-world pedestrian dataset, we validate the efficacy of our approach. Inference from the network indicates that our method can upsample mmWave radar point clouds with enhanced density, uniformity, and closer alignment to the ground truth LiDAR point clouds, which is the first of its kind network to do so.

Human and Small Animal Detection Using Multiple Millimeter-Wave Radars and Data Fusion: Enabling Safe Applications.

Millimeter-wave (mmWave) radars attain high resolution without compromising privacy while being unaffected by environmental factors such as rain, dust, and fog. This study explores the challenges of using mmWave radars for the simultaneous detection of people and small animals, a critical concern in applications like indoor wireless energy transfer systems. This work proposes innovative methodologies for enhancing detection accuracy and overcoming the inherent difficulties posed by differences in target size and volume. In particular, we explore two distinct positioning scenarios that involve up to four mmWave radars in an indoor environment to detect and track both humans and small animals. We compare the outcomes achieved through the implementation of three distinct data-fusion methods. It was shown that using a single radar without the application of a tracking algorithm resulted in a sensitivity of 46.1%. However, this sensitivity significantly increased to 97.10% upon utilizing four radars using with the optimal fusion method and tracking. This improvement highlights the effectiveness of employing multiple radars together with data fusion techniques, significantly enhancing sensitivity and reliability in target detection.

MMHTSR: In-Air Handwriting Trajectory Sensing and Reconstruction Based on mmWave Radar

MMHTSR: In-Air Handwriting Trajectory Sensing and Reconstruction Based on mmWave Radar

Person Identification Method Based on PointNet++ and Adversarial Network for mmWave Radar

Person Identification Method Based on PointNet++ and Adversarial Network for mmWave Radar

TRANS-CNN-Based Gesture Recognition for mmWave Radar.

In order to improve the real-time performance of gesture recognition by a micro-Doppler map of mmWave radar, the point cloud based gesture recognition for mmWave radar is proposed in this paper. Two steps are carried out for mmWave radar-based gesture recognition. The first step is to estimate the point cloud of the gestures by 3D-FFT and the peak grouping. The second step is to train the TRANS-CNN model by combining the multi-head self-attention and the 1D-convolutional network so as to extract the features in the point cloud data at a deeper level to categorize the gestures. In the experiments, TI mmWave radar sensor IWR1642 is used as a benchmark to evaluate the feasibility of the proposed approach. The results show that the accuracy of the gesture recognition reaches 98.5%. In order to prove the effectiveness of our approach, a simply 2Tx2Rx radar sensor is developed in our lab, and the accuracy of recognition reaches 97.1%. The results show that our proposed gesture recognition approach achieves the best performance in real time with limited training data in comparison with the existing methods.

Towards mmWave Altimetry for UAS: Exploring the Potential of 77 GHz Automotive Radars

Precise altitude data are indispensable for flight navigation, particularly during the autonomous landing of unmanned aerial systems (UASs). Conventional light and barometric sensors employed for altitude estimation are limited by poor visibility and temperature conditions, respectively, whilst global positioning system (GPS) receivers provide the altitude from the mean sea level (MSL) marred with a slow update rate. To cater to the landing safety requirements, UASs necessitate precise altitude information above ground level (AGL) impervious to environmental conditions. Radar altimeters, a mainstay in commercial aviation for at least half a century, realize these requirements through minimum operational performance standards (MOPSs). More recently, the proliferation of 5G technology and interference with the universally allocated band for radar altimeters from 4.2 to 4.4 GHz underscores the necessity to explore novel avenues. Notably, there is no dedicated MOPS tailored for radar altimeters of UASs. To gauge the performance of a radar altimeter offering for UASs, existing MOPSs are the de facto choice. Historically, frequency-modulated continuous wave (FMCW) radars have been extensively used in a broad spectrum of ranging applications including radar altimeters. Modern monolithic millimeter wave (mmWave) automotive radars, albeit designed for automotive applications, also employ FMCW for precise ranging with a cost-effective and compact footprint. Given the technology maturation with excellent size, weight, and power (SWaP) metrics, there is a growing trend in industry and academia to explore their efficacy beyond the realm of the automotive industry. To this end, their feasibility for UAS altimetry remains largely untapped. While the literature on theoretical discourse is prevalent, a specific focus on mmWave radar altimetry is lacking. Moreover, clutter estimation with hardware specifications of a pure look-down mmWave radar is unreported. This article argues the applicability of MOPSs for commercial aviation for adaptation to a UAS use case. The theme of the work is a tutorial based on a simplified mathematical and theoretical discussion on the understanding of performance metrics and inherent intricacies. A systems engineering approach for deriving waveform specifications from operational requirements of a UAS is offered. Lastly, proposed future research directions and insights are included.