Mm-wave Sensor Research Articles

Improving the Accuracy of mmWave Radar for Ethical Patient Monitoring in Mental Health Settings.

Monitoring patient safety in high-risk mental health environments is a challenge for clinical staff. There has been a recent increase in the adoption of contactless sensing solutions for remote patient monitoring. mmWave radar is a technology that has high potential in this field due it its low cost and protection of privacy; however, it is prone to multipath reflections and other sources of environmental noise. This paper discusses some of the challenges in mmWave remote sensing applications for patient safety in mental health wards. In line with these challenges, we propose a novel low-data solution to mitigate the impact of multipath reflections and other sources of noise in mmWave sensing. Our solution uses an unscented Kalman filter for target tracking over time and analyses features of movement to determine whether targets are human or not. We chose a commercial off-the-shelf radar and compared the accuracy and reliability of sensor measurements before and after applying our solution. Our results show a marked decrease in false positives and false negatives during human target tracking, as well as an improvement in spatial location detection in a two-dimensional space. These improvements demonstrate how a simple low-data solution can improve existing mmWave sensors, making them more suitable for patient safety solutions in high-risk environments.

A mmWave Sensor and Camera Fusion System for Indoor Occupancy Detection and Tracking

A mmWave Sensor and Camera Fusion System for Indoor Occupancy Detection and Tracking

Intelligent Fall Detection for Elders

In today's fast-paced world, many elderly individuals live alone, increasing their risk of falls without immediate help. To address this, we propose a system with wall-mounted cameras and mmWave sensors in the home to detect falls quickly and accurately. When a fall is detected, the system alerts medical services and caregivers for prompt assistance. All fall-related data is securely stored on a website, accessible to medical professionals and family members for real-time monitoring. This system ensures the safety of the elderly and provides peace of mind for their loved ones by bridging technology and caregiving. Key Word: Elderly individuals, wall-mounted cameras, mmWave sensors, falls, secure website, monitoring, alert.

Design and Implementation of Nursing-Secure-Care System with mmWave Radar by YOLO-v4 Computing Methods

In computer vision and image processing, the shift from traditional cameras to emerging sensing tools, such as gesture recognition and object detection, addresses privacy concerns. This study navigates the Integrated Sensing and Communication (ISAC) era, using millimeter-wave signals as radar via a Convolutional Neural Network (CNN) model for event sensing. Our focus is on leveraging deep learning to detect security-critical gestures, converting millimeter-wave parameters into point cloud images, and enhancing recognition accuracy. CNNs present complexity challenges in deep learning. To address this, we developed flexible quantization methods, simplifying You Only Look Once (YOLO)-v4 operations with an 8-bit fixed-point number representation. Cross-simulation validation showed that CPU-based quantization improves speed by 300% with minimal accuracy loss, even doubling the YOLO-tiny model’s speed in a GPU environment. We established a Raspberry Pi 4-based system, combining simplified deep learning with Message Queuing Telemetry Transport (MQTT) Internet of Things (IoT) technology for nursing care. Our quantification method significantly boosted identification speed by nearly 2.9 times, enabling millimeter-wave sensing in embedded systems. Additionally, we implemented hardware-based quantization, directly quantifying data from images or weight files, leading to circuit synthesis and chip design. This work integrates AI with mmWave sensors in the domain of nursing security and hardware implementation to enhance recognition accuracy and computational efficiency. Employing millimeter-wave radar in medical institutions or homes offers a strong solution to privacy concerns compared to conventional cameras that capture and analyze the appearance of patients or residents.

Intelligent Millimeter-Wave System for Human Activity Monitoring for Telemedicine.

Telemedicine has the potential to improve access and delivery of healthcare to diverse and aging populations. Recent advances in technology allow for remote monitoring of physiological measures such as heart rate, oxygen saturation, blood glucose, and blood pressure. However, the ability to accurately detect falls and monitor physical activity remotely without invading privacy or remembering to wear a costly device remains an ongoing concern. Our proposed system utilizes a millimeter-wave (mmwave) radar sensor (IWR6843ISK-ODS) connected to an NVIDIA Jetson Nano board for continuous monitoring of human activity. We developed a PointNet neural network for real-time human activity monitoring that can provide activity data reports, tracking maps, and fall alerts. Using radar helps to safeguard patients' privacy by abstaining from recording camera images. We evaluated our system for real-time operation and achieved an inference accuracy of 99.5% when recognizing five types of activities: standing, walking, sitting, lying, and falling. Our system would facilitate the ability to detect falls and monitor physical activity in home and institutional settings to improve telemedicine by providing objective data for more timely and targeted interventions. This work demonstrates the potential of artificial intelligence algorithms and mmwave sensors for HAR.

Outage performance and energy efficiency optimization of wireless-powered millimeter-wave sensor networks

AbstractWith the widespread use of wireless sensor networks, one of the most pressing concerns is extending the lifetime of the sensors. By deploying directional antenna arrays, millimeter wave (mmWave) is a possible candidate for wireless energy transfer (WPT). This paper investigates a beneficial combination of WPT and data transmission in a typical mmWave sensor network with Rayleigh channels, where a transmission interval can be divided into two sub-intervals. During the first sub-interval, one hybrid access point (HAP) employs beamforming techniques to transfer energy for serving multiple sensors within the service sector. The sensors then transmit their individual signal in turn to the HAP based on time division multiple address (TDMA) strategy by using the whole harvested energy. According to stochastic geometry, the exact and approximate expressions of beam outage probability for the considered system are determined, respectively. The optimal time allocation of energy harvesting and data transmission for sensors is examined in order to maximize the energy efficiency of the system. The optimization problem can be translated into corresponding parametric form, and the resulting optimization problem can be solved using the Lagrange dual method with Karush–Kuhn–Tucker (KKT) conditions. The numerical results show the variation trend of the beam outage probability under various parameters and verify the accuracy of the theoretical analyses. Furthermore, the simulation results illustrate that the proposed optimal time allocation strategy can significantly enhance the overall energy efficiency of the system compared with a similar scheme.

Simultaneous Tracking and Recognizing Drone Targets with Millimeter-Wave Radar and Convolutional Neural Network

In this paper, a framework for simultaneous tracking and recognizing drone targets using a low-cost and small-sized millimeter-wave radar is presented. The radar collects the reflected signals of multiple targets in the field of view, including drone and non-drone targets. The analysis of the received signals allows multiple targets to be distinguished because of their different reflection patterns. The proposed framework consists of four processes: signal processing, cloud point clustering, target tracking, and target recognition. Signal processing translates the raw collected signals into spare cloud points. These points are merged into several clusters, each representing a single target in three-dimensional space. Target tracking estimates the new location of each detected target. A novel convolutional neural network model was designed to extract and recognize the features of drone and non-drone targets. For the performance evaluation, a dataset collected with an IWR6843ISK mmWave sensor by Texas Instruments was used for training and testing the convolutional neural network. The proposed recognition model achieved accuracies of 98.4% and 98.1% for one and two targets, respectively.

An Indoor People Counting and Tracking System using mmWave sensor and sub-sensors

The automatic detection and recognition system has many applications in implementing the smart city. Device-based and optical methods perform well in detecting and recognizing people, but several issues remain, such as device limitations, environmental limitations, and privacy concerns. To address these issues, we propose a real-time people counting and tracking system mainly based on a millimeter wave radar (mmWave) as an alternative to optical solutions in restaurants and retail. With the aid of sub-sensors, such as another mmWave radar or camera, this system also may feed the need for static people detection and security. It visualizes customer information, including trajectories and the number of people, helping the restaurant to prevent crowds during the pandemic of COVID-19 and analyzing customer visiting patterns for efficient management and marketing. The mmWave single-chip solution achieves an 83.5 % overall counting accuracy for walking people with a median position error of 0.17 m. The mmWave radar with sub-sensor cameras system approaches an overall detection accuracy of 93.1 %.

A Black Ice Detection Method Based on 1-Dimensional CNN Using mmWave Sensor Backscattering

Black ice on the road can be dangerous, as it renders the road slippery and is difficult to identify, owing to its transparency. Although studies on black ice detection using cameras, optical sensors, and infrared sensors have been conducted, these sensors have limitations, as they are affected by low light conditions and sunlight. To detect black ice regardless of low light conditions or sunlight, in this study, we incorporate a mmWave sensor that is consistent with varying light conditions. In the proposed method, a frequency modulated continuous wave is transmitted to the surface by the mmWave sensor, and the mmWave sensor backscattering is modulated by the surface medium and roughness. The proposed method also includes preprocessing to calculate the Range-FFT result of the mmWave sensor backscattering and a classification based on a 1-dimensional convolutional neural network to precisely detect the presence of black ice from the Range-FFT result. As a result of the indoor experiment, the proposed black ice detection method achieves an accuracy of 98.2% on dry, wet, and black ice surfaces. Additionally, under low light conditions and in an outdoor environment with sunlight, the proposed method achieves accuracies of 95.6% and 98.5%, respectively.

MTransSee

Gesture recognition using millimeter-wave radios facilitates natural human-computer interactions, but existing works require a consistent environment, i.e., the neural networks for recognition are trained and tested for the same users and at some fixed positions. In this case, their performance will decrease rapidly when they enter into a new environment. To make the model applicable in different environments, a straightforward approach is to collect and re-train the model for the gesture samples on every possible position upon each new user. However, it may ask the users to spend unacceptable time to accomplish such adaptation, which makes it difficult to be widely used in practice. In this paper, we first collect an abundant mmWave gesture dataset containing 59,280 samples as a benchmark to investigate the impact of the environment changes quantitatively. Then we propose a novel transfer-learning approach called mTransSee, which can serve the gestures in practice using pre-learned experience by least adaptation, i.e., retraining using only 8 samples per gesture for the same accuracy. mTransSee reduces dozens of workloads for the environment adaptation. We implement mTransSee on a commodity mmWave sensor and make a user study to compare the advance of mTransSee over the state-of-the-art solution in terms of user experience during adaptation.

M-Gesture: Person-Independent Real-Time In-Air Gesture Recognition Using Commodity Millimeter Wave Radar

Millimeter wave (mmWave) sensing promises to enable contactless and high-precision “in-air” gesture-based human–computer interaction (HCI). While previous works have demonstrated its feasibility, they require tedious gesture collecting for person-independent recognition and they operate in an off-line mode without considering practical issues, such as segmenting gesture and recognition latency. In this work, we propose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M-Gesture</i> , a person-independent real-time mmWave gesture recognition solution. We first build a compact gesture model with a custom-designed neural network to distill the unique features underlying each gesture, while suppressing personalized discrepancy across different users without extra collection and retraining. Furthermore, we design a system status transition (SST) to decide when a gesture begins and ends, which enables automatic gesture segmentation and hence real-time recognition. We prototype <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M-Gesture</i> on a commodity mmWave sensor and demonstrate its advantages using two practical applications: 1) a contactless music player and 2) camera. Extensive experiments and user studies show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M-Gesture</i> has an accuracy of 99% and a short response latency within 25 ms. Moreover, we also collect and release a comprehensive mmWave gesture data set consisting of 54 620 instances from 144 persons, which may have an independent value of facilitating future research.

A Data-Driven Waveform Adaptation Method for Mm-Wave Gait Classification at the Edge

The ever-increasing need for real-time 3D perception for autonomy has resulted in development of a new generation of high-resolution perception sensors paired with powerful edge processors and advanced perception algorithms. However, processing of the large data volumes generated by such high-resolution sensors still imposes a great challenge for resource-limited edge devices and systems. In this work, we visit this problem in the context of pedestrian gait analysis using high-resolution mm-Wave sensors with application in autonomous driving. We propose reducing the total computational load of a resource-limited perception system by dynamically controlling the mm-Wave sensor's resolution on the fly. We provide results on three dynamic sensor adaptation approaches including an end-to-end Deep Q-Learning approach that can overcome the resource and performance limitations inherent to conventional static methods.

FG-LiquID

Contact-less liquid identification via wireless sensing has diverse potential applications in our daily life, such as identifying alcohol content in liquids, distinguishing spoiled and fresh milk, and even detecting water contamination. Recent works have verified the feasibility of utilizing mmWave radar to perform coarse-grained material identification, e.g., discriminating liquid and carpet. However, they do not fully exploit the sensing limits of mmWave in terms of fine-grained material classification. In this paper, we propose FG-LiquID, an accurate and robust system for fine-grained liquid identification. To achieve the desired fine granularity, FG-LiquID first focuses on the small but informative region of the mmWave spectrum, so as to extract the most discriminative features of liquids. Then we design a novel neural network, which uncovers and leverages the hidden signal patterns across multiple antennas on mmWave sensors. In this way, FG-LiquID learns to calibrate signals and finally eliminate the adverse effect of location interference caused by minor displacement/rotation of the liquid container, which ensures robust identification towards daily usage scenarios. Extensive experimental results using a custom-build prototype demonstrate that FG-LiquID can accurately distinguish 30 different liquids with an average accuracy of 97%, under 5 different scenarios. More importantly, it can discriminate quite similar liquids, such as liquors with the difference of only 1% alcohol concentration by volume.

Detection and Localization of Unmanned Aircraft Systems Using Millimeter-Wave Automotive Radar Sensors

This letter reports the measurements and results of tests conducted on the detection of unmanned aircraft systems (UASs) or drones using a millimeter-wave (mmWave) automotive radar sensor. Designed for the detection of automotive targets with radar cross section (RCS) in excess of 20 dBsm, these miniaturized radar sensors have demonstrated detection and ranging performance in detecting a UAS, whose RCS is typically less than -15 dBsm. The sensor operates in the 76-81 GHz (mmWave) band and employs the frequency-modulated continuous-wave principle with a maximum output power of 16 mW. The detection capability of the sensor in terms of range and radial velocity measurements was investigated using a Type-I micro-UAS with a mean RCS less than $-$20 dBsm. In addition to detection, ranging, and velocity measurements, the RCS and the micro-Doppler signature measurements of the UAS in flight are also presented. The measurements and analysis suggest that these mmWave sensors are ideally suited as low-cost sensors for the detection of micro- and mini-classes of UASs at short ranges. The micro-Doppler signature analysis reveals the versatility of mmWave sensors in the extraction of micro-Doppler signatures, which serves as an effective tool for the classification of UASs from other airborne nuisance targets such as birds and moving clutter.

Hybrid precoder designs for decentralized parameter estimation in millimeter wave (mmWave) sensor networks with perfect and imperfect CSI

This work conceives techniques for the design of linear hybrid precoders toward decentralized parameter estimation in a millimeter wave (mmWave) wireless sensor network (WSN). To achieve this objective, a novel system model is proposed for mmWave WSNs, wherein the sensors pre-process their observations using hybrid baseband and radio frequency (RF) precoders, prior to their transmission over a coherent multiple access channel (MAC) to the fusion center (FC). The pertinent combiner is constrained to satisfy the constant signal gain condition, thus ensuring a distortionless estimate at the FC without requiring any additional post-processing. The precoders are designed to achieve the minimum mean square error (MSE) of parameter estimation at the FC. Furthermore, robust precoder designs are also developed to mitigate the effects of imperfect channel state information (CSI) in a mmWave sensor network, which is inevitable due to practical effects such as limited pilot overhead, channel drift, quantization error, among others. Robust designs are presented considering both the probabilistic as well as bounded CSI uncertainty models, which makes the design framework comprehensive since it considers both average and worst case distortion minimization. The centralized MMSE lower bound is also derived to characterize the estimation performance of the proposed schemes. Simulation results are presented to demonstrate the performance of the proposed designs and also verify the analytical propositions.

Indoor Detection and Tracking of People Using mmWave Sensor

This paper proposes a new indoor people detection and tracking system using a millimeter‐wave (mmWave) radar sensor. Firstly, a systematic approach for people detection and tracking is presented—a static clutter removal algorithm used for removing mmWave radar data’s static points. Two efficient clustering algorithms are used to cluster and identify people in a scene. The recursive Kalman filter tracking algorithm with data association is used to track multiple people simultaneously. Secondly, a fast indoor people detection and tracking system is designed based on our proposed algorithms. The method is lightweight enough for scalability and portability, and we can execute it in real time on a Raspberry Pi 4. Finally, the proposed method is validated by comparing it with the Texas Instruments (TI) system. The proposed system’s experimental accuracy ranged from 98% (calculated by misclassification errors) for one person to 65% for five people. The average position errors at positions 1, 2, and 3 are 0.2992 meters, 0.3271 meters, and 0.3171 meters, respectively. In comparison, the Texas Instruments system had an experimental accuracy ranging from 96% for one person to 45% for five people. The average position errors at positions 1, 2, and 3 are 0.3283 meters, 0.3116 meters, and 0.3343 meters, respectively. The proposed method’s advantage is demonstrated in terms of tracking accuracy, computation time, and scalability.

&lt;i&gt;Flying Among Stars:&lt;/i&gt; Jamming-resilient Channel Selection for UAVs through Aerial Constellations&lt;i&gt;&lt;/i&gt;

Wireless communication between an unmanned aerial vehicle (UAV) and the ground base station is susceptible to adversarial jamming. In such situations, it is important for the UAV to indicate a new channel to the BS. This paper describes a method of creating spatial codes that map the chosen channel to the location of the UAVs in space, wherein the latter physically traverses the space from a given so called to another. These points create patterns in the sky, analogous to modulation constellations in classical wireless communications, and are detected at the BS through a millimeter-wave radar sensor. A constellation point represents a distinct n-bit field mapped to a specific channel, allowing simultaneous frequency switching at both ends without any RF transmissions. The main contributions of this paper are: (i) We conduct experimental studies to demonstrate how such constellations may be formed using COTS UAVs and mmWave sensors, (ii) We develop a theoretical framework that maps a desired constellation design to error and band switching time, including multi-user scenario-specific challenges, (iii) We compare our approach against current FHSS technology and (iv) We experimentally demonstrate jamming resilient communications and validate system goodput for links formed by UAV-mounted software defined radios.

Monitoring for Rare Events in a Wireless Powered Communication mmWave Sensor Network.

The use of a wireless sensor network to monitor an area of interest for possible hazardous events has become a common practice. The difficulty of replacing or recharging sensor batteries dictates the use of energy harvesting as a means to extend the network’s lifetime. To this end, energy beamforming is used in a millimeter wave wireless power sensor network with randomly deployed nodes. A simple protocol is proposed that allows nodes to report their charging conditions in an effort to select efficient energy-beamforming strategies. Analytical expressions for the probability of successful information reception and successful reporting are provided for two benchmark schemes: the random and the circular energy-beamforming scheme. A Markov chain is used for the former to model the energy level of sensor nodes. Simple sector selection strategies are presented and their performance, in terms of delay and failure information delivery, is assessed through simulations.

Development and Demonstration of MIMO-SAR mmWave Imaging Testbeds

Multiple-input multiple-output (MIMO) radars and synthetic aperture radar (SAR) techniques are well researched and have been effectively combined for many imaging applications ranging from remote sensing to security. Despite numerous studies that apply MIMO concepts to SAR imaging, the design process of a MIMO-SAR system is non-trivial, especially for millimeter-wave (mmWave) imaging systems. Many issues have to be carefully addressed. Besides, compared with conventional monostatic sampling schemes or MIMO-only solutions, efficient image reconstruction methods for MIMO-SAR topologies are more complicated in short-range applications. To address these issues, we present highly-integrated and reconfigurable MIMO-SAR testbeds, along with examples of three-dimensional (3-D) image reconstruction algorithms optimized for MIMO-SAR configurations. The presented testbeds utilize commercially available wideband mmWave sensors and motorized rail platforms. Several aspects of the MIMO-SAR testbed design process, including MIMO array calibration, electrical/mechanical synchronization, system-level verification, and performance evaluation, are described. We present three versions of MIMO-SAR testbeds with different implementation costs and accuracies to provide alternatives for other researchers who want to implement their testbed framework. Several representative examples in various real-world imaging applications are presented to demonstrate the capabilities of the proposed testbeds and algorithms.

Combining Commercially Available Active and Passive Sensors Into a Millimeter-Wave Imager for Concealed Weapon Detection

Current imaging technology for concealed weapon detection in the millimeter-wave (mm-wave) and terahertz ranges is traditionally limited to a single type of sensor. In this paper, a system combining active and passive mm-wave sensors along with a depth-sensing camera is presented. These sensors are combined at a hardware stage (being integrated into a single imager) as well as at software level (via simultaneous processing and sensor fusion). The resulting images of the individual sensors are combined via image fusion techniques, using a consumer-grade optical sensor as an image noise-removing tool and highlighting the threat over the subject's body. Each sensor is able to compensate some of the imaging limitations of the others, thus offering better overall detection capabilities. Furthermore, the final imager has been formed by combining commercially available devices, which provides a reduced-cost alternative to custom-made equipment.