Radar Sensor Data Research Articles

Radar-Based Ego-Motion Estimation of Autonomous Robot for Simultaneous Localization and Mapping

To perform radar-based simultaneous localization and mapping, information on a robot’s ego-motion such as its rotation angle and velocity should be considered along with radar sensor data. In this paper, we propose a method to estimate the robot’s ego-motion using only a radar sensor without any other devices. To estimate the rotation angle of a robot, we use the distribution of detected points on a two-dimensional (2D) plane. The distributions of the detected points at successive time instants are correlated with each other, and the rotation angle can be estimated based on this correlation. In addition, the moving velocity of the robot is estimated from the trend line formed by the detected points on the 2D plane. To evaluate the performance of the proposed estimation method, the information about the robot’s ego-motion received from the robot motor is compared with the information estimated from the radar sensor data. The comparison of the yaw rate obtained through each method shows that our proposed method can estimate the rotation angle of the robot within an error of 3°. In addition, the ego-velocity of the radar-equipped robot can be estimated within an error of 0.073 m/s.

Simultaneous Target Classification and Moving Direction Estimation in Millimeter-Wave Radar System.

In this study, we propose a method to identify the type of target and simultaneously determine its moving direction in a millimeter-wave radar system. First, using a frequency-modulated continuous wave (FMCW) radar sensor with the center frequency of 62 GHz, radar sensor data for a pedestrian, a cyclist, and a car are obtained in the test field. Then, a You Only Look Once (YOLO)-based network is trained with the sensor data to perform simultaneous target classification and moving direction estimation. To generate input data suitable for the deep learning-based classifier, a method of converting the radar detection result into an image form is also proposed. With the proposed method, we can identify the type of each target and its direction of movement with an accuracy of over 95%. Moreover, the pre-trained classifier shows an identification accuracy of 85% even for newly acquired data that have not been used for training.

Radar target classification considering unknown classes using deep convolutional neural network ensemble

The target classification of unknown classes using radar sensor data is discussed. The neural network-based classifier shows high classification accuracy for the learned class targets. However, there is a risk of false decision for the untrained class target owing to an overconfidence problem. The output confidence of the classifier is calibrated using the deep convolutional neural network ensemble structure to propose a method to set the proper threshold for output confidence to decide unknown class targets. When using the proposed method, the accuracy of the learned target is maintained similar to that of the existing single neural network-based classifier, whereas the unknown class target is better identified. Further analysis verifies the effectiveness of the proposed method using commercial automotive radar. The proposed method can classify learned targets with an accuracy of 95% and distinguish unknown class targets with an accuracy of at least 85%. Based on the interaction with other sensors, individual sensors need to make reserved decisions about uncertain information. It is expected that the proposed ensemble network will be efficient in designing the classifier to perform target classification including unknown class decision.

CNN‐based estimation of heading direction of vehicle using automotive radar sensor

Modern autonomous vehicles are being equipped with various automotive sensors to perform special functions. Especially, it is important to predict the heading direction of the front vehicle to adjust the speed of the ego-vehicle and select appropriate actions. Here, we propose a method for estimating the instantaneous heading direction of a vehicle using automotive radar sensor data. First, using a frequency-modulated continuous wave (FMCW) radar in the 77 GHz band, we accumulate the automotive radar sensor data for different movements of the front vehicle (e.g., stop, going ahead, reversing, turning left, and turning right). To distinguish the different movements of the vehicle, we use the convolutional neural network (CNN) and train it using the acquired radar sensor data. Because the CNN algorithm usually uses image data as input, it is essential to convert radar sensor data into image data. Therefore, we apply a high-resolution angle estimation algorithm to the obtained radar data and convert it into a two-dimensional range map. After the CNN model is trained with the obtained radar sensor data, various movements of the front vehicle can be classified with over 94% of accuracy.

Control and Validation of an Autonomous Drag Racing Vehicle

<div class="section abstract"><div class="htmlview paragraph">This paper studies the implementation and validation of control algorithms for an autonomous drag racing vehicle. The previously developed modeling equations are first implemented with control in a realistic simulation environment complete with synthetic sensor data and decision-making algorithms. The controller is then transformed into an embedded on-board processing unit for on-vehicle testing. Camera, lidar, and radar sensor data are investigated and algorithms are created to provide information from physical sensors rather than synthetic data. The control related to actuation of the steering, brake, throttle, and shifting systems are further discussed, along with human-vehicle interaction in terms of handoff and emergency takeovers. The control algorithms are then validated on the research vehicle. This is demonstrated by completing a fully autonomous quarter-mile drag race, complete with camera detection for the staging sequence and MPC trajectory following. Two additional safety scenarios are presented and controlled: starting the race off center and fishtailing.</div></div>

Identification of Human Motion Using Radar Sensor in an Indoor Environment.

In this paper, we propose a method of identifying human motions, such as standing, walking, running, and crawling, using a millimeter wave radar sensor. In our method, two signal processing is performed in parallel to identify the human motions. First, the moment at which a person’s motion changes is determined based on the statistical characteristics of the radar signal. Second, a deep learning-based classification algorithm is applied to determine what actions a person is taking. In each of the two signal processing, radar spectrograms containing the characteristics of the distance change over time are used as input. Finally, we evaluate the performance of the proposed method with radar sensor data acquired in an indoor environment. The proposed method can find the moment when the motion changes with an error rate of 3%, and also can classify the action that a person is taking with more than 95% accuracy.

Combining Radar and Optical Sensor Data to Measure Player Value in Baseball.

Evaluating a player’s talent level based on batted balls is one of the most important and difficult tasks facing baseball analysts. An array of sensors has been installed in Major League Baseball stadiums that capture seven terabytes of data during each game. These data increase interest among spectators, but also can be used to quantify the performances of players on the field. The weighted on base average cube model has been used to generate reliable estimates of batter performance using measured batted-ball parameters, but research has shown that running speed is also a determinant of batted-ball performance. In this work, we used machine learning methods to combine a three-dimensional batted-ball vector measured by Doppler radar with running speed measurements generated by stereoscopic optical sensors. We show that this process leads to an improved model for the batted-ball performances of players.

Radar-Based Face Recognition: One-Shot Learning Approach

In this paper, we propose a method for recognizing human faces by applying one-shot learning to radar sensor data. First, we use a small-sized radar sensor with a center frequency of 61 GHz and accumulate radar signals reflected from different human faces. Different spatial characteristics of each human face are reflected in the signals received through multiple receiving channels. Then, we apply the one-shot learning approach for obtaining effective recognition performance even on small data sets, to distinguish different faces. The one-shot learning method has the advantages for extracting feature information from small labelled samples and adapting to new sample not studied previously. To generate the input of the one-shot learning model, the signals received from the multiple channels are concatenated in parallel. The proposed one-shot learning method based on a Siamese network shows a recognition accuracy of almost 97.6% and demonstrates a better recognition performance than the conventional deep neural networks, when applied to the radar signal data of eight different faces.

Radar Data Integrity Verification Using 2D QIM-Based Data Hiding.

The modern-day vehicle is evolved in a cyber-physical system with internal networks (controller area network (CAN), Ethernet, etc.) connecting hundreds of micro-controllers. From the traditional core vehicle functions, such as vehicle controls, infotainment, and power-train management, to the latest developments, such as advanced driver assistance systems (ADAS) and automated driving features, each one of them uses CAN as their communication network backbone. Automated driving and ADAS features rely on data transferred over the CAN network from multiple sensors mounted on the vehicle. Verifying the integrity of the sensor data is essential for the safety and security of occupants and the proper functionality of these applications. Though the CAN interface ensures reliable data transfer, it lacks basic security features, including message authentication, which makes it vulnerable to a wide array of attacks, including spoofing, replay, DoS, etc. Using traditional cryptography-based methods to verify the integrity of data transmitted over CAN interfaces is expected to increase the computational complexity, latency, and overall cost of the system. In this paper, we propose a light-weight alternative to verify the sensor data’s integrity for vehicle applications that use CAN networks for data transfers. To this end, a framework for 2-dimensional quantization index modulation (2D QIM)-based data hiding is proposed to achieve this goal. Using a typical radar sensor data transmission scenario in an autonomous vehicle application, we analyzed the performance of the proposed framework regarding detecting and localizing the sensor data tampering. The effects of embedding-induced distortion on the applications using the radar data were studied through a sensor fusion algorithm. It was observed that the proposed framework offers the much-needed data integrity verification without compromising on the quality of sensor fusion data and is implemented with low overall design complexity. This proposed framework can also be used on any physical network interface other than CAN, and it offers traceability to in-vehicle data beyond the scope of the in-vehicle applications.

Face identification using millimetre‐wave radar sensor data

In this Letter, the authors propose a face identification method using radar sensor data. They use a frequency-modulated continuous wave radar sensor that utilises a centre frequency of 61 GHz and a bandwidth of 6 GHz. Then, they accumulate radar data by transmitting and receiving radar signals on human faces. Finally, they use a convolutional neural network (CNN) to distinguish radar signals reflected from different human faces. In this network, signals received from multiple antenna elements are synthesised in parallel to make the radar signals into an image that is the input form of the CNN. The accuracy of face recognition through the CNN is > 98 % . In addition, they also collect radar data when the same subjects wear cotton masks. Within their entire dataset, wearing a mask does not significantly affect the accuracy of the radar-based face recognition method.

Indoor human activity recognition using high-dimensional sensors and deep neural networks

Many smart home applications rely on indoor human activity recognition. This challenge is currently primarily tackled by employing video camera sensors. However, the use of such sensors is characterized by fundamental technical deficiencies in an indoor environment, often also resulting in a breach of privacy. In contrast, a radar sensor resolves most of these flaws and maintains privacy in particular. In this paper, we investigate a novel approach toward automatic indoor human activity recognition, feeding high-dimensional radar and video camera sensor data into several deep neural networks. Furthermore, we explore the efficacy of sensor fusion to provide a solution in less than ideal circumstances. We validate our approach on two newly constructed and published data sets that consist of 2347 and 1505 samples distributed over six different types of gestures and events, respectively. From our analysis, we can conclude that, when considering a radar sensor, it is optimal to make use of a three-dimensional convolutional neural network that takes as input sequential range-Doppler maps. This model achieves 12.22% and 2.97% error rate on the gestures and the events data set, respectively. A pretrained residual network is employed to deal with the video camera sensor data and obtains 1.67% and 3.00% error rate on the same data sets. We show that there exists a clear benefit in combining both sensors to enable activity recognition in the case of less than ideal circumstances.

Assimilating Soil Moisture Retrieved from Sentinel-1 and Sentinel-2 Data into WOFOST Model to Improve Winter Wheat Yield Estimation

Crop yield estimation at a regional scale over a long period of time is of great significance to food security. In past decades, the integration of remote sensing observations and crop growth models has been recognized as a promising approach for crop growth monitoring and yield estimation. Optical remote sensing data are susceptible to cloud and rain, while synthetic aperture radar (SAR) can penetrate through clouds and has all-weather capabilities. This allows for more reliable and consistent crop monitoring and yield estimation in terms of radar sensor data. The aim of this study is to improve the accuracy for winter wheat yield estimation by assimilating time series soil moisture images, which are retrieved by a water cloud model using SAR and optical data as input, into the crop model. In this study, SAR images were acquired by C-band SAR sensors boarded on Sentinel-1 satellites and optical images were obtained from a Sentinel-2 multi-spectral instrument (MSI) for Hengshui city of Hebei province in China. Remote sensing data and ground data were all collected during the main growing season of winter wheat. Both the normalized difference vegetation index (NDVI), derived from Sentinel-2, and backscattering coefficients and polarimetric indicators, computed from Sentinel-1, were used in the water cloud model to derive time series soil moisture (SM) images. To improve the prediction of crop yields at the field scale, we incorporated remotely sensed soil moisture into the World Food Studies (WOFOST) model using the Ensemble Kalman Filter (EnKF) algorithm. In general, the trend of soil moisture inversion was consistent with the ground measurements, with the coefficient of determination (R2) equal to 0.45, 0.53, and 0.49, respectively, and RMSE was 9.16%, 7.43%, and 8.53%, respectively, for three observation dates. The winter wheat yield estimation results showed that the assimilation of remotely sensed soil moisture improved the correlation of observed and simulated yields (R2 = 0.35; RMSE =934 kg/ha) compared to the situation without data assimilation (R2 = 0.21; RMSE = 1330 kg/ha). Consequently, the results of this study demonstrated the potential and usefulness of assimilating SM retrieved from both Sentinel-1 C-band SAR and Sentinel-2 MSI optical remote sensing data into WOFOST model for winter wheat yield estimation and could also provide a reference for crop yield estimation with data assimilation for other crop types.

Automotive radar and camera fusion using Generative Adversarial Networks

Radar sensors are considered to be very robust under harsh weather and poor lighting conditions. Largely owing to this reputation, they have found broad application in driver assistance and highly automated driving systems. However, radar sensors have considerably lower precision than cameras. Low sensor precision causes ambiguity in the human interpretation of the measurement data and makes the data labeling process difficult and expensive. On the other hand, without a large amount of high-quality labeled training data, it is difficult, if not impossible, to ensure that the supervised machine learning models can predict, classify, or otherwise analyze the phenomenon of interest with the required accuracy. This paper presents a method for fusing the radar sensor measurements with the camera images. A proposed fully-unsupervised machine learning algorithm converts the radar sensor data to artificial, camera-like, environmental images. Through such data fusion, the algorithm produces more consistent, accurate, and useful information than that provided solely by the radar or the camera. The essential point of the work is the proposal of a novel Conditional Multi-Generator Generative Adversarial Network (CMGGAN) that, being conditioned on the radar sensor measurements, can produce visually appealing images that qualitatively and quantitatively contain all environment features detected by the radar sensor.

Detection of Breathing and Heart Rates in UWB Radar Sensor Data Using FVPIEF-Based Two-Layer EEMD

Ultra-wideband (UWB) radar is an important remote sensing tool of life detection or a non-contact monitor of the vital signals. By processing the received UWB pulse echoes reflected from the body, different signals corresponding to heart activity and breathing, corrupted by body motion and the environment noise, are wanted to be separated clearly. However, the heartbeat signal is so tiny that it is covered by breathing harmonics and clutters. At the same time, since the frequencies of the vital signals are very close, usually around 1 Hz, it is difficult to apply an ordinary frequency filter to separate them apart. This problem induces that the vital signal detection method, usually, only detects the large breath signal, not the heartbeat signal. To solve this problem, a novel method is provided, in this paper, to extract the heartbeat and the breath information simultaneously. The method uses the feature time index with the first valley peak of the energy function of intrinsic mode functions (FVPIEF) calculated by pseudo bi-dimension ensemble empirical mode decomposition method and extracts the vital signals by the ensemble empirical mode decomposition (EEMD). Both simulation and experiment results evidently show that the proposed FVPIEF based two-layer EEMD method is effective for separating the small heartbeat signal from the large breath signal and significantly improves the evaluation of heart and breathing rates in both hold-breathing and breathing conditions.

Propagating sensor uncertainty to better infer office occupancy in smart building control

Occupant presence and behaviour in buildings is considered a key element towards building intelligent and pervasive environments. Yet, practical applications of energy intelligent buildings typically suffer from high sensor unreliability. In this work, we propose a layered probabilistic framework for occupancy-based control in intelligent buildings. We adopt a cascade of layers, where each layer addresses different aspects of the occupancy detection problem in a probabilistic manner rather than in a hard rule engine. We show that propagating uncertainty through each layer instead of standard hard decision outcomes improves the overall system performance. This finding suggests that smart building interfaces and communication data formats may need to input and output probabilistic data rather than simple discrete classification outputs. System performance and user comfort were evaluated with real life radar sensor data, based on an algorithm that allows real-time (casual) processing. Energy savings of up to 30% were obtained, compared to baseline measurements, while maintaining user comfort.

A random finite set approach for dynamic occupancy grid maps with real-time application

Grid mapping is a well-established approach for environment perception in robotic and automotive applications. Early work suggests estimating the occupancy state of each grid cell in a robot’s environment using a Bayesian filter to recursively combine new measurements with the current posterior state estimate of each grid cell. This filter is often referred to as binary Bayes filter. A basic assumption of classical occupancy grid maps is a stationary environment. Recent publications describe bottom-up approaches using particles to represent the dynamic state of a grid cell and outline prediction-update recursions in a heuristic manner. This paper defines the state of multiple grid cells as a random finite set, which allows to model the environment as a stochastic, dynamic system with multiple obstacles, observed by a stochastic measurement system. It motivates an original filter called the probability hypothesis density / multi-instance Bernoulli (PHD/MIB) filter in a top-down manner. The paper presents a real-time application serving as a fusion layer for laser and radar sensor data and describes in detail a highly efficient parallel particle filter implementation. A quantitative evaluation shows that parameters of the stochastic process model affect the filter results as theoretically expected and that appropriate process and observation models provide consistent state estimation results.

Extended Object Tracking and Classification Using Radar and ESM Sensor Data

Extended object tracking and classification (EOTC) using multisensor kinematic and attribute data is a challenging and highly coupled problem. It is a joint decision and estimation (JDE) problem. A good solution has to handle effectively the coupling between tracking and classification and also make good use of multisensor data. For this purpose and based on the recently proposed JDE framework, this letter proposes a conditional JDE (CJDE) risk, which integrates the tracking error and the classification cost using heterogeneous sensor data. Object classes differ from each other in maneuverability and feature attribute. A suitable model is identified and used to describe object classes differing in maneuverability. Also presented are attribute evolution and measurement models. Then, an EOTC algorithm optimizing the CJDE risk is proposed, which considers the coupling and the information contained in various data. Simulation results demonstrate the superiority of the proposed EOTC algorithm in joint performance.

Estimating freeway travel time and its reliability using radar sensor data

ABSTRACTTravel time and its reliability are intuitive system performance measures for freeway traffic operations. This paper proposes a method to estimate travel times based on data collected from roadside radar sensors, considering spatially correlated traffic conditions. Link-level and corridor-level travel time distributions are estimated using these travel time estimates and compared with the ones estimated based on probe vehicle data. The maximum likelihood estimation is used to estimate the parameters of Weibull, gamma, normal, and lognormal distributions. According to the log-likelihood values, lognormal distribution is the best fit among all the tested distributions. Corridor-level travel time reliability measures are extracted from the travel time distributions. The proposed travel time estimation model can well capture the temporal pattern of travel time and its distribution.

Combining satellite data for better tropical forest monitoring

Implementation of policies to reduce forest loss challenges the Earth observation community to improve forest monitoring. An important avenue for progress is the use of new satellite missions and the combining of optical and synthetic aperture radar sensor data.

A comparative analysis of ALOS PALSAR L-band and RADARSAT-2 C-band data for land-cover classification in a tropical moist region

This paper explores the use of ALOS (Advanced Land Observing Satellite) PALSARL-band (Phased Array type L-band Synthetic Aperture Radar) and RADARSAT-2 C-band data for land-cover classification in a tropical moist region. Transformed divergence was used to identify potential textural images which were calculated with the gray-level co-occurrence matrix method. The standard deviation of selected textural images and correlation coefficients between them were then used to determine the best combination of texture images for land-cover classification. Classification results based on different scenarios with maximum likelihood classifier were compared. Based on the identified best scenarios, different classification algorithms – maximum likelihood classifier, classification tree analysis, Fuzzy ARTMAP (a neural-network method), k-nearest neighbor, object-based classification, and support vector machine were compared for examining which algorithm was suitable for land-cover classification in the tropical moist region. This research indicates that the combination of radiometric images and their textures provided considerably better classification accuracies than individual datasets. The L-band data provided much better land-cover classification than C-band data but neither L-band nor C-band was suitable for fine land-cover classification system, no matter which classification algorithm was used. L-band data provided reasonably good classification accuracies for coarse land-cover classification system such as forest, succession, agropasture, water, wetland, and urban with an overall classification accuracy of 72.2%, but C-band data provided only 54.7%. Compared to the maximum likelihood classifier, both classification tree analysis and Fuzzy ARTMAP provided better performances, object-based classification and support vector machine had similar performances, and k-nearest neighbor performed poorly. More research should address the use of multitemporal radar data and the integration of radar and optical sensor data for improving land-cover classification.