Online Sensor Data Research Articles

A cloud-based infrastructure to deploy supervisory forecast models for predictive coagulant dosing control

Abstract Advanced optimal dosing control based on multiple online sensor data is operational in several treatment facilities in Norway. The benefits of the dosing control system in maintaining stable phosphate/solids removal and saving coagulant usage are documented in the literature. The dosing algorithm is currently implemented in a programmable logic controller (PLC) connected to the treatment plant's Supervisory Control and Data Acquisition System (SCADA) system. The PLC receives online sensor data from the plant's SCADA, calculates the optimal dosing values, and transmits optimal dosage values back to the SCADA system. The dosing algorithm is frequently updated to keep in sync with the process and equipment upgrades of the treatment plant and advances in control algorithm schemes. The upgrades include new regulatory feedback loops structural changes to the dose equation, and the addition of conditional setpoints. Each maintenance and upgrade routine entails operational downtime where the dosing algorithm is set to a sub-optimal flow-proportional dose. This paper presents a non-intrusive Internet of Things (IoT) infrastructure to implement a predictive/forecast component to an existing dosing control algorithm. The benefits of the new cloud-based system in improving nutrient removal, increasing operational flexibility, and reducing maintenance downtime are presented in this work.

A digital twin model for contaminant fate and transport in urban and natural drainage networks with online state estimation

Increased pollutant loads caused by urbanization and climate change have led to widespread impairment of surface water systems. To better manage these threats, water managers are seeking digital twins that combine online models with sensor data to respond to water quality hazards in real-time. This study introduces Pipedream-WQ, a new model for contaminant transport in drainage networks that combines a novel implicit solver for the unsteady advection–reaction–diffusion (ARD) equation with an efficient data assimilation scheme based on Kalman Filtering. We show that this solver reliably reproduces analytical solutions to the ARD equation in steady conditions, and accurately captures unsteady contaminant transport behavior in a complex drainage network. Furthermore, we show that online sensor data assimilation enables better estimation of contaminant concentrations at ungauged locations compared to a model-only approach. This model will enable improved pollutant tracking and source identification, and active water quality management through real-time control of hydraulic infrastructure.

Prediction of Gearbox Oil Degradation Based on Online Sensor Data and Machine Learning Algorithms

Prediction of Gearbox Oil Degradation Based on Online Sensor Data and Machine Learning Algorithms

Transformer-enabled generative adversarial imputation network with selective generation (SGT-GAIN) for missing region imputation

Although data have been extensively leveraged for process monitoring and control in advanced manufacturing, it still suffers from the connection issues among sensors, machines, and computers, which may lead to significant data loss, i.e., missing region in the collected data, in the application of data-driven monitoring. To address the missing region issues, one popular way is to perform missing data imputation. With the advances of machine learning, many approaches have been developed for the missing data imputation, such as the popular Generative Adversarial Imputation Network (GAIN), which is based on the Generative Adversarial Network (GAN). However, the inherent shortcomings of generative adversarial architecture may still lead to unstable training. More importantly, the collected online sensor data in manufacturing are in sequential order whereas GAIN considered the input data independently. Hence, to address these two limitations, this work proposes a novel approach termed transformer-enabled GAIN with selective generation (SGT-GAIN). The contributions of the proposed SGT-GAIN consist of three aspects: (i) the architecture for transformer-enabled generation is developed to capture the sequential information among the data; (ii) a selective multi-generation framework is proposed to further reduce the imputation bias; and (iii) an ensemble learning framework is applied to enhance the imputation robustness. Both the numerical simulation study and a real-world case study in additive manufacturing demonstrated the effectiveness of the proposed SGT-GAIN.

Fast DNA-analyses for surveillance of microbial communities in full-scale deammonification tanks: Potential for control and troubleshooting

The partial nitritation/anammox process is a popular process for sidestream nitrogen removal, but the process is sensitive to disturbances and requires extensive surveillance and monitoring for optimal performance. We followed two parallel sidestream full-scale deammonification reactors treating digester centrate for a year with high time-resolution of both online sensor data and microbial community as measured by Nanopore DNA sequencing. DNA surveillance revealed system disturbances and allowed for detection of process and equipment upsets, and it facilitated remediating operational actions. Surveillance of anammox bacteria (Ca. Brocadia) revealed unexpected variations, and the composition and dynamics of the flanking community indicated causes for occasional process disturbances with poor nitrogen removal. Monitoring the ammonia oxidizing bacteria (AOB) and nitrite oxidizing bacteria (NOB) could potentially allow reactor operation with increased dissolved oxygen (DO), yielding higher ammonia conversion while keeping NOB in control. The use of fast and frequent DNA sequencing (sampling 3–5 times a week, analysed once per week) was an important supplement, and in many cases superior, to the online sensor data for process surveillance, understanding and control.

Dynamic alpha factor prediction with operating data - a machine learning approach to model oxygen transfer dynamics in activated sludge

Aeration is an energy-intensive process of aerobic biological wastewater treatment. An accurate model of oxygen transfer dynamics in activated sludge tanks would improve design and operation of aeration systems. Such a model should consider spatial and diurnal variation of α-factor as well as site-specific conditions that impact oxygen transfer. For this dynamic prediction a machine learning approach was used for the first time. The data-driven method was based on long-term ex-situ off-gas measurements with pilot-scale reactors (5.8 m height, 8.3 m3 vol) coupled to full-scale activated sludge tanks on the sites of two conventional and a two-stage activated sludge treatment plant. The ex-situ off-gas method allowed to quantify theoretical off-gas parameters in non-aerated zones and thus consider the whole activated sludge tank. We introduced the α0-factor to compare aerated and non-aerated zones under nonsteady-state conditions. Like the established α-factor for steady-state conditions, the α0-factor describes oxygen transfer inhibiting effects in activated sludge. α0-factor was lowest in upstream denitrification zones. This indicates an anoxic elimination of oxygen transfer inhibiting wastewater contaminants which improved oxygen transfer in subsequent aerobic zones. Random Forest models predicted α0-factor reliably in all examined activated sludge tanks even for stormwater events and seasonal variation. Model development only required online sensor data already available to operators. Our results suggest that machine learning models can dynamically predict α-factors in a variety of activated sludge processes, thus considering site-specific conditions in model training without manual calibration.

Augmented Time Regularized Generative Adversarial Network (ATR-GAN) for Data Augmentation in Online Process Anomaly Detection

Supervised machine learning techniques, such as classification models, have been widely applied to online process anomaly detection in advanced manufacturing. However, since abnormal process states rarely occur in regular manufacturing settings, the data collected for model training may be highly imbalanced, which may result in significant training bias for supervised learning and, thus, further deteriorate the anomaly detection accuracy. To reduce the training bias, a natural idea is to incorporate data augmentation techniques to generate effective artificial sample data for the abnormal process states. However, most of the existing data augmentation methods do not effectively consider the temporal orders of the sensor signals, and they also usually require large amounts of actual samples to ensure satisfactory augmentation performance. To address these limitations, this article developed a novel data-driven methodology termed augmented time regularized generative adversarial network (ATR-GAN). By incorporating a proposed augmented generator, ATR-GAN is capable of generating more effective artificial samples for training supervised learning models. The novelty of this augmented generator in the proposed methodology can be summarized into three aspects: 1) an augmented filter layer is introduced in the augmented generator to identify the high-quality artificial samples; 2) in the augmented filter layer, a new distance metric termed time-regularized Hausdorff (TRH) distance is developed to accurately measure the similarity between actual samples and the generated artificial samples; and 3) batching techniques are also employed in the proposed augmented generator to further increase the diversity of the artificial data and fully utilize the relatively limited training data. In addition, the effectiveness of the proposed ATR-GAN is also validated by both numerical simulation and a real-world case study in additive manufacturing. Note to Practitioners—Online process anomaly detection currently plays a significant role in advanced manufacturing since unexpected anomalies may damage product quality and even result in catastrophic loss. In practice, processes are mostly under normal conditions, and anomalies rarely occur. Therefore, the data collected under abnormal conditions are very limited compared to normal conditions, which causes the data imbalanced issue, leading to deterioration in detection accuracy. Many existing data augmentation methods, such as generative adversarial network (GAN), cannot synthesize diversified high-quality artificial samples only using relatively limited actual samples. There is an urgent need in developing an effective data augmentation methodology to address the data imbalanced issue in process anomaly detection. This article developed a novel approach called augmented time regularized GAN (ATR-GAN) for online sensor data augmentation. With this new approach, the applications in additive manufacturing demonstrate that the performance of data augmentation can be improved effectively, and thereafter, the anomaly detection accuracy is also increased significantly. Moreover, the developed methodology is inherently integrated into a generic framework. Thus, it can be further transformed for applications in many other areas that need data augmentation.

Exploration of deep learning models for real-time monitoring of state and performance of anaerobic digestion with online sensors

The immediate response to the state disturbances of anaerobic digestion is essential to prevent anaerobic digestion failure. However, frequent monitoring of the state and performance of anaerobic digestion is challenging. Thus, deep learning models were investigated to predict the state and performance variables from online sensor data. The online sensor data, including pH, electric conductivity, and oxidation–reduction potential, were used as the input features to build deep learning models. The state and performance data measured offline were used as the labels. The model performance was compared for several deep learning models of convolutional neural network (CNN), long short-term memory (LSTM), dense layer, and their combinations. The combined model of CNN and bidirectional LSTM was robust and well-generalized in predicting the state and performance variables (R2 = 0.978, root mean square error = 0.031). The combined model is an excellent soft sensor for monitoring the state and performance of anaerobic digestion from electrochemical sensors.

A Multimodal Perception-Driven Self Evolving Autonomous Ground Vehicle.

Increasingly complex automated driving functions, specifically those associated with free space detection (FSD), are delegated to convolutional neural networks (CNNs). If the dataset used to train the network lacks diversity, modality, or sufficient quantities, the driver policy that controls the vehicle may induce safety risks. Although most autonomous ground vehicles (AGVs) perform well in structured surroundings, the need for human intervention significantly rises when presented with unstructured niche environments. To this end, we developed an AGV for seamless indoor and outdoor navigation to collect realistic multimodal data streams. We demonstrate one application of the AGV when applied to a self-evolving FSD framework that leverages online active machine-learning (ML) paradigms and sensor data fusion. In essence, the self-evolving AGV queries image data against a reliable data stream, ultrasound, before fusing the sensor data to improve robustness. We compare the proposed framework to one of the most prominent free space segmentation methods, DeepLabV3+ [1]. DeepLabV3+ [1] is a state-of-the-art semantic segmentation model composed of a CNN and an autodecoder. In consonance with the results, the proposed framework outperforms DeepLabV3+ [1]. The performance of the proposed framework is attributed to its ability to self-learn free space. This combination of online and active ML removes the need for large datasets typically required by a CNN. Moreover, this technique provides case-specific free space classifications based on the information gathered from the scenario at hand.

An integrated manifold learning approach for high-dimensional data feature extractions and its applications to online process monitoring of additive manufacturing

As an effective dimension reduction and feature extraction technique, manifold learning has been successfully applied to high-dimensional data analysis. With the rapid development of sensor technology, a large amount of high-dimensional data such as image streams can be easily available. Thus, a promising application of manifold learning is in the field of sensor signal analysis, particular for the applications of online process monitoring and control using high-dimensional data. The objective of this study is to develop a manifold learning-based feature extraction method for process monitoring of Additive Manufacturing (AM) using online sensor data. Due to the non-parametric nature of most existing manifold learning methods, their performance in terms of computational efficiency, as well as noise resistance has yet to be improved. To address this issue, this study proposes an integrated manifold learning approach termed multi-kernel metric learning embedded isometric feature mapping (MKML-ISOMAP) for dimension reduction and feature extraction of online high-dimensional sensor data such as images. Based on the extracted features with the utilization of supervised classification and regression methods, an online process monitoring methodology for AM is implemented to identify the actual process quality status. In the numerical simulation and real-world case studies, the proposed method demonstrates excellent performance in both prediction accuracy and computational efficiency.

Wheel Loader Scooping Controller Using Deep Reinforcement Learning

This article presents a deep reinforcement learning-based controller for an unmanned ground vehicle with a custom-built scooping mechanism. The robot's aim is to autonomously perform earth scooping cycles with three degrees of freedom: lift, tilt and the robot's velocity. While the majority of previous studies on automated scooping processes are based on data recorded by expert operators, we present a method to autonomously control a wheel loader to perform the scooping cycle using deep reinforcement learning methods without any user-provided demonstrations. The controller's learning approach is based on the actorcritic, Deep Deterministic Policy Gradient algorithm which we use to map online sensor data as input to continuously update the actuator commands. The training of the scooping policy network is done solely in a simplified simulation environment using a virtual physics engine, which converges to an average of a 65% fill factor from the full bucket capacity and a 5 [sec] average cycle time. We illustrate the performance of the trained policy in simulations and in real-world experiments with 3 different inclination angles of the earth. An additional scooping experiment compared the performance of our controller to remote manual human control. Overall, the deep reinforcement learning-based controller exhibited good performance in terms of both achieved visually bucket fill with varying scooped earth weights of 4.1 - 7.2[kg], and a 5.1 - 7.1[sec] cycle time. The experimental results confirm the ability of our planner to fill bucket as required, indicating that our controller can be used for excavation purposes.

Investigations on quasi-arithmetic means for machine condition monitoring

Machine condition monitoring aims to use on-line sensor data to evaluate machine health conditions. One of the most crucial steps is construction of a health index for incipient fault detection and monotonic degradation assessment. Moreover, observations of a health index can be used as inputs to prognostic models for machine remaining useful life prediction. Even though significant outcomes about sparsity measures, such as kurtosis, the ratio of Lp to Lq norm, pq-mean, smoothness index, negative entropy, and Gini index, for machine health monitoring have been achieved during recent years, construction of a health index for simultaneously realizing incipient fault detection and monotonic degradation assessment is not fully explored due to unexpected variances of repetitive transients caused by rotating machine faults. To solve this problem, in this paper, quasi-arithmetic means (QAMs) are thoroughly investigated. Moreover, the aforementioned sparsity measures can be respectively reformulated as the ratios of different QAMs. Further, a generalized framework based on the ratio of different QAMs for machine health monitoring is proposed. Experimental results demonstrate that some special cases of the generalized framework can simultaneously detect incipient rotating faults, exhibit a monotonic degradation tendency and be robust to impulsive noises, and they are better than existing sparsity measures for machine health monitoring.

A Generic Framework for Degradation Modeling Based on Fusion of Spectrum Amplitudes

Prognostics and health management aims to use on-line sensor data to monitor and predict current and future health conditions of degraded systems and components. Nowadays, constructing composite health indices for characterizing current health conditions of a system from multiple degradation-based sensor data has attracted much attention. These kinds of “data-level” models show strong ability to provide a better degradation characterization of a degraded system than a model solely depending on data from an individual sensor. Although numerous efforts have been made to propose “data-level” fusion methodologies for process data, such as temperature, pressure, speeds, etc., little research has targeted “data-level” fusion models for nonprocess data, such as vibration and acoustic signals. In this article, a methodology for constructing a composite health index from fusion of spectrum amplitudes is proposed. Here, each spectrum amplitude can be regarded as “an individual sensor.” The goal of this article is that a composite health index generated from fusion of spectrum amplitudes can simultaneously detect incipient faults and provide a monotonically increasing trend for degradation assessment. Our proposed methodology was verified by two illustrative examples including gearbox run-to-failure vibration data and bearing run-to-failure vibration data. Results showed that our proposed methodology is better than popular sparse measures for gear and bearing health monitoring and degradation assessment. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Process data, such as temperature, pressure, speed, etc., are capable of directly showing degradation trends of degraded systems and components. “Data-level” fusion models can be directly used to fuse process data from multiple sensors to show a better-fused degradation trend than a sole trend obtained from an individual process data sensor. Being different from process data, nonprocess data, such as vibration and acoustic data, cannot be used to directly show degradation trends unless they are transformed into a health index. One of the benefits of nonprocess data is that they have been proved to be sensitive to incipient machine faults. Nevertheless, due to complicated transmission paths and multiple responses, transforming nonprocess data into a health index is still a challenging task. This article presents a methodology to fuse spectrum amplitudes to form a health index that can simultaneously detect incipient machine faults and assess monotonic machine degradation. The main idea of this article is to regard each spectrum amplitude as “an individual sensor” and the sum of weighted spectrum amplitudes as a health index. To implement the proposed methodology, it is necessary: 1) to transform temporal nonprocess data into frequency spectra by using the well-known Fourier transform; 2) to know two essential properties about detecting incipient machine faults and assessing machine degradation; and 3) to train weights based on the essential properties by any convex optimization algorithms. Once optimal weights are obtained, the health index has ability to simultaneously detect incipient machine faults and monotonically assess machine degradation.

Hands-Free User Interface for AR/VR Devices Exploiting Wearer's Facial Gestures Using Unsupervised Deep Learning.

Developing a user interface (UI) suitable for headset environments is one of the challenges in the field of augmented reality (AR) technologies. This study proposes a hands-free UI for an AR headset that exploits facial gestures of the wearer to recognize user intentions. The facial gestures of the headset wearer are detected by a custom-designed sensor that detects skin deformation based on infrared diffusion characteristics of human skin. We designed a deep neural network classifier to determine the user’s intended gestures from skin-deformation data, which are exploited as user input commands for the proposed UI system. The proposed classifier is composed of a spatiotemporal autoencoder and deep embedded clustering algorithm, trained in an unsupervised manner. The UI device was embedded in a commercial AR headset, and several experiments were performed on the online sensor data to verify operation of the device. We achieved implementation of a hands-free UI for an AR headset with average accuracy of 95.4% user-command recognition, as determined through tests by participants.

The future of WRRF modelling - outlook and challenges.

The wastewater industry is currently facing dramatic changes, shifting away from energy-intensive wastewater treatment towards low-energy, sustainable technologies capable of achieving energy positive operation and resource recovery. The latter will shift the focus of the wastewater industry to how one could manage and extract resources from the wastewater, as opposed to the conventional paradigm of treatment. Debatable questions arise: can the more complex models be calibrated, or will additional unknowns be introduced? After almost 30 years using well-known International Water Association (IWA) models, should the community move to other components, processes, or model structures like 'black box' models, computational fluid dynamics techniques, etc.? Can new data sources - e.g. on-line sensor data, chemical and molecular analyses, new analytical techniques, off-gas analysis - keep up with the increasing process complexity? Are different methods for data management, data reconciliation, and fault detection mature enough for coping with such a large amount of information? Are the available calibration techniques able to cope with such complex models? This paper describes the thoughts and opinions collected during the closing session of the 6th IWA/WEF Water Resource Recovery Modelling Seminar 2018. It presents a concerted and collective effort by individuals from many different sectors of the wastewater industry to offer past and present insights, as well as an outlook into the future of wastewater modelling.

Activity Recognition in Sensor Data Streams for Active and Assisted Living Environments

In active and assisted living environments, a major service that can be provided is the automated assessment of elderly people’s well-being. Therefore, activity recognition is required to detect what types of help disabled persons need to support them in their daily life activities. Unfortunately, it is still a difficult task to estimate the size of the required window for online sensor data streams to recognize a specific activity, especially when new sensor events are recorded. This paper proposes a windowing algorithm, which presents promising results to recognize complex human activities for multi-resident homes. The approach is based on the analysis of the sensor data to identify the best fitting sensors that should be considered in a specified window. Moreover, the second part of this paper proposes a set of different statistical spatio-temporal features to recognize human activities. In order to check the overall performance, this approach is tested using the CASAS data set and artificially generated laboratory data using our HBMS simulator. The results show high performance based on different evaluation metrics compared to other approaches. We believe that the proposed windowing approach provides a good approximation of the required window size in order to robustly detect human activities in comparison to other windowing approaches.

Robust Sparse Representation-Based Classification Using Online Sensor Data for Monitoring Manual Material Handling Tasks

Sensor-based online process monitoring has extensive applications, such as in manufacturing and service industries. In real environments, though, sensor data are often contaminated with noise, leading to severe challenges in accurate data analysis. In the existing literature, noise is generally modeled as Gaussian to analyze sensor data for various applications, for example in fault detection and diagnostics. However, in some applications, such as due to challenging field conditions, sensor data may be disturbed by high levels of outliers such that the Gaussian assumption of sensor noise is inadequate, thus leading to large estimation errors. This paper focuses on online classification applications. A robust sparse representation classification method is proposed, which considers non-Gaussian noise, and thus can effectively analyze sensor data with higher levels of outliers. Case studies were completed, based on both numerically simulated sensor data and actual wearable sensor data from occupational manual material handling process monitoring. The proposed classification method could effectively analyze sensor data with non-Gaussian noise, and outperformed commonly used methods in the literature. Thus, this new method may be advantageous for solving classification problems in challenging field conditions, to address the difficulties of high levels of sensor outliers. Note to Practitioners —This paper proposes a fast, robust classification method for online sensor data classification. The proposed method is designed to cope with high levels of sensor outliers. The robustness of the method and its computational efficiency make it particularly appealing for online sensor data classification in challenging field conditions in which the presence of sensor outliers causes practical difficulties for most existing classification algorithms.

Combined Coagulation and Ultrafiltration Process to Counteract Increasing NOM in Brown Surface Water

Membrane hybrid processes—coagulation coupled with ultrafiltration (UF)—have become a common method to comply with the legal, chemical, and microbiological requirements for drinking water. The main advantages of integrating coagulation with membrane filtration are the enhanced removal of natural organic matter (NOM) and reduced membrane fouling. With in-line coagulation, coagulants are patched into the feed stream directly prior to the membrane process, without removing the coagulated solids. Compared with conventional coagulation/sedimentation, in-line coagulation/membrane reduces the treatment time and footprint. Coagulant dosing could be challenging in raw water of varying quality; however, with relatively stable specific ultraviolet absorbance (SUVA), dosing can be controlled. Recent studies indicate that UV absorbance correlates well with humic substances (HS), the major fraction to be removed during coagulation. This paper describes and evaluates a 30-month UF pilot trial on the surface water of Lake Neden (Sweden), providing drinking water to 60,000 residents. In this study, automatic coagulant dosing based on online measurement was successfully applied. Online sensor data were used to identify the current optimal aluminium coagulation conditions (0.5–0.7 mg L−1) and the potential boundaries (0.9–1.2 mg L−1) for efficient future (2040) NOM removal. The potential increase in NOM could affect the Al dose and drinking water quality significantly within 20 years, should the current trends in dissolved organic carbon (DOC) prevail. UV absorbance, the freshness index, and liquid chromatography-organic carbon detection (LC-OCD) measurements were used to optimise the process. Careful cross-calibration of raw and filtered samples is recommended when using online sensor data for process optimisation, even in low-turbidity water (formazin nephelometric unit (FNU) &lt; 5).

Automatic feed phase identification in multivariate bioprocess profiles by sequential binary classification

In this paper, we propose a new strategy for retrospective identification of feed phases from online sensor-data enriched feed profiles of an Escherichia Coli (E. coli) fed-batch fermentation process. In contrast to conventional (static), data-driven multi-class machine learning (ML), we exploit process knowledge in order to constrain our classification system yielding more parsimonious models compared to static ML approaches. In particular, we enforce unidirectionality on a set of binary, multivariate classifiers trained to discriminate between adjacent feed phases by linking the classifiers through a one-way switch. The switch is activated when the actual classifier output changes. As a consequence, the next binary classifier in the classifier chain is used for the discrimination between the next feed phase pair etc. We allow activation of the switch only after a predefined number of consecutive predictions of a transition event in order to prevent premature activation of the switch and undertake a sensitivity analysis regarding the optimal choice of the (time) lag parameter. From a complexity/parsimony perspective the benefit of our approach is three-fold: i) The multi-class learning task is broken down into binary subproblems which usually have simpler decision surfaces and tend to be less susceptible to the class-imbalance problem. ii) We exploit the fact that the process follows a rigid feed cycle structure (i.e. batch-feed-batch-feed) which allows us to focus on the subproblems involving phase transitions as they occur during the process while discarding off-transition classifiers and iii) only one binary classifier is active at the time which keeps effective model complexity low. We further use a combination of logistic regression and Lasso (i.e. regularized logistic regression, RLR) as a wrapper to extract the most relevant features for individual subproblems from the whole set of high-dimensional sensor data. We train different soft computing classifiers, including decision trees (DT), k-nearest neighbors (k-NN), support vector machines (SVM) and an own developed fuzzy classifier and compare our method with conventional multi-class ML. Our results show a remarkable out-performance of the here proposed method over static ML approaches in terms of accuracy and robustness. We achieved close to error free feed phase classification while reducing the misclassification rates in 17 out of 20 investigated test cases in the range between 39% and 98.2% depending on feature set and classifier architecture. Models trained on features based on selection by RLR significantly outperformed those trained on features suggested by experts and their predictive performance was considerably less affected by the choice of the lag parameter.

3D FEATURE POINT EXTRACTION FROM LIDAR DATA USING A NEURAL NETWORK

Abstract. Accurate positioning of vehicles plays an important role in autonomous driving. In our previous research on landmark-based positioning, poles were extracted both from reference data and online sensor data, which were then matched to improve the positioning accuracy of the vehicles. However, there are environments which contain only a limited number of poles. 3D feature points are one of the proper alternatives to be used as landmarks. They can be assumed to be present in the environment, independent of certain object classes. To match the LiDAR data online to another LiDAR derived reference dataset, the extraction of 3D feature points is an essential step. In this paper, we address the problem of 3D feature point extraction from LiDAR datasets. Instead of hand-crafting a 3D feature point extractor, we propose to train it using a neural network. In this approach, a set of candidates for the 3D feature points is firstly detected by the Shi-Tomasi corner detector on the range images of the LiDAR point cloud. Using a back propagation algorithm for the training, the artificial neural network is capable of predicting feature points from these corner candidates. The training considers not only the shape of each corner candidate on 2D range images, but also their 3D features such as the curvature value and surface normal value in z axis, which are calculated directly based on the LiDAR point cloud. Subsequently the extracted feature points on the 2D range images are retrieved in the 3D scene. The 3D feature points extracted by this approach are generally distinctive in the 3D space. Our test shows that the proposed method is capable of providing a sufficient number of repeatable 3D feature points for the matching task. The feature points extracted by this approach have great potential to be used as landmarks for a better localization of vehicles.