Unreliable Sensor Data Research Articles

Stormwater digital twin with online quality control detects urban flood hazards under uncertainty

Urban drainage systems face increased floods and combined sewer overflows due to climate change and population growth. To manage these hazards, cities are seeking stormwater digital twins that integrate sensor data with hydraulic models for real-time response. However, these efforts are complicated by unreliable sensor data, imperfect hydrologic models, and inaccurate rainfall forecasts. To address these issues, we introduce a stormwater digital twin system that uses online data assimilation to estimate stormwater depths and discharges under sensor and model uncertainty. We first derive a novel state estimation scheme based on Extended Kalman Filtering that fuses sensor data into a hydraulic model while simultaneously detecting and removing faulty measurements. The system’s accuracy is evaluated through a long-term deployment in Austin’s flood-prone Waller Creek watershed. The digital twin model demonstrates enhanced accuracy in estimating stormwater depths at ungauged locations and delivers more accurate near-term forecasts. Moreover, it effectively identifies and removes sensor faults from streaming data, achieving a Receiver Operating Characteristic Area Under the Curve (ROC AUC) of over 0.99 and significantly reducing the potential for false flood alarms. This study provides a complete software implementation, offering water managers a reliable framework for real-time monitoring, rapid flood response, predictive maintenance, and active control of sewer systems.

Conducting Vessel Data Imputation Method Selection Based on Dataset Characteristics

Time series datasets collected from marine sensors inevitably undergo missing data problems. This cause unreliable sensor data to assist the decision-making process. Many methods are offered to impute missing values. However, selecting the best imputation method is not a trivial task, as it usually requires domain expertise and several trial-and-error iterations. Furthermore, when imputations are carried out in a careless way, it generates a high error factor that can lead stakeholders to wrong assumptions. This paper provides a systematic approach that is able to extract characteristics of underlying data and, based on it, recommends the less error-prone imputation method. We evaluate our proposed method using nine real-world vessel datasets. In total, we generated 3859 data samples consisting of 17 inputs and 1 target feature. Experimental results show that the proposed approach is capable of obtaining a weighted F1-Score of 92.6%. Additionally, when compared with the application of careless selected imputation methods, our work is able to gain up to 86% on the average imputation score, with the worst case gain being of 5%. We empirically demonstrate that the proposed approach is efficient when selecting the best imputation methods.

Improved Indoor Positioning by Means of Occupancy Grid Maps Automatically Generated from OSM Indoor Data

In recent years, there is a growing interest in indoor positioning due to the increasing amount of applications that employ position data. Current approaches determining the location of objects in indoor environments are facing problems with the accuracy of the sensor data used for positioning. A solution to compensate inaccurate and unreliable sensor data is to include further information about the objects to be positioned and about the environment into the positioning algorithm. For this purpose, occupancy grid maps (OGMs) can be used to correct such noisy data by modelling the occupancy probability of objects being at a certain location in a specific environment. In that way, improbable sensor measurements can be corrected. Previous approaches, however, have focussed only on OGM generation for outdoor environments or require manual steps. There remains need for research examining the automatic generation of OGMs from detailed indoor map data. Therefore, our study proposes an algorithm for automated OGM generation using crowd-sourced OpenStreetMap indoor data. Subsequently, we propose an algorithm to improve positioning results by means of the generated OGM data. In our study, we used positioning data from an Ultra-wideband (UWB) system. Our experiments with nine different building map datasets showed that the proposed method provides reliable OGM outputs. Furthermore, taking one of these generated OGMs as an example, we demonstrated that integrating OGMs in the positioning algorithm increases the positioning accuracy. Consequently, the proposed algorithms now enable the integration of environmental information into positioning algorithms to finally increase the accuracy of indoor positioning applications.

Dempster-Shafer Theory for Modeling and Treating Uncertainty in IoT Applications Based on Complex Event Processing.

The Internet of Things (IoT) has emerged from the proliferation of mobile devices and objects connected, resulting in the acquisition of periodic event flows from different devices and sensors. However, such sensors and devices can be faulty or affected by failures, have poor calibration, and produce inaccurate data and uncertain event flows in IoT applications. A prominent technique for analyzing event flows is Complex Event Processing (CEP). Uncertainty in CEP is usually observed in primitive events (i.e., sensor readings) and rules that derive complex events (i.e., high-level situations). In this paper, we investigate the identification and treatment of uncertainty in CEP-based IoT applications. We propose the DST-CEP, an approach that uses the Dempster–Shafer Theory to treat uncertainties. By using this theory, our solution can combine unreliable sensor data in conflicting situations and detect correct results. DST-CEP has an architectural model for treating uncertainty in events and its propagation to processing rules. We describe a case study using the proposed approach in a multi-sensor fire outbreak detection system. We submit our solution to experiments with a real sensor dataset, and evaluate it using well-known performance metrics. The solution achieves promising results regarding Accuracy, Precision, Recall, F-measure, and ROC Curve, even when combining conflicting sensor readings. DST-CEP demonstrated to be suitable and flexible to deal with uncertainty.

Anomalous IoT Sensor Data Detection: An Efficient Approach Enabled by Nonlinear Frequency-Domain Graph Analysis

The detection of anomalous Internet-of-Things (IoT) sensor data is extremely important in many industrial applications due to the catastrophic consequences of the faulty or unreliable sensor data. Good anomalous data detection performance with high detection efficiency is indeed a dilemma since it is difficult to derive an explicit detection function to characterize the relationships between the anomalous values and the detection indicator. To overcome this difficulty, the location information of the IoT sensors is exploited in this study to characterize and reconstruct the relationships among the sensor data based on a second-order nonlinear polynomial graph filter (NPGF). The analysis of the sensor data reconstruction model is then conducted in the frequency domain based on the 2-D inverse graph Fourier transform (GFT), and the reconstruction error function for the sensor data is analytically derived based on the second-order GFT coefficients. It is shown that the detection efficiency can be greatly improved if the input graph signal is designed to be bandlimited. The anomalous sensor data detection is then conducted in the frequency domain as high-frequency components are more sensitive to the deviation values. An NPGF-based frequency-domain algorithm is proposed for the anomalous sensor data detection, which is illustrated and validated with a real-world data set for temperature monitoring. The simulation results demonstrate the detection performance and efficiency improvement of the proposed algorithm in anomaly detection.

Outlier Detection in Wireless Sensor Networks Using Model Selection-Based Support Vector Data Descriptions.

Wireless sensor networks (WSNs) are often deployed in harsh and unattended environments, which may cause the generation of abnormal or low quality data. The inaccurate and unreliable sensor data may increase generation of false alarms and erroneous decisions, so it’s very important to detect outliers in sensor data efficiently and accurately to ensure sound scientific decision-making. In this paper, an outlier detection algorithm (TSVDD) using model selection-based support vector data description (SVDD) is proposed. Firstly, the Toeplitz matrix random feature mapping is used to reduce the time and space complexity of outlier detection. Secondly, a novel model selection strategy is realized to keep the algorithm stable under the low feature dimensions, this strategy can select a relatively optimal decision model and avoid both under-fitting and overfitting phenomena. The simulation results on SensorScope and IBRL datasets demonstrate that, TSVDD achieves higher accuracy and lower time complexity for outlier detection in WSNs compared with existing methods.

On Exploiting Structured Human Interactions to Enhance Sensing Accuracy in Cyber-physical Systems

In this article, we describe a general methodology for enhancing sensing accuracy in cyber-physical systems that involve structured human interactions in noisy physical environment. We define structured human interactions as domain-specific workflow. A novel workflow-aware sensing model is proposed to jointly correct unreliable sensor data and keep track of states in a workflow. We also propose a new inference algorithm to handle cases with partially known states and objects as supervision. Our model is evaluated with extensive simulations. As a concrete application, we develop a novel log service called Emergency Transcriber , which can automatically document operational procedures followed by teams of first responders in emergency response scenarios. Evaluation shows that our system has significant improvement over commercial off-the-shelf (COTS) sensors and keeps track of workflow states with high accuracy in noisy physical environment.

An Indoor Localization System for Telehomecare Applications

In this paper, we present a novel probabilistic technique, based on the Bayes filter, able to estimate the user location, even with unreliable sensor data coming only from fixed sensors in the monitored environment. Our approach has been extensively tested in a home-like environment, as well as in a real home, and achieves very good results. We present results on two datasets, representative of real life conditions, collected during the testing phase. We detect the patient location with subroom accuracy, an improvement over the state of the art for localization using only environmental sensors. The main drawback is that it is only suitable for applications where a single person is present in the environment, like as with other approaches that do not use any mobile device. For this reason, we introduced the “telehomecare” term, therefore differentiating from generic telemedicine applications, where many people can be in the same environment at the same time.

Model-predictive asset guarding by team of autonomous surface vehicles in environment with civilian boats

In this paper, we present a contract-based, decentralized planning approach for a team of autonomous unmanned surface vehicles (USV) to patrol and guard an asset in an environment with hostile boats and civilian traffic. The USVs in the team have to cooperatively deal with the uncertainty about which boats pose an actual threat and distribute themselves around the asset to optimize their guarding opportunities. The developed planner incorporates a contract-based algorithm for allocating tasks to the USVs through forward simulating the mission and assigning estimated utilities to candidate task allocation plans. The task allocation process uses a form of marginal cost-based contracting that allows decentralized, cooperative task negotiation among neighboring agents. The task allocation plans are realized through a corresponding set of low-level behaviors. In this paper, we demonstrate the planner using two mission scenarios. However, the planner is general enough to be used for a variety of scenarios with mission-specific tasks and behaviors. We provide detailed analysis of simulation results and discuss the impact of communication interruptions, unreliable sensor data, and simulation inaccuracies on the performance of the planner.

Smart Urban Signal Networks: Initial Application of the SURTRAC Adaptive Traffic Signal Control System

In this paper, we describe a pilot implementation and field test of a recently developed approach to real-time adaptive traffic signal control. The pilot system, called SURTRAC (Scalable Urban Traffic Control), follows the perspective of recent work in multi-agent planning and implements a decentralized, schedule-driven approach to traffic signal control. Under this approach, each intersection independently (and asynchronously) computes a schedule that optimizes the flow of currently approaching traffic through that intersection, and uses this schedule to decide when to switch green phases. The traffic outflows projected by this schedule are then communicated to the intersection's downstream neighbors, to increase visibility of vehicles entering their respective planning horizons. This process is repeated as frequently as once per second in rolling horizon fashion, to provide real-time responsiveness to changing traffic conditions and coordinated signal network behavior. After summarizing this basic approach to adaptive traffic signal control and the domain challenges it is intended to address, we describe the pilot implementation of SURTRAC and its application to a nine-intersection road network in Pittsburgh, Pennsylvania. Both the SURTRAC architecture for interfacing with the detection equipment, hardware controller and communication network at a given intersection and the extensions required to account for unreliable sensor data are discussed. Finally, we present the results of a pilot test of the system, where SURTRAC is seen to achieve major reductions in travel times and vehicle emissions over pre-existing signal timings.

Continuity analysis of programs

We present an analysis to automatically determine if a program represents a continuous function, or equivalently, if infinitesimal changes to its inputs can only cause infinitesimal changes to its outputs. The analysis can be used to verify the robustness of programs whose inputs can have small amounts of error and uncertainty---e.g., embedded controllers processing slightly unreliable sensor data, or handheld devices using slightly stale satellite data. Continuity is a fundamental notion in mathematics. However, it is difficult to apply continuity proofs from real analysis to functions that are coded as imperative programs, especially when they use diverse data types and features such as assignments, branches, and loops. We associate data types with metric spaces as opposed to just sets of values, and continuity of typed programs is phrased in terms of these spaces. Our analysis reduces questions about continuity to verification conditions that do not refer to infinitesimal changes and can be discharged using off-the-shelf SMT solvers. Challenges arise in proving continuity of programs with branches and loops, as a small perturbation in the value of a variable often leads to divergent control-flow that can lead to large changes in values of variables. Our proof rules identify appropriate ``synchronization points'' between executions and their perturbed counterparts, and establish that values of certain variables converge back to the original results in spite of temporary divergence. We prove our analysis sound with respect to the traditional epsilon-delta definition of continuity. We demonstrate the precision of our analysis by applying it to a range of classic algorithms, including algorithms for array sorting, shortest paths in graphs, minimum spanning trees, and combinatorial optimization. A prototype implementation based on the Z3 SMT-solver is also presented.

Constructing dependable certainty grids from unreliable sensor data

Measurements from sensors as they are used for robotic grid map applications typically show behavior like degradation or discalibration over time, which affects the quality of the generated maps. This paper presents two novel algorithms for the generation of certainty grids dealing with this behavior. The first algorithm named Fault-Tolerant Certainty Grid (FTCG) performs voting over multiple sensor readings. This approach removes up to ( n − 1 ) / 2 faulty measurements for grid cells that are updated by n independent sensors, however it requires that each grid cell is covered by at least three different independent sensors. The second algorithm named Robust Certainty Grid (RCG) uses a sensor validation method that detects abnormal sensor measurements and adjusts a confidence value for each sensor. This method also supports reintegration of recovered sensors from transient faults and sensor maintenance by providing a measurement for the operability of a sensor. The RCG algorithm works with at least three sensors with a partially overlapping sensing range and needs fewer sensor inputs and less memory than the FTCG approach. Results from simulation and an experimental evaluation on an autonomous mobile robot show that under the presence of unreliable sensor data, both algorithms perform better than the Bayesian approach typically used for certainty grids.