Spatio-temporal Outliers Research Articles

Outlier-Concerned Data Completion Exploiting Intra- and Inter-Data Correlations in Sparse CrowdSensing

Mobile CrowdSensing (MCS) is a popular data collection paradigm which usually faces the problem of sparse sensed data because of the limited sensing cost. In order to address the situation of sparse data, sparse MCS recruits users to sense important areas and infers completed data by data completion, which is crucial in sparse MCS for urban sensing applications (e.g. enhancing data expression, improving urban analysis, guiding city planning, etc.) To achieve accurate completion results, previous methods usually utilize the universal similarity and conventional tendency while incorporating only a single dataset to infer the full map. However, in real-world scenarios, there may exist many kinds of data (inter-data), that could help to complement each other. Moreover, for each kind of data (intra-data), there usually exist a few but important outliers caused by the special events (e.g., parking peak, traffic congestion, or festival parade), which may behave in a different way as the statistical patterns. These outliers cannot be ignored, while it is difficult to detect and recover them in data completion because of the following challenges: 1) the infrequency and unpredictability of outliers’ occurrence, 2) the large deviations against the means compared to normal values, and 3) the complex spatiotemporal relations among inter-data. To this end, focusing on spatiotemporal data with both intra- and inter-data correlations, we propose a matrix completion method that takes outliers’ effects into consideration and exploits both intra- and inter-data correlations for enhancing performance. Specifically, we first conduct the Deep Matrix Factorization (DMF) with multiple auxiliary Neural Networks, which named Stacked Deep Matrix Factorization (SDMF). Note that the loss function of SDMF is no longer the previous MSE loss function, but replaced with an Outlier Value Loss (OVL) function to effectively detect and recover the outliers. Moreover, a spatiotemporal outlier value memory network is added for further enhancing the outlier inference. Finally, we take extensive qualitative and quantitative experiments on two popular datasets each with two types of sensing data, and the experimental results indicate the advantages of our method that outperforms the state-of-the-art methods.

On Robustness for Spatio-Temporal Data

The spatio-temporal variogram is an important factor in spatio-temporal prediction through kriging, especially in fields such as environmental sustainability or climate change, where spatio-temporal data analysis is based on this concept. However, the traditional spatio-temporal variogram estimator, which is commonly employed for these purposes, is extremely sensitive to outliers. We approach this problem in two ways in the paper. First, new robust spatio-temporal variogram estimators are introduced, which are defined as M-estimators of an original data transformation. Second, we compare the classical estimate against a robust one, identifying spatio-temporal outliers in this way. To accomplish this, we use a multivariate scale-contaminated normal model to produce reliable approximations for the sample distribution of these new estimators. In addition, we define and study a new class of M-estimators in this paper, including real-world applications, in order to determine whether there are any significant differences in the spatio-temporal variogram between two temporal lags and, if so, whether we can reduce the number of lags considered in the spatio-temporal analysis.

Long term exposure to ambient air particulate matter and mortality effects in Megacity of Tehran, Iran: 2012–2017

Ambient particulate matter (PM) is the major environmental concern in Tehran, Iran. Exposure to such particles is a serious global health concern and substantial evidence indicates that exposure to particulate matter contributes to cardiovascular and respiratory disease. Here, we provide a detailed health impact of ambient air pollution in Tehran by linking PM pollution with mortality caused by cardiovascular and respiratory disease. Air quality data of PM obtained from Tehran Air Quality Monitoring Stations (TAQMSs) during 2012–2017. Data mining methods conducted by Spatio-temporal outlier detection and “Air Pollution and Health: a European Approach (APHEA) project” was applied to impute missing values. Time-series data were analyzed to estimate the associations between PM pollution and cause-specific mortality at different lags in Tehran. Based on descriptive analysis, PM2.5 is expected to be a major influencing factor in the increased cardiovascular-related death (CVD) and respiratory dysfunction related death (RD) deaths. The cross-correlation analysis showed the same source for PM2.5 and PM10 in the ambient air of Tehran. Also, the results indicated delays of about 7–15 days from the increased concentrations of PMs and the observed rising in CVD and RD death. Trend analysis showed a more decreasing trend for PM2.5 pollutions than PM10 pollution and a slightly increasing trend for cause-specific deaths during 2012–2017. Long term exposure to high PM levels increases the CVD and RD deaths and control strategies along with control regulations should be considered to control PM.

Deep learning detection of anomalous patterns from bus trajectories for traffic insight analysis

Existing data-driven methods for traffic anomaly detection are modeled on taxi trajectory datasets. The concern is that the data may contain much inaccuracy about the actual traffic situations, because taxi drivers often choose optimal routes to evade from the congestions caused by traffic anomalies. We use bus trajectory data in this work. Bus trajectories can capture real traffic conditions in the road networks without drivers’ preference, which are more objective and appropriate for accurately detecting anomalous patterns for a broad range of insight analyses on traffics. We proposed a deep learning-based feature visualization method to map 3-dimensional features into a red–green–blue (RGB) color space. A color trajectory (CT) is then derived by encoding a trajectory with the RGB colors. With the spatial and temporal properties extracted from the CT, spatio-temporal outliers are detected by a novel offline detection method. We then conduct GIS map fusion to obtain insights for better understanding the traffic anomaly locations, and more importantly the influences on the road affected by the corresponding anomalies. Extended from the offline detection, an online detection method is developed for real-time detection of anomalous patterns. Our proposed methods were tested on 3 real-world bus trajectory datasets to demonstrate the performance of high accuracies, high detection rates and relatively low false alarm rates.

Big spatio‐temporal data mining for emergency management information systems

Nowadays, spatio-temporal data analytics plays an important role in different applications based on geographic information system (GIS). One of such applications is emergency management information systems (EMISs) which process a huge amount of spatio-temporal data that can be considered as Big Data (BD). Effective EMIS data analysis will allow authorities to improve the management of emergency events, including those related to traffic problems (traffic accidents, traffic jams, etc.). However, the stack of modern technologies dealing with BD does not provide natively supported functions for spatio-temporal data analysis, so they have to be developed and implemented for EMIS. In this study, the authors developed and evaluated several algorithms and a framework for big spatio-temporal data mining in the Russian EMIS GLONASS + 112112. They suggested improved spatio-temporal co-location patterns mining technique based on DBSCAN, FPGrowth, and natural language processing (NLP). They detected spatio-temporal outliers and spatial autocorrelation. Finally, they evaluated the scalability and time performance of algorithms. They found that using NLP and aggregation of an emergency dataset with weather to enhance the number of emergency categories could significantly increase the quality of association rules. They discovered valuable association rules related to traffic accidents in the dedicated area of Kazan city.

Detecting grassland cover changes through spatiotemporal outlier analysis using remotely sensed time-series data: a case study from Xilingol, China

Significant land cover change (SLCC) in grassland ecosystem includes both conversions in land cover types and critical modifications to specific land cover properties without land cover conversions. A statistics-based framework, known as spatiotemporal outlier analysis (STOA), was proposed to detect SLCC from time-series remote sensing data considering both spatial and temporal contexts. The proposed STOA combines local spatial association analysis (spatial context) and temporal variation analysis (temporal context) to extract spatiotemporal outliers. As the case study, STOA was applied to mapping SLCC on the grassland of Xilinhot, China using time-series MODIS vegetation index during 2000–2015. The results clearly revealed the anisotropic characteristics and spatio-temporal variations in the extracted SLCC, demonstrating meaningful patterns in the land cover changes (LCC). Considering the scale-effect, it is inferred that the detected SLCC is most likely attributed to human-induced impact. We conclude that STOA is a promising tool to quantify LCC for grassland ecosystems.

Mapping the Changes in Urban Greenness Based on Localized Spatial Association Analysis under Temporal Context Using MODIS Data

Vegetation plays an irreplaceable role for urban ecosystem services. Urban greenness represents all vegetation cover in and around cities. Understanding spatiotemporal patterns of the changes in urban greenness (CUG) provides fundamental clues for urban planning. The impact on CUG can be roughly categorized as being climate-induced and human-induced. Methods for mapping human-induced CUG (H-CUG) are rare. In this paper, a new framework, known as Localized Spatial Association Analysis under Temporal Context (LSAA-TC), was proposed to explore H-CUG. Localized spatial association analysis (LSAA) was performed first to extract local spatial outliers (LSOs), or locations that differ significantly in urban greenness from those located in the neighborhood. LSOs were then analyzed under the temporal context to map their intertemporal variations known as spatiotemporal outliers. We applied LSAA-TC to mapping H-CUG in the Wuhan Metropolitan Area, China during 2000–2015 using the vegetation index from Moderate-resolution Imaging Spectroradiometer (MODIS) 13Q1 as the proxy for urban greenness. The computed H-CUG demonstrated apparent spatiotemporal patterns. The result is consistent with the fact that the traditional downtown area presents the lowest H-CUG, while it is found that the peripheral area in the circular belt within 14–20 km from the urban center demonstrates the most significant H-CUG. We conclude that LSAA-TC can be a widely applicable framework to understand H-CUG patterns and is a promising tool for informative urban planning.

A spatiotemporal outlier detection method based on partial least squares discriminant analysis and area Delaunay triangulation for image-based process monitoring

ABSTRACTOver the past two decades, statistical process control has evolved from monitoring individual data points to linear profiles to image data. Image sensors are now being deployed in complex systems at increasing rates due to the rich information they can provide. As a result, image data play an important role in process monitoring in different application domains ranging from manufacturing to service systems. Many of the existing process monitoring methods fail to take full advantage of the image data due to the data's complex nature in both the spatial and temporal domains. This article proposes a spatiotemporal outlier detection method based on the partial least squares discriminant analysis and a control statistic based on the area Delaunay triangulation of the squared prediction errors to improve the performance of an image-based monitoring scheme. First, the discriminant analysis of the partial least squares is used to efficiently extract the most important features from the high-dimensional image data to identify the benchmark images of the products and obtain the pixel value errors. Next, the squared errors resulting from the previous step are connected using a Delaunay triangulation to form a surface, the area of which is used as the control statistic for the purpose of outlier detection. A real case study at a paper product manufacturing company is used to compare the performance of the proposed method in detecting different types of outliers with some of the existing methods and demonstrate the merit of the proposed method.

Spatio-temporal outlier detection algorithms based on computing behavioral outlierness factor

A major task in spatio-temporal outlier detection is to identify objects that exhibit abnormal behavior either spatially, and/or temporally. There have only been a few algorithms proposed for detecting spatial and/or temporal outliers. One example is the Local Density-Based Spatial Clustering of Applications with Noise (LDBSCAN). Density-Based Spatial Clustering of Applications with Noise (DBSCAN) is mainly for clustering; it just tells us whether an object belongs to a cluster or it is an outlier. A measure known as Local Outlier Factor (LOF) gives a quantitative measure of outlierness to each object, where a high LOF score means it is potentially an outlier. LDBSCAN algorithm, which combines the above notions, considers only the spatial context. Furthermore, the notion of a cluster is defeated (i.e. LDBSCAN may report clusters having less than the minimum required points in a cluster), and some of the outliers may not be detected because of the limitation of the existing conditions in the LDBSCAN algorithm. In this paper, we propose two algorithms, namely Spatio-Temporal Behavioral Density-based Clustering of Applications with Noise (ST-BDBCAN) and Approx-ST-BDBCAN. ST-BDBCAN algorithm adopts the proposed, new concept, called Spatio-Temporal Behavioral Outlier Factor (ST-BOF), which is a spatio-temporal extension to LOF. It also uses both spatial and temporal attributes simultaneously to define the context. By doing so, the relative importance of spatial continuity or temporal continuity appropriate to the application at hand can be established. The Approx-ST-BDBCAN algorithm achieves improved scalability, with minimal loss of detection accuracy by partitioning data points for parallel processing. Experimental results on synthetic, and buoy datasets suggest that our proposed algorithms are accurate and computationally efficient. Additionally, new Outlier Association with Hurricane Intensity Index (OAHII) measures are introduced for quantitative evaluation of the results from buoy dataset.

Online Detection of Low-Quality Synchrophasor Measurements: A Data-Driven Approach

In this paper, an online data-driven approach is proposed for the detection of low-quality synchrophasor measurements. The proposed method leverages the spatio-temporal similarities among multiple-time-instant synchrophasor measurements and formulates the low-quality synchrophasor data as spatio-temporal outliers. A density-based local outlier detection technique is proposed to detect the spatio-temporal outliers. This data-driven approach involves no system modeling information. The detection algorithm can operate under both normal and fault-on system conditions, with fast computation speed suitable for online applications. Case studies on both synthetic and real-world synchrophasor data verify the effectiveness of the proposed approach.

ST TD outlier detection

Through vehicle-to-everything traffic information propagation often causes data outliers, due to data delay, data loss, inaccurate data and inconsistent data. Traffic data (TD) with outliers may incorrectly describe traffic conditions and decline the reliability and stability of transportation cyber physical system. This study develops some research approaches to detect spatiotemporal (ST) data outliers for the development of transportation systems. These research approaches include the theorisation of ST traffic outliers, the creation of an innovative firefly algorithm (IFA), the discussion of TD synchronisation methods and the development of the FA-based ST outlier detection mechanism (IFA-STODM). The experimental results show that the proposed IFA-STODM is an effective and efficient method for the detection of ST TD outliers.

基于DBSCAN聚类算法的异常轨迹检测

Existing traditional trajectory outlier detection algorithms always focus on spatial outliers and ignore temporal outliers, and the accuracy is relatively low. To solve these problems, a simple and effective approach based on enhanced clustering algorithm was proposed to detect spatio-temporal trajectory outliers. Firstly, each original trajectory was simplified into a set of sequential line segments with the velocity-based minimum description length (VMDL) partition principle. Secondly, the distance formula between line segments was improved to enhance the clustering performance. Using DBSCAN algorithm, the line segments were classified into different groups which could represent local normal behaviors. Thirdly, outliers were detected using two-level detection algorithm which first detected spatial outliers and then detected temporal outliers. Experimental results on multiple trajectory data sets demonstrate that the proposed algorithm could successfully detect three kinds of spatio-temporal outliers, position, angle and velocity. Compared with other methods, the precision and accuracy make great improvement.

Identification and influence of spatio-temporal outliers in urban air quality measurements

Forty eight potential outliers in air pollution measurements taken simultaneously in Detroit, Michigan, USA and Windsor, Ontario, Canada in 2008 and 2009 were identified using four independent methods: box plots, variogram clouds, difference maps, and the Local Moran's I statistic. These methods were subsequently used in combination to reduce and select a final set of 13 outliers for nitrogen dioxide (NO2), volatile organic compounds (VOCs), total benzene, toluene, ethyl benzene, and xylene (BTEX), and particulate matter in two size fractions (PM2.5 and PM10). The selected outliers were excluded from the measurement datasets and used to revise air pollution models. In addition, a set of temporally-scaled air pollution models was generated using time series measurements from community air quality monitors, with and without the selected outliers. The influence of outlier exclusion on associations with asthma exacerbation rates aggregated at a postal zone scale in both cities was evaluated.Results demonstrate that the inclusion or exclusion of outliers influences the strength of observed associations between intraurban air quality and asthma exacerbation in both cities. The box plot, variogram cloud, and difference map methods largely determined the final list of outliers, due to the high degree of conformity among their results. The Moran's I approach was not useful for outlier identification in the datasets studied. Removing outliers changed the spatial distribution of modeled concentration values and derivative exposure estimates averaged over postal zones. Overall, associations between air pollution and acute asthma exacerbation rates were weaker with outliers removed, but improved with the addition of temporal information. Decreases in statistically significant associations between air pollution and asthma resulted, in part, from smaller pollutant concentration ranges used for linear regression. Nevertheless, the practice of identifying outliers through congruence among multiple methods strengthens confidence in the analysis of outlier presence and influence in environmental datasets.

Spatiotemporal Outlier Detection: Did Buoys Tell Where the Hurricanes Were?

ABSTRACTData points that exhibit abnormal behavior, either spatially, temporally, or both, are considered spatiotemporal outliers. Spatiotemporal outlier detection is important for the discovery of exceptional events due to the rapidly increasing amount of spatiotemporal data available, and the need to understand such data. A tropical cyclone system or a hurricane can be considered an abnormal activity of the atmosphere system. Discovery of such an abnormality usually leverages data from a satellite or radar. Not many people have thought about using a weather buoy, a floating device that provides meteorological and environmental information in real time for open ocean and coastal zones. The aim of this research is to see if a spatiotemporal outlier approach can help to discover the evolution and movement of the hurricane system from weather buoy observations. This article leverages an algorithm, spatiotemporal local density-based clustering of applications with noise (ST-LDBCAN), which has been developed and used by the authors to detect outliers in various scenarios. The ST-LDBCAN has a novel way of defining spatiotemporal context and can handle multivariate data, which is its advantage over existing algorithms. The results show a good correlation between detected spatiotemporal outliers and the paths and evolution of Hurricanes Katrina and Gustav.

Spatiotemporal Data Mining: A Computational Perspective

Explosive growth in geospatial and temporal data as well as the emergence of new technologies emphasize the need for automated discovery of spatiotemporal knowledge. Spatiotemporal data mining studies the process of discovering interesting and previously unknown, but potentially useful patterns from large spatiotemporal databases. It has broad application domains including ecology and environmental management, public safety, transportation, earth science, epidemiology, and climatology. The complexity of spatiotemporal data and intrinsic relationships limits the usefulness of conventional data science techniques for extracting spatiotemporal patterns. In this survey, we review recent computational techniques and tools in spatiotemporal data mining, focusing on several major pattern families: spatiotemporal outlier, spatiotemporal coupling and tele-coupling, spatiotemporal prediction, spatiotemporal partitioning and summarization, spatiotemporal hotspots, and change detection. Compared with other surveys in the literature, this paper emphasizes the statistical foundations of spatiotemporal data mining and provides comprehensive coverage of computational approaches for various pattern families. ISPRS Int. J. Geo-Inf. 2015, 4 2307 We also list popular software tools for spatiotemporal data analysis. The survey concludes with a look at future research needs.

Erratum to: A fuzzy index for detecting spatiotemporal outliers

Erratum to: A fuzzy index for detecting spatiotemporal outliers

A fuzzy index for detecting spatiotemporal outliers

The detection of spatial outliers helps extract important and valuable information from large spatial datasets. Most of the existing work in outlier detection views the condition of being an outlier as a binary property. However, for many scenarios, it is more meaningful to assign a degree of being an outlier to each object. The temporal dimension should also be taken into consideration. In this paper, we formally introduce a new notion of spatial outliers. We discuss the spatiotemporal outlier detection problem, and we design a methodology to discover these outliers effectively. We introduce a new index called the fuzzy outlier index, FoI, which expresses the degree to which a spatial object belongs to a spatiotemporal neighbourhood. The proposed outlier detection method can be applied to phenomena evolving over time, such as moving objects, pedestrian modelling or credit card fraud.

Spatio-Temporal outliers detection within the space-time framework

Spatio-Temporal outliers detection within the space-time framework

Toward an interactive system for checking spatio-temporal data quality

The wide dissemination and use of statistics about territories has induced a need for more validation and understanding about those data. We propose an interface that eases the exploration of spatio-temporal data using an interactive cartographic tool by showing data with their metadata simultaneously. This system helps to check the consistency of a set of data through the use of known and efficient methods for detecting spatio-temporal outliers. Finally, on the basis of those observations, and of their own knowledge about the data, experts can generate and export a report on data quality, conform to ISO 19115 standard for geographic metadata.

Spatio-Temporal Outlier Detection in Large Databases

Outlier detection is one of the major data mining methods. This paper proposes a three-step approach to detect spatio-temporal outliers in large databases. These steps are clustering, checking spatial neighbors, and checking temporal neighbors. In this paper, we introduce a new outlier detection algorithm to find small groups of data objects that are exceptional when compared with the remaining large amount of data. In contrast to the existing outlier detection algorithms, the new algorithm has the ability of discovering outliers according to the non-spatial, spatial and temporal values of the objects. In order to demonstrate the new algorithm, this paper also presents an example of application using a data warehouse.