Real-world Data Streams Research Articles

Exemplar-based data stream clustering toward Internet of Things

Dealing with dynamic data stream has become one of the most active research fields for Internet of Things (IoT). Specifically, clustering toward dynamic data stream is a necessary foundation for numerous IoT platforms. In this paper, we focus on dynamic exemplar-based clustering models. In terms of the maximum a priori principle, under the probability framework, we first summarize a unified explanation for two typical exemplar-based clustering models, namely enhanced $$\alpha$$-expansion move (EEM) and affinity propagation (AP). Then, a new dynamic exemplar-based data stream clustering algorithm called DSC is proposed accordingly. The distinctive merit of the proposed algorithm DSC is that we can simply utilize the framework of EEM algorithm through modifying the definitions of several variables and do not need to design another optimization mechanism. Moreover, algorithm DSC is capable of dealing to two cases of similarities. In contrast to both AP and EEM, our experimental results indicate the power of algorithm DSC for real-world IoT data streams.

PALM: An Incremental Construction of Hyperplanes for Data Stream Regression

Data stream has been the underlying challenge in the age of big data because it calls for real-time data processing with the absence of a retraining process and/or an iterative learning approach. In the realm of the fuzzy system community, data stream is handled by algorithmic development of self-adaptive neuro-fuzzy systems (SANFS) characterized by the single-pass learning mode and the open structure property that enables effective handling of fast and rapidly changing natures of data streams. The underlying bottleneck of SANFSs lies in its design principle, which involves a high number of free parameters (rule premise and rule consequent) to be adapted in the training process. This figure can even double in the case of the type-2 fuzzy system. In this paper, a novel SANFS, namely parsimonious learning machine (PALM), is proposed. PALM features utilization of a new type of fuzzy rule based on the concept of hyperplane clustering, which significantly reduces the number of network parameters because it has no rule premise parameters. PALM is proposed in both type-1 and type-2 fuzzy systems where all of which characterize a fully dynamic rule-based system. That is, it is capable of automatically generating, merging, and tuning the hyperplane-based fuzzy rule in the single-pass manner. Moreover, an extension of PALM, namely recurrent PALM, is proposed and adopts the concept of teacher-forcing mechanism in the deep learning literature. The efficacy of PALM has been evaluated through numerical study with six real-world and synthetic data streams from public database and our own real-world project of autonomous vehicles. The proposed model showcases significant improvements in terms of computational complexity and number of required parameters against several renowned SANFSs, while attaining comparable and often better predictive accuracy.

Skyline queries over incomplete data streams

Nowadays, efficient and effective processing over massive stream data has attracted much attention from the database community, which are useful in many real applications such as sensor data monitoring, network intrusion detection, and so on. In practice, due to the malfunction of sensing devices or imperfect data collection techniques, real-world stream data may often contain missing or incomplete data attributes. In this paper, we will formalize and tackle a novel and important problem, named skyline query over incomplete data stream (Sky-iDS), which retrieves skyline objects (in the presence of missing attributes) with high confidences from incomplete data stream. In order to tackle the Sky-iDS problem, we will design efficient approaches to impute missing attributes of objects from incomplete data stream via differential dependency (DD) rules. We will propose effective pruning strategies to reduce the search space of the Sky-iDS problem, devise cost-model-based index structures to facilitate the data imputation and skyline computation at the same time, and integrate our proposed techniques into an efficient Sky-iDS query answering algorithm. Extensive experiments have been conducted to confirm the efficiency and effectiveness of our Sky-iDS processing approach over both real and synthetic data sets.

A Content Level Based Deduplication on Streaming Data using Poisson Process Filter Technique (PPFT)

This paper proposed on Poisson process-based algorithm is to carry out content-level deduplication for the streaming data. Since Poisson processes are meant to do the counting of different events happening over a period of time and space, it becomes appropriate to use it for identifying duplications of data as it gets streamed based on time and space, which can allow the deduplication process to be carried out in tandem. Some of the research on deduplication has been focusing on File-level and Block-level deduplication while the focus can be brought to content-level, as data get streamed lively and becomes more dynamic. With this approach, the content-level deduplication will allow the data to be scanned intelligently and at the same time, it can save the deduplication operation time. Also, streaming data has its randomness which is innately there and by having Poisson process based deduplication it will address the random behaviour of the data transfer and can work efficiently in the dynamically connected environment. The proposed method identifies the unique data to store in the Database. Based on the experimental result, the Poisson Process-based algorithm produce 0.912 Area Under Curve (AUC) accuracy on real-world streaming data, which means that if AUC is greater than 0.8 then the performance of algorithm is pretty good. So, the machine intelligence-based deduplication model produced reliable and robust deduplication on streaming data compared to existing approaches.

A heterogeneous online learning ensemble for non-stationary environments

Learning in non-stationary environments is a challenging task which requires the updating of predictive models to deal with changes in the underlying probability distribution of the problem, i.e., dealing with concept drift. Most work in this area is concerned with updating the learning system so that it can quickly recover from concept drift, while little work has been dedicated to investigating what type of predictive model is most suitable at any given time. This paper aims to investigate the benefits of online model selection for predictive modelling in non-stationary environments. A novel heterogeneous ensemble approach is proposed to intelligently switch between different types of base models in an ensemble to increase the predictive performance of online learning in non-stationary environments. This approach is Heterogeneous Dynamic Weighted Majority (HDWM). It makes use of “seed” learners of different types to maintain ensemble diversity, overcoming problems of existing dynamic ensembles that may undergo loss of diversity due to the exclusion of base learners. The algorithm has been evaluated on artificial and real-world data streams against existing well-known approaches such as a heterogeneous Weighted Majority Algorithm (WMA) and a homogeneous Dynamic Weighted Majority (DWM). The results show that HDWM performed significantly better than WMA in non-stationary environments. Also, when recurring concept drifts were present, the predictive performance of HDWM showed an improvement over DWM.

Developing an online general type-2 fuzzy classifier using evolving type-1 rules

General type-2 fuzzy systems have been shown to handle more levels of uncertainty present in the majority of real-world applications. Nevertheless, the rapid growth of information generation does not allow utilizing general type-2 models for their complex learning process. This paper introduces a novel online general type-2 fuzzy classifier (called oGT2FC). It aims to reduce the computations needed to get the type-2 fuzzy sets. As in most online and evolving fuzzy schemes, the initial rule-base in oGT2FC is empty, and then the fuzzy rules are generated in completely online manner; without storing the samples. To specify the type-2 fuzzy sets, oGT2FC employs some experts' opinions, drawn from training data, to generate automatically some diverse type-1 fuzzy rule-bases. These type-1 rule-bases are updated/evolved by incoming new samples and are used to construct the general type-2 model. By defining a type-2 fuzzy set as the union of vertical slices, oGT2FC performs the type reduction in a fast and efficient manner. The efficiency of the proposed oGT2FC is assessed experimentally, using synthetic and real-world data streams, via comparing with other type-2 and type-1 evolving fuzzy classifiers as well as some state-of-the-art incremental algorithms. In addition, oGT2FC is compared against some fuzzy classifiers in its ability to model uncertainty.

A non-canonical hybrid metaheuristic approach to adaptive data stream classification

Data stream classification techniques have been playing an important role in big data analytics recently due to their diverse applications (e.g. fraud and intrusion detection, forecasting and healthcare monitoring systems) and the growing number of real-world data stream generators (e.g. IoT devices and sensors, websites and social network feeds). Streaming data is often prone to evolution over time. In this context, the main challenge for computational models is to adapt to changes, known as concept drifts, using data mining and optimisation techniques. We present a novel ensemble technique called RED-PSO that seamlessly adapts to different concept drifts in non-stationary data stream classification tasks. RED-PSO is based on a three-layer architecture to produce classification types of different size, each created by randomly selecting a certain percentage of features from a pool of features of the target data stream. An evolutionary algorithm, namely, Replicator Dynamics (RD), is used to seamlessly adapt to different concept drifts; it allows good performing types to grow and poor performing ones to shrink in size. In addition, the selected feature combinations in all classification types are optimised using a non-canonical version of the Particle Swarm Optimisation (PSO) technique for each layer individually. PSO allows the types in each layer to go towards local (within the same type) and global (in all types) optimums with a specified velocity. A set of experiments are conducted to compare the performance of the proposed method to state-of-the-art algorithms using real-world and synthetic data streams in immediate and delayed prequential evaluation settings. The results show a favourable performance of our method in different environments.

NETS

This paper addresses the problem of efficiently detecting outliers from a data stream as old data points expire from and new data points enter the window incrementally. The proposed method is based on a newly discovered characteristic of a data stream that the change in the locations of data points in the data space is typically very insignificant. This observation has led to the finding that the existing distance-based outlier detection algorithms perform excessive unnecessary computations that are repetitive and/or canceling out the effects. Thus, in this paper, we propose a novel set-based approach to detecting outliers, whereby data points at similar locations are grouped and the detection of outliers or inliers is handled at the group level. Specifically, a new algorithm NETS is proposed to achieve a remarkable performance improvement by realizing set-based early identification of outliers or inliers and taking advantage of the "net effect" between expired and new data points. Additionally, NETS is capable of achieving the same efficiency even for a high-dimensional data stream through two-level dimensional filtering . Comprehensive experiments using six real-world data streams show 5 to 25 times faster processing time than state-of-the-art algorithms with comparable memory consumption. We assert that NETS opens a new possibility to real-time data stream outlier detection.

DG2CEP: a near real-time on-line algorithm for detecting spatial clusters large data streams through complex event processing

Spatial concentrations (or spatial clusters) of moving objects, such as vehicles and humans, is a mobility pattern that is relevant to many applications. Fast detection of this pattern and its evolution, e.g., if the cluster is shrinking or growing, is useful in numerous scenarios, such as detecting the formation of traffic jams or detecting a fast dispersion of people in a music concert. On-Line detection of this pattern is a challenging task because it requires algorithms that are capable of continuously and efficiently processing the high volume of position updates in a timely manner. Currently, the majority of approaches for spatial cluster detection operate in batch mode, where moving objects location updates are recorded during time periods of a certain length and then batch-processed by an external routine, thus delaying the result of the cluster detection until the end of the time period. Further, they extensively use spatial data structures and operators, which can be troublesome to maintain or parallelize in on-line scenarios. To address these issues, in this paper we propose DG2CEP, a parallel algorithm that combines the well-known density-based clustering algorithm DBSCAN with the data stream processing paradigm Complex Event Processing (CEP) to achieve continuous and timely detection of spatial clusters. Our experiments with real-world data streams indicate that DG2CEP is able to detect the formation and dispersion of clusters with small latency while having higher similarity to DBSCAN than batch-based approaches.

EACD: evolutionary adaptation to concept drifts in data streams

This paper presents a novel ensemble learning method based on evolutionary algorithms to cope with different types of concept drifts in non-stationary data stream classification tasks. In ensemble learning, multiple learners forming an ensemble are trained to obtain a better predictive performance compared to that of a single learner, especially in non-stationary environments, where data evolve over time. The evolution of data streams can be viewed as a problem of changing environment, and evolutionary algorithms offer a natural solution to this problem. The method proposed in this paper uses random subspaces of features from a pool of features to create different classification types in the ensemble. Each such type consists of a limited number of classifiers (decision trees) that have been built at different times over the data stream. An evolutionary algorithm (replicator dynamics) is used to adapt to different concept drifts; it allows the types with a higher performance to increase and those with a lower performance to decrease in size. Genetic algorithm is then applied to build a two-layer architecture based on the proposed technique to dynamically optimise the combination of features in each type to achieve a better adaptation to new concepts. The proposed method, called EACD, offers both implicit and explicit mechanisms to deal with concept drifts. A set of experiments employing four artificial and five real-world data streams is conducted to compare its performance with that of the state-of-the-art algorithms using the immediate and delayed prequential evaluation methods. The results demonstrate favourable performance of the proposed EACD method in different environments.

Resample-Based Ensemble Framework for Drifting Imbalanced Data Streams

Machine learning in real-world scenarios is often challenged by concept drift and class imbalance. This paper proposes a Resample-based Ensemble Framework for Drifting Imbalanced Stream (RE-DI). The ensemble framework consists of a long-term static classifier to handle gradual and multiple dynamic classifiers to handle sudden concept drift. The weights of the ensemble classifier are adjusted from two aspects. First, a time-decayed strategy decreases the weights of the dynamic classifiers to make the ensemble classifier focus more on the new concept of the data stream. Second, a novel reinforcement mechanism is proposed to increase the weights of the base classifiers that perform better on the minority class and decrease the weights of the classifiers that perform worse. A resampling buffer is used for storing the instances of the minority class to balance the imbalanced distribution over time. In our experiment, we compare the proposed method with other state-of-the-art algorithms on both real-world and synthetic data streams. The results show that the proposed method achieves the best performance in terms of both Prequential AUC and accuracy.

Online Tool Condition Monitoring Based on Parsimonious Ensemble.

Existing methodologies for tool condition monitoring (TCM) still rely on batch approaches which cannot cope with a fast sampling rate of a metal cutting process. Furthermore, they require a retraining process to be completed from scratch when dealing with a new set of machining parameters. This paper presents an online TCM approach based on Parsimonious Ensemble+ (pENsemble+). The unique feature of pENsemble+ lies in its highly flexible principle where both the ensemble structure and base-classifier structure can automatically grow and shrink on the fly based on the characteristics of data streams. Moreover, the online feature selection scenario is integrated to actively sample relevant input attributes. This paper presents advancement of a newly developed ensemble learning algorithm, pENsemble, where the online active learning scenario is incorporated to reduce the operator's labeling effort. The ensemble merging scenario is proposed which allows reduction of ensemble complexity while retaining its diversity. Experimental studies utilizing two real-world manufacturing data streams: 1) metal turning and 2) 3-D-printing processes and comparisons with well-known algorithms were carried out. Furthermore, the efficacy of pENsemble+ was examined using benchmark concept drift data streams. It has been found that pENsemble+ incurs low structural complexity and results in a significant reduction of the operator's labeling effort.

A novel approach using incremental oversampling for data stream mining

Data stream mining is very popular in recent years with advanced electronic devices generating continuous data streams. The performance of standard learning algorithms is been compromised with imbalance nature present in real world data streams. In this paper we propose a novel algorithm dubbed as increment over sampling for data streams (IOSDS) which uses an unique over sampling technique to almost balance the data sets to minimize the effect of imbalance in stream mining process. The experimental analysis is conducted on 15 data chunks of data streams with varied sizes and different imbalance ratios. The results suggests that the proposed IOSDS algorithm improves the knowledge discovery over benchmark algorithms like C4.5 and Hoeffding tree in terms of standard performance measures namely accuracy, AUC, precision, recall and F-measure.

Mining microscopic and macroscopic changes in network data streams

Network data streams are unbounded sequences of complex data produced at high rate which represent complex systems that evolve continuously over time. In this scenario, a problem worthy of being studied is the analysis of the changes, which may concern a complex system as a whole or small parts of it. In this paper, these are distinguished into macroscopic changes and microscopic changes: macroscopic changes have impact on a substantial part of the network, whereas microscopic changes concern variations occurring in specific portions of the network. The algorithm we propose, called KARMA, combines the frequent pattern mining framework with an automatic time-window detection approach. In this way, it is able to detect changes on the frequent subnetworks mined from different time-windows: network changes are then represented as variations of structural regularities frequently observed over the stream. KARMA takes an holistic perspective, in which the two kinds of change are related each other. This is the main novelty with respect to the recent studies, which do not simultaneously extract microscopic and macroscopic changes. Experiments on several real-world network data streams show the effectiveness and efficiency of our approach in comparison with competing algorithms and the usefulness of the changes detected.

An effective pattern-based Bayesian classifier for evolving data stream

Abstract One of the hot topics in graph-based machine learning is to build Bayesian classifier from large-scale dataset. An advanced approach to Bayesian classification is based on exploited patterns. However, traditional pattern-based Bayesian classifiers cannot adapt to the evolving data stream environment. For that, an effective Pattern-based Bayesian classifier for Data Stream (PBDS) is proposed. First, a data-driven lazy learning strategy is employed to discover local frequent patterns for each test record. Furthermore, we propose a summary data structure for compact representation of data, and to find patterns more efficiently for each class. Greedy search and minimum description length combined with Bayesian network are applied to evaluating extracted patterns. Experimental studies on real-world and synthetic data streams show that PBDS outperforms most state-of-the-art data stream classifiers.

Analysing real world data streams with spatio-temporal correlations: Entropy vs. Pearson correlation

Smart Cities use different Internet of Things (IoT) data sources and rely on big data analytics to obtain information or extract actionable knowledge crucial for urban planners for efficiently use and plan the construction infrastructures. Big data analytics algorithms often consider the correlation of different patterns and various data types. However, the use of different techniques to measure the correlation with smart cities data and the exploitation of correlations to infer new knowledge are still open questions. This paper proposes a methodology to analyse data streams, based on spatio-temporal correlations using different correlation algorithms and provides a discussion on co-occurrence vs. causation. The proposed method is evaluated using traffic data collected from the road sensors in the city of Aarhus in Denmark.

Concept Drift Adaptation by Exploiting Historical Knowledge.

Incremental learning with concept drift has often been tackled by ensemble methods, where models built in the past can be retrained to attain new models for the current data. Two design questions need to be addressed in developing ensemble methods for incremental learning with concept drift, i.e., which historical (i.e., previously trained) models should be preserved and how to utilize them. A novel ensemble learning method, namely, Diversity and Transfer-based Ensemble Learning (DTEL), is proposed in this paper. Given newly arrived data, DTEL uses each preserved historical model as an initial model and further trains it with the new data via transfer learning. Furthermore, DTEL preserves a diverse set of historical models, rather than a set of historical models that are merely accurate in terms of classification accuracy. Empirical studies on 15 synthetic data streams and 5 real-world data streams (all with concept drifts) demonstrate that DTEL can handle concept drift more effectively than 4 other state-of-the-art methods.

Parsimonious random vector functional link network for data streams

The majority of the existing work on random vector functional link networks (RVFLNs) is not scalable for data stream analytics because they work under a batch learning scenario and lack a self-organizing property. A novel RVLFN, namely the parsimonious random vector functional link network (pRVFLN), is proposed in this paper. pRVFLN adopts a fully flexible and adaptive working principle where its network structure can be configured from scratch and can be automatically generated, pruned and recalled from data streams. pRVFLN is capable of selecting and deselecting input attributes on the fly as well as capable of extracting important training samples for model updates. In addition, pRVFLN introduces a non-parametric type of hidden node which completely reflects the real data distribution and is not constrained by a specific shape of the cluster. All learning procedures of pRVFLN follow a strictly single-pass learning mode, which is applicable for online time-critical applications. The advantage of pRVFLN is verified through numerous simulations with real-world data streams. It was benchmarked against recently published algorithms where it demonstrated comparable and even higher predictive accuracies while imposing the lowest complexities.

Generating High-Dimensional Datastreams for Change Detection

Abstract A popular testbed for change-detection algorithms consists in detecting changes that have been synthetically injected in real-world datastreams. Unfortunately, most of experimental practices in the literature lead to injecting changes whose magnitude is unknown and can not be controlled. As a consequence, results are difficult to interpret, reproduce, and compare with. Most importantly, controlling the change magnitude is a primary requirement to investigate the change-detection performance when data dimension scales, which is an issue to be typically addressed in big data scenarios. Here we present a best practice to inject changes in multivariate/high-dimensional datastreams: “Controlling Change Magnitude” (CCM) is a rigorous method to generate datastreams affected by a change having a desired magnitude at a known location. In CCM, changes are introduced by directly applying a roto-translation to the data, and the change magnitude is measured by the symmetric Kullback–Leibler divergence between the pre- and post-change data distributions. The roto-translation parameters yielding the desired change magnitude are identified by two iterative algorithms whose convergence is here proven. Our experiments show that CCM can effectively control the change magnitude in real-world datastreams, while traditional experimental practices might not be appropriate for assessing the performance of change-detection algorithms in high-dimensional data.

An Incremental Learning of Concept Drifts Using Evolving Type-2 Recurrent Fuzzy Neural Networks

The age of online data stream and dynamic environments results in the increasing demand of advanced machine learning techniques to deal with concept drifts in large data streams. Evolving fuzzy systems (EFS) are one of recent initiatives from the fuzzy system community to resolve the issue. Existing EFSs are not robust against data uncertainty, temporal system dynamics, and the absence of system order, because a vast majority of EFSs are designed in the type-1 feedforward network architecture. This paper aims to solve the issue of data uncertainty, temporal behavior, and the absence of system order by developing a novel evolving recurrent fuzzy neural network, called evolving type-2 recurrent fuzzy neural network (eT2RFNN). eT2RFNN is constructed in a new recurrent network architecture, featuring double recurrent layers. The new recurrent network architecture evolves a generalized interval type-2 fuzzy rule, where the rule premise is built upon the interval type-2 multivariate Gaussian function, whereas the rule consequent is crafted by the nonlinear wavelet function. The eT2RFNN adopts a holistic concept of evolving systems, where the fuzzy rule can be automatically generated, pruned, merged, and recalled in the single-pass learning mode. eT2RFNN is capable of coping with the problem of high dimensionality because it is equipped with online feature selection technology. The efficacy of eT2RFNN was experimentally validated using artificial and real-world data streams and compared with prominent learning algorithms. eT2RFNN produced more reliable predictive accuracy, while retaining lower complexity than its counterparts.