Data Stream Mining Research Articles

Analytical Engineering for Data Stream

The analytical capacity of massive data has become increasingly necessary, given the high volume of data that has been generated daily by different sources. The data sources are varied and can generate a huge amount of data, which can be processed in batch or stream settings. The stream setting corresponds to the treatment of a continuous sequence of data that arrives in real-time flow and needs to be processed in real-time. The models, tools, methods and algorithms for generating intelligence from data stream culminate in the approaches of Data Stream Mining and Data Stream Learning. The activities of such approaches can be organized and structured according to Engineering principles, thus allowing the principles of Analytical Engineering, or more specifically, Analytical Engineering for Data Stream (AEDS). Thus, this article presents the AEDS conceptual framework composed of four pillars (Data, Model, Tool, People) and three processes (Acquisition, Retention, Review). The definition of these pillars and processes is carried out based on the main components of data stream setting, corresponding to four pillars, and also on the necessity to operationalize the activities of an Analytical Organization (AO) in the use of AEDS four pillars, which determines the three proposed processes. The AEDS framework favors the projects carried out in an AO, that is, its Analytical Projects (AP), to favor the delivery of results, or Analytical Deliverables (AD), carried out by the Analytical Teams (AT) in order to provide intelligence from stream data.

A Clustering Algorithm in Stream Data Using Strong Coreset

There are various applications of clustering in the fields of machine learning, data mining, data compression along with pattern recognition. The existent techniques like the Llyods algorithm (sometimes called k-means) were affected by the issue of the algorithm which converges to a local optimum along with no approximation guarantee. For overcoming these shortcomings, an efficient k-means clustering approach is offered by this paper for stream data mining. Coreset is a popular and fundamental concept for k-means clustering in stream data. In each step, reduction determines a coreset of inputs, and represents the error, where P represents number of input points according to nested property of coreset. Hence, a bit reduction in error of final coreset gets n times more accurate. Therefore, this motivated the author to propose a new coreset-reduction algorithm. The proposed algorithm executed on the Covertype dataset, Spambase dataset, Census 1990 dataset, Bigcross dataset, and Tower dataset. Our algorithm outperforms with competitive algorithms like Streamkm++, BICO (BIRCH meets Coresets for k-means clustering), and BIRCH (Balance Iterative Reducing and Clustering using Hierarchies.

Data Stream Mining Between Classical and Modern Applications: A Review

Data mining (DM) is an amazing innovation with incredible potential to help organizations centre on the main data in the information they have gathered about the conduct of their clients and expected clients. It finds data inside the information that questions and reports can't viably uncover. For the most part, DM is the way toward examining information from alternate points of view and summing up it into helpful data - data that can be utilized to expand income, reduces expenses, or both. There are four types of DM: 1) Classification and regression, 2) Clustering, 3) Association Rule Mining, and 4) Outlier/Anomaly Detection. Tending to the velocity part of Big Data (BD) has as of late pulled in a lot of revenue in the investigation local area because of its critical effect on information from pretty much every area of life like medical services, financial exchange, and interpersonal organizations, and so on. Many research works have investigated this velocity issue through mining data streams. Most existing data stream mining research centres on adjusting the primary classifications of approaches, methods and methods for static information to the dynamic information circumstance. This research explores widely the current writing in the field of data stream mining and recognizes the fundamental preparing units supporting different existing methods. This study not simply benefits examiner to make strong assessment subjects and separate gaps in the field yet moreover helps specialists for DM and BD application structure headway.

Efficient strategies for incremental mining of frequent closed itemsets over data streams

Mining frequent closed itemsets over data streams is an important data mining problem. Mining data streams is more challenging than mining static data because of the nature of data streams, including high arrival rate, massive volume of incoming data, and concept drift. The existing algorithms for mining frequent closed itemsets over data streams suffer from scalability and efficiency bottlenecks. This paper proposes a novel algorithm for mining frequent closed itemsets over data streams both for the sliding window model and for the landmark model. An indexed prefix closed itemset tree is proposed for compressing all closed itemsets and for quick searching of closed itemsets, and novel search strategies are proposed to prune the search space in updating the set of closed itemsets. The proposed algorithm outperforms the state-of-the-art intersection-based algorithms, CICLAD, ConPatSet, and CloStream, by several times to 2 orders of magnitude in efficiency, and also outperforms the state-of-the-art pattern enumeration algorithm, Moment, by up to 2 orders of magnitude over data streams with large windows and sparse data streams. The proposed algorithm is also superior in scalability.

Concept drift type identification based on multi-sliding windows

Concept drift is a common and important issue in streaming data analysis and mining. Thus far, many concept drift detection methods have been proposed but may not be able to identify the type of concept drift, which will result in some difficulties, such as extracting the wrong key information, inadequate model learning and poor detection efficiency. To solve these problems, a concept drift type identification method is proposed based on multi-sliding windows (CDT_MSW). This method consists of three processes. During the first detection process, the drift position is detected by sliding the basic window forward. Then, in the growth process, the drift length is detected using the growth of the adjoint window, and the drift category is identified according to the drift length. Finally, during tracking process, the drift subcategory can be accurately identified according to the different tracking flow ratio curves generated during window tracking. Experimental results show that the proposed method can effectively identify the type of concept drift, accurately analyze the key information during online learning and improve the efficiency and generalization performance of streaming data analysis and mining.

Time Series Segmentation Based on Stationarity Analysis to Improve New Samples Prediction.

A wide range of applications based on sequential data, named time series, have become increasingly popular in recent years, mainly those based on the Internet of Things (IoT). Several different machine learning algorithms exploit the patterns extracted from sequential data to support multiple tasks. However, this data can suffer from unreliable readings that can lead to low accuracy models due to the low-quality training sets available. Detecting the change point between high representative segments is an important ally to find and thread biased subsequences. By constructing a framework based on the Augmented Dickey-Fuller (ADF) test for data stationarity, two proposals to automatically segment subsequences in a time series were developed. The former proposal, called Change Detector segmentation, relies on change detection methods of data stream mining. The latter, called ADF-based segmentation, is constructed on a new change detector derived from the ADF test only. Experiments over real-file IoT databases and benchmarks showed the improvement provided by our proposals for prediction tasks with traditional Autoregressive integrated moving average (ARIMA) and Deep Learning (Long short-term memory and Temporal Convolutional Networks) methods. Results obtained by the Long short-term memory predictive model reduced the relative prediction error from 1 to 0.67, compared to time series without segmentation.

Hierarchical Multiresolution Representation of Streaming Time Series

• Generic time series representation model supporting streaming time series data mining at multiple time resolutions. • New representation based on the proposed model, considering fluctuations and continuity of modeled processes. • The application of the proposed model leads to higher and more stable processing speed at lower RAM consumption. • The proposed representation improves subsequent time series clustering. Real-time monitoring, analysis and operations in large industrial systems require an accurate but compact data model created on the basis of a large number of data sources continuously generating massive amounts of data modeled as streaming time series. This paper proposes a generic time series representation approach for reducing data model size and supporting streaming time series data mining at multiple time resolutions. The proposed Hierarchical Multiresolution Time Series Representation model utilizes a buffer-based approach that combines one-pass stream processing with hierarchical aggregation to achieve high processing speed without excessive hardware requirements. In addition, this paper presents a new representation based on the proposed model, Hierarchical Multiresolution Linear-function-based Piecewise Statistical Approximation. The proposed representation considers fluctuations and continuity of modeled processes in order to preserve fundamental characteristics of time series at reduced dimensionality. The usefulness of the proposed solution was proven in a case study. The case study results for generated data set confirm that the proposed model leads to higher and more stable processing speed at lower RAM consumption comparing to related model, especially when dealing with greater number of time resolutions. The case study results for real UK smart meter data confirm that the proposed representation leads to a reduced amount of information loss and an improvement in subsequent time series clustering comparing to related time series representation. Therefore, this paper's main contribution is multiresolution streaming time series data mining support convenient for application in large industrial systems.

CURIE: a cellular automaton for concept drift detection

Data stream mining extracts information from large quantities of data flowing fast and continuously (data streams). They are usually affected by changes in the data distribution, giving rise to a phenomenon referred to as concept drift. Thus, learning models must detect and adapt to such changes, so as to exhibit a good predictive performance after a drift has occurred. In this regard, the development of effective drift detection algorithms becomes a key factor in data stream mining. In this work we propose $$\textit{CURIE}$$ , a drift detector relying on cellular automata. Specifically, in $$\textit{CURIE}$$ the distribution of the data stream is represented in the grid of a cellular automata, whose neighborhood rule can then be utilized to detect possible distribution changes over the stream. Computer simulations are presented and discussed to show that $$\textit{CURIE}$$ , when hybridized with other base learners, renders a competitive behavior in terms of detection metrics and classification accuracy. $$\textit{CURIE}$$ is compared with well-established drift detectors over synthetic datasets with varying drift characteristics.

Design of adaptive ensemble classifier for online sentiment analysis and opinion mining.

DataStream mining is a challenging task for researchers because of the change in data distribution during classification, known as concept drift. Drift detection algorithms emphasize detecting the drift. The drift detection algorithm needs to be very sensitive to change in data distribution for detecting the maximum number of drifts in the data stream. But highly sensitive drift detectors lead to higher false-positive drift detections. This paper proposed a Drift Detection-based Adaptive Ensemble classifier for sentiment analysis and opinion mining, which uses these false-positive drift detections to benefit and minimize the negative impact of false-positive drift detection signals. The proposed method creates and adds a new classifier to the ensemble whenever a drift happens. A weighting mechanism is implemented, which provides weights to each classifier in the ensemble. The weight of the classifier decides the contribution of each classifier in the final classification results. The experiments are performed using different classification algorithms, and results are evaluated on the accuracy, precision, recall, and F1-measures. The proposed method is also compared with these state-of-the-art methods, OzaBaggingADWINClassifier, Accuracy Weighted Ensemble, Additive Expert Ensemble, Streaming Random Patches, and Adaptive Random Forest Classifier. The results show that the proposed method handles both true positive and false positive drifts efficiently.

Rule‐based preprocessing for data stream mining using complex event processing

AbstractData preprocessing is known to be essential to produce accurate data from which mining methods are able to extract valuable knowledge. When data constantly arrives from one or more sources, preprocessing techniques need to be adapted to efficiently handle these data streams. To help domain experts to define and execute preprocessing tasks for data streams, this paper proposes the use of active rule‐based systems and, more specifically, complex event processing (CEP) languages and engines. The main contribution of our approach is the formulation of preprocessing procedures as event detection rules, expressed in an SQL‐like language, that provide domain experts a simple way to manipulate temporal data. This idea is materialized into a publicly available solution that integrates a CEP engine with a library for online data mining. To evaluate our approach, we present three practical scenarios in which CEP rules preprocess data streams with the aim of adding temporal information, transforming features and handling missing values. Experiments show how CEP rules provide an effective language to express preprocessing tasks in a modular and high‐level manner, without significant time and memory overheads. The resulting data streams do not only help improving the predictive accuracy of classification algorithms, but also allow reducing the complexity of the decision models and the time needed for learning in some cases.

Selective ensemble-based online adaptive deep neural networks for streaming data with concept drift

Concept drift is an important issue in the field of streaming data mining. However, how to maintain real-time model convergence in a dynamic environment is an important and difficult problem. In addition, the current methods have limited ability to deal with the problem of streaming data classification for complex nonlinear problems. To solve these problems, a selective ensemble-based online adaptive deep neural network (SEOA) is proposed to address concept drift. First, the adaptive depth unit is constructed by combining shallow features with deep features and adaptively controls the information flow in the neural network according to changes in streaming data at adjacent moments, which improves the convergence of the online deep learning model. Then, the adaptive depth units of different layers are regarded as base classifiers for ensemble and weighted dynamically according to the loss of each classifier. In addition, a dynamic selection of base classifiers is adopted according to the fluctuation of the streaming data to achieve a balance between stability and adaptability. The experimental results show that the SEOA can effectively contend with different types of concept drift and has good robustness and generalization.

Comparative Analysis of Drift Detection Based Adaptive Ensemble Model with Different Drift Detection Techniques

Stream data mining is a popular research area these days. The concept drift detection and drift handling are the biggest challenges of stream data mining. Several drift detection algorithms have been developed which can accurately detect various drifts but have the problem of false-positive drift detection. The false-positive drift detection leads to the performance degradation of the classifier because of unnecessary training in between analyses. Classifier ensemble has shown its efficiency for drift detection, drift handling, and classification. But the ensemble classifiers could not detect the exact position of drift occurrence, so it has to update itself at some fixed interval, which leads to an unnecessary computational burden on the system. Combining the drift detection algorithm with an ensemble classifier can improve the performance and also solve the problems of false-positive drift detection and unnecessary updating of the ensemble classifier. In this paper, a model is proposed that creates a weighted adaptive ensemble classifier by updating it only when a drift detection signal is given by the used drift detection method. The proposed model is evaluated on text-based stream data for sentiment analysis and opinion mining with multiple drift detection algorithms and with multiple classification algorithms as base classifiers for the ensemble. A comparative analysis has been done, and the results have shown the efficiency of the proposed models.

Classification of Imbalanced Data Stream: Techniques and Challenges

As the number of generated data increases every day, this has brought the importance of data mining and knowledge extraction. In traditional data mining, offline status can be used for knowledge extraction. Nevertheless, dealing with stream data mining is different due to continuously arriving data that can be processed at a single scan besides the appearance of concept drift. As the pre-processing stage is critical in knowledge extraction, imbalanced stream data gain significant popularity in the last few years among researchers. Many real-world applications suffer from class imbalance including medical, business, fraud detection and etc. Learning from the supervised model includes classes whether it is binary- or multi-classes. These classes are often imbalance where it is divided into the majority (negative) class and minority (positive) class, which can cause a bias toward the majority class that leads to skew in predictive performance models. Handles imbalance streaming data is mandatory for more accurate and reliable learning models. In this paper, we will present an overview of data stream mining and its tools. Besides, summarize the problem of class imbalance and its different approaches. In addition, researchers will present the popular evaluation metrics and challenges prone from imbalanced streaming data.

ONLINE FUZZY CLUSTERING OF INCOMPLETE DATA USING CREDIBILISTIC APPROACH AND SIMILARITY MEASURE OF SPECIAL TYPE

Context. In most clustering (classification without a teacher) tasks associated with real data processing, the initial information is usually distorted by abnormal outliers (noise) and gaps. It is clear that “classical” methods of artificial intelligence (both batch and online) are ineffective in this situation.The goal of the paper is to propose the procedure of fuzzy clustering of incomplete data using credibilistic approach and similarity measure of special type.
 Objective. The goal of the work is credibilistic fuzzy clustering of distorted data, using of credibility theory.
 Method. The procedure of fuzzy clustering of incomplete data using credibilistic approach and similarity measure of special type based on the use of both robust goal functions of a special type and similarity measures, insensitive to outliers and designed to work both in batch and its recurrent online version designed to solve Data Stream Mining problems when data are fed to processing sequentially in real time.
 Results. The introduced methods are simple in numerical implementation and are free from the drawbacks inherent in traditional methods of probabilistic and possibilistic fuzzy clustering data distorted by abnormal outliers (noise) and gaps.
 Conclusions. The conducted experiments have confirmed the effectiveness of proposed methods of credibilistic fuzzy clustering of distorted data operability and allow recommending it for use in practice for solving the problems of automatic clusterization of distorted data. The proposed method is intended for use in hybrid systems of computational intelligence and, above all, in the problems of learning artificial neural networks, neuro-fuzzy systems, as well as in the problems of clustering and classification.

Fast and Accurate Terrain Image Classification for ASTER Remote Sensing by Data Stream Mining and Evolutionary-EAC Instance-Learning-Based Algorithm

Remote sensing streams continuous data feed from the satellite to ground station for data analysis. Often the data analytics involves analyzing data in real-time, such as emergency control, surveillance of military operations or scenarios that change rapidly. Traditional data mining requires all the data to be available prior to inducing a model by supervised learning, for automatic image recognition or classification. Any new update on the data prompts the model to be built again by loading in all the previous and new data. Therefore, the training time will increase indefinitely making it unsuitable for real-time application in remote sensing. As a contribution to solving this problem, a new approach of data analytics for remote sensing for data stream mining is formulated and reported in this paper. Fresh data feed collected from afar is used to approximate an image recognition model without reloading the history, which helps eliminate the latency in building the model again and again. In the past, data stream mining has a drawback in approximating a classification model with a sufficiently high level of accuracy. This is due to the one-pass incremental learning mechanism inherently exists in the design of the data stream mining algorithm. In order to solve this problem, a novel streamlined sensor data processing method is proposed called evolutionary expand-and-contract instance-based learning algorithm (EEAC-IBL). The multivariate data stream is first expanded into many subspaces, and then the subspaces, which are corresponding to the characteristics of the features are selected and condensed into a significant feature subset. The selection operates stochastically instead of deterministically by evolutionary optimization, which approximates the best subgroup. Followed by data stream mining, the model learning for image recognition is done on the fly. This stochastic approximation method is fast and accurate, offering an alternative to the traditional machine learning method for image recognition application in remote sensing. Our experimental results show computing advantages over other classical approaches, with a mean accuracy improvement at 16.62%.

Novel evolutionary-EAC instance-learning-based algorithm for fast data stream mining in assisted living with extreme connectivity

In modern healthcare, sensing technologies such as IoT empower the quality of assisted living service by knowing what a resident is doing in real-time. Using extreme connectivity and cloud computing in a smart home, where a collection of sensors is installed, the sensors sample continuously from the movements of the resident as well as ambient data from the surrounding inside the house. Automatic human activity recognition of the resident's activities is one of the key components of assisted living in smart home. For monitoring in-home safety, the ability in recognizing abnormal activities such as accident, falling, acute disease attack (e.g. asthma, stroke, etc.), fainting, wobbling, is particularly important. The detection and machine learning process must be both accurate and fast, to cope with the real-time activity recognition. To this end, a novel streamlined sensor data processing method is proposed called Evolutionary Expand-and-Contract Instance-based Learning algorithm (EEAC-IBL). The multivariate data stream is first expanded into many subspaces, then the subspaces which are corresponding to the characteristics of the features are selected and condensed into a significant feature subset. The selection operates scholastically instead of deterministically by evolutionary optimization which approximates the best subgroup. Followed by data stream mining, the machine learning for activity recognition is done on the fly. This approach is unique and suitable for such extreme connectivity scenario where precise feature selection is not required, and the relative importance of each feature among the sensor data changes over time. This stochastic approximation method is fast and accurate, offering an alternative to traditional machine learning method for smart home activity recognition application. Our experimental results show computing advantages over other classical approaches.

A clustering and ensemble based classifier for data stream classification

In the era of data mining, the research industry has great attention to data stream mining as well as it has a great impact on a wide range of applications like networking, telecommunication, education, banking, weather forecasting, a stock market, and so on. Because of these data stream mining having more attention from researchers. The handling of concept drifting data streams is one of the major issues and challenges in the data stream mining field. In the presence of the concept drift, the performance of the learning algorithm always degrades. In this paper, a hybrid method has been proposed which are the combination of an ensemble, and grid and density-based clustering methods. The proposed method is tested on both synthetic as well as real data. The proposed method works well in the presence of concept drift and performance is measured in terms of time, accuracy, and memory. As compared with the state-of-art algorithms, the proposed method performed well and gave better accuracy using synthetic datasets like 88.29%, 71.34%, and 75.39% for Hyperplane, RBF, and LED respectively and for real datasets 86.17%, 86.28%, 95.15%, and 99.83% for Adult, Census-Income, KDDCup99%–10%, and Covertype respectively.

A Frequent Pattern Conjunction Heuristic for Rule Generation in Data Streams

This paper introduces a new and expressive algorithm for inducing descriptive rule-sets from streaming data in real-time in order to describe frequent patterns explicitly encoded in the stream. Data Stream Mining (DSM) is concerned with the automatic analysis of data streams in real-time. Rapid flows of data challenge the state-of-the art processing and communication infrastructure, hence the motivation for research and innovation into real-time algorithms that analyse data streams on-the-fly and can automatically adapt to concept drifts. To date, DSM techniques have largely focused on predictive data mining applications that aim to forecast the value of a particular target feature of unseen data instances, answering questions such as whether a credit card transaction is fraudulent or not. A real-time, expressive and descriptive Data Mining technique for streaming data has not been previously established as part of the DSM toolkit. This has motivated the work reported in this paper, which has resulted in developing and validating a Generalised Rule Induction (GRI) tool, thus producing expressive rules as explanations that can be easily understood by human analysts. The expressiveness of decision models in data streams serves the objectives of transparency, underpinning the vision of ‘explainable AI’ and yet is an area of research that has attracted less attention despite being of high practical importance. The algorithm introduced and described in this paper is termed Fast Generalised Rule Induction (FGRI). FGRI is able to induce descriptive rules incrementally for raw data from both categorical and numerical features. FGRI is able to adapt rule-sets to changes of the pattern encoded in the data stream (concept drift) on the fly as new data arrives and can thus be applied continuously in real-time. The paper also provides a theoretical, qualitative and empirical evaluation of FGRI.

Addressing Event-Driven Concept Drift in Twitter Stream: A Stance Detection Application

The content posted by users on Social Networks represents an important source of information for a myriad of applications in the wide field known as `social sensing'. The Twitter platform in particular hosts the thoughts, opinions and comments of its users, expressed in the form of tweets: as a consequence, tweets are often analyzed with text mining and natural language processing techniques for relevant tasks, ranging from brand reputation and sentiment analysis to stance detection. In most cases the intelligent systems designed to accomplish these tasks are based on a classification model that, once trained, is deployed into the data flow for online monitoring. In this work we show how this approach turns out to be inadequate for the task of stance detection from tweets. In fact, the sequence of tweets that are collected everyday represents a data stream. As it is well known in the literature on data stream mining, classification models may suffer from concept drift, i.e. a change in the data distribution can potentially degrade the performance. We present a broad experimental campaign for the case study of the online monitoring of the stance expressed on Twitter about the vaccination topic in Italy. We compare different learning schemes and propose yet a novel one, aimed at addressing the event-driven concept drift.

A hybrid network intrusion detection using darwinian particle swarm optimization and stacked autoencoder hoeffding tree.

Cybersecurity experts estimate that cyber-attack damage cost will rise tremendously. The massive utilization of the web raises stress over how to pass on electronic information safely. Usually, intruders try different attacks for getting sensitive information. An Intrusion Detection System (IDS) plays a crucial role in identifying the data and user deviations in an organization. In this paper, stream data mining is incorporated with an IDS to do a specific task. The task is to distinguish the important, covered up information successfully in less amount of time. The experiment focuses on improving the effectiveness of an IDS using the proposed Stacked Autoencoder Hoeffding Tree approach (SAE-HT) using Darwinian Particle Swarm Optimization (DPSO) for feature selection. The experiment is performed in NSL_KDD dataset the important features are obtained using DPSO and the classification is performed using proposed SAE-HT technique. The proposed technique achieves a higher accuracy of 97.7% when compared with all the other state-of-art techniques. It is observed that the proposed technique increases the accuracy and detection rate thus reducing the false alarm rate.