Problem Of Data Streams Research Articles

A novel framework for concept drift detection using autoencoders for classification problems in data streams

A novel framework for concept drift detection using autoencoders for classification problems in data streams

Revisiting maximum satisfiability and related problems in data streams

We revisit the maximum satisfiability problem (Max-SAT) in the data stream model. In this problem, the stream consists of m clauses that are disjunctions of literals drawn from n Boolean variables. The objective is to find an assignment to the variables that maximizes the number of satisfied clauses. Chou et al. (FOCS 2020) showed that Ω(n) space is necessary to yield a 2/2+ε approximation of the optimum value; they also presented an algorithm that yields a 2/2−ε approximation of the optimum value using O(ε−2log⁡n) space.In this paper, we not only focus on approximating the optimum value, but also on obtaining the corresponding Boolean assignment using sublinear o(mn) space. We present randomized single-pass algorithms that w.h.p.1 yield:•A 1−ε approximation using O˜(n/ε3) space and exponential post-processing time•A 3/4−ε approximation using O˜(n/ε) space and polynomial post-processing time. Our ideas also extend to dynamic streams. However, we show that the streaming k-SAT problem, which asks whether one can satisfy all size-k input clauses, must use Ω(nk) space.We also consider the related minimum satisfiability problem (Min-SAT), introduced by Kohli et al. (SIAM J. Discrete Math. 1994), that asks to find an assignment that minimizes the number of satisfied clauses. For this problem, we give a O˜(n2/ε2) space algorithm, which is sublinear when m=ω(n), that yields an α+ε approximation where α is the approximation guarantee of the offline algorithm. If each variable appears in at most f clauses, we show that a 2fn approximation using O˜(n) space is possible.Finally, for the Max-AND-SAT problem where clauses are conjunctions of literals, we show that any single-pass algorithm that approximates the optimal value up to a factor better than 1/2 with success probability at least 2/3 must use Ω(mn) space.

Hybrid Sampling and Dynamic Weighting-Based Classification Method for Multi-Class Imbalanced Data Stream

The imbalance and concept drift problems in data streams become more complex in multi-class environment, and extreme imbalance and variation in class ratio may also exist. To tackle the above problems, Hybrid Sampling and Dynamic Weighted-based classification method for Multi-class Imbalanced data stream (HSDW-MI) is proposed. The HSDW-MI algorithm deals with imbalance and concept drift problems through the hybrid sampling and dynamic weighting phases, respectively. In the hybrid sampling phase, adaptive spectral clustering is proposed to sample the data after clustering, which can maintain the original data distribution; then the sample safety factor is used to determine the samples to be sampled for each class; the safe samples are oversampled and the unsafe samples are under-sampled in each cluster. If the data stream is extremely imbalanced, the sample storage pool is used to extract samples with a high safety factor to add to the data stream. In the dynamic weighting phase, a dynamic weighting method based on the G-mean value is proposed. The G-mean values are used as the weights of each base classifier in the ensemble and the ensemble is dynamically updated during the processing of the data stream to accommodate the occurrence of concept drift. Experiments were conducted with LB, OAUE, ARF, BOLE, MUOB, MOOD, CALMID, and the proposed HSDW-MI on 10 multi-class synthetic data streams with different class ratios and concept drifts and 3 real multi-class imbalanced streams with unknown drifts, and the results show that the proposed HSDW-MI has better classification capabilities and performs more consistently compared to all other algorithms.

An Ensemble-Based Fuzzy Rough Active Learning Approach for Broken Rotor Bar Detection in Nonstationary Environment

Most existing fault detection methods rely on batch learning, which assumes that all data is available and can be accessed over and over again. Such an approach is impossible in the face of nonstationary environments. This paper proposes an online combination of an Ensemble framework, Fuzzy Rough Active Learning, and drift detection, namely EFRAL, in which the structure of the ensemble automatically grows according to the data distribution. Since obtaining a real industrial data label is costly and time-consuming in practice, this paper incorporates an active learning scenario based on fuzzy rough sets to reduce operator labeling effort. This method gives weight to training samples based on the fuzzy rough sets theory and the samples are selected based on their weight. The proposed algorithm is applied to detect a broken rotor bar fault of induction motors to show its feasibility in industrial environments. The attained results show the efficiency of the proposed method. Furthermore, the efficiency of EFRAL is evaluated through data stream problems.

EMM-CLODS: An Effective Microcluster and Minimal Pruning CLustering-Based Technique for Detecting Outliers in Data Streams

Detecting outliers in data streams is a challenging problem since, in a data stream scenario, scanning the data multiple times is unfeasible, and the incoming streaming data keep evolving. Over the years, a common approach to outlier detection is using clustering-based methods, but these methods have inherent challenges and drawbacks. These include to effectively cluster sparse data points which has to do with the quality of clustering methods, dealing with continuous fast-incoming data streams, high memory and time consumption, and lack of high outlier detection accuracy. This paper aims at proposing an effective clustering-based approach to detect outliers in evolving data streams. We propose a new method called Effective Microcluster and Minimal pruning CLustering-based method for Outlier detection in Data Streams (EMM-CLODS). It is a clustering-based outlier detection approach that detects outliers in evolving data streams by first applying microclustering technique to cluster dense data points and effectively handle objects within a sliding window according to the relevance of their status to their respective neighbors or position. The analysis from our experimental studies on both synthetic and real-world datasets shows that the technique performs well with minimal memory and time consumption when compared to the other baseline algorithms, making it a very promising technique in dealing with outlier detection problems in data streams.

Scalable teacher forcing network for semi-supervised large scale data streams

The large-scale data stream problem refers to high-speed information flow which cannot be processed in scalable manner under a traditional computing platform. This problem also imposes expensive labelling cost making the deployment of fully supervised algorithms unfeasible. On the other hand, the problem of semi-supervised large-scale data streams is little explored in the literature because most works are designed in the traditional single-node computing environments while also being fully supervised approaches. This paper offers Weakly Supervised Scalable Teacher Forcing Network (WeScatterNet) to cope with the scarcity of labelled samples and the large-scale data streams simultaneously. WeScatterNet is crafted under distributed computing platform of Apache Spark with a data-free model fusion strategy for model compression after parallel computing stage. It features an open network structure to address the global and local drift problems while integrating a data augmentation, annotation and auto-correction (DA3) method for handling partially labelled data streams. The performance of WeScatterNet is numerically evaluated in the six large-scale data stream problems with only 25% label proportions. It shows highly competitive performance even if compared with fully supervised learners with 100% label proportions.

Adaptive Machine Learning for Automated Modeling of Residential Prosumer Agents

An efficient participation of prosumers in power system management depends on the quality of information they can obtain. Prosumers actions can be performed by automated agents that are operating in time-changing environments. Therefore, it is essential for them to deal with data stream problems in order to make reliable decisions based on the most accurate information. This paper provides an in-depth investigation of data and concept drift issues in accordance with residential prosumer agents. Additionally, the adaptation techniques, forgetting mechanisms, and learning strategies employed to handle these issues are explored. Accordingly, an approach is proposed to adapt the prosumer agent models to overcome the gradual and sudden concept drift concurrently. The suggested method is based on triggered adaptation techniques and performance-based forgetting mechanism. The results obtained in this study demonstrate that the proposed approach is capable of constructing efficient prosumer agents models with regard to the concept drift problem.

An Online Robust Support Vector Regression for Data Streams

Since support vector regression (SVR) is a flexible regression algorithm, its computational complexity does not depend on the dimensionality of the input space, and it has excellent generalization capability. However, a central assumption with SVRs is that all the required data is available at the time of construction, which means these algorithms cannot be used with data streams. Incremental SVR has been offered as a potential solution, but its accuracy suffers with noise and learning speeds are slow. To overcome these two limitations, we propose a novel incremental regression algorithm, called online robust support vector regression (ORSVR). ORSVR solves nonparallel bound functions simultaneously. Hence, the large quadratic programming problem (QPP) in classical v-SVR are decomposed into two smaller QPPs. An incremental learning algorithm then solves each QPP step-by-step. The results of a series of comparative experiments demonstrate that the ORSVR algorithm efficiently solves regression problems in data streams, with or without noise, and speeds up the learning process.

Exploiting evolving micro-clusters for data stream classification with emerging class detection

Learning non-stationary data streams is challenging due to the unique characteristics of infinite length and evolving property. Current existing works often concentrate on the concept-drift problem in data streams. Concept evolution, indicating novel classes are emerged in data streams, has gained growing attention recently due to its practical values in many real-world applications. Thereby, how to design a new robust learning model on data streams to handle concept drift, concept evolution and outliers simultaneously, is of significant importance. To this end, we propose a new data stream classification approach, called EMC, which dynamically learns the Evolving Micro-Clusters to examine both concept drift and evolution. Specifically, to capture time-changing concept, EMC dynamically maintains a set of online micro-clusters and learns their importance with error-based representative learning. Building upon the evolving micro-clusters, the novel class detector is introduced based on a local density perspective, which allows handling the data streams with complex class distribution. Beyond, EMC allows distinguishing concept drift and evolution from noisy instances. Extensive experiments on both synthetic and real-world data sets show that our method has good classification and novel class detection performance compared to state-of-the-art algorithms.

Online Sequential Extreme Learning Machine With Dynamic Forgetting Factor

Online sequential extreme learning machine (OS-ELM) and its variants provide a promising way to solve data stream problems, but most of them do not take the timeliness of the problems into account, which may degrade the performance of the model. The main reason is that these algorithms are unable to adapt to the latest data accordingly when the distribution of the data stream changes. To mitigate this limitation, the forgetting factor is introduced into the relevant models, which is used to balance the relative importance of past data and new data when necessary. However, there is no efficient way to set the forgetting factor properly so far. In this paper, we have developed a novel updating strategy for setting the forgetting factor and proposed a dynamic forgetting factor based OS-ELM algorithm (DOS-ELM). In the sequential learning phase of DOS-ELM, the forgetting factor can be adjusted dynamically according to the change degree of the model accuracy in each learning epoch. This updating process does not require setting any parameters artificially and thus greatly improves the flexibility of the model. The experimental results on ten classification problems, five regression problems, one time-series problem show that DOS-ELM can deal well with both stationary and non-stationary data stream problems. In addition, we have extended DOS-ELM to an online deep model named ML-DOS-ELM, which can handle more complex tasks such as the face recognition problem and the handwritten digit recognition problem. Our experimental evaluations show that both DOS-ELM and ML-DOS-ELM can achieve higher prediction accuracy compared to the other similar algorithms.

Adaptive learning on mobile network traffic data

ABSTRACTMachine learning based mobile traffic classification has become a popular topic in recent years. As mobile traffic data is dynamic in nature, the static model has become ineffective for the task of classifying future traffic. This is known as the concept drift problem in data streams. To this end, this paper presents an adaptive mobile traffic classification method. Specifically, a method based on the fuzzy competence model is devised to detect concept drift, and a dynamic learning method is presented to update the classification model, so as to adapt to an ever-changing environment at an appropriate time. The concept drift detection method relies on the data distribution instead of the classification error rate. Furthermore, the weights of flow samples are dynamically updated and flow samples are resampled for training a new model when a concept drift is detected. Moreover, recently trained models are saved and used for classification in weighted voting. The weight of each model is updated according to the performance it obtains on the most recent flow samples. On mobile traffic data, experimental results show that our proposed method obtains lower classification error rate with less time consumption on updating models as compared to related methods designed for handling concept drift problems.

Width of Points in the Streaming Model

In this article, we show how to compute the width of a dynamic set of low-dimensional points in the streaming model. In particular, we assume that the stream contains both insertions of points and deletions of points to a set S , and the goal is to compute the width of the set S , namely the minimal distance between two parallel hyperplanes sandwiching the point set S . Our algorithm (1 + ϵ) approximates the width of the set S using space polylogarithmic in the size of S and the aspect ratio of S . This is the first such algorithm that supports both insertions and deletions of points to the set S : previous algorithms for approximating the width of a point set only supported additions [Agarwal et al. 2004; Chan 2006], or a sliding window [Chan and Sadjad 2006]. This solves an open question from the “2009 Kanpur list” of open problems in data streams, property testing, and related topics [Indyk et al. 2011].

Arthur–Merlin streaming complexity

We study the power of Arthur–Merlin probabilistic proof systems in the data stream model. We show a canonical AM streaming algorithm for a class of data stream problems. The algorithm offers a tradeoff between the length of the proof and the space complexity that is needed to verify it.As an application, we give an AM streaming algorithm for the Distinct Elements problem. Given a data stream of length m over alphabet of size n, the algorithm uses O˜(s) space and a proof of size O˜(w), for every s,w such that s⋅w≥n (where O˜ hides a polylog(m,n) factor). We also prove a lower bound, showing that every MA streaming algorithm for the Distinct Elements problem that uses s bits of space and a proof of size w, satisfies s⋅w=Ω(n). Furthermore, the lower bound also holds for approximating the number of distinct elements within a multiplicative factor of 1±1/n.As a part of the proof of the lower bound for the Distinct Elements problem, we show a new lower bound of Ω(n) on the MA communication complexity of the Gap Hamming Distance problem, and prove its tightness.

Competitive analysis of maintaining frequent items of a stream

We study the classic frequent items problem in data streams, but from a competitive analysis point of view. We consider the standard worst-case input model, as well as a weaker distributional adversarial setting. We are primarily interested in the single-slot memory case and for both models we give (asymptotically) tight bounds of Θ(N) and Θ(N3) respectively, achieved by very simple and natural algorithms, where N is the stream's length. We also provide lower bounds, for both models, in the more general case of arbitrary memory sizes of k≥1

Robust on-line neural learning classifier system for data stream classification tasks

The increasing integration of technology in the different areas of science and industry has resulted in the design of applications that generate large amounts of data on-line. Most often, extracting information from these data is key, in order to gain a better understanding of the processes that the data are describing. Learning from these data poses new challenges to traditional machine learning techniques, which are not typically designed to deal with data in which concepts and noise levels may vary over time. The purpose of this paper is to present supervised neural constructivist system (SNCS), an accuracy-based neural-constructivist learning classifier system that makes use of multilayer perceptrons to learn from data streams with a fast reaction capacity to concept changes. The behavior of SNCS on data stream problems with different characteristics is carefully analyzed and compared with other state-of-the-art techniques in the field. This comparison is also extended to a large collection of real-world problems. The results obtained show that SNCS can function in a variety of problem situations producing accurate classification of data, whether the data are static or in dynamic streams.

Exploiting empirical variance for data stream classification

Classification, using the decision tree algorithm, is a widely studied problem in data streams. The challenge is when to split a decision node into multiple leaves. Concentration inequalities, that exploit variance information such as Bernstein’s and Bennett’s inequalities, are often substantially strict as compared with Hoeffding’s bound which disregards variance. Many machine learning algorithms for stream classification such as very fast decision tree (VFDT) learner, AdaBoost and support vector machines (SVMs), use the Hoeffding’s bound as a performance guarantee. In this paper, we propose a new algorithm based on the recently proposed empirical Bernstein’s bound to achieve a better probabilistic bound on the accuracy of the decision tree. Experimental results on four synthetic and two real world data sets demonstrate the performance gain of our proposed technique.

Conceptual modeling rules extracting for data streams

Data take the form of continuous data streams rather than traditional stored databases in a growing number of applications, including network traffic monitoring, network intrusion detection, sensor networks, fraudulent transaction detection, financial monitoring, etc. People are interested in the potential rules in data streams such as association rules and decision rules. Compared with much work on developing algorithms of data streams mining, there is little attention paid on the modeling data mining and data streams mining. Considering the problem of conceptual modeling data streams mining, we put forward a data streams oriented decision logic language as a granular computing formal approach and a rules extracting model based on granular computing. In this model, we propose the notion of granular drifting, which accurately interpret the concept drifting problem in data streams. This model is helpful to understand the nature of data streams mining. Based on this model, new algorithms and techniques of data streams mining could be developed.