Multi-dimensional Data Streams Research Articles

The ubiquitous self-organizing map for non-stationary data streams

The Internet of things promises a continuous flow of data where traditional database and data-mining methods cannot be applied. This paper presents improvements on the Ubiquitous Self-Organized Map (UbiSOM), a novel variant of the well-known Self-Organized Map (SOM), tailored for streaming environments. This approach allows ambient intelligence solutions using multidimensional clustering over a continuous data stream to provide continuous exploratory data analysis. The average quantization error and average neuron utility over time are proposed and used to estimating the learning parameters, allowing the model to retain an indefinite plasticity and to cope with changes within a multidimensional data stream. We perform parameter sensitivity analysis and our experiments show that UbiSOM outperforms existing proposals in continuously modeling possibly non-stationary data streams, converging faster to stable models when the underlying distribution is stationary and reacting accordingly to the nature of the change in continuous real world data streams.

Discovering Analytic Associate Rule Filtering on Multi-Dimensional Data Streams

Multidimensional data points discover structural and chronological association inside the data streams. The PaDSkyline framework in a distributed environment utilized intra group optimization and multi filtering technique for skyline query processing. Skyline query processes within each group of distributed data sets, but dynamic filtering point selection was not performed with cost effective system. Similarity- Profiled temporal Association MINing mEthod (SPAMINE) used reference time sequences and threshold value to filter the information from real world data. Different similarity models for filtering temporal patterns were not very effective for performing the phase shift in time. To attain minimal phase shift based cost effective filtering on multidimensional data stream, Analytic Associate Rule Filtering (AARF) mechanism is proposed in this study. The main objective of AARF is to identify the relationship between attributes on multidimensional data and to filter out the independent attributes from the data streams. Initially, analytic association rule uses the weight computing factor to identify relationship and make inferences while testing multidimensional samples. Secondly, with the help of the analyzed relationship, the AARF mechanism uses the attribute independent criterion to discard negligible weight from association rule. Finally, to filter the analytic association rule with specified phase shift time, the ‘if-then’ strategy is used in AARF mechanism. AARF mechanism has an ability to make an analytic filtering with minimal phase shift time on multidimensional test dataset. The minimal phase shift time reduces the execution time factor and attains cost effective filtering system. Experiment is conducted using Japanese vowel multidimensional data set extracted from UCI repository for measuring the factors such as the average precision level, execution time, filtering query traffic efficiency and true positive rate.

Uncertain canonical correlation analysis for multi-view feature extraction from uncertain data streams

Abstract Canonical correlation analysis (CCA) is a well-known technique to extract common features from a pair of multivariate data. In uncertain data stream situations, however, it does not extract useful features because of the existence of data uncertainty which is widespread in a variety of applications. This paper describes an uncertain CCA method called UCCA for feature extraction from uncertain multidimensional data streams. By using the information of uncertainty, UCCA can well represent an uncertain linear structure in the projected space. The approach is tested on a variety of real datasets and its effectiveness in terms of multi-view classification based on dimensionality reduction is validated.

Toward a new cognitive neuroscience: modeling natural brain dynamics.

Toward a new cognitive neuroscience: modeling natural brain dynamics.

Finding Spatio-Temporal Patterns in Multidimensional Data Streams

In the last few decades, advances in data acquisition technology have contributed to generation of huge volumes of data in diverse application areas, creating new research challenges in knowledge discovery. The analysis of these data has become an important task in several domains such as sensor networks, web-logs, financial transactions and climate change monitoring. In this article, we propose the Spatio-Temporal Behavior Meter (STB-meter) method to identify spatio-temporal patterns in multidimensional evolving data streams. Our approach combines a multi-resolution hierarchical structure to deal with spatial information with fractal-based analysis to monitor non spatial information of the multidimensional data stream. Experimental evaluation on real climate data shows that our method allows finding relevant spatio-temporal patterns in evolving data at different spatial and temporal resolutions and therefore it can be a useful tool to assist domain specialists in climate change researches.

Development of optimization procedures for application-specific chemical sensing

We present methods for selecting optimized operating conditions from tunable sensor systems that produce multi-dimensional data streams. To demonstrate our approach in a case study, a chemical sensing dataset was collected using a microsensor array with temperature-modulated elements. The top electrodes of the array elements were coated with SnO2, In2O3, and CuO sensing films that were formed on the individual microhotplate platforms by annealing microcapillary deposited metal-hydroxide sol–gel films. Chemical sensing data was collected while interfacial interactions were influenced via pulsed temperature programming. During the collection of the chemical sensing database an analyte exposure schedule consisting of a dry air background and μmol/mol concentrations of three reducing analytes was cycled as the temperature program operating the microsensor array was systematically varied. The data was processed using linear discriminant analysis and principle component analysis to quantify analyte selectivity. The methods discussed in this manuscript produced sensing materials-specific temperature programs optimized for the studied analytes which contained a factor of 2.6 fewer temperature pulses than the original inspection temperature programs. These shortened temperature programs increased sample frequency, preserved analyte concentration information, and improved target cross-selectivity.

A Precise Statistical approach for concept change detection in unlabeled data streams

Recently data stream has been extensively explored due to its emergence in a great deal of applications such as sensor networks, web click streams and network flows. One of the most important challenges in data streams is concept change where data underlying distributions change from time to time. A vast majority of researches in the context of data stream mining are devoted to labeled data, whereas, in real word human practice label of data are rarely available to the learning algorithms. Moreover, most of the methods that detect changes in unlabeled data stream merely deal with numerical data sets, and also, they are facing considerable difficulty when dimension of data tends to increase. In this paper, we present a Precise Statistical approach for Concept Change Detection in unlabeled data streams, which, abbreviated as PSCCD, detects changes using an exchangeable test. This hypothesis test is driven from a martingale which is based on Doob’s Maximal Inequality. The advantages of our approach are three fold. First, it does not require a sliding window on the data stream whose size is a well-known challenging issue; second, it works well in multi-dimensional data stream, and last but not the least, it is applicable to different types of data including categorical, numerical and mixed-attribute data streams. To explore the advantages of our approach, quite a lot of experiments with different settings and specifications are conducted. The obtained results are very promising.

ApproxCCA: An approximate correlation analysis algorithm for multidimensional data streams

Correlation analysis is regarded as a significant challenge in the mining of multidimensional data streams. Great emphasis is generally placed on one-dimensional data streams with the existing correlation analysis methods for the mining of data streams. Therefore, the identification of underlying correlation among multivariate arrays (e.g. Sensor data) has long been ignored. The technique of canonical correlation analysis (CCA) has rarely been applied in multidimensional data streams. In this study, a novel correlation analysis algorithm based on CCA, called ApproxCCA, is proposed to explore the correlations between two multidimensional data streams in the environment with limited resources. By introducing techniques of unequal probability sampling and low-rank approximation to reduce the dimensionality of the product matrix composed by the sample covariance matrix and sample variance matrix, ApproxCCA successfully improves computational efficiency while ensuring the analytical precision. Experimental results of synthetic and real data sets have indicated that the computational bottleneck of traditional CCA can be overcome with ApproxCCA, and the correlations between two multidimensional data streams can also be detected accurately.

Detecting anomalies from high-dimensional wireless network data streams: a case study

In this paper, we study the problem of anomaly detection in wireless network streams. We have developed a new technique, called Stream Projected Outlier deTector (SPOT), to deal with the problem of anomaly detection from multi-dimensional or high-dimensional data streams. We conduct a detailed case study of SPOT in this paper by deploying it for anomaly detection from a real-life wireless network data stream. Since this wireless network data stream is unlabeled, a validating method is thus proposed to generate the ground-truth results in this case study for performance evaluation. Extensive experiments are conducted and the results demonstrate that SPOT is effective in detecting anomalies from wireless network data streams and outperforms existing anomaly detection methods.

HIDS

In the research of high-speed data streams, large amounts of synthetic data are needed. These days, more and more researchers focus on hierarchical multi-dimensional data streams or data sets, which is beyond the ability of traditional synthetic data generators. In this paper we propose a two-phased method to generate hierarchical multi-dimensional data streams, in which a tree-like structure is built first, and then an unlimited number of items chosen among the tree leaves according to a distribution are inserted into the stream. Our generator, HIDS, integrates all of the functions of existing data generators, and can customize the tree structure according to usersý requirements, producing tree structures such as equal-depth trees, equal-fan-out trees, balanced trees and different-fan-out trees. An experimental study using real data streams shows that HIDS can generate data streams tailored to specific applications.

Efficiently tracing clusters over high-dimensional on-line data streams

A good clustering method should provide flexible scalability on the number of dimensions as well as the size of a data set. This paper proposes a method of efficiently tracing the clusters of a high-dimensional on-line data stream. While tracing the one-dimensional clusters of each dimension independently, a technique which is similar to frequent itemset mining is employed to find the set of multi-dimensional clusters. By finding a frequently co-occurred set of one-dimensional clusters, it is possible to trace a multi-dimensional rectangular space whose range is defined by the one-dimensional clusters collectively. In order to trace such candidates over a multi-dimensional online data stream, a cluster-statistics tree (CS-Tree) is proposed in this paper. A k-depth node( k ⩽ d) in the CS-tree is corresponding to a k-dimensional rectangular space. Each node keeps track of the density of data elements in its corresponding rectangular space. Only a node corresponding to a dense rectangular space is allowed to have a child node. The scalability on the number of dimensions is greatly enhanced while sacrificing the accuracy of identified clusters slightly.

Cell trees: An adaptive synopsis structure for clustering multi-dimensional on-line data streams

To effectively trace the clusters of recently generated data elements in an on-line data stream, a sibling list and a cell tree are proposed in this paper. Initially, the multi-dimensional data space of a data stream is partitioned into mutually exclusive equal-sized grid-cells. Each grid-cell monitors the recent distribution statistics of data elements within its range. The old distribution statistics of each grid-cell are diminished by a predefined decay rate as time goes by, so that the effect of the obsolete information on the current result of clustering can be eliminated without maintaining any data element physically. Given a partitioning factor h, a dense grid-cell is partitioned into h equal-size smaller grid-cells. Such partitioning is continued until a grid-cell becomes the smallest one called a unit grid-cell. Conversely, a set of consecutive sparse grid-cells can be merged into a single grid-cell. A sibling list is a structure to manage the set of all grid-cells in a one-dimensional data space and it acts as an index for locating a specific grid-cell. Upon creating a dense unit grid-cell on a one-dimensional data space, a new sibling list for another dimension is created as a child of the grid-cell. In such a way, a cell tree is created. By repeating this process, a multi-dimensional dense unit grid-cell is identified by a path of a cell tree. Furthermore, in order to confine the usage of memory space, the size of a unit grid-cell is adaptively minimized such that the result of clustering becomes as accurate as possible at all times. The proposed method is comparatively analyzed by a series of experiments to identify its various characteristics.

Range Counting over Multidimensional Data Streams

We consider the problem of approximate range counting over a stream of d-dimensional points. In the data stream model the algorithm makes a single scan of the data, which is presented in an arbitrary order, and computes a compact summary data structure. The summary, whose size depends on the approximation parameter ε, can be used to count the number of points inside a query range within additive error εn, where n is the size of the stream seen so far. We present several results, deterministic and randomized, for both rectangle and halfspace ranges.

Adaptive Spatial Partitioning for Multidimensional Data Streams

We propose a space-efficient scheme for summarizing multidimensional data streams. Our sketch can be used to solve spatial versions of several classical data stream queries efficiently. For instance, we can track ε-hot spots, which are congruent boxes containing at least an ε fraction of the stream, and maintain hierarchical heavy hitters in d dimensions. Our sketch can also be viewed as a multidimensional generalization of the ε-approximate quantile summary. The space complexity of our scheme is O((1/ε) log R) if the points lie in the domain [0, R]d, where d is assumed to be a constant. The scheme extends to the sliding window model with a log (ε n) factor increase in space, where n is the size of the sliding window. Our sketch can also be used to answer ε-approximate rectangular range queries over a stream of d-dimensional points.