Data Streams In Time Research Articles

Big Data Classification Using Distributed Optimized Hoeffding Trees

Large usage of social media, online shopping or transactions gives birth to voluminous data. Visual representation and analysis of this large amount of data is one of the major research topics today. As this data is changing over the period of time, we need an approach which will take care of velocity of data as well as volume and variety. In this paper, author has proposed a distributed method which will handle three dimensions of data and gives good results as compared to other method. Traditional algorithms are based on global optima which are basically memory resident programs. Our approach which is based on optimized hoeffding bound uses local optima method and distributed map-reduce architecture. It does not require copying whole data set onto a memory. As the model build is frequently updated on multiple nodes concurrently, it is more suitable for time varying data. Hoeffding bound is basically suitable for real time data stream. We have proposed very efficient distributed map-reduce architecture to implement hoeffding tree efficiently. We have used deep learning at leaf level to optimize the hoeffding tree. Drift detection is taken care by the architecture itself no separate provision is required for this. In this paper, with experimental results it is proved that our method takes less learning time with more accuracy. Also distributed algorithm for hoeffding tree implementation is proposed.

Data Stream Classification using Random Forest and Very Fast Decision Tree

Objective: Data Stream Classification is a big problem. Ensemble based learning methods are used to tackle the data Stream classification problem. Method: Methods such as Random Forest and Very Fast Decision Tree (VFDT) for classification are used in the system. Findings: This hybrid approach is an Effective method for calculating hidden data and maintains accuracy in the large data set. It also calculates the neighbors between each pair of cases that can be used in clusters. It is used for finding unknown or (by scaling) gives informational views over the data. This hybrid approach achieves 85 % accuracy and result proves that the hybrid approach performs well when compared to other algorithms in terms of accuracy. This is the application where we can download data streams of any application at faster rate. Many methods are available to process data streams. But the proposed algorithm performs well when compared to other algorithms. Applications: Many real time data streams are downloaded and uploaded to test and train various Applications. Data streams are used in many applications such as medical applications and educational applications.

Weighted FP-Tree Mining Algorithms for Conversion Time Data Flow

The data distribution in the data streams usually changes dynamically with time. Traditional mining algorithms based on transaction are difficult to establish the correlation between time characteristics and relationship features, thus making the results inaccurate. By analyzing the problems in the processing of time data stream, we put forward the concept of time gap degrees and design a mining algorithms based on weighted FP-Tree. We introduce the concept of FP-Tree node weights to transform the time data dynamically and excavate the data stream association rules. The experiments performed on the actual data set show that the algorithm can improve the recall and precision while consumes comparable computational time.

A strained silicon cold electron bolometer using Schottky contacts

We describe optical characterisation of a strained silicon cold electron bolometer (CEB), operating on a 350 mK stage, designed for absorption of millimetre-wave radiation. The silicon cold electron bolometer utilises Schottky contacts between a superconductor and an n++ doped silicon island to detect changes in the temperature of the charge carriers in the silicon, due to variations in absorbed radiation. By using strained silicon as the absorber, we decrease the electron-phonon coupling in the device and increase the responsivity to incoming power. The strained silicon absorber is coupled to a planar aluminium twin-slot antenna designed to couple to 160 GHz and that serves as the superconducting contacts. From the measured optical responsivity and spectral response, we calculate a maximum optical efficiency of 50% for radiation coupled into the device by the planar antenna and an overall noise equivalent power, referred to absorbed optical power, of 1.1×10−16 W Hz−1/2 when the detector is observing a 300 K source through a 4 K throughput limiting aperture. Even though this optical system is not optimized, we measure a system noise equivalent temperature difference of 6 mK Hz−1/2. We measure the noise of the device using a cross-correlation of time stream data, measured simultaneously with two junction field-effect transistor amplifiers, with a base correlated noise level of 300 pV Hz−1/2 and find that the total noise is consistent with a combination of photon noise, current shot noise, and electron-phonon thermal noise.

Analysis of the 3DVAR filter for the partially observed Lorenz'63 model

The problem of effectively combining data with a mathematical model constitutes a major challenge in applied mathematics. It is particular challenging for high-dimensional dynamical systems where data is received sequentially in time and the objective is to estimate the system state in an on-line fashion; this situation arises, for example, in weather forecasting. The sequential particle filter is then impractical and ad hoc filters, which employ some form of Gaussian approximation, are widely used. Prototypical of these ad hoc filters is the 3DVAR method. The goal of this paper is to analyze the 3DVAR method, using the Lorenz '63 model to exemplify the key ideas. The situation where the data is partial and noisy is studied, and both discrete time and continuous time data streams are considered. The theory demonstrates how the widely used technique of variance inflation acts to stabilize the filter, and hence leads to asymptotic accuracy.

The TDAQ Analytics Dashboard: a real-time web application for the ATLAS TDAQ control infrastructure

The ATLAS Trigger and Data Acquisition (TDAQ) infrastructure is responsible for filtering and transferring ATLAS experimental data from detectors to mass storage systems. It relies on a large, distributed computing system composed of thousands of software applications running concurrently. In such a complex environment, information sharing is fundamental for controlling applications behavior, error reporting and operational monitoring. During data taking, the streams of messages sent by applications and data published via information services are constantly monitored by experts to verify the correctness of running operations and to understand problematic situations. To simplify and improve system analysis and errors detection tasks, we developed the TDAQ Analytics Dashboard, a web application that aims to collect, correlate and visualize effectively this real time flow of information. The TDAQ Analytics Dashboard is composed of two main entities that reflect the twofold scope of the application. The first is the engine, a Java service that performs aggregation, processing and filtering of real time data stream and computes statistical correlation on sliding windows of time. The results are made available to clients via a simple web interface supporting SQL-like query syntax. The second is the visualization, provided by an Ajax-based web application that runs on client's browser. The dashboard approach allows to present information in a clear and customizable structure. Several types of interactive graphs are proposed as widgets that can be dynamically added and removed from visualization panels. Each widget acts as a client for the engine, querying the web interface to retrieve data with desired criteria. In this paper we present the design, development and evolution of the TDAQ Analytics Dashboard. We also present the statistical analysis computed by the application in this first period of high energy data taking operations for the ATLAS experiment.

A Data Stream Outlier Detection Algorithm Based on Reverse K Nearest Neighbors

This paper proposes a new data stream outlier detection algorithm SODRNN based on reverse nearest neighbors. We deal with the sliding window model, where outlier queries are performed in order to detect anomalies in the current window. The update of insertion or deletion only needs one scan of the current window, which improves efficiency. The capability of queries at arbitrary time on the whole current window is achieved by Query Manager Procedure, which can capture the phenomenon of concept drift of data stream in time. Results of experiments conducted on both synthetic and real data sets show that SODRNN algorithm is both effective and efficient.

SEVEN-YEAR WILKINSON MICROWAVE ANISOTROPY PROBE ( WMAP ) OBSERVATIONS: GALACTIC FOREGROUND EMISSION

[Abridged] We present updated estimates of Galactic foreground emission using seven years of WMAP data. Using the power spectrum of differences between multi-frequency template-cleaned maps, we find no evidence for foreground contamination outside of the updated (KQ85y7) foreground mask. We place a 15 microKelvin upper bound on rms foreground contamination in the cleaned maps used for cosmological analysis. We find no indication in the polarization data of an extra "haze" of hard synchrotron emission from energetic electrons near the Galactic center. We provide an updated map of the cosmic microwave background (CMB) using the internal linear combination (ILC) method, updated foreground masks, and updates to point source catalogs with 62 newly detected sources. Also new are tests of the Markov chain Monte Carlo (MCMC) foreground fitting procedure against systematics in the time-stream data, and tests against the observed beam asymmetry. Within a few degrees of the Galactic plane, WMAP total intensity data show a rapidly steepening spectrum from 20-40 GHz, which may be due to emission from spinning dust grains, steepening synchrotron, or other effects. Comparisons are made to a 1-degree 408 MHz map (Haslam et al.) and the 11-degree ARCADE 2 data (Singal et al.). We find that spinning dust or steepening synchrotron models fit the combination of WMAP and 408 MHz data equally well. ARCADE data appear inconsistent with the steepening synchrotron model, and consistent with the spinning dust model, though some discrepancies remain regarding the relative strength of spinning dust emission. More high-resolution data in the 10-40 GHz range would shed much light on these issues.

Finding top-k elements in a time-sliding window

Identifying the top-k most frequent elements is one of the many problems associated with data streams analysis. It is a well-known and difficult problem, especially if the analysis is to be performed and maintained up to date in near real time. Analyzing data streams in time sliding window model is of particular interest as only the most recent, more relevant events are considered. Approximate answers are usually adequate when dealing with this problem. This paper presents a new and innovative algorithm, the Filtered Space-Saving with Sliding Window Algorithm (FSW) that addresses this problem by introducing in the Filtered Space Saving (FSS) algorithm an approximated time sliding window counter. The algorithm provides the top-k list of elements, their frequency and an error estimate for each frequency value within the sliding window. It provides strong guarantees on the results, depending on the elements real frequencies. Experimental results detail performance on real life cases.

An Effective Streams Clustering Method for Biomedical Signals

In the healthcare industry, the ability to monitor patients via biomedical signals assists healthcare professionals in detecting early signs of conditions such as blocked arteries and abnormal heart rhythms. Using data clustering, it is possible to interpret these signals to look for patterns that may indicate emerging or developing conditions. This can be accomplished by basing monitoring systems on a fast clustering algorithm that processes fast-paced streams of raw data effectively. This paper presents a clustering method, POD-Clus, which can be useful in computer-aided diagnosis. The proposed method clusters data streams in linear time and outperforms a competing algorithm in capturing changes of clusters in data streams.

Auditory Augmentation

With auditory augmentation, the authors describe building blocks supporting the design of data representation tools, which unobtrusively alter the auditory characteristics of structure-borne sounds. The system enriches the structure-borne sound of objects with a sonification of (near) real time data streams. The object’s auditory gestalt is shaped by data-driven parameters, creating a subtle display for ambient data streams. Auditory augmentation can be easily overlaid to existing sounds, and does not change prominent auditory features of the augmented objects like the sound’s timing or its level. In a peripheral monitoring situation, the data stay out of the users’ attention, which thereby remains free to focus on a primary task. However, any characteristic sound change will catch the users’ attention. This article describes the principles of auditory augmentation, gives an introduction to the Reim Software Toolbox, and presents the first observations made in a preliminary long-term user study.

Guest editorial: Thematic issue on ‘Adaptive Soft Computing Techniques and Applications’

In the last 20years, we have witnessed the remarkable progress in computational-intelligence modelling for various applications. However, a majority of these researches make one fundamental assumption that sufficient and representative data are provided in advance for training. Because this assumption often does not hold in many real applications, recent efforts on adaptive soft computing techniques, such as evolving connectionist systems (ECOS), incremental learning and other adaptive learning models aim to relax the “sufficiency” requirement by continuously updating a model which keeps learning from data streams. ECOS addresses the learning from a data stream or chunks of data whose underlying distribution changes over time, by training a neural network continuously and adapting its structure and functionality through repeated interactions with the environment or other learning systems. Similarly, incremental learning develops the ability of a computational model to continuously accumulate knowledge learned over time from noisy and/or incomplete data. In practice, incremental learning is also featured by one-pass property, which enables the algorithm to work with real time data streams presented only once to the learning machine. Other adaptive soft computing

Generalized dimension-reduction framework for recent-biased time series analysis

Recent-biased approximations have received increased attention recently as a mechanism for learning trend patterns from time series or data streams. They have shown promise for clustering time series and incrementally pattern maintaining. In this paper, we design a generalized dimension-reduction framework for recent-biased approximations, aiming at making traditional dimension-reduction techniques actionable in recent-biased time series analysis. The framework is designed in two ways: equi-segmented scheme and vari-segmented scheme. In both schemes, time series data are first partitioned into segments and a dimension-reduction technique is applied to each segment. Then, more coefficients are kept for more recent data while fewer kept for older data. Thus, more details are preserved for recent data and fewer coefficients are kept for the whole time series, which improves the efficiency greatly. We experimentally evaluate the proposed approach, and demonstrate that traditional dimension-reduction techniques, such as SVD, DFT, DWT, PIP, PAA, and landmarks, can be embedded into our framework for recent-biased approximations over streaming time series.

Efficient mining method for retrieving sequential patterns over online data streams

With the usefulness of data mining in various fields of information science, various mining methods have been proposed in previous research. Recently, in these fields, data has taken the form of continuous data streams rather than finite stored data sets. In this paper, a mining method of sequential patterns over an online sequence data stream is proposed, which is useful for retrieving embedded knowledge in the data stream. The proposed method can minimize memory usage of the mining process while an error is allowed in its mining result, and supports flexible trade-off between memory usage and mining accuracy. However, the error is minimized by an accurate estimation method for the count of a sequence, which considers the ordering information of items. The proposed method can catch a recent change in a sequence data stream in a short time, by a decaying mechanism gracefully discarding old information that may be no longer useful.

Computable total functions on metric algebras, universal algebraic specifications and dynamical systems

Data such as real and complex numbers, discrete and continuous time data streams, waveforms, scalar and vector fields, and many other functions, are fundamental for many kinds of computation. In the theory of data, such data types are modelled using topological, or metric, many-sorted algebras and continuous homomorphisms. A theory of such topological data types is needed to answer the general questions: 1. What are the computable functions on topological algebras? 2. What methods exist to axiomatically specify functions on topological algebras? 3. Can all computable functions be specified? Such a theory seems to be in its infancy: there are many approaches to computability theory on general and specific spaces, and few approaches to specification theory. In some earlier papers, we have studied the questions 1 and 2 with the needs of data type theory in mind, and built a bridge between computations and specifications to try to answer 3. In this paper, we extend and combine several of our results, to prove new theorems that (i) show the equivalence of some six deterministic or non-deterministic models of computation on various metric algebras and, in particular, on spaces R n of real numbers; (ii) provide finite universal algebraic specifications for all the functions that can be computably approximated on metric algebras and, in particular, on Euclidean n-space R n ; (iii) show the existence of finite universal algebraic specifications of computably approximable finite dimensional deterministic dynamical systems. A technical issue is the localisation of uniform continuity using exhaustions of open sets. We use specifications composed of conditional equations, inequalities and, for convenience, new exhaustion primitives, that define functions uniquely up to isomorphism.

CMB map-making and power spectrum estimation

CMB data analysis is in general performed through two main steps: map-making of the time data streams and power spectrum extraction from the maps. The latter basically consists in the separation between the variance of the CMB and that of the noise in the map. Noise must therefore be deeply understood so that the estimation of CMB variance (the power spectrum) is unbiased. General techniques to make maps from time streams and to extract the power spectrum from them are presented in this article. We will see that exact, maximum likelihood solutions are in general too slow and hard to deal with to be used in modern experiments such as Archeops and should be replaced by approximate, iterative or Monte Carlo approaches that lead to similar precision. To cite this article: J.-Ch. Hamilton, C. R. Physique 4 (2003).

A simple graphical technique for assessing timer accuracy of computer systems

Timing accuracy is of perennial concern to researchers conducting reaction time studies, especially for those using computers that were not designed with millisecond accuracy in mind. Inaccuracies can manifest themselves in the form of quantization of the continuous time data stream. The method proposed here is simply to inspect the graph of the order statistics because quantization is readily apparent in such graphs. The method is illustrated with timing measurements of keypresses on an IBM PC.

The effect of the detector response time on bolometric cosmic microwave background anisotropy experiments

We analyze the effects of the detector response time on bolometric measurements of the anisotropy of the cosmic microwave background (CMB). We quantify the effect in terms of a single dimensionless parameter L defined as the ratio between the time the beam sweeps its own size and the bolometer response time. As L decreases below ∼ 2.5 the point source response of the experiment becomes elongated. We introduce a window function matrix based on the timestream data to assess the effects of the elongated beam. We find that the values of the window function matrix elements decrease slowly as a function L. Our analysis and results apply to other cases of beam asymmetry. For the High Frequency Instrument on board the Planck Surveyor satellite we show that for a broad range of L the ability of the experiment to extract the cosmological parameters is not degraded. Our analysis enhances the flexibility in tuning the design parameters of CMB anisotropy experiments.

Temporal relations and synchronization agents

Temporal relationships, inevitable among distributed multimedia objects, require synchronization of multiple data streams in time. We propose using synchronization agents as "smart" distributed objects to deal with scheduling, integrating, and synchronizing distributed multimedia streams. A formal specification model, interoperable Petri nets, describes the agents' behavior and captures the temporal semantics existing among distributed multimedia objects.

A high speed bit stream generator for HDTV

In developing high speed digital video decoder systems such as MPEG I, II, multimedia decoder or high-definition TV (HDTV), it is essential to have a real time data stream generation apparatus. In this paper, a real time bit stream generator using conventional SCSI-II type hard disc is proposed and implemented, and its effectiveness is verified in developing the prototype HDTV decoder system. The design concept and implementation are described. The bit stream generator and HDTV decoder had been successfully demonstrated.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>