Approximate Aggregation Research Articles

(ε, δ)-Approximate Aggregation Algorithms in Dynamic Sensor Networks

Aggregation operations are important in WSN applications. Since large numbers of applications only require approximate aggregation results rather than the exact ones, some approximate aggregation algorithms have been proposed to save energy. However, the error bounds of these algorithms are fixed and it is impossible to adjust the error bounds automatically, so they cannot meet the requirement of arbitrary precision required by various users. Thus, a uniform sampling-based algorithm was proposed by the authors of this paper to satisfy arbitrary precision requirement. Unfortunately, this uniform sampling-based algorithm is only suitable for static sensor networks. To overcome the shortcoming of the uniform sampling-based algorithm, this paper proposes four Bernoulli sampling-based and distributed approximate aggregation algorithms to process the snapshot and continuous aggregation queries in dynamic sensor networks. Theoretical analysis and experimental results show that the proposed algorithms have high performance in terms of accuracy and energy consumption.

Fixed Interval Smoothing of Nonlinear/Non-Gaussian Dynamic Systems in Cell Space

State estimation problems such as optimal filtering and smoothing do not lend themselves to analytical treatment in general nonlinear/non-Gaussian dynamic systems. The fixed interval smoothing problem aims to construct the marginal conditional probability density function of the state given past and future measurements relative to the state. For linear Gaussian systems it is derived as the Rauch-Tung-Striebel smoother. Theoretical solutions are available for nonlinear systems in the form of forward-filter-backward-smoother strategy and the two-filter-smoother formula that combines the results of two independent filters. Recently, these methods have been implemented in the context of sequential Monte Carlo filters with importance sampling. In this paper the fixed interval smoothing is numerically approximated by recursive computation of the conditional density as a piecewise constant function. The key for this algorithm is a coarse-grained representation of the system dynamics as an approximate aggregate Markov chain in discretized state space or cell space. The proposed approach is demonstrated with a simulation example involving a nonlinear CSTR model.

Approximate aggregation of a two time scales periodic multi-strain SIS epidemic model: A patchy environment with fast migrations

In this work we consider a spatially distributed periodic multi strain SIS epidemic model. We let susceptible and infected individuals migrate between patches, with periodic migration rates. Considering that migrations are much faster than the epidemic process, we build up a less dimensional (aggregated) system that allows to study some features of the asymptotic behavior of the original model. In particular, we are able to define global reproduction numbers in the non-spatialized aggregated system that serve to decide the eradication or endemicity of the epidemic in the initial spatially distributed nonautonomous model. Comparing these global reproductive numbers with those corresponding to isolated patches we show that adequate periodic fast migrations can in many cases reverse local endemicity and get global eradication of the epidemic.

Approximate Aggregations in Structured P2P Networks

In corporate networks, daily business data are generated in gigabytes or even terabytes. It is costly to process aggregate queries in those systems. In this paper, we propose PACA, a probably approximately correct aggregate query processing scheme, for answering aggregate queries in structured Peer-to-Peer (P2P) network. PACA retrieves random samples from peers' databases and applies the samples to process queries. Instead of scanning the entire database of each peer, PACA only accesses a small random number of data. Moreover, based on the query distribution,PACA publishes a precomputed synopsis and uses the synopsis to answer future queries. Most queries are expected to be answered by the precomputed synopsis partially or fully. And the synopsis is adaptively tuned to follow the query distribution. Experiments on the PlanetLab show the effectiveness of the approach.

Approximate aggregation of linear discrete models with two time scales: re-scaling slow processes to the fast scale

Mathematical models used in ecology often inherit the complexity found in nature and thus are governed by a large number of variables. Aggregation of variables methods is used to make such models mathematically tractable by building an approximate system governing fewer variables. We extend here aggregation methods for linear discrete models with processes occurring at different time scales. In practical cases, some processes that occur at a fast time scale are often only measured at a slow time scale, like mortality. We present a general class of models with two time scales involving such kind of processes. We show how they should be re-scaled in order to be taken into account at the fast time scale in a more realistic approach. The approximate aggregation of these models is undertaken and justified in mathematical terms. We also provide an application to a model of a structured population in a two-patch environment.

(ε,δ )-Approximate Aggregation Algorithm in Wireless Sensor Networks

This paper proposes an approximate aggregation algorithm based on Bernoulli sampling to satisfy the requirement of arbitrary precision in wireless sensor networks(WSN).Besides,two sample data adaptive algorithms are also provided.One is to adapt the sample to the varying precision requirement.The other is to adapt the sample to the varying sensed data in networks.Theoretical analysis and experimental results show that the proposed algorithms have good performance in terms of accuracy and energy cost.

Approximate Aggregation of Time-Varying Data in P2P Networks

随着P2P 技术在电子商务等领域的广泛应用,对分布在P2P 网络中的数据进行聚集操作的需求越来越迫切.但是,由于P2P 网络的大规模及分散性,这种聚集操作的实现颇具挑战性.而且在很多应用中,P2P 网络中的数据往往是随时间变化的,这进一步增加了聚集操作的难度.现有P2P 网络中的聚集算法均假定网络中的数据是非时变的,如果将其直接应用在存在时变数据的P2P 网络中,则会因为其聚集时间过长而导致聚集过程中数据已经发生变化的问题.为此,提出了一种P2P 网络中基于均衡采样的时变数据近似聚集算法,理论分析和实验结果表明,该聚集算法在处理时变数据时优于已有的算法,可以有效地应用于存在时变数据的P2P 网络中.;With the wide application of peer-to-peer (P2P) technologies in many fields such as E-commerce, it is increasingly necessary to do aggregation queries in P2P networks. However, due to the large scale and decentralization of P2P networks it is rather difficult to do this kind of operation. Aggregation queries will become even more difficult in case that the data in P2P networks are time-varying which is often occurs in practice. The existing aggregation methods for data in P2P networks all assume that the data are time-invariant. If these methods are directly applied to P2P networks with time-varying data, some problems will arise because the data used in aggregation processing would have changed owing to the long time of aggregation. So, this paper proposes an approximate aggregation method for time-varying data in P2P networks based on uniform sampling. The theoretical analysis and experimental results show that this aggregation method outperforms the existing methods and can effectively be applied to P2P networks with time-varying data.

Approximate Continuous Aggregation via Time Window Based Compression and Sampling in WSNs

In many applications continuous aggregation of sensed data is usually required. The existing aggregation schemes usually compute every aggregation result in a continuous aggregation either by a complete aggregation procedure or by partial data update at each epoch. To further reduce the energy cost, we propose a sampling-based approach with time window based linear regression for approximate continuous aggregation. We analyze the approximation error of the aggregation results and discuss the determinations of parameters in our approach. Simulation results verify the effectiveness of our approach.

Approximate Aggregation Methods in Discrete Time Stochastic Population Models

Approximate aggregation techniques consist of introducing certain approximations that allow one to reduce a complex system involving many coupled variables obtaining a simpler ʽʽaggregated systemʼʼ governed by a few variables. Moreover, they give results that allow one to extract information about the complex original system in terms of the behavior of the reduced one. Often, the feature that allows one to carry out such a reduction is the presence of different time scales in the system under consideration. In this work we deal with aggregation techniques in stochastic discrete time models and their application to the study of multiregional models, i.e., of models for an age structured population distributed amongst different spatial patches and in which migration between the patches is usually fast with respect to the demography (reproduction-survival) in each patch. Stochasticity in population models can be of two kinds: environmental and demographic. We review the formulation and the main properties of the dynamics of the different models for populations evolving in discrete time and subjected to the effects of environmental and demographic stochasticity. Then we present different stochastic multiregional models with two time scales in which migration is fast with respect to demography and we review the main relationships between the dynamics of the original complex system and the aggregated simpler one. Finally, and within the context of models with environmental stochasticity in which the environmental variation is Markovian, we make use these techniques to analyze qualitatively the behavior of two multiregional models in which the original complex system is intractable. In particular we study conditions under which the population goes extinct or grows exponentially.

Time-decaying Sketches for Robust Aggregation of Sensor Data

We present a new sketch for summarizing network data. The sketch has the following properties which make it useful in communication-efficient aggregation in distributed streaming scenarios, such as sensor networks: the sketch is duplicate insensitive, i.e., reinsertions of the same data will not affect the sketch and hence the estimates of aggregates. Unlike previous duplicate-insensitive sketches for sensor data aggregation [S. Nath et al., Synposis diffusion for robust aggregation in sensor networks, in Proceedings of the 2nd International Conference on Embedded Network Sensor Systems, (2004), pp. 250–262], [J. Considine et al., Approximate aggregation techniques for sensor databases, in Proceedings of the 20th International Conference on Data Engineering (ICDE), 2004, pp. 449–460], it is also time decaying, so that the weight of a data item in the sketch can decrease with time according to a user-specified decay function. The sketch can give provably approximate guarantees for various aggregates of data, including the sum, median, quantiles, and frequent elements. The size of the sketch and the time taken to update it are both polylogarithmic in the size of the relevant data. Further, multiple sketches computed over distributed data can be combined without loss of accuracy. To our knowledge, this is the first sketch that combines all the above properties.

Approximating the distribution of population size in stochastic multiregional matrix models with fast migration

In this work we deal with a multiregional model in discrete time for an age-structured population which lives in an environment that changes randomly with time and is distributed in different spatial patches. In addition, and as is often the case in applications, we assume that migration is fast with respect to demography. Using approximate aggregation techniques we make use of the existence of different time scales in the model and reduce the dimension of the system obtaining a stochastic Leslie model in which the variables are the total population in each age class. Literature shows that, under reasonable conditions, the distribution of population size in matrix models with environmental stochasticity is asymptotically lognormal, and is characterized by two parameters, stochastic growth rate (s.g.r.) and scaled logarithmic variance (s.l.v.), that, in most practical cases, cannot be computed exactly. We show that the s.g.r. and the s.l.v. of the original multiregional model can be approximated by those corresponding to the reduced stochastic Leslie model, therefore simplifying its analysis. Moreover, we illustrate the usefulness of the reduction procedure by presenting some practical cases in which, although the explicit computation of the s.g.r. and the s.l.v. of the original multiregional model is not feasible, we can calculate its analogues for the reduced model.

Collective adjustment of pension rights in ALM models

Collective adjustment of pension rights is a way to keep defined benefit systems tenable. In asset liability management (ALM) models presented in the literature these decisions are modeled both at the aggregate level of the liabilities as a whole and at a more detailed level. In this paper we compare the approximate aggregate approach to the accurate detailed approach for the average earnings scheme with conditional indexation. We prove that the aggregate approach leads to one-sided errors. Moreover, we show that for semi-realistic data these biases are considerable.

An approximation for kanban controlled assembly systems

An approximation is proposed to evaluate the steady-state performance of kanban controlled two-stage assembly systems. The development of the approximation is as follows. The considered continuous-time Markov chain is aggregated keeping the model exact, and this aggregate model is approximated replacing some state-dependent transition rates with constant rates. The approximate aggregate model is, then, decomposed into submodels and a product-form steady-state distribution is obtained for each submodel. Finally, the submodels are combined in such a way that the size of the problem becomes independent of the number of kanbans. This leads to the computational advantage in solving the combined model using numerical matrix-geometric solution algorithms. Numerical comparisons of the combined model with simulation, exact model, approximate aggregate model and an approximation in the literature show that the proposed approximation performs well in terms of accuracy and computational burden.

Robust approximate aggregation in sensor data management systems

In the emerging area of sensor-based systems, a significant challenge is to develop scalable, fault-tolerant methods to extract useful information from the data the sensors collect. An approach to this data management problem is the use of sensor database systems, which allow users to perform aggregation queries such as MIN, COUNT, and AVG on the readings of a sensor network. In addition, more advanced queries such as frequency counting and quantile estimation can be supported. Due to energy limitations in sensor-based networks, centralized data collection is generally impractical, so most systems use in-network aggregation to reduce network traffic. However, even these aggregation strategies remain bandwidth-intensive when combined with the fault-tolerant, multipath routing methods often used in these environments. To avoid this expense, we investigate the use of approximate in-network aggregation using small sketches. We present duplicate-insensitive sketching techniques that can be implemented efficiently on small sensor devices with limited hardware support and we analyze both their performance and accuracy. Finally, we present an experimental evaluation that validates the effectiveness of our methods.

The spatial and vertical distribution of living (stained) benthic foraminifera from a tropical, intertidal environment, north Queensland, Australia

We investigated the distribution of living (stained) benthic foraminifera across a tropical, intertidal shoreline adjacent to Cocoa Creek, Queensland, Australia for the purpose of better understanding the nature of test production and ultimately fossil assemblage development within such environments. Short cores (up to 1 m) were collected during the wet and dry season, along an elevational gradient comprising non-vegetated intertidal mudflat and higher-intertidal mangrove forest environments. The distribution of stained specimens can be broadly delineated into assemblages characterising ‘upper mangrove’ (2.64–2.91 m above Lowest Astronomical Tide (LAT)) and ‘low mangrove-mudflat’ (1.62–2.18 m above LAT) environments. Agglutinated species were generally limited to upper mangrove stations. Calcareous species occurred within all of the intertidal environments examined but differ in their composition between upper and lower intertidal settings. Upper mangrove faunas were characterised by the agglutinated species Arenoparrella mexicana, Haplophragmoides wilberti, Miliammina fusca, Miliammina obliqua and Trochammina inflata and the calcareous species Helenina anderseni. Live (stained) assemblages at lower intertidal elevations were dominated by the calcareous species Ammonia aoteana, as well as Rosalina spp., Elphidium oceanicum, Triloculina oblonga, Ammonia pustulosa and Shackoinella globosa. Absolute densities of stained specimens were lowest within the low mangrove sediments (6–8 specimens per cm 2) but higher within the upper mangrove (25–249 specimens per cm 2) and mudflat (8–77 specimens per cm 2). Abundances within mudflat sediments varied seasonally (being significantly larger in the wet season) whereas those within mangrove zone sediments changed little through the year. Stained foraminifera were found to a maximum depth of 50 cm at Cocoa Creek, extending to the greatest depth at the upper mangrove stations, and were generally shallowest beneath the lower mangroves. All stations had over 50% of their foraminiferal standing crop below the upper 1 cm, although at most stations < 10% of living specimens occurred within this layer. Comparison of this data with dead assemblages suggests that: (1) abundances of stained specimens, averaged over two sampling events, reasonably approximate aggregate in situ test production; and (2) a clear enrichment of infaunal species occurs within the mangrove. The latter effect was less conspicuous on the mudflat due to high rates of mixing and assemblage time-averaging, which obscure the signal of test production within the dead assemblages. In both cases, the validity and stratigraphic resolution of surface assemblages for sea-level and other palaeoenvironmental reconstructions is questionable.

Approximate reduction of non-linear discrete models with two time scales

The aim of this work is to present a general class of nonlinear discrete time models with two time scales whose dynamics is susceptible of being approached by means of a reduced system. The reduction process is included in the so-called approximate aggregation of variables methods which consist of describing the dynamics of a complex system involving many coupled variables through the dynamics of a reduced system formulated in terms of a few global variables. For the time unit of the discrete system we use that of the slow dynamics and assume that fast dynamics acts a large number of times during it. After introducing a general two-time scales nonlinear discrete model we present its reduced accompanying model and the relationships between them. The main result proves that certain asymptotic behaviours, hyperbolic asymptotically stable (A.S.) periodic solutions, to the aggregated system entail that to the original system.

Dynamic adaptive data structures for monitoring data streams

The monitoring of data streams is a very important issue in many different areas. Aspects such as accuracy, the speed of response, the use of memory and the adaptability to the changing nature of data may vary in importance depending on the situation. Examples such as Web page access monitoring, approximate aggregation in relational queries or IP message routing are clear examples of a varied range of those needs. There are different data structures that deal with this problem such as the counting bloom filters, the spectral bloom filters and the dynamic count filters. Those data structures range from static to complex dynamic representations of the data stream that keep an approximate count of the number of occurrences for each data value. In this paper, we focus on three main aspects. First, we analyze the problem in perspective and review the existing static and dynamic solutions. Second, we propose and analyze in depth a simple yet powerful partitioning strategy that reinforces the advantages of the methods proposed up to now solving most of their drawbacks. Finally, using real executions and mathematical models, we evaluate the existing methods alone and in combination with our partitioning strategy. We show that with our partitioning strategy, it is possible to reduce the memory requirements and average response time, improving the adaptiveness to changing data characteristics and leaving the accuracy of the partitioned dynamic data structures intact.

Bottom-k sketches

A Bottom-k sketch is a summary of a set of items with nonnegative weights. Each such summary allows us to compute approximate aggregates over the set of items. Bottom- k sketches are obtained by associating with each item in a ground set an independent random rank drawn from a probability distribution that depends on the weight of the item. For each subset of interest, the bottom- k sketch is the set of the k minimum ranked items and their ranks. Bottom- k sketches have numerous applications. We develop and analyze data structures and estimators for bottom- k sketches to facilitate their deployment. We develop novel estimators and algorithms that show that they are a superior alternative to other sketching methods in both efficiency of obtaining the sketches and the accuracy of the estimates derived from the sketches.

The Clustered AGgregation (CAG) technique leveraging spatial and temporal correlations in wireless sensor networks

Sensed data in Wireless Sensor Networks (WSN) reflect the spatial and temporal correlations of physical attributes existing intrinsically in the environment. In this article, we present the Clustered AGgregation (CAG) algorithm that forms clusters of nodes sensing similar values within a given threshold (spatial correlation), and these clusters remain unchanged as long as the sensor values stay within a threshold over time (temporal correlation). With CAG, only one sensor reading per cluster is transmitted whereas with Tiny AGgregation (TAG) all the nodes in the network transmit the sensor readings. Thus, CAG provides energy efficient and approximate aggregation results with small and often negligible and bounded error. In this article we extend our initial work in CAG in five directions: First, we investigate the effectiveness of CAG that exploits the temporal as well as spatial correlations using both the measured and modeled data. Second, we design CAG for two modes of operation (interactive and streaming) to enable CAG to be used in different environments and for different purposes. Interactive mode provides mechanisms for one-shot queries, whereas the streaming mode provides those for continuous queries. Third, we propose a fixed range clustering method, which makes the performance of our system independent of the magnitude of the sensor readings and the network topology. Fourth, using mica2 motes, we perform a large-scale measurement of real environmental data (temperature and light, both indoor and outdoor) and the wireless radio reliability, which were used for both analytical modeling and simulation experiments. Fifth, we model the spatially correlated data using the properties of our real world measurements. Our experimental results show that when we compute the average of sensor readings in the network using the CAG interactive mode with the user-provided error threshold of, 20%, we can save 68.25% of transmissions over TAG with only 2.46% inaccuracy in the result. The streaming mode of CAG can save even more transmissions (up to 70.24% in our experiments) over TAG, when data shows high spatial and temporal correlations. We expect these results to hold in reality, because we used the mica2 radio profile and empirical datasets for our simulation study. CAG is the first system that leverages spatial and temporal correlations to improve energy efficiency of in-network aggregation. This study analytically and empirically validates CAG's effectiveness.

Error minimization in approximate range aggregates

Histogram techniques have been used in many commercial database management systems to estimate a query result size. Recently, it has been shown that they are very effective to support approximation of query processing especially aggregates. In this paper, we investigate the problem of minimizing average errors of approximate aggregates using histogram techniques. Firstly, we present a novel linear-spline histogram model that is more accurate than the existing models. Secondly, we propose a novel histogram construction technique for minimizing such average errors, which is shown to generate a near optimal histogram. Our experiment results demonstrate that the new histogram construction techniques lead to a great accuracy improvement on the existing techniques.