Number Of Buckets Research Articles

Enhancing histograms by tree-like bucket indices

Histograms are used to summarize the contents of relations into a number of buckets for the estimation of query result sizes. Several techniques have been proposed in the past for determining bucket boundaries which provide accurate estimations. However, while search strategies for optimal bucket boundaries are rather sophisticated, no much attention has been paid for estimating queries inside buckets and all of the above techniques adopt naive methods for such an estimation. This paper focuses on the problem of improving the estimation inside a bucket once its boundaries have been fixed. The proposed technique is based on the addition, to each bucket, of a memory-word additional information (organized into a tree-like index), storing approximate cumulative frequencies in a hierarchical fashion. Both theoretical analysis and experimental results show that the proposed approach improves the accuracy of the estimation inside buckets, w.r.t. both classical approaches (like continuous value assumption and uniform spread assumption) and a number of alternative ways to organize the additional information. The index is later added to state-of-the-art histograms obtaining the non-obvious result that despite the spatial overhead which reduces the number of allowed buckets once the storage space has been fixed, the original methods are strongly improved in terms of accuracy.

Compressed histograms with arbitrary bucket layouts for selectivity estimation

Selectivity estimation is an important step of query optimization in a database management system, and multi-dimensional histogram techniques have proved promising for selectivity estimation. Recent multi-dimensional histogram techniques such as GenHist and STHoles use an arbitrary bucket layout. This layout has the advantage of requiring a smaller number of buckets to model tuple densities than those required by the traditional grid or recursive layouts. However, the arbitrary bucket layout brings an inherent disadvantage of requiring more memory to store each bucket location information. This diminishes the advantage of requiring fewer buckets and, therefore, has an adverse effect on the resulting selectivity estimation accuracy. To our knowledge, however, no existing histogram-based technique with arbitrary layout addresses this issue. In this paper, we introduce the idea of bucket location compression and then demonstrate its effectiveness for improving selectivity estimation accuracy by proposing the STHoles+ technique. STHoles+ extends STHoles by quantizing each coordinate of a bucket relative to the coordinate of the smallest enclosing bucket. This quantization increases the number of histogram buckets that can be stored in the histogram. Our quantization scheme allows STHoles+ to trade precision of histogram bucket locations for storing more buckets. Experimental results show that STHoles+ outperforms STHoles on various data distributions, query distributions, and other factors such as available memory size, quantization resolution, and dimensionality of the data space.

Estimating dispersal rate of the silky cane weevil (Coleoptera: Curculionidae)*

Abstract: The objective of this study was to estimate the silky cane weevil rate of dispersal under near‐natural conditions inside a screened enclosure where an array of buckets was baited with cut sugarcane stalks. One hundred weevils were released and weevils inside the buckets were counted hourly for 8 h, and then 24 and 48 h after release. A passive diffusion model was used to estimate the weevil's dispersal and disappearance rates, within and between rows of buckets with sugarcane. The weevils concentrated around the release point and slowly moved towards the boundaries of the experimental plot over time with an overall average dispersal rate of 2.8 ± 3.58 cm2/h. Dispersal and disappearance rates within and between rows were not significantly different among the time intervals considered (1–8, 8–24 and 24–48 h after release) except for the 1–8 time interval on the array representing the release point when the dispersal rate, D, was significantly higher than those at other time intervals. Continuum of the substratum to disperse from one side of the array to another via a wooden bridge may explain the higher dispersal rate through this array. The number of buckets exposed to the sun during the morning hours was significantly higher on those rows exposed to the sun (south side of the screen enclosure) than on the shaded side. Longer times of bucket exposure to the sun may explain the predominant distribution of weevils in that area suggesting that the weevil population is constantly expanding and retracting according to micro environmental conditions.

Perfect Hashing Schemes for Mining Association Rules

Hashing schemes are widely used to improve the performance of data mining association rules, as in the DHP algorithm that utilizes the hash table in identifying the validity of candidate itemsets according to the number of the table's bucket accesses. However, since the hash table used in DHP is plagued by the collision problem, the process of generating large itemsets at each level requires two database scans, which leads to poor performance. In this paper we propose perfect hashing schemes to avoid collisions in the hash table. The main idea is to employ a refined encoding scheme, which transforms large itemsets into large 2-itemsets and thereby makes the application of perfect hashing feasible. Our experimental results demonstrate that the new method is also efficient (about three times faster than DHP), and scalable when the database size increases. We also propose another variant of the perfect hash scheme with reduced memory requirements. The properties and performances of several perfect hashing schemes are also investigated and compared.

부하 균형 유지를 고려한 파이프라인 해시 조인 방법

다중 조인 연산의 파이프라인 방식 처리에서 조인 어트리뷰트의 자료 불균형(data skew)이 성능에 주는 영향을 연구하고, 자료 불균형을 대비하여 적재부하를 라운드-로빈 방식으로 정적 분할하는 방법과 자료분포도를 이용하여 적응적으로 분할하는 두 가지 파이프라인 해시 조인 알고리즘을 제안한다. 해시 기반 조인을 사용하면 여러 개의 조인을 파이프라인 방식으로 처리할 수 있다. 다중 조인의 파이프라인 방식 처리는 조인 중간 결과를 디스크를 통하지 않고 다른 프로세서에게 직접 전달하므로 효율적이다. 파이프라인 해시 조인 알고리즘이 자료 불균형을 대비한 부하 균형 유지 메커니즘을 갖고 있지 않다면 자료 불균형은 성능에 매우 심각한 영향을 줄 수 있다. 본 논문은 자료 불균형의 영향과 제안된 두 가지 기법을 비교하기 위하여 파이프라인 세그먼트의 실행 모형, 비용 모형, 그리고 시뮬레이터를 개발한다. 다양한 파라미터로 모의 실험을 한 결과에 의하면 자료 불균형은 조인 선택도와 릴레이션 크기에 비례하여 시스템 성능을 떨어뜨림을 보여준다. 그러나 제안된 파이프 라인 해시 조인 알고리즘은 다수의 버켓 사용과 분할의 조율을 통해 자료 불균형도가 심한 경우에도 좋은 성능을 갖게 한다. We investigate the effect of the data skew of join attributes on the performance of a pipelined multi-way hash join method, and propose two new hash join methods with load balancing capabilities. The first proposed method allocates buckets statically by round-robin fashion, and the second one allocates buckets adaptively via a frequency distribution. Using hash-based joins, multiple joins can be pipelined so that the early results from a join, before the whole join is completed, are sent to the next join processing without staying on disks. Unless the pipelining execution of multiple hash joins includes some load balancing mechanisms, the skew effect can severely deteriorate system performance. In this paper, we derive an execution model of the pipeline segment and a cost model, and develop a simulator for the study. As shown by our simulation with a wide range of parameters, join selectivities and sizes of relations deteriorate the system performance as the degree of data skew is larger. But the proposed method using a large number of buckets and a tuning technique can offer substantial robustness against a wide range of skew conditions.

Synchronization of picosecond laser pulses to the target X-ray pulses at SPring-8

Synchronization system between an intense picosecond laser and the target X-ray pulses has been developed at SPring-8. The intense laser pulses were obtained by amplification of the pulses picked up from a mode-locked Ti:sapphire laser synchronized with the radio frequency of the storage ring. The repetition rate of amplified laser pulses was controlled to be 1/ n of the RF, where n is a multiple of the number of RF buckets in the ring, so that the laser pulses meet the SR pulses originated from a particular electron bunch in partial filling patterns. The temporal overlap of the laser and the target X-ray pulses was achieved as monitored with a streak camera in synchroscan and repetitive single shot operation modes, and was stable with a precision of a few ps for several hours.

Global optimization of histograms

Histograms are frequently used to represent the distribution of data values in an attribute of a relation. Most previous work has focused on identifying the optimal histogram (given a limited number of buckets) for a single attribute independent of other attributes/histograms . In this paper, we propose the idea of global optimization of histograms , i.e., single-attribute histograms for a set of attributes are optimized collectively so as to minimize the overall error in using the histograms. The idea is to allocate more buckets to histograms whose attributes are more frequently used and/or distributions are highly skewed. While the accuracy of some histograms is penalized (being assigned fewer buckets), we expect the global error to be low compared to the traditional method (of allocating equal number of buckets to each histogram). We propose two algorithms to determine the histograms to construct for a collection of attributes. The first is based on dynamic programming, and the second is a greedy algorithm. We compare the overall error of these algorithms against the traditional method. Extensive experiments are conducted and the results confirm the benefits of global optimal histograms in reducing the overall error. The extent of improvement depends on the data and query distributions, ranging from no benefit when there is no significant differences in the data distributions to over a factor of 100 reduction in error in some cases we tried. The time to compute global optimal histogram using dynamic programming is much longer than the time to compute optimal histograms separately for each attribute, and the difference widens at a faster rate as the number of histograms increases. With the greedy algorithm, the time penalty is small, but the error reduction is somewhat less as well. We propose a third algorithm, called greedy algorithm with remedy, that has running time similar to the greedy algorithm, but produces results close to global optimum. In fact, in every experiment that we tried, this algorithm found the exact global optimum.

GSn: A new service type for integrated services on the internet

AbstractNetwork services with deterministic guarantees are based on a worst—case description of user—generated traffic. When designing a policing and scheduling algorithm for guaranteed services on the Internet, accuracy of description of the traffic profile has to be traded with simplicity of implementation. The result of this trade off is often expressed as the number of token buckets required by the service along with the choice of their parameters. The GS type of service proposed by the IETF uses two token buckets both for characterizing the traffic and for policing it. The choice of using only two token buckets is primarily driven by policing costs. In this paper we propose a method that allows the number of token buckets used for characterizing the traffic to be greater than what is actually needed to police it. This means we can obtain an accurate profile of the traffic while keeping policing simple. The method consists of computing a profile of the traffic which involves a number of token buckets of the order of ten, and then doing the policing using only the first token bucket, plus another one which is chosen depending on the delay requirements of the receivers. This paper shows that with this simple enhancement we obtain a guaranteed service whose performance closely approaches the theoretical limits of services with deterministic guarantees.

Analytic variations on bucket selection and sorting

We provide complete average-case as well as probabilistic analysis of the cost of bucket selection and sorting algorithms. Two variations of bucketing (and flavors therein) are considered: distributive bucketing (large number of buckets) and radix bucketing (recursive with a small number of buckets, suitable for digital computation). For Distributive Selection a compound Poisson limit is established. For all other flavors of bucket selection and sorting, central limit theorems underlying the cost are derived by asymptotic techniques involving perturbation of Rice's integral and contour integration (saddle point methods). In the case of radix bucketing, periodic fluctuations appear in the moments of both the selection and sorting algorithms.

Bucket Hashing and Its Application to Fast Message Authentication

We introduce a new technique for constructing a family of universal hash functions. At its center is a simple metaphor: to hash a string x , cast each of its words into a small number of buckets; xor the contents of each bucket; then collect up all the buckets' contents. Used in the context of Wegman—Carter authentication, this style of hash function provides a fast approach for software message authentication.

Traffic characterization algorithms for VBR video in multimedia networks

A network that offers deterministic, i.e., worst case, quality-of-service guarantees to variable-bit-rate (VBR) video must provide a resource reservation mechanism that allocates bandwidth, buffer space, and other resources for each video stream. Such a resource reservation scheme must be carefully designed, otherwise network resources are wasted. A key component for the design of a resource reservation scheme is the traffic characterization method that specifies the traffic arrivals on a video stream. The traffic characterization should accurately describe the actual arrivals, so that a large number of streams can be supported; but it must also map directly into efficient traffic-policing mechanisms that monitor arrivals on each stream. In this study, we present a fast and accurate traffic characterization method for stored VBR video in networks with a deterministic service. We use this approximation to obtain a traffic characterization that can be efficiently policed by a small number of leaky buckets. We present a case study where we apply our characterization method to networks that employ a dynamic resource reservation scheme with renegotiation. We use traces from a set of 25–30-min MPEG sequences to evaluate our method against other characterization schemes from the literature.

Efficiently extendible mappings for balanced data distribution

In data storage applications, a large collection of consecutively numbered data “buckets” are often mapped to a relatively small collection of consecutively numbered storage “bins.” For example, in parallel database applications, buckets correspond to hash buckets of data and bins correspond to database nodes. In disk array applications, buckets correspond to logical tracks and bins correspond to physical disks in an array. Measures of the “goodness” of a mapping method include: One contribution of this paper is to give a new mapping method, theInterval-Round-Robin (IRR) method. The IRR method has optimal balance and relocation cost, and its time complexity and storage requirements compare favorably with known methods. Specifically, ifm is the number of times that the number of bins and/or buckets has increased, then the time complexity isO(logm) and the storage isO(m 2). Another contribution of the paper is to identify the concept of ahistory-independent mapping, meaning informally that the mapping does not “remember” the past history of expansions to the number of buckets and bins, but only the current number of buckets and bins. Thus, such mappings require very little information to be stored. Assuming that balance and relocation are optimal, we prove that history-independent mappings are possible if the number of buckets is fixed (so only the number of bins can increase), but not possible if the number of bins and buckets can both increase.

Optimizing sort order query execution in balanced and nested grid files

Disk input/output (I/O) efficient query execution is an important topic with respect to DBMS performance. In this context, we elaborate on the construction of disk access plans for sort order queries in balanced and nested grid files. The key idea is to use the order information contained in the directory of the multiattribute search structure. The presented algorithms are shown to yield a significant decrease in the number of disk I/O operations by appropriate use of the order information. Two algorithms for the construction of appropriate disk access plans are proposed, namely a greedy approach and a heuristic divide-and-conquer approach. Both approaches yield considerable I/O savings compared to straightforward query processing without consideration of any directory order information. The former performs well for small buffer page allocations, i.e., for a small number of buffer pages relative to the number of data buckets processed in the query. The latter is superior to the greedy algorithm with respect to the total number of I/O operations and with respect to the overall maximum of buffer pages needed to achieve the minimal number of disk I/O operations. Both approaches rely on a binary trie as a temporary data structure. This trie is used as an explicit representation of the order information. The storage consumption of the temporary data structure is shown to be negligible in realistic cases, Even for pathological cases with respect to degenerated balanced and nested grid files, reasonable upper bounds can be given. >

Chaos in a Real System

Many studies of chaos involve using simplified equations with little relation to a real system. Lorenz proposed a real system that demonstrates chaotic behavior - the classic Lorenz water wheel. This wheel, a mechanical analog to fluid heating in a box, consists of a number of buckets on the outside of a spoked wheel which rotates in a vertical plane. A filling source at the top of the wheel fills buckets as they pass under it. The filled buckets cause rotational dynamics, which at times exhibit a chaotic behavior. The requisite equations for a four-bucket wheel (sixth-order nonlinear model) are developed and implementation considerations enumerated. Some typical responses are presented.

On the optimal ordering of multiple-field tables

Consider a table of n d-dimensional records, grouped into b buckets, and a set Q = {( Q h , w h )} of weighed partial-match conditions, where w h is the relative frequency of Q h . Let n( Q h ) be the number of records which satisfy Q h , and b( Q h ) the number of buckets in which these records are found. The problem we consider is the individuation of the optimal ordering field which minimizes the sum of accessed buckets, B( Q) = Σ h w h × b( Q h ). An exact solution requires the records to be sorted according to the values of each of the d fields in turn with an overall time complexity, evaluated as a function of bucket accesses, of O( d × b log b). We present an approximate O( b) algorithm which estimates the optimal ordering without the need to sort the table at all. The algorithm makes use of a probabilistic counting technique, known as linear counting, which requires a single scan of the table. Experimental results show that in most cases the approximate solution agrees with the exact one.

The Joint Distribution of Elastic Buckets in Multiway Search Trees

Random search trees are studied when they grow under a general computer memory management scheme. In a general scheme, the space is released in buckets of certain predesignated sizes. For a search tree with branch factor m, the nodes may hold up to $m - 1$ keys. Suppose the buckets of the memory management scheme that can hold less than m keys have key capacities $c_1 , \ldots ,c_r $. The search tree must then be implemented with multitype nodes of these capacities. After n insertions, let $X_n^{(i)} $ be the number of buckets of type i (i.e., of capacity $c_i $,$1 \leqslant i \leqslant p$. The multivariate structure of the tree is investigated. For the vector ${\bf X}_n = ( \leqslant X_n^{(1)} , \ldots ,X_n^{(p)} )^T $, the asymptotic mean and covariance matrix are determined. Under practical memory management schemes, all variances and covariances experience a phase transition: For $3 \leqslant m \leqslant 26$, all variances and covariances are asymptotically linear in n; for higher branch factors the variances and covariances become a superlinear (but ubquadratic) function of n. The joint distribution of ${\bf X}_n $ is shown to be multivariate normal in a range of m. While the tree is growing, conversions between types are necessary. A multivariate problem concerning these conversions with an asymptotic multivariate normal distribution is also studied. The fixed bucket, exact fit, and buddy system allocation schemes will serve as illustrating examples.

Maximum Size of a Dynamic Data Structure: Hashing with Lazy Deletion Revisited

The dynamic data structure management technique called hashing with lazy deletion (HwLD) is studied. A table managed under HwLD is built by a sequence of insertions and deletions of items. When hashing with lazy deletions, one does not delete items as soon as possible but keeps more items in the data structure than would be the case with immediate-deletion strategies. This deferral allows the use of a simpler deletion algorithm, leading to a lower overhead—in space and time—for the HwLD implementation. It is of interest to know how much extra space is used by HwLD. This paper investigates the maximum size and the excess space used by HwLD, under general probabilistic assumptions, by using the methodology of queueing theory. In particular, for the Poisson arrivals and general lifetime distribution of items, the excess space does not exceed the number of buckets in HwLD. As a byproduct of the analysis, the limiting distribution of the maximum queue length in an $M|G|\infty $ queueing system is also derived....

Discharge and mechanical efficiency of Egyptian water wheels

Water wheels are used throughout Egypt to lift water from a supply canal to field level. The most common wheel design employs spiral channels to lift the water. The discharge per revolution of these wheels is an important parameter in evaluating their performance. A technique was developed to determine the discharge per wheel revolution from data on the geometry of the submerged area collected using a Cartesian coordinate measuring device. Data on thirty water wheels from the Mansuriya and Kafr El-Sheikh regions of Egypt were collected. In addition, it was found that water wheels with curves conforming to the Archimedes spiral are geometrically similar if they have the same number of buckets and either the same characteristic angle or the same characteristic submergence. This led to normalized characteristic curves for four and six bucket wheels that can be used to rapidly estimate the discharge per revolution, in the field, from a few easily measured parameters. The relative benefits of this method compared to flumes are discussed. In addition, the mechanical efficiencies of eleven field installations were determined with values ranging from 54% to 22%. The sources of inefficiency are also discussed herein.

Performance comparison of extendible hashing and linear hashing techniques

Based on seven assumptions, the following comparison factors are used to compare the performance of linear hashing with extendible hashing: 1. storage utilization; 2. average unsuccessful search cost; 3. average successful search cost; 4. split cost; 5. insertion cost; 6. number of overflow buckets. The simulation is conducted with the bucket sizes of 10, 20, and 50 for both hashing techniques. In order to observe their average behavior, the simulation uses 50,000 keys which have been generated randomly.According to our simulation results, extendible hashing has an advantage of 5% over linear hashing in terms of storage utilization. Successful search, unsuccessful search, and insertions are less costly in linear hashing. However, linear hashing requires a large overflow space to handle the overflow records. Simulation shows that approximately 10% of the space should be marked as overflow space in linear hashing.Directory size is a serious bottleneck in extendible hashing. Based on the simulation results, the authors recommend linear hashing when main memory is at a premium.

On the design of multiple key hashing files for concurrent orthogonal range retrieval between two disks

This paper concerns the following problem: given a set of multi-attribute records, a fixed number of buckets and a two-disk system, arrange the records into the buckets and then store the buckets between the disks in such a way that, over all possible orthogonal range queries (ORQs), the disk access concurrency is maximized. We shall adopt the multiple key hashing (MKH) method for arranging records into buckets and use the disk modulo (DM) allocation method for storing buckets onto disks. Since the DM allocation method has been shown to be superior to any other allocation methods for allocating an MKH file onto a two-disk system for answering ORQs, the real issue is knowing how to determine an optimal way for organizing the records into buckets based upon the MKH concept. A performance formula that can be used to evaluate the average response time, over all possible ORQs, of an MKH file in a two-disk system using the DM allocation method is first presented. Based upon this formula, it is shown that our design problem is related to a notoriously difficult problem, namely the Prime Number Problem. Then a performance lower bound and an efficient algorithm for designing optimal MKH files in certain cases are presented. It is pointed out that in some cases the optimal MKH file for ORQs in a two-disk system using the DM allocation method is identical to the optimal MKH file for ORQs in a single-disk system and the optimal average response time in a two-disk system is slightly greater than one half of that in a single-disk system.