Iceberg Queries Research Articles

Increasing the Performance of Iceberg Query Through Summary Tables

Increasing the Performance of Iceberg Query Through Summary Tables

SecBerg: Secure and Practical Iceberg Queries in Cloud

Secure queries are fundamental to data security, particularly in cloud databases. In data analytics, one of the common and practical queries is the iceberg query that can find aggregate values above a specified threshold. However, existing secure aggregate query schemes: i) are unable to support secure iceberg queries equipped with the HAVING clause; ii) only consider additive aggregate functions; and iii) suffer from performance issues due to the use of homomorphic encryption to encrypt databases. In this paper, we present a secure iceberg query scheme, SecBerg, to support both addition-based and comparison-based aggregate functions and ensure high efficiency and security simultaneously. To make it possible, we propose a secure bitmap index system to encode database values and pioneer the use of the arithmetic secret sharing technique to protect databases in the cloud environment. Furthermore, we carefully design efficient and secure protocols over arithmetic secret sharing to construct our SecBerg. Extensive evaluations are conducted, and the results indicate that SecBerg is significantly more efficient than the state-of-the-art relevant scheme in computational overhead and can attain orders of magnitude performance improvement at best.

Computing Iceberg Queries on Map Reduce Framework

Computing Iceberg Queries on Map Reduce Framework

Pallam

Pallam

Domain Partitioning: Approach to Computing Average Iceberg Queries

: Data analytics and data mining systems work on data which stored in files, the files are not store relationships among the data, from such kind of data we compute aggregate values over the set of required attributes for find insights of data, find attributes values which aggregation values greater than threshold such kind of queries called iceberg queries. Computing iceberg queries with average aggregate function is default, because limited memory available. Existing method are suffers with re-computation of candidate. We proposed a Record Traction Algorithm(RTA) ,it use Domain partitioning approach, it avoid re-computation of candidate in during next scan of data set, it use bit vector and bitmap numbers for Domain Partitioning the data, our experiment reveals that our approach generate a candidate only once and input data will reduced in further candidate sets.

Computing SUM and COUNT aggregate functions of Iceberg query using LAM strategy

International Journal of Computer Sciences and Engineering (A UGC Approved and indexed with DOI, ICI and Approved, DPI Digital Library) is one of the leading and growing open access, peer-reviewed, monthly, and scientific research journal for scientists, engineers, research scholars, and academicians, which gains a foothold in Asia and opens to the world, aims to publish original, theoretical and practical advances in Computer Science,Information Technology, Engineering (Software, Mechanical, Civil, Electronics & Electrical), and all interdisciplinary streams of Computing Sciences. It intends to disseminate original, scientific, theoretical or applied research in the field of Computer Sciences and allied fields. It provides a platform for publishing results and research with a strong empirical component. It aims to bridge the significant gap between research and practice by promoting the publication of original, novel, industry-relevant research.

A novel approach for iceberg query evaluation on multiple attributes using set representation

Iceberg query (IBQ) can be an really identifying kind of aggregation question that calculate aggregations up-on user given threshold(T). In data mining field, effective investigation of compounding queries was because of by the majority of investigators because the tremendous generation of information outside of industrial and businesses industries. Conclusion assist database and discovery of the majority of information connected systems largely calculate the worthiness of most fascinating features having an critical level of information from data foundations that may be tremendous. By means of the paper, we propose that an initial Manner of calculating IBQ, which builds a choice for every attribute nicely value, but additionally includes a One of a Kind events Inside the attribute column also plays specify operations for creating closing Outcomes. We formulated highly effective GUI software for just 2 characteristics, numerous traits employing egotistical prepare and several features utilizing lively plan. If data collection comprises two traits, then it truly is substantially more advanced than apply just two traits. In the event of information collection comprises multiple traits, predicated up on anyone choice suitable module could potentially be decided on. If characteristic uniqueness changes from characteristic in to the following characteristic, then vibrant variety approach is very powerful. This strategy somewhat reduces performance memory and time space contrast with additional processes. A experiment using artificial Statistics collection and actual info demonstrates our strategy will be considerably more effective compared to present apps for Nearly Every threshold.

MAC: Missing Tag Iceberg Queries for Multi-Category RFID Systems

Recently radio frequency identification (RFID) is promoting the rapid development of Internet of Things and has been widely applied in numerous industrial applications such as inventory management, object tracking and smart logistics, etc. Tags in some RFID applications can be classified into multiple categories according to the types of objects they are attached to. One of the important functionalities for the multi-category RFID systems is the missing tag cardinality estimation of each category. This paper focuses on solving the missing tag iceberg query problem for multi-category RFID systems, i.e., to determine a set of categories, whose missing tags are more than a threshold, with a required reliability. We firstly propose two basic m issing t a g i c eberg query schemes called MAC-SZE and MAC-HZE, which adopt the singleton-zero estimator and homogeneous-zero estimator, respectively. To further improve the query efficiency, we then propose two segmented enhanced schemes called MAC-SSZE and MAC-SHZE, which eliminate the unnecessary slots in each frame. We conduct theoretical analysis to guarantee the required reliability is satisfied. The simulations are finally conducted and the results illustrate that the proposed missing tag iceberg query schemes greatly outperform other existing one.

A Survey: Computing Iceberg Queries

Knowledge discovery, tasks are deals with huge no of records. Queries are needed to identify unique attributes values and their aggregate that is above a predefined threshold from this huge number of records. This type of queries is called iceberg queries. Iceberg queries requires huge amount of main memory and takes longer time to answer the query. As computer system has limited amount of main memory, the processing of iceberg queries is a challenging task. This paper discusses different methods that are in literature for processing iceberg queries , we also explore pros and cons of these methods and future scope.

Evaluating Aggregate Functions of Iceberg Query Using Priority Based Bitmap Indexing Strategy

Aggregate function and iceberg queries are important and common in many applications of data warehouse because users are generally interested in looking for variance or unusual patterns. Normally, the nature of the queries to be executed on data warehouse are the queries with aggregate function followed by having clause, these type of queries are known as iceberg query. Especially to have efficient techniques for processing aggregate function of iceberg query is very important because their processing cost is much higher than that of the other basic relational operations such as SELECT and PROJECT. Presently available iceberg query processing techniques faces the problem of empty bitwise AND,OR XOR operation and requires more I/O access and time.To overcome these problems proposed research provides efficient algorithm to execute iceberg queries using priority based bitmap indexing strategy. Priority based approach consider bitmap vector to be executed as per the priority.Intermediate results are evaluated to find probability of result.Fruitless operations are identified and skipped in advance which help to reduce I/O access and time.Time and iteration required to process query is reduced [45-50] % compare to previous strategy. Experimental result proves the superiority of priorty based approach compare to previous bitmap processing approach.

Scalable and adaptive collaborative filtering by mining frequent item co-occurrences in a user feedback stream

Neighborhood-based methods are one of the mainstream approaches to collaborative filtering. A common problem with these methods is scalability to large number of users and items. Consequently, the adaptivity of a neighborhood-based model to system dynamics is often compromised due to model constraints and prolonged training intervals. These drawbacks can be important in designing demanding applications of today and the future. In this paper, we propose a novel real-time scalable and adaptive collaborative filtering algorithm, SASCF, suitable for personalized and item-to-item recommendations, in which the underlying neighborhood-based model is updated on-the-fly with the streaming user feedback. The algorithm does not perform an offline search for finding nearest neighbors in a full item similarity matrix. Instead, taking a landmark window over the user feedback stream, a space-efficient summary structure is maintained. This structure corresponds to the result of a standing iceberg query for finding every item's top-k frequently co-occurring items over a specified support threshold. Mining such frequent co-occurrences can facilitate approximate computation of several useful item similarity measures. The algorithm offers scalability thanks to the space-efficient summary structure which handles ever-changing users, items, and item similarities in a resource aware fashion. It also offers adaptivity in the sense that newly arriving user-item interactions are immediately integrated into the model. The model is always up-to-date and it can readily be used to recommend items to users with the most recent information.

A parallel space saving algorithm for frequent items and the Hurwitz zeta distribution

We present a message-passing based parallel version of the Space Saving algorithm designed to solve the k–majority problem. The algorithm determines in parallel frequent items, i.e., those whose frequency is greater than a given threshold, and is therefore useful for iceberg queries and many other different contexts. We apply our algorithm to the detection of frequent items in both real and synthetic datasets whose probability distribution functions are a Hurwitz and a Zipf distribution respectively. Also, we compare its parallel performances and accuracy against a parallel algorithm recently proposed for merging summaries derived by the Space Saving or Frequent algorithms.

Efficient Protocols for Collecting Histograms in Large-Scale RFID Systems

Collecting histograms over RFID tags is an essential premise for effective aggregate queries and analysis in large-scale RFID-based applications. In this paper we consider an efficient collection of histograms from the massive number of RFID tags, without the need to read all tag data. In order to achieve time efficiency, we propose a novel, ensemble sampling-based method to simultaneously estimate the tag size for a number of categories. We first consider the problem of basic histogram collection, and propose an efficient algorithm based on the idea of ensemble sampling. We further consider the problems of advanced histogram collection, respectively, with an iceberg query and a top- $k$ query. Efficient algorithms are proposed to tackle the above problems such that the qualified/unqualified categories can be quickly identified. This ensemble sampling-based framework is very flexible and compatible to current tag-counting estimators, which can be efficiently leveraged to estimate the tag size for each category. Experiment results indicate that our ensemble sampling-based solutions can achieve a much better performance than the basic estimation/identification schemes.

A Framework to Process Iceberg Queries using Set-intersection and Set-Difference Operations

Many data mining queries are basically identified as iceberg queries. Applications are required to be compute aggregate functions over an interesting attributes to find aggregate values above some specified threshold. Such queries are called as iceberg queries. We propose set operations instead of bitwise-AND operations to evaluate iceberg queries efficiently using very little memory and significantly fewer passes over data, as compared to current techniques that use Dynamic pruning approaches and Vector alignment algorithms. Set operations reduces the execution time and make evaluation process of Iceberg query very effective by reducing the number of bitmaps that are needed. The exhaustive experimentation gives better results than existing strategies.

Efficient Iceberg Query Processing in Sensor Networks

Efficient Iceberg Query Processing in Sensor Networks

A Density based Priority Queue Strategy to Evaluate Iceberg Queries Efficiently using Compressed Bitmap Indices

iceberg query is a special class of aggregation query that computes aggregated values upon user interested threshold (T).The bitmap index is a common data structure for fast retrieval of matching tuples from data base table. These resultant tuples are useful to compute aggregations such as SUM, COUNT, AVG, MIN, MAX, and RANK. In this paper, we propose a density based bitmap pruning strategy to evaluate iceberg queries efficiently using compressed bitmap indices. The strategy prioritizes the vectors to be enter in to priority queue by allowing high density of 1's count that achieve optimal pruning effect. Extensive experimentation demonstrates our proposed approach is much more efficient than existing strategy.

Methods for Evaluating Iceberg Queries

Methods for Evaluating Iceberg Queries

An Algorithm to Evaluate Iceberg Query using Compacted Bitmap Vector

The data storing and retrieving are playing a major role in the data clustering and data warehousing techniques. The effectiveness of a data retrieving method depends upon the data specific queries for retrieving the data from the database. Iceberg query is a unique class of aggregation query, which computes aggregate values above a given threshold. Many data mining queries are basically ice berg queries. The major part taken into the consideration about the AND operation in the iceberg queries. The reduced number of AND operation increases the effectiveness of the iceberg query. In this work, an efficient iceberg query evaluation process is proposed by reducing the bitwise AND operations needed to find the item pairs. In the proposed approach two solutions are introduced to reduce the bitwise AND operations. Randomly identifying 'N' 1-bit positions instead of first 1-bit position and reducing the zero bit values from the Most Significant Side so that the bit map vector will be reduced in such a way, the bitwise operations needed is reduced. The experimentation is conducted on two datasets in order to evaluate the performance of the proposed iceberg query evaluation algorithm. In the case of retail datasets, the time required for processing 100000 tuples is 864 milliseconds. On the other hand, the same for synthetic dataset, T10I4D100K, is 910 milliseconds.

Efficient Iceberg Query Evaluation Using Compressed Bitmap Index

Decision support and knowledge discovery systems often compute aggregate values of interesting attributes by processing a huge amount of data in very large databases and/or warehouses. In particular, iceberg query is a special type of aggregation query that computes aggregate values above a user-provided threshold. Usually, only a small number of results will satisfy the threshold constraint. Yet, the results often carry very important and valuable business insights. Because of the small result set, iceberg queries offer many opportunities for deep query optimization. However, most existing iceberg query processing algorithms do not take advantage of the small-result-set property and rely heavily on the tuple-scan-based approach. This incurs intensive disk accesses and computation, resulting in long processing time especially when data size is large. Bitmap index, which builds one bitmap vector for each attribute value, is gaining popularity in both column-oriented and row-oriented databases in recent years. It occupies less space than the raw data and gives opportunities for more efficient query processing. In this paper, we exploited the property of bitmap index and developed a very effective bitmap pruning strategy for processing iceberg queries. Our index-pruning-based approach eliminates the need of scanning and processing the entire data set (table) and thus speeds up the iceberg query processing significantly. Experiments show that our approach is much more efficient than existing algorithms commonly used in row-oriented and column-oriented databases.

Finding frequent items in parallel

AbstractWe present a deterministic parallel algorithm for the k‐majority problem, that can be used to find in parallel frequent items, i.e. those whose multiplicity is greater than a given threshold, and is therefore useful to process iceberg queries and in many other different contexts of applied mathematics and information theory. The algorithm can be used both in the online (stream) context and in the offline setting, the difference being that in the former case we are restricted to a single scan of the input elements, so that verifying the frequent items that have been determined is not allowed (e.g. network traffic streams passing through internet routers), while in the latter a parallel scan of the input can be used to determine the actual k‐majority elements. To the best of our knowledge, this is the first parallel algorithm solving the proposed problem. Copyright © 2011 John Wiley & Sons, Ltd.