Query Processing Cost Research Articles

Exploiting SSDs in operational multiversion databases

Multiversion databases store both current and historical data. Rows are typically annotated with timestamps representing the period when the row is/was valid. We develop novel techniques to reduce index maintenance in multiversion databases, so that indexes can be used effectively for analytical queries over current data without being a heavy burden on transaction throughput. To achieve this end, we re-design persistent index data structures in the storage hierarchy to employ an extra level of indirection. The indirection level is stored on solid-state disks that can support very fast random I/Os, so that traversing the extra level of indirection incurs a relatively small overhead. The extra level of indirection dramatically reduces the number of magnetic disk I/Os that are needed for index updates and localizes maintenance to indexes on updated attributes. Additionally, we batch insertions within the indirection layer in order to reduce physical disk I/Os for indexing new records. In this work, we further exploit SSDs by introducing novel DeltaBlock techniques for storing the recent changes to data on SSDs. Using our DeltaBlock, we propose an efficient method to periodically flush the recently changed data from SSDs to HDDs such that, on the one hand, we keep track of every change (or delta) for every record, and, on the other hand, we avoid redundantly storing the unchanged portion of updated records. By reducing the index maintenance overhead on transactions, we enable operational data stores to create more indexes to support queries. We have developed a prototype of our indirection proposal by extending the widely used generalized search tree open-source project, which is also employed in PostgreSQL. Our working implementation demonstrates that we can significantly reduce index maintenance and/or query processing cost by a factor of 3. For the insertion of new records, our novel batching technique can save up to 90 % of the insertion time. For updates, our prototype demonstrates that we can significantly reduce the database size by up to 80 % even with a modest space allocated for DeltaBlocks on SSDs.

모바일 P2P 네트워크에서 효율적인 스카이라인 질의 처리 기법

본 논문에서는 모바일 P2P 환경에서 질의 처리 정확성 및 통신 비용을 향상시키기 위한 새로운 스카이라인 질의 처리 기법을 제안한다. 제안하는 기법은 사전 스카이라인 구축, 질의 배포 범위 확장 정책과 연속 스카이라인 질의 처리 과정으로 구성된다. 사전 스카이라인에서는 질의 처리 이전에 필터링 객체로 선정될 가능성을 가지고 있는 후보 객체 집합을 선정하여 질의 처리에서의 필터링 비용을 감소시킨다. 질의 배포 범위 확장 정책에서 질의 배포 범위를 확장함으로써 정확성을 향상시킨다. 또한, 최초 스카이라인 질의 처리한 후 모니터링을 수행하여 연속 스카이라인 질의를 처리한다. 제안하는 기법의 우수성을 입증하기 위해 기존 기법과 성능 평가를 수행한 결과 제안하는 기법이 기존 기법보다 우수한 성능을 보였다. In this paper, we propose a new skyline query processing scheme to enhance accuracy of query processing and communication cost in mobile P2P environments. The proposed scheme consists of three stages such as the pre-skyline processing, the query transmission range extension policy, and the continuous skyline query processing. In the pre-skyline processing, a peer selects the candidate filtering objects who have the potential to be selected. By doing so, the proposed scheme reduces the filtering cost when processing the query. In the query transmission range extension policy, we have improved the accuracy by extending the query transmission range. In addition, it can handle continuous skyline query by performing the monitoring after the first skyline query processing. In order to show the superiority of the proposed method, we compare it with the existing schemes through performance evaluation. As a result, it was shown that the proposed scheme outperforms the existing schemes.

Exploiting RASP Data Perturbation to BuildConfidential Query Services in the Cloud

With the advent of cloud computing technology, using clouds for hosting data query service has become increasingly popular because of the low cost of computation, hosting applications and content storage. As in cloud the data management and infrastructure management is provided by third-party security and privacy are the biggest concern. Until and unless data confidentiality and secure query processing are guaranteed it is always a risk for the data owner to move the sensitive data to the cloud. Workload must be reduced to fully realize the benefits of cloud computing. Therefore to meet the above said requirements RASP method is proposed where RASP stands for Random Space Perturbation. This data perturbation technique ensures that the data is not d istorted and does not lead to a security breach by allowing users to ascertain key summary information. Cloud computing enables outsourcing the management of the data related to individuals and organizations to a service provider as the hardware cost and t he maintenance cost is less. RASP provides exclusive security features for hosting query services in the cloud by satisfying the CPEL criteria where CPEL stands for data Confidentiality, query Privacy, Efficient query processing and Low working cost .KNN-R algorithm is used to process the range query and the KNN query. Here users have been authorized by the randomly generated product key value provided by the admin after successful registration followed by activation by admin thus maintaining confidentiality. User queries are retrieved within a very short span of time. Also analysed how the RASP method provides confidentiality of data and increases the working process of query.

An Approach for Query Optimization by using Schema Object Base View

Mining of Data is the extraction of hidden prognosticative information from large databases or set of data, is a strong new technology with great prospective to help companies focus on the most important information in their data base. Query optimization is a purpose of many relational database management systems. The query optimizer experiments to dictate the most efficient way to implement a given query by examining the possible query plans. There are different techniques is given for optimizing query using schema based and materialized views in data base namelyQuery Graph, Tableaus, Optimization of Queries having Aggregates. In this paper we are using Different query optimization parameter and create an effective approach by using this approach we are reduce query execution cost, query space and more effectible for the query. The complexity of Queries severely increase the execution cost of the queries and have a critical effect on performance and productivity of decision support systems. It is required to perform expensive join and aggregation operations frequently on the databases. Now if they are not pre calculated in advanced then it leads to reduce query performance. Schema object improve query performance by pre calculating expensive join and aggregation operations on the database prior to execution and storing the results in the database. Schema object define not only relationships, but also allow you to recompute expensive joins and aggregations which lead to optimized query performance in possible ways. Schema object leads to the decrease Query processing cost and Query Maintenance cost in terms of Time factor. Schema object improve query performance by pre calculating expensive join and aggregation operations on the database prior to execution and storing the results in the database. The big advantage of a Schema object based views is extremely fast retrieval of aggregate data, since it is precomputed and stored, at the expense of insert/update/delete so that it increase query performance than the ordinary view and table. Schema object based view is also called Materialized view.

Indexing Earth Mover’s Distance over Network Metrics

The Earth Mover’s Distance (EMD) is a well-known distance metric for data represented as probability distributions over a predefined feature space. Supporting EMD-based similarity search has attracted intensive research effort. Despite the plethora of literature, most existing solutions are optimized for $L^p$ feature spaces (e.g., Euclidean space); while in a spectrum of applications, the relationships between features are better captured using networks. In this paper, we study the problem of answering $k$ -nearest neighbor ( $k$ -NN) queries under network-based EMD metrics (NEMD). We propose Oasis , a new access method which leverages the network structure of feature space and enables efficient NEMD-based similarity search. Specifically, Oasis employs three novel techniques: (i) Range Oracle , a scalable model to estimate the range of $k$ -th nearest neighbor under NEMD, (ii) Boundary Index , a structure that efficiently fetches candidates within given range, and (iii) Network Compression Hierarchy , an incremental filtering mechanism that effectively prunes false positive candidates to save unnecessary computation. Through extensive experiments using both synthetic and real data sets, we confirmed that Oasis significantly outperforms the state-of-the-art methods in query processing cost.

Design and Implementation of Spatial Operators and Energy-Efficient Query Processing Strategy in Wireless Sensor Network Database System

Database applications in wireless sensor networks very often demand data collection from sensor nodes of specific target regions. Design and development of spatial query expressions and energy-efficient query processing strategy are important issues for sensor network database systems. The existing sensor network database systems lack the needed sophistication for the space calculation of the target sensor nodes; hence, unnecessary query/data transmissions are required between the sensor nodes and the server. This paper describes our spatial operations and energy-efficient query processing methods that are designed and implemented in our sensor network database system called [Formula: see text]. With a set of spatial operators based on geometric parameters, such as Envelope, NearBy, Distance, Direction, and set theoretic operators, [Formula: see text] allows sensor network applications to easily specify the target space of interest. Our energy-efficient query processing strategy implements an in-network query management based on the lowest common ancestor (LCA) algorithm, so that the query processing cost for calculating the target spaces is greatly reduced by avoiding the need of heavy query/data transmissions between the base-station and target nodes. Performance evaluation shows that our proposed design and implementation of spatial query expressions and processing strategy achieve improved energy efficiency for database operations in the wireless sensor network.

Hilbert curve-based cryptographic transformation scheme for spatial query processing on outsourced private data

Research on preserving location data privacy in outsourced databases has been spotlighted with the development of cloud computing. However, the existing spatial transformation schemes are vulnerable to various attack models. The existing cryptographic transformation scheme provides good data privacy, but it has a high query processing cost. To improve privacy and reduce cost, we propose a Hilbert curve-based cryptographic transformation scheme to preserve the privacy of the spatial data from various attacks on outsourced databases. We also provide efficient range and k-NN query processing algorithms using a Hilbert-order index. A performance analysis confirms that the proposed scheme is robust against attack models and achieves better query processing performance than the existing cryptographic transformation scheme.

BUILDING CONFIDENTIAL AND EFFICIENT QUERY SERVICES IN THE CLOUD WITH RASP DATA PERTURBATION

With the advent of Cloud computing technology, the cost of computation, application hosting, content storage and delivery is reduced significantly. So now-a-days at an increased speed cloud computing infrastructures are used by the people. Security and Privacy is the biggest concern about cloud computing, since data management and infrastructure management in cloud is provided by third-party, it is always a risk to handover the sensitive information to such providers until and unless data confidentiality and secure query processing is guaranteed by the cloud service provider. To fully realize the benefits of cloud computing the workload must be reduced and efficient query processing must be provided. Therefore, to provide confidential and efficient query service RASP method is proposed, where RASP denotes Random Space Perturbation. Data Perturbation technique allows users to ascertain key summary information about the data that is not distorted and does not lead to a security breach. Exclusive security features are provided by the RASP. The RASP approach satisfies the data Confidentiality, query Privacy, Efficient query processing and Low working cost (CPEL) criteria for hosting queries in the cloud. KNN-R algorithm is used here to process the Range query to the kNN query. Users have been authorized by using the randomly generated product key value provided by the admin after successful registration by the user thus maintaining confidentiality. User queries are retrieved within a very short period of time i.e., less than a second. Outsourcing the management of the data related to organizations and individuals to a service provider are enabled by cloud computing services in order to save on the hardware costs and reducing maintenance cost. Also analyzed how the RASP method will provide confidentiality of data and will increase the working process of query.

그리드 인덱스 기반 뷰 선택 기법을 이용한 효율적인 Top-k 질의처리 알고리즘

Research on top-k query processing algorithms for analyzing big data have been spotlighted recently. However, because existing top-k query processing algorithms do not provide an efficient index structure, they incur high query processing costs and cannot support various types of queries. To solve these problems, we propose a top-k query processing algorithm using a view selection method based on a grid index. The proposed algorithm reduces the query processing time by retrieving the minimum number of grid cells for the query range, by using a grid index-based view selection method. Finally, we show from our performance analysis that the proposed scheme outperforms an existing scheme, in terms of both query processing time and query result accuracy.

A Locality Sensitive Hashing Filter for Encrypted Vector Databases

We introduce a filtering methodology based on locality-sensitive hashing LSH and whitening transformation to reduce candidate tuples between which encrypted vector databases EVDBs must compute similarity for query processing. The LSH hashing methodology is efficient for estimating similarities between two vectors. It hashes a vector space using randomly chosen vectors. We can filter vectors that are less similar to the querying vectors by recording which hashed space each vector belongs to. However, if vectors in EVDBs are found locally, then most vectors are in the same hashed space, so the filter will not work. Because we can treat those cases using whitening transformation to distribute the vectors broadly, our proposed filtering methodology will work effectively on any vector space. We also show that our filter reduces the server's query processing cost.

Distributed query plan generation using multi-objective particle swarm optimisation

Processing of distributed queries in an efficient manner entails generating query plans that would minimise the total query processing cost. The number of possible query plans increases exponentially with an increase in the number of relations accessed by the distributed query, as well as the number of sites where these relations reside. Consequently, it becomes infeasible to explore all possible query plans. In this paper, this problem has been addressed as a bi-objective optimisation problem with the two objectives being the minimisation of the number of sites involved in processing the distributed query and the maximisation of the concentration of relations in these involved sites. This problem has been solved using the set-based comprehensive learning parallel particle swarm optimisation (S-CLPPSO). Experimental results show that the S-CLPPSO-based distributed query plan generation algorithm is able to generate Top-K query plans that would result in efficient processing of a distributed query.

IJARCCE - Computer and Communication Engineering

In this paper, we describe the effective design of materialized view selection and preservation in a data warehousing system. This design implementation aims to aid users in retrieving data effectively for business analysis. The skeleton design of this data warehousing system employs the dimensional modeling concepts of snowflake as well as star schemes. Here, some of frequently accessed queries are stored in various user files on which we apply materialized view selection process to create materialized views in order to minimize the query processing cost. A cost analysis model was developed to enable the estimate the total cost and benefit involved in selecting each materialized view. For effective materialized views selection and preservation methodology, WMVS and PMV algorithms has been implemented. This algorithm takes into account an effective cost variables associated with the materialized views Selection and preservation method which includes query access frequencies, materialized view access frequencies, query processing costs, materialized view access cost ,query storage cost , materialized view storage cost and the availability of the system's storage. The algorithm has been applied to dummy tables containing student information to create cost effective set of materialized summary views, , thereby resulting in an efficient data warehousing system where storage and query processing of the system is optimized.

IJARCCE - Computer and Communication Engineering

Database is the collection of files or table and Database Management System is used to manage the overall activities of the database. We have two approaches for storing and managing database namely Centralised Database management system and Distributed Database management system. Query Processing and its optimization is one of the major key areas of Distributed Database Systems. Query optimization is much more difficult in Distributed Database as compared to the Centralised Database System. This literature review paper is based in the query optimization in Distributed Database System. It mainly focused on to analyse the performance and behaviour of the use of semi-joins and joins. Various factors are considered like CPU cost, I/O cost, Total Cost, Response time and Total time etc. Minimizing the amount of data transmission is important to reduce the query processing cost. To minimize this communication cost, the exhaustive enumerative technique has used to bring dynamism in determining the sub operations (join or semi join) to various sites.

Research on Materialized Views under Disk-Space Constraint in Data Warehouse

Materialized view is an important topic in data warehouse research, and also affects the query efficiency and maintenance cost. The disk-space view-selection problem is to select a set of materialized views for the purpose of minimizing the total query processing cost and the total maintenance cost. In this paper we introduce evolutionary algorithm using stochastic ranking algorithm, which can enable materialized view selection under disk-space constraint. The algorithm improve the stochastic ranking algorithm, which can find a near-optimal feasible solution. This paper use the algorithm in Police data warehouse.

An ant-based approach to cluster peers in P2P database systems

Peer-to-Peer (P2P) systems with distinct advantages such as scalability, robustness and low cost have developed quickly in recent years. The resource search problem, that is to find out a peer having a desired resource, is a fundamental problem to be initially solved in most P2P applications. P2P systems can also be categorized under database systems. One of the most important problems of P2P database systems is the query processing algorithms. Even a rather small improvement over these algorithms can have a significant impact on the performance of a P2P database system. One of the most effective methods to reduce query processing cost is peer clustering. Hence, clustering peers with similar contents makes the queries directed to the appropriate cluster and thus to be efficiently answered. In this paper, we present a new peer clustering algorithm for P2P database systems that uses ant-based clustering algorithms. We have adjusted ant-based clustering algorithm to peer clustering problem. Performance evaluation of the algorithm shows its strength in clustering peers in P2P database systems to be of a higher quality and accuracy compared to the existing systems, to have a relatively short running time and to quickly converge to an optimum. Moreover, our algorithm achieves a high scalability to the increasing problem size.

Dynamic and fast processing of queries on large-scale RDF data

As RDF data continue to gain popularity, we witness the fast growing trend of RDF datasets in both the number of RDF repositories and the size of RDF datasets. Many known RDF datasets contain billions of RDF triples (subject, predicate and object). One of the grant challenges for managing these huge RDF data is how to execute RDF queries efficiently. In this paper, we address the query processing problems against the billion triple challenges. We first identify some causes for the problems of existing query optimization schemes, such as large intermediate results, initial query cost estimation errors. Then, we present our block-oriented dynamic query plan generation approach powered with pipelining execution. Our approach consists of two phases. In the first phase, a near-optimal execution plan for queries is chosen by identifying the processing blocks of queries. We group the join patterns sharing a join variable into building blocks of the query plan since executing them first provides opportunities to reduce the size of intermediate results generated. In the second phase, we further optimize the initial pipelining for a given query plan. We employ optimization techniques, such as sideways information passing and semi-join, to further reduce the size of intermediate results, improve the query processing cost estimation and speed up the performance of query execution. Experimental results on several RDF datasets of over a billion triples demonstrate that our approach outperforms existing RDF query engines that rely on dynamic programming based static query processing strategies.

Distributed query plan generation using multiobjective genetic algorithm.

A distributed query processing strategy, which is a key performance determinant in accessing distributed databases, aims to minimize the total query processing cost. One way to achieve this is by generating efficient distributed query plans that involve fewer sites for processing a query. In the case of distributed relational databases, the number of possible query plans increases exponentially with respect to the number of relations accessed by the query and the number of sites where these relations reside. Consequently, computing optimal distributed query plans becomes a complex problem. This distributed query plan generation (DQPG) problem has already been addressed using single objective genetic algorithm, where the objective is to minimize the total query processing cost comprising the local processing cost (LPC) and the site-to-site communication cost (CC). In this paper, this DQPG problem is formulated and solved as a biobjective optimization problem with the two objectives being minimize total LPC and minimize total CC. These objectives are simultaneously optimized using a multiobjective genetic algorithm NSGA-II. Experimental comparison of the proposed NSGA-II based DQPG algorithm with the single objective genetic algorithm shows that the former performs comparatively better and converges quickly towards optimal solutions for an observed crossover and mutation probability.

Efficient processing of label-constraint reachability queries in large graphs

In this paper, we study a variant of reachability queries, called label-constraint reachability (LCR) queries. Specifically, given a label set S and two vertices u1 and u2 in a large directed graph G, we check the existence of a directed path from u1 to u2, where edge labels along the path are a subset of S. We propose the path-label transitive closure method to answer LCR queries. Specifically, we t4ransform an edge-labeled directed graph into an augmented DAG by replacing the maximal strongly connected components as bipartite graphs. We also propose a Dijkstra-like algorithm to compute path-label transitive closure by re-defining the “distance” of a path. Comparing with the existing solutions, we prove that our method is optimal in terms of the search space. Furthermore, we propose a simple yet effective partition-based framework (local path-label transitive closure+online traversal) to answer LCR queries in large graphs. We prove that finding the optimal graph partition to minimize query processing cost is a NP-hard problem. Therefore, we propose a sampling-based solution to find the sub-optimal partition. Moreover, we address the index maintenance issues to answer LCR queries over the dynamic graphs. Extensive experiments confirm the superiority of our method.

Minimizing roundtrip response time in distributed databases with vertical fragmentation

The main purpose of this paper is to show the advantage of using a model proposed by us, which minimizes roundtrip response time versus traditional models that minimize query transmission and processing costs for the design of a distributed database with vertical fragmentation. To this end, an experiment was conducted to compare the roundtrip response time of the optimal solution obtained using our model versus the roundtrip response time of the optimal solution obtained using a traditional model. The experimental results show that for most cases the optimal solution from a traditional model yields a response time which is larger than the response time of the optimal solution obtained from our model, and sometimes it can be thrice as large.

Bayesian networks for supporting query processing over incomplete autonomous databases

As the information available to naive users through autonomous data sources continues to increase, mediators become important to ensure that the wealth of information available is tapped effectively. A key challenge that these information mediators need to handle is the varying levels of incompleteness in the underlying databases in terms of missing attribute values. Existing approaches such as QPIAD aim to mine and use Approximate Functional Dependencies (AFDs) to predict and retrieve relevant incomplete tuples. These approaches make independence assumptions about missing values--which critically hobbles their performance when there are tuples containing missing values for multiple correlated attributes. In this paper, we present a principled probabilistic alternative that views an incomplete tuple as defining a distribution over the complete tuples that it stands for. We learn this distribution in terms of Bayesian networks. Our approach involves mining/"learning" Bayesian networks from a sample of the database, and using it to do both imputation (predict a missing value) and query rewriting (retrieve relevant results with incompleteness on the query-constrained attributes, when the data sources are autonomous). We present empirical studies to demonstrate that (i) at higher levels of incompleteness, when multiple attribute values are missing, Bayesian networks do provide a significantly higher classification accuracy and (ii) the relevant possible answers retrieved by the queries reformulated using Bayesian networks provide higher precision and recall than AFDs while keeping query processing costs manageable.