Query Processing In Distributed Databases Research Articles

Comparative Studies on Intelligent Swarming Network (iSWAN) Geno-Generative Algorithm and Top-K Query Processing Algorithm

This paper proposed an enhanced Top-k query processing in a real time distributed database system. The system employs a Particle Swarm Optimizer (PSO) based Geno-Generative iSWAN Model technique that enhances and allows multi-task concurrent query processing in a real time co-simulation data acquisition platform and as part of refinement to an existing Top-k query processing Technique. In this paper, the proposed system is compared for efficiency with the Top-K Query Algorithm, which is emerging as an alternative to more conventional technique for real time query processing in distributed databases. Dynamic simulations were performed with a real time small testbed comprising of physical and non-physical devices to test and evaluate the performance and efficiency of the two systems. Considering the estimated and expected temperatures, the result of simulation study proves that the Intelligent Swarming Network (iSWAN) Geno-Generative Model is more preferred over Top-K Query Algorithm due to its 70% accuracy over the Top-K Model, which reported a lower accuracy level of 40%.

A Gitbash-Generative Model for an Enhanced Query Processing In a Distributed Database System

This research study is directly focused on an enhanced query processing system using GitBash- Deep Generative Model, it is use to improve the performance of big data. Due to Poor query execution plan and absence of an improved aggregate query method which impose threats to qualities of good query processing in distributed databases which also cause memory pressure in a database, inflated central processing unit (CPU) and an overall reduction in concurrency. Also the volume of data use in any organization is progressively increasing in it seize, thereby resulting or demand a large storage location or space system. The availability of intended software that will confront the problem of storage contents and execution plan for query processing is a vital challenge facing big data in a processing system, which can prompt to information loss and missing record of vital document in an enterprises. The study will boost solution and addresses the problem of latencies, inability to transform and store queries, and poor query processing plans for future improvement of distributed databases, using an unsupervised learning approach such as deep generative model. The adopted methodology is dynamic system development method (DSDM). php programming language and mongoDB database is adopted for implementation. The datasets used for the proposed system was generated from e-library server/data.json (query) and is inserted into MongoDB database json format. The datasets are all trained and recognized by the Deep Generative algorithm through the Git Bash server. The proposed system result shows highly increase in the efficiency of management of big data and improved query processing. General Term Query Processing System Keywords Big Data, Deep Generative Model, Query Processing System, Information Management, Gitbash.

Distributed Query Plan Generation using Cuckoo Search Algorithm

Query processing in distributed databases involves data transmission amongst sites capable of providing answers to a distributed query. For this, a distributed query processing strategy, which generates efficient query processing plans for a given distributed query, needs to be devised. Since in distributed databases, the data is fragmented and replicated at multiple sites, the number of query plans increases exponentially with increase in the number of sites capable of providing answers to a distributed query. As a result, generating efficient query processing plans, from amongst all possible query plans, becomes a complex problem. This distributed query plan generation (DQPG) problem has been addressed using the Cuckoo Search Algorithm (CSA) in this paper. Accordingly, a CSA based DQPG algorithm (DQPGCSA) that aims to generate Top-K query plans having minimum cost of processing a distributed query has been proposed. Experimental based comparison of DQPGCSA with the existing GA based DQPG algorithm shows that the former is able to generate Top-K query plans that have a comparatively lower query processing cost. This, in turn, reduces the query response time resulting in efficient decision making.

Skyline-join query processing in distributed databases

The skyline-join operator, as an important variant of skylines, plays an important role in multi-criteria decision making problems. However, as the data scale increases, previous methods of skyline-join queries cannot be applied to new applications. Therefore, in this paper, it is the first attempt to propose a scalable method to process skyline-join queries in distributed databases. First, a tailored distributed framework is presented to facilitate the computation of skyline-join queries. Second, the distributed skyline-join query algorithm (DSJQ) is designed to process skyline-join queries. DSJQ contains two phases. In the first phase, two filtering strategies are used to filter out unpromising tuples from the original tables. The remaining tuples are transmitted to the corresponding data nodes according a partition function, which can guarantee that the tuples with the same join value are transferred to the same node. In the second phase, we design a scheduling plan based on rotations to calculate the final skyline-join result. The scheduling plan can ensure that calculations are equally assigned to all the data nodes, and the calculations on each data node can be processed in parallel without creating a bottleneck node. Finally, the effectiveness of DSJQ is evaluated through a series of experiments.

Beyond the mean in fork-join queues: Efficient approximation for response-time tails

Fork-join queues are natural models for various computer and communications systems that involve parallel multitasking and the splitting and resynchronizing of data, such as parallel computing, query processing in distributed databases, and parallel disk access. Job response time in a fork-join queue is a critical performance indicator but its exact analysis is challenging. We introduce a stochastic model for K-node homogeneous fork-join queues (K≥2) that focuses on the difference in length between any node-queue and the shortest one, truncating the state space such that the maximum difference is at most a constant C. Whilst most previous methods focus on the mean response time, our model is also able to evaluate the response time distribution, as well as accommodating phase-type processing times and Markovian arrival processes. In order to tackle scenarios with high loads, which require a large value of C to provide sufficient accuracy, we develop an efficient algorithm using matrix-analytic methods. Tests against simulation show that the proposed model yields accurate results for 2-node fork-join queues. As the model becomes numerically intractable for large values of K, we further propose an approximate approach, based on properties of order statistics and extreme values. The approximation gives a high degree of accuracy on response time tails, and has the advantage of being efficient and scalable, requiring only the analytical results for a single-node and 2-node fork-join queues, which we obtain with the aforementioned matrix-analytic model. Comparison with simulation results shows that our approximation yields good fits for the tails, even in very large cases with general processing and inter-arrival times.

Static vs Dynamic Techniques for Selectivity Evaluation in Distributed Query Optimization

In distributed database query optimization, query optimizers have traditionally relied upon statically estimated table cardinalities when evaluating the cost of the query plans. This paper analyses static vs. dynamic calculation for selectivity of intermediate relations generated in query processing. The objective of this research is to overcome the disadvantages of previously formulated static methods which are relatively inaccurate in a distributed database environment. A Dynamic selectivity evaluation tool (DSET) has been proposed to optimize cost for a distributed database query processing environment. The results have shown that dynamic evaluation of selectivity factor of sub query operation is feasible and can significantly reduced the total query cost than its static estimation.

Research on Query Optimization Technology in Distributed Database

Through the in-depth study of the existing distributed database query processing technology, this paper proposes a distributed database query processing program. This program optimizes the existing query processing, stores the commonly used query results according to the query frequency, to be directly used by the subsequent queries or used as intermediate query results, thus avoiding possible transmission of a large number of data, thereby reducing the query time and improving query efficiency.

Distributed Queries in the E-learning Environment

This paper presents the advantages of e-Learning education and focus on query processing in distributed databases. For this, we present the factors that influence distributed query optimization and examine the optimization strategies of query execution. The goal of this paper is to present strategies of optimizing query execution in e-learning systems used mainly by universities with geographically distributed locations, in order to obtain a system response time as low as possible and to minimize the total cost of implementation.

Distributed stream join query processing with semijoins

This paper addresses the distributed stream processing of window-based multi-way join queries considering the semijoin as a key join operator. In distributed stream processing, data streams arriving at remote sites need to be shipped to the processing site for query execution. This typically introduces high communication overhead. Our observation is that semijoin, effective in reducing communication overhead in distributed database query processing, can be also effective in distributed stream query processing. The challenge, however, lies in the streaming nature of the tuples, as it requires continuous and incremental processing of an unbounded sequence of tuples instead of one-time processing of a set of stored tuples. This paper describes our comprehensive work done to address the challenge. Specifically, we first propose a distributed stream join processing model that handles the issue of network delays introduced from the shipment of data streams, and allows for efficient batch processing. Then, based on the model, we propose join algorithms in a multi-way join case: first, one-way join algorithms for different combinations of join placement and join method and, then, multi-way join algorithms assuming linear join ordering. Regarding the join method, two distributed join methods are introduced: (1) simple join, in which full tuples are forwarded to the query processing site and (2) semijoin-based join, in which partial tuples are forwarded. A semijoin-based join can be executed with different possible semijoin strategies which incur different communication overheads. We present a complete set of join algorithms considering all possible semijoin strategies, and propose an optimization algorithm. The join algorithms are executed continuously in an incremental manner as tuples arrive, and never ship tuples redundantly. The optimization algorithm constructs an efficient multi-way join plan by using a greedy heuristic which adds to the plan one stream with the minimum join execution cost in each step. Through extensive experiments, we conduct comparative studies of the performance among the proposed one-way join algorithms and the efficiency of the generated plan between the optimization algorithm based on the greedy heuristic and the exhaustive search, respectively.

Coloured Petri net‐based modeling for distributed relational database queries

In this paper, a new application of colour timed Petri net (CTPN) based methodology for distributed relational database queries modeling and corresponding simulation is addressed. This work, first based on parsed query definitions, converts logical query plans into designed CTPN models. Later, the improved cost‐based functions are then established and appended to CTPN with a general purpose CPN simulator‐Design/CPN to observe the thresholds of query operations. The propose CTPN can be used in the design phase as an experimental prototype to automatically simulate distributed database query processing which, in turn, may considerably reduce the load of developing the actual query processing software in the logical design phase of a distributed database system. Also, since all the essential details of query processing in CTPN have been simulated, the results of this study can be closely related to real world applications.

Query processing in distributed databases with nondisjoint data

In a distributed relational database, relations are divided into disjoint fragments. These fragments are allocated to different sites in the database using some allocation scheme to improve the data retrieval time. Allocation schemes that are not constrained by other features of the DBMS will make the allocation easier to implement and provide desired performance (retrieval time). However, at present, the allocation schemes are constrained by the assumptions made by the existing query processing schemes. Most of the existing query processing schemes assume a restricted form of fragment allocation. Some of these assume that a fragment is allocated to only one site, while others assume that the sets of fragments allocated to two different sites are either disjoint or the same. This paper emphasizes the importance of nondisjoint data among sites in a distributed database environment and presents query processing framework for such an allocation. Using the framework, a number of query processing schemes can be implemented. This paper also presents a heuristic query processing scheme using this framework. The heuristics presented here attempts to use the redundant data to eliminate the expensive join, I/O, and communication cost.