Inter-query Parallelism Research Articles

Achieving new SQL query performance levels through parallel execution in SQL Server

This article provides an in-depth look at implementing parallel SQL query processing using the Microsoft SQL Server database management system. It examines how parallelism can significantly accelerate query execution by leveraging multi-core processors and clustered environments. The article explores SQL Server's sophisticated parallel processing capabilities including automatic query parallelization, intra-query parallelism techniques like parallel joins and parallel data aggregation, as well as inter-query parallelism for concurrent query execution. It covers key considerations around effective parallelization such as managing concurrency and locks, handling data skew, resource governance, and monitoring. Challenges like debugging parallel plans and potential bottlenecks from excessive parallelism are also discussed along with mitigation strategies. Real-world examples demonstrate how judicious application of parallel processing helps optimize complex analytics workloads involving massive datasets. The insights presented provide guidance to database developers and administrators looking to enable parallel SQL query execution in SQL Server environments for substantial performance gains and scalability.

Partitioned signature files: design issues and performance evaluation

A signature file acts as a filtering mechanism to reduce the amount of text that needs to be searched for a query. Unfortunately, the signature file itself must be exhaustively searched, resulting in degraded performance for a large file size. We propose to use a deterministic algorithm to divide a signature file into partitions, each of which contains signatures with the same “key.” The signature keys in a partition can be extracted and represented as the partition's key. The search can then be confined to the subset of partitions whose keys match the query key. Our main concern here is to study methods for obtaining the keys and their performance in terms of their ability to reduce the search space. Owing to the reduction of search space, partitioning a signature file has a direct benefit in a sequential search (single-processor) environment. In a parallel environment, search can be conducted in parallel effectively by allocating one or more partitions to a processor. Partitioning the signature tile with a deterministic method (as opposed to a random partitioning scheme) provides intraquery parallelism as well as interquery parallelism. In this paper, we outline the criteria for evaluating partitioning schemes. Three algorithms are described and studied. An analytical study of the performance of the algorithms is provided and the results are verified with simulation.