Query Processing Technology Research Articles

Location-based skyline query processing technology in road networks

Location-based skyline query processing technology in road networks

Application of processing technology based on skyline query in computer network

With the rapid development of network data communication technology, the complexity of the network environment makes the data in the data stream have uncertain characteristics, and a large number of data streams are generated in many fields. Therefore, it is necessary to find an efficient and accurate processing method to process a large amount of data in a computer network. This paper takes the skyline probability calculation and the actual efficiency of the unknown object index structure in the skyline query processing technology on the uncertain data stream as the research object. Based on the skyline query technology, we study an efficient skyline query processing method on the unknown network traffic based on the model SUMG. The method includes two algorithms: dynamic modeling algorithm DMG and skyline query algorithm GST. The DMG algorithm samples the data in the sliding window of the uncertain data stream and establishes a model to convert the data stream into the parameter stream in the uncertain object probability density function, the GST algorithm establishes the R-tree index structure, which is in order to reduce the amount of calculation, it uses the parameter flow of the model. In both methods, the set of local skyline results is first obtained at the distributed node, and the skyline query is performed again on the union of the local skyline results to obtain the global skyline result set. The experimental results show that compared with the skyline query method BNL for unknown network traffic without an index structure, the SUMG method can not only effectively model the link-type unknown object to assist the skyline query, but also effectively prune the uncertain data object and improve the skyline. The distributed skyline query method can cope with the skyline query task on the distributed data stream.

Research on Practical Wireless Sensor Network Query Processing Technology

WSN(Wireless Sensor Network) is intended for obtaining sensory data so that the query processing technology of high efficiency is the precondition of widely used WSN. Existing query processing technology assumes that the WSN deployed in two-dimensional or three-dimensional free space are not suitable for those deployed in such limited circumstances as roads, pipelines and indoor environments. This paper explores the problems of existing query processing technology, does research on perception data query processing method and technology, and seeks out the existing problem and its solution by means of using the WSN actually deployed in a building as an experimental platform.

Research on OLAP query processing technology for asymmetric in-memory computing platform

Research on OLAP query processing technology for asymmetric in-memory computing platform

A Three-phase Large Scale Skyline Service Selection Framework in Clouds

For the large scale services with high-dimensional QoS attributes and distributed environment, traditional service selection approaches are faced with unprecedented challenges in terms of efficiency and performance of QoS. To address these challenges, we propose a three-phase large scale Skyline service selection framework for service composition in clouds. This framework adopts distributed parallel Skyline computation with MapReduce to prune redundant candidate services, and employs parallel multiobjective optimization algorithm based on MapReduce to select Skyline services from the tremendous amount of Skyline services warehouse for composing single service into a set of more powerful Skyline composite services, then applies Top-k query processing technology or multiple attribute decision making support method to select k Skyline composite services from the set of Skyline composite services. Through theoretical analysis, the framework can efficiently solve the service selection problem with large scale services, high-dimensional QoS in cloud computing environment, and quickly generate better composite services with the global optimal QoS.

Research on Big Data Query Online Analysis and Processing Technology on the Basis of Decision Tree Model

Research on Big Data Query Online Analysis and Processing Technology on the Basis of Decision Tree Model

Dynamic Guaranteed Cost Compression for Time Series Big Data

Query processing technology has recently received a lot of attention in the business intelligence and information service communities. However, the existing approaches can not efficiently optimize the query performance in the uncertain big data environment. In this paper, we propose QPPUBG, a novel and efficient query processing platform for uncertain big data. QPPUBG mainly includes four modules: (i) query equivalence reconstructing for uncertain big data; (ii) multiple query optimization over probability relation components; (iii) query execution plan constructing over probability relation components, and (iv) physical implementation solution of query for uncertain big data. Specially, QPPUBG can support the possible world instance semantics and efficiently handle arbitrary decision spaces. Moreover, QPPUBG can seamlessly integrate the above four modules into the modern parallel computation frameworks. We present the extensive experiments that demonstrate QPPUBG is both efficient and effective.

DB2 with BLU acceleration

DB2 with BLU Acceleration deeply integrates innovative new techniques for defining and processing column-organized tables that speed read-mostly Business Intelligence queries by 10 to 50 times and improve compression by 3 to 10 times, compared to traditional row-organized tables, without the complexity of defining indexes or materialized views on those tables. But DB2 BLU is much more than just a column store. Exploiting frequency-based dictionary compression and main-memory query processing technology from the Blink project at IBM Research - Almaden, DB2 BLU performs most SQL operations - predicate application (even range predicates and IN-lists), joins, and grouping - on the compressed values, which can be packed bit-aligned so densely that multiple values fit in a register and can be processed simultaneously via SIMD (single-instruction, multipledata) instructions. Designed and built from the ground up to exploit modern multi-core processors, DB2 BLU's hardware-conscious algorithms are carefully engineered to maximize parallelism by using novel data structures that need little latching, and to minimize data-cache and instruction-cache misses. Though DB2 BLU is optimized for in-memory processing, database size is not limited by the size of main memory. Fine-grained synopses, late materialization, and a new probabilistic buffer pool protocol for scans minimize disk I/Os, while aggressive prefetching reduces I/O stalls. Full integration with DB2 ensures that DB2 with BLU Acceleration benefits from the full functionality and robust utilities of a mature product, while still enjoying order-of-magnitude performance gains from revolutionary technology without even having to change the SQL, and can mix column-organized and row-organized tables in the same tablespace and even within the same query.

A New Approach of Distributed Query Processing Based on Database Grid

Distributed query processing technology is a difficult problem of data query, at the moment, the research and result of the query processing technology which is in view of grid environment is seldom. On the basis of the current grid standard and technology and aiming at the research of distributed relation join algorithm which adapts to grid environment, the paper designs the relevant solutions, achieves the efficient distributed query processing. For the characteristics of database grid, come up with the algorithm of relation join operation in database grid-symmetrical same-ordered semi-join algorithm, this algorithm can reduces the cost of communication and improves the parallelism, and is suitable for multi-join; Second, for the algorithm characteristics, researching the relevant SQL query processing method in order to generates efficient execution plan of symmetrical same-ordered semi-join algorithm.

Research on Query Optimization Technology in Distributed Database

Through the in-depth study of the existing distributed database query processing technology, this paper proposes a distributed database query processing program. This program optimizes the existing query processing, stores the commonly used query results according to the query frequency, to be directly used by the subsequent queries or used as intermediate query results, thus avoiding possible transmission of a large number of data, thereby reducing the query time and improving query efficiency.

The design and implementation of OGSA-DQP: A service-based distributed query processor

Service-based approaches are rising to prominence because of their potential to meet the requirements for distributed application development in e-business and e-science. The emergence of a service-oriented view of hardware and software resources raises the question as to how database management systems and technologies can best be deployed or adapted for use in such an environment. This paper explores one aspect of service-based computing and data management, viz., how to integrate query processing technology with a service-based architecture suitable for a Grid environment. The paper addresses this by describing in detail the design and implementation of a service-based distributed query processor. The query processor is service-based in two orthogonal senses: firstly, it supports querying over data storage and analysis resources that are made available as services, and, secondly, its internal architecture factors out as services the functionalities related to the construction and execution of distributed query plans. The resulting system both provides a declarative approach to service orchestration, and demonstrates how query processing can benefit from a service-based architecture. As well as describing and motivating the architecture used, the paper also describes usage scenarios, and, using a bioinformatics application, presents performance results that benchmark the system and illustrate the benefits provided by the service-based architecture.

Tight lower bounds for query processing on streaming and external memory data

It is generally assumed that databases have to reside in external, inexpensive storage because of their sheer size. Current technology for external storage systems presents us with a reality that, performance-wise, a small number of sequential scans of the data is strictly preferable over random data accesses. Database technology–in particular query processing technology–has developed around a notion of memory hierarchies with layers of greatly varying sizes and access times. It seems that the current technologies scale up to their tasks and are very successful, but on closer investigation it may appear that our theoretical understanding of the problems involved–and of optimal algorithms for these problems–is not quite as developed. Recently, data stream processing has become an object of study by the database management community, but from the viewpoint of database theory, this is really a special case of the query processing problem on data in external storage where we are limited to a single scan of the input data. In the present paper we study a clean machine model for external memory and stream processing. We establish tight bounds for the data complexity of Core XPath evaluation and filtering. We show that the number of scans of the external data induces a strict hierarchy (as long as internal memory space is sufficiently small, e.g., polylogarithmic in the size of the input). We also show that neither joins nor sorting are feasible if the product of the number r ( n ) of scans of the external memory and the size s ( n ) of the internal memory buffers is sufficiently small, i.e., of size o ( n ) .

Accelerating XPath evaluation in any RDBMS

This article is a proposal for a database index structure, the XPath accelerator , that has been specifically designed to support the evaluation of XPath path expressions. As such, the index is capable to support all XPath axes (including ancestor, following, preceding-sibling, descendant-or-self, etc.). This feature lets the index stand out among related work on XML indexing structures which had a focus on the child and descendant axes only. The index has been designed with a close eye on the XPath semantics as well as the desire to engineer its internals so that it can be supported well by existing relational database query processing technology: the index (a) permits set-oriented (or, rather, sequence-oriented) path evaluation, and (b) can be implemented and queried using well-established relational index structures, notably B-trees and R-trees.We discuss the implementation of the XPath accelerator on top of different database backends and show that the index performs well on all levels of the memory hierarchy, including disk-based and main-memory based database systems.