Query Operations Research Articles

Distributed Maximal Clique Computation and Management

Maximal cliques are elementary substructures in a graph and instrumental in graph analysis such as the structural analysis of many complex networks, graph clustering and community detection, network hierarchy detection, emerging pattern mining, vertex importance measures, etc. However, the number of maximal cliques is also notoriously large even for many small real world graphs. This size problem gives rise to challenges in both computing and managing the set of maximal cliques. Many algorithms for computing maximal cliques have been proposed in the literature; however, most of them are sequential algorithms that cannot scale due to the high complexity of the problem, while existing parallel algorithms for computing maximal cliques are mostly immature and especially suffer from skewed workload. As for managing the set of maximal cliques, which is essential due to its large size, there is barely any efficient method for querying or updating the set of maximal cliques. In this paper, we first propose a distributed algorithm built on a share-nothing architecture for computing the set of maximal cliques. We effectively address the problem of skewed workload distribution due to high-degree vertices, which also leads to drastically reduced worst-case time complexity for computing maximal cliques in common real-world graphs. Then, we propose a set of fundamental query operations and efficient algorithms to process the queries, to aid more efficient and effective analysis of the set of maximal cliques. Finally, we also devise algorithms to support efficient update maintenance of the set of maximal cliques when the underlying graph is updated. We verify the efficiency of our algorithms for computing, querying, and updating the set of maximal cliques with a range of real-world graphs from different application domains.

Channel-Oblivious Counting Algorithms for Large-Scale RFID Systems

Scalable, low-latency and accurate RFID counting algorithms have recently been proposed as a fundamental building block to support more complex query operations in a large-scale RFID system. One distinct feature of these algorithms is that they do not require explicit identification of individual tags and therefore can eliminate the latency bottleneck caused by serialization during multiple access control. However, these algorithms all assume reliable communications between the reader and the tags. While this assumption is also adopted by many tag-identification protocols in the current RFID standards, it is practically unachievable given the current technology and low-cost requirement of RFID tags. In fact, recent empirical studies have found that the communication between an RFID reader and a set of seemingly “in-range” tags are still unreliable and highly non-deterministic due to the varying channel conditions. In this paper, we discuss the design and performance analysis of a set of channel-oblivious RFID counting algorithms which can estimate the size of a tag-set of interest over unreliable wireless channels. The proposed schemes can provide accurate cardinality estimates without any prior knowledge of the channel parameters. We first propose a series of algorithms and analyze their performance under a simplified memoryless lossy channel model. We then extend them to handle the impact due to backscattering effects and correlated losses found in practical RFID systems. Our proposed designs only require simple modifications to standard RFID tags and readers and can be implemented using current technologies with minimal increase in tag/ reader cost. Our designs can also be extended to other RFID counting algorithms which assumed reliable communication channels.

Query-competitive algorithms for cheapest set problems under uncertainty

Considering the model of computing under uncertainty where element weights are uncertain but can be obtained at a cost by query operations, we study the problem of identifying a cheapest (minimum-weight) set among a given collection of feasible sets using a minimum number of queries of element weights. For the general case we present an algorithm that makes at most d⋅OPT+d queries, where d is the maximum cardinality of any given set and OPT is the optimal number of queries needed to identify a cheapest set. For the minimum multi-cut problem in trees with d terminal pairs, we give an algorithm that makes at most d⋅OPT+1 queries. For the problem of computing a minimum-weight base of a given matroid, we give an algorithm that makes at most 2⋅OPT queries, generalizing a known result for the minimum spanning tree problem. For each of the above algorithms we give matching lower bounds. We also settle the complexity of the verification version of the general cheapest set problem and the minimum multi-cut problem in trees under uncertainty.

3D PARTITION-BASED CLUSTERING FOR SUPPLY CHAIN DATA MANAGEMENT

Abstract. Supply Chain Management (SCM) is the management of the products and goods flow from its origin point to point of consumption. During the process of SCM, information and dataset gathered for this application is massive and complex. This is due to its several processes such as procurement, product development and commercialization, physical distribution, outsourcing and partnerships. For a practical application, SCM datasets need to be managed and maintained to serve a better service to its three main categories; distributor, customer and supplier. To manage these datasets, a structure of data constellation is used to accommodate the data into the spatial database. However, the situation in geospatial database creates few problems, for example the performance of the database deteriorate especially during the query operation. We strongly believe that a more practical hierarchical tree structure is required for efficient process of SCM. Besides that, three-dimensional approach is required for the management of SCM datasets since it involve with the multi-level location such as shop lots and residential apartments. 3D R-Tree has been increasingly used for 3D geospatial database management due to its simplicity and extendibility. However, it suffers from serious overlaps between nodes. In this paper, we proposed a partition-based clustering for the construction of a hierarchical tree structure. Several datasets are tested using the proposed method and the percentage of the overlapping nodes and volume coverage are computed and compared with the original 3D R-Tree and other practical approaches. The experiments demonstrated in this paper substantiated that the hierarchical structure of the proposed partitionbased clustering is capable of preserving minimal overlap and coverage. The query performance was tested using 300,000 points of a SCM dataset and the results are presented in this paper. This paper also discusses the outlook of the structure for future reference.

Symbolic Trajectories

Due to the proliferation of GPS-enabled devices in vehicles or with people, large amounts of position data are recorded every day and the management of such mobility data, also called trajectories, is a very active research field. A lot of effort has gone into discovering “semantics” from the raw geometric trajectories by relating them to the spatial environment or finding patterns, for example, by data mining techniques. A question is how the resulting “meaningful” trajectories can be represented or further queried. In this article, we propose a systematic study of annotated trajectory databases . We define a very simple generic model called symbolic trajectory to capture a wide range of meanings derived from a geometric trajectory. Essentially, a symbolic trajectory is just a time-dependent label; variants have sets of labels, places, or sets of places. They are modeled as abstract data types and integrated into a well-established framework of data types and operations for moving objects. Symbolic trajectories can represent, for example, the names of roads traversed obtained by map matching, transportation modes, speed profile, cells of a cellular network, behaviors of animals, cinemas within 2km distance, and so forth. Symbolic trajectories can be combined with geometric trajectories to obtain annotated trajectories. Besides the model, the main technical contribution of the article is a language for pattern matching and rewriting of symbolic trajectories. A symbolic trajectory can be represented as a sequence of pairs (called units) consisting of a time interval and a label. A pattern consists of unit patterns (specifications for time interval and/or label) and wildcards, matching units and sequences of units, respectively, and regular expressions over such elements. It may further contain variables that can be used in conditions and in rewriting. Conditions and expressions in rewriting may use arbitrary operations available for querying in the host DBMS environment, which makes the language extensible and quite powerful. We formally define the data model and syntax and semantics of the pattern language. Query operations are offered to integrate pattern matching, rewriting, and classification of symbolic trajectories into a DBMS querying environment. Implementation of the model using finite state machines is described in detail. An experimental evaluation demonstrates the efficiency of the implementation. In particular, it shows dramatic improvements in storage space and response time in a comparison of symbolic and geometric trajectories for some simple queries that can be executed on both symbolic and raw trajectories.

PatchTable

This paper presents a data structure that reduces approximate nearest neighbor query times for image patches in large datasets. Previous work in texture synthesis has demonstrated real-time synthesis from small exemplar textures. However, high performance has proved elusive for modern patch-based optimization techniques which frequently use many exemplar images in the tens of megapixels or above. Our new algorithm, PatchTable, offloads as much of the computation as possible to a pre-computation stage that takes modest time, so patch queries can be as efficient as possible. There are three key insights behind our algorithm: (1) a lookup table similar to locality sensitive hashing can be precomputed, and used to seed sufficiently good initial patch correspondences during querying, (2) missing entries in the table can be filled during pre-computation with our fast Voronoi transform, and (3) the initially seeded correspondences can be improved with a precomputed k-nearest neighbors mapping. We show experimentally that this accelerates the patch query operation by up to 9× over k-coherence, up to 12× over TreeCANN, and up to 200× over PatchMatch. Our fast algorithm allows us to explore efficient and practical imaging and computational photography applications. We show results for artistic video stylization, light field super-resolution, and multi-image editing.

Designs, analyses, and optimizations for attribute-shuffling obfuscation to protect information from malicious cloud administrators

We developed an attribute-shuffling obfuscation for database applications in cloud environment and studied its potential in preventing information leaks to malicious administrators at cloud providers, who have unlimited and non-censored accesses to any local resources, possibly including the system security logs. The proposed obfuscation allows database management systems at cloud servers to perform fundamental query operations while cloud users' information is protected against leaks to malicious administrators. We studied the performance of the proposed obfuscation to find that the inflation of obfuscated tables at cloud servers and the increase in the network traffic load will be the major overhead. We developed algorithms that mitigate the inflation of obfuscated tables using 'α' parameter and the increase in the network traffic load using the query constructor. The former achieved a linear relation between the obfuscated table size and the degree of obfuscation how hard for malicious administrators to understand the meaning of users' information. The latter achieved that the network traffic load nearly converged to that of no obfuscation for busy systems. We conclude that the proposed attribute-shuffling obfuscation will be feasible and efficient for busy and large database systems, while it adapts to database systems with diverse configurations. Copyright © 2015 John Wiley & Sons, Ltd.

NAND 플래쉬메모리에서 고정그리드화일 색인의 성능 평가

Since a NAND-flash memory is able to keep data during electricity-off and has small cost to store data per bytes, it is widely used on hand-held devices. It is necessary to use an index in order to process mass data effectively on the flash memory. However, since the flash memory requires high cost for a write operation and does not support an overwrite operation, it is possible to reduce the performance of the index when the disk b ased index is exploited. In this paper, we implement the fixed grid file index and evaluate the performance of the index on various conditions. To do this, we measure the average processing time by the ratio of query operations and update operations. We also the compare the processing times of the flash memory with those of the magnetic disk.

Succinct Dynamic Cardinal Trees

Cardinal trees (or tries of degree $$k$$k) are a fundamental data structure, in particular for many text-processing algorithms. Each node in a cardinal tree has at most $$k$$k children, each labeled with a symbol from the alphabet $$\{1,\ldots , k\}$${1,?,k}. In this paper we introduce succinct representations for dynamic cardinal trees on $$n$$n nodes, requiring $$2n + n\lg {k} + o(n\lg {k})$$2n+nlgk+o(nlgk) bits of space. These are the first dynamic cardinal tree representations that support a (fairly) complete set of operations while using almost optimal space. For $$k= \mathcal {O}(\mathrm{polylog}(n))$$k=O(polylog(n)), we show how the navigational and query operations on the tree can be supported in $$\mathcal {O}(1)$$O(1) time, while supporting insertions and deletions of leaves in $$\mathcal {O}(1)$$O(1) amortized time. For $$k= \omega ({\mathrm{polylog}(n)})$$k=?(polylog(n)) [and $$\mathcal {O}(n)$$O(n)], we show that the same set of operations can be supported in $$\mathcal {O}(\lg {k}/\lg {\lg {k}})$$O(lgk/lglgk) time (amortized in the case of insertions/deletions). We also show how to associate $$b$$b-bit satellite data to the tree nodes using $$bn+o(n)$$bn+o(n) extra bits. Finally, we show how the machinery introduced for dynamic cardinal trees can be adapted to represent dynamic binary trees using $$2n+o(n)$$2n+o(n) bits of space, so that the tree operations are supported in $$\mathcal {O}(1)$$O(1) time (amortized in the case of insert/delete operations). We support adding satellite data to the tree nodes using $$bn+o(n)$$bn+o(n) extra bits [(vs. $$bn+o(bn)$$bn+o(bn) extra bits of the fastest previous dynamic representation from the literature], while providing several trade-offs for accessing/modifying the data.

Fault-tolerant resource allocation for query processing in grid environments

In this paper, we propose a new algorithm for fault-tolerant resource allocation for query processing in grid environments. For this, we propose an initial resource allocation algorithm followed by a fault-tolerance protocol. The proposed fault-tolerance protocol is based on the passive replication of stateful operators in queries. We provide theoretical analyses of the proposed algorithms and consolidate our analyses with the simulations.

A Three-Phase Top-k Query Based Distributed Data Collection Scheme in Wireless Sensor Networks

We propose a three-phase top-[Formula: see text] query based distributed data collection scheme which is designed for clustered or multisink wireless sensor networks. The proposed scheme consists of a distributed iterative hard thresholding algorithm and a three-phase top-[Formula: see text] query algorithm. In the distributed iterative hard thresholding algorithm, the cluster heads or sink nodes reconstruct the compressed data in a distributed and cooperative manner. Meanwhile, the top-[Formula: see text] query operation in the above algorithm is realized by pruning unnecessary elements among cluster heads or sink nodes in the three-phase top-[Formula: see text] query algorithm. Simulation results show that there is no obvious difference in the performance of data reconstruction between our proposed scheme and existing compressive sensing theory based data collection schemes. However, both the number of interactions and the amount of transmitted data among cluster heads or sink nodes can be effectively reduced in the proposed scheme. The performance of the proposed scheme is analyzed in detail in this paper to support the claims.

Applied Research of Association Mining Technology in Student Achievement Analysis System

Most colleges and universities have built a database of student achievement, but only a simple query and statistical operations, while hiding behind the data in these achievements even more valuable information has not been excavated and use. To solve this problem, this paper proposes the use of data mining association mining method on student achievement dig deeper; get relevant information between different courses for school administrators in decision analysis, the teacher's lesson plans and student learning arrangements.

Hardware‐assisted estimation of entropy norm for high‐speed network traffic

The computation of the entropy of a high-speed data stream in a one-pass fashion is crucial to many network security applications. Motivated by the work of Lall et al., this study examines the design trade-off of processing speed and accuracy for estimating the entropy norm. The proposed scheme leverages the Count Sketch with constant memory access on counter update and point query operations. With a bounded relative error and a constant memory access cycle, the design can process incoming traffic with a throughput of 30 Gbit/s.

A Formal Description of XML Tree Pattern Query for XQuery Language

In order to express tree pattern query in query plan and take advantage of formal method to analyze its behavioral characteristics, this paper present a formal description of tree pattern query based on functional language and denotational semantics. This description major focuses on behavior of a tree pattern query on matching against an eXtensible Markup Language (XML) document tree. First, we introduce a formal definition for a kind of extended generalized tree pattern (GTP++). Then we present a functional tree pattern description language (XTPL) for GTP++ and give its complete denotational semantics based on a novel data structure, named WTree, which efficiently organizes this typical XML data query results and provides flexible data access method. In the end, we present the formal semantics of identifying tree pattern from path expressions. By using formal methods, the semantics of tree pattern query is consistent and analyzable. As the core operation of XML query, this formal description can provide an initial step for analyzing the correctness of XML queries, and improves the reliability and robustness of query processing methods.

Dynamic algorithms for monotonic interval scheduling problem

We investigate dynamic algorithms for the interval scheduling problem. We focus on the case when the set of intervals is monotonic. This is when no interval properly contains another interval. We provide two data structures for representing the intervals that allow efficient insertion, removal and various query operations. The first dynamic algorithm, based on the data structure called compatibility forest, runs in amortised time O(log2⁡n) for insertion and removal and O(log⁡n) for query. The second dynamic algorithm, based on the data structure called linearised tree, runs in time O(log⁡n) for insertion, removal and query. We discuss differences and similarities of these two data structures through theoretical and experimental results.

Consultas con agrupamiento basado en similitud

The fuzzy sets allow for the representation of vague concepts, where membership of an object could be gradual. The use of this theory in the data bases has been proposed. As result of that, the fuzzy relational model GEFRED has emerged, as well as the fuzzy extended Entity-Relationship model FuzzyEER and the fuzzy querying language FSQL, among others. These extensions include some forms of fuzzy attributes, such as the so-called Type 3. These attributes have values in a set of labels provided with a fuzzy similarity relation, which is an extension of equivalence relations in the fuzzy set theory. The implications of such attributes in partitioned query operators have not been previously explored. The contribution of this paper is on the syntactic and semantic definition of an extension to SQL, in order to allow for partitioned queries with Type 3 fuzzy attributes in grouping criteria. This extension increases the expressive power of the database language with respect to previous proposals

Performance Modeling in CUDA Streams - A Means for High-Throughput Data Processing.

Push-based database management system (DBMS) is a new type of data processing software that streams large volume of data to concurrent query operators. The high data rate of such systems requires large computing power provided by the query engine. In our previous work, we built a push-based DBMS named G-SDMS to harness the unrivaled computational capabilities of modern GPUs. A major design goal of G-SDMS is to support concurrent processing of heterogenous query processing operations and enable resource allocation among such operations. Understanding the performance of operations as a result of resource consumption is thus a premise in the design of G-SDMS. With NVIDIA's CUDA framework as the system implementation platform, we present our recent work on performance modeling of CUDA kernels running concurrently under a runtime mechanism named CUDA stream. Specifically, we explore the connection between performance and resource occupancy of compute-bound kernels and develop a model that can predict the performance of such kernels. Furthermore, we provide an in-depth anatomy of the CUDA stream mechanism and summarize the main kernel scheduling disciplines in it. Our models and derived scheduling disciplines are verified by extensive experiments using synthetic and real-world CUDA kernels.

Stochastic Heuristic Optimization based Multi-Query Processing in Wireless Sensor Network using Genetic Algorithm

Sensor Network is an infrastructure comprising of sensing, and computing. The communication elements in sensor network give capability to instrument, watch, and respond to events and phenomenon in a particular situation. Query processing in sensor network first transfers the query generated position to the node where the result obtained is similar to this query. Collaborative Query-Centric Framework (COSE) heterogeneous sensor networks are effectual and well-organized for processing of queries. Query processing with respect to energy efficiency attain single pipeline of query processing but fails to address the issues related to multiple pipeline of query processing. COSE major drawback is that it is unable to attain an optimal solution for multiple pipelines. Next, Pocket Driven Trajectories (PDT) algorithm monitors the query processing based on spatial layout of the selected nodes. PDT algorithm efficiently adapts to different types of data collection paths but not effective for multicast query phase. To develop a multiple query processing strategy in wireless sensor network, Stochastic Heuristic Optimization using Genetic Algorithm (SHO-GA) is introduced. The SHO- GA framework process the multiple query plans based on the closeness of nodes required to answer the user query. The multi-query processing using Genetic Algorithm (GA) takes the associations with FROM clause and other multiple query operators. The SHO-GA framework carries the probability of crossover, mutation and the pre-specified number of generations as input, and produces the top 'n' multi-query processing as output. The stochastic heuristic optimization fitness value provides the best fit chromosome to find the improvement in its query processing parameters. The stochastic heuristic for multi-query called base station optimization (BSO) eliminates the redundancy from the original set. The multi-query plans processed involve minimum processing time for answering the user query leading to efficient query processing sensor network system. Experimental evaluation is performed on factors such as average processing cost, cumulative distribution, multi-query processing time, query answer transmitted speed, query processing delay and user accessibility level. KeywordsNetwork, Genetic Algorithm, Multi-Query Processing, Plans Formation, FROM clause, Crossover, Mutation rate

Ibex

Modern data appliances face severe bandwidth bottlenecks when moving vast amounts of data from storage to the query processing nodes. A possible solution to mitigate these bottlenecks is query off-loading to an intelligent storage engine , where partial or whole queries are pushed down to the storage engine. In this paper, we present Ibex , a prototype of an intelligent storage engine that supports off-loading of complex query operators. Besides increasing performance, Ibex also reduces energy consumption, as it uses an FPGA rather than conventional CPUs to implement the off-load engine. Ibex is a hybrid engine, with dedicated hardware that evaluates SQL expressions at line-rate and a software fallback for tasks that the hardware engine cannot handle. Ibex supports GROUP BY aggregation, as well as projection - and selection - based filtering. GROUP BY aggregation has a higher impact on performance but is also a more challenging operator to implement on an FPGA.

Static vs Dynamic Techniques for Selectivity Evaluation in Distributed Query Optimization

In distributed database query optimization, query optimizers have traditionally relied upon statically estimated table cardinalities when evaluating the cost of the query plans. This paper analyses static vs. dynamic calculation for selectivity of intermediate relations generated in query processing. The objective of this research is to overcome the disadvantages of previously formulated static methods which are relatively inaccurate in a distributed database environment. A Dynamic selectivity evaluation tool (DSET) has been proposed to optimize cost for a distributed database query processing environment. The results have shown that dynamic evaluation of selectivity factor of sub query operation is feasible and can significantly reduced the total query cost than its static estimation.