Multidimensional Access Methods Research Articles

Packing R-trees with Space-filling Curves

The massive amount of data and large variety of data distributions in the big data era call for access methods that are efficient in both query processing and index management, and over both practical and worst-case workloads. To address this need, we revisit two classic multidimensional access methods—the R-tree and the space-filling curve. We propose a novel R-tree packing strategy based on space-filling curves. This strategy produces R-trees with an asymptotically optimal I/O complexity for window queries in the worst case. Experiments show that our R-trees are highly efficient in querying both real and synthetic data of different distributions. The proposed strategy is also simple to parallelize, since it relies only on sorting. We propose a parallel algorithm for R-tree bulk-loading based on the proposed packing strategy and analyze its performance under the massively parallel communication model. To handle dynamic data updates, we further propose index update algorithms that process data insertions and deletions without compromising the optimal query I/O complexity. Experimental results confirm the effectiveness and efficiency of the proposed R-tree bulk-loading and updating algorithms over large data sets.

Theoretically optimal and empirically efficient r-trees with strong parallelizability

The massive amount of data and large variety of data distributions in the big data era call for access methods that are efficient in both query processing and index bulk-loading, and over both practical and worst-case workloads. To address this need, we revisit a classic multidimensional access method - the R-tree. We propose a novel R-tree packing strategy that produces R-trees with an asymptotically optimal I/O complexity for window queries in the worst case. Our experiments show that the R-trees produced by the proposed strategy are highly efficient on real and synthetic data of different distributions. The proposed strategy is also simple to parallelize, since it relies only on sorting. We propose a parallel algorithm for R-tree bulk-loading based on the proposed packing strategy, and analyze its performance under the massively parallel communication model. Experimental results confirm the efficiency and scalability of the parallel algorithm over large data sets.

Qatris iManager: a general purpose CBIR system

Content-based image retrieval (CBIR) has drawn much interest from the research community over the past decade, as a good number of CBIR techniques, methods and systems have emerged, contributing new solutions to the issue of storing, managing and retrieving images, as database management systems do with structured data. There is undoubtedly a crucial need to characterize image content as well as subjectivity in the interpretation of this content (for which the community has coined the term "semantic gap"). In this paper the CBIR system developed by our research group, Qatris iManager, is described as a positive proposal to cope with chief issues in the field, especially the semantic gap, from a novel and original perspective. Based on color, texture and shape features, our system provides a broad range of useful operations to facilitate the storage, management, retrieval and browsing of large image collections. Local and remote image loading processes enable the population of image collections. Classification methods allow users to organize the collections according to their own interests. A multidimensional access method contributes to the efficiency in similarity searches. Parameterized similarity functions give flexibility to the search by content processes. Finally, the integrated automatic learning methods for classification and search processes teach the system about the user's information needs. The proposed system is the result of a joint effort with different research tasks. This paper extensively describes all the system functionalities, techniques, processes and algorithms implemented.

Calculations of Mapping from Two Dimensional Plane to Integer Line and the Reverse using Hilbert Curve

Location management systems require multi-dimensional access methods to allow efficient handling of spatial queries. Because there is no total ordering of locations that preserves the spatial locality between objects, it is difficult to design multi-dimensional access method in the way as traditional one-dimensional access methods. However, mapping multidimensional data into a single dimension makes it possible to answer these queries in an efficient way. Space filling curves are used to connect all the points on a plane. Hilbert curve preserves locality than any other space filling curves. Hilbert curve is used for sequencing the points in a plane and use the order for storage. This paper discusses the efficient way of calculating the mapping from two dimensional planes to integer line or real line and the vice versa. The time complexity for encoding and decoding using the proposed algorithm is less compared with existing algorithm. The memory requirement is almost constant compared to state machine approach. This algorithm can be used for storing spatial data in efficient way so that the access time becomes minimal.

Accelerating Range Query Processing on R-Tree Using Graphics Processing Units

Recently, various research efforts have been conducted to develop strategies for accelerating multi-dimensional query processing using the graphics processing units (GPUs). However, well-known multi-dimensional access methods such as the R-tree, B-tree, and their variants are hardly applicable to GPUs in practice, mainly due to the characteristics of a hierarchical index structure. More specifically, the hierarchical structure not only causes frequent transfers of small volumes of data but also provides limited opportunity to exploit the advanced data parallelism of GPUs. To address these problems, we propose an approach that uses GPUs as a buffer. The main idea is that object entries in recently visited leaf nodes are buffered in the global memory of GPUs and processed by massive parallel threads of the GPUs. Through extensive performance studies, we observed that the proposed approach achieved query performance up to five times higher than that of the original R-tree.

A Functional Measure-Based Framework for Evaluation of Multi-Dimensional Point Access Methods

Abstract Recent improvement in hardware technology has lead to producing lots of multi-dimensional data such as text, image, graphics and video in various applications. Multi-dimensional access methods have developed for supporting fast retrieval of multi-dimensional data from multi-dimensional databases. In the literature multi-dimensional access methods totally have been classified in to two classes including point access methods and spatial access methods. In this study, we focused on multi-dimensional point access methods, and due to variety of methods have developed in recent decades, we proposed a functional measure-based framework to evaluating multi-dimensional point access methods. In this framework, in order to present a comprehensive evaluation of multi-dimensional point access methods, firstly, we extended related classification of multi-dimensional point access methods in the literatures, secondly, we presented some important functional measures to evaluate multi-dimensional point access methods and we evaluated multi-dimensional point access methods based on these functional measures. We hope this study will lead to development of more efficient methods to support search process in multi-dimensional point data bases.

Adaptive similarity search in streaming time series with sliding windows

The challenge in a database of evolving time series is to provide efficient algorithms and access methods for query processing, taking into consideration the fact that the database changes continuously as new data become available. Traditional access methods that continuously update the data are considered inappropriate, due to significant update costs. In this paper, we use the IDC-Index (Incremental DFT Computation – Index), an efficient technique for similarity query processing in streaming time series. The index is based on a multidimensional access method enhanced with a deferred update policy and an incremental computation of the Discrete Fourier Transform (DFT), which is used as a feature extraction method. We focus both on range and nearest-neighbor queries, since both types are frequently used in modern applications. An important characteristic of the proposed approach is its ability to adapt to the update frequency of the data streams. By using a simple heuristic approach, we manage to keep the update frequency at a specified level to guarantee efficiency. In order to investigate the efficiency of the proposed method, experiments have been performed for range queries and k-nearest-neighbor queries on real-life data sets. The proposed method manages to reduce the number of false alarms examined, achieving high answers vs. candidates ratio. Moreover, the results have shown that the new techniques exhibit consistently better performance in comparison to previously proposed approaches.

Estimating the performance of multidimensional access methods based on nonoverlapping regions

Most cost models for the prediction of the performance of multidimensional access methods are based on Minkowski operations. However, this approach does not correspond to the space-partitioning strategy of multidimensional access methods that produce nonoverlapping index regions, for example, the KDB-tree and its variants. This article proposes a new cost model for the prediction of the selection (search) performance of the related access methods, including its adaptation for nonuniform data. The results of an extensive set of experiments conducted on both simulated and real data show that the cost model and its extension provide a basis for a highly accurate analysis of these structures in both low- and high-dimensional situations. © 2007 Wiley Periodicals, Inc. Int J Int Syst 22: 259–277, 2007.

Processing partially specified queries over high-dimensional databases

In recent years, high-dimensional database applications deal with multidimensional ad hoc queries that refer to an arbitrary number of arbitrarily (unpredictably) chosen dimensions of high-dimensional data. This paper thoroughly and systematically investigates possible secondary-storage based solutions to the problems of processing multidimensional ad hoc query in transactional or semi-transactional environments. Then a complementary solution called the indexed and transposed access method (ITAM) is proposed. This method is based on two complementary measures – a multidimensional access method and a proposed access method called the Opus-path. The performance of multidimensional access methods deteriorates rapidly as the ratio of query dimensionality to data dimensionality decreases. On the other hand, the Opus-path shows retrieval performance that is actually better when the ratio is low.

Branch-and-bound processing of ranked queries

Despite the importance of ranked queries in numerous applications involving multi-criteria decision making, they are not efficiently supported by traditional database systems. In this paper, we propose a simple yet powerful technique for processing such queries based on multi-dimensional access methods and branch-and-bound search. The advantages of the proposed methodology are: (i) it is space efficient, requiring only a single index on the given relation (storing each tuple at most once), (ii) it achieves significant (i.e., orders of magnitude) performance gains with respect to the current state-of-the-art, (iii) it can efficiently handle data updates, and (iv) it is applicable to other important variations of ranked search (including the support for non-monotone preference functions), at no extra space overhead. We confirm the superiority of the proposed methods with a detailed experimental study.

On Approximate Algorithms for Distance-Based Queries using R-trees

In modern database applications the similarity or dissimilarity of complex objects is examined by performing distance-based queries (DBQs) on data of high dimensionality. The R-tree and its variations are commonly cited multidimensional access methods that can beused for answering such queries. Although the related algorithms work well for low-dimensional data spaces, their performance degrades as the number of dimensions increases (dimensionality curse). In order to obtain acceptable response time in high-dimensional data spaces, algorithms that obtain approximate solutions can be used. Approximation techniques, like N-consider (based on the tree structure), α-allowance and e-approximate (based on distance), or Time-consider (based on time) can be applied in branch-and-bound algorithms for DBQs inorder to control the trade-off between cost and accuracy of the result. In this paper, we improve previous approximate DBQ algorithms by applying a combination of the approximation techniques in the same query algorithm (hybrid approximation scheme). We investigate the performance of these improvements for one of the most representative DBQs (the K-closest pairs query, K-CPQ) in high-dimensional data spaces, as well as the influence of the algorithmic parameters on the control of the trade-off between the response time and the accuracy of the result. The outcome of the experimental evaluation, using synthetic and real datasets, is the derivation of the outperforming DBQ approximate algorithm for large high-dimensional point datasets.

Perspectives in astrophysical databases

Astrophysics has become a domain extremely rich of scientific data. Data mining tools are needed for information extraction from such large data sets. This asks for an approach to data management emphasizing the efficiency and simplicity of data access; efficiency is obtained using multidimensional access methods and simplicity is achieved by properly handling metadata. Moreover, clustering and classification techniques on large data sets pose additional requirements in terms of computation and memory scalability and interpretability of results. In this study we review some possible solutions.

On accessing data in high-dimensional spaces: A comparative study of three space partitioning strategies

While experience shows that contemporary multi-dimensional access methods perform poorly in high-dimensional spaces, little is known about the underlying causes of this important problem. One of the factors that has a profound effect on the performance of a multi-dimensional structure in high-dimensional situations is its space partitioning strategy. This paper investigates the partitioning strategies of KDB-trees, the Pyramid Technique, and a new point access method called the Θ s Technique. The paper reveals important dimensionality problems associated with these strategies and shows how each strategy affects the retrieval performance across a range of spaces with varying dimensionalities. The Pyramid Technique, which is frequently regarded as the state-of-the-art access method for high-dimensional data, suffers from numerous problems that become particularly severe with highly skewed data in heavily sparse spaces. Although the partitioning strategy of KDB-trees incurs several problems in high-dimensional spaces, it exhibits a remarkable adaptability to the changing data distributions. However, the experimental evidence gathered on both simulated and real data sets shows that the Θ s Technique generally outperforms the other two schemes in high-dimensional spaces, usually by a significant margin.

Processing OLAP queries in hierarchically clustered databases

On-Line Analytical Processing (OLAP) is a technology that encompasses applications requiring a multidimensional and hierarchical view of data. OLAP applications often require fast response time to complex grouping/aggregation queries on enormous quantities of data. Commercial relational database management systems use mainly multiple one-dimensional indexes to process OLAP queries that restrict multiple dimensions. However, in many cases, multidimensional access methods outperform one-dimensional indexing methods. We present an architecture for multidimensional databases that are clustered with respect to multiple hierarchical dimensions. It is based on the star schema and is called CSB star. We focus on processing OLAP queries over this schema using multidimensional access methods. Users can still formulate their queries over a traditional star schema, which are then rewritten by the query processor over the CSB star. We exploit the different clustering features of the CSB star to efficiently process a class of typical OLAP queries. We detect cases where the construction of an evaluation plan can be simplified, and other cases where additional processing techniques can be applied.

A retrieval technique for high-dimensional data and partially specified queries

While the persistent data of many advanced database applications, such as OLAP and scientific studies, are characterized by very high dimensionality, typical queries posed on these data appeal to a small number of relevant dimensions. Unfortunately, the multi-dimensional access methods designed for high-dimensional data perform rather poorly for these partially specified queries. The retrieval technique proposed in this paper uses a combination of two complementary measures to support efficient processing of partial queries over high-dimensional data. First, an elaborate storage organization, called the inverted space, allows the system administrator to control the size of individual indexes in order to avoid the negative impact of extremely high data dimensionality on the retrieval performance. Second, a new indexing structure, which is designed to support the inverted-space storage organization, enables efficient query processing in projected spaces with moderate dimensionality. This indexing mechanism is a general-purpose point access method that effectively attacks the limitations of KDB-trees in spaces with many dimensions, while preserving the simplicity and relatively good performance of the later structure in low-dimensional spaces. The analytical and experimental results show that the new indexing scheme outperforms two other variants of KDB-trees investigated in the paper for both fully and partially specified queries.

Efficient concurrency control in multidimensional access methods

The importance of multidimensional index structures to numerous emerging database applications is well established. However, before these index structures can be supported as access methods (AMs) in a “commercial-strength” database management system (DBMS), efficient techniques to provide transactional access to data via the index structure must be developed. Concurrent accesses to data via index structures introduce the problem of protecting ranges specified in the retrieval from phantom insertions and deletions (the phantom problem ). This paper presents a dynamic granular locking approach to phantom protection in Generalized Search Trees(GiSTs), an index structure supporting an extensible set of queries and data types. The granular locking technique offers a high degree of concurrency and has a low lock overhead. Our experiments show that the granular locking technique (1) scales well under various system loads and (2) similar to the B-tree case, provides a significantly more efficient implementation compared to predicate locking for multidimensional AMs as well. Since a wide variety of multidimensional index structures can be implemented using GiST, the developed algorithms provide a general solution to concurrency control in multidimensional AMs. To the best of our knowledge, this paper provides the first such solution based on granular locking.

Multidimensional access methods

Search operations in databases require special support at the physical level. This is true for conventional databases as well as spatial databases, where typical search operations include thepoint query(find all objects that contain a given search point) and theregion query(find all objects that overlap a given search region). More than ten years of spatial database research have resulted in a great variety of multidimensional access methods to support such operations. We give an overview of that work. After a brief survey of spatial data management in general, we first present the class ofpoint access methods, which are used to search sets of points in two or more dimensions. The second part of the paper is devoted tospatial access methodsto handle extended objects, such as rectangles or polyhedra. We conclude with a discussion of theoretical and experimental results concerning the relative performance of various approaches.

Designing access methods for bitemporal databases

By supporting the valid and transaction time dimensions, bitemporal databases represent reality more accurately than conventional databases. The authors examine the issues involved in designing efficient access methods for bitemporal databases, and propose the partial-persistence and the double-tree methodologies. The partial-persistence methodology reduces bitemporal queries to partial persistence problems for which an efficient access method is then designed. The double-tree methodology "sees" each bitemporal data object as consisting of two intervals (a valid-time and a transaction-time interval) and divides objects into two categories according to whether the right endpoint of the transaction time interval is already known. A common characteristic of both methodologies is that they take into account the properties of each time dimension. Their performance is compared with a straightforward approach that "sees" the intervals associated with a bitemporal object as composing one rectangle, which is stored in a single multidimensional access method. Given that some limited additional space is available, the experimental results show that the partial-persistence methodology provides the best overall performance, especially for transaction timeslice queries. For those applications that require ready, off-the-shelf, access methods, the double-tree methodology is a good alternative.

Point representation of spatial objects and query window extension: a new technique for spatial access methods

The use of Spatial Access Methods (SAMs) in spatial database systems, such as Geographical Information Systems, is necessary to achieve efficient retrieval of data items according to their spatial properties. Existing SAMs organizing minimum bounding rectangle (MBR) approximations of spatial objects can be classified into four groups. Each group is characterized by the special technique adopted for managing MBRs: (a) Ordering, (b) Transformation, (c) Clipping, and (d) Overlapping. This paper introduces a new technique. The basic idea of this technique is to represent all spatial objects by their MBRs and further reduce them into points of the same dimensionality, so that any multidimensional Point Access Method (PAM) may be used to support access. Essential for the functionality of the new method is the query window extension. The results of both analytical and experimental work show that SAMs using the new technique clearly outperform popular SAMs, such as the R- and R*-trees for data sets consisting of equal-sized MBRs. As for data sets of varying MBR sizes a competitive performance can be obtained.

On the data model and access method of summary data management

A data model and an access method for summary data management are presented. Summary data, represented as a trinary tuple (statistical function, category, summary), are metaknowledge summarized by a statistical function of a category of individual information typically stored in a conventional database. For instance, (average-income, female engineer with 10 years' experience and master's degree, $45000) is a summary datum. The computational complexity of the derivability problem has been found intractable in general, and the proposed summary data model, enforcing the disjointness constraint, alleviates the intractable problem without loss of information. In order to store, manage, and access summary data, a multidimensional access method called summary data (SD) tree is proposed. By preserving the category hierarchy, the SD tree provides for efficient operations, including summary data search, derivation, insertion, and deletion.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>