Multidimensional Indexing Research Articles

A grid middleware for data management exploiting peer-to-peer techniques

In this paper, we describe a service-oriented middleware architecture for Grid environments which enables efficient data management. Our design introduces concepts from Peer-to-Peer computing in order to provide a scalable and reliable infrastructure for storage, search and retrieval of annotated content. To ensure fast file lookups in the distributed repositories, our system incorporates a multidimensional indexing scheme which serves the need for supporting both exact match and range queries over a group of metadata attributes. Finally, file transfers are conducted using GridTorrent, a grid-enabled, Peer-to-Peer mechanism that performs efficient data transfers by enabling cooperation among participating nodes and balances the cost of file transfer among them. The proposed architecture is the middleware component used by the GREDIA project, in which both media and banking partners plan to share large loads of annotated content.

FOOD Index: A Multidimensional Index Structure for Similarity-Based Fuzzy Object Oriented Database Models

A fuzzy object-oriented data model is a fuzzy logic-based extension to an object-oriented database model that permits uncertain data to be explicitly represented. The fuzzy object-oriented database (FOOD) model is one of the proposed models in the literature to handle uncertainty in object-oriented databases. Several kinds of fuzziness are dealt with in the FOOD model, including fuzziness at attribute level and between object and class and between class and superclass relations. The traditional index structures do not allow efficient access to both crisp and fuzzy objects for fuzzy object-oriented databases since they are not efficient enough in processing both crisp and fuzzy queries. In this study, we propose a new index structure, namely a FOOD index (FI), to deal with different kinds of fuzziness in fuzzy object-oriented databases and to support multidimensional indexing. In this paper, we describe this proposed index structure and show how it supports various types of flexible queries, and evaluate its performance for exact, range, and fuzzy queries.

RCUBE

This article addresses the query performance issue for Relational OLAP (ROLAP) datacubes. We present RCUBE, a distributed multidimensional ROLAP indexing scheme which is practical to implement, requires only a small communication volume, and is fully adapted to distributed disks. Our solution is efficient for spatial searches in high dimensions and scalable in terms of data sizes, dimensions, and number of processors. Our method is also incrementally maintainable. Using “surrogate” group-bys, it allows for the efficient processing of arbitrary OLAP queries on partial cubes, where not all of the group-bys have been materialized. Our experiments with RCUBE show that the ROLAP advantage of better scalability, in comparison to MOLAP, can be maintained while providing a fast and flexible index for OLAP queries.

Skyframe: a framework for skyline query processing in peer-to-peer systems

This paper looks at the processing of skyline queries on peer-to-peer (P2P) networks. We propose Skyframe, a framework for efficient skyline query processing in P2P systems, which addresses the challenges of quick response time, low network communication cost and query load balancing among peers. Skyframe consists of two querying methods: one is optimized for network communication while the other focuses on query response time. These methods are different in the way in which the query search space is defined. In particular, the first method uses a high dominating point that has a large dominating region to prune the search space to achieve a low cost in network communication. On the other hand, the second method relaxes the search space in order to allow parallel query processing to speed up query response. Skyframe achieves query load balancing by both query load conscious data space splitting/merging during the join/departure of nodes and dynamic load migration. We further show how to apply Skyframe to both the P2P systems supporting multi-dimensional indexing and the P2P systems supporting single-dimensional indexing. Finally, we have conducted extensive experiments on both real and synthetic data sets over two existing P2P systems: CAN (Ratnasamy in A scalable content-addressable network. In: Proceedings of SIGCOMM Conference, pp. 161---172, 2001) and BATON (Jagadish et al. in A balanced tree structure for peer-to-peer networks. In: Proceedings of VLDB Conference, pp. 661---672, 2005) to evaluate the effectiveness and scalability of Skyframe.

KpyrRec: a recursive multidimensional indexing structure

The emergence of digital technologies in the audio–visual sector requires the use of powerful tools for fast data access. We focus on high multidimensional indexing structures for content-based similarity search in very large audiovisual databases of companies specialized in business films. This paper summarizes our study on analytical and experimental results for two new indexing structures we propose. These structures are integrated and tested in a search tool intended for professional use in large company film databases.

Content-based image retrieval from a large image database

In this paper, a content-based image retrieval system is built based on the image local descriptors and database indexing and searching techniques. The proposed system is designed for a powerful computer system-the ReMIX platform. The ReMIX system manages hundreds of GB memories and a number of power CPUs. Using this ReMIX system, the image retrieval over a large database becomes an easy and efficient task. In this paper, firstly, the image local descriptors, the ReMIX system's architectures and multi-dimensional database indexing and searching techniques are described. Secondly, the specific database searching strategies based on the ReMIX system are explained. Thirdly, due to the hardware requirement of the ReMIX platform, integer instead of float or double is the best data format for hardware implementation. Therefore, the novel float-to-integer conversion is presented with the step by step procedures. Finally, in the experimental results, the comparisons among using float values and integer values and also using different integer conversion parameters are presented. Several images generated with large image variations are queried into a large database to demonstrate the image retrieval power of the ReMIX system. From the results, it is clear that the ReMIX system is a powerful tool for the content-based image retrieval.

An efficient peer-to-peer indexing tree structure for multidimensional data

As one of the most important technologies for implementing large-scale distributed systems, peer-to-peer (P2P) computing has attracted much attention in both research and industrial communities, for its advantages such as high availability, high performance, and high flexibility to the dynamics of networks. However, multidimensional data indexing remains as a big challenge to P2P computing, because of the inefficiency in search and network maintenance caused by the complicated existing index structures, which greatly limits the scalability of applications and dimensionality of the data to be indexed. We propose SDI (Swift tree structure for multidimensional Data Indexing), a swift index scheme with a simple tree structure for multidimensional data indexing in large-scale distributed systems. While keeping the query efficiency in O ( log N ) in terms of routing hops, SDI has extremely low maintenance costs which is proved through theoretical analysis. Furthermore, SDI overcomes the root-bottleneck problem existing in most other tree-based distributed indexing systems. Extensive empirical study verifies the superiority of SDI in both query and maintenance performance.

P2P case storage and retrieval with an unspecified ontology

Traditional approaches for similarity-based retrieval of structured data, such as Case-Based Reasoning (CBR), have been largely implemented using centralized storage systems. In such systems, when the cases contain both numeric and free-text attributes, similarity-based retrieval cannot exploit standard speedup techniques based on multi-dimensional indexing, and the retrieval is implemented by an exhaustive comparison of the case to be solved with the whole set of stored cases. In this work, we review current research on Peer-to-Peer (P2P) and distributed CBR techniques and propose a novel approach for storage of the case-base in a decentralized Peer-to-Peer environment using the notion of Unspecified Ontology to improve the performance of the case retrieval stage and build CBR systems that can scale up to large case-bases. We develop an algorithm for efficient retrieval of approximated most-similar cases, which exploits inherent characteristics of the unspecified ontology in order to improve the performance of the case retrieval stage in the CBR problem solving cycle. The experiments show that the algorithm successfully retrieves cases close to the most-similar cases, while reducing the number of cases to be compared. Hence, it improves the performance of the retrieval stage. Moreover, the distributed nature of our approach eliminates the computational bottleneck and single point of failure of the centralized storage systems.

Multidimensional indexing tools for the virtual observatory

AbstractThe last decade has seen a dramatic change in the way astronomy is carried out. The dawn of the the new microelectronic devices, like CCDs has dramatically extended the amount of observed data. Large, in some cases all sky surveys emerged in almost all the wavelength ranges of the observable spectrum of electromagnetic waves. This large amount of data has to be organized, published electronically and a new style of data retrieval is essential to exploit all the hidden information in the multiwavelength data. Many statistical algorithms required for these tasks run reasonably fast when using small sets of in‐memory data, but take noticeable performance hits when operating on large databases that do not fit into memory. We utilize new software technologies to develop and evaluate fast multidimensional indexing schemes that inherently follow the underlying, highly non‐uniform distribution of the data: they are layered uniform indices, hierarchical binary space partitioning, and sampled flat Voronoi tessellation of the data. These techniques can dramatically speed up operations such as finding similar objects by example, classifying objects or comparing extensive simulation sets with observations. (© 2007 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

DHR-Trees: Enabling Multidimensional Queries in P2P Systems

There is an increasing requirement for supporting complex multidimensional queries in Peer-to-Peer systems. In the centralized spatial database world, R-Trees and its variant structures are widely accepted due to their capabilities to manage complex multidimensional queries. In this paper, we propose a new multidimensional indexing structure for P2P systems, called Distributed Hilbert R-Trees (DHR-Trees), in which peers organize themselves into an overlay network, dynamically maintain routing tables with region information and collaboratively execute complex multidimensional queries, such as range query and k-nearest neighbors query, efficiently. DHR-Trees has similar topology to the P-Trees P2P system. The peers' routing tables are enhanced with spatial region information, which allow multidimensional query predicates to be adapted into P2P systems with minor modification. The structure design and two major multidimensional query algorithms are presented. Our experimental results demonstrate that it performs well on range queries and k-nearest neighbors queries with multidimensional data set.

A multi-dimensional indexing approach for timestamped event sequence matching

This paper addresses the problem of timestamped event sequence matching, a new type of similar sequence matching that retrieves the occurrences of interesting patterns from timestamped sequence databases. The sequential-scan-based method, the trie-based method, and the method based on the iso-depth index are well-known approaches to this problem. In this paper, we point out their shortcomings, and propose a new method that effectively overcomes these shortcomings. The proposed method employs an R ∗-tree, a widely accepted multi-dimensional index structure that efficiently supports timestamped event sequence matching. To build the R ∗-tree, this method extracts time windows from every item in a timestamped event sequence and represents them as rectangles in n-dimensional space by considering the first and last occurring times of each event type. Here, n is the total number of disparate event types that may occur in a target application. To resolve the dimensionality curse in the case when n is large, we suggest an algorithm for reducing the dimensionality by grouping the event types. Our sequence matching method based on the R ∗-tree performs with two steps. First, it efficiently identifies a small number of candidates by searching the R ∗-tree. Second, it picks out true answers from the set of candidates. We prove its robustness formally, and also show its effectiveness via extensive experiments.

Efficient Physical Organization of R-Trees Using Node Clustering

R-Tree is a multidimensional indexing structure that forms basis for all the multidimensional indexing structures based on data partitioning. A number of attempts have been made in the past to improve the performance of R-Tree by manipulating the tree parameters and the data parameters. But hardly any attempt had been made to use external parameters such as disk parameters to enhance the performance. This work attempts to improve the performance of R-Tree by efficiently clustering the nodes into input-output units of the hard disk with in the constraint that the independence between the logical and physical organization of the R-Tree should be preserved. Moreover, to preserve the structural and functional properties of R-Tree at any point in the process of clustering, this paper introduces a concept called ‘controlled duplication’. Extensive experiments were conducted and the results are tabulated. The improvements are significant and open more avenues for exploration.

Compressing Data Cube in Parallel OLAP Systems

This paper proposes an efficient algorithm to compress the cubes in the progress of the parallel data cube generation. This low overhead compression mechanism provides block-by-block and record-by-record compression by using tuple difference coding techniques, thereby maximizing the compression ratio and minimizing the decompression penalty at run-time. The experimental results demonstrate that the typical compression ratio is about 30:1 without sacrificing running time. This paper also demonstrates that the compression method is suitable for Hilbert Space Filling Curve, a mechanism widely used in multi-dimensional indexing.

A dynamic programming scheme with multidimensional step indexing

The classical dynamic programming scheme with computation for one sequence of steps is generalized to the case of several indexed sequences in which the solution at every step depends on the results found at preceding steps of each of these sequences. The Bellman equations are generalized and proved, the complexity of algorithms is estimated, computer-aided realization is described, and applied problems whose formalization leads to a dynamic programming problem with multidimensional step indexing are stated.

Content-based object organization for efficient image retrieval in image databases

Much research has focused on content-based image retrieval (CBIR) methods that can be automated in image classification and query processing. In this paper, we propose a blob-centric image retrieval scheme based on the blobworld representation. The blob-centric scheme consists of several newly proposed components, including an image classification method, an image browsing method based on semantic hierarchy of representative blobs, and a blob search method based on multidimensional indexing. We present the database structures and their maintenance algorithms for these components and conduct a performance comparison of three image retrieval methods, the naive method, the representative-blobs method, and the indexed-blobs method. Our quantitative analysis shows significant reduction in query response time by using the representative-blobs method and the indexed-blobs method.

The Stepwise Dimensionality Increasing (SDI) Index for High-Dimensional Data

Similarity search is a powerful paradigm for image and multimedia databases, time series databases, and DNA and protein sequence databases. Objects are represented by high-dimensional feature vectors based on color, texture, and shape, in the case of images, for example object similarity is usually implemented via k-nearest neighbor (k-NN) queries, determined by the distance of the endpoints of the feature vectors. The cost of processing k-NN queries via a sequential scan increases with the number of objects and the number of dimensions. Multi-dimensional indexing structures can be used to improve the efficiency of k-NN query processing, but lose their effectiveness as the dimensionality increases. The curse of dimensionality manifests itself in the form of increased overlap among the nodes of the index, so that a high fraction of index pages are touched in processing k-NN queries. The increased dimensionality results in a reduced fanout and an increased index height. We propose a stepwise dimensionality increasing (SDI)-tree index, which aims at reducing the number of disk accesses and CPU processing cost. The index is built using feature vectors transformed via principal component analysis. Dimensions are retained in non-increasing order of their variance according to a parameter p, which specifies the incremental fraction of variance at each level of the index. The optimal value for p is determined experimentally. Experiments on three datasets have shown that SDI-trees access fewer disk pages and incur less CPU time than SR-trees, VAMSR-trees, vector approximation (VA)-Files and the recently proposed iDistance method. In CPU time SDI outperforms the sequential scan and OMNI methods.

Design of content-based retrieval system in remote sensing image database

To retrieve the object region efficaciously from massive remote sensing image database, a model for content-based retrieval of remote sensing image is given according to the characters of remote sensing image application firstly, and then the algorithm adopted for feature extraction and multidimensional indexing, and relevance feedback by this model are analyzed in detail. Finally, the contents intending to be researched about this model are proposed.

SaIL: A Spatial Index Library for Efficient Application Integration

With the proliferation of spatial and spatio-temporal data that are produced everyday by a wide range of applications, Geographic Information Systems (GIS) have to cope with millions of objects with diverse spatial characteristics. Clearly, under these circumstances, substantial performance speed up can be achieved with the use of spatial, spatio-temporal and other multi-dimensional indexing techniques. Due to the increasing research effort on developing new indexing methods, the number of available alternatives is becoming overwhelming, making the task of selecting the most appropriate method for indexing the data according to application needs rather challenging. Therefore, developing a library that can combine a variety of indexing techniques under a common application programming interface can prove to be a valuable tool. In this paper we present SaIL (SpAtial Index Library), an extensible framework that enables easy integration of spatial and spatio-temporal index structures into existing applications. We focus on design issues and elaborate on techniques for making the framework generic enough, so that it can support user defined data types, customizable spatial queries, and a broad range of spatial (and spatio-temporal) index structures, in a way that does not compromise functionality, extensibility and, primarily, ease of use. SaIL is publicly available and has already been successfully utilized for research and commercial applications.

Analysis and Comparison of Color Features for Content Based Image Retrieval

Creation of a content-based image retrieval system implies solving a number of difficult problems, including analysis of low-level image features and construction of feature vectors, multidimensional indexing, design of user interface, and data visualization. Color is one of the important features used in CBIR systems. The methods of characterizing color fall into two major categories:Â Histograms and Statistical. An experimental comparison of a number of different color features for content-based image retrieval presented in these paper. The primary goal is to determine which color feature is most efficient in representing the spatial distribution of images. In this paper, we analyze and evaluate both Statistical and Structural color features. For the experiments, publicly available image databases are used. Analysis and comparison of individual color features are presented

VECTOR SPACE INDEXING FOR BIOSEQUENCE SIMILARITY SEARCHES

We present a multi-dimensional indexing approach for fast sequence similarity search in DNA and protein databases. In particular, we propose effective transformations of subsequences into numerical vector domains and build efficient index structures on the transformed vectors. We then define distance functions in the transformed domain and examine properties of these functions. We experimentally compared their (a) approximation quality for k-Nearest Neighbor (k-NN) queries and both (b) pruning ability and (c) approximation quality for ε-range queries. Results for k-NN queries, which we present here, show that our proposed distances FD2 and WD2 (i.e. Frequency and Wavelet Distance functions for 2-grams) perform significantly better than the others. We then develop effective index structures, based on R-trees and scalar quantization, on top of transformed vectors and distance functions. Promising results from the experiments on real biosequence data sets are presented.