Efficient Indexing Scheme Research Articles

Optimizing subgraph retrieval and matching with an efficient indexing scheme

Optimizing subgraph retrieval and matching with an efficient indexing scheme

Personalized PageRank on Evolving Graphs with an Incremental Index-Update Scheme

\em Personalized PageRank (PPR) stands as a fundamental proximity measure in graph mining. Given an input graph G with the probability of decay α, a source node s and a target node t, the PPR score π(s,t) of target t with respect to source s is the probability that an α-decay random walk starting from s stops at t. A \em single-source PPR (SSPPR) query takes an input graph G with decay probability α and a source s, and then returns the PPR π(s,v) for each node v ∈ V. Since computing an exact SSPPR query answer is prohibitive, most existing solutions turn to approximate queries with guarantees. The state-of-the-art solutions for approximate SSPPR queries are index-based and mainly focus on static graphs, while real-world graphs are usually dynamically changing. However, existing index-update schemes can not achieve a sub-linear update time. Motivated by this, we present an efficient indexing scheme for single-source PPR queries on evolving graphs. Our proposed solution is based on a classic framework that combines the forward-push technique with a random walk index for approximate PPR queries. Thus, our indexing scheme is similar to existing solutions in the sense that we store pre-sampled random walks for efficient query processing. One of our main contributions is an incremental updating scheme to maintain indexed random walks in expected O(1) time after each graph update. To achieve O(1) update cost, we need to maintain auxiliary data structures for both vertices and edges. To reduce the space consumption, we further revisit the sampling methods and propose a new sampling scheme to remove the auxiliary data structure for vertices while still supporting O(1) index update cost on evolving graphs. Extensive experiments show that our update scheme achieves orders of magnitude speed-up on update performance over existing index-based dynamic schemes without sacrificing the query efficiency.

Determining respiratory rate from photoplethysmogram and electrocardiogram signals using respiratory quality indices and neural networks.

Continuous and non-invasive respiratory rate (RR) monitoring would significantly improve patient outcomes. Currently, RR is under-recorded in clinical environments and is often measured by manually counting breaths. In this work, we investigate the use of respiratory signal quality quantification and several neural network (NN) structures for improved RR estimation. We extract respiratory modulation signals from the electrocardiogram (ECG) and photoplethysmogram (PPG) signals, and calculate a possible RR from each extracted signal. We develop a straightforward and efficient respiratory quality index (RQI) scheme that determines the quality of each moonddulation-extracted respiration signal. We then develop NNs for the estimation of RR, using estimated RRs and their corresponding quality index as input features. We determine that calculating RQIs for modulation-extracted RRs decreased the mean absolute error (MAE) of our NNs by up to 38.17%. When trained and tested using 60-sec waveform segments, the proposed scheme achieved an MAE of 0.638 breaths per minute. Based on these results, our scheme could be readily implemented into non-invasive wearable devices for continuous RR measurement in many healthcare applications.

An Efficient Indexing Scheme for Network Traffic Collection and Retrieval System

Historical network traffic retrieval, both at the packet and flow level, has been applied in many fields of network security, such as network traffic analysis and network forensics. To retrieve specific packets from a vast number of packet traces, it is an effective solution to build indexes for the query attributes. However, it brings challenges of storage consumption and construction time overhead for packet indexing. To address these challenges, we propose an efficient indexing scheme called IndexWM based on the wavelet matrix data structure for packet indexing. Moreover, we design a packet storage format based on the PcapNG format for our network traffic collection and retrieval system, which can speed up the extraction of index data from packet traces. Offline experiments on randomly generated network traffic and actual network traffic are performed to evaluate the performance of the proposed indexing scheme. We choose an open-source and widely used bitmap indexing scheme, FastBit, for comparison. Apart from the native bitmap compression method Word-Aligned Hybrid (WAH), we implement an efficient bitmap compression method Scope-Extended COMPAX (SECOMPAX) in FastBit for performance evaluation. The comparison results show that our scheme outperforms the selected bitmap indexing schemes in terms of time consumption, storage consumption and retrieval efficiency.

AlphaTims: Indexing Trapped Ion Mobility Spectrometry–TOF Data for Fast and Easy Accession and Visualization

High-resolution MS-based proteomics generates large amounts of data, even in the standard LC–tandem MS configuration. Adding an ion mobility dimension vastly increases the acquired data volume, challenging both analytical processing pipelines and especially data exploration by scientists. This has necessitated data aggregation, effectively discarding much of the information present in these rich datasets. Taking trapped ion mobility spectrometry (TIMS) on a quadrupole TOF (Q-TOF) platform as an example, we developed an efficient indexing scheme that represents all data points as detector arrival times on scales of minutes (LC), milliseconds (TIMS), and microseconds (TOF). In our open-source AlphaTims package, data are indexed, accessed, and visualized by a combination of tools of the scientific Python ecosystem. We interpret unprocessed data as a sparse four-dimensional matrix and use just-in-time compilation to machine code with Numba, accelerating our computational procedures by several orders of magnitude while keeping to familiar indexing and slicing notations. For samples with more than six billion detector events, a modern laptop can load and index raw data in about a minute. Loading is even faster when AlphaTims has already saved indexed data in an HDF5 file, a portable scientific standard used in extremely large-scale data acquisition. Subsequently, data accession along any dimension and interactive visualization happens in milliseconds. We have found AlphaTims to be a key enabling tool to explore high-dimensional LC-TIMS-Q-TOF data and have made it freely available as an open-source Python package with a stand-alone graphical user interface at https://github.com/MannLabs/alphatims or as part of the AlphaPept ‘ecosystem’.

Zero-Shot Learning to Index on Semantic Trees for Scalable Image Retrieval.

In this study, we develop a new approach, called zero-shot learning to index on semantic trees (LTI-ST), for efficient image indexing and scalable image retrieval. Our method learns to model the inherent correlation structure between visual representations using a binary semantic tree from training images which can be effectively transferred to new test images from unknown classes. Based on predicted correlation structure, we construct an efficient indexing scheme for the whole test image set. Unlike existing image index methods, our proposed LTI-ST method has the following two unique characteristics. First, it does not need to analyze the test images in the query database to construct the index structure. Instead, it is directly predicted by a network learnt from the training set. This zero-shot capability is critical for flexible, distributed, and scalable implementation and deployment of the image indexing and retrieval services at large scales. Second, unlike the existing distance-based index methods, our index structure is learnt using the LTI-ST deep neural network with binary encoding and decoding on a hierarchical semantic tree. Our extensive experimental results on benchmark datasets and ablation studies demonstrate that the proposed LTI-ST method outperforms existing index methods by a large margin while providing the above new capabilities which are highly desirable in practice.

Towards the design of efficient hash-based indexing scheme for growing databases on non-volatile memory

The index is a fundamental component in data intensive systems to accelerate data retrieval operations. In the design of Non-Volatile Memory (NVM) based indexes, the hash-based structure is one of the most promising candidates since it can take full advantages of byte-addressable property of NVM to perform query operations with constant time complexity. However, we found that the basic operation, “rehash operation”, may incur a large number of write activities on NVM, which is harmful to the endurance of NVM, and will cause drastic performance degradation. Additionally, range query operations cannot be efficiently conducted on hash-based indexes. In this paper, we first investigate how to design an NVM-friendly hash-based structure with the considerations of endurance and performance issues. Then, we propose a novel indexing scheme called “Bucket Hash”, which can significantly reduce the overhead caused by rehash operations and range query operations. We evaluate the proposed Bucket Hash using YCSB workloads. Compared with existing indexes, Bucket Hash achieves 40% reduction on average in the number of NVM writes, meanwhile gaining 30% improvement on timing performance.

An efficient and scalable multi-dimensional indexing scheme for modular data centers

An efficient distributed indexing scheme plays an important role in improving the performance of cloud storage systems. To achieve concurrent query service and high manageability, the indexing scheme should meet the requirements of high scalability and low latency. In this paper, we propose RB-Index, an efficient and scalable multi-dimensional indexing scheme for modular data centers with the BCube topology. RB-Index is a two-layer indexing scheme integrating the BCube based routing protocol and the R-tree based indexing structure. In RB-Index, we build several distinct indexing spaces with dimensions selected according to query history. Each server takes responsibility for a portion of the indexing space according to a mapping scheme. A data pretreatment method and a publishing scheme are presented to uniformly distribute the global index across all the servers in the network. Index maintenance strategies are designed to keep the system cost at a low level. Efficient and complete query strategies are also introduced to support highly concurrent queries. We conduct experiments on Amazon EC2 platform to evaluate the performance of RB-Index and compare its performance with RT-CAN and FT-Index. Experiment results manifest the efficiency and scalability of our indexing scheme.

SLC-index: A scalable skip list-based index for cloud data processing

Due to the increasing number of cloud applications, the amount of data in the cloud shows signs of growing faster than ever before. The nature of cloud computing requires cloud data processing systems that can handle huge volumes of data and have high performance. However, most cloud storage systems currently adopt a hash-like approach to retrieving data that only supports simple keyword-based enquiries, but lacks various forms of information search. Therefore, a scalable and efficient indexing scheme is clearly required. In this paper, we present a skip list-based cloud index, called SLC-index, which is a novel, scalable skip list-based indexing for cloud data processing. The SLC-index offers a two-layered architecture for extending indexing scope and facilitating better throughput. Dynamic load-balancing for the SLC-index is achieved by online migration of index nodes between servers. Furthermore, it is a flexible system due to its dynamic addition and removal of servers. The SLC-index is efficient for both point and range queries. Experimental results show the efficiency of the SLC-index and its usefulness as an alternative approach for cloud-suitable data structures.

STaRS: Simulating Taxi Ride Sharing at Scale

As urban populations grow, cities face many challenges related to transportation, resource consumption, and the environment. Ride sharing has been proposed as an effective approach to reduce traffic congestion, gasoline consumption, and pollution. However, despite great promise, researchers and policy makers lack adequate tools to assess the tradeoffs and benefits of various ride-sharing strategies. In this paper, we propose a real-time, data-driven simulation framework that supports the efficient analysis of taxi ride sharing. By modeling taxis and trips as distinct entities, our framework is able to simulate a rich set of realistic scenarios. At the same time, by providing a comprehensive set of parameters, we are able to study the taxi ride-sharing problem from different angles, considering different stakeholders' interests and constraints. To address the computational complexity of the model, we describe a new optimization algorithm that is linear in the number of trips and makes use of an efficient indexing scheme, which combined with parallelization, makes our approach scalable. We evaluate our framework through a study that uses data about 360 million trips taken by 13,000 taxis in New York City during 2011 and 2012. We describe the findings of the study which demonstrate that our framework can provide insights into strategies for implementing city-wide ride-sharing solutions. We also carry out a detailed performance analysis which shows the efficiency of our approach.

An Efficient Block Index Scheme with Segmentation for Spatio-Textual Similarity Join

An Efficient Block Index Scheme with Segmentation for Spatio-Textual Similarity Join

대규모 RDF 데이터의 특성을 고려한 효율적인 색인 기법

대규모 RDF 데이터의 특성을 고려한 효율적인 색인 기법

Content-based Image Retrieval using Visual Attention Point Features

One of the challenges in the development of a content-based image indexing and retrieval application is to achieve an efficient and robust indexing scheme. Color is a fundamental image feature used in content-based image retrieval (CBIR) systems. This paper proposes a robust and effective image retrieval scheme, which is based on the weighed color histogram of visual attention points. Firstly, the fully affine invariant visual attention points are extracted from the origin color image by using the Affine-SIFT (scale-invariant feature transform) detector. Secondly, according to the color complexity measure (CCM) theory, the visual weight values for the significant visual attention points are calculated to reflect the image local variation. Then, the weighed color histogram of visual attention points is constructed. Finally, the similarity between color images is computed by using the weighed color histogram of visual attention points. Experimental results show that the proposed image retrieval is not only more accurate and efficient in retrieving the user-interested images, but also yields higher retrieval accuracy than some state-of-the-art image retrieval schemes for various test DBs.

Going over the three dimensional protein structure similarity problem

This article presents in detail our novel proposed methodology for detecting similarity between or among three dimensional protein structures. The innovation of our algorithm relies on the fact that during the similarity process, it has the ability to combine many attributes together and fulfill lots of preconditions, which are extensively discussed throughout the paper. Our concept is also supported by an efficient and effective indexing scheme, that provides convincing results comparing to other known methods.

Evaluating Path Queries over Frequently Updated Route Collections

The recent advances in the infrastructure of Geographic Information Systems (GIS), and the proliferation of GPS technology, have resulted in the abundance of geodata in the form of sequences of points of interest (POIs), waypoints, etc. We refer to sets of such sequences as route collections. In this work, we consider path queries on frequently updated route collections: given a route collection and two points n <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</sub> and n <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</sub> , a path query returns a path, i.e., a sequence of points, that connects n <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</sub> to n <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</sub> . We introduce two path query evaluation paradigms that enjoy the benefits of search algorithms (i.e., fast index maintenance) while utilizing transitivity information to terminate the search sooner. Efficient indexing schemes and appropriate updating procedures are introduced. An extensive experimental evaluation verifies the advantages of our methods compared to conventional graph-based search.

Text Document Classification: An Approach Based on Indexing

In this paper we propose a new method of classifying text documents. Unlike conventional vector space models, the proposed method preserves the sequence of term occurrence in a document. The term sequence is effectively preserved with the help of a novel datastructure called ‘Status Matrix’. Further the corresponding classification technique has been proposed for efficient classification of text documents. In addition, in order to avoid sequential matching during classification, we propose to index the terms in Btree, an efficient index scheme. Each term in B-tree is associated with a list of class labels of those documents which contain the term. Further the corresponding classification technique has been proposed. To corroborate the efficacy of the proposed representation and status matrix based classification, we have conducted extensive experiments on various datasets.

Exploring interest correlation for peer-to-peer socialized video sharing

The last five years have witnessed an explosion of networked video sharing, represented by YouTube, as a new killer Internet application. Their sustainable development however is severely hindered by the intrinsic limit of their client/server architecture. A shift to the peer-to-peer paradigm has been widely suggested with success already shown in live video streaming and movie-on-demand. Unfortunately, our latest measurement demonstrates that short video clips exhibit drastically different statistics, which would simply render these existing solutions suboptimal, if not entirely inapplicable. Our long-term measurement over five million YouTube videos, on the other hand, reveals interesting social networks with strong correlation among the videos, thus opening new opportunities to explore. In this article, we present NetTube, a novel peer-to-peer assisted delivering framework that explores the user interest correlation for short video sharing. We address a series of key design issues to realize the system, including a bi-layer overlay, an efficient indexing scheme, a delay-aware scheduling mechanism, and a prefetching strategy leveraging interest correlation. We evaluate NetTube through both simulations and prototype experiments, which show that it greatly reduces the server workload, improves the playback quality and scales well.

EpiExplorer: live exploration and global analysis of large epigenomic datasets.

Epigenome mapping consortia are generating resources of tremendous value for studying epigenetic regulation. To maximize their utility and impact, new tools are needed that facilitate interactive analysis of epigenome datasets. Here we describe EpiExplorer, a web tool for exploring genome and epigenome data on a genomic scale. We demonstrate EpiExplorer's utility by describing a hypothesis-generating analysis of DNA hydroxymethylation in relation to public reference maps of the human epigenome. All EpiExplorer analyses are performed dynamically within seconds, using an efficient and versatile text indexing scheme that we introduce to bioinformatics. EpiExplorer is available at http://epiexplorer.mpi-inf.mpg.de.

An effective and efficient indexing scheme for audio fingerprinting

With the proliferation of multimedia data, there is an increasing need to support the ability to automatically analyse and interpret their contents. In this paper, we propose a new effective and efficient indexing scheme for a fingerprint database that stores songs. A song is represented by a fingerprint extracted from the audio fingerprinting technique and it can be recognised by matching an extracted fingerprint to a database of known audio fingerprints. We are given a fingerprint database of songs and focus our attention on the problem of effective and efficient database search. However, the nature of high dimensionality of fingerprints suffers from the curse of dimensionality, i.e., as the dimension increases, the search performance decreases exponentially. In order to tackle this problem, we propose a new indexing scheme based on the inverted file and duplicate fingerprint indexes with toggled bits. In addition, we also present a new search algorithm based on the offset match principle to avoid expen...

Score level fusion of voting strategy of geometric hashing and SURF for an efficient palmprint-based identification

This paper proposes an efficient indexing scheme for palmprint-based identification system. The proposed system uses geometric hashing of SURF key-points to index the palmprint into hash table and makes score level fusion of voting strategy based on geometric hashing and SURF score to identify the live palmprint. All ordered pairs of SURF key-points of the palmprint are scaled and mapped to a predefined coordinate system and all other points are similarity transformed. The new location after transformation serves as the index of the hash table. During identification, all ordered pairs of key-points of live palmprint are scaled and mapped to the coordinate system while remaining points are similarity transformed. A vote is casted to all images in the corresponding bins. Images having votes more than certain threshold are considered as candidate images of the live palmprint. SURF features of the live palmprint and the candidate images are compared for matching. Matching scores which are based on SURF key-points and vote of the corresponding candidate image are fused using weighted sum rule. The candidate image with the highest fused score is considered as the best match. The system is tested on IITK, CASIA and PolyU datasets. It has been observed that penetration rate of the proposed system is less than 30% for 0% bin miss rate (BMR) and has the identification accuracy of more than 97% for all three datasets. Further, the system is evaluated for robustness on downscaled and rotated. It has been found that the identification accuracy of the system for top best match is more than 90% for images downscaled up to 49% and accuracy is more than 85% when images are rotated at any angle.