Index-based Algorithms Research Articles

Efficient and Provable Effective Resistance Computation on Large Graphs: An Index-based Approach

Effective resistance (ER) is a fundamental metric for measuring node similarities in a graph, and it finds applications in various domains including graph clustering, recommendation systems, link prediction, and graph neural networks. The state-of-the-art algorithm for computing effective resistance relies on a landmark technique, which involves selecting a node that is easy to reach by all the other nodes as a landmark. The performance of this technique heavily depends on the chosen landmark node. However, in many real-life graphs, it is not always possible to find an easily reachable landmark node, which can significantly hinder the algorithm's efficiency. To overcome this problem, we propose a novel multiple landmarks technique which involves selecting a set of landmark nodes V l such that the other nodes in the graph can easily reach any one of a landmark node in V l . Specifically, we first propose several new formulas to compute ER with multiple landmarks, utilizing the concept of Schur complement. These new formulas allow us to pre-compute and maintain several small-sized matrices related to V l as a compact index. With this powerful index technique, we demonstrate that both single-pair and single-source ER queries can be efficiently answered using a newly-developed V l -absorbed random walk sampling or V l -absorbed push technique. Comprehensive theoretical analysis shows that all proposed index-based algorithms achieve provable performance guarantees for both single-pair and single-source ER queries. Extensive experiments on 5 real-life datasets demonstrate the high efficiency of our multiple landmarks-based index techniques. For instance, our algorithms, with a 1.5 GB index size, can be up to 4 orders of magnitude faster than the state-of-the-art algorithms while achieving the same accuracy on a large road network.

Efficient List Intersection Algorithm for Short Documents by Document Reordering

List intersection plays a pivotal role in various domains such as search engines, database systems, and social networks. Efficient indexes and query strategies can significantly enhance the efficiency of list intersection. Existing inverted index-based algorithms fail to utilize the length information of documents and require excessive list intersections, resulting in lower efficiency. To address this issue, in this paper, we propose the LDRpV (Length-based Document Reordering plus Verification) algorithm. LDRpV filters out documents that are unlikely to satisfy the intersection results by reordering documents based on their length, thereby reducing the number of candidates. Additionally, to minimize the number of list intersection operations, an intersection and verification strategy is designed, where only the first m lists are intersected, and the resulting candidate set is directly verified. This approach effectively improves the efficiency of list intersection. Experimental results on four real datasets demonstrate that LDRpV can achieve a maximum efficiency improvement of 46.69% compared to the most competitive counterparts.

Efficient Resistance Distance Computation: The Power of Landmark-based Approaches

Resistance distance is a fundamental metric to measure the similarity between two nodes in graphs which has been widely used in many real-world applications. In this paper, we study two problems on approximately computing resistance distance: (i) single-pair query which aims at calculating the resistance distance r(s, t) for a given pair of nodes (s, t); and (ii) single-source query which is to compute all the resistance distances r(s, u) for all nodes u in the graph with a given source node s. Existing algorithms for these two resistance distance query problems are often costly on large graphs. To efficiently solve these problems, we first establish several interesting connections among resistance distance, a new concept called v-absorbed random walk, random spanning forests, and a newly-developed v-absorbed push procedure. Based on such new connections, we propose three novel and efficient sampling-based algorithms as well as a deterministic algorithm for single-pair query; and we develop an online and two index-based approximation algorithms for single-source query. We show that the two index-based algorithms for single-source query take almost the same running time as the algorithms for single-pair query with the aid of a linear-size index. The striking feature of all our algorithms is that they are allowed to select an easy-to-hit node by random walks on the graph. Such an easy-to-hit landmark node v can make the v-absorbed random walk sampling, spanning tree sampling, as well as the v-absorbed push more efficient, thus significantly improving the performance of our algorithms. Extensive experiments on 5 real-life datasets show that our algorithms substantially outperform the state-of-the-art algorithms for two resistance distance query problems in terms of both running time and estimation errors.

FPGA Implementation of Efficient CFAR Algorithm for Radar Systems.

The constant false-alarm rate (CFAR) algorithm is essential for detecting targets during radar signal processing. It has been improved to accurately detect targets, especially in nonhomogeneous environments, such as multitarget or clutter edge environments. For example, there are sort-based and variable index-based algorithms. However, these algorithms require large amounts of computation, making them difficult to apply in radar applications that require real-time target detection. We propose a new CFAR algorithm that determines the environment of a received signal through a new decision criterion and applies the optimal CFAR algorithms such as the modified variable index (MVI) and automatic censored cell averaging-based ordered data variability (ACCA-ODV). The Monte Carlo simulation results of the proposed CFAR algorithm showed a high detection probability of 93.8% in homogeneous and nonhomogeneous environments based on an SNR of 25 dB. In addition, this paper presents the hardware design, field-programmable gate array (FPGA)-based implementation, and verification results for the practical application of the proposed algorithm. We reduced the hardware complexity by time-sharing sum and square operations and by replacing division operations with multiplication operations when calculating decision parameters. We also developed a low-complexity and high-speed sorter architecture that performs sorting for the partial data in leading and lagging windows. As a result, the implementation used 8260 LUTs and 3823 registers and took 0.6 μs to operate. Compared with the previously proposed FPGA implementation results, it is confirmed that the complexity and operation speed of the proposed CFAR processor are very suitable for real-time implementation.

Probabilistic Load Flow for Wind Integrated Power System Considering Node Power Uncertainties and Random Branch Outages

This paper proposes an analytical probabilistic load flow (PLF) approach that considers conventional generator outages, load variability, and random branch outages. The branch outages are modeled as 0-1 distributions of fictitious power injections at the appropriate nodes. The distribution of state variables and line power flows is then obtained using a combined Cumulant and Gram-Charlier series expansion approach. The proposed PLF performs contingency sequencing with fuzzy logic to eliminate random line checking and avoid masking mistakes faced by performance index-based algorithms. The Jacobian inverse calculation in the traditional Cumulant method is eliminated to conserve storage space and speed up the computation using the Gauss-Jordan method. The correlations among loads and wind power generations has been modeled using the Nataf transformation process. Results of 24-bus and 259-bus equivalent systems of the Indian southern and western power grids are analyzed and validated with those obtained using the Monte Carlo simulation method. The suggested method's efficacy is justified by its accuracy and low computational burden.

Efficient index-independent approaches for the collective spatial keyword queries

In abundant location-based service applications, it is necessary to process continuous spatial keyword queries over geo-textual data streaming. As an important spatial keyword query, the collective spatial keyword (CSK) query aims to find a set of objects such that it covers all the given keywords collectively, the objects within the set are nearest to the query point, and it has the minimum distance between different objects. The existing approaches for the CSK query are mostly index-based algorithms. Although these approaches gain superior performance, their applicability is significantly limited by the necessity to create an index to organize the dataset.Therefore, these index-based approaches cannot be utilized to process data streaming that prevalently exists in most location-based service applications. In addition, the existing algorithms have much room for improvement as the distances between different objects are overlooked when generating feasible candidate sets. Moreover, the results returned by the proposed algorithms could be further refined to offer better decision support for users. In this paper, a greedy algorithm and an approximate algorithm with a provable approximate bound are proposed for the CSK query. Our approaches are appropriate to the CSK queries where the datasets are not suitable to be organized by indexes and can get better query results with less objects and smaller function scores. To boost the query performance, new pruning strategies and heuristic rules are developed. The experimental results demonstrate scalability, efficiency, and effectiveness of the proposed algorithms.

ROAM: A Fundamental Routing Query on Road Networks with Efficiency

Novel road-network applications often recommend a moving object (e.g., a vehicle) about interesting services or tasks on its way to a destination. A taxi-sharing system, for instance, suggests a new passenger to a taxi while it is serving another one. The traveling cost is then shared among these passengers. A fundamental query is: given two nodes $s$ s and $t$ t , and an area $\mathcal {A}$ A on road network graph $\mathcal {G}$ G , is there a “good” route (e.g., short enough path) $P$ P from $s$ s to $t$ t that crosses $\mathcal {A}$ A in $\mathcal {G}$ G ? In a taxi-sharing system, $s$ s and $t$ t can be a taxi's current and destined locations, and $\mathcal {A}$ A contains all the places to which a person waiting for a taxi is willing to walk. Answering this Route and Area Matching (ROAM) Query allows the application involved to recommend appropriate services to users efficiently. In this paper, we examine efficient ROAM query algorithms. Particularly, we develop solutions for finding a $\rho$ ρ -route , which is an $s$ s - $t$ t path that passes $\mathcal {A}$ A , with a length of at most $(1+\rho)$ ( 1 + ρ ) times the shortest distance between $s$ s and $t$ t . The existence of a $\rho$ ρ -route implies that a service or task located at $\mathcal {A}$ A can be found for a given moving object $m$ m , and that $m$ m only deviates slightly from its current route. We present comprehensive studies on index-free and index-based algorithms for answering ROAM queries. Comprehensive experiments show that our algorithm runs up to 30 times faster than baseline algorithms.

Performance evaluation and comparative assessment of Sentinel-2B, Landsat-8, and AVIRIS-NG sensor imageries for extraction of road and roof surfaces using different algorithms

Urbanization in India has been rapid over the past few decades, which results in a substantial replacement of the natural surfaces into built-up lands. We present a performance evaluation of Sentinel-2B, Landsat-8 multispectral, and AVIRIS-NG hyperspectral imagery for extraction of road and roof surfaces using proposed spectral index-based and other conventional algorithms. The new road extraction index (NREI) and new building extraction index (NBEI) are developed for extraction of road and roof surfaces, respectively. Moreover, existing spectral angle mapper (SAM), spectral information divergence (SID), matched filter (MF), and support vector machine (SVM) are utilized as angle, information, filtering, and machine learning-based algorithms, respectively, for detection of both the surfaces. The results of our study suggest that the performance of AVIRIS-NG sensor is the best in comparison to aforesaid multispectral sensors, whereas Sentinel-2B performs better in comparison to Landsat-8 for extraction of road and roof surfaces. The comparison of various algorithms suggests that proposed indices, MF, and SVM produce the best results for extraction of road and roof surfaces, while SAM and SID are superior algorithms for extraction of both the surfaces in AVIRIS-NG imagery. Further, NREI and MF performed well for extraction of roads followed by NBEI and SAM for roofs in Landsat-8. Finally, NREI, SAM, and SID are found to be efficient for extraction of roads subsequently NBEI, SAM, and SID for roofs in Sentinel-2B imagery.

Efficient Algorithms for Approximate Single-Source Personalized PageRank Queries

Given a graph G , a source node s, and a target node t , the personalized PageRank ( PPR ) of t with respect to s is the probability that a random walk starting from s terminates at t . An important variant of the PPR query is single-source PPR ( SSPPR ), which enumerates all nodes in G and returns the top- k nodes with the highest PPR values with respect to a given source s . PPR in general and SSPPR in particular have important applications in web search and social networks, e.g., in Twitter’s Who-To-Follow recommendation service. However, PPR computation is known to be expensive on large graphs and resistant to indexing. Consequently, previous solutions either use heuristics, which do not guarantee result quality, or rely on the strong computing power of modern data centers, which is costly. Motivated by this, we propose effective index-free and index-based algorithms for approximate PPR processing, with rigorous guarantees on result quality. We first present FORA, an approximate SSPPR solution that combines two existing methods—Forward Push (which is fast but does not guarantee quality) and Monte Carlo Random Walk (accurate but slow)—in a simple and yet non-trivial way, leading to both high accuracy and efficiency. Further, FORA includes a simple and effective indexing scheme, as well as a module for top- k selection with high pruning power. Extensive experiments demonstrate that the proposed solutions are orders of magnitude more efficient than their respective competitors. Notably, on a billion-edge Twitter dataset, FORA answers a top-500 approximate SSPPR query within 1s, using a single commodity server.

An optimal and progressive approach to online search of top-k influential communities

Community search over large graphs is a fundamental problem in graph analysis. Recent studies propose to compute top- k influential communities, where each reported community not only is a cohesive subgraph but also has a high influence value. The existing approaches to the problem of top- k influential community search can be categorized as index-based algorithms and online search algorithms without indexes. The index-based algorithms, although being very efficient in conducting community searches, need to pre-compute a special-purpose index and only work for one built-in vertex weight vector. In this paper, we investigate online search approaches and propose an instance-optimal algorithm LocalSearch whose time complexity is linearly proportional to the size of the smallest subgraph that a correct algorithm needs to access without indexes. In addition, we also propose techniques to make LocalSearch progressively compute and report the communities in decreasing influence value order such that k does not need to be specified. Moreover, we extend our framework to the general case of top- k influential community search regarding other cohesiveness measures. Extensive empirical studies on real graphs demonstrate that our algorithms outperform the existing online search algorithms by several orders of magnitude.

Continuous range-based skyline queries in road networks

With the ever-growing popularity of smartphone devices in recent years, skyline queries over spatial Web objects in road networks have received increasing attention. In the literature, various techniques have been developed to tackle skyline queries that take both spatial and non-spatial attributes into consideration. However, the existing solutions only focus on solving point-based queries, where the query location is a spatial point. We observe that in many real-life applications, the user location is often represented by a spatial range. Thus, in this paper, we study a new problem of range-based skyline queries (CRSQs) in road networks. Two efficient algorithms named landmark-based (LBA) and index-based (IBA) algorithms are proposed. We also present incremental versions of LBA and IBA to handle continuous range-based skyline queries over moving objects. Extensive experiments using real road network datasets demonstrate the effectiveness and efficiency of our proposed algorithms.

Fast online and index-based algorithms for approximate search of RNA sequence-structure patterns

BackgroundIt is well known that the search for homologous RNAs is more effective if both sequence and structure information is incorporated into the search. However, current tools for searching with RNA sequence-structure patterns cannot fully handle mutations occurring on both these levels or are simply not fast enough for searching large sequence databases because of the high computational costs of the underlying sequence-structure alignment problem.ResultsWe present new fast index-based and online algorithms for approximate matching of RNA sequence-structure patterns supporting a full set of edit operations on single bases and base pairs. Our methods efficiently compute semi-global alignments of structural RNA patterns and substrings of the target sequence whose costs satisfy a user-defined sequence-structure edit distance threshold. For this purpose, we introduce a new computing scheme to optimally reuse the entries of the required dynamic programming matrices for all substrings and combine it with a technique for avoiding the alignment computation of non-matching substrings. Our new index-based methods exploit suffix arrays preprocessed from the target database and achieve running times that are sublinear in the size of the searched sequences. To support the description of RNA molecules that fold into complex secondary structures with multiple ordered sequence-structure patterns, we use fast algorithms for the local or global chaining of approximate sequence-structure pattern matches. The chaining step removes spurious matches from the set of intermediate results, in particular of patterns with little specificity. In benchmark experiments on the Rfam database, our improved online algorithm is faster than the best previous method by up to factor 45. Our best new index-based algorithm achieves a speedup of factor 560.ConclusionsThe presented methods achieve considerable speedups compared to the best previous method. This, together with the expected sublinear running time of the presented index-based algorithms, allows for the first time approximate matching of RNA sequence-structure patterns in large sequence databases. Beyond the algorithmic contributions, we provide with RaligNAtor a robust and well documented open-source software package implementing the algorithms presented in this manuscript. The RaligNAtor software is available at http://www.zbh.uni-hamburg.de/ralignator.

Speeding up tandem mass spectral identification using indexes.

Tandem mass spectrometry (MS/MS) has been routinely used in proteomics studies. Post-translational modification (PTM) identification is a challenging problem in tandem mass spectral analysis. In this article, we define two scoring functions for identifying peptides/proteins with PTMs from MS/MS spectra: match scores and diagonal scores, as well as two spectral identification problems based on the two scores. We propose several index-based algorithms for the two problems. Both theoretical and experimental analyses show that the index-based algorithms significantly improve on speed when compared with existing algorithms.

Finding Significant Matches of Position Weight Matrices in Linear Time

Position weight matrices are an important method for modeling signals or motifs in biological sequences, both in DNA and protein contexts. In this paper, we present fast algorithms for the problem of finding significant matches of such matrices. Our algorithms are of the online type, and they generalize classical multipattern matching, filtering, and superalphabet techniques of combinatorial string matching to the problem of weight matrix matching. Several variants of the algorithms are developed, including multiple matrix extensions that perform the search for several matrices in one scan through the sequence database. Experimental performance evaluation is provided to compare the new techniques against each other as well as against some other online and index-based algorithms proposed in the literature. Compared to the brute-force O(mn) approach, our solutions can be faster by a factor that is proportional to the matrix length m. Our multiple-matrix filtration algorithm had the best performance in the experiments. On a current PC, this algorithm finds significant matches (p = 0.0001) of the 123 JASPAR matrices in the human genome in about 18 minutes.

Query processing issues in region-based image databases

Many modern image database systems adopt a region-based paradigm, in which images are segmented into homogeneous regions in order to improve the retrieval accuracy. With respect to the case where images are dealt with as a whole, this leads to some peculiar query processing issues that have not been investigated so far in an integrated way. Thus, it is currently hard to understand how the different alternatives for implementing the region-based image retrieval model might impact on performance. In this paper, we analyze in detail such issues, in particular the type of matching between regions (either one-to-one or many-to-many). Then, we propose a novel ranking model, based on the concept of Skyline, as an alternative to the usual one based on aggregation functions and k-Nearest Neighbors queries. We also discuss how different query types can be efficiently supported. For all the considered scenarios we detail efficient index-based algorithms that are provably correct. Extensive experimental analysis shows, among other things, that: (1) the 1–1 matching type has to be preferred to the N–M one in terms of efficiency, whereas the two have comparable effectiveness, (2) indexing regions rather than images performs much better, and (3) the novel Skyline ranking model is consistently the most efficient one, even if this sometimes comes at the price of a reduced effectiveness.