Suffix Tree Research Articles

Linear-time String Indexing and Analysis in Small Space

The field of succinct data structures has flourished over the past 16 years. Starting from the compressed suffix array by Grossi and Vitter (STOC 2000) and the FM-index by Ferragina and Manzini (FOCS 2000), a number of generalizations and applications of string indexes based on the Burrows-Wheeler transform (BWT) have been developed, all taking an amount of space that is close to the input size in bits. In many large-scale applications, the construction of the index and its usage need to be considered as one unit of computation. For example, one can compare two genomes by building a common index for their concatenation and by detecting common substructures by querying the index. Efficient string indexing and analysis in small space lies also at the core of a number of primitives in the data-intensive field of high-throughput DNA sequencing. We report the following advances in string indexing and analysis: We show that the BWT of a string T ∈ {1,…,σ} n can be built in deterministic O ( n ) time using just O ( n log σ) bits of space, where σ ≤ n . Deterministic linear time is achieved by exploiting a new partial rank data structure that supports queries in constant time and that might have independent interest. Within the same time and space budget, we can build an index based on the BWT that allows one to enumerate all the internal nodes of the suffix tree of T . Many fundamental string analysis problems, such as maximal repeats, maximal unique matches, and string kernels, can be mapped to such enumeration and can thus be solved in deterministic O ( n ) time and in O ( n log σ) bits of space from the input string by tailoring the enumeration algorithm to some problem-specific computations. We also show how to build many of the existing indexes based on the BWT, such as the compressed suffix array , the compressed suffix tree , and the bidirectional BWT index , in randomized O ( n ) time and in O ( n log σ) bits of space. The previously fastest construction algorithms for BWT, compressed suffix array and compressed suffix tree, which used O ( n log σ) bits of space, took O ( n log log σ) time for the first two structures and O ( n log ϵ n ) time for the third, where ϵ is any positive constant smaller than one. Alternatively, the BWT could be previously built in linear time if one was willing to spend O ( n log σ log log σ n ) bits of space. Contrary to the state-of-the-art, our bidirectional BWT index supports every operation in constant time per element in its output.

Analysis of the trajectory shapes of moving objects in the video sequence with use of structural description

This article proposes using structural description for graphical objects to solve an urgent task of trajectory analysis. A range of modern trajectory analysis approaches were analyzed and the best that is based on Graph Convolutional Neural Networks and Suffix Tree Clustering algorithm was chosen. Descripted ways to reduce computational sources for this neural network approach. This neural network was adapted to analyze structural description and advantages of this approach are shown.

Dendrogeomorphic reconstruction of floods in a dynamic tropical river

Tropical regions are frequently affected by intense floods causing substantial human and economic losses. A proper management of floods and the prevention of disasters is, however, often hampered by a generalized paucity of systematic discharge measurements, which in turn renders any assessment of the frequency and magnitude of extreme floods challenging or impossible. Here, we analyze the suitability of trees impacted by floods and their growth-ring records to provide insights into past flood activity and to allow estimation of their magnitude. We base this exploratory study on the extreme floods triggered by the passage of tropical storm Nate on October 5, 2017 and investigate whether dendrogeomorphic approaches can be employed to date and quantify floods in the catchment of tropical Río General (Costa Rica). To this end, we sampled 91 trees showing scars in three river reaches and tested their potential to serve as paleostage indicators (PSI). High-resolution (0.5 m) digital surface and elevation models were then obtained with an Unmanned Aerial Vehicle to run a step-backwater hydraulic simulation aimed at defining flood peak discharge for which the mean squared errors between PSI heights and simulated water tables could be minimized. In a last analytical step, we investigated which hydraulic (i.e., Froude number, flow velocity) and fluvial landform characteristics explained deviations between scar heights and modeled water tables best by using a generalized linear model. Our analysis confirms that scarred trees can indeed be used for the reconstruction of past floods in tropical river systems and that the geomorphic position of trees will exert control on deviations between modeled water tables and scar height, with cut banks being most suited for scar-based flood reconstruction.

Efficient Path Query Processing Over Massive Trajectories on the Cloud

A path query aims to find trajectories passing a given sequence of connected road segments within a time period. It is very useful in many urban applications: 1) traffic modeling, 2) frequent path mining, 3) intersection coordination, and 4) traffic anomaly detection. Existing solutions for path query processing are implemented based on single machines, which are not efficient for the following tasks: 1) indexing large-scale historical data; 2) handling real-time trajectory updates; and 3) processing concurrent path queries from urban data mining applications. In this paper, we design and implement a cloud-based path query processing framework based on Microsoft Azure. We modify existing suffix tree structure to index trajectories using Azure Table. The proposed system consists of two main parts: 1) back-end processing , which performs pre-processing (i.e., parsing and map-matching) and index building tasks with a distributed computing platform (i.e., Storm) used to efficiently handle massive real-time trajectory updates; and 2) query processing , which answers path queries using Azure Storm to improve efficiency and overcome I/O bottleneck. Extensive experiments are performed based on the real-time taxi trajectories from Guiyang City, the capital of Guizhou Province, China to confirm the system efficiency. We also demonstrate a real deployed traffic analysis system based on our query processing framework.

Geocoding of trees from street addresses and street-level images

We introduce an approach for updating older tree inventories with geographic coordinates using street-level panorama images and a global optimization framework for tree instance matching. Geolocations of trees in inventories until the early 2000s where recorded using street addresses whereas newer inventories use GPS. Our method retrofits older inventories with geographic coordinates to allow connecting them with newer inventories to facilitate long-term studies on tree mortality etc. What makes this problem challenging is the different number of trees per street address, the heterogeneous appearance of different tree instances in the images, ambiguous tree positions if viewed from multiple images and occlusions. To solve this assignment problem, we (i) detect trees in Google street-view panoramas using deep learning, (ii) combine multi-view detections per tree into a single representation, (iii) and match detected trees with given trees per street address with a global optimization approach. Experiments for >50000 trees in 5 cities in California, USA, show that we are able to assign geographic coordinates to 38% of the street trees, which is a good starting point for long-term studies on the ecosystem services value of street trees at large scale.

Zygnematophyceae: from living algae collections to the establishment of future models.

The class of conjugating green algae, Zygnematophyceae (Conjugatophyceae), is extremely rich in species and has attracted the interest of phycologists for a long time. It is now widely accepted that this class of charophyte algae holds a key position in the phylogenetic tree of streptophytes, where they represent the closest relatives to all land plants (embryophytes). It is increasingly evident that robust model plants that can be easily cultivated and genetically transformed are necessary to better understand the process of terrestrialization and the related molecular, cellular, and physiological adaptations. Living algae collections play an important role, not only for phylogenomic-based taxonomy but also for screening for suitable model organisms. For this review, we screened six major public algae collections for Zygnematophyceae strains and established a cumulative list comprising 738 different taxa (including species, subspecies, varieties, and forms). From the described biodiversity with 8883 registered taxa (AlgaeBase) the cultured Zygnematophyceae taxa worldwide cover only ~8.3%. We review the past research on this clade of algae and discuss it from the perspective of establishing a model organism. We present data on the life cycle of the genera Micrasterias and Spirogyra, representing the orders Desmidiales and Zygnematales, and outline the current status of genetic transformation of Zygnematophyceae algae and future research perspectives.

A trunk-based SLAM backend for smartphones with online SLAM in large-scale forest inventories

Reliable forest resource information is needed to assess the forest development status and design management plans for forest maintenance and conservation. The forest field sample inventory is a vital forest resource inventory method. Thus, forest inventory reliability depends on tree attribute estimation accuracy and the quantity and quality of field samples. Simultaneous localization and mapping (SLAM)-based mobile laser scanners (MLSs) are convenient inventory tools due to their mobility and global navigation satellite system (GNSS) signal independence. However, such scanners may be heavy, expensive and unable to verify results on-site. With the improved SLAM algorithm and chip computing capabilities, a smartphone can deploy an online SLAM system, which allows the smartphone to perform the relative positioning in forests without GNSS signals. Previous research studies have demonstrated this simple, portable, and economical device for estimating the tree position and diameter at breast height (DBH) through tree-by-tree measurements in real time. However, the device might face a challenge in large-scale forest inventories because the image-feature-based backend may not work well in forests that are not well constructed for traditional SLAM systems. In this paper, an online trunk-based backend was designed to accurately estimate tree position and correct pose drift in large-scale forest inventories in real time. Specifically, a trunk-based loop closure detection algorithm was designed for detecting whether an earlier observed tree is re-observed to provide nodes and constraints for tree position graph optimization; this algorithm uses the provided nodes and constraints to build and optimize the tree position graph and then correct the current pose based on the optimized globally consistent tree position graph. This new backend was integrated with the previous work as an executable program that can be deployed on a smartphone with an online RGB-D SLAM system. The method was tested in 5 field sample plots (32 × 32 m), and the reference tree positions were collected using terrestrial laser scanning (TLS) through multi-scan mode. The distance mean between the estimated and reference tree positions was 0.133 m when using our new backend, and it was 0.759 m when estimated with the RTAB-Map. The tree position estimates were unbiased and had root mean square errors (RMSEs) of less than 0.09 m in the x-axis, y-axis and z-axis directions when using the trunk-based backend. However, the estimates had biases of −0.125 m, −0.261 m and 0.262 m and RMSEs of more than 0.30 m in the x-axis, y-axis and z-axis directions without the new backend. The results showed that the designed trunk-based backend allows a smartphone with an online SLAM system to function as an accurate and efficient tool for large-scale forest inventories. However, the method was tested only in 32 × 32 m square field sample plots. More tests must be performed in larger plots in the future, although enough loop-closure constraints can theoretically guarantee the accuracy of the tree position graph and current pose.

Finding all maximal perfect haplotype blocks in linear time

Recent large-scale community sequencing efforts allow at an unprecedented level of detail the identification of genomic regions that show signatures of natural selection. Traditional methods for identifying such regions from individuals’ haplotype data, however, require excessive computing times and therefore are not applicable to current datasets. In 2019, Cunha et al. (Advances in bioinformatics and computational biology: 11th Brazilian symposium on bioinformatics, BSB 2018, Niterói, Brazil, October 30 - November 1, 2018, Proceedings, 2018. https://doi.org/10.1007/978-3-030-01722-4_3) suggested the maximal perfect haplotype block as a very simple combinatorial pattern, forming the basis of a new method to perform rapid genome-wide selection scans. The algorithm they presented for identifying these blocks, however, had a worst-case running time quadratic in the genome length. It was posed as an open problem whether an optimal, linear-time algorithm exists. In this paper we give two algorithms that achieve this time bound, one conceptually very simple one using suffix trees and a second one using the positional Burrows–Wheeler Transform, that is very efficient also in practice.

Individual Tree Position Extraction and Structural Parameter Retrieval Based on Airborne LiDAR Data: Performance Evaluation and Comparison of Four Algorithms

Information for individual trees (e.g., position, treetop, height, crown width, and crown edge) is beneficial for forest monitoring and management. Light Detection and Ranging (LiDAR) data have been widely used to retrieve these individual tree parameters from different algorithms, with varying successes. In this study, we used an iterative Triangulated Irregular Network (TIN) algorithm to separate ground and canopy points in airborne LiDAR data, and generated Digital Elevation Models (DEM) by Inverse Distance Weighted (IDW) interpolation, thin spline interpolation, and trend surface interpolation, as well as by using the Kriging algorithm. The height of the point cloud was assigned to a Digital Surface Model (DSM), and a Canopy Height Model (CHM) was acquired. Then, four algorithms (point-cloud-based local maximum algorithm, CHM-based local maximum algorithm, watershed algorithm, and template-matching algorithm) were comparatively used to extract the structural parameters of individual trees. The results indicated that the two local maximum algorithms can effectively detect the treetop; the watershed algorithm can accurately extract individual tree height and determine the tree crown edge; and the template-matching algorithm works well to extract accurate crown width. This study provides a reference for the selection of algorithms in individual tree parameter inversion based on airborne LiDAR data and is of great significance for LiDAR-based forest monitoring and management.

Counter based suffix tree for DNA pattern repeats

In recent years, the string datasets have increased exponentially, so is the need to process them. Most of these datasets have been deeply rooted in the field of bioinformatics since the entire characteristics of any living organism is encoded in their genes. Genes consist of nucleic bases which will, therefore, makeup the entire genome. A genome is made of a concatenation of different types of nucleic bases. To efficiently extract the information encrypted in these sequences there is a need to use algorithms to decrypt it. Most available methods use the data structure commonly referred to as the suffix tree. They have tremendously evolved over the years, and the on-line construction of the suffix tree is deemed as the best data structure, however, it is not optimal when it comes to finding repeated sequences because of many traversals algorithm will have to do when identifying repeats. To improve the speed and of finding repeats we developed a counter based suffix tree algorithm. Our work presents a novel algorithm of constructing a counter based suffix tree without losing its properties. The counter based suffix tree time complexity is θ(n) where n represents the length of a string. Which is the same as the fastest suffix tree implementation. We have shown that the counter based suffix tree will reduce the search time when identifying repeats. We have proved that a counter based suffix tree can be developed during construction.

A compressed dynamic self-index for highly repetitive text collections

We present a novel compressed dynamic self-index for highly repetitive text collections. ESP-index is a static self-index of this type and has a large disadvantage of slow pattern search for short patterns. We obtain faster pattern search by leveraging the idea behind a truncated suffix tree (TST) to develop the first compressed dynamic self-index, called TST-index, that supports not only fast pattern search but also dynamic update operations for highly repetitive texts. Experiments with a benchmark dataset show that the pattern search performance of the TST-index is significantly better than that of ESP-index for short patterns.

Fully Functional Suffix Trees and Optimal Text Searching in BWT-Runs Bounded Space

Indexing highly repetitive texts—such as genomic databases, software repositories and versioned text collections—has become an important problem since the turn of the millennium. A relevant compressibility measure for repetitive texts is r , the number of runs in their Burrows-Wheeler Transforms (BWTs). One of the earliest indexes for repetitive collections, the Run-Length FM-index, used O ( r ) space and was able to efficiently count the number of occurrences of a pattern of length m in a text of length n (in O ( m log log n ) time, with current techniques). However, it was unable to locate the positions of those occurrences efficiently within a space bounded in terms of r . In this article, we close this long-standing problem, showing how to extend the Run-Length FM-index so that it can locate the occ occurrences efficiently (in O ( occ log log n ) time) within O ( r ) space. By raising the space to O ( r log log n ), our index counts the occurrences in optimal time, O ( m ), and locates them in optimal time as well, O ( m + occ ). By further raising the space by an O ( w / log σ) factor, where σ is the alphabet size and w = Ω (log n ) is the RAM machine size in bits, we support count and locate in O (⌈ m log (σ)/ w ⌉) and O (⌈ m log (σ)/ w ⌉ + occ ) time, which is optimal in the packed setting and had not been obtained before in compressed space. We also describe a structure using O ( r log ( n / r )) space that replaces the text and extracts any text substring of length ℓ in the almost-optimal time O (log ( n / r )+ℓ log (σ)/ w ). Within that space, we similarly provide access to arbitrary suffix array, inverse suffix array, and longest common prefix array cells in time O (log ( n / r )), and extend these capabilities to full suffix tree functionality, typically in O (log ( n / r )) time per operation. Our experiments show that our O ( r )-space index outperforms the space-competitive alternatives by 1--2 orders of magnitude in time. Competitive implementations of the original FM-index are outperformed by 1--2 orders of magnitude in space and/or 2--3 in time.

Near-Complete Genome Sequences of Rice Yellow Mottle Virus Isolates from Senegal.

Rice yellow mottle virus in Senegal is reported here for the first time. The near-complete genomic sequences of two isolates (Se1 and Se5) were obtained. A comparison with 18 sequences from West Africa revealed a new cluster with an isolate from Gambia, located at a basal position in the phylogenetic tree.

Frequency Response Curves and Dynamic Characteristics of a Ginkgo Tree in Different Growth Periods

HighlightsThe frequency domain modal parameters identification method was applied to a ginkgo tree.Dynamic characteristics of the ginkgo tree were tested during five phenological periods.Almost all resonance frequencies were near the peaks of the frequency response curves.Leaves caused the number of natural frequencies of the ginkgo tree to be greatly reduced.Abstract. Understanding the dynamic characteristics of fruit trees is the premise of effective mechanized harvesting. This study performed a tracking test on a ginkgo tree in five phenological periods from the dormant period to the leaf-unfolding period. The frequency domain modal parameters identification method was applied to the ginkgo tree, and the relationship between the natural frequencies and resonance frequencies of the ginkgo tree was obtained. The main factors affecting the fundamental frequency and damping ratio of the ginkgo tree were not the elastic modulus and moisture content but rather the growth of the leaves. The growth of leaves caused the number of natural frequencies in the low-frequency band to be greatly reduced, and the value of the natural frequencies exhibited a slightly decreasing tendency. The damping caused by leaves had a significant weakening effect on the transmission of vibrational energy on the lateral branches. The resonance frequencies that caused strong response of the ginkgo tree were mostly near the peak frequencies of the frequency response curves (natural frequencies), but eccentric motor excitation could not effectively stimulate all the natural frequencies of the ginkgo tree to reach resonance. In the frequency response curves of the ginkgo tree, the main natural frequency with the maximum energy might not cause the maximum vibration response of the ginkgo tree, even if this excitation frequency could induce resonance. Resonance could be used to maximize the transfer of excitation energy, but each position of the tree had its own independent frequency spectrum characteristics. A single excitation frequency could not cause all positions of the ginkgo tree to resonate simultaneously. Changing the excitation frequency of harvesting equipment within a small frequency range could achieve the maximum resonance response of most positions on fruit trees. Keywords: Dynamic characteristics, Growth periods, Leaves, Natural frequencies, Resonance.

Fault Tolerant Suffix Trees

Classical algorithms and data structures assume that the underlying memory is reliable, and the data remain safe during or after processing. However, the assumption is perilous as several studies have shown that large and inexpensive memories are vulnerable to bit flips. Thus, the correctness of output of a classical algorithm can be threatened by a few memory faults. Fault tolerant data structures and resilient algorithms are developed to tolerate a limited number of faults and provide a correct output based on the uncorrupted part of the data. Suf- fix tree is one of the important data structures that has widespread applications including substring search, super string problem and data compression. The fault tolerant version of the suffix tree presented in the literature uses complex techniques of encodable and decodable error-correcting codes, blocked data structures and fault-resistant tries. In this work, we use the natural approach of data replication to develop a fault tolerant suffix tree based on the faulty memory random access machine model. The proposed data structure stores copies of the indices to sustain memory faults injected by an adversary. We develop a resilient version of the Ukkonen’s algorithm for constructing the fault tolerant suffix tree and derive an upper bound on the number of corrupt suffixes.

Stereoscopic Image Feature Indexing Based on Hybrid Grid Multiple Suffix Tree and Hierarchical Clustering

In order to achieve content-based binocular stereoscopic image or video retrieval efficiently, a feature indexing algorithm based on hybrid grid multiple suffix tree and hierarchical clustering is proposed. With the RGB-D image model, the shape features of depth map obtained from the matching of binocular stereoscopic left image and right image and the color features of left image are extracted respectively. The features are quantified and hashed, and the optimized underlying features are sorted as leaf nodes of hierarchical indexing. Then the shape and color feature values of leaf nodes are mapped to the two-dimensional coordinates, and the 2D feature points are put into different grid hash areas respectively by clustering and labeled with multiple suffix tree. Furthermore, to construct the global index, a pointer to an array of clustering grid center point is defined according to the computation of grid area feature values. The experimental results show that compared to the double grid suffix tree and typical stereoscopic image feature indexing structure, the proposed algorithm can effectively reduce the indexing construction complexity While maintaining high recall, it can also greatly improve the query efficiency, which can better realize the feature indexing of binocular stereoscopic images or videos.

A Gray Box for Visualizing Instruction Sequence Based on Improved Suffix Tree

Gray box is a kind of device in which the working process of a program or system is locally recognized. Gray box testing, also known as gray box analysis, is a software debugging method based on the limited cognition of the internal details of the program. Testers may know how system components interact with each other, but they lack a detailed understanding of internal program functions and operation. So the construction of gray box is particularly important. The most original gray boxes are static debugger and dynamic debugger. And then reflexion model, which reduces the manual work greatly, is developed and applied. The latest gray boxes are focus on regarding instructions as a natural language using the mature mathematical model to mine their internal value. Adhering to the idea of latest researches, our paper improves the original suffix tree and use the improved suffix tree as a mathematical models to analyse and visualize the internal logic of instructions. Our gray box aims at solving three problems in practical application. In addition, we explain the complexity of instruction sequence and put forward a prediction formula for the building part. By experiment, we prove the time complexity of each part and the correctness of the prediction formula, and show the effect of visualizing part.

Comparison of accuracy of tree height and position focusing on differences in position determination method in UAV measurement

Comparison of accuracy of tree height and position focusing on differences in position determination method in UAV measurement

How the position of trees planting can improve the near-road air quality exposed to CO<SUB align="right">2 emission from transportation

How the position of trees planting can improve the near-road air quality exposed to CO<SUB align="right">2 emission from transportation

РАЗРАБОТКА ОПТИМАЛЬНЫХ СХЕМ ЛЕСОПОСАДКИ МЕТОДОМ ИМИТАЦИОННОГО МОДЕЛИРОВАНИЯ

One way to increase the productivity of artificial forestation is to control the density and relative position of trees. In this regard, there are tasks of finding the optimal planting schemes to obtain the maximum wood stock of a given diameter. Based on the developed simulation model, a comparative analysis of the productivity of spruce stands with different planting schemes was carried out. The optimal forest planting schemes for spruce stands, which provide the maximum stock of small, medium and large wood, are determined. It is shown that in order to maximize the stock of wood of different size classes, different initial planting densities are required. The higher the size classes of the grown wood, the lower the optimal planting density.