Sequence Search Algorithms Research Articles

Noise reduction algorithm of GPR data based on MMSE‐PDS

AbstractGround‐penetrating radar (GPR) technology is widely used in tunnel engineering detection; however, various factors, such as environmental interference and low signal‐to‐noise ratio characteristics of the echo data, limit the detection accuracy. A noise and interference suppression algorithm based on improved singular value decomposition is proposed in this paper. Compared with traditional filtering methods, the proposed method has the advantages of thorough denoising, no clutter, efficient improvement of profile resolution and less dependence on parameters. The main features of the proposed algorithm are as follows. (1) Given the global characteristics of the noise disturbance on the signal space, the minimum mean square error estimation is employed to approximate the effective signal, introducing the correction factor to suppress the larger singular value from the noise output in the reconstructing process of the effective signal subspace and to eliminate the strong direct wave interference to avoid producing false signals. (2) A positive difference sequence search algorithm based on rank order variance as well as the method of selecting correction factors are proposed to improve the processing accuracy. In order to verify the design, the tunnel lining simulation model and the actual tunnel lining detection data are used. The results show good performance for noise and interference suppression, providing technical support for improving GPR data quality and tunnel detection accuracy.

Near-Duplicate Sequence Search at Scale for Large Language Model Memorization Evaluation

Recent studies show that large language models (LLM) unintendedly memorize part of the training data, which brings serious privacy risks. For example, it has been shown that over 1% of tokens generated unprompted by an LLM are part of sequences in the training data. However, current studies mainly focus on the exact memorization behaviors. In this paper, we propose to evaluate how many generated texts have near-duplicates (e.g., only differ by a couple of tokens out of 100) in the training corpus. A major challenge of conducting this evaluation is the huge computation cost incurred by near-duplicate sequence searches. This is because modern LLMs are trained on larger and larger corpora with up to 1 trillion tokens. What's worse is that the number of sequences in a text is quadratic to the text length. To address this issue, we develop an efficient and scalable near-duplicate sequence search algorithm in this paper. It can find (almost) all the near-duplicate sequences of the query sequence in a large corpus with guarantees. Specifically, the algorithm generates and groups the min-hash values of all the sequences with at least t tokens (as very short near-duplicates are often irrelevant noise) in the corpus in linear time to the corpus size. We formally prove that only 2 n+1/t+1 -1 min-hash values are generated for a text with n tokens in expectation. Thus the index time and size are reasonable. When a query arrives, we find all the sequences sharing enough min-hash values with the query using inverted indexes and prefix filtering. Extensive experiments on a few large real-world LLM training corpora show that our near-duplicate sequence search algorithm is efficient and scalable.

AIomics: Exploring More of the Proteome Using Mass Spectral Libraries Extended by Artificial Intelligence.

The unbounded permutations of biological molecules, including proteins and their constituent peptides, present a dilemma in identifying the components of complex biosamples. Sequence search algorithms used to identify peptide spectra can be expanded to cover larger classes of molecules, including more modifications, isoforms, and atypical cleavage, but at the cost of false positives or false negatives due to the simplified spectra they compute from sequence records. Spectral library searching can help solve this issue by precisely matching experimental spectra to library spectra with excellent sensitivity and specificity. However, compiling spectral libraries that span entire proteomes is pragmatically difficult. Neural networks that predict complete spectra containing a full range of annotated and unannotated ions can be used to replace these simplified spectra with libraries of fully predicted spectra, including modified peptides. Using such a network, we created predicted spectral libraries that were used to rescore matches from a sequence search done over a large search space, including a large number of modifications. Rescoring improved the separation of true and false hits by 82%, yielding an 8% increase in peptide identifications, including a 21% increase in nonspecifically cleaved peptides and a 17% increase in phosphopeptides.

Low-Thrust, Multiple-Near-Earth-Asteroid Mission Design with Sample Return to Earth Using Machine Learning

Sample return missions to near-Earth asteroids (NEAs) are invaluable for the scientific community to learn more about the initial stages of the solar system formation and life evolution. Low-thrust propulsion technology enables missions with multiple-asteroid rendezvouses to collect samples and eventually return to Earth, thanks to its high specific impulse. To identify the best asteroid sequences with return to Earth, this work proposes to employ machine learning techniques and, specifically, artificial neural networks (ANNs), to quickly estimate the cost of each transfer between asteroids. The ANN is integrated within a sequence search algorithm based on a tree search, which identifies the asteroid sequences and selects the best ones in terms of propellant mass required and interest value. This algorithm can design NEA sequences so that specific asteroids of interest, for which a sample return would be more valuable, can be targeted. A pseudospectral optimal control solver is then used to find the optimal trajectory and control history. The performance of the proposed methodology is assessed by analyzing three distinctive NEA sequences ending with return to Earth and rendezvous. Near-term low-thrust propulsion enables to rendezvous five asteroids, and return samples to Earth in about 10 years from launch. It is demonstrated that visiting more scientifically interesting asteroids increases the appeal of the sequence at the cost of more propellant mass required.

Gene Structure-Based Homology Search Identifies Highly Divergent Putative Effector Gene Family.

Homology of highly divergent genes often cannot be determined from sequence similarity alone. For example, we recently identified in the aphid Hormaphis cornu a family of rapidly evolving bicycle genes, which encode novel proteins implicated as plant gall effectors, and sequence similarity search methods yielded few putative bicycle homologs in other species. Coding sequence-independent features of genes, such as intron-exon boundaries, often evolve more slowly than coding sequences, however, and can provide complementary evidence for homology. We found that a linear logistic regression classifier using only structural features of bicycle genes identified many putative bicycle homologs in other species. Independent evidence from sequence features and intron locations supported homology assignments. To test the potential roles of bicycle genes in other aphids, we sequenced the genome of a second gall-forming aphid, Tetraneura nigriabdominalis and found that many bicycle genes are strongly expressed in the salivary glands of the gall forming foundress. In addition, bicycle genes are strongly overexpressed in the salivary glands of a non-gall forming aphid, Acyrthosiphon pisum, and in the non-gall forming generations of H. cornu. These observations suggest that Bicycle proteins may be used by multiple aphid species to manipulate plants in diverse ways. Incorporation of gene structural features into sequence search algorithms may aid identification of deeply divergent homologs, especially of rapidly evolving genes involved in host-parasite interactions.

Design of New Word Retrieval Algorithm for Chinese-English Bilingual Parallel Corpus

Natural language processing is an important direction in the field of computer science and artificial intelligence. It can realize various theories and methods of effective communication between humans and computers using natural language. Machine learning is a branch of natural language processing research, which is based on a large-scale English-Chinese database. Due to the relatively poor alignment corpus of English and Chinese bilingual sentences containing unknown words, machine translation is unprofessional and unbalanced, which is the problem studied in this paper. The purpose of this paper is to design and implement a length-based system for sentence alignment between English and Chinese bilingual texts. The research content of this paper is mainly divided into the following parts. First, the evaluation function of bilingual sentence alignment is designed, and on this basis, the bilingual sentence alignment algorithm based on the length and the optimal sentence pair sequence search algorithm is designed. In this paper, China National Knowledge Infrastructure (CNKI) is selected as an English-Chinese bilingual candidate website and English-Chinese bilingual web pages are downloaded. After analyzing the downloaded pages, nontext content such as page tags is removed, and bilingual text information is stored so as to establish an English-Chinese bilingual corpus based on segment alignment and retain English-Chinese bilingual keywords in the web pages. Second, extract the dictionary from the software StarDict, analyze the original dictionary format, and turn it into a custom dictionary format, which is convenient and better to use the double-sentence sentence alignment system, which is conducive to expanding the number of dictionaries and increasing the professionalism of vocabulary. Finally, we extract the stems of English words from the established corpus to simplify the complexity of English word processing, reduce the noise caused by the conversion of word parts of speech, and improve the operation efficiency. A bilingual sentence alignment system based on length is implemented. Finally, the system parameters are adjusted for comparative experiments to test the system performance.

Design of multiple space debris removal missions using machine learning

Active debris removal (ADR) allows for the disposal of inactive satellites and larger objects, preventing the build-up of space junk and allowing to replace aging agents in a constellation. To make ADR missions more commercially viable, the removal and disposal of multiple debris objects using a single spacecraft are investigated. This paper proposes the use of artificial neural networks (ANNs) to quickly estimate the cost and duration of the transfers to de-orbit a range of debris objects, so that it is possible to identify the optimal sequence of objects which minimizes the cost and/or the duration of the mission, for the maximum number of de-orbited objects. To this end, the ANN is integrated within a sequence search algorithm based on a tree search. The performance of the proposed methodology is assessed by analyzing three distinctive sequences of multiple space debris removals. A near-term low-thrust propulsion technology enables to dispose of up to 13 debris objects within 10 years, when the optimal design parameters are chosen. The use of ANN allows for this solution to be found 26 times faster than current methods, while enabling the selection of faster and less expensive (being the propellant mass required lower) options.

Shared ancestry of core-histone subunits and non-histone plant proteins containing the Histone Fold Motif (HFM).

The three helical Histone Fold Motif (HFM) of core histone proteins provides an evolutionarily favored site for the protein-DNA interface. Despite significant variation in sequence, the HFM retains a distinctive structural fold that has diversified into several non-histone protein families. In this work, we explore the ancestry of non-histone HFM containing families in the plant kingdom. A sequence search algorithm was developed using iterative profile Hidden Markov Models to identify remote homologs of core-histone proteins. The resulting hits were functionally annotated, classified into families, and subjected to comprehensive phylogenetic analyses via Maximum likelihood and Bayesian methods. We have identified 4390 HFM containing proteins in the plant kingdom that are not histones, mostly existing as diverse transcription factor families, distributed widely within and across taxonomic groups. Patterns of homology suggest that core histone subunit H2A has evolved into newer families like NF-YC and DRAP1, whereas the H2B subunit of core histones shares a common ancestry with NF-YB and DR1 class of TFs. Core histone subunits H3 and H4 were found to have evolved into DPE and TAF proteins, respectively. Taken together these results provide insights into diversification events during the evolution of the HFM, including sub-functionalization and neo-functionalization of the HFM.

Deciphering the Subtype Differentiation History of SARS-CoV-2 Based on a New Breadth-First Searching Optimized Alignment Method Over a Global Data Set of 24,768 Sequences.

SARS-CoV-2 has caused a worldwide pandemic. Existing research on coronavirus mutations is based on small data sets, and multiple sequence alignment using a global-scale data set has yet to be conducted. Statistical analysis of integral mutations and global spread are necessary and could help improve primer design for nucleic acid diagnosis and vaccine development. Here, we optimized multiple sequence alignment using a conserved sequence search algorithm to align 24,768 sequences from the GISAID data set. A phylogenetic tree was constructed using the maximum likelihood (ML) method. Coronavirus subtypes were analyzed via t-SNE clustering. We performed haplotype network analysis and t-SNE clustering to analyze the coronavirus origin and spread. Overall, we identified 33 sense, 17 nonsense, 79 amino acid loss, and 4 amino acid insertion mutations in full-length open reading frames. Phylogenetic trees were successfully constructed and samples clustered into subtypes. The COVID-19 pandemic differed among countries and continents. Samples from the United States and western Europe were more diverse, and those from China and Asia mainly contained specific subtypes. Clades G/GH/GR are more likely to be the origin clades of SARS-CoV-2 compared with clades S/L/V. Conserved sequence searches can be used to segment long sequences, making large-scale multisequence alignment possible, facilitating more comprehensive gene mutation analysis. Mutation analysis of the SARS-CoV-2 can inform primer design for nucleic acid diagnosis to improve virus detection efficiency. In addition, research into the characteristics of viral spread and relationships among geographic regions can help formulate health policies and reduce the increase of imported cases.

An effective coded exposure photography framework using optimal fluttering pattern generation

Motion blurring often occurs during camera imaging process due to the relative motion between the scene and imaging sensor, which would degrade the quality of captured image. In this paper, an effective coded exposure photography framework is proposed to help obtain better motion deblurring results. An integrated optimal fluttering pattern generation criterion with several factors is designed to guarantee the limited frequency information loss using coded exposure photography. Furthermore, an effective fluttering code sequence search algorithm is put forward to obtain the optimal code sequence quickly with perfect balance consideration of global search and local search. The differential evolution algorithm is utilized to carry out effective global searching. Meanwhile, simulate annealing algorithm is introduce to implement supplementary local optimal search, which could help avoid falling into local optimizations and reduce the iteration numbers significantly. Various groups of synthetic and real-world deblurring experiments are carried out to verify the effectiveness of the designed imaging framework. Experimental results demonstrate that the optimal fluttering pattern could be calculated quickly by the presented approach and the deblurring images have superior subjective visual perception and objective image quality assessment performance.

Stretch Profile: A pruning technique to accelerate DNA sequence search

Abstract DNA sequence similarity search has been used by scientists to facilitate biological research. Over the years, more sequences are added to databases, making them constantly larger. Existing sequence search techniques usually focus on the improvement of sequence search algorithms to make the search faster, ignoring the possibility of reducing unrelated sequences from the search. This paper presents a pruning technique to accelerate DNA sequence search based on a novel Stretch Profile created from stretches of consecutive base characters: A-Stretch, C-Stretch, G-Stretch, and T-Stretch. The Stretch Profile is pre-generated for each sequence in a sequence database. During the search, the Stretch Profile of the query sequence is generated for comparison. The sequences in the database whose profiles do not match the Stretch Profile of the query sequence are excluded from the search, resulting in the reduction of search space, and consequently, search time. For evaluation, we compare sequence retrievals from the Greengenes database and processing time when using only BLAST and when using the proposed pruning technique with BLAST. The results show that the proposed pruning technique can reduce the search time by 30.43% up to 63.74% depending on the length of input query, while maintaining a sensitivity of 1.00 when compared to the result of the original BLAST search.

Signal-based Context Comparative Analysis for Identification of Similar Manufacturing Modules

Industry 4.0 connects different machines and their modules to each other. Integrating already existing non-modular machines and establishing the required modularization in such a scenario requires a lot of time-consuming analysis. But Industry 4.0 also allows previously unconnected machines to establish a comparative analysis between each other by comparing monitored results of new and old machines. This analysis allows finding behavior that overlaps between machines and allows to identify parts that can be encapsulated in new or substituted by already known modules. In this paper, we propose a method to identify those overlapping parts by exploiting learned behavior models during runtime and combining production paths with sequence search algorithms by using context information of observable event signals.

The HMMER Web Server for Protein Sequence Similarity Search.

Protein sequence similarity search is one of the most commonly used bioinformatics methods for identifying evolutionarily related proteins. In general, sequences that are evolutionarily related share some degree of similarity, and sequence-search algorithms use this principle to identify homologs. The requirement for a fast and sensitive sequence search method led to the development of the HMMER software, which in the latest version (v3.1) uses a combination of sophisticated acceleration heuristics and mathematical and computational optimizations to enable the use of profile hidden Markov models (HMMs) for sequence analysis. The HMMER Web server provides a common platform by linking the HMMER algorithms to databases, thereby enabling the search for homologs, as well as providing sequence and functional annotation by linking external databases. This unit describes three basic protocols and two alternate protocols that explain how to use the HMMER Web server using various input formats and user defined parameters. © 2017 by John Wiley & Sons, Inc.

Improving the Mapping of Smith-Waterman Sequence Database Searches onto CUDA-Enabled GPUs.

Sequence alignment lies at heart of the bioinformatics. The Smith-Waterman algorithm is one of the key sequence search algorithms and has gained popularity due to improved implementations and rapidly increasing compute power. Recently, the Smith-Waterman algorithm has been successfully mapped onto the emerging general-purpose graphics processing units (GPUs). In this paper, we focused on how to improve the mapping, especially for short query sequences, by better usage of shared memory. We performed and evaluated the proposed method on two different platforms (Tesla C1060 and Tesla K20) and compared it with two classic methods in CUDASW++. Further, the performance on different numbers of threads and blocks has been analyzed. The results showed that the proposed method significantly improves Smith-Waterman algorithm on CUDA-enabled GPUs in proper allocation of block and thread numbers.

Short pulse groups searching based on isomorphic sequences

Conventional pulse sorting algorithms have difficulty in extracting short pulse groups which are dispersed in massive pulse streams. An isomorphic sequence search algorithm, based on a suffix array and the longest common prefix, is used to extract short pulse repetition interval-repeated pulse groups.

Approaches for algebraic Bayesian networks primary structure cyclicity elimination

One of the conditions for the effectiveness of the algorithms of logical and probabilistic inference in algebraic Bayesian network (ABN) is an acyclicity of its graphical representation. Introduction of hypergraphic representation of ABN structures allowed applying the methods of converting this graph to an acyclic form, basing on the methods of the theory of tree decomposition. The general scheme of the method of converting the network to acyclic one with the help of elimination sequences is considered. The main classes of heuristic elimination sequence search algorithms which are appropriate in the context of the transformation of the ABN, as well as evaluation of their complexity and quality of the results are presented.

A Gram-Based String Paradigm for Efficient Video Subsequence Search

The unprecedented increase in the generation and dissemination of video data has created an urgent demand for the large-scale video content management system to quickly retrieve videos of users' interests. Traditionally, video sequence data are managed by high-dimensional indexing structures, most of which suffer from the well-known “curse of dimensionality” and lack of support of subsequence retrieval. Inspired by the high efficiency of string indexing methods, in this paper, we present a string paradigm called VideoGram for large-scale video sequence indexing to achieve fast similarity search. In VideoGram, the feature space is modeled as a set of visual words. Each database video sequence is mapped into a string. A gram-based indexing structure is then built to tackle the effect of the “curse of dimensionality” and support video subsequence matching. Given a high-dimensional query video sequence, retrieval is performed by transforming the query into a string and then searching the matched strings from the index structure. By doing so, expensive high-dimensional similarity computations can be completely avoided. An efficient sequence search algorithm with upper bound pruning power is also presented. We conduct an extensive performance study on real-life video collections to validate the novelties of our proposal.

Improved performance of sequence search algorithms in remote homology detection

The protein sequence space is vast and diverse, spanning across different families. Biologically meaningful relationships exist between proteins at superfamily level. However, it is highly challenging to establish convincing relationships at the superfamily level by means of simple sequence searches. It is necessary to design a rigorous sequence search strategy to establish remote homology relationships and achieve high coverage. We have used iterative profile-based methods, along with constraints of sequence motifs, to specify search directions. We address the importance of multiple start points (queries) to achieve high coverage at protein superfamily level. We have devised strategies to employ a structural regime to search sequence space with good specificity and sensitivity. We employ two well-known sequence search methods, PSI-BLAST and PHI-BLAST, with multiple queries and multiple patterns to enhance homologue identification at the structural superfamily level. The study suggests that multiple queries improve sensitivity, while a pattern-constrained iterative sequence search becomes stringent at the initial stages, thereby driving the search in a specific direction and also achieves high coverage. This data mining approach has been applied to the entire structural superfamily database.

A FAST LOCAL SEARCH ALGORITHM USING HISTOGRAM FEATURES FOR DNA SEQUENCE DATABASE

DNA sequence search is a very important topic in bioinformatics algorithm development. However, this task usually spends much computational time to search on large DNA sequence database. In this paper, we propose an efficient hierarchical DNA sequence search algorithm to improve the search speed while the accuracy is being kept constant. For a given query DNA sequence, firstly, a fast local search algorithm using histogram features is used as a filtering mechanism before scanning the sequences in the database. An overlapping processing is newly added to improve the robustness of the algorithm. A large number of DNA sequences with low similarity will be excluded for latter searching. The Smith-Waterman algorithm is then applied to each remainder sequences. Experimental results using GenBank se- quence data show the proposed algorithm combining histogram information and Smith-Waterman algorithm is more efficient for DNA se- quence search. Keywords- Fast search, DNA sequence, Histogram feature, Smith-Waterman algorithm, Local search.

Search for identical octapeptides in unrelated proteins: Structural plasticity revisited

Since proteins are dynamic in nature, they can alter their local structure in response to changes in their environment factors such as temperature, pH, phosphorylation, and binding of other small molecules. These conformational changes are extremely important for the correct folding and functioning of proteins. There are also a number of diseases associated with protein conformational change such as amyloid diseases. To stimulate research into the above factors which specify one conformation over another, different theoretical models have been proposed and tested against sequence similar distant structure protein fragments. In order to simplify the computational complexity of identifying conformational changes in proteins, various local sequence search algorithms were employed and the structural plasticity in unrelated proteins was examined by various research groups. In the present work, we revisit the mechanism of structural plasticity in unrelated proteins with increased number of structures in Protein Data Bank by comparing identical octapeptides in unrelated proteins with dictionary of protein secondary structure extracted from existing experimental data. Our goal is to bring out the influence of hydrophobic residues, hydrophilic residues, flanking residues, difference in secondary structural propensities of surrounding residues, difference in phi-psi angles and local and nonlocal interactions in identical octapeptides adopting different conformations. Also we have used surrounding hydrophobicity, environment dependent interaction energy, atomic mean force potential, structural unit contacts and difference profiles models to explore the factors which cause structural plasticity. The results discussed here may provide insights into protein folding, design and function.