Edit Distance Research Articles

Improving SpellChecking: an effective Ad-Hoc probabilistic lexical measure for general typos

<p>Since the era of learning to write by human beings, mistakes made in typing words have occupied a privileged place in linguistic studies, integrating new disciplines into school curricula such as spelling and dictation. According to exhaustive studies that we have done in the field of spellchecking errors made in typing Arabic texts, very few research works that deal with typographical errors specifically caused by the insertion or missing of the blank-space in words. On the other hand, spelling correction software remains ineffective for handling this type of errors. Failure to process errors due to the insertion/missing of blankspace between and in words leads and brings us back to situations of ambiguity and incomprehension of the meaning of the typed text. To remedy this limitation of correction, we propose in this article an ad-hoc probabilistic method which is based jointly on two approaches. The first approach treats the errors due to deletion or missing of blank-space between or inside words, while the second puts emphasis in correcting space insertion errors in a word of course in addition to other kinds of elementary editing errors (addition, deletion, permutation of characters). Our new approach combines edit distance with n-gram language models to correct the errors already mentioned. Our new approach gave an accuracy rate that reaches 98,14% for missing blank-space errors (noted MBSE) and 89,5% for insertion blank-dpace errors (noted IBSE), which gives an average correction rate of around 95,26%. These results are very encouraging and show the interest and the importance of our approach.</p>

The effect of genome graph expressiveness on the discrepancy between genome graph distance and string set distance.

MotivationIntra-sample heterogeneity describes the phenomenon where a genomic sample contains a diverse set of genomic sequences. In practice, the true string sets in a sample are often unknown due to limitations in sequencing technology. In order to compare heterogeneous samples, genome graphs can be used to represent such sets of strings. However, a genome graph is generally able to represent a string set universe that contains multiple sets of strings in addition to the true string set. This difference between genome graphs and string sets is not well characterized. As a result, a distance metric between genome graphs may not match the distance between true string sets.ResultsWe extend a genome graph distance metric, Graph Traversal Edit Distance (GTED) proposed by Ebrahimpour Boroojeny et al., to FGTED to model the distance between heterogeneous string sets and show that GTED and FGTED always underestimate the Earth Mover’s Edit Distance (EMED) between string sets. We introduce the notion of string set universe diameter of a genome graph. Using the diameter, we are able to upper-bound the deviation of FGTED from EMED and to improve FGTED so that it reduces the average error in empirically estimating the similarity between true string sets. On simulated T-cell receptor sequences and actual Hepatitis B virus genomes, we show that the diameter-corrected FGTED reduces the average deviation of the estimated distance from the true string set distances by more than 250%.Availability and implementationData and source code for reproducing the experiments are available at: https://github.com/Kingsford-Group/gtedemedtest/.Supplementary information Supplementary data are available at Bioinformatics online.

Information Support for Personalities Socialization Processes Based on Common Interests

The main objective of this article is to create an information system project for socialization by personal interests on the basis of SEO-technologies and methods of machine learning. The main purpose of this information system is to identify the user within the system using neural networks and to select similar users by analysing the user's current information. An information system was created that, through Identity and JWT tokens, provides optimized and secure authorization, logging, and support functions for the current system user session. Finding a face in a user's photo and checking the presence of a similar user in the database are implemented using convolutional and Siamese neural networks. The analysis and formation of similar user beeps were implemented using fuzzy search algorithms, the Levenshtein algorithm and the Noisy Channel model, which made it possible to maximize the automation of the user selection process and to optimize the time spent in this process. Tools have also been created to view other users’ profiles, preferences and private correspondence. All private correspondence and information about it are stored in the current database. Each user of the system can view all information about sent and received messages. The created information system implements the process of user identification, analysis, selection and further socialization of system users.

Automatic English Pronunciation Evaluation Algorithm Based on Sequence Matching and Feature Fusion

This article focuses on the question answering type of automatic scoring system for large-scale spoken English examinations and scores using a method called multifeature fusion. Three types of features are extracted to score using speech recognition text as the research object. The three types of features are similarity, syntactic, and phonetic. There are nine distinct characteristics that describe the relationship between examinee responses and expert ratings. Manhattan distance is improved as a measure of similarity in the similarity feature. Simultaneously, a feature of keyword coverage based on editing distance is proposed, and the phenomenon of word variation in text recognition is fully considered, in order to provide examinees with an objective and fair score. To obtain the machine score, all extracted features were fused using a multiple linear regression model. The experimental results demonstrate that the extracted features are extremely effective for machine scoring, with the system scoring performance of the examinee as a unit equaling 98.4 percent of expert scoring performance.

Solar activity facilitates daily forecasts of large earthquakes.

Despite the extensive literature related to earthquakes, an effective method to forecast and avoid occasional seismic hazards that cause substantial damage is lacking. The Sun has recently been identified as a potential precursor to earthquakes, although no causal relationship between its activity and the Earth's seismicity has been established. This study was aimed at investigating whether such a relationship exists and whether it can be used to improve earthquake forecasting. The edit distances between earthquake point processes were combined with delay-coordinate distances for sunspot numbers. The comparison of these two indicated the existence of unidirectional causal coupling from solar activity to seismicity on Earth, and a radial basis function regressor showed accuracy improvements in the largest magnitude prediction of next days by 2.6%-17.9% in the odds ratio when sunspot distances were included.

Branch Decomposition‐Independent Edit Distances for Merge Trees

AbstractEdit distances between merge trees of scalar fields have many applications in scientific visualization, such as ensemble analysis, feature tracking or symmetry detection. In this paper, we propose branch mappings, a novel approach to the construction of edit mappings for merge trees. Classic edit mappings match nodes or edges of two trees onto each other, and therefore have to either rely on branch decompositions of both trees or have to use auxiliary node properties to determine a matching. In contrast, branch mappings employ branch properties instead of node similarity information, and are independent of predetermined branch decompositions. Especially for topological features, which are typically based on branch properties, this allows a more intuitive distance measure which is also less susceptible to instabilities from small‐scale perturbations. For trees with 𝒪(n) nodes, we describe an 𝒪(n4) algorithm for computing optimal branch mappings, which is faster than the only other branch decomposition‐independent method in the literature by more than a linear factor. Furthermore, we compare the results of our method on synthetic and real‐world examples to demonstrate its practicality and utility.

FLCSS: A fuzzy‐based longest common subsequence method for uncertainty management in trajectory similarity measures

AbstractThe large quantity of movement data collected from various sources can be inherently uncertain and heterogeneous. In the movement data analysis and mining spectrum, computing the similarity of trajectories while considering the uncertainty and heterogeneity has been less addressed. Generally, two factors of sampling and positioning error cause uncertainty in trajectory databases. Therefore, in this research, a method based on the longest common subsequence (LCSS), named FLCSS, is proposed that uses fuzzy theory and the bead model to consider the uncertainty of trajectories originated from positioning and sampling errors. The performance of FLCSS is evaluated by implementations on real and synthetic datasets, and compared with six important and commonly used similarity measurement methods, namely, LCSS, edit distance on real sequence (EDR), dynamic time warping (DTW), edit distance with real penalty (ERP), Hausdorff distance (HD), and Fréchet distance (FD). The results show that FLCSS has a better performance compared to other methods, in terms of sensitivity to point displacement, noise, and different sampling rates. Furthermore, the high correlation between FLCSS and LCSS (ρ = 0.91) confirms the robustness of the proposed method in considering uncertainty in the trajectory databases.

Automatic construction of real‐world‐based typing‐error test dataset

In this study, we aim to automatically construct a test dataset for testing the performance of spelling error correction systems. The Google Web 1T corpus, which includes data on 10 quadrillion phrases, is used for this purpose. Therefore, error words used in the test dataset use error words generated by real web users. There are seven types of error words. In order to obtain the error word, a word set that appears simultaneously with the surrounding context (3-g range) of the location of the error word generation is searched. In this calculation, we exclude error words with wide edit distances that cause the resolution of original words to become exceedingly difficult. In order to select the final error word from the word set, a word with a high value is selected by calculating the context probability using 3-g. In the experiment, the performance was measured for two systems (grammarly, MS Word) in service and the recently announced spelling error correction system (Neuspell). The highest performance was the F1 score of 56%, which shows the overall performance, indicating the need for research on spelling errors.

A Faster Reduction of the Dynamic Time Warping Distance to the Longest Increasing Subsequence Length

The similarity between a pair of time series, i.e., sequences of indexed values in time order, is often estimated by the dynamic time warping (DTW) distance, instead of any in the well-studied family of measures including the longest common subsequence (LCS) length and the edit distance. Although it may seem as if the DTW and the LCS(-like) measures are essentially different, we reveal that the DTW distance can be represented by the longest increasing subsequence (LIS) length of a sequence of integers, which is the LCS length between the integer sequence and itself sorted. For a given pair of time series of length n such that the dissimilarity between any elements is an integer between zero and c, we propose an integer sequence that represents any substring-substring DTW distance as its band-substring LIS length. The length of the produced integer sequence is O(c n^2), which can be translated to O(n^2) for constant dissimilarity functions. To demonstrate that techniques developed under the LCS(-like) measures are directly applicable to analysis of time series via our reduction of DTW to LIS, we present time-efficient algorithms for DTW-related problems utilizing the semi-local sequence comparison technique developed for LCS-related problems.

Content-Based Image Retrieval for Songket Motifs using Graph Matching

Indonesia is a country that has abundant cultural wealth. One of the characteristics of Indonesian culture is Songket. Songket is a typical Malay woven cloth that has many variants of motifs, each of which represents a different meaning and philosophy. Songket is often found in Sumatra Island with different motifs in each region. With so many types of songket motifs, not everyone can recognize and distinguish between one songket motif and another, even Indonesian citizens themselves. With the help of computers, it is easier to find information about a songket motif or to find a similar songket motif. The field that can play a role in solving this problem is Content-Based Image Retrieval (CBIR). This study aims to carry out a content retrieval process on the songket core motif using graph matching-based processing. In this study, the method used is felzenzswalb segmentation, and graph matching through the VF2 isomorphism algorithm and graph edit distance. The number of songket core motif images used as data is 180 data in the form of color images measuring 64 x 64 pixels. Based on the results of the study, it was found that the optimal graph matching algorithm and parameters in this study were the VF2 algorithm for artificial images with an f-1 score of 91.05%, and Graph Edit Distance with GED≤8 parameters for songket motif images with an f1-score. by 53.36%.

Accurate and Language Agnostic Code Clone Detection by Measuring Edit Distance of ANTLR Parse Tree

In spite of significant research done in the past 3 decades introducing more than 250 clone detection tools/ techniques for finding the same language clones, there exists no single framework to detect and classify all 4 basic types of clones with great accuracy (precision and recall). In this paper, we propose an accurate and language agnostic technique to classify 4 types of clones. The method first generates an ANTLR parse tree for the input program file using freely available ANTLR grammar files then finds the edit distance between the two parse trees using the Levenshtein distance algorithm and converts the edit distance into similarity using. We obtained 100% precision and recall in detecting type 1 & 2 clone types and achieve 98.50 and 98.12 respectively for type 3 and 4 clone types for our datasets containing microprograms of C, CPP, and Java. This paper provides evidence that the Levenshtein distance on ANTLR parse tree is the good choice to build a complete and accurate software clone detector and act as proper validation tools to detect code plagiarism.

Approximating the Geometric Edit Distance

Edit distance is a measurement of similarity between two sequences such as strings, point sequences, or polygonal curves. Many matching problems from a variety of areas, such as signal analysis, bioinformatics, etc., need to be solved in a geometric space. Therefore, the geometric edit distance (GED) has been studied. In this paper, we describe the first strictly sublinear approximate near-linear time algorithm for computing the GED of two point sequences in constant dimensional Euclidean space. Specifically, we present a randomized \(O(n\log ^2n)\) time \(O(\sqrt{n})\)-approximation algorithm. Then, we generalize our result to give a randomized \(\alpha \)-approximation algorithm for any \(\alpha \in [\sqrt{\log n}, \sqrt{n/\log n}]\), running in time \(O(n^2/\alpha ^2 \log n)\). Both algorithms are Monte Carlo and return approximately optimal solutions with high probability.

A Contrastive Learning Pre-Training Method for Motif Occupancy Identification.

Motif occupancy identification is a binary classification task predicting the binding of DNA motif instances to transcription factors, for which several sequence-based methods have been proposed. However, through direct training, these end-to-end methods are lack of biological interpretability within their sequence representations. In this work, we propose a contrastive learning method to pre-train interpretable and robust DNA encoding for motif occupancy identification. We construct two alternative models to pre-train DNA sequential encoder, respectively: a self-supervised model and a supervised model. We augment the original sequences for contrastive learning with edit operations defined in edit distance. Specifically, we propose a sequence similarity criterion based on the Needleman–Wunsch algorithm to discriminate positive and negative sample pairs in self-supervised learning. Finally, a DNN classifier is fine-tuned along with the pre-trained encoder to predict the results of motif occupancy identification. Both proposed contrastive learning models outperform the baseline end-to-end CNN model and SimCLR method, reaching AUC of 0.811 and 0.823, respectively. Compared with the baseline method, our models show better robustness for small samples. Specifically, the self-supervised model is proved to be practicable in transfer learning.

A Linear Time Solution to the Labeled Robinson-Foulds Distance Problem.

A large variety of pairwise measures of similarity or dissimilarity have been developed for comparing phylogenetic trees, for example, species trees or gene trees. Due to its intuitive definition in terms of tree clades and bipartitions and its computational efficiency, the Robinson–Foulds (RF) distance is the most widely used for trees with unweighted edges and labels restricted to leaves (representing the genetic elements being compared). However, in the case of gene trees, an important information revealing the nature of the homologous relation between gene pairs (orthologs, paralogs, and xenologs) is the type of event associated to each internal node of the tree, typically speciations or duplications, but other types of events may also be considered, such as horizontal gene transfers. This labeling of internal nodes is usually inferred from a gene tree/species tree reconciliation method. Here, we address the problem of comparing such event-labeled trees. The problem differs from the classical problem of comparing uniformly labeled trees (all labels belonging to the same alphabet) that may be done using the Tree Edit Distance (TED) mainly due to the fact that, in our case, two different alphabets are considered for the leaves and internal nodes of the tree, and leaves are not affected by edit operations. We propose an extension of the RF distance to event-labeled trees, based on edit operations comparable to those considered for TED: node insertion, node deletion, and label substitution. We show that this new Labeled Robinson–Foulds (LRF) distance can be computed in linear time, in addition of maintaining other desirable properties: being a metric, reducing to RF for trees with no labels on internal nodes and maintaining an intuitive interpretation. The algorithm for computing the LRF distance enables novel analyses on event-label trees such as reconciled gene trees. Here, we use it to study the impact of taxon sampling on labeled gene tree inference and conclude that denser taxon sampling yields trees with better topology but worse labeling. [Algorithms; combinatorics; gene trees; phylogenetics; Robinson–Foulds; tree distance.]

Using open surgery simulation kinematic data for tool and gesture recognition.

The use of motion sensors is emerging as a means for measuring surgical performance. Motion sensors are typically used for calculating performance metrics and assessing skill. The aim of this study was to identify surgical gestures and tools used during an open surgery suturing simulation based on motion sensor data. Twenty-five participants performed a suturing task on a variable tissue simulator. Electromagnetic motion sensors were used to measure their performance. The current study compares GRU and LSTM networks, which are known to perform well on other kinematic datasets, as well as MS-TCN++, which was developed for video data and was adapted in this work for motion sensors data. Finally, we extended all architectures for multi-tasking. In the gesture recognition task the MS-TCN++ has the highest performance with accuracy of [Formula: see text] and F1-Macro of [Formula: see text], edit distance of [Formula: see text] and F1@10 of [Formula: see text] In the tool usage recognition task for the right hand, MS-TCN++ performs the best in most metrics with an accuracy score of [Formula: see text], F1-Macro of [Formula: see text], F1@10 of [Formula: see text], and F1@25 of [Formula: see text]. The multi-task GRU performs best in all metrics in the left-hand case, with an accuracy of [Formula: see text], edit distance of [Formula: see text], F1-Macro of [Formula: see text], F1@10 of [Formula: see text], and F1@25 of [Formula: see text]. In this study, using motion sensor data, we automatically identified the surgical gestures and the tools used during an open surgery suturing simulation. Our methods may be used for computing more detailed performance metrics and assisting in automatic workflow analysis. MS-TCN++ performed better in gesture recognition as well as right-hand tool recognition, while the multi-task GRU provided better results in the left-hand case. It should be noted that our multi-task GRU network is significantly smaller and has achieved competitive results in the rest of the tasks as well.

Visualization of Topological Pharmacophore Space with Graph Edit Distance.

A topological pharmacophore (TP) is a chemical graph-based pharmacophore representation, where nodes are pharmacophoric features (PF) and edges are topological distances between PFs. Previously proposed sparse pharmacophore graphs (SPhGs) for TPs were shown to be effective in identifying structurally different active compounds while maintaining the interpretability of the graphs. However, one limitation of using SPhGs as queries is that many structurally similar SPhGs can be identified from a set of active compounds, requiring the classification and visualization of SPhGs, followed by an understanding of the pharmacophore hypotheses. In this study, we propose a scheme for SPhG analysis based on dimensionality reduction techniques with the graph edit distance (GED) metric. This metric enables measuring similarities among SPhGs in a quantitative manner. The visualization of SPhGs, which themselves are the graphs shared by active compounds, can help us understand the pharmacophore hypotheses as well as the data set. As a proof-of-concept study, we generated two-dimensional SPhG-maps using three dimensionality reduction techniques for six biological targets. A comparison with other pharmacophore representations was also conducted. We demonstrated knowledge extraction (interpretation of the data set) from the generated maps. Our findings include a suitable mapping algorithm as well as a pharmacophore hypothesis analysis procedure using an SPhG-map.

Edit Distance and Persistence Diagrams over Lattices

We build a functorial pipeline for persistent homology. The input to this pipeline is a filtered simplicial complex indexed by any finite metric lattice, and the output is a persistence diagram defined as the Möbius inversion of its birth-death function. We adapt the Reeb graph edit distance to each of our categories and prove that both functors in our pipeline are 1-Lipschitz, making our pipeline stable. Our constructions generalize the classical persistence diagram, and in this setting, the bottleneck distance is strongly equivalent to the edit distance.

An efficient multikeyword fuzzy ciphertext retrieval scheme based on distributed transmission for Internet of Things

As traditional computing and cloud computing integrate, the Internet of Things (IoT) has evolved into a layered and cloud-network-edge-end architecture. However, most searchable encryption models still use triples, in which hierarchical structures are neglected, and insecure intermediate nodes are exposed to external environment. Meanwhile, mainstream schemes adopting accurate retrieval are incompatible with IoT end users' features of differentiation. To address these issues, we innovatively design an efficient and credible search model with an accurate multikeyword fuzzy ciphertext retrieval scheme in the context of IoT. First, based on network coding and key sharing, data are grouped, encoded, and transmitted in parallel to the receiver node through middle-layer nodes, with high efficiency and reliability. Second, to realize fuzzy retrieval of IoT, edit distance is selected as the standard of difference between keywords, and then document index vector and query vector are created based on locality sensitive hashing (LSH) and Bloom Filter. Furthermore, to improve the traditional scheme, query keywords are split into multiple single-word forms, inner products between each trapdoor of single word and encryption index vector are calculated, respectively, for the sum of each inner product and thus top $\mathrm{top}$ - k $k$ sorting search. Ultimately, feasibility, safety, and efficiency of our improved scheme are verified by security analysis, while simulation results support that our scheme has better accuracy and efficiency.

Input-driven pushdown automata for edit distance neighborhood

Edit distance ℓ-neighborhood of a language L is the set of all strings that can be obtained by at most ℓ elementary edit operations—deleting or inserting one symbol in the string—from some string in L. We shall show that if L is recognized by a nondeterministic input-driven pushdown automaton (pda) with |Γ| pushdown symbols and |Q| states, then its edit distance ℓ-neighborhood can be recognized by a nondeterministic input-driven pda with 2⋅|Γ|+1 pushdown symbols and O(|Q|⋅|Γ|ℓ) states. We shall also prove the corresponding lower bound: if |Q|>|Γ|≥2, there exists a language L that is recognized by a deterministic input-driven pda using |Γ| pushdown symbols and |Q| states but any nondeterministic input-driven pda for its edit distance ℓ-neighborhood must use at least (|Q|−1)⋅|Γ|ℓ states. This improves the known upper bound from literature and gives the bound that is asymptotically the best possible. If, beside deleting or inserting one symbol in the string, the measure of edit distance includes the operation of rewriting one symbol with another, the edit distance ℓ-neighborhood can be recognized with 2⋅|Γ|+1 pushdown symbols and O(|Q|⋅|Γ|2⋅ℓ) states.

A novel kernel based approach to arbitrary length symbolic data with application to type 2 diabetes risk

Predictive modeling of clinical data is fraught with challenges arising from the manner in which events are recorded. Patients typically fall ill at irregular intervals and experience dissimilar intervention trajectories. This results in irregularly sampled and uneven length data which poses a problem for standard multivariate tools. The alternative of feature extraction into equal-length vectors via methods like Bag-of-Words (BoW) potentially discards useful information. We propose an approach based on a kernel framework in which data is maintained in its native form: discrete sequences of symbols. Kernel functions derived from the edit distance between pairs of sequences may then be utilized in conjunction with support vector machines to classify the data. Our method is evaluated in the context of the prediction task of determining patients likely to develop type 2 diabetes following an earlier episode of elevated blood pressure of 130/80 mmHg. Kernels combined via multi kernel learning achieved an F1-score of 0.96, outperforming classification with SVM 0.63, logistic regression 0.63, Long Short Term Memory 0.61 and Multi-Layer Perceptron 0.54 applied to a BoW representation of the data. We achieved an F1-score of 0.97 on MKL on external dataset. The proposed approach is consequently able to overcome limitations associated with feature-based classification in the context of clinical data.