Levenshtein Distance Algorithm Research Articles

3GOLD: optimized Levenshtein distance for clustering third-generation sequencing data

BackgroundThird-generation sequencing offers some advantages over next-generation sequencing predecessors, but with the caveat of harboring a much higher error rate. Clustering-related sequences is an essential task in modern biology. To accurately cluster sequences rich in errors, error type and frequency need to be accounted for. Levenshtein distance is a well-established mathematical algorithm for measuring the edit distance between words and can specifically weight insertions, deletions and substitutions. However, there are drawbacks to using Levenshtein distance in a biological context and hence has rarely been used for this purpose. We present novel modifications to the Levenshtein distance algorithm to optimize it for clustering error-rich biological sequencing data.ResultsWe successfully introduced a bidirectional frameshift allowance with end-user determined accommodation caps combined with weighted error discrimination. Furthermore, our modifications dramatically improved the computational speed of Levenstein distance. For simulated ONT MinION and PacBio Sequel datasets, the average clustering sensitivity for 3GOLD was 41.45% (S.D. 10.39) higher than Sequence-Levenstein distance, 52.14% (S.D. 9.43) higher than Levenshtein distance, 55.93% (S.D. 8.67) higher than Starcode, 42.68% (S.D. 8.09) higher than CD-HIT-EST and 61.49% (S.D. 7.81) higher than DNACLUST. For biological ONT MinION data, 3GOLD clustering sensitivity was 27.99% higher than Sequence-Levenstein distance, 52.76% higher than Levenshtein distance, 56.39% higher than Starcode, 48% higher than CD-HIT-EST and 70.4% higher than DNACLUST.ConclusionOur modifications to Levenshtein distance have improved its speed and accuracy compared to the classic Levenshtein distance, Sequence-Levenshtein distance and other commonly used clustering approaches on simulated and biological third-generation sequenced datasets. Our clustering approach is appropriate for datasets of unknown cluster centroids, such as those generated with unique molecular identifiers as well as known centroids such as barcoded datasets. A strength of our approach is high accuracy in resolving small clusters and mitigating the number of singletons.

Mitigation Impact of Energy and Time Delay for Computation Offloading in an Industrial IoT Environment Using Levenshtein Distance Algorithm

Due to the explosive growth of the Internet of things (IoT) devices and the emergence of diverse new applications, network traffic volume is growing exponentially. The traditional centralized network architecture cannot fulfill IoT devices demand because of the heavy network traffic in industrial IoT. Moreover, IoT devices have limited computational ability and battery power. Energy consumption and time delay problems during computation offloading are fundamental issues. A new architecture known as mobile edge computing (MEC) was introduced to overcome these issues, which brings cloud services and its contents to the edge of the network. IoT devices can offload the data for computation to the cloud server or edge nodes. Different schemes have been proposed to overcome this problem under many scenarios (i.e., single-user, multiuser, and vehicular networks). In this paper, we proposed a modified delay mitigation Levenshtein distance algorithm (MDML). We consider an industrial scenario with multiple IoT devices and multiple servers (edge nodes). Each edge node consists of one MEC server. The proposed algorithm solves the offloading optimization problem of energy and mitigation of time delay with much lower complexity while significantly reducing offloading tasks’ execution time. It works on the basis of dynamic programming, where we break down a complex problem into subproblems. Performance evaluation of our proposed algorithm shows that it can achieve satisfactory energy efficiency and mitigate time delay in the industrial IoT environment.

Topical Fluoride Applications Related Posts Analysis on Twitter Using Natural Language Processing.

Social media is today a comprehensive source of data that can serve as a guide to professionals in issues related to public health. The purpose of this paper is to investigate the content of topical fluoride-related Twitter posts made over a 3-year period in order to improve our understanding of Twitter users' perceptions and treatment experiences. A continuous cross-sectional sample of Tweets on the subject of 'approaches to the topical fluoride treatment of tooth decay' was collected from the Twitter social networking platform between 1 January 2017 and 1 January 2020 using a software application developed for this research that makes use of the Twitter advanced search API. The words and phrases used for the identification of related Tweets were determined through a screening of the topical fluoride keywords of previous studies, and a search was conducted in the English language. To better arrange the collected Tweets and to make the data more meaningful, firstly one of the natural language process techniques - Tokenization - was applied, after which the Tweets were converted into a set of meaningful words and regular expressions. The Tweets were then compared with each other, word-by-word, with the help of a word-based Levenshtein distance algorithm, after which two experts in the computational social science domain labelled each Tweet. A total of 132,358 Tweeter posts referencing topical fluoride applications were collected, of which 110,847 were eliminated through the use of a word-based Levenshtein distance algorithm, and the remaining corpus of 21,511 posts was analysed and evaluated for specific content. Within the garnered data, 48.5% (n = 10,428) of the Tweeter posts concerned topical fluoride treatments, and 7% (n = 1,507) reported experiences with topical fluoride treatment. Negative Tweeter posts about topical fluoride treatment (5,679, 26.4%) vastly outnumbered those that were positive (3,897, 18.1%). The current study achieved its main objectives of analysing topical fluoride application-related posts made on social media. From the garnered Twitter data, it can be understood that Twitter users regularly share their concerns and negative sentiments about the side effects of topical fluoride treatments on the platform. Future explorations of social media may aid public health and dental professionals in the development of strategies to educate the public and to raise awareness of the importance of topical fluoride applications.

STEMMING BAHASA JAWA MENGGUNAKAN DAMERAU LEVENSHTEIN DISTANCE (DLD)

Stemming is one of the essential stages of text mining. This process removes prefixes and suffixes to produce root words in a text. This study uses a string matching algorithm, namely Damerau Levenshtein Distance (DLD), to find the basic word forms of Javanese. Test data of 300 words that have a prefix, insertion, suffix, a combination of prefix and suffix, and word repetition. The results of this study indicate that the Damerau Levenshtein Distance (DLD) algorithm can be used for Stemming Javanese text with an accuracy value of 49.6%.

Enhancing text classification performance by preprocessing misspelled words in Indonesian language

Supervised learning using shallow machine learning methods is still a popular method in processing text, despite the rapidly advancing sector of unsupervised methodologies using deep learning. Supervised text classification for application user feedback sentiments in Indonesian Language is one of the applications which is quite popular in both the research community and industry. However, due to the nature of shallow machine learning approaches, various text preprocessing techniques are required to clean the input data. This research aims to implement and evaluate the role of Levenshtein distance algorithm in detecting and preprocessing misspelled words in Indonesian language, before the text data is then used to train a user feedback sentiment classification model using multinomial Naïve Bayes. This research experimented with various evaluation scenarios, and found that preprocessing misspelled words in Indonesian language using the Levenshtein distance algorithm could be useful and showed a promising 8.2% increase on the accuracy of the model’s ability to classify user feedback text according to their sentiments.

Designing a word recommendation application using the Levenshtein Distance algorithm

Good scriptwriting or reporting requires a high level of accuracy. The basic problem is that the level of accuracy of the authors is not the same. The low level of accuracy allows for mistyping of words in a sentence. Typing errors caused the word to become non-standard. Even worse, the word became meaningless. In this case, the recommendation application serves to provide word-writing recommendations in case of a typing error. This application can reduce the error rate of the writer when typing. One method to improve word spelling is Approximate String Matching. This method applies an approach to the string search process. The Levenshtein Distance algorithm is a part of the Approximate String-Matching method. This method, firstly, is necessary to go through the preprocessing stage to correct an incorrectly written word using the Levenshtein Distance algorithm. The application testing phase uses ten texts composed of 100 words, ten texts composed of 100 to 250 words, and ten texts composed of 250 to 500 words. The average accuracy rate of these test results was 95%, 94%, and 90%.

Integrated modelling of automobile maintenance expert system based on knowledge graph

Using online information resources to build knowledge bases to provide knowledge answering services would help auto companies or third-party platforms to gain competitive advantages. Therefore, a construction plan of automobile maintenance expert system based on knowledge graph was proposed by integrated modelling. In terms of the entity recognition algorithm, the BM LSTM (Boyer-Moore Long Short-Term Memory) algorithm was proposed by integrating hidden Markov model, Conditional Random Field (CRF), Bi-directional Long Short-Term Memory (BiLSTM), BiLSTM-CRF and Lattice LSTM, which improved the accuracy index F1-score. In terms of the text quality evaluation algorithm, a secondary text quality evaluation system was designed. It evaluated the matching quality of the problem based on the word toolkit Synonyms and Levenshtein Distance algorithm. And it evaluated the quality of the answer text based on the TF-IDF (Term Frequency-Inverse Document Frequency) similarity algorithm and centered on completeness, accuracy, reliability, and argument strength. Finally, experiments are carried out on the proposed model and algorithm to prove its effectiveness.

Dig That Lick (DTL): Analyzing Large-Scale Data for Melodic Patterns in Jazz Performances

Dig That Lick (DTL): Analyzing Large-Scale Data for Melodic Patterns in Jazz Performances

Analyzing and Experimenting Open Source OCR Engines in RPA with Levenshtein Distance Algorithm

Robotic Process Automation is a platform used to automate boring and repetitive computer processes using software bots so that humans could involve in tasks which include creativity and decision making which could not be done by robots. Optical Character Recognition takes out printed characters in an image and converts it to text. Google Tesseract OCR and Microsoft OCR were the commonly used OCR engines available in UiPath, a tool for Robotic Process Automation. In Previous, research on comparing those two open source OCR engine, there we made comparison on basic factors which included speed, hardware requirements, accuracy ,but in that case, accuracy was been calculated manually which gave us results but with less precise, as it was a manual process to substitute scraped data to that formulas, In this research we’ve made results with more precision by performing a String comparison algorithm named, “Levenshtein Distance Algorithm” which is deployed in UiPath.

Hybrid Spelling Correction and Query Expansion for Relevance Document Searching

A digital library is a type of information retrieval (IR) system. The existing IR methodologies generally have problems on keyword searching. Some of search engine has not been able to provide search results with partial matching and typographical error. Therefore, it is required to be able to provide search results that are relevant to keywords provided by the user. We proposed a model to solve the problem by combining the spell correction and query expansion. Searching is starting with indexing the title of the document by preprocessing the title of all incoming document data and then weighting the Term Frequency – Inverse Document Frequency (TF-IDF) against all terms of the whole document. Levenshtein Distance algorithm is used in the search process to correct typo-indicated keywords. Before calculating the relevance between the keywords and the documents using Cosine Similarity, the keywords are expanded using Query Expansion to increase number of documents retrieved. Calculation results using Cosine Similarity are then added to Query Expansion weight calculation to get final ranking result. Results show improvements over IR system compared with system without spell check and query expansion. The results of the study in the form of web-based application conducted testing for 50 times with number of data of 2,045. The system was able to correct typo-indicated keywords and search documents with average recall value of 95.91%, average precision value of 63.82% and average Non Interpolated Average Precision (NIAP) value of 86.29%.

Spelling Checker using Algorithm Damerau Levenshtein Distance and Cosine Similarity

Writing is an embodiment of the author's ideas that are to be conveyed to others. A writer often experiences typos in typing the script, so that it can influence the meaning of the text. Therefore, a system is needed to detect word errors. In this study, checking is done by using the Dictionary Lookup method and giving the candidate words using the Damerau Levenshtein Distance algorithm. Candidates will then determine the ranking by breaking the word into Bigram form and calculating the similarity value using the Cosine Similarity algorithm. The test results based on the data used yield different Mean Reciprocal Rank (MRR) values for each type of error. The type of error deletion produces an MRR value of 88.89%, the type of insertion error produces an MRR value of 97.78%, the type of substitution error produces an MRR value of 88.89%, the type of transposition error produces an MRR value of 89%

Analisis Sentimen Kinerja KPU Pemilu 2019 Menggunakan Algoritma K-Means Dengan Algoritma Confix Stripping Stemmer

The performance of the KPU in the 2019 elections was a lively conversation between the general public and the political elite. Many people or parties commented on the process of calculating the results of the 2019 elections through social media. KPU's Facebook fan page is also busy being attacked by positive or negative comments. Sentiment analysis of public opinion can be investigated to find out how much percentage of a positive and negative sentiment of this policy through comments that have been sent to social-social media. Data analyzed were 200 data taken from Facebook KPU, divided into 150 training data and 50 test data. This study uses the K-Means algorithm with a value of k = 2 to determine the final sentiments of positive and negative, the Levenshtein Distance algorithm for word normalization and the Confix Stripping Stemmer algorithm in the stemming process. The results obtained from the public sentiment on the performance of the KPU are more negative than positive. The results of the accuracy obtained from the use of the K-Means algorithm are 84% with a lower accuracy value compared to the combination of the algorithm above, namely 86%. Suggestions for further research should use even more data and use the k-fold cross-validation accuracy calculation technique as a further trial

Sistem Pendeteksi Dini Plagiarisme Menggunakan Algoritma Levenshtein Distance

The academic world in Indonesia has growing rapidly which is marked by the development of science and technology. With all the facilities offered, technology has a positive and negative impact on life. One of negative impacts is plagiarism. The plagiarism often occurs among students, therefore detection of plagiarism needs to be done to avoid plagiarism. This reseacrh to detect the similarity of the text content of the document using the Levenshtein Distance algorithm. The type of document used is .pdf . The documents used are thesis proposals and publication papers. In the calculation speed test, the source document which count words 4405 with 3 comparative documents that have words 13465 count produces a calculation duration 3.57 seconds.

Autocorrect on Drugs e-Dictionary Search Module Using Levenshtein Distance Algorithm

The Dictionary of Medicine in the form of a physical book has many drawbacks, one of them is its thickness makes it impractical to be carried. This becomes a motivation to develop drug dictionary applications in the form of a Drugs e-Dictionary. One of the developed Drugs e-Dictionary uses A-Z index-based approach to discover any drug terms. This approach is less effective and less efficient timewise. Therefore, it is necessary to add a search function that has an autocorrect feature to aid the user. The purpose of this study is to build a search module that has an autocorrect feature on Drugs e-Dictionary using the Levenshtein Distance algorithm. The methodology or the stages of this research divided into the construction of a search module on Drugs e-Dictionary, implementation of the Levenshtein Distance algorithm, and autocorrect validation test. The results of the algorithm implementation show that the search module with the autocorrect feature can detect typing errors in the inputted terms by producing the closest drug term output in the database, then automatically provide suggestions for improvement and display the results of the improved drug terms to the user. it reaches 90% accuracy of inputted query, with 90% precision and 90% recall.

PERANCANGAN SISTEM PENDETEKSI BERITA HOAX MENGGUNAKAN ALGORITMA LEVENSHTEIN DISTANCE BERBASIS PHP

In the 4.0 era where the Internet is an important part of life today, information can be easily accessed anytime, anywhere. But not all information distributed through the internet is in the form of facts. Data presented by the Ministry of Communication and Information based on a survey conducted in 2018 said that as many as 800,000 sites in Indonesia indicated that non-fact or hoax news spreaders were indicated. As a result of hoax news generated is very dangerous because it attacks the minds of the human subconscious, so it is needed a system that can detect hoax news. In this study used a database containing hoax news documents. The algorithm applied is the TF-IDF algorithm to measure the weight of a word in a hoax document and combined with the Levenshtein Distance (LD) algorithm to measure the distance between words in a document. The application of the Levenshtein Distance Method in the Hoax Detection System has several stages that begin with the pre-processing of the word (prepocessing text) followed by the TF-IDF calculation phase and then the minimum distance calculation between words using the Levenshtein Distance algorithm. The result of a limit of 0.1 on 40 documents that have been classified as test data has high Precision, Recall and Accuracy values, namely Precision 1; Recall 0.71; and Accuracy 80%.

KWA: A New Method of Calculation and Representation Accuracy for Speech Keyword Spotting in String Results

This study proposes a new method to measure and represent accuracy for Keyword Spotting (KWS) problem in non-aligned string results. Our approach, called Keyword Spotting Accuracy (KWA), was improved from the Levenshtein Distance algorithm, that used to evaluate the accuracy of the keywords in KWS by measuring the minimum distance between two strings. The main improved algorithm is to show the status of each keyword in training phase for predicted and true labels. In which, representing which words are correct, which ones need to be inserted, substituted or deleted when comparing the prediction labels with true ones during the training phase. In addition, a new method of presenting the multiple keywords in results was proposed to indicate the accuracy of each keyword. This method can display detailed results by keywords, from which, we can obtain the accuracy, distribution, and balance of the keywords in the training dataset by actual speech variance, not by counting keywords in true labels as usual.

Similarity Based Information Retrieval Using Levenshtein Distance Algorithm

Sentence similarity plays an important role in many text-related research and applications such as information retrieval, information recommendation, natural language processing, machine translation and translation memory, and etc. Calculating similarity between sentences is the basis of measuring the similarity between texts which is the key to document classification and clustering. Sentence similarity partially depends on the word similarity. This system will display a similar text of field, areas and other facts in document retrieval. This paper uses a sentence matching method of the Levenshtein Distance algorithm. Thesimilarity between words can be calculated from the spelling of words or the meaning of words. Sentence similarity: The similarities between words in different sentences have a great influence on the similarity between two sentences. This system is retrieved from a similar sentence that included Theories, Methods and other facts in the document database.

Similarity detection based on document matrix model and edit distance algorithm

This paper presents a new algorithm with an objective of analyzing the similarity measure between two text documents. Specifically, the main idea of the implemented method is based on the structure of the so-called “edit distance matrix” (similarity matrix). Elements of this matrix are filled with a formula based on Levenshtein distances between sequences of sentences. The Levenshtein distance algorithm (LDA) is used as a replacement for various implementations of stemming or lemmatization methods. Additionally, the proposed algorithm is fast, precise, and may be implemented for analyzing very large documents (e.g., books, diploma works, newspapers, etc.). Moreover, it seems to be versatile for the most common European languages such as Polish, English, German, French and Russian. The presented tool is intended for all employees and students of the university to detect the level of similarity regarding analyzed documents. Results obtained in the paper were confirmed in the tests shown in the article.

Query Suggestion on Drugs e-Dictionary Using the Levenshtein Distance Algorithm

Dictionary of medicine in the form of a thick book has many disadvantages, one of which is impractical. This is the reason for Indonesian developers to create drugs e-Dictionary. But the drugs e-Dictionary that has been developed is still in the form of a letter index so that users must search the terms one by one in sequential order. This has become so inefficient and ineffective that it is necessary to add a search function and query suggestion feature to the drug e-dictionary. The purpose of this study is to build a query suggestion facility on drugs e-Dictionary using the Levenshtein Distance algorithm. The stages of this research consist of the Development of web-based drugs e-Dictionary, Implementation of the Levenshtein Distance Algorithm, Query Suggestion Testing, and Usage. Based on the results of the implementation of the Levenshtein Distance algorithm and test results, Drugs e-Dictionary can evaluate words that are not in the database. The query suggestion function works by producing the closest word output contained in the database.

Fuzzy string implementation matching on android-based encyclopedia and anatomy quiz

Encyclopedia anatomy is an android application to introduce anatomy, equipped with a search feature, multiple choice quizzes and guess image quizzes. Due to some anatomical term is uptake from foreign languages and frequently typing errors by a smartphone user, then search feature in this application implement fuzzy string matching method which uses Levenshtein distance algorithm. However, the results obtained are still not accurate, then added method of splitting and indexing, so the results are more accurate and search process fast enough. Due to more simple than a book and supported by a good appearance, this application can get a positive response from the users.