Difference Algorithm Research Articles

Confidence-Based Cheat Detection Through Constrained Order Inference of Temporal Sequences

For particular domains, duplication may be indicative of cheating or an adversarial act intended to skew data. For Sony’s PlayStation Network (PSN) that services the world’s most popular gaming platform, we observe cheating through duplication of user data in the context of trophies/achievements. This particular domain is representative of the challenges of increasingly prevalent temporal data, where conventional similarity and distance-based deduplication techniques struggle in the context of deduplication. We leverage the Adaptive Sorted Neighborhood Method (ASNM) for temporal domains by applying ASNM, inferring attribute metadata, and performing inference of temporal ordering requirements using subsequence discovery techniques Longest Common Subsequence (LCS) and Needleman–Wunsch (NW). For records of a shared type, we split each record’s time-ordered events into constrained and unconstrained sequences. Through both a binary classification and confidence-based approach, we indicate suspicious (errant) records that do not adhere to the inferred constrained order and may indicate a record as a duplicate if its unconstrained order matches that of another record. ASNM, [Formula: see text] and [Formula: see text] were evaluated against a labeled dataset of 22,794 records from PSN trophy data where duplication may be indicative of cheating. [Formula: see text] resulted in an F1 of 0.949 using the confidence-based approach, outperforming ASNM and [Formula: see text]. ASNM’s best performance was an F1 of 0.708 at the 0.99 similarity threshold; [Formula: see text]’s best performance was an F1 of 0.942 using the confidence-based approach. The significant performance improvement costs little overhead as [Formula: see text] and [Formula: see text] averaged only 3.79% and 5.75% additional runtime, respectively.

Trajectory Clustering-Based Anomaly Detection in Indoor Human Movement

Human movement anomalies in indoor spaces commonly involve urgent situations, such as security threats, accidents, and fires. This paper proposes a two-phase framework for detecting indoor human trajectory anomalies based on density-based spatial clustering of applications with noise (DBSCAN). The first phase of the framework groups datasets into clusters. In the second phase, the abnormality of a new trajectory is checked. A new metric called the longest common sub-sequence using indoor walking distance and semantic label (LCSS_IS) is proposed to calculate the similarity between trajectories, extending from the longest common sub-sequence (LCSS). Moreover, a DBSCAN cluster validity index (DCVI) is proposed to improve the trajectory clustering performance. The DCVI is used to choose the epsilon parameter for DBSCAN. The proposed method is evaluated using two real trajectory datasets: MIT Badge and sCREEN. The experimental results show that the proposed method effectively detects human trajectory anomalies in indoor spaces. With the MIT Badge dataset, the proposed method achieves 89.03% in terms of F1-score for hypothesized anomalies and above 93% for all synthesized anomalies. In the sCREEN dataset, the proposed method also achieves impressive results in F1-score on synthesized anomalies: 89.92% for rare location visit anomalies (τ = 0.5) and 93.63% for other anomalies.

Efficient Trajectory Clustering with Road Network Constraints Based on Spatiotemporal Buffering

The analysis of individuals’ movement behaviors is an important area of research in geographic information sciences, with broad applications in smart mobility and transportation systems. Recent advances in information and communication technologies have enabled the collection of vast amounts of mobility data for investigating movement behaviors using trajectory data mining techniques. Trajectory clustering is one commonly used method, but most existing methods require a complete similarity matrix to quantify the similarities among users’ trajectories in the dataset. This creates a significant computational overhead for large datasets with many user trajectories. To address this complexity, an efficient clustering-based method for network constraint trajectories is proposed, which can help with transportation planning and reduce traffic congestion on roads. The proposed algorithm is based on spatiotemporal buffering and overlapping operations and involves the following steps: (i) Trajectory preprocessing, which uses an efficient map-matching algorithm to match trajectory points to the road network. (ii) Trajectory segmentation, where a Compressed Linear Reference (CLR) technique is used to convert the discrete 3D trajectories to 2D CLR space. (iii) Spatiotemporal proximity analysis, which calculates a partial similarity matrix using the Longest Common Subsequence similarity indicator in CLR space. (iv) Trajectory clustering, which uses density-based and hierarchical clustering approaches to cluster the trajectories. To verify the proposed clustering-based method, a case study is carried out using real trajectories from the GeoLife project of Microsoft Research Asia. The case study results demonstrate the effectiveness and efficiency of the proposed method compared with other state-of-the-art clustering-based methods.

COMPARATIVE STUDY AND EVALUATION OF TEXT CLASSIFICATION APPROACHES IN NLP USING DIFFERENT ALGORITHMS

COMPARATIVE STUDY AND EVALUATION OF TEXT CLASSIFICATION APPROACHES IN NLP USING DIFFERENT ALGORITHMS

Research on LCS Word Segmentation Expansion and Optimization of Subsequence Solving Algorithm

As one of the most common and critical tasks in natural language processing (NLP), similarity calculation has a wide range of applications in fields such as censorship detection and information retrieval. In order to improve the accuracy of text similarity calculation, based on the in-depth analysis of traditional LCS algorithms, this paper proposes an extended LCS algorithm based on word separation and synonym matching. The algorithm combines the new achievements in natural language processing research, and addresses the problems that LCS cannot screen the common means of plagiarism and nesting when used for text similarity comparison, as well as the high time complexity and weak performance of the backtracking algorithm used in solving the longest common subsequence, and realizes the matching of synonyms between sequences by calculating the similarity between words through synonym word forest on the basis of word separation. It achieves the screening of plagiarism and copying means such as synonym substitution to the original text. At the same time, the algorithm improves the traditional algorithm for solving LCS subsequences by recording the associated positions of the characters of the longest common subsequence in each sequence and appropriately increasing the spatial complexity to realize the chain marking of common sequences. The experimental results show that the LCS extension algorithm proposed in this paper can accurately identify the substitution of synonyms in the text, and the calculation result of text similarity is more accurate, while the time complexity of solving the LCS subsequence is reduced from <I>O</I>(2^{<i>max</i>(<i>m</i>,<i>n</i>)}) to linear level <I>O</I>(<i>n</i>).

Automatic identification system-based trajectory clustering framework to identify vessel movement pattern

&lt;span lang="EN-US"&gt;Automatic identification system (AIS) is a vessel radio navigation equipment that has been determined by international maritime organization (IMO). Historical AIS data can be utilized for anomaly detection, trajectory prediction, and vessel trajectory planning. These benefits can be achieved by identifying the vessel's trajectory pattern through trajectory clustering. However, more effort is needed in trajectory clustering using AIS data due to their large volume and the significant number of deficiencies. In addition, trajectory clustering cannot be directly applied to trajectory data, which also applies to vessel trajectory. Therefore, we propose a trajectory clustering framework by combining douglas peucker (DP), longest common subsequence (LCSS), multi-dimensional scaling (MDS), and density-based spatial clustering of applications with noise (DBSCAN). Our experiments, carried out with AIS data for the Lombok Strait, Indonesia, showed that the trajectory compression with DP significantly accelerates the similarity measurement process. Moreover, we found that the LCSS is the optimal algorithm for similarity measurement of vessel trajectories based on AIS data. We also applied the right combination of MDS and DBSCAN in density-based clustering. The proposed framework can distinguish trajectoriess in different directions, identify the noise, and produce good quality clusters in relatively fast total processing time.&lt;/span&gt;

A novel smart photoelectric lock system: Speech transmitted by laser and speech to text

We propose a circuit that modulates a speech signal to a laser, using which the speech signal can be transmitted using the laser. Also, it shows the use of a platform based on embedded ARM (Advanced RISC Machine), running a small deep learning model based on TDNN (Time delay neural network) and LSTM (Long short-term memory), and converting speech to text, and use the text cipher for unlocking. This research implements a smart lock system that can set a pre-record speech cipher and verify the similarity through a laser transmission speech cipher to unlock it. In our experiment result, the English speech of laser transmission can reach a WER (Word error rate) of 14.06% through the deep learning model to recognize the content of the speech cipher. We also design a similarity comparison algorithm based on LCS (Longest common subsequence) to compare the character set of the laser transmission speech compare and the prerecord speech cipher to calculate the similarity rate. Through the similarity comparison algorithm, when the WER is 27.27%, the male speech samples used in this study still have a 95% unlocking success rate, while the female speech samples have a 100% unlocking success rate. Compared with only using automatic speech recognition (ASR) to unlock, the method we propose is to compare the similarity of the content of speech cipher. The method significantly improves the unlocking fault tolerance of using lasers to transmit audio. Therefore, by using the laser to transmit the speech cipher, the usability of the photoelectric smart lock system has been significantly improved. At the same time, the characteristics of the laser are not easy to eavesdrop on the cipher, which can also improve security.

Diffusion Alignment Coefficient (DAC): A Novel Similarity Metric for Protein-Protein Interaction Network.

Interaction networks can be used to predict the functions of unknown proteins using known interactions and proteins with known functions. Many graph theory or diffusion-based methods have been proposed, using the assumption that the topological properties of a protein in a network are related to its biological function. Here we seek to improve function prediction by finding more similar neighbors with a new diffusion-based alignment technique to overcome the topological information loss of the node. In this study, we introduce the Diffusion Alignment Coefficient (DAC) algorithm, which combines diffusion, longest common subsequence, and longest common substring techniques to measure the similarity of two nodes in protein interaction networks. As a proof of concept, our experiments, conducted on a real PPI networks S.cerevisiae and Homo Sapiens, demonstrated that our method obtained better results than competitors for MIPS and MSigDB Collections hallmark gene set functional categories. This is the first study to develop a measure of node function similarity using alignment to consider the positions of nodes in protein-protein interaction networks. According to the experimental results, the use of spatial information belonging to the nodes in the network has a positive effect on the detection of more functionally similar neighboring nodes.

Software System of Checking for Plagiarism of Ukrainian Texts

The purpose of this work is to describe the methodology of building a software system (application) for plagiarism checking of scientific publications in the Ukrainian language using two machine learning models, Word2Vec and BERT. We consider the detection of external plagiarism in Ukrainian texts.Plagiarism is usually defined as the passing off someone else’s ideas as your own. As the Internet becomes more and more accessible every day, a huge amount of data becomes available to people. Nowadays, it is quite easy to find a suitable study and plagiarize it instead of developing one’s own from scratch.Plagiarism undermines the efforts of the researcher whose work has been plagiarized and gives the plagiarist the opportunity to over-praise himself; such a person can be detrimental when appointed to an important position.Many fields of life are susceptible to plagiarism, including research and education. Plagiarism can also take many forms: from straight up copy-paste to paraphrasing and sentence restructuring. This makes plagiarism a rather complex problem, where methods, such as longest common subsequence or n-grams, based on finding shared words between documents, might not work. Therefore, we might consider applying deep learning to the problem of plagiarism detection.In this article we discussed the concept of plagiarism and listed its types. Two machine learning models have been proposed for plagiarism detection: Word2Vec and BERT. We also provided an overview of both models and described how they could be used in the problem of plagiarism detection.A web application for plagiarism detection in the Ukrainian language has been developed. This application features React, a JavaScript framework, on the frontend and Python on the backend. To store application data, MongoDB is used.This application allows a user to input a text that will be compared with the texts from the application database using cosine similarity or Euclidean distance as metrics. Comparison is performed using word embeddings, calculated by pre-trained BERT or Word2Vec model. A user can choose the model and similarity metrics using the application’s UI.The application can be further improved to not only output similarity metric but also highlight the similar sentences in the texts.

Study on typical process route mining method based on multilevel longest common subsequence information entropy and intelligent clustering model

ABSTRACT A large number of manufacturing cases are accumulated by manufacturing enterprises in the process of operation and development, and therefore, one of the most effective ways to improve manufacturing efficiency and support innovation is to reuse these case resources reasonably. In reality, the first problem to be solved is to determine the case resources with high reuse value potential, so as to realize the high-value reuse of manufacturing case resources. With the purpose of scientific determination of the reuse objects and improvement of the reuse flexibility, a typical process route mining method based on multilevel longest common subsequence (LCS) information entropy and intelligent clustering model is proposed in this paper. First, a similarity calculation method of machining process route based on multilevel (LCS) information entropy is proposed, which can more comprehensively and accurately evaluate the similarity of machining process. On this basis, a process route clustering model based on spectral clustering idea and particle swarm optimization-Kmeans clustering algorithm is proposed, which realizes the clustering of process routes as per the similarity; in the end, the typical and representative process routes in each cluster are extracted, and the typical process routes are mined for reuse. In the end, it shows that the method proposed in this paper can effectively mine high-value process reuse objects and then can further support manufacturing case reuse through three verification cases.

A novel system for discovery and reuse of typical process route based on information entropy and PSO-Kmeans clustering algorithm

Manufacturing enterprises will accumulate a large number of manufacturing instances as they run and develop. Being able to excavate and reuse the instance resources reasonably is one of the most effective ways to improve manufacturing and support innovation. To determine the reuse object scientifically and raise the reuse flexibility, a novel system for discovery and reuse of typical process route based on the information entropy and PSO-Kmeans clustering algorithm is proposed in this paper. In this system, a similarity measurement method of machining process routes based on the information entropy of multistage longest common subsequence is developed. Then a discovery method of typical process route based on the spectral clustering idea and PSO-Kmeans clustering algorithm is invented, and the two reuse approaches based on the typical process route are analyzed and discussed. Finally, the three case studies are rendered and the results reveal that the proposed system can provide better support for manufacture instance reuse.

Behavior Cloning and Replay of Humanoid Robot via a Depth Camera

The technique of behavior cloning is to equip a robot with the capability of learning control skills through observation, which can naturally perform human–robot interaction. Despite many related studies in the context of humanoid robot behavior cloning, the problems of the unnecessary recording of similar actions and more efficient storage forms than recording actions by joint angles or motor counts are still worth discussing. To reduce the storage burden on robots, we implemented an end-to-end humanoid robot behavior cloning system, which consists of three modules, namely action emulation, action memorization, and action replay. With the help of traditional machine learning methods, the system can avoid recording similar actions while storing actions in a more efficient form. A jitter problem in the action replay is also handled. In our system, an action is defined as a sequence of many pose frames. We propose a revised key-pose detection algorithm to keep minimal poses of each action to minimize storage consumption. Subsequently, a clustering algorithm for key poses is implemented to save each action in the form of identifiers series. Finally, a similarity equation is proposed to avoid the unnecessary storage of similar actions, in which the similarity evaluation of actions is defined as an LCS problem. Experiments on different actions have shown that our system greatly reduces the storage burden of the robot while ensuring that the errors are within acceptable limits. The average error of the revised key-pose detection algorithm is reduced by 69% compared to the original and 26% compared to another advanced algorithm. The storage consumption of actions is reduced by 97% eventually. Experimental results demonstrate that the system can efficiently memorize actions to complete behavioral cloning.

An Algorithm for the Longest Common Subsequence and Substring Problem

In this note, we first introduce a new problem called the longest common subsequence and substring problem. Let X and Y be two strings over an alphabet ∑. The longest common subsequence and substring problem for X and Y is to find the longest string, which is a subsequence of X and a substring of Y. We propose an algorithm to solve the problem.

Автоматическое определение сходства Javadoc-комментариев

Code comments are an essential part of software documentation. Many software projects suffer the problem of low-quality comments that are often produced by copy-paste. In case of similar methods, classes, etc. copy-pasted comments with minor modifications are justified. However, in many cases this approach leads to degraded documentation quality and, subsequently, to problematic maintenance and development of the project. In this study, we address the problem of near-duplicate code comments detection, which can potentially improve software documentation. We have conducted a thorough evaluation of traditional string similarity metrics and modern machine learning methods. In our experiment, we use a collection of Javadoc comments from four industrial open-source Java projects. We have found out that LCS (Longest Common Subsequence) is the best similarity algorithm taking into account both quality (Precision 94%, Recall 74%) and performance.

A Novel Model to Generate Heterogeneous and Realistic Time-Series Data for Post-Stroke Rehabilitation Assessment.

The application of machine learning-based tele-rehabilitation faces the challenge of limited availability of data. To overcome this challenge, data augmentation techniques are commonly employed to generate synthetic data that reflect the configurations of real data. One such promising data augmentation technique is the Generative Adversarial Network (GAN). However, GANs have been found to suffer from mode collapse, a common issue where the generated data fails to capture all the relevant information from the original dataset. In this paper, we aim to address the problem of mode collapse in GAN-based data augmentation techniques for post-stroke assessment. We applied the GAN to generate synthetic data for two post-stroke rehabilitation datasets and observed that the original GAN suffered from mode collapse, as expected. To address this issue, we propose a Time Series Siamese GAN (TS-SGAN) that incorporates a Siamese network and an additional discriminator. Our analysis, using the longest common sub-sequence (LCSS), demonstrates that TS-SGAN generates data uniformly for all elements of two testing datasets, in contrast to the original GAN. To further evaluate the effectiveness of TS-SGAN, we encode the generated dataset into images using Gramian Angular Field and classify them using ResNet-18. Our results show that TS-SGAN achieves a significant accuracy increase of classification accuracy (35.2%-42.07%) for both selected datasets. This represents a substantial improvement over the original GAN.

Implementation of Rapid Code Transformation Process Using Deep Learning Approaches

Our previous work has introduced the newly generated program using the code transformation model GPT-2, verifying the generated programming codes through simhash (SH) and longest common subsequence (LCS) algorithms. However, the entire code transformation process has encountered a time-consuming problem. Therefore, the objective of this study is to speed up the code transformation process significantly. This paper has proposed deep learning approaches for modifying SH using a variational simhash (VSH) algorithm and replacing LCS with a piecewise longest common subsequence (PLCS) algorithm to faster the verification process in the test phase. Besides the code transformation model GPT-2, this study has also introduced Microsoft MASS and Facebook BART for a comparative analysis of their performance. Meanwhile, the explainable AI technique using local interpretable model-agnostic explanations (LIME) can also interpret the decision-making of AI models. The experimental results show that VSH can reduce the number of qualified programs by 22.11%, and PLCS can reduce the execution time of selected pocket programs by 32.39%. As a result, the proposed approaches can significantly speed up the entire code transformation process by 1.38 times on average compared with our previous work.

A central limit theorem for the length of the longest common subsequences in random words

Let $(X_i)_{i \geq 1}$ and $(Y_i)_{i\geq1}$ be two independent sequences of independent identically distributed random variables taking their values in a common finite alphabet and having the same law. Let $LC_n$ be the length of the longest common subsequences of the two random words $X_1\cdots X_n$ and $Y_1\cdots Y_n$. Under a lower bound assumption on the order of its variance, $LC_n$ is shown to satisfy a central limit theorem. This is in contrast to the limiting distribution of the length of the longest common subsequences in two independent uniform random permutations of $\{1, \dots, n\}$, which is shown to be the Tracy-Widom distribution.

Image Stitching Method of Bone Stick Fragment Based on Similarity Freeman Code Matching

A large number of bone stick fragments were unearthed at the Weiyang Palace, and the manual stitching method is inefficient and time-consuming, therefore Image Stitching Method of Bone Stick Fragment Based on Similarity Freeman code Matching is proposed by this paper to improve bone stick stitching efficiency. The Freeman code is used to represent the contour of the true color RGB image of bone stick fragments, and the direction change dense feature of the bone stick fragment fracture is fused with the Freeman code difference to represent the change direction feature to locate the fracture boundary. At the same time, the integer basic operation is used to replace the complex calculus calculation to improve the efficiency of the algorithm; The longest common subsequence algorithm is used to perform local similarity rough matching on Freeman code of the contour at the fragment fracture, and then the angle and distance error threshold is set according to the prior knowledge of manual stitching to reduce the matching error; Finally, according to the matching results, the complete bone stick contour binary image is drawn. Compared with other algorithms, the contour Cosine similarity accuracy of the bone stick image stitched by this algorithm is improved by 21.68%, which can be accurate to 97.80%; the Editing distance similarity accuracy is improved by 14.29%, which can be accurate to 91.87%;the Peak signal-to-noise ratio is improved by 83%, which can be accurate to 31.13%;the Structural Similarity is improved by 29%, which can be accurate to 0.87; the Figure of merit is improved by 39%, which can be accurate to 0.85. The experiment results show that the algorithm proposed in this paper can effectively assist the stitching work of bone stick cultural relics.

FEATURE ENGINEERING WITH SENTENCE SIMILARITY USING THE LONGEST COMMON SUBSEQUENCE FOR EMAIL CLASSIFICATION

Feature selection plays a prominent role in email classification since selecting the most relevant features enhances the accuracy and performance of the learning classifier. Due to the exponential increase rate in the usage of emails, the classification of such emails posed a fitting problem. Therefore, there is a requirement for a proper classification system. Such an email classification system requires an efficient feature selection method for the accurate classification of the most relevant features. This paper proposes a novel feature selection method for sentence similarity using the longest common subsequence for email classification. The proposed feature selection method works in two main phases: First, it builds the longest common subsequence vector of features by comparing each email with all other emails in the dataset. Later, a template is constructed for each class using the closest features of emails of a particular class. Further, email classification is tested for unseen emails using these templates. The performance of the proposed method is compared with traditional feature selection methods such as TF-IDF, Information Gain, Chi-square, and semantic approach. The experimental results showed that the proposed method performed well with 96.61% accuracy.

Using sequence mining to study students’ calculator use, problem solving, and mathematics achievement in the National Assessment of Educational Progress (NAEP)

Using appropriate tools strategically to aid in problem solving is a crucial skill identified in K-12 mathematics curriculum standards. As more assessments transition from paper-and-pencil to digital formats, a variety of interactive tools have been made available to test takers in digital testing platforms. Using onscreen calculators as an example, this study illustrates how process data obtained from student interactions with a digitally-based large-scale assessment can be leveraged to explore how and how well test takers use interactive tools and unveil their mathematical problem-solving processes and strategies. Specifically, sequence mining techniques using the longest common subsequence were applied on process data collected from a nationally representative sample who took the National Assessment of Educational Progress (NAEP) mathematics assessment to examine patterns of eighth-grade students’ calculator-use behaviors and the content of calculator input across a series of items. Sequences of keystrokes executed on the onscreen calculator by test takers were compared to reference sequences identified by content experts as proficient and efficient use to infer how well and how consistently the calculator was used. Results indicated that calculator-use behaviors and content differed by item characteristics. Students were more likely to use calculators on calculation-demanding items that involve intensive and complex computations than on items that involve simple or no computation. Using the calculator on more calculation-demanding items and using it in a manner that is more efficient and more similar to reference sequences on these items were related to higher mathematical proficiency. Findings have implications for assessment design and can be used in educational practices to provide educators with actionable process-related information on tool use and problem solving.