Similarity Algorithm Research Articles

Intelligent recommendation model of tourist places based on collaborative filtering and user preferences

ABSTRACT Providing personalized recommendation service for users and improving the accuracy of recommendation and user satisfaction are the main research tasks of current travel recommendation systems. The intelligent recommendation model of tourist places requires the algorithm to be able to accurately recommend tourist attractions according to the user’s interests. Big data and artificial intelligence technologies have driven the development of intelligent recommendation systems. However, realistic data are often sparse, and the lack of common rating items among users makes some traditional similarity measures impossible to calculate. In addition, traditional collaborative filtering algorithms ignore the issue of user preferences, which can cause a decrease in recommendation accuracy. To address these issues, this paper analyzes the factors affecting users’ interest preferences in terms of their global and local rating information. The interest preferences of users are calculated by computing the probability distribution of user rating information globally and using Hamming approach degree. A similarity algorithm about user preferences is derived using Jeffries-Matusita distance. The similarity algorithm is effectively combined with the traditional similarity algorithm to propose a model of collaborative filtering recommendation algorithm for tourist attractions based on user preferences under sparse data. The paper aims to improve the accuracy of tourist attraction recommendation systems by incorporating user preferences and addressing the issue of sparse data and the lack of common rating items among users that traditional similarity measures cannot calculate. The experimental results show that the improved algorithm model outperforms the traditional collaborative filtering algorithm and other algorithms. And it also has high accuracy rate on more sparse tourism data set.

The choice of spectral similarity algorithms influences suspected soil sample provenance

Similarity algorithms are commonly used in soil forensic applications to help identify similar samples from an existing reference library as possible source locations of unknown target samples. These algorithms are well-suited to compare soil spectra. However, different similarity algorithms may lead to different clusters of similar samples, and thus different strengths of evidence in forensic investigations. To quantify this, we conducted a study to evaluate the influence of seven similarity algorithms on soil provenance, using as a sample set a soil spectral library consisting of 280 soil profiles from Anhui Province, China. This library includes three spatial scales of datasets: provincial (DSp), county (DSc) and field (DSf). A set of ten samples covering a wide range of spectra variations were selected from the DSf dataset as the “unknown” samples, with the remaining being used as the reference samples. This study aimed to: (1) evaluate how several commonly-used similarity algorithms, namely Euclidean distance (ED), Mahalanobis distance (MD), Spectral angle mapper (SAM), and Spectral information divergence (SID), as well as variants of several of these measured in standardized principal component space computed from the spectra (ED_PCA, MD_PCA and SAM_PCA), influence the identification of the matched similar samples; (2) determine the overlap in sample selection between different similarity algorithms; (3) propose best practices for similarity algorithms applied to soil forensic analysis using spectroscopy. The use of different similarity algorithms did influence the selection of most similar samples. The similarity algorithms calculated in PC space (ED_PCA, MD_PCA and SAM_PCA) performed slightly better than their counterparts calculated in spectral space. Due to the availability of a detailed spectral library, regardless of the different similarity algorithms used, the matched most similar samples were all located close to the unknowns, mostly within 3 km, with one exception. That is, the varied choices of different similarity algorithms hardly influenced the conclusion of soil provenance in this case. In general, MD_PCA, SAM and ED were the best similarity algorithms overall. However, since there was no single best algorithms for all cases, we recommend the joint use of MD_PCA, SAM and ED as an ensemble. Indications of possible sample provenance from these similarity measured can be useful evidence to complement evidence from other methods in a forensic investigation.

Towards a better similarity algorithm for host-based intrusion detection system

Abstract An intrusion detection system plays an essential role in system security by discovering and preventing malicious activities. Over the past few years, several research projects on host-based intrusion detection systems (HIDSs) have been carried out utilizing the Australian Defense Force Academy Linux Dataset (ADFA-LD). These HIDS have also been subjected to various algorithm analyses to enhance their detection capability for high accuracy and low false alarms. However, less attention is paid to the actual implementation of real-time HIDS. Our principal objective in this study is to create a performant real-time HIDS. We propose a new model, “Better Similarity Algorithm for Host-based Intrusion Detection System” (BSA-HIDS), using the same dataset ADFA-LD. The proposed model uses three classifications to represent the attack folder according to certain criteria, the entire system call sequence is used. Furthermore, this work uses textual distance and compares five algorithms like Levenshtein, Jaro–Winkler, Jaccard, Hamming, and Dice coefficient, to classify the system call trace as attack or non-attack based on the notions of interclass decoupling and intra-class coupling. The model can detect zero-day attacks because of the threshold definition. The experimental results show a good detection performance in real-time for Levenshtein/Jaro–Winkler algorithms, 99–94% in detection rate, 2–5% in false alarm rate, and 3,300–720 s in running time, respectively.

Recommendation of Scientific Publications—A Real-Time Text Analysis and Publication Recommendation System

Studies have shown that although having more information improves the quality of decision-making, information overload causes adverse effects on decision quality. Visual analytics and recommendation systems counter this adverse effect on decision-making. Accurately identifying relevant information can reduce the noise during exploration and improve decision-making. These countermeasures also help scientists make correct decisions during research. We present a novel and intuitive approach that supports real-time collaboration. In this paper, we instantiate our approach to scientific writing and propose a system that supports scientists. The proposed system analyzes text as it is being written and recommends similar publications based on the written text through similarity algorithms. By analyzing text as it is being written, it is possible to provide targeted real-time recommendations to improve decision-making during research by finding relevant publications that might not have been otherwise found in the initial research phase. This approach allows the recommendations to evolve throughout the writing process, as recommendations begin on a paragraph-based level and progress throughout the entire written text. This approach yields various possible use cases discussed in our work. Furthermore, the recommendations are presented in a visual analytics system to further improve scientists’ decision-making capabilities.

A marker-less assembly stage recognition method based on corner feature

Fast and robust product assembly stage recognition is a key step in human–machine cooperative assembly. To solve the recognition problem of similar adjacent assembly stages, a marker-less assembly stage recognition method is proposed based on corner feature between an assembling product and a digital model. Considering the geometric features of mechanical products, a corner identification method is proposed based on the circumferential angle difference (CADF). Then a corner matching method based on distance constraint is studied for ICP registration to realize the point cloud registration between the product and digital model. Based on the registration relationship, a similarity algorithm based on proximity point proportion is used to calculate the similarity between models and the input assembling product. The model with the greatest similarity is taken as the stage recognition result. In experiments on four group assembling products, the average stage recognition accuracy is 96.15%, which indicates that the proposed method can solve the stage recognition problem. The corner identification method based on the CADF outperforms the Harris-3D corner detection method in the efficiency of assembly stage recognition.

A Comparison of Document Similarity Algorithms

Document similarity is an important part of Natural Language Processing and is most commonly used forplagiarism-detection and text summarization. Thus, finding the overall most effective document similarity algorithm could have a major positive impact on the field of Natural Language Processing. This report setsout to examine the numerous document similarity algorithms, and determine which ones are the mostuseful. It addresses the most effective document similarity algorithm by categorizing them into 3 types ofdocument similarity algorithms: statistical algorithms, neural networks, and corpus/knowledge-basedalgorithms. The most effective algorithms in each category are also compared in our work using a series of benchmark datasets and evaluations that test every possible area that each algorithm could be used in.

Human scent signature on cartridge case survives gun being fired: A preliminary study on a potential of scent residues as an identification tool.

This paper focuses on a chemical analysis of human scent samples that were obtained from cartridge cases after being fired and their comparison with scent samples collected under laboratory conditions. Scent samples were analyzed by comprehensive two-dimensional gas chromatography coupled with the time-of-flight mass spectrometer. The results obtained from the chemical analyzes confirmed the desired stability of the human scent evidence and outlined the possible application for forensic purposes. The qualitative results of the study converge with the findings of previous studies on the composition of human scent and the chemical composition of human fingerprints. Furthermore, statistical analyzes were performed employing similarity algorithms such as Pearson's and Spearman's correlations, or Kendall's tau. The resulting comparison of the scent samples secured on fired cartridge cases compared with those samples collected under laboratory conditions yielded ten out of ten correct identifications of the scent inflictor.

Comparison of Two Approaches to Detecting Switched Class Labels in LCA Simulations: Class Assignment vs. Class Similarity

The detection of switched class labels is required in latent class analysis (LCA) simulations involving parameter estimation. The present study proposed a class similarity (CS) algorithm to detect switched class labels based on the similarity of conditional probabilities between true and estimated classes, in contrast to Tueller et al.’s class assignment (CA) algorithm considering the number of participants in each true class assigned to every estimated class. The performances of CS and CA were compared by examining the average class assignment accuracy and the bias of parameter estimates in a numerical experiment. CS and CA were shown to perform similarly that either method can be used to detect switched class labels in future LCA simulations. The performance of the two algorithms warrants further investigation under a wider simulation condition.

Quantum fingerprinting for heterogeneous devices localization

In this paper, we present QHFP, a device-independent quantum fingerprint matching algorithm that addresses two of the issues for realizing worldwide ubiquitous large-scale location tracking systems: scalability of storage space and running time as well as devices heterogeneity. In particular, we present a quantum algorithm with a complexity that is exponentially better than the classical techniques, both in space and running time. QHFP also has provisions for handling the inherent localization error due to building the large-scale fingerprint using heterogeneous devices. We give the details of the entire system starting from extracting device-independent features from the raw received signal strength, mapping the classical feature vectors to their quantum counterparts, and showing a quantum cosine similarity algorithm for fingerprint matching.We implement our quantum algorithm and deploy it in a real testbed using the IBM Quantum machine simulator. We also study the effect of the different quantum error sources, such as quantum gates error and quantum measurement error, on the system accuracy. Based on this, we discuss the suitable provisions to mitigate the effect of the quantum error sources on the localization accuracy.Results confirm the ability of QHFP to obtain the correct estimated location with an exponential improvement in space and running time compared to the traditional classical counterparts. In addition, the proposed device-independent features lead to more than 20% better accuracy in median error. This highlights the promise of our algorithm for future ubiquitous large-scale worldwide device-independent fingerprinting localization systems.

A hunger-based scheduling strategy for distributed crawler

Distributed crawling is one of the mainstream text data collection technologies, which is essential for mining boundless data available on the Internet for users. Internet information is clustered by the correlation of keywords, and users employ search engines to retrieve relevant keywords to get the information they care about. Distributed crawlers are used to mine the Internet information by simulating users’ behavior, the more important keywords that users care about, the higher the correlation of data to keywords. In order to preferentially collect information that users care about with minimal resource consumption, in this paper, we design a scheduling framework and propose a novel scheduling strategy based on hunger for distributed crawler. We first define the load capacity of distributed crawler as hunger which reflects the ability to complete tasks and divide keywords queues into sub-queues based on the hunger of distributed crawlers. Then, we use a vector space model and cosine similarity algorithm to learn the correlation of keywords to text data and apply the optimized logistic algorithm to measure the importance of keywords. Meanwhile, we design a comprehensive evaluation algorithm to quantify the contribution of keywords, so that updating sub-queues order. Finally, new sub-queues are used in the deeper scheduling to preferentially get data that users desire and sacrifice the least number of resources. Experimental results demonstrate that our method optimizes the scheduling procedures and makes crawling more efficient with less run time.

A Comprehensive Strategy Combining Feature Selection and Local Optimization Algorithm to Optimize the Design of Low-Density Chip for Genomic Selection

Design of low-density SNP chips provides an opportunity for wide application of genomic selection at lower cost. A novel strategy referred to as the “block-free” method is proposed in this study to select a subset of SNPs from a high-density chip to form a low-density panel. In this method, Feature Selection using a Feature Similarity (FSFS) algorithm was first performed to remove highly correlated SNPs, and then a Multiple-Objective, Local-Optimization (MOLO) algorithm was used to pick SNPs for the low-density panel. Two other commonly used methods called the “uniform” method and the “block-based” method were also implemented for comparison purposes. A real pig dataset with 7967 individuals from three breeds containing 43,832 SNPs was used for comparison of the methods. In terms of genotype imputation accuracy and genomic prediction accuracy, our strategy was superior in most cases when the densities were lower than 1K. The genotype imputation accuracy from the low-density chip compared to the original high-density chip was higher than 90% in all pig breeds as the density increased to 1K. In addition, the accuracies of predicted genomic breeding values (GEBV) calculated using the imputed panel were nearly 90% of estimates from the original chip for all traits and breeds. Our strategy is effective to design low-density chips by making full use of information of close relationships for genomic selection in animals and plants.

Detecting Urban Commercial Districts by Fusing Points of Interest and Population Heat Data with Region-Growing Algorithms

Reasonable urban commercial planning must clarify the location and scope of urban commercial districts (UCDs). However, existing studies typically detect spurious UCDs owing to the bias in a single data source while ignoring the continuity and ambiguity of commercial district boundaries. Therefore, in this study, we designed a two-stage approach for detecting UCDs. First, points of interest and population heat data were fused through hotspot and overlay analyses to detect core commercial areas. The boundaries of the UCDs were then identified by considering adjacent blocks using adjusted cosine similarity and region-growing algorithms. Finally, an experiment was conducted in Xiamen, revealing concentrated businesses on Xiamen Island and sparse businesses outside Xiamen Island. An experimental comparison with other strategies confirmed the improved modeling ability of this approach for the edge ambiguity of UCDs. This framework provides tools for urban commercial planning and helps recognize urban commercial patterns in a timely manner.

Investigating the Validity and Reliability of a Comprehensive Essay Evaluation Model of Integrating Manual Feedback and Intelligent Assistance

How to respond effectively and efficiently to students’ writing and to maximize the potential of feedback to promote student writing skills deserves careful consideration. The use of intelligent algorithms to assist teachers in the manual evaluation of students’ essays is of practical value and importance in the context of "artificial intelligence + education". The existing intelligent evaluation techniques are subject to the interference of many factors such as openness of questions and students' language expression abilities. For this reason, this study conducts a study on comprehensive essay evaluation method with intelligent assistance and manual feedback and on reliability and validity tests. Before the intelligent evaluation, the semantic integrity of students' essays is analyzed, and a semantic integrity analysis model of students' essays based on BERT model is constructed. A fusion similarity algorithm for essay answer key points is proposed by extracting these characteristics that have an impact on the evaluation results, such as technique preferences, paragraph content and paragraph topic of the essays. The Siamese and ESIM networks are combined to propose an intelligent evaluation model for students’ essays, and the model framework and working principle are described in detail. The experimental results verify the effectiveness of the constructed model.

Norm and interpolation based absolute stress evaluation in steel members using the Lcr wave

The uniaxial absolute stress in steel members is a vital indicator for evaluating the performance of steel structures. Non-destructive ultrasonic detection is widely applied in the absolute stress measurement of steel members with the advantages of easy implementation, no limits on materials and high precision. The acquisition algorithm of time-of-flight (TOF), the sampling rate of ultrasonic signal and the processing of outliers determine the accuracy of the stress measurement. In this study, we proposed a norm and interpolation based algorithm for absolute stress measurement using the longitudinal critically refracted (Lcr) waves, which improves the accuracy and velocity of the absolute stress measurement. Firstly, the norm was designed to obtain the TOF of the ultrasonic signal. When only the Lcr wave arose in the ultrasonic signal, the two-norm similarity algorithm was found to be consistent with the cross-correlation-based algorithm and the corresponding theoretical expressions were derived. With the non-Lcr wave in the ultrasonic echo, the two-norm similarity algorithm has stronger anti-interference ability. Secondly, the pchipslopes, spline and linear interpolation algorithms were introduced via which the TOF of ultrasound with low sampling rate reaches the same accuracy as that of high sampling rate signal. Finally, a special class of outliers in TOF was explored, and a threshold based on the number of shift points and the stress feature was proposed to calibrate these outliers to normal values. Experiments demonstrate the proposed algorithm improves the accuracy and velocity of uniaxial absolute stress measurement in structural steel members.

Kurdish Kurmanji Lemmatization and Spell-checker with Spell-correction

There are many studies about using lemmatization and spell-checker with spell-correction regarding English, Arabic, and Persian languages but only few studies found regarding low-resource languages such as Kurdish language and more specifically for Kurmanji dialect, which increased the need of creating such systems. Lemmatization is the process of determining a base or dictionary form (lemma) for a specific surface pattern, whereas spell-checkers and spell-correctors determine whether a word is correctly spelled also correct a range of spelling errors, respectively. This research aims to present a lemmatization and a word-level error correction system for Kurdish Kurmanji Dialect, which are the first tools for this dialect based on our knowledge. The proposed approach for lemmatization is built on morphological rules, and a hybrid approach that relies on the n-gram language model and the Jaccard Coefficient Similarity algorithm was applied to the spell-checker and spell-correction. The process results for lemmatization, as detailed in this article, rates of 97.7% and 99.3% accuracy for noun and verb lemmatization, correspondingly. Furthermore, for spell-checker and spell-correction, accordingly, accuracy rates of 100% and 90.77% are attained.

Weighted edit distance optimized using genetic algorithm for SMILES-based compound similarity

Weighted edit distance optimized using genetic algorithm for SMILES-based compound similarity

Comparison of String Similarity Algorithm in post-processing OCR

The Optical Character Recognition (OCR) problem that often occurs is that the image used, has a lot of noise covering letters in a word partially. This can cause misspellings in the process of word recognition or detection in the image. After the OCR process, we must do some post-processing for correcting the word. The words will be corrected using a string similarity algorithm. So what is the best algorithm? We conducted a comparison algorithm including the Levenshtein distance, Hamming distance, Jaro-Winkler, and Sørensen – Dice coefficient. After testing, the most effective algorithm is the Sørensen-Dice coefficient with a value of 0.88 for the value of precision, recall, and F1 score

Improved Feature Extraction and Similarity Algorithm for Video Object Detection

Video object detection is an important research direction of computer vision. The task of video object detection is to detect and classify moving objects in a sequence of images. Based on the static image object detector, most of the existing video object detection methods use the unique temporal correlation of video to solve the problem of missed detection and false detection caused by moving object occlusion and blur. Another video object detection model guided by an optical flow network is widely used. Feature aggregation of adjacent frames is performed by estimating the optical flow field. However, there are many redundant computations for feature aggregation of adjacent frames. To begin with, this paper improved Faster RCNN by Feature Pyramid and Dynamic Region Aware Convolution. Then the S-SELSA module is proposed from the perspective of semantic and feature similarity. Feature similarity is obtained by a modified SSIM algorithm. The module can aggregate the features of frames globally to avoid redundancy. Finally, the experimental results on the ImageNet VID and DET datasets show that the mAP of the method proposed in this paper is 83.55%, which is higher than the existing methods.

Research on online test question recommendation technology based on multi-dimensional user dynamic portrait

This paper proposes a research method of personalized recommendation of test questions for users by fully collecting and studying the user’s characteristic data in multiple dimensions, establishing a dynamic user portrait, and mining and analyzing the characteristic information of test questions in the test question bank. Firstly, collect the user’s data, calculate the user’s similarity by analyzing the user’s behavior sequence, and use the cosine similarity algorithm to calculate the text similarity of the test questions. Then, the tag system is established to analyze the user activity, and the TF-IDF algorithm is used to calculate the user’s proficiency in different keyword classification questions. Based on the above information, a hybrid recommendation mechanism is adopted to provide users with personalized test question recommendation services.

An intelligent medical guidance and recommendation model driven by patient-physician communication data.

Based on the online patient-physician communication data, this study used natural language processing and machine learning algorithm to construct a medical intelligent guidance and recommendation model. First, based on 16,935 patient main complaint data of nine diseases, this study used the word2vec, long-term and short-term memory neural networks, and other machine learning algorithms to construct intelligent department guidance and recommendation model. Besides, taking ophthalmology as an example, it also used the word2vec, TF-IDF, and cosine similarity algorithm to construct an intelligent physician recommendation model. Furthermore, to recommend physicians with better service quality, this study introduced the information amount of physicians' feedback to the recommendation evaluation indicator as the text and voice service quality. The results show that the department guidance model constructed by long-term and short-term memory neural networks has the best effect. The precision is 82.84%, and the F1-score is 82.61% in the test set. The prediction effect of the LSTM model is better than TextCNN, random forest, K-nearest neighbor, and support vector machine algorithms. In the intelligent physician recommendation model, under certain parameter settings, the recommendation effect of the hybrid recommendation model based on similar patients and similar physicians has certain advantages over the model of similar patients and similar physicians.