Jaccard Similarity Measure Research Articles

A novel approach to text clustering using shift k-medoid

As the amount of data is growing day by day, there is a need to convert it into some effective manner so as to extract some useful information from huge data. Text mining is used to perform this task. In this paper, text clustering is used to convert the large data into different cluster forms to extract the meaningful information for the purpose of analysis so as to get the summarised data. Three partitioning-based clustering techniques, i.e., k-means, k-means fast and k-medoids are compared, and a new algorithm named shift k-medoid is proposed, which is hybrid of k-medoid and mean shift clustering algorithms. Cosine similarity, correlation coefficient and Jaccard similarity measures are used to check the performance of the algorithms and two measures, i.e., randomised feature and normalised mutual information (NMI) feature are used to test the accuracy of the algorithms. The outcomes demonstrate that the best performance is accomplished by using the proposed algorithm.

A novel approach to text clustering using shift k-medoid

As the amount of data is growing day by day, there is a need to convert it into some effective manner so as to extract some useful information from huge data. Text mining is used to perform this task. In this paper, text clustering is used to convert the large data into different cluster forms to extract the meaningful information for the purpose of analysis so as to get the summarised data. Three partitioning-based clustering techniques, i.e., k-means, k-means fast and k-medoids are compared, and a new algorithm named shift k-medoid is proposed, which is hybrid of k-medoid and mean shift clustering algorithms. Cosine similarity, correlation coefficient and Jaccard similarity measures are used to check the performance of the algorithms and two measures, i.e., randomised feature and normalised mutual information (NMI) feature are used to test the accuracy of the algorithms. The outcomes demonstrate that the best performance is accomplished by using the proposed algorithm.

Molecular epidemiology of ESBL-producing E. coli and K. pneumoniae: establishing virulence clusters.

ObjectiveTo genetically characterize clusters of virulence factors (VFs) among extended spectrum β-lactamase (ESBL)-producing Escherichia coli and Klebsiella pneumoniae and assess whether these clusters are associated with genetic determinants or clinical outcomes.MethodsOne hundred forty-eight E. coli and 82 K. pneumoniae clinical isolates were obtained from 213 patients in Paris, France. Isolates underwent ESBL characterization, MultiLocus Sequence Typing (MLST) typing and phylogenetic group identification. Detection of ten E. coli and seven K. pneumoniae VF-encoding genes were assessed, from which a k-medians partition algorithm with Jaccard similarity measure was used to construct clusters.ResultsCTX-M was the predominant ESBL and susceptibility to trimethoprim–sulfamethoxazole (32%), ciprofloxacin (22%) and aminoglycosides (32%) was low. In E. coli, there were five identified clusters, with significantly different distributions of ESBL-sequence type (P<0.001), ST131 (P<0.001) and phylogenetic group (P=0.001) between clusters. “Siderophore exclusive”, “siderophore exclusive with iroN ” and “adhesin sfa/papGIII-rich” clusters had higher 12-month mortality rates compared to others (49% vs 22%, respectively, P=0.02). In K. pneumoniae, three different clusters, with significantly different distributions of aminoglycoside-sensitivity (P<0.004), MLST-type (P<0.001) and relaxase plasmids (P=0.001) were described.ConclusionDistinct clusters of E. coli and K. pneumoniae VFs are observed within ESBL-producing isolates and are strongly associated with several genetic determinants. Their association with overall morbidity and mortality requires further evidence.

Qualitative topological relations between interval type‐2 fuzzy spatial objects

AbstractThe topological relations between fuzzy objects are crucial in geographical spatial analysis. A qualitative topological relationship analysis method for interval type‐2 fuzzy spatial objects (IT2 FSOs) is proposed in this article. Similarities between the interval type‐2 fuzzy topological relation matrices and the corresponding crisp topological relation matrices are calculated using Jaccard's similarity measure and sorted to identify the two major relations. Next, the proximity degrees and quantifiers that correspond to these two major relations can be determined. Thus, two methods can be used to describe the relation between two IT2 FSOs; the method proposed in this article conforms to human cognitive habits and meets the requirements of a variety of applications in geographical information systems (GIS). Users can select different boundary definitions, intersection matrix types, and calculation functions for each cell in the intersection matrix, as well as different description methods according to the requirements of different GIS applications.

Pixel-Label-Based Segmentation of Cross-Sectional Brain MRI Using Simplified SegNet Architecture-Based CNN.

Using deep neural networks for segmenting an MRI image of heterogeneously distributed pixels into a specific class assigning a label to each pixel is the concept of the proposed approach. This approach facilitates the application of the segmentation process on a preprocessed MRI image, with a trained network to be utilized for other test images. As labels are considered expensive assets in supervised training, fewer training images and training labels are used to obtain optimal accuracy. To validate the performance of the proposed approach, an experiment is conducted on other test images (available in the same database) that are not part of the training; the obtained result is of good visual quality in terms of segmentation and quite similar to the ground truth image. The average computed Dice similarity index for the test images is approximately 0.8, whereas the Jaccard similarity measure is approximately 0.6, which is better compared to other methods. This implies that the proposed method can be used to obtain reference images almost similar to the segmented ground truth images.

A Concise Proof of the Triangle Inequality for the Jaccard Distance

The Jaccard similarity measure is widely used in many applications, such as data mining and information retrieval [3] or even comparing DNA sequences [6]. The Jaccard similarity coefficient [2] of ...

Graph-based Representation for Sentence Similarity Measure : A Comparative Analysis

Textual data are a rich source of knowledge; hence, sentence comparison has become one of the important tasks in text mining related works. Most previous work in text comparison are performed at document level, research suggest that comparing sentence level text is a non-trivial problem. One of the reason is two sentences can convey the same meaning with totally dissimilar words. This paper presents the results of a comparative analysis on three representation schemes i.e. term frequency inverse document frequency, Latent Semantic Analysis and Graph based representation using three similarity measures i.e. Cosine, Dice coefficient and Jaccard similarity to compare the similarity of sentences. Results reveal that the graph based representation and the Jaccard similarity measure outperforms the others in terms of precision, recall and F-measures.

Sequence clustering approach for clustering web user session

Clustering web usage data is useful to discover interesting patterns pertaining to user traversals, behaviour and their usage characteristics. It is also useful for trend discovery as well as for building personalisation and recommendation engines. Since web is dynamic, clustering web user transactions results in arbitrary shapes. Moreover, users accesses web pages in an order in which they are interested and hence incorporating sequence nature of their usage is crucial for clustering web transactions. In this paper, we present an approach to cluster web usage sequence data and removing noise using DBSCAN algorithm. We also study the impact of clustering process when both sequence and content information is incorporated while computing similarity measure. We use sequence and set similarity (S3M) measure to capture both the order of occurrence of page visits and the page information itself, and compared the results with Euclidean distance and Jaccard similarity measures. The inter-cluster and intra-cluster distances are computed using average Levensthein distance (ALD) to demonstrate the usefulness of the proposed approach in the context of web usage mining.

Vector similarity measures of hesitant fuzzy linguistic term sets and their applications.

In decision making, similarity measure and distance between two objects are crucial to be able to determine the relationship between those objects. Many researchers have received much attention for their research on this subject. In this study, we propose two novel similarity measures between hesitant fuzzy linguistic term sets (HFLTSs). In addition, two extensions of Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) are proposed in the hesitant fuzzy linguistic environments. Furthermore, an example of an application concerning traditional Chinese medical diagnosis and an MCDM problem have been given to illustrate the applicability and validation of these similarity measures of HFLTSs. Furthermore, the results of examples demonstrate that the Dice and Jaccard similarity measures are more reasonable than the cosine similarity measure with respect to HFLTSs.

Forecasting Model Based on Neutrosophic Logical Relationship and Jaccard Similarity

The daily fluctuation trends of a stock market are illustrated by three statuses: up, equal, and down. These can be represented by a neutrosophic set which consists of three functions—truth-membership, indeterminacy-membership, and falsity-membership. In this paper, we propose a novel forecasting model based on neutrosophic set theory and the fuzzy logical relationships between the status of historical and current values. Firstly, the original time series of the stock market is converted to a fluctuation time series by comparing each piece of data with that of the previous day. The fluctuation time series is then fuzzified into a fuzzy-fluctuation time series in terms of the pre-defined up, equal, and down intervals. Next, the fuzzy logical relationships can be expressed by two neutrosophic sets according to the probabilities of different statuses for each current value and a certain range of corresponding histories. Finally, based on the neutrosophic logical relationships and the status of history, a Jaccard similarity measure is employed to find the most proper logical rule to forecast its future. The authentic Taiwan Stock Exchange Capitalization Weighted Stock Index (TAIEX) time series datasets are used as an example to illustrate the forecasting procedure and performance comparisons. The experimental results show that the proposed method can successfully forecast the stock market and other similar kinds of time series. We also apply the proposed method to forecast the Shanghai Stock Exchange Composite Index (SHSECI) to verify its effectiveness and universality.

Multicriteria Decision-Making Method Based on Similarity Measures under Single-Valued Neutrosophic Refined and Interval Neutrosophic Refined Environments

In this paper, we propose three similarity measure methods for single-valued neutrosophic refined (SVNR) sets and interval neutrosophic refined (INR) sets based on Jaccard, Dice, and Cosine similarity measures of SVN-sets and interval neutrosophic sets. Furthermore, we suggest two multicriteria decision-making (MCDM) methods under SVNR environment and INR environment, and give applications of proposed MCDM methods. Finally, we suggest a consistency analysis method for proposed similarity measures between INR-sets and give an application to demonstrate process of the method.

A Generic Method for Accelerating LSH-Based Similarity Join Processing

Locality sensitive hashing (LSH) is an efficient method for solving the problem of approximate similarity search in high-dimensional spaces. Through LSH, a high-dimensional similarity join can be processed in the same way as hash join, making the cost of joining two large datasets linear. By judicially analyzing the properties of multiple LSH algorithms, we propose a generic method to speed up the process of joining two large datasets using LSH. The crux of our method lies in the way which we identify a set of representative points to reduce the number of LSH lookups. Theoretical analyzes show that our proposed method can greatly reduce the number of lookup operations and retain the same result accuracy compared to executing LSH lookups for every query point. Furthermore, we demonstrate the generality of our method by showing that the same principle can be applied to LSH algorithms for three different metrics: the Euclidean distance (QALSH), Jaccard similarity measure (MinHash), and Hamming distance (sequence hashing). Results from experimental studies using real datasets confirm our error analyzes and show significant improvements of our method over the state-of-the-art LSH method: to achieve over 0.95 recall, we only need to operate LSH lookups for at most 15 percent of the query points.

Estimation of the joint roughness coefficient (JRC) of rock joints by vector similarity measures

Accurate determination of joint roughness coefficient (JRC) of rock joints is essential for evaluating the influence of surface roughness on the shear behavior of rock joints. The JRC values of rock joints are typically measured by visual comparison against Barton’s standard JRC profiles. However, its accuracy is strongly affected by personal bias. In the present study, a new comparison method is proposed for JRC evaluation to overcome the drawback of conventional visual comparison methods based on vector similarity measures (VSMs). The feature vectors are obtained by analyzing the angular variation of line segments of both standard JRC profiles and test profiles obtained from three kinds of natural rocks with a sampling interval of 0.5 mm. The roughness similarity degrees between test profiles and standard profiles are evaluated by the Jaccard, Dice, and cosine similarity measures. The JRC values of the test profiles are then determined according to the maximum relation index based on the similarity degrees. In the present study, a comparative analysis between the VSMs method and the JRC evaluation method using different roughness parameters demonstrated that the VSMs method is effective and accurate for JRC measurement.

Multi-criteria decision making method based on similarity measures under single-valued neutrosophic refined and interval neutrosophic refined environments

In this paper, we propose three similarity measure methods for single-valued neutrosophic refined sets and interval neutrosophic re¯ned sets based on Jaccard, Dice and Cosine similarity measures of single-valued neutrosophic sets and interval neutrosophic sets. Furthermore, we suggest two multi-criteria decision making methods under single-valued neutrosophic refined environment and interval neutrosophic refined environment, and give applications of proposed multi-criteria decision making methods. Finally we suggest a consistency analysis method for proposed similarity measures between interval neutrosophic refined sets and give an application to demonstrate process of the method.

Improved similarity measure in neutrosophic environment and its application in finding minimum spanning tree

Minimum spanning tree finds its huge application in network designing, approximation algorithms for NP-hard problems, clustering problems and many more. Many research works have been done to find minimum spanning tree due to its various applications. But, till date very few research works are available in finding minimum spanning tree in neutrosophic environment. This paper contributes significantly by defining the weight of each network edge using single valued neutro- sophic set (SVNS) and introduce a new approach using similarity measure to find minimum spanning tree in neutrosophic environment. Use of SVNS makes the problem realistic as it can describe the uncertainty, indeterminacy and hesitancy of the real world in a better way. We introduce two new and simple similarity measures to overcome some disadvantages of existing Jaccard, Dice and Cosine similarity measures of SVNSs for ranking the alternatives. Further from the similarity measures we have developed two formulas for the entropy measure proving a fundamental relation between similarity measure and entropy measure. The new entropy measures define the uncertainty more explicitly in comparison to other entropy measure existing in the literature which has been established using an example.

Mathematical properties of soft cardinality: Enhancing Jaccard, Dice and cosine similarity measures with element-wise distance

The soft cardinality function generalizes the concept of counting measure of the classic cardinality of sets. This function provides an intuitive measure of the amount of elements in a collection (i.e. a set or a bag) exploiting the similarities among them. Although soft cardinality was first proposed in an ad-hoc way, it has been successfully used in various tasks in the field of natural language processing. In this paper, a formal definition of soft cardinality is proposed together with an analysis of its boundaries, monotonicity property and a method for constructing similarity functions. Additionally, an empirical evaluation of the model was carried out using synthetic data.

An activation detection based similarity measure for intuitionistic fuzzy sets

Intuitionistic fuzzy sets (IF-sets), with mechanisms to represent both the degree of membership and hesitancy of a given entity with respect to a concept under consideration, have been proven to be a useful extension to Zadeh's fuzzy set theory. Noteworthy efforts by various researchers have been devoted to defining a robust similarity measure for a given pair of IF-sets, as we often need to quantify the similarity between given entities in application domains ranging from medical diagnosis to multiple criteria decision making. These efforts have shown that it is highly non-trivial to construct a truly robust IF-set similarity measure with easy-to-understand interpretations. In this article, grounded on native concepts from activation detection in medical image analysis, a model for determining the degree of similarity between IF-sets is proposed. An IF-set similarity measure (termed the activation detection based similarity measure) is then systematically built from this model. We show that the proposed measure produces results that are intuitively appealing, easy to understand, and can be robustly interpreted. Moreover, we demonstrate that the proposed measure obeys standard conventions regarding set definition in the classical setting, and is equivalent to the Jaccard's similarity measure as we transition from the intuitionistic fuzzy setting to the classical setting. The source code of the numerical implementation of the proposed measure is available from the author upon request.

Service Recommendation Based on Ranking Using Keywords in Hadoop

Recommendation system acts as a tool in providing the most appropriate service to the user. Currently, information through online services is increased. This leads to the overhead of data in online and there is a possibility of getting less accurate prediction. In previous approaches, recommendation of services does not consider the suggestion of the user at a time, was in need of searching for the particular service. The proposed system deals with the implementation of personalized recommendation to provide services for hotel reservation system. Candidate service is created as the combination of keyword list and Domain Thesaurus which consist of semantically annotated words. Preferences are collected from the active user about particular service for each application. Similar user’s opinions are taken from the reviews using keyword extraction method. Similarity is calculated between user preferences with reviews of the previous user using jaccard and cosine similarity measures. From this most similar keywords are provided to the user as a recommended service using MapReduce framework. It outperforms about 8% when compared with previous approaches, in providing the accurate prediction of relevant service to the active user.

An Ontology Based Framework for Intelligent Web Based e-Learning

E-Learning is a fast, just-in-time, and non-linear learning process, which is now widely applied in distributed and dynamic environments such as the World Wide Web. Ontology plays an important role in capturing and disseminating the real world knowledge for effective human computer interactions. However, engineering of domain ontologies is very labor intensive and time consuming. Some machine learning methods have been explored for automatic or semi-automatic discovery of domain ontologies. Nevertheless, both the accuracy and the computational efficiency of these methods need to be improved. While constructing large scale ontology for real-world applications such as e-learning, the ability to monitor the progress of students' learning performance is a critical issue. In this paper, a system is proposed for analyzing students' knowledge level obtained using Kolb's classification based on the students level of understanding and their learning style using cluster analysis. This system uses fuzzy logic and clustering algorithms to arrange their documents according to the level of their performance. Moreover, a new domain ontology discovery method is proposed uses contextual information of the knowledge sources from the e-Learning domain. This proposed system constructs ontology to provide an effective assistance in e-Learning. The proposed ontology discovery method has been empirically tested in an e-Learning environment for teaching the subject Database Management Systems. The salient contributions of this paper are the use of Jaccard Similarity measure and K-Means clustering algorithm for clustering of learners and the use of ontology for concept understanding and learning style identification. This helps in adaptive e-learning by providing suitable suggestions for decision making and it uses decision rules for providing intelligent e-Learning.

Detecting overlapping community in complex network based on node similarity

Overlapping communities in complex network is a common phenomenon in real world network. The overlapping community structure can more accurately obtain the actual structure information in the network. But at present the study of overlapping community division algorithm is relatively less, facing the problems of the low accurate rate. Based on this, this paper presents algorithms OCNS for detecting community overlapping base on node similarity. The algorithm calculates similarity between two nodes in the network by means of Jaccard similarity measure formula. Then the related nodes are adaptive merged according to the similarity value, combining with the community according to the change of modularity. The process of partitioning can not only accurately merge closely linked nodes in the network, but also find the overlapping nodes and bridge nodes between communities. The experiment proved the algorithm is effective to detect the overlapping community and has obvious advantages in the division of baseline social network Zachary and dolphin network, and the quality of division better than other existing partitioning algorithm.