Model For Document Clustering Research Articles

Topics in the Haystack: Enhancing Topic Quality through Corpus Expansion

Abstract Extracting and identifying latent topics in large text corpora has gained increasing importance in Natural Language Processing (NLP). Most models, whether probabilistic models similar to Latent Dirichlet Allocation (LDA) or neural topic models, follow the same underlying approach of topic interpretability and topic extraction. We propose a method that incorporates a deeper understanding of both sentence and document themes, and goes beyond simply analyzing word frequencies in the data. Through simple corpus expansion, our model can detect latent topics that may include uncommon words or neologisms, as well as words not present in the documents themselves. Additionally, we propose several new evaluation metrics based on intruder words and similarity measures in the semantic space. We present correlation coefficients with human identification of intruder words and achieve near-human level results at the word-intrusion task. We demonstrate the competitive performance of our method with a large benchmark study, and achieve superior results compared to state-of-the-art topic modeling and document clustering models.

Deep document clustering via adaptive hybrid representation learning

Document clustering plays an important role in text mining and machine learning. Recently, various studies have indicated that a hybrid representation contributes to document clustering tasks. Adaptively adjusting representation learning to make each representation of documents mutually reinforced remains an open problem. In this study, we propose a deep document clustering model, DCAHR, to improve the document clustering performance by adaptively learning a hybrid representation. Specifically, an adaptive representation enhanced network (AREN) is designed using a composite Gated Linear Units (GLU) mechanism to adaptively capture the consistent information of semantic and structure representations, and utilize it to adaptively adjust each representation learning for studying the corresponding enhanced representations. A hybrid representation was then obtained by normalizing the two enhanced representations. Furthermore, an adaptive joint objective function was developed to learn document partitioning and supervise the entire model by adaptively learning the weights of each objective function. Experimental results on nine commonly used real-world datasets demonstrated the outstanding performance of the proposed DCAHR model.

Improved Meta-Heuristic Model for Text Document Clustering by Adaptive Weighted Similarity

This paper intends to develop a novel framework for text document clustering with the aid of a new improved meta-heuristic algorithm. Initially, the features are selected from the text document by subjecting each word under Term Frequency-Inverse Document Frequency (TF-IDF) computation. Subsequently, centroid selection plays a vital role in cluster formation, which is done using a new Improved Lion Algorithm (LA) termed as Cross over probability-based LA model (CP-LA). As a novelty, this paper introduced a new inter and intracluster similarity model. Moreover, this centroid selection is made in such a way that the proposed adaptive weighted similarity should be minimal. Based on the characteristics of the document, the weights are automatically adapted with the similarity measure. The proposed adaptive weighted similarity function involves the inter-cluster, and intra-cluster similarity of both ordered and unordered documents. Finally, the superiority of the proposed over other models is proved under different performance measures.

Intention-guided deep semi-supervised document clustering via metric learning

The intention expresses the user’s preference for document structure division. Intention-guided document structure division is an important task in the field of text mining. To achieve this goal, deep semi-supervised document clustering provides a promising solution to personalized document clustering. However, traditional deep semi-supervised clustering models suffer from the problem of the limited number of constraints which is insufficient for intention-guided document clustering. Moreover, documents normally have various emphases on their representations to reflect different structural opinions. In this paper, we proposed an intention-guided deep semi-supervised document clustering model, namely IGSC, to divide document structure based on a small amount of user-provided supervised information. IGSC designs a deep metric learning network to solve the above problems. The deep metric learner explores the user’s global intention and outputs an intention matrix. The intention is explored from the small amount user provided pairwise constraints and is used to guide the representation learning. Moreover, IGSC uses the intention matrix to guide the clustering process, to get the clustering results that best meet the user’s intention. This paper compares IGSC with a number of document clustering models on four real-world text datasets, namely Reu-10k, BBC, ACM, and Abstract. The results show that IGSC evidently improves the clustering performance and outperforms the best result of benchmark models with 7% on average. The comparison with other models and the visualization results can demonstrate that IGSC is effective.

Construction of suffix tree using key phrases for document using down-top incremental conceptual hierarchical text clustering approach

With development of technologies in the World Wide Web, usage of document increases day by day. In order to access the document easily, document clustering technique is introduced. In the field of data mining, document clustering plays a vital role. Organizing the unstructured and unlabeled document is one of the major problems and it is ever growing and complex. Handling of such unorganized documents causes more expensive. Hence, challenges raised by the continuing growth of unstructured and unlabeled documents are handled in this proposed work. Document clustering is one of the most powerful methods to solve the problem of organizing unstructured documents. There are numerous clustering methods available. In this we were proposed phrase-based clustering algorithm, which is based on the applications of suffix tree document clustering model. The proposed algorithm is designed to use the suffix tree document clustering (STDC) model for accurate representation of document and similarity measurement of the similar documents. This proposed algorithm gives more than 90% accuracy.

Hierarchical Dirichlet Multinomial Allocation Model for Multi-Source Document Clustering

Mining a document structure from multiple data sources in terms of their underlying topics has become an important task of document clustering. The traditional document clustering approach cannot be applied directly to the multi-source document clustering problem. There are three typical difficulties: 1) The topics of different data sources are related but not the same. 2) Usually, each data source has its own focus on topics. 3) The number of clusters of the data sources are not necessarily the same and are not known beforehand. In this paper, based on our previous research, we design a novel multi-source document clustering model, namely, the hierarchical Dirichlet multinomial allocation (HDMA) model, to solve all the above problems. The HDMA model is investigated with a two-step hierarchical topic generation process. Topics are learnt to share their general characteristics across data source, while at the same time preserve the local characteristics of the data source. Each data source is applied with an exclusive topic partition to learn the source-level topic emphasis. A Gibbs sampling algorithm is then used to learn the number of clusters for each data source as well as the parameters of the HDMA model at the same time. Experimental results demonstrate that the HDMA model is effective.

Clustering Scientific Document Based on an Extended Citation Model

With the number of published scientific paper increasing exponentially, scientific document clustering is becoming a challenging task. Therefore, a scientific document clustering model with high quality is needed. In this paper, we propose an extended citation model for scientific document clustering. On the one hand, the proposed model considers that 1) the high frequency and the wide distribution of a scientific document cited in other documents will result in the high similarity between the citing and the cited documents; and 2) the close location of two scientific documents cited in a scientific document will also result in the high similarity between these two documents. On the other hand, the proposed model combines a citation networks and textual similarity network to enhance the performance of scientific document clustering. To evaluate the performance of our proposed model, we collect scientific documents from PMC and PubMed databases in the field of oncology as a case study. It is proved that our proposed model can obtain reasonably clustering results by comparing it with traditional scientific documents clustering models, such as traditional bibliographic coupling model and textual similarity model, according to the indices of precision, recall, and F1-score.

An novel cluster based feature selection and document classification model on high dimension trec data

TREC text documents are complex to analyze the features its relevant similar documents using the traditional document similarity measures. As the size of the TREC repository is increasing, finding relevant clustered documents from a large collection of unstructured documents is a challenging task. Traditional document similarity and classification models are implemented on homogeneous TREC data to find essential features for document entities that are similar to the TREC documents. Also, most of the traditional models are applicable to limited text document sets for text analysis. The main issues in the traditional text mining models in TREC repository include :1) Each document is represented in vector form with many sparsity values 2) Failed to find the document semantic similarity between the intra and inter clusters 3) High mean squared error rate. In this paper, novel feature selection based clustered and classification model is proposed on large number of different TREC repositories. Traditional latent Semantic Indexing and document clustering models are failed to find the topic relevance on large number of TREC clinical text document sets due to computational memory and time. Proposed document feature selection and clustered based classification model is applied on TREC clinical benchmark datasets. From the experimental results, it is proved that the proposed model is efficient than the existing models in terms of computational memory, accuracy and error rate are concerned.

Document representation and clustering models for bilingual documents clustering

ABSTRACTCurrently, the internet has created many documents in languages other than English. People face challenges when seeking and using information; for example, non‐native English‐speaking students tend to have problems when utilizing libraries in North American universities. To help people efficiently organize information, bilingual documents clustering has advantages for practical utilization, it can divide documents into groups with the same topic and there is no need for a training dataset. Document representation and clustering models are two important parts in clustering. This paper compares four popular representation methods, vector space model (VSM), latent semantic indexing (LSI), latent Dirichlet allocation (LDA) and doc2vec (D2V), together with four different types of clustering algorithms, K‐means++, BIRCH, DBSCAN and affinity propagation (AP) to identify appropriate combinations for bilingual documents clustering. Parallel corpus and comparable corpus are all used for the bilingual datasets. Experimental results show that, clustering performance varies when combining different representation methods with clustering algorithms. It's important to make good choice of models for better documents organization.

Design and Develop Semantic Textual Document Clustering Model

Design and Develop Semantic Textual Document Clustering Model SK Ahammad Fahad, Wael Mohamed Shaher Yafooz Abstract The utilization of textual documents is spontaneously increasing over the internet, email, web pages, reports, journals, articles and they stored in the electronic database format. It is challenging to find and access these documents without proper classification mechanisms. To overcome such difficulties we proposed a semantic document clustering model and develop this model. The document pre-processing steps, semantic information from WordNet help us to be bioavailable the semantic relation from raw text. By reminding the limitation of traditional clustering algorithms on the natural language, we consider semantic clustering by COBWEB conceptual clustering. Clustering quality and high accuracy were one of the most important aims of our research, and we chose F-Measure evaluation for ensuring the purity of clustering. However, there still exist many challenges, like the word, high spatial property, extracting core linguistics from texts, and assignment adequate description for the generated clusters. By the help of Word Net database, we eliminate those issues. In this research paper, there have a proposed framework and describe our development evaluation with evaluation. Full Text: PDF DOI: 10.15640/jcsit.v5n2a4

MODC: Multi-Objective Distance based Optimal Document Clustering by GA

Background/Objective: Unsupervised learning of text documents is an essential and significant process of knowledge discovery and data mining. The concept, context and semantic relevancy are the important and exclusive factors in text mining, where as in the case of unsupervised learning of record structured data, these factors are not in scope. Methods/Statistical Analysis: The current majority of benchmarking document clustering models is keen and relies on term frequency, and all these models are not considering the concept, context and semantic relations during document clustering. In regard to this, our earlier works introduced a novel document clustering approaches and one of that named as Document Clustering by Conceptual, Contextual and Semantic Relevance (DC3SR). The lessons learned from the empirical study of this contribution motivated us to propose aMulti-Objective Distance based optimal document Clustering (MODC) approach that optimizes resultant clusters using the well-known evolutionary computation technique called Genetic Algorithm.Findings: The significant contribution of this proposal is feature formation by concept, context and semantic relevance and optimizing resultant clusters by genetic algorithm. An unsupervised learning approach to form the initial clusters that estimates similarity between any two documents by concept, context and semantic relevance score and further optimizes by genetic algorithm is proposed. This novel method represents the concept as correlation between arguments and activities in given documents, context as correlation between meta-text of the documents and the semantic relevance is assessed by estimating the similarity between documents through the hyponyms of the arguments. The meta-text of the documents considered for context assessment contains the authors list, keywords list and list of document versioning time schedules. Application/Improvements:The experiments were conducted to assess the significance of the proposed model.The results obtained from experiments concluding that the MODC is performing exceptionally well under divergent document count and evincing the cluster formation accuracy as 97%. The dimensionality reduction by concept, context and semantic relevance is left for future enhancement of the proposed model.

Advances in Document Clustering with Evolutionary-Based Algorithms

Document clustering is the process of organizing a particularelectronic corpus of documents into subgroups of similar text features.Formerly, a number of conventional algorithms had been applied to performdocument clustering. There are current endeavors to enhance clusteringperformance by employing evolutionary algorithms. Thus, such endeavors becamean emerging topic gaining more attention in recent years. The aim of this paperis to present an up-to-date and self-contained review fully devoted to documentclustering via evolutionary algorithms. Itfirstly provides a comprehensive inspection to the document clustering model revealingits various components with its related concepts. Then it shows and analyzesthe principle research work in this topic. Finally, it compiles and classifiesvarious objective functions, the core of the evolutionary algorithms, from therelated collection of research papers. The paper ends up by addressing someimportant issues and challenges that can be subject of future work.

Subject based Clustering for Digital Forensic Investigation with Subject Suggestion

digital forensics has become a prominent activity in crime investigation since computers are increasingly used as tools to commit crimes. During forensic investigation the digital devices such as desktops, notebooks, smart phones etc. found at the crime scene are collected for further investigation. Investigators have to go through humongous amount of data stored on these devices to gather evidence. This activity exceeds the expert's ability of analyzing and interpreting the data. In this context data mining techniques such as clustering are used for automated data analysis. This research work focuses on a novel document clustering model that allows an investigator to semantically cluster the documents stored on a suspect's digital devices with the help of subject suggestions initially provided to him. Providing subject suggestion improves the accuracy and speeds up the process of searching the evidence. Without subject suggestion, the investigators are heedless about the suspect's dataset and fail to give appropriate search query which may delay the process of investigation.

Dirichlet Process Mixture Model for Document Clustering with Feature Partition

Finding the appropriate number of clusters to which documents should be partitioned is crucial in document clustering. In this paper, we propose a novel approach, namely DPMFP, to discover the latent cluster structure based on the DPM model without requiring the number of clusters as input. Document features are automatically partitioned into two groups, in particular, discriminative words and nondiscriminative words, and contribute differently to document clustering. A variational inference algorithm is investigated to infer the document collection structure as well as the partition of document words at the same time. Our experiments indicate that our proposed approach performs well on the synthetic data set as well as real data sets. The comparison between our approach and state-of-the-art document clustering approaches shows that our approach is robust and effective for document clustering.

Subject-based semantic document clustering for digital forensic investigations

Computers are increasingly used as tools to commit crimes such as unauthorized access (hacking), drug trafficking, and child pornography. The proliferation of crimes involving computers has created a demand for special forensic tools that allow investigators to look for evidence on a suspect's computer by analyzing communications and data on the computer's storage devices. Motivated by the forensic process at Sûreté du Québec (SQ), the Québec provincial police, we propose a new subject-based semantic document clustering model that allows an investigator to cluster documents stored on a suspect's computer by grouping them into a set of overlapping clusters, each corresponding to a subject of interest initially defined by the investigator.

Phrase based Clustering Scheme of Suffix Tree Document Clustering Model

Document clustering is one of the difficult and recent research fields in the search engine research. Most of the existing documents clustering techniques use a group of keywords from each document to cluster the documents. Document clustering arises from information retrieval domains, and “It finds grouping for a set of documents belonging to the same cluster are similar and documents belongs to the different cluster are dissimilar”. The nformation retrieval plays an important role in data mining for extracting the relevant information for related to user request. Information retrieval finds the file contents and identifies their similarity. It measures the performance of the documents by using the precision and recall. In this paper we proposed a phrase based clustering scheme which based on application of Suffix Tree Document Clustering (STDC) model. The proposed algorithm is designed to use the STDC model for accurate equivalent representation of document and similarity measurement of the similar documents. This method of clustering reduces the grouping time and similarity accuracy as compared to other existing methods.

A nonnegative matrix factorization framework for semi-supervised document clustering with dual constraints

In this paper, we propose a new semi-supervised co-clustering algorithm Orthogonal Semi-Supervised Nonnegative Matrix Factorization (OSS-NMF) for document clustering. In this new approach, the clustering process is carried out by incorporating both prior domain knowledge of data points (documents) in the form of pair-wise constraints and category knowledge of features (words) into the NMF co-clustering framework. Under this framework, the clustering problem is formulated as the problem of finding the local minimizer of objective function, taking into account the dual prior knowledge. The update rules are derived, and an iterative algorithm is designed for the co-clustering process. Theoretically, we prove the correctness and convergence of our algorithm and demonstrate its mathematical rigorous. Our experimental evaluations show that the proposed document clustering model presents remarkable performance improvements with those constraints.

A Similarity Rough Set Model for Document Representation and Document Clustering

Document clustering is a textmining technique for unsupervised document organization. It helps the users browse and navigate large sets of documents. Ho et al. proposed a Tolerance Rough Set Model (TRSM) [1] for improving the vector space model that represents documents by vectors of terms and applied it to document clustering. In this paper we analyze their model to propose a new model for efficient clustering of documents. We introduce Similarity Rough Set Model (SRSM) as another model for presenting documents in document clustering. The model is evaluated by experiments on test collections. The experiment results show that the SRSM document clusteringmethod outperforms the one with TRSM and the results of SRSM are less affected by the value of parameter than TRSM.

개체명을 이용한 주제기반 웹 문서 클러스터링

종래의 클러스터링 기법은 단순히 키워드를 추출에 기반한 단어간 유사도에 의한 그룹핑 방식을 구사함으로써 비교해야 할 대상 키워드 수 및 종류가 매우 다양하여 계산량이 증가함으로써 속도가 느리고 정확도도 높지 않은 편이다. 본 논문은 이러한 단점을 해소하기 위해 웹 문서를 대상으로 기존 명사 위주의 키워드 뿐 아니라 인명, 지명, 회사명, 물품명 등을 자동으로 인식하는 개체명 인식 결과를 이용하는 웹클러스터링 기법을 제안하고자 한다. 실험을 통해 기존 키워드 기반 클러스터링 결과에 비해 개체명 기반클러스터링의 품질이 우수함을 증명하였으며, 문서 집합 특성에 따른 클러스터링 결과도 비교 분석하였다. Past clustering researches are focused on extraction of keyword for word similarity grouping. However, too many candidates to compare and compute bring high complexity, low speed and low accuracy. To overcome these weaknesses, this paper proposed a topical web document clustering model using not only keyword but also named entities such as person name, organization, location, and so on. By several experiments, we prove effects of our model compared with traditional model based on only keyword and analyze how different effects show according to characteristics of document collection.

An online document clustering technique for short web contents

Document clustering techniques have been applied in several areas, with the web as one of the most recent and influential. Both general-purpose and text-oriented techniques exist and can be used to cluster a collection of documents in many ways. This work proposes a novel heuristic online document clustering model that can be specialized with a variety of text-oriented similarity measures. An experimental evaluation of the proposed model was conducted in the e-commerce domain. Performances were measured using a clustering-oriented metric based on F-Measure and compared with those obtained by other well-known approaches. The obtained results confirm the validity of the proposed method both for batch scenarios and online scenarios where document collections can grow over time.