Text Classification System Research Articles

워드넷 기반 특징 추상화를 통한 웹문서 자동분류시스템의 성능향상

본 논문은 기계학습 기법에 기반한 웹문서 자동분류 시스템의 성능을 높이기 위한 새로운 형태의 특징가공 기법을 제안한다. 제안 기법은 하이퍼텍스트 웹문서에 대한 자동분류를 효과적으로 수행하기 위해 하이퍼링크 관계를 활용하여 특징 집합을 확장시킨다. 웹문서는 하이퍼링크 관계를 통해 서로 연결된 구조를 가지며, 그 관계는 많은 경우 연관도가 높은 문서들 간에 존재한다. 이러한 링크 정보가 분류모델의 주요 인자가 되는 특징 집합의 질을 높이는 중요한 역할을 수행할 수 있다. 제안 기법의 기본 아이디어는 워드넷 온톨로지를 기반으로 분류 대상 문서와 인접 문서들에 포함된 단어(특징)들 간의 의미적 유사도를 평가함으로써 다수의 특징들로 구성된 추상화된 개념적 특징을 생성하는 것이다. 여기서 유사도 함수는 워드넷 안에서 특징들 간의 상/하위어 관계 정보를 정량적으로 계산하게 된다. 분류모델의 구축시 추상화된 개념 특징은 일반 특징과 동일하게 간주하여 보다 정확한 분류 모델을 구축하는데 기여한다. Web-KB 문서집합을 이용한 실험을 통해 제안 기법이 기존 기법 보다 우수함을 보였다. This paper presents a novel feature engineering technique that can improve the conventional machine learning-based text classification systems. The proposed method extends the initial set of features by using hyperlink relationships in order to effectively categorize hypertext web documents. Web documents are connected to each other through hyperlinks, and in many cases hyperlinks exist among highly related documents. Such hyperlink relationships can be used to enhance the quality of features which consist of classification models. The basic idea of the proposed method is to generate a sort of ed concept feature which consists of a few raw feature words; for this, the method computes the semantic similarity between a target document and its neighbor documents by utilizing hierarchical relationships in the WordNet ontology. In developing classification models, the ed concept features are equated with other raw features, and they can play a great role in developing more accurate classification models. Through the extensive experiments with the Web-KB test collection, we prove that the proposed methods outperform the conventional ones.

Collaborative Filtering Recommender Systems

Recommender Systems are software tools and techniques for suggesting items to users by considering their preferences in an automated fashion. The suggestions provided are aimed at support users in various decision- making processes. Technically, recommender system has their origins in different fields such as Information Retrieval (IR), text classification, machine learning and Decision Support Systems (DSS). Recommender systems are used to address the Information Overload (IO) problem by recommending potentially interesting or useful items to users. They have proven to be worthy tools for online users to deal with the IO and have become one of the most popular and powerful tools in E-commerce. Many existing recommender systems rely on the Collaborative Filtering (CF) and have been extensively used in E-commerce .They have proven to be very effective with powerful techniques in many famous E-commerce companies. This study presents an overview of the field of recommender systems with current generation of recommendation methods and examines comprehensively CF systems with its algorithms.

Researching in Web Technology Classification Based on Improved Support Vector Machine

Abstract. Web as the main platform of information manufacturing, issuing, processing and transacting, has emerged massive isomerous dynamic semi-structural or non-structural information resources [1], so how to extract useful information from the mass Web resources has become a question which need to be solved. This paper introduces the background and status from domestic and abroad, and expounds the related theory and technology of text classification, constructs the Web text classification system model, gives a Web text collection algorithm and collection system. It verified the theoretical method which proposed in this paper by experiment, the result show that the extraction of information and Web classification is more accurate.

Creation and Use of Ontology Related to Genes, Syndromes, Diseases and Symptoms for the Classification of Biomedical Texts

research focuses on analyzing and classifying biomedical articles in the field of neuroscience, with a particular emphasis on scientific articles related to hearing loss. To carry out this task in a more efficient manner, resources as the elimination of stopwords were used. As well, it was implemented the n- gram-based text categorization system along with the use of a domain ontology related with genes, diseases and syndromes, obtaining promising results.

DACS Dewey index-based Arabic Document Categorization System

This paper is devoted to the development of Arabic Text Categorization System. First, a stop-words list is generated using statistical approach which captures the inflation of different Arabic words. Second, a feature representation model based on Hidden Markov Model is developed to extract roots and morphological weights. Third, a semantic synonyms merge technique is presented for feature reduction. Finally a Dewey-Index Based Back-propagation Artificial Neural Network is developed for Arabic Document Categorization. The system was compared with other classifiers and the results reveal a promising architecture.

Design of Text Categorization System Based on SVM

This paper introduces the design of a text categorization system based on Support Vector Machine (SVM). It analyzes the high dimensional characteristic of text data, the reason why SVM is suitable for text categorization. According to system data flow this system is constructed. This system consists of three subsystems which are text representation, classifier training and text classification. The core of this system is the classifier training, but text representation directly influences the currency of classifier and the performance of the system. Text feature vector space can be built by different kinds of feature selection and feature extraction methods. No research can indicate which one is the best method, so many feature selection and feature extraction methods are all developed in this system. For a specific classification task every feature selection method and every feature extraction method will be tested, and then a set of the best methods will be adopted.

Ontology-guided feature engineering for clinical text classification

In this study we present novel feature engineering techniques that leverage the biomedical domain knowledge encoded in the Unified Medical Language System (UMLS) to improve machine-learning based clinical text classification. Critical steps in clinical text classification include identification of features and passages relevant to the classification task, and representation of clinical text to enable discrimination between documents of different classes. We developed novel information-theoretic techniques that utilize the taxonomical structure of the Unified Medical Language System (UMLS) to improve feature ranking, and we developed a semantic similarity measure that projects clinical text into a feature space that improves classification. We evaluated these methods on the 2008 Integrating Informatics with Biology and the Bedside (I2B2) obesity challenge. The methods we developed improve upon the results of this challenge’s top machine-learning based system, and may improve the performance of other machine-learning based clinical text classification systems. We have released all tools developed as part of this study as open source, available at http://code.google.com/p/ytex.

Semantic Similarity Metric and its Application in Text Classification

Text classification is the task of assigning natural language textual documents to predefined categories based on their context. The main concern is this paper is to improve the accuracy of text classification system combined an improved CHI method and semantic similarity metric. Firstly, use an improved CHI method to select features from the raw features aim to reduce the dimensions of the features. Secondly, calculates the semantic distance between text feature vector and categorization feature vector so as to determine the document categorization. Finally, we carried out a series of experiments compared with other methods using the F1-measure. Experimental results show that our new method makes an important improvement in all categories.

Text Classification Combined an Improved CHI and Category Relevance Factor

Text classification is the task of assigning natural language textual documents to predefined categories based on their context. The main concern in this paper is to improve the accuracy of text classification system combined an improved CHI method and category relevance factor. Firstly, use an improved CHI method to select features from the raw features aim to reduce the dimensions of the features. Secondly, through the TF-CRF method to calculate the feature weight, this method mainly consider that the features have different distributions in different categories. Finally, we carried out a series of experiments compared with other methods using the F1-measure. Experimental results show that our new method makes an important improvement in all categories.

Efficient representation of text with multiple perspectives

An effective text representation scheme dominates the performance of text categorization system. However, based on the assumption of independent terms, the traditional schemes which tediously use term frequency (TF) and document frequency (DF) are insufficient for capturing enough information of a document and result in poor performance. To overcome this limitation, we investigate exploring the relationships between different terms of the same class tendency and the way of measuring the importance of a repetitive term in a document. In this paper, a group of novel term weighting factors are proposed to enhance the category contribution for each term. Then, based on a novel strategy of generating passages from document, we present two schemes, the weighted co-contributions of different terms corresponding to the class tendency and the weighted co-contributions for each term in different passages, to achieve improvements on text representation. The prior scheme works in a dimensionality reduction mode while the second one runs in the conventional way. By employing the support vector machine (SVM) classifier, experiments on four benchmark corpora show that the proposed schemes could achieve a consistent better performance than the conventional methods in both efficiency and accuracy. Further analysis also confirms some promising directions for the future works.

Multiclass Boosting with Adaptive Group‐Based kNN and Its Application in Text Categorization

AdaBoost is an excellent committee‐based tool for classification. However, its effectiveness and efficiency in multiclass categorization face the challenges from methods based on support vector machine (SVM), neural networks (NN), naïve Bayes, and k‐nearest neighbor (kNN). This paper uses a novel multi‐class AdaBoost algorithm to avoid reducing the multi‐class classification problem to multiple two‐class classification problems. This novel method is more effective. In addition, it keeps the accuracy advantage of existing AdaBoost. An adaptive group‐based kNN method is proposed in this paper to build more accurate weak classifiers and in this way control the number of basis classifiers in an acceptable range. To further enhance the performance, weak classifiers are combined into a strong classifier through a double iterative weighted way and construct an adaptive group‐based kNN boosting algorithm (AGkNN‐AdaBoost). We implement AGkNN‐AdaBoost in a Chinese text categorization system. Experimental results showed that the classification algorithm proposed in this paper has better performance both in precision and recall than many other text categorization methods including traditional AdaBoost. In addition, the processing speed is significantly enhanced than original AdaBoost and many other classic categorization algorithms.

Text Classification Using Support Vector Machine with Mixture of Kernel

Recent studies have revealed that emerging modern machine learning techniques are advantageous to statistical models for text classification, such as SVM. In this study, we discuss the applications of the support vector machine with mixture of kernel (SVM-MK) to design a text classification system. Differing from the standard SVM, the SVM-MK uses the 1-norm based object function and adopts the convex combinations of single feature basic kernels. Only a linear programming problem needs to be resolved and it greatly reduces the computational costs. More important, it is a transparent model and the optimal feature subset can be obtained automatically. A real Chinese corpus from FudanUniversityis used to demonstrate the good performance of the SVM- MK.

A Novel Category Vector-Based Cross Language Text Categorization Method

Due to the globalization on the Web, monolingual Text Categorization can be reformulated as a cross language TC task. To establish a practical English-Chinese CLTC system, a feature translation method and a fast text categorization algorithm based on a novel Category Vector Space Model are proposed in this paper. Provided a Chinese-English bilingual dictionary in scientific and technological fields, parallel corpora was employed to append translation probability value to bilingual dictionary so as to disambiguate translation results. The experiment results show that the CLTC system which was established by method in the paper is practical and valuable. The performance of Cross-Language text categorization system exceeds that of Mono-Lingual text categorization system and the result is exciting.

Semi-Automatic Labeling of Training Data Sets in Text Classification

Web includes digital libraries and billions of text documents. A fast and simple search through this sizeable set is important for users and researchers. Since manual or rule based document classification is a difficult, time consuming process, automatic classification systems are absolutely needed. Automatic text classification systems demand extensive and proper training data sets. To provide these data sets, usually, numerous unlabeled documents are labeled manually by experts. Manual labeling of documents is a difficult and time consuming process. Moreover, in manual labeling, due to human exhaustion and carelessness, there is the possibility of mistakes.In this study, semi-automatic creation of training data set has been proposed in a way that only a small percentage of this extensive set’s documents is labeled manually and the remaining percentage is done automatically. Results show that by labeling only ten percent of the training set, remaining documents can be automatically labeled with 98 percent of accuracy. It is worth mentioning that this reduction in accuracy only occurs in standard data sets, while for large practical data sets, this reduction is trivial compared to the accuracy reduction resulted by human exhaustion and carelessness.

Feature sub-set selection metrics for Arabic text classification

Feature sub-set selection (FSS) is an important step for effective text classification (TC) systems. This paper presents an empirical comparison of seventeen traditional FSS metrics for TC tasks. The TC is restricted to support vector machine (SVM) classifier and only for Arabic articles. Evaluation used a corpus that consists of 7842 documents independently classified into ten categories. The experimental results are presented in terms of macro-averaging precision, macro-averaging recall and macro-averaging F 1 measures. Results reveal that Chi-square and Fallout FSS metrics work best for Arabic TC tasks.

Solving multi-label text categorization problem using support vector machine approach with membership function

The pervasiveness of information available on the internet means that increasing numbers of documents must be classified. Text categorization is not only undertaken by domain experts, but also by automatic text categorization systems. Therefore, a text categorization system with a multi-label classifier is necessary to process the large number of documents. In this study, a proposed multi-label text categorization system is developed to classify multi-label documents. Data mapping is performed to transform data from a high-dimensional space to a lower-dimensional space with paired SVM output values, thus lowering the complexity of the computation. A pairwise comparison approach is applied to set the membership function in each predicted class to judge all possible classified classes. To better explain the proposed model, a comparative study using Reuter's data sets is performed on several multi-label approaches such as Naive Bayes, Multi-Label Mixture, Jaccard Kernel and Bp-MLL. Though the comparative results of the empirical experiment indicate that the proposed multi-label text categorization system performs better than other methods in terms of overall performance indices, these comparisons are done under the conditions without knowing original settings of parameters. From these comparative studies, it is found that these probabilities of documents appearing in correctly predicted classes and those of documents appearing in the wrongly predicted classes are important properties and we conclude that the probability of 0.5 for model membership function is a good criterion to judge between correctly and incorrectly classified documents from the results of the empirical experiment.

A Hybrid Arabic Text Summarization Technique Based on Text Structure and Topic Identification

We present a hybrid approach to the problem of Arabic text summarization. Our approach focuses on segment extraction and ranking using heuristic methods that assign weighted scores to segments of text. Also, we use a text categorization system and the Arabic WordNet to identify the thematic structure of the input text in order to select the most relevant sentences obtained from the statistical analysis process. We use a tokenizer, a stemmer and other statistical tools borrowed from traditional information retrieval to identify relevant segments in the text. The source document is segmented into its major units (title, paragraphs and lines) and then, text-lines are interpreted to extract relevant segments for inclusion in the summary. The summarization system was tested by 1200 human evaluators, who were each given a copy of a newspaper article and a system-generated summary and asked to classify them as rejected, not-related, satisfactory, good, or accepted. 76.92% of the summaries were judged t...

Symbolic rule-based classification of lung cancer stages from free-text pathology reports

To classify automatically lung tumor-node-metastases (TNM) cancer stages from free-text pathology reports using symbolic rule-based classification. By exploiting report substructure and the symbolic manipulation of systematized nomenclature of medicine-clinical terms (SNOMED CT) concepts in reports, statements in free text can be evaluated for relevance against factors relating to the staging guidelines. Post-coordinated SNOMED CT expressions based on templates were defined and populated by concepts in reports, and tested for subsumption by staging factors. The subsumption results were used to build logic according to the staging guidelines to calculate the TNM stage. The accuracy measure and confusion matrices were used to evaluate the TNM stages classified by the symbolic rule-based system. The system was evaluated against a database of multidisciplinary team staging decisions and a machine learning-based text classification system using support vector machines. Overall accuracy on a corpus of pathology reports for 718 lung cancer patients against a database of pathological TNM staging decisions were 72%, 78%, and 94% for T, N, and M staging, respectively. The system's performance was also comparable to support vector machine classification approaches. A system to classify lung TNM stages from free-text pathology reports was developed, and it was verified that the symbolic rule-based approach using SNOMED CT can be used for the extraction of key lung cancer characteristics from free-text reports. Future work will investigate the applicability of using the proposed methodology for extracting other cancer characteristics and types.

A parametric methodology for text classification

Finding the correct category (class) a new unclassified document belongs to is an interesting and difficult problem, with a wide range of applications. Our methodology for narrative text classification is based on two techniques: we calculate the distance (similarity) between the new unclassified document and all the pre-classified documents of each class and also calculate the similarity of the new document to the ‘average class document’ of each class. In both cases we use key phrases (text phrases or key terms) as the distinctive features of our text classification methodology and eventually the proposed text classification method is based on the automatic extraction of an authority list of key phrases that is appropriate for discriminating between different classes. In this paper, we apply this methodology in handling Greek text and we present the key concepts, the algorithms, and some critical decisions. A number of parameters of the mining algorithm are also fine tuned. The actual text classification system, the adopted (embedded) ideas and the alternative values of parameters are evaluated using two training sets (test collections).

Distributed Text Classification With an Ensemble Kernel-Based Learning Approach

Constructing a single text classifier that excels in any given application is a rather inviable goal. As a result, ensemble systems are becoming an important resource, since they permit the use of simpler classifiers and the integration of different knowledge in the learning process. However, many text-classification ensemble approaches have an extremely high computational burden, which poses limitations in applications in real environments. Moreover, state-of-the-art kernel-based classifiers, such as support vector machines and relevance vector machines, demand large resources when applied to large databases. Therefore, we propose the use of a new systematic distributed ensemble framework to tackle these challenges, based on a generic deployment strategy in a cluster distributed environment. We employ a combination of both task and data decomposition of the text-classification system, based on partitioning, communication, agglomeration, and mapping to define and optimize a graph of dependent tasks. Additionally, the framework includes an ensemble system where we exploit diverse patterns of errors and gain from the synergies between the ensemble classifiers. The ensemble data partitioning strategy used is shown to improve the performance of baseline state-of-the-art kernel-based machines. The experimental results show that the performance of the proposed framework outperforms standard methods both in speed and classification.