Large Scale Hierarchical Classification Research Articles

Incremental feature selection for large-scale hierarchical classification with the arrival of new samples

Incremental feature selection for large-scale hierarchical classification with the arrival of new samples

Uncertainty instructed multi-granularity decision for large-scale hierarchical classification

Hierarchical classification identifies a sample from the root node to a leaf node along the hierarchical structures of labels. It is often difficult to perform leaf-node prediction owing to ambiguous or incomplete information In such scenarios, a multi-granularity decision needs to be designed to stop the sample at a coarse-grained node rather than going rashly to a wrong leaf node. Conventionally, the probabilities of assigning the sample to its child nodes are regarded as evidence to decide whether to let the sample stop or go. However, the fact that the output of an unreliable classifier cannot be used to appropriately form the basis of decision making, especially with the existence of data uncertainties, is overlooked by existing methods. Therefore, we model the multi-granularity decision problem from an uncertainty perspective and consider that the uncertainties of multi-granularity decision emanate from the prediction of the classifier and the intrinsic information of the data. Inspired by the theory of fuzzy rough sets, a new measure is proposed to describe the intrinsic uncertainty in data. Integrating with the prediction uncertainty of the classifier, the hierarchical structure is used to design an effective optimization method that ensures proper multi-granularity decisions. Experiments show that the proposed algorithm achieves state-of-the-art performance.

Kernelized fuzzy rough sets based online streaming feature selection for large-scale hierarchical classification

In recent years, many online streaming feature selection approaches focus on flat data, which means that all data are taken as a whole. However, in the era of big data, not only the feature space of data has unknown and evolutionary characteristics, but also the label space of data exists hierarchical structure. To address this problem, an online streaming feature selection framework for large-scale hierarchical classification task is proposed. The framework consists of three parts: (1) a new hierarchical data-oriented kernelized fuzzy rough model with sibling strategy is constructed, (2) the online important feature is selected based on feature correlation analysis, and (3) the online redundant feature is deleted based on feature redundancy. Finally, an empirical study using several hierarchical classification data sets manifests that the proposed method outperforms other state-of-the-art online streaming feature selection methods.

Fuzzy Rough Set Based Feature Selection for Large-Scale Hierarchical Classification

The classification of high-dimensional tasks remains a significant challenge for machine learning algorithms. Feature selection is considered to be an indispensable preprocessing step in high-dimensional data classification. In the era of big data, there may be hundreds of class labels, and the hierarchical structure of the classes is often available. This structure is helpful in feature selection and classifier training. However, most current techniques do not consider the hierarchical structure. In this paper, we design a feature selection strategy for hierarchical classification based on fuzzy rough sets. First, a fuzzy rough set model for hierarchical structures is developed to compute the lower and upper approximations of classes organized with a class hierarchy. This model is distinguished from existing techniques by the hierarchical class structure. A hierarchical feature selection problem is then defined based on the model. The new model is more practical than existing feature selection approaches, as many real-world tasks are naturally cast in terms of hierarchical classification. A feature selection algorithm based on sibling nodes is proposed, and this is shown to be more efficient and more versatile than flat feature selection. Compared with the flat feature selection algorithm, the computational load of the proposed algorithm is reduced from 98.0% to 6.5%, while the classification performance is improved on the SAIAPR dataset. The related experiments also demonstrate the effectiveness of the hierarchical algorithm.

Improving large-scale hierarchical classification by rewiring: a data-driven filter based approach

Hierarchical Classification (HC) is a supervised learning problem where unlabeled instances are classified into a taxonomy of classes. Several methods that utilize the hierarchical structure have been developed to improve the HC performance. However, in most cases apriori defined hierarchical structure by domain experts is inconsistent; as a consequence performance improvement is not noticeable in comparison to flat classification methods. We propose a scalable data-driven filter based rewiring approach to modify an expert-defined hierarchy. Experimental comparisons of top-down hierarchical classification with our modified hierarchy, on a wide range of datasets shows classification performance improvement over the baseline hierarchy (i.e., defined by expert), clustered hierarchy and flattening based hierarchy modification approaches. In comparison to existing rewiring approaches, our developed method (rewHier) is computationally efficient, enabling it to scale to datasets with large numbers of classes, instances and features. We also show that our modified hierarchy leads to improved classification performance for classes with few training samples in comparison to flat and state-of-the-art hierarchical classification approaches. Source Code: https://cs.gmu.edu/~mlbio/TaxMod/

Review on hierarchical learning methods for large-scale classification task

Hierarchical classification is a task that uses hierarchy of categories in data. It can handle large-scale data. In recent years, significant research has emerged in this field, which is receiving increasingly more attention. In this paper, we first introduce the definition of hierarchical classification and thereafter review the important studies on several basic issues in large-scale hierarchical classification tasks based on different problem-solving strategies. First, we define the hierarchy formally and introduce some hierarchical evaluation metrics. Second, we explain how to construct the hierarchy, how to learn classifiers and perform feature selection using the information in the hierarchy, and how to design stopping strategies and introduce some representative studies on each issue. Finally, we summarize the features of large-scale hierarchical classification task and discuss the possible future work in this field.

HierFlat: flattened hierarchies for improving top-down hierarchical classification

Large-scale classification of structured data where classes are organized in a hierarchical structure is an important area of research. Top-down approaches that leverage the hierarchy during the learning and prediction phase are efficient for solving large-scale hierarchical classification. However, accuracy of top-down approaches is poor due to error propagation, i.e., prediction errors made at higher levels in the hierarchy cannot be corrected at lower levels. One of the main reasons behind errors at the higher levels is the presence of inconsistent nodes that are introduced due to the arbitrary process of creating these hierarchies by domain experts. In this paper, we propose two different data-driven approaches (local and global) for hierarchical structure modification that identifies and flattens inconsistent nodes present in the hierarchy. Our extensive empirical evaluation of the proposed approaches on several image and text datasets with varying distribution of features, classes and training instances per class shows improved classification performance over competing hierarchical modification approaches. Specifically, we see an improvement up to 7 $$\%$$ in Macro-F1 score with our best approach over best top-down baseline. We also show that our modified hierarchy leads to improved classification performance for classes with few training samples in comparison with flat approach. Further, our experimental evaluation shows that combining the prediction from flat and hierarchical model in an ensemble setting results in better classification performance.

Large-scale hierarchical classification with rare categories and inconsistencies

Structuring data is crucial for managing massive amount of available data. Hierarchy (taxonomy) provides a natural and convenient way to organize the information. It has been extensively used in several domains, such as gene taxonomy for organizing gene sequences, international patent hierarchy for easy browsing and retrieval of patent documents, DMOZ taxonomy for web-pages categorization, and ImageNet database for indexing millions of images according to WordNet hierarchy. Given, a hierarchy containing thousands of classes (categories) and millions of instances (examples), there is an essential need to develop an efficient and automated approaches to categorize unlabeled test instances. This problem is referred to as Hierarchical Classification (HC) task. HC is an important machine learning problem that has been researched and explored extensively in the past few years (Silla Jr & Freitas, 2011). The popularity of large-scale HC problem is evident from various HC competitions organized in the past few years such as LSHTC 1 , BioASQ 2 and ILSVRC 3 . HC poses several challenges due to the following reasons: (i) Data imbalance with large number of classes having very few positive examples for training (rare categories), (ii) Multi-label classification, (iii) Feature selection, (iv) Inconsistent hierarchy due to domain experts manual design, and (v) Scalability due to large number of examples, features and classes. Several approaches that address these issues individually (or multiple issues together) have been developed over the years (Gopal & Yang, 2013; Babbar et al., 2013), however there are many possibilities of improving the existing methods. Specifically, we have developed the methods for handling rare categories and inconsistent hierarchy problem.

A Meta-Top-Down Method for Large-Scale Hierarchical Classification

Recent large-scale hierarchical classification tasks typically have tens of thousands of classes on which the most widely used approach to multiclass classification--one-versus-rest--becomes intractable due to computational complexity. The top-down methods are usually adopted instead, but they are less accurate because of the so-called error-propagation problem in their classifying phase. To address this problem, this paper proposes a meta-top-down method that employs metaclassification to enhance the normal top-down classifying procedure. The proposed method is first analyzed theoretically on complexity and accuracy, and then applied to five real-world large-scale data sets. The experimental results indicate that the classification accuracy is largely improved, while the increased time costs are smaller than most of the existing approaches.

Hierarchical classification with a topic taxonomy via LDA

Large scale hierarchical classification problem researches how to classify documents into a predefined taxonomy with thousands of categories. As the skewed category distribution over documents, that is, most categories have very few labeled documents, the data sparseness problem in the rare categories lead to a low classification performance. In this paper, we study the problem of web-page classification over the topic taxonomy of the DMOZ directory. For this hard task, we proposed a hierarchical classification model based on Latent Dirichlet allocation (LDA). We use LDA model as the feature extraction technique to extract latent topics to reduce the effects of data sparseness, and construct topic feature vectors associated with the corpus for training more robust classification models for rare categories. Experiments were conducted on the dataset of web pages from the Chinese Simplified branch of the DMOZ directory. The results show that our method achieves a performance improvement for rare categories over the hierarchical classification methods based on full-term and feature-word, and further improves the performance over the whole topic taxonomy.

Research and Development of Large Scale Hierarchical Classification Problem

With the development of information technology,Web information management and access become much difficult to some extent as rapid increase in Internet data.A large scale class hierarchy of concepts or topics was used to label the web information to make information access easier.In this process,large scale hierarchical classification problem researches how to classify the Web documents into the categories among the class hierarchy,which is surveyed in this paper.Firstly,a definition of large scale hierarchical classification problem is proposed,which is used to describe the problem in Abstraction level.Meanwhile,strategies for conquering the problem are also investigated.Secondly,classification of solving methods for this problem is analyzed,and on the basis of the classification,many typical solving methods are introduced and compared.Lastly,future research trends of the solving methods for this problem are reviewed.

The ECIR 2010 large scale hierarchical classification workshop

This paper reports on the Large Scale Hierarchical Classification workshop (http://kmi.open.ac.uk/events/ecir2010/workshops-tutorials), held in conjunction with the European Conference on Information Retrieval (ECIR) 2010. The workshop was associated with the PASCAL 2 Large-Scale Hierarchical Text Classification Challenge (http://lshtc.iit.demokritos.gr), which took place in 2009. We first provide information about the challenge, presenting the data used, the tasks and the evaluation measures and then we provide an overview of the approaches proposed by the participants of the workshop, together with a summary of the results of the challenge.