Extreme Multi-label Classification Research Articles

Performance improvement of extreme multi-label classification using K-way tree construction with parallel clustering algorithm

eXtreme Multi-Label Classification (XMLC) is the particular case of Multi-Label Classification, which deals with an extremely high number of labels. The main goal is to learn extreme classifier which extracts the subset of relevant labels from extremely large label space. In an extreme environment, one of the big issues is to deal with an extreme number of features, labels and instances which affects the performance of the classifier. The high dimensional feature space and label space makes existing approaches intractable in terms of data scalability, data sparsity, training and prediction cost. The appropriate input representation technique can be used to maintain the interdependency among labels and correlation between feature space and label space. The proposed approach called “K-way Tree based eXtreme Multi-Label Classifier (KTXMLC)“ works on tree based classifier to maintain correlations using feature-label input representation technique and node partitioning using a clustering algorithm. KTXMLC constructs multi-way multiple trees using a parallel clustering algorithm, which leads to fast computational cost. KTXMLC outperforms over the existing tree based classifier in terms of ranking based measures on six datasets named Delicious, Mediamill, Eurlex-4K, Wiki10-31K, AmazonCat-13K, Delicious-200K.

GHS-NET a generic hybridized shallow neural network for multi-label biomedical text classification

Exponential growth of biomedical literature and clinical data demands more robust yet precise computational methodologies to extract useful insights from biomedical literature and to perform accurate assignment of disease-specific codes. Such approaches can largely enhance the effectiveness of diverse biomedicine and bioinformatics applications. State-of-the-art computational biomedical text classification methodologies either solely leverage discrimintaive features extracted through convolution operations performed by deep convolutional neural network or contextual information extracted by recurrent neural network. However, none of the methodology takes advantage of both convolutional and recurrent neural networks. Further, existing methodologies lack to produce decent performance for the classification of different genre biomedical text such as biomedical literature or clinical notes. We, for the very first time, present a generic deep learning based hybrid multi-label classification methodology namely GHS-NET which can be utilized to accurately classify biomedical text of diverse genre. GHS-NET makes use of convolutional neural network to extract most discriminative features and bi-directional Long Short-Term Memory to acquire contextual information. GHS-NET effectiveness is evaluated for extreme multi-label biomedical literature classification and assignment of ICD-9 codes to clinical notes. For the task of extreme multi-label biomedical literature classification, performance comparison of GHS-Net and state-of-the-art deep learning based methodology reveals that GHS-Net marks the increment of 1%, 6%, and 1% for hallmarks of cancer dataset, 10%, 16%, and 11% for chemical exposure dataset in terms of precision, recall, and F1-score. For the task of clinical notes classification, GHS-Net outperforms previous best deep learning based methodology over Medical Information Mart for Intensive Care dataset (MIMIC-III) by the significant margin of 6%, 8% in terms of recall and F1-score. GHS-NET is available as a web service at11http://210.56.21.196/bio-medical-text-classification. and potentially can be used to accurately classify multi-variate disease and chemical exposure specific text.

Multi-label legal document classification: A deep learning-based approach with label-attention and domain-specific pre-training

Multi-label document classification has a broad range of applicability to various practical problems, such as news article topic tagging, sentiment analysis, medical code classification, etc. A variety of approaches (e.g., tree-based methods, neural networks and deep learning systems that are specifically based on pre-trained language models) have been developed for multi-label document classification problems and have achieved satisfying performance on different datasets. In the legal domain, however, one is often faced with several key challenges when working with multi-label classification tasks. One critical challenge is the lack of high-quality human labeled datasets, which prevents researchers and practitioners from achieving decent performance on respective tasks. Also, existing methods on multi-label classification typically focus on the majority classes, which results in an unsatisfying performance for other important classes that do not have sufficient training samples. In order to tackle the above challenges, in this paper, we first present POSTURE50K, a novel legal extreme multi-label classification dataset, which we will release to the research community. The dataset contains 50,000 legal opinions and their manually labeled legal procedural postures. Labels in this dataset follow a Zipfian distribution, leaving many of the classes with only a few samples. Furthermore, we propose a deep learning architecture that adopts domain-specific pre-training and a label-attention mechanism for multi-label document classification. We evaluate our proposed architecture on POSTURE50K and another legal multi-label dataset EUROLEX57K, and show that our approach achieves better performances than two baseline systems and another four recent state-of-the-art methods on both datasets.

Bonsai: diverse and shallow trees for extreme multi-label classification

Extreme multi-label classification (XMC) refers to supervised multi-label learning involving hundreds of thousands or even millions of labels.In this paper, we develop a suite of algorithms, called Bonsai, which generalizes the notion of label representation in XMC, and partitions the labels in the representation space to learn shallow trees.We show three concrete realizations of this label representation space including: (i) the input space which is spanned by the input features, (ii) the output space spanned by label vectors based on their co-occurrence with other labels, and (iii) the joint space by combining the input and output representations. Furthermore, the constraint-free multi-way partitions learnt iteratively in these spaces lead to shallow trees.By combining the effect of shallow trees and generalized label representation, Bonsai achieves the best of both worlds—fast training which is comparable to state-of-the-art tree-based methods in XMC, and much better prediction accuracy, particularly on tail-labels. On a benchmark Amazon-3M dataset with 3 million labels, Bonsai outperforms a state-of-the-art one-vs-rest method in terms of prediction accuracy, while being approximately 200 times faster to train. The code for Bonsai is available at https://github.com/xmc-aalto/bonsai.

Comprehensive study and Analysis of Extreme Multi-Label Classification Approach

Comprehensive study and Analysis of Extreme Multi-Label Classification Approach

Combining instance and feature neighbours for extreme multi-label classification

Extreme multi-label classification problems occur in different applications such as prediction of tags or advertisements. We propose a new algorithm that predicts labels using a linear ensemble of labels from instance- and feature-based nearest neighbours. In the feature-based nearest neighbours method, we precompute a matrix containing the similarities between each feature and label. For the instance-based nearest neighbourhood, we create an algorithm that uses an inverted index to compute cosine similarity on sparse datasets efficiently. We extend this baseline with a new top-k query algorithm that combines term-at-a-time and document-at-a-time traversal with tighter pruning based on a partition of the dataset. On ten real-world datasets, we find that our method outperforms state-of-the-art methods such as multi-label k-nearest neighbours, instance-based logistic regression, binary relevance with support vector machines and FastXml on different evaluation metrics. We also find that our algorithm is orders of magnitude faster than these baseline algorithms on sparse datasets and requires less than 20 ms per instance to predict labels for extreme datasets without the need for expensive hardware.

Extreme multi-label learning : A large scale classification approach in machine learning

In digital world, the amount of data is growing exponentially in day to day life. It is difficult to analyze and extract knowledge from large amount of data with millions of categories in Big Data environment. Therefore, it is challenging problem to develop model that classify large volume of documents available on Internet. However, Multi-Label Classification approach is used to classify data with multiple categories or labels but it is inefficient way to deal with millions of categories. Hence Extreme Multi-Label Classification approach is used to overcome this limitation by selecting subset of labels for the new instance from millions of labels. Recently Extreme Multi-Label Classification has attracted research attention in different application areas like document categorization in Wikipedia, people identification in social networking, gene prediction in bio-informatics etc. Extreme Multi-Label Classification is also opened up new challenge to reformulate existing machine learning problems like ranking, tagging and recommendation. This survey paper focuses on approaches and reviewing current research challenges on eXtreme Multi Label Classification. Also discussed state-of-the-art algorithms to handle eXtreme Multi-Label Classification Problem.

Data scarcity, robustness and extreme multi-label classification

The goal in extreme multi-label classification (XMC) is to learn a classifier which can assign a small subset of relevant labels to an instance from an extremely large set of target labels. The distribution of training instances among labels in XMC exhibits a long tail, implying that a large fraction of labels have a very small number of positive training instances. Detecting tail-labels, which represent diversity of the label space and account for a large fraction (upto 80%) of all the labels, has been a significant research challenge in XMC. In this work, we pose the tail-label detection task in XMC as robust learning in the presence of worst-case perturbations. This viewpoint is motivated by a key observation that there is a significant change in the distribution of the feature composition of instances of these labels from the training set to test set. For shallow classifiers, our robustness perspective to XMC naturally motivates the well-known ell _1-regularized classification. Contrary to the popular belief that Hamming loss is unsuitable for tail-labels detection in XMC, we show that minimizing (convex upper bound on) Hamming loss with appropriate regularization surpasses many state-of-the-art methods. Furthermore, we also highlight the sub-optimality of the co-ordinate descent based solver in the LibLinear package, which, given its ubiquity, is interesting in its own right. We also investigate the spectral properties of label graphs for providing novel insights towards understanding the conditions governing the performance of Hamming loss based one-vs-rest scheme vis-à-vis label embedding methods.

EDGE2VEC: Edge Representations for Large-Scale Scalable Hierarchical Learning

In present front-line of Big Data, prediction tasks over the nodes and edges in complex deep architecture needs a careful representation of features by assigning hundreds of thousands, or even millions of labels and samples for information access system, especially for hierarchical extreme multi-label classification. We introduce edge2vec, an edge representations framework for learning discrete and continuous features of edges in deep architecture. In edge2vec, we learn a mapping of edges associated with nodes where random samples are augmented by statistical and semantic representations of words and documents. We argue that infusing semantic representations of features for edges by exploiting word2vec and para2vec is the key to learningricherrepresentationsforexploringtargetnodes or labels in the hierarchy. Moreover, we design and implement a balanced stochastic dual coordinate ascent (DCA)-based support vector machine for speeding up training. We introduce a global decision-based top-down walks instead of random walks to predict the most likelihood labels in the deep architecture. We judge the efficiency of edge2vec over the existing state-of-the-art techniques on extreme multi-label hierarchical as well as flat classification tasks. The empirical results show that edge2vec is very promising and computationally very efficient in fast learning and predicting tasks. In deep learning workbench, edge2vec represents a new direction for statistical and semantic representations of features in task-independent networks.

IN-DEDUCTIVE and DAG-Tree Approaches for Large-Scale Extreme Multi-label Hierarchical Text Classification

This paper presents a large-scale extreme multi-label hierarchical text classification method that employs a large-scale hierarchical inductive learning and deductive classification (IN-DEDUCTIVE) approach using different efficient classifiers, and a DAG-Tree that refines the given hierarchy by eliminating nodes and edges to generate a new hierarchy. We evaluate our method on the standard hierarchical text classification datasets prepared for the PASCAL Challenge on Large-Scale Hierarchical Text Classification (LSHTC). We compare several classification algorithms on LSHTC including DCD-SVM, SVMperf, Pegasos, SGD-SVM, and Passive Aggressive, etc. Experimental results show that IN-DEDUCTIVE approach based systems with DCD-SVM, SGD-SVM, and Pegasos are promising and outperformed other learners as well as the top systems participated in the LSHTC3 challenge on Wikipedia medium dataset. Furthermore, DAG-Tree based hierarchy is effective especially for very large datasets since DAG-Tree exponentially reduce the amount of computation necessary for classification. Our system with IN-DEDUCIVE and DAG-Tree approaches outperformed the top systems participated in the LSHTC4 challenge on Wikipedia large dataset.

SLEEC: a space station ambulance

Restricted accessMoreSectionsView PDF ToolsAdd to favoritesDownload CitationsTrack Citations ShareShare onFacebookTwitterLinked InRedditEmail Cite this article Nonweiler Terence R. F. 1999SLEEC: a space station ambulancePhil. Trans. R. Soc. A.3572157–2176http://doi.org/10.1098/rsta.1999.0426SectionRestricted accessSLEEC: a space station ambulance Terence R. F. Nonweiler Terence R. F. Nonweiler Victoria University, Wellington, New Zealand and APECS Ltd, 3 Hawkley Hurst, Hawkley, Hants GU33 6NS, UK Google Scholar Find this author on PubMed Search for more papers by this author Terence R. F. Nonweiler Terence R. F. Nonweiler Victoria University, Wellington, New Zealand and APECS Ltd, 3 Hawkley Hurst, Hawkley, Hants GU33 6NS, UK Google Scholar Find this author on PubMed Search for more papers by this author Published:01 August 1999https://doi.org/10.1098/rsta.1999.0426AbstractThe assembly of the International Space Station calls for many hundreds of hours of extra-vehicular activity (EVA), and for maintenance thereafter. Occupation will ultimately involve as many as eight residents at a time. This level of human activity introduces significant risk of injury or illness and the need to return the patient to Earth. This return must avoid delays in orbit and unnecessary physical stress to the patient.SLEEC22 is a novel kind of ‘space ambulance’ based in orbit that is intended to return a patient and attendant back to Earth in a gentle winged descent, restricting the peak g experienced to less than a tenth above normal ground–level values. It nonetheless allows a wider than usual ‘re–entry window’ that would permit a landing at any of an extensive choice of airfields. It makes use of a lightweight waverider wing with only shallow anhedral and ‘sharp’ metallic leading edges. Cooling of the structure, both exterior and interior, is effected by conduction–assisted radiation. Previous ArticleNext Article VIEW FULL TEXT DOWNLOAD PDF FiguresRelatedReferencesDetailsCited by Chen S, Wang L, Li W and Zhang K (2019) Deep Learning Method with Attention for Extreme Multi-label Text Classification PRICAI 2019: Trends in Artificial Intelligence, 10.1007/978-3-030-29894-4_14, (179-190), . Townend L (2001) Domain of the Scramjet, Journal of Propulsion and Power, 10.2514/2.5865, 17:6, (1205-1213), Online publication date: 1-Nov-2001. This Issue01 August 1999Volume 357Issue 1759Theme Issue ‘Hypersonic aircraft: lifting re–entry and launch’ compiled by L. H. Townend Article InformationDOI:https://doi.org/10.1098/rsta.1999.0426Published by:Royal SocietyPrint ISSN:1364-503XOnline ISSN:1471-2962History: Published online01/08/1999Published in print01/08/1999 License: Citations and impact Keywordsspace station ambulanceheat protectionwaverider designchoice of descentlow–g re–entrycentreline stand–off angle