Multi-label Classification Model Research Articles

Leak detection in water distribution network using machine learning techniques

ABSTRACT Leakage in the water distribution system (WDS) and its control has been challenging for water resources fraternity for management of precious water demand. This study examines an inverse engineering technique to find the leaks in water supply pipelines. The main objective of the study has been to identify the patterns of deviations in the pressure/flow in the network, due to a single leak in the network, by solving classification and regression problems using artificial neural networks (ANNs) and support vector machines (SVMs). The leak detections were solved using two scenarios, wherein, (a) only pressure measurements and (b) only flow measurements, are undertaken in the system. The multi-layered perceptron (MLP) model and multi-label multi-class SVM classification and regression models were developed and trained using the pressure and flow signals, separately. It was found that the ANN model performed better than the SVM model in pressure- and flow-based leak detection in both classification and regression problems. The model performance could also be improved by optimizing the number of inputs to the model during the training phase. The present study would be useful for water supply management while applying the techniques for minimizing the losses in the water supply network due to leakages.

Early and Intelligent Bearing Fault Detection Using Adaptive Superlets

Faults in bearings interrupt the operation of three-phase induction motors (IMs) and disturb factory production. Vibration signal analysis, feature extraction, and incorporation of artificial intelligence can enhance the ability of fault detection in bearings. However, implementing these techniques is challenging in industries due to harsh environmental conditions and massive computational power. Prioritizing the issue, this article presents a fusion method considering the time-frequency contents of vibration signals via the super-resolution property of wavelets and a two-dimensional convolutional neural network (2-D-CNN). First, a laboratory-based real-time process is considered with Variable Frequency Drive (VFD) driven IM which is also exposed to environmental vibration. In the present work, a piezoelectric sensor is used to collect the vibration of IM during operation. Then the collected time-series signals are stored in a computer through a Data Acquisition System (DAS). Further, signals are segmented in small windows to extract respective feature images using Continuous Wavelet Transform (CWT), Stockwell Transform (ST), and Adaptive Superlet Transform (ASLT). Thereafter, the image resolution is optimally resized and fed to a 2-D-CNN model for multi-label classification of the bearings. It is observed that ASLT-2-D-CNN with 32×32×3 resolution outperforms the other two fusion methods as it reports high accuracy and reduces runtime (training and testing) along with memory usage. The robustness of ASLT-2-D-CNN is also validated with two open-source bearing datasets. Thus, the result indicates that the proposed method for bearing fault detection can be deployed in the field to make IMs safe which reduces the maintenance cost of the industry.

Diabetic retinopathy detection using principal component analysis multi-label feature extraction and classification

Diabetic Retinopathy (DR) is the most common cause of eyesight loss that affects millions of people worldwide. Although there are recognized screening procedures for detecting the condition, such as fluorescein angiography and optical coherence tomography, the majority of patients are unaware and fail to have such tests at the proper time. Prompt identification of the condition is critical in avoiding vision loss, which occurs when Diabetes Mellitus (DM) is left untreated for an extended length of time. Several Machine Learning (ML) and Deep Learning (DL) algorithms have been used on DR datasets for disease prediction and classification, however, the majority of them have ignored the element of data pre-processing and dimensionality reduction, which are known as a major gap resulting in biased findings. In the first line of this research, data preprocessing was performed on the color Fundus Photographs (CFPs). Subsequently, we performed feature extraction with Principal Component Analysis (PCA). A Deep Learning Multi-Label Feature Extraction and Classification (ML-FEC) model based on pre-trained Convolutional Neural Network (CNN) architecture was proposed. Then, transfer learning was applied to train a subset of the images using three state-of-the-art CNN architectures, namely, ResNet50, ResNet152, and SqueezeNet1 with parameter-tuning to identify and classify the lesions. The experimental findings revealed an accuracy of 93.67% with a hamming loss of 0.0603 for ResNet 50, an accuracy of 91.94%, and Hamming Loss of 0.0805 for Squeezenet1 and an accuracy of 94.40% with Hamming loss of 0.0560 was achieved by ResNet 152 which demonstrates the suitability of the model for implementation in daily clinical practice and to support large scale DR screening programs.

MecDDI: Clarified Drug-Drug Interaction Mechanism Facilitating Rational Drug Use and Potential Drug-Drug Interaction Prediction.

Drug-drug interactions (DDIs) are a major concern in clinical practice and have been recognized as one of the key threats to public health. To address such a critical threat, many studies have been conducted to clarify the mechanism underlying each DDI, based on which alternative therapeutic strategies are successfully proposed. Moreover, artificial intelligence-based models for predicting DDIs, especially multilabel classification models, are highly dependent on a reliable DDI data set with clear mechanistic information. These successes highlight the imminent necessity to have a platform providing mechanistic clarifications for a large number of existing DDIs. However, no such platform is available yet. In this study, a platform entitled "MecDDI" was therefore introduced to systematically clarify the mechanisms underlying the existing DDIs. This platform is unique in (a) clarifying the mechanisms underlying over 1,78,000 DDIs by explicit descriptions and graphic illustrations and (b) providing a systematic classification for all collected DDIs based on the clarified mechanisms. Due to the long-lasting threats of DDIs to public health, MecDDI could offer medical scientists a clear clarification of DDI mechanisms, support healthcare professionals to identify alternative therapeutics, and prepare data for algorithm scientists to predict new DDIs. MecDDI is now expected as an indispensable complement to the available pharmaceutical platforms and is freely accessible at: https://idrblab.org/mecddi/.

Continuous Prediction of Web User Visual Attention on Short Span Windows Based on Gaze Data Analytics.

Understanding users' visual attention on websites is paramount to enhance the browsing experience, such as providing emergent information or dynamically adapting Web interfaces. Existing approaches to accomplish these challenges are generally based on the computation of salience maps of static Web interfaces, while websites increasingly become more dynamic and interactive. This paper proposes a method and provides a proof-of-concept to predict user's visual attention on specific regions of a website with dynamic components. This method predicts the regions of a user's visual attention without requiring a constant recording of the current layout of the website, but rather by knowing the structure it presented in a past period. To address this challenge, the concept of visit intention is introduced in this paper, defined as the probability that a user, while browsing, will fixate their gaze on a specific region of the website in the next period. Our approach uses the gaze patterns of a population that browsed a specific website, captured via an eye-tracker device, to aid personalized prediction models built with individual visual kinetics features. We show experimentally that it is possible to conduct such a prediction through multilabel classification models using a small number of users, obtaining an average area under curve of 84.3%, and an average accuracy of 79%. Furthermore, the user's visual kinetics features are consistently selected in every set of a cross-validation evaluation.

A Federated Learning Framework for Fingerprinting-Based Indoor Localization in Multibuilding and Multifloor Environments

The participatory nature of federated learning (FL) makes it attractive for fingerprinting-based indoor localization in multibuilding and multifloor environments. A group of sensing clients can collaboratively leverage their private, local fingerprint data to help their edge server update a location prediction model. However, it is challenging to jointly handle the two involved issues, i.e., building-floor classification (BFC) and latitude–longitude regression (LLR), in a wide 3-D space through enabling FL on decentralized yet heterogeneous data and over an imperfect wireless network. In this article, we confront these challenges and propose an FL framework, FedLoc3D, for both BFC and LLR. Specifically, the former issue is addressed by an FedDSC-BFC approach, which generates a multilabel classification model based on a convolutional neural network with depthwise separable convolutions. The latter issue is addressed by an FedADA-LLR approach, which develops a multitarget regression model based on a deep neural network with autoencoder and data augmentation. Extensive experiments on a real-world data set of WiFi fingerprints are carried out, and our approaches with enhanced capabilities of feature extraction, generalization, and convergence are validated to improve both localization accuracy and learning efficiency under data heterogeneity and network instability.

Cross-modal multi-label image classification modeling and recognition based on nonlinear

Abstract Recently, it has become a popular strategy in multi-label image recognition to predict those labels that co-occur in a picture. Previous work has concentrated on capturing label correlation but has neglected to correctly fuse picture features and label embeddings, which has a substantial influence on the model’s convergence efficiency and restricts future multi-label image recognition accuracy improvement. In order to better classify labeled training samples of corresponding categories in the field of image classification, a cross-modal multi-label image classification modeling and recognition method based on nonlinear is proposed. Multi-label classification models based on deep convolutional neural networks are constructed respectively. The visual classification model uses natural images and simple biomedical images with single labels to achieve heterogeneous transfer learning and homogeneous transfer learning, capturing the general features of the general field and the proprietary features of the biomedical field, while the text classification model uses the description text of simple biomedical images to achieve homogeneous transfer learning. The experimental results show that the multi-label classification model combining the two modes can obtain a hamming loss similar to the best performance of the evaluation task, and the macro average F1 value increases from 0.20 to 0.488, which is about 52.5% higher. The cross-modal multi-label image classification algorithm can better alleviate the problem of overfitting in most classes and has better cross-modal retrieval performance. In addition, the effectiveness and rationality of the two cross-modal mapping techniques are verified.

Automatic Multilabel Classification of Multiple Fundus Diseases Based on Convolutional Neural Network With Squeeze-and-Excitation Attention.

Automatic multilabel classification of multiple fundus diseases is of importance for ophthalmologists. This study aims to design an effective multilabel classification model that can automatically classify multiple fundus diseases based on color fundus images. We proposed a multilabel fundus disease classification model based on a convolutional neural network to classify normal and seven categories of common fundus diseases. Specifically, an attention mechanism was introduced into the network to further extract information features from color fundus images. The fundus images with eight categories of labels were applied to train, validate, and test our model. We employed the validation accuracy, area under the receiver operating characteristic curve (AUC), and F1-score as performance metrics to evaluate our model. Our proposed model achieved better performance with a validation accuracy of 94.27%, an AUC of 85.80%, and an F1-score of 86.08%, compared to two state-of-the-art models. Most important, the number of training parameters has dramatically dropped by three and eight times compared to the two state-of-the-art models. This model can automatically classify multiple fundus diseases with not only excellent accuracy, AUC, and F1-score but also significantly fewer training parameters and lower computational cost, providing a reliable assistant in clinical screening. The proposed model can be widely applied in large-scale multiple fundus disease screening, helping to create more efficient diagnostics in primary care settings.

MRChexNet: Multi-modal bridge and relational learning for thoracic disease recognition in chest X-rays.

While diagnosing multiple lesion regions in chest X-ray (CXR) images, radiologists usually apply pathological relationships in medicine before making decisions. Therefore, a comprehensive analysis of labeling relationships in different data modes is essential to improve the recognition performance of the model. However, most automated CXR diagnostic methods that consider pathological relationships treat different data modalities as independent learning objects, ignoring the alignment of pathological relationships among different data modalities. In addition, some methods that use undirected graphs to model pathological relationships ignore the directed information, making it difficult to model all pathological relationships accurately. In this paper, we propose a novel multi-label CXR classification model called MRChexNet that consists of three modules: a representation learning module (RLM), a multi-modal bridge module (MBM) and a pathology graph learning module (PGL). RLM captures specific pathological features at the image level. MBM performs cross-modal alignment of pathology relationships in different data modalities. PGL models directed relationships between disease occurrences as directed graphs. Finally, the designed graph learning block in PGL performs the integrated learning of pathology relationships in different data modalities. We evaluated MRChexNet on two large-scale CXR datasets (ChestX-Ray14 and CheXpert) and achieved state-of-the-art performance. The mean area under the curve (AUC) scores for the 14 pathologies were 0.8503 (ChestX-Ray14) and 0.8649 (CheXpert). MRChexNet effectively aligns pathology relationships in different modalities and learns more detailed correlations between pathologies. It demonstrates high accuracy and generalization compared to competing approaches. MRChexNet can contribute to thoracic disease recognition in CXR.

A new Classifier Chain method of BERT Models For Multi-label Classification of Arabic Abusive Language on Social Media

Moving beyond classical single-label classification tasks proves to be crucial nowadays since the majority of real-life problems are inherently multi-label. Various tasks require involving, for one single instance as an input, several classes simultaneously or none of them. Particularly, detecting abusive texts in Arabic social networks intuitively carries several manifestations with regard to the targets’ referred characteristics like gender, race, sexual orientation, etc. Existing multi-label algorithms either invest considerable complexity to handle labels’ correlation or explicitly ignore this information to deal with each label as an independent binary problem. To remedy the abovementioned problems, this paper proposes a new multi-label classification solution based on deep neural networks. The main idea is to exploit the correlation between all the dataset's labels to break it into ordered subsets. Then, follow that order to implement a chain of BERT classifiers. Each classifier is therefore trained on a corresponding subset and passes its output via feature space to the following one. For a new instance, we gather all the classifiers’ results and pick the best one for each class. Experiments showed that our approach significantly outperformed the single classifier's method using BERT as well as other neural network architectures on our Arabic abusive language dataset. 91%,76%, and 82% were achieved respectively for precision, recall, and F1-score. Moreover, we further investigated the performance of our approach on Kaggle multi-label research paper dataset and achieved very promising results.

A Hybrid TF-IDF and RNN Model for Multi-label Classification of the Deep and Dark Web

A Hybrid TF-IDF and RNN Model for Multi-label Classification of the Deep and Dark Web

Combing Type-Aware Attention and Graph Convolutional Networks for燛vent燚etection

Event detection (ED) is aimed at detecting event occurrences and categorizing them. This task has been previously solved via recognition and classification of event triggers (ETs), which are defined as the phrase or word most clearly expressing event occurrence. Thus, current approaches require both annotated triggers as well as event types in training data. Nevertheless, triggers are non-essential in ED, and it is time-wasting for annotators to identify the “most clearly” word from a sentence, particularly in longer sentences. To decrease manual effort, we evaluate event detection without triggers. We propose a novel framework that combines Type-aware Attention and Graph Convolutional Networks (TA-GCN) for event detection. Specifically, the task is identified as a multi-label classification problem. We first encode the input sentence using a novel type-aware neural network with attention mechanisms. Then, a Graph Convolutional Networks (GCN)-based multi-label classification model is exploited for event detection. Experimental results demonstrate the effectiveness.

SHO-CNN: A Metaheuristic Optimization of a Convolutional Neural Network for Multi-Label News Classification

News media always pursue informing the public at large. It is impossible to overestimate the significance of understanding the semantics of news coverage. Traditionally, a news text is assigned to a single category; however, a piece of news may contain information from more than one domain. A multi-label text classification model for news is proposed in this paper. The proposed model is an automated expert system designed to optimize CNN’s classification of multi-label news items. The performance of a CNN is highly dependent on its hyperparameters, and manually tweaking their values is a cumbersome and inefficient task. A high-level metaheuristic optimization algorithm, spotted hyena optimizer (SHO), has higher advanced exploration and exploitation capabilities. SHO generates a collection of solutions as a group of hyperparameters to be optimized, and the process is repeated until the desired optimal solution is achieved. SHO is integrated to automate the tuning of the hyperparameters of a CNN, including learning rate, momentum, number of epochs, batch size, dropout, number of nodes, and activation function. Four publicly available news datasets are used to evaluate the proposed model. The tuned hyperparameters and higher convergence rate of the proposed model result in higher performance for multi-label news classification compared to a baseline CNN and other optimizations of CNNs. The resulting accuracies are 93.6%, 90.8%, 68.7%, and 95.4% for RCV1-v2, Reuters-21578, Slashdot, and NELA-GT-2019, respectively.

A Complex Heterogeneous Network Model of Disease Regulated by Noncoding RNAs: A Case Study of Unstable Angina Pectoris.

MicroRNAs (miRNAs) are important types of noncoding RNAs, and there is a lack of holistic and systematic understanding of the functions they play in disease. We proposed a research strategy, including two parts network analysis and network modelling, to analyze, model, and predict the regulatory network of miRNAs from a network perspective, using unstable angina pectoris as an example. In the network analysis section, we proposed the WGCNA & SimCluster method using both correlation and similarity to find hub miRNAs, and validation on two datasets showed better results than the methods using correlation or similarity alone. In the network modelling section, we used six knowledge graph or graph neural network models for link prediction of three types of edges and multilabel classification of two types of nodes. Comparative experiments showed that the RotatE model was a good model for link prediction, while the RGCN model was the best model for multilabel classification. Potential target genes were predicted for hub miRNAs and validation of hub miRNA-target gene interactions, target genes as biomarkers and target gene functions were performed using a three-step validation approach. In conclusion, our study provides a new strategy to analyze and model miRNA regulatory networks.

A COLLABORATIVE NETWORK INTRUSION DETECTION ARCHITECTURE FOR A PROGRAMMABLE DATA PLANE

For early detection and response to network threats, a network intrusion detection system should be executed on a data plane. However, due to high model complexity, an intrusion detection model based on advanced machine learning techniques becomes unsuitable for limited-resource switches. To address this problem, we propose a lightweight joint detection model that is inspired by classification parallelism and neuron pruning. Specifically, the traditional multi-label classification model is decoupled into several class-specific sub-models and each sub-model takes charge of detecting one or several traffic classes. In our model, the number of participating switches can vary based on network traffic and available computing resources of edge devices. Moreover, to reduce the size of sub-models, magnitude pruning is applied for each sub-model to only keep salient connections. Evaluation experiments are conducted with various network parameters and results show that the proposed architecture achieves much lower model complexity than the traditional multi-label classifier without a reduction in classification performance.

A Curriculum Batching Strategy for Automatic ICD Coding with Deep Multi-Label Classification Models.

The International Classification of Diseases (ICD) has an important role in building applications for clinical medicine. Extremely large ICD coding label sets and imbalanced label distribution bring the problem of inconsistency between the local batch data distribution and the global training data distribution into the minibatch gradient descent (MBGD)-based training procedure for deep multi-label classification models for automatic ICD coding. The problem further leads to an overfitting issue. In order to improve the performance and generalization ability of the deep learning automatic ICD coding model, we proposed a simple and effective curriculum batching strategy in this paper for improving the MBGD-based training procedure. This strategy generates three batch sets offline through applying three predefined sampling algorithms. These batch sets satisfy a uniform data distribution, a shuffling data distribution and the original training data distribution, respectively, and the learning tasks corresponding to these batch sets range from simple to complex. Experiments show that, after replacing the original shuffling algorithm-based batching strategy with the proposed curriculum batching strategy, the performance of the three investigated deep multi-label classification models for automatic ICD coding all have dramatic improvements. At the same time, the models avoid the overfitting issue and all show better ability to learn the long-tailed label information. The performance is also better than a SOTA label set reconstruction model.

Multi-label sub-pixel classification of red and black soil over sparse vegetative areas using AVIRIS-NG airborne hyperspectral image

The limited spatial resolution of the hyperspectral (Hx) images corrupts the spectral information of pure materials and their distribution in an image. The accuracy of characterising or classifying the soil using Hx or Mx images decreases when surfaces are covered by vegetation. In the presence of vegetation, a single pixel can be labelled as either vegetation or a specific soil type. In this context, we have studied the usefulness of the multi-label classification (MLC) approach to classify the soil colour in the presence of vegetation cover. We have evaluated its performance on airborne Hx (Airborne Visible InfraRed Imaging Spectrometer - Next Generation, AVIRIS-NG) images acquired over Berambadi catchment, Karnataka, India. The potential of MLC to classify soil types using simulated Sentinel-2 images (Sen-S) was also explored in this study. The surface soil colour in the Berambadi catchment was classified into two soil types (“black” and “red” soils). The proposed MLC approach consists of (1) simulating the mixed spectra of vegetation, red soil, black soil and non-photosynthesis-vegetation (NPV) using linear-mixture-model (LMM) and bi-linear-mixture-model (BLM) to generate a well-balanced calibration data set, and (2) labelling of each pixel into multiple classes using MLC approaches. Performances of classical and deep-neural-network (DNN) based MLC models were compared to identify the best performing model. Our results showed significant performance for the cost-sensitive-multi-label-embedding (CLEMS) model when applied to both AVIRIS-NG (OA=97%) and Sen-S (OA=93%) images. The proposed method requires a limited number of ground-truth samples, and it is operationally practical for large Hx and Mx images.

Machine-learning models utilizing CYP3A4*1G show improved prediction of hypoglycemic medication in Type 2 diabetes.

The effectiveness and side effects of Type 2 diabetes (T2D) medication are related to individual genetic background. SNPs CYP3A4 and CYP2C19 were introduced to machine-learning models to improve the performance of T2D medication prediction. Two multilabel classification models, ML-KNN and WRank-SVM, trained with clinical data and CYP3A4/CYP2C19 SNPs were evaluated. Prediction performance was evaluated with Hamming loss, one-error, coverage, ranking loss and average precision. The average precision of ML-KNN and WRank-SVM using clinical data was 92.74% and 92.9%, respectively. Combined with CYP2C19*2*3, the average precision dropped to 88.84% and 89.93%, respectively. While combined with CYP3A4*1G, the average precision was enhanced to 97.96% and 97.82%, respectively. Results suggest that CYP3A4*1G can improve the performance of ML-KNN and WRank-SVM models in predicting T2D medication performance.

Joint optimization of scoring and thresholding models for online multi-label classification

Existing online multi-label classification works cannot well handle the online label thresholding problem and lack regret analysis for their online algorithms. This paper proposes a novel framework of joint optimization of scoring and thresholding models for online multi-label classification, with the aim to overcome the above drawbacks. The key feature of our framework is that both scoring and thresholding models are included as important components of the online multi-label classifier and are incorporated into one online optimization problem. Based on this framework, we present two adaptive label thresholding algorithms and two fixed thresholding algorithms. For each type of algorithms, a first-order method and a second-order one are provided for updating the online multi-label classifier. Both methods enjoy a closed-form update. Our proposed algorithms are proved to achieve a sub-linear regret. Using Mercer kernels, two first-order algorithms can be extended to handle nonlinear multi-label prediction tasks. Experiments show the advantage of the adaptive and the fixed thresholding algorithms, in terms of various multi-label performance metrics.

The Emerging Trends of Multi-Label Learning.

Exabytes of data are generated daily by humans, leading to the growing needs for new efforts in dealing with the grand challenges for multi-label learning brought by big data. For example, extreme multi-label classification is an active and rapidly growing research area that deals with classification tasks with extremely large number of classes or labels; utilizing massive data with limited supervision to build a multi-label classification model becomes valuable for practical applications, etc. Besides these, there are tremendous efforts on how to harvest the strong learning capability of deep learning to better capture the label dependencies in multi-label learning, which is the key for deep learning to address real-world classification tasks. However, it is noted that there have been a lack of systemic studies that focus explicitly on analyzing the emerging trends and new challenges of multi-label learning in the era of big data. It is imperative to call for a comprehensive survey to fulfil this mission and delineate future research directions and new applications.