Multi-label K-nearest Neighbor Research Articles

Enhancing smart home appliance recognition with wavelet and scalogram analysis using data augmentation

The development of smart homes, equipped with devices connected to the Internet of Things (IoT), has opened up new possibilities to monitor and control energy consumption. In this context, non-intrusive load monitoring (NILM) techniques have emerged as a promising solution for the disaggregation of total energy consumption into the consumption of individual appliances. The classification of electrical appliances in a smart home remains a challenging task for machine learning algorithms. In the present study, we propose comparing and evaluating the performance of two different algorithms, namely Multi-Label K-Nearest Neighbors (MLkNN) and Convolutional Neural Networks (CNN), for NILM in two different scenarios: without and with data augmentation (DAUG). Our results show how the classification results can be better interpreted by generating a scalogram image from the power consumption signal data and processing it with CNNs. The results indicate that the CNN model with the proposed data augmentation performed significantly higher, obtaining a mean F1-score of 0.484 (an improvement of +0.234), better than the other methods. Additionally, after performing the Friedman statistical test, it indicates that it is significantly different from the other methods compared. Our proposed system can potentially reduce energy waste and promote more sustainable energy use in homes and buildings by providing personalized feedback and energy savings tips.

The Implementation of Multi Label K- Nearest Neighbor Algorithm To Classifying Essay Answers

One way of assessing essays is to use the STAR (Situation, Task, Action, Result) method. This method aims to classify whether the essay already reflects the values of the situation, task, action, and outcome labels presented by the prospective driving teacher. Multi-label classification refers to a classification task in which each data sample can be classified into multiple categories or labels simultaneously. Various methods have been developed to perform multi-label text classification, such as MLKNN, MLTSVM, and MLP. This research aims to determine the value of the accuracy of the score. Therefore, the Multi-Label K-Nearest Neighbor, or MLKNN, algorithm can be implemented to classify essays with five labels: no star, situation, task, action, and result. With a total dataset of 110, which will be obtained from one of the assessors from the driving teacher. The MLKNN algorithm produces an accuracy rate with an average value of 15.6%.

Emotion classification for short texts: an improved multi-label method

The process of computationally identifying and categorizing opinions expressed in a piece of text is of great importance to support better understanding and services to online users in the digital environment. However, accurate and fast multi-label automatic classification is still insufficient. By considering not only individual in-sentence features but also the features in the adjacent sentences and the full text of the tweet, this study adjusted the Multi-label K-Nearest Neighbors (MLkNN) classifier to allow iterative corrections of the multi-label emotion classification. It applies the new method to improve both the accuracy and speed of emotion classification for short texts on Twitter. By carrying out three groups of experiments on the Twitter corpus, this study compares the performance of the base classifier of MLkNN, the sample-based MLkNN (S-MLkNN), and the label-based MLkNN (L-MLkNN). The results show that the improved MLkNN algorithm can effectively improve the accuracy of emotion classification of short texts, especially when the value of K in the MLkNN base classifier is 8, and the value of α is 0.7, and the improved L-MLkNN algorithm outperforms the other methods in the overall performance and the recall rate reaches 0.8019. This study attempts to obtain an efficient classifier with smaller training samples and lower training costs for sentiment analysis. It is suggested that future studies should pay more attention to balancing the efficiency of the model with smaller training sample sizes and the completeness of the model to cover various scenarios.

A Hybrid Feature Selection and Multi-Label Driven Intelligent Fault Diagnosis Method for Gearbox

Gearboxes are utilized in practically all complicated machinery equipment because they have great transmission accuracy and load capacities, so their failure frequently results in significant financial losses. The classification of high-dimensional data remains a difficult topic despite the fact that numerous data-driven intelligent diagnosis approaches have been suggested and employed for compound fault diagnosis in recent years with successful outcomes. In order to achieve the best diagnostic performance as the ultimate objective, a feature selection and fault decoupling framework is proposed in this paper. That is based on multi-label K-nearest neighbors (ML-kNN) as classifiers and can automatically determine the optimal subset from the original high-dimensional feature set. The proposed feature selection method is a hybrid framework that can be divided into three stages. The Fisher score, information gain, and Pearson’s correlation coefficient are three filter models that are used in the first stage to pre-rank candidate features. In the second stage, a weighting scheme based on the weighted average method is proposed to fuse the pre-ranking results obtained in the first stage and optimize the weights using a genetic algorithm to re-rank the features. The optimal subset is automatically and iteratively found in the third stage using three heuristic strategies, including binary search, sequential forward search, and sequential backward search. The method takes into account the consideration of feature irrelevance, redundancy and inter-feature interaction in the selection process, and the selected optimal subsets have better diagnostic performance. In two gearbox compound fault datasets, ML-kNN performs exceptionally well using the optimal subset with subset accuracy of 96.22% and 100%. The experimental findings demonstrate the effectiveness of the proposed method in predicting various labels for compound fault samples to identify and decouple compound faults. The proposed method performs better in terms of classification accuracy and optimal subset dimensionality when compared to other existing methods.

An Improved Multilabel k-Nearest Neighbor Algorithm Based on Value and Weight

Multilabel data share important features, including label imbalance, which has a significant influence on the performance of classifiers. Because of this problem, a widely used multilabel classification algorithm, the multilabel k-nearest neighbor (ML-kNN) algorithm, has poor performance on imbalanced multilabel data. To address this problem, this study proposes an improved ML-kNN algorithm based on value and weight. In this improved algorithm, labels are divided into minority and majority, and different strategies are adopted for different labels. By considering the label of latent information carried by the nearest neighbors, a value calculation method is proposed and used to directly classify majority labels. Additionally, to address the misclassification problem caused by a lack of nearest neighbor information for minority labels, weight calculation is proposed. The proposed weight calculation converts distance information with and without label sets in the nearest neighbors into weights. The experimental results on multilabel datasets from different benchmarks demonstrate the performance of the algorithm, especially for datasets with high imbalance. Different evaluation metrics show that the results are improved by approximately 2–10%. The verified algorithm could be applied to a multilabel classification of various fields involving label imbalance, such as drug molecule identification, building identification, and text categorization.

ML-k’sNN: Label Dependent k Values for Multi-Label k-Nearest Neighbor Rule

Multi-label classification as a data mining task has recently attracted increasing interest from researchers. Many current data mining applications address problems with instances that belong to more than one category. These problems require the development of new, efficient methods. Multi-label k-nearest neighbors rule, ML-kNN, is among the best-performing methods for multi-label problems. Current methods use a unique k value for all labels, as in the single-label method. However, the distributions of the labels are frequently very different. In such scenarios, a unique k value for the labels might be suboptimal. In this paper, we propose a novel approach in which each label is predicted with a different value of k. Obtaining the best k for each label is stated as an optimization problem. Three different algorithms are proposed for this task, depending on which multi-label metric is the target of our optimization process. In a large set of 40 real-world multi-label problems, our approach improves the results of two different tested ML-kNN implementations.

Scheduling Post-Disaster Power System Repair With Incomplete Failure Information: A Learning-to-Rank Approach

This paper proposes a novel repair rule set (RRS) for scheduling the power system infrastructure repair after the occurrence of extreme events. RRS is made up of multiple repair rules, each of them can be applied in arbitrary post-disaster failure scenarios to rank the repair actions by priority. A learning-to-rank technique called AdaRank is used to train the repair rules by combining the weak learners derived from the dynamic repair scheduling model. Then, RRS is constructed by iteratively clustering the training cases and retraining the repair rule for each cluster. Increasing the number of repair rules within RRS allows it to differentiate various types of failure scenarios, thereby improving its performance. Further combined with multi-label K nearest neighbor (ML-KNN) technique, RRS is able to schedule the repair without the full knowledge of real-time failure information, such as the estimated repair time. The results of case studies on IEEE-118 test systems show that the proposed method has a desirable performance compared to the exact mathematical optimization model. Moreover, it reduces the requirement for failure information while significantly improving the computational efficiency.

Local-based [formula omitted] values for multi-label [formula omitted]-nearest neighbors rule

Multi-label learning is a growing field in machine learning research. Many applications address instances that simultaneously belong to many categories, which cannot be disregarded if optimal results are desired. Among the many algorithms developed for multi-label learning, the multi-label k-nearest neighbor method is among the most successful. However, in a difficult classification task, such as multi-label learning, a challenge that arises in the k-nearest neighbor approach is the assignment of the appropriate value of k. Although a suitable value might be obtained using cross-validation, it is unlikely that the same value will be optimal for the whole space spanned by the training set. It is evident that different regions of the feature space would have different distributions of instances and labels that would require different values of k. The very complex boundaries among the many present labels make the necessity of local k values even more important than in the case with a single-label k-nearest neighbor. We present a simple yet powerful approach for setting a local value of k. We associate a potentially different k with every prototype and obtain the best value of that k by optimizing the criterion consisting of the local effect of the different k values in the neighborhood of the prototype. The proposed method has a fast training stage, as it only uses the neighborhood of each training instance to set the local k value. The complexity of the proposed method in terms of the testing time is similar to that of the standard multi-label k-nearest neighbor approach. Experiments performed on a set of 20 problems show that not only does our proposed method significantly outperform the standard multi-label k-nearest neighbor rule but also the locally adaptive multi-label k-nearest neighbor method can benefit from a local k value.

Intelligent diagnosis of resistance variant multiple fault locations of mine ventilation system based on ML-KNN.

The resistance variant faults (RVFs) observed in the mine ventilation system can utterly restrict mine safety production. Herein, a machine learning model, which is based on multi-label k-nearest neighbor (ML-KNN), is proposed to solve the problem of the rapid and accurate diagnosis of the RVFs that occur at multiple locations within the mine ventilation system. The air volume that passes through all the branches of the ventilation network, including the residual branches, was used as the diagnostic model input after the occurrence of multiple faults, whereas the label vector of the fault locations was used as the model’s output. In total, seven evaluation indicators and 1800 groups of randomly simulated faults at the typical locations in a production mine with 153 nodes and 223 branches were considered to evaluate the feasibility of the proposed model to solve for multiple fault locations diagnostic and verify the model’s generalization ability. After ten-fold cross-validation of the training sets containing 1600 groups of fault instances, the diagnostic accuracy of the model tested with the air volume of all 223 branches and the 71 residual branches’ air volume as input was 73.6% and 72.3%, respectively. On the other hand, To further evaluate the diagnostic performance of the model, 200 groups of the multiple fault instances that were not included in the training were tested. The accuracy of the fault location diagnosis was 76.5% and 73.5%, and the diagnostic time was 9.9s and 12.16s for the multiple faults instances with all 223 branches’ air volume and the 71 residual branches’ air volume as observation characteristics, respectively. The data show that the machine learning model based on ML-KNN shows good performance in the problem of resistance variant multiple fault locations diagnoses of the mine ventilation system, the multiple fault locations diagnoses can be carried out with all the branches’ air volume or the residual branches’ air volume as the input of the model, the diagnostic average accuracy is higher than 70%, and the average diagnosis time is less than one minute. Hence, the proposed model’s diagnostic accuracy and speed can meet the engineering requirements for the diagnosis of multiple fault locations for a real ventilation system in the field, and this model can effectively replace personnel to discover ventilation system failures, and also lays a good foundation for the construction of intelligent ventilation systems.

Automated Multi Label Classification on Fertilizer Themed Patent Documents in Indonesia


 Patent literature research has a high scientific value for the industrial, commercial, legal, and policymaking communities. Therefore, patent analysis has become crucial. Patent topic classification is an important process in patent topic modeling analysis. However, the classification process is time-consuming and expensive, as it is usually carried out manually by an expert. Moreover, a patent document may be categorised in more than one category or label, further complicating the task. As the number of patent documents submitted increases, creating an automated patent classification system that yields accurate results becomes increasingly critical. Therefore, in this paper, we analyse the performance of two algorithms with regard to multi-label classification in patent documents: multi-label k-nearest neighbor (ML-KNN) and classifier chain k-nearest neighbor (CC-KNN), combined with latent Dirichlet allocation (LDA). These two methods have a considerable advantage in handling the continuously updated dataset; they also exhibit superior performance compared to other multi-label learning algorithms. This study also compares these two algorithms with the term frequency (TF)-weighting measure. The optimal value obtained is based on the following evaluation parameters: micro F1, accuracy, Hamming loss, and one error. The result shows that the ML-KNN method is better than the CC-KNN method and that the multi-label classification based on topics (patent LDA) is better than the TF-weighting technique.

Hyperparameter tuning for multi-label classification of feedbacks in online courses

In this work, we propose the extension of a methodology for the multi-label classification of feedback according to the Hattie and Timperley feedback model, incorporating a hyperparameter tuning stage. It is analyzed whether the incorporation of the hyperparameter tuning stage prior to the execution of the algorithms support vector machines, random forest and multi-label k-nearest neighbors, improves the performance metrics of multi-label classifiers that automatically locate the feedback generated by a teacher to the activities sent by students in online courses on the Blackboard platform at the task, process, regulation, praise and other levels proposed in the feedback model by Hattie and Timperley. The grid search strategy is used to refine the hyperparameters of each algorithm. The results show that the adjustment of the hyperparameters improves the performance metrics for the data set used.

Classification of drugs based on mechanism of action using machine learning techniques

The mechanism of action is an important aspect of drug development. It can help scientists in the process of drug discovery. This paper provides a machine learning model to predict the mechanism of action of a drug. The machine learning models used in this paper are Binary Relevance K Nearest Neighbors (Type A and Type B), Multi-label K-Nearest Neighbors and a custom neural network. These machine learning models are evaluated using the mean column-wise log loss. The custom neural network model had the best accuracy with a log loss of 0.01706. This neural network model is integrated into a web application using Flask framework. A user can upload a custom testing features dataset, which contains the gene expression and the cell viability levels. The web application will output the top classes of drugs, along with the scatter plots for each of the drug.

Seismic resilient three-stage enhancement for gas distribution network using computational optimization algorithms

A computer-simulation-based three-stage optimization strategy is proposed for the resilience enhancement of urban gas distribution networks (GDNs). In stage I (pre-earthquake stage), the Fixed Proportion and Direct Comparison Genetic Algorithm (FPDC-GA) is applied to select key pipelines need to be strenthened or replaced under limited funding in preparation for future potential earthquakes. In stage II (post-earthquake stage), pressure tests must be carried out according to the gas leakage situation reported by users or detected by devices. The Multi-Label K-Nearest-Neighbor (ML-KNN) algorithm is used to predict the corresponding failed pipelines and optimize the pipeline pressure test order. In stage III (repair stage), a strategy based on a greedy algorithm is applied to optimize the pipeline repair sequence. The proposed methods were applied to the GDN of a city in northern China. The following conclusions were drawn from the results: (1) The FPDC-GA enhanced the robustness and resourcefulness of the GDN system to the maximum level within the available funding budget. (2) The pipeline pressure test order calculated using the ML-KNN algorithm was significantly improved compared with a random pressure test order or one based on the empirical failure probability of pipelines. (3) After optimization using a greedy algorithm, the performance recovery curves under different earthquake conditions were shaped as an exponential function, which indicates that the performance of the GDNs recovered in the most efficient manner. The findings of this study could be useful as tools for the seismic resilience enhancement of GDNs in different stages. The proposed optimization algorithms can also be extended to the lifeline of other networks.

Spatial Distribution Assessment of Terrorist Attack Types Based on I-MLKNN Model

Terrorist attacks are harmful to lives and property and seriously affect the stability of the international community and economic development. Exploring the regularity of terrorist attacks and building a model for assessing the risk of terrorist attacks (a kind of public safety risk, and it means the possibility of a terrorist attack) are of great significance to the security and stability of the international community and to global anti-terrorism. We propose a fusion of Inverse Distance Weighting (IDW) and a Multi-label k-Nearest Neighbor (I-MLKNN)-based assessment model for terrorist attacks, which is in a grid-scale and considers 17 factors of socio-economic and natural environments, and applied the I-MLKNN assessment model to assess the risk of terrorist attacks in Southeast Asia. The results show the I-MLKNN multi-label classification algorithm is proven to be an ideal tool for the assessment of the spatial distribution of terrorist attacks, and it can assess the risk of different types of terrorist attacks, thus revealing the law of distribution of different types of terrorist attacks. The terrorist attack risk assessment results indicate that Armed Attacks, Bombing/Explosions and Facility/Infrastructure Attacks in Southeast Asia are high-risk terrorist attack events, and the southernmost part of Thailand and the Philippines are high-risk terrorist attack areas for terrorism. We do not only provide a reference for incorporating spatial features in multi-label classification algorithms, but also provide a theoretical basis for decision-makers involved in terrorist attacks, which is meaningful to the implementation of the international counter-terrorism strategy.

Clinical decision-making framework against over-testing based on modeling implicit evaluation criteria

Different statistical methods include various subjective criteria that can prevent over-testing. However, no unified framework that defines generalized objective criteria for various diseases is available to determine the appropriateness of diagnostic tests recommended by doctors. We present the clinical decision-making framework against over-testing based on modeling the implicit evaluation criteria (CDFO-MIEC). The CDFO-MIEC quantifies the subjective evaluation process using statistics-based methods to identify over-testing. Furthermore, it determines the test’s appropriateness with extracted entities obtained via named entity recognition and entity alignment. More specifically, implicit evaluation criteria are defined—namely, the correlation among the diagnostic tests, symptoms, and diseases, confirmation function, and exclusion function. Additionally, four evaluation strategies are implemented by applying statistical methods, including the multi-label k-nearest neighbor and the conditional probability algorithms, to model the implicit evaluation criteria. Finally, they are combined into a classification and regression tree to make the final decision. The CDFO-MIEC also provides interpretability by decision conditions for supporting each clinical decision of over-testing. We tested the CDFO-MIEC on 2,860 clinical texts obtained from a single respiratory medicine department in China with the appropriate confirmation by physicians. The dataset was supplemented with random inappropriate tests. The proposed framework excelled against the best competing text classification methods with a Mean_F1 of 0.9167. This determined whether the appropriate and inappropriate tests were properly classified. The four evaluation strategies captured the features effectively, and they were imperative. Therefore, the proposed CDFO-MIEC is feasible because it exhibits high performance and can prevent over-testing.

Modified ML-kNN and Rank SVM for Multi-label Pattern Classification

To develop an efficient multi-label classifier is the main objective of this paper. In multi-label learning tasks such as classification, each example is associated with a set of labels, and the task is to predict the label set whose size is unknown apriory for each unseen example. In a realistic scenario each object or entity belongs to a multi-label category. Multi-Label k-Nearest Neighbor (ML-kNN), Rank-SVM (Ranking Support Vector Machine) are two popular techniques used for multi-label pattern classification. ML-kNN is a multi-label version of standard kNN and Rank SVM is a multi-label extension of standard SVM. The main aim of this work is to enhance the performance of these methods. Multi-label classifiers generally consider ranking loss, Hamming loss, one error, average precision and coverage as a performance metrics.

Text GCN-SW-KNN: a novel collaborative training multi-label classification method for WMS application themes by considering geographic semantics

ABSTRACT Without explicit description of map application themes, it is difficult for users to discover desired map resources from massive online Web Map Services (WMS). However, metadata-based map application theme extraction is a challenging multi-label text classification task due to limited training samples, mixed vocabularies, variable length and content arbitrariness of text fields. In this paper, we propose a novel multi-label text classification method, Text GCN-SW-KNN, based on geographic semantics and collaborative training to improve classification accuracy. The semi-supervised collaborative training adopts two base models, i.e. a modified Text Graph Convolutional Network (Text GCN) by utilizing Semantic Web, named Text GCN-SW, and widely-used Multi-Label K-Nearest Neighbor (ML-KNN). Text GCN-SW is improved from Text GCN by adjusting the adjacency matrix of the heterogeneous word document graph with the shortest semantic distances between themes and words in metadata text. The distances are calculated with the Semantic Web of Earth and Environmental Terminology (SWEET) and WordNet dictionaries. Experiments on both the WMS and layer metadata show that the proposed methods can achieve higher F1-score and accuracy than state-of-the-art baselines, and demonstrate better stability in repeating experiments and robustness to less training data. Text GCN-SW-KNN can be extended to other multi-label text classification scenario for better supporting metadata enhancement and geospatial resource discovery in Earth Science domain.

Degradation State Partition and Compound Fault Diagnosis of Rolling Bearing Based on Personalized Multilabel Learning

The prognostic and health management (PHM) of rolling bearings has been a popular research area. Since bearing fault is inevitable during degradation, how to improve the PHM performance based on both degradation states and fault types is still an open problem. In this study, two multilabel learning algorithms are proposed for PHM of rolling bearings, named personalized binary relevance (PBR) and hierarchical multilabel K-nearest neighbor (HML-KNN), respectively. Degradation states and fault types are used as the labels of the bearing data so that each sample has a corresponding label sequence, that is to say, the PHM problem is converted to a multilabel learning problem. Both algorithms have a personalized search process, which can not only help samples build a personalized model to improve classification accuracy but also solve the problem of data imbalance between labels. At the same time, the two algorithms also have their own characteristics and focus on different application situations. The PBR algorithm has faster modeling speed, more flexible use, and replaceable subclassifiers. HML-KNN is a high-order algorithm with global information analysis capabilities through the hierarchical processing of data and the conversion of label information. Both methods have achieved good enough results in the XJTU-SY bearing dataset. In order to illustrate the practicality of the algorithm, the experimental part further increases the difficulty. Using only a single faulty sample as the training set to determine the type of compound fault, the algorithms also show high performance. In the era of industrial big data, the depth of data mining and the design of algorithm models will help us better manage equipment health.

MpsLDA-ProSVM: Predicting multi-label protein subcellular localization by wMLDAe dimensionality reduction and ProSVM classifier

Multi-label proteins play a significant role in life processes such as cell growth, development, and reproduction. Exploring protein subcellular localization (SCL) is a direct way to better understand the function of multi-label proteins in cells. This paper firstly presents a new prediction model named MpsLDA-ProSVM which predicts the SCL of multi-label proteins. Firstly, we utilize four coding algorithms including pseudo position-specific scoring matrix (PsePSSM), gene ontology (GO), conjoint triad (CT) and pseudo amino acid composition (PseAAC) to draw the feature information from protein sequences. Then, for the first time, we use a weighted multi-label linear discriminant analysis framework based on entropy weight form (wMLDAe) to refine and purify features. Finally, we input the optimal feature subset into the multi-label learning with label-specific features (LIFT) and multi-label k-nearest neighbor (ML-KNN) algorithms to obtain a synthetic ranking of relevant labels, and then use Prediction and Relevance Ordering based SVM (ProSVM) classifier to predict the SCLs. Tested by leave-one-out cross-validation (LOOCV), the overall actual accuracy on virus, plant, Gram-positive bacteria and Gram-negative bacteria datasets are 98.06%, 98.97%, 99.81% and 98.49%, which are 0.56%–9.16%, 1.07%–30.87%, 0.21%–6.91% and 3.99%–8.59% higher than other advanced methods respectively. By comparison, the model MpsLDA-ProSVM can effectively predict the specific location of multi-label proteins in cells.

Key Quality Indicators Prediction for Web Browsing with Embedded Filter Feature Selection

In this paper, the prediction of over-the-top service quality is discussed, which is a promising way for mobile network engineers to tackle service deterioration as early as possible. Currently, traditional mobile network operation often takes appropriate remedial measures, when receiving customers’ complaints about service problems. With the popularity of over-the-top services, this problem has become increasingly serious. Based on the service perception data crowd-sensed from massive smartphones in the mobile network, we first investigated the application of multi-label ReliefF, a well-known method of feature selection, in determining the feature weights of the perception data and propose a unified multi-label ReliefF (UML-ReliefF) algorithm. Then a feature-weighted multi-label k-nearest neighbor (ML-kNN) algorithm is proposed for the key quality indicators (KQI) prediction, by combining the UML-ReliefF and ML-kNN together in the learning. The experimental results for web browsing service show that UML-ReliefF can effectively identify the most influential features of the data and thus, lead to better performance for KQI prediction. The experiments also show that the feature-weighted KQI prediction is superior to its unweighted counterpart, since the former takes full advantage of all the features in the learning. Although there is still much room of improvement in the precision of the prediction, the proposed method is highly potential for network engineers to find the deterioration of service quality promptly and take measures before it is too late.