Semi-supervised Ensemble Learning Research Articles

Semi-supervised ensemble learning for human activity recognition in casas Kyoto dataset

—The automatic identification of human physical activities, commonly referred to as Human Activity Recognition (HAR), has garnered significant interest and application across various sectors, including entertainment, sports, and notably health. Within the realm of health, a myriad of applications exists, contingent upon the nature of experimentation, the activities under scrutiny, and the methodology employed for data and information acquisition. This diversity opens doors to multifaceted applications, including support for the well-being and safeguarding of elderly individuals afflicted with neurodegenerative diseases, especially in the context of smart homes. Within the existing literature, a multitude of datasets from both indoor and outdoor environments have surfaced, significantly contributing to the activity identification processes. One prominent dataset, the CASAS project developed by Washington State University (WSU) University, encompasses experiments conducted in indoor settings. This dataset facilitates the identification of a range of activities, such as cleaning, cooking, eating, washing hands, and even making phone calls. This article introduces a model founded on the principles of Semi-supervised Ensemble Learning, enabling the harnessing of the potential inherent in distance-based clustering analysis. This technique aids in the identification of distinct clusters, each encapsulating unique activity characteristics. These clusters serve as pivotal inputs for the subsequent classification process, which leverages supervised techniques. The outcomes of this approach exhibit great promise, as evidenced by the quality metrics' analysis, showcasing favorable results compared to the existing state-of-the-art methods. This integrated framework not only contributes to the field of HAR but also holds immense potential for enhancing the capabilities of smart homes and related applications.

Research on habitat quality assessment and decision-making based on Semi-supervised Ensemble Learning method—Daxia River Basin, China

Habitat quality is an indicator of the ecological evolution of a region, and evaluating habitat quality utilizing Machine Learning approaches can reflect the ecological status of a region more objectively. Gridded statistical ecological factors in the study region were utilized to build the Ecological Information State Layer (EISL) in the grid space. Calculate the Environmental Quality Index (EQI) of the sampled grid to evaluate the performance index of the models on the sampled grid. The model with the optimal performance index is selected for the task of habitat quality classification in the research region, and then especially decision-making advice is offered by inverting the degree of influence of ecological elements on habitat quality. The results show that: (1) The Semi-supervised Ensemble Learning model (Tri-training) Accuracy, Kappa coefficient, and F1-score are 0.93, 0.89, and 0.92, respectively, as the optimal model for the habitat quality classification task. (2) Based on the results of the ecological categories calculation, the current ecosystem quality of the southwestern section of the Daxia River Basin is maintained as a positive indicator. the ecosystem quality inside the living space centered on the cities of Linxia and Hezuo shows negative indicator changes. (3) The calculation results of the inverse importance of ecological factors show that vegetation index and human population density are the key factors impacting the habitat quality of the Daxia River Basin. In the ecological management of Daxia River Basin, the degree of spatial aggregation of people's settlements should be reduced while preserving and expanding vegetation cover. Using Tri-training's comprehensive analysis of multiple ecological factors, regional habitat quality can be accurately assessed.

A local semi-supervised ensemble learning strategy for the data‐driven soft sensor of the power prediction in wind power generation

Soft sensors have been widely employed to estimate the power prediction in wind power generation that are difficult to measure in real-time. However, strongly nonlinear, dynamic time-varying, and multi-rate data characteristics are major causes of poor soft sensor performance. To address this problem, a strategy of building a soft sensor model based on local semi-supervised ensemble learning of least squares support vector regression (LS-EL-LSSVR) is proposed in this paper. Firstly, to reduce the influence of nonlinearity, a localization method based on the combination of time and space criteria is proposed, which can overcome the drawback of the clustering method based on a spatial criterion. Subsequently, the self-training semi-supervised classification method is used to achieve semi-supervised clustering of unlabeled samples to improve the generalization performance of the soft sensor model. Afterward, based on the local semi-supervised clustering sample dataset, the soft sensor model is established according to the least squares support vector machine method. Moreover, a selective ensemble learning method based on the prediction accuracy of local models and an adaptive combination weight calculation method of sub-models is proposed to realize selective ensemble learning and improve the prediction accuracy. The parameters of the proposed method are finalized automatically by the particle swarm optimization technique. The proposed method is simulated by using the actual spatial dynamic wind power prediction dataset, the results demonstrate the effectiveness of the proposed method in dealing with nonlinear, dynamic time-varying, and multi-rate data regression problems in wind power generation process.

Multi-DS Strategy for Source Camera Identification in Few-Shot Sample Data Sets

Source camera identification (SCI) is an intriguing problem in digital forensics, which identifies the source device of given images. However, most existing works require sufficient training samples to ensure performance. In this work, we propose a method based on semi-supervised ensemble learning (multi-DS) strategy, which extends labeled data set by multi-distance-based clustering strategy and then calibrate the pseudo-labels through a self-correction mechanism. Next, we iteratively perform the calibration-appending-training process to improve our model. We design comprehensive experiments, and our model achieves satisfactory performance on benchmark public databases (i.e., Dresden, VISION, and SOCRatES).

Imbalanced fault diagnosis based on semi-supervised ensemble learning

Imbalanced fault diagnosis based on semi-supervised ensemble learning

A semi-supervised decision support system to facilitate antibiotic stewardship for urinary tract infections

Urinary Tract Infections (UTIs) are among the most frequently occurring infections in the hospital. Urinalysis and urine culture are the main tools used for diagnosis. Whereas urinalysis is sufficiently sensitive for detecting UTI, it has a relatively low specificity, leading to unnecessary treatment with antibiotics and the risk of increasing antibiotic resistance. We performed an evaluation of the current diagnostic process with an expert-based label for UTI as outcome, retrospectively established using data from the Electronic Health Records. We found that the combination of urinalysis results with the Gram stain and other readily available parameters can be used effectively for predicting UTI. Based on the obtained information, we engineered a clinical decision support system (CDSS) using the reliable semi-supervised ensemble learning (RESSEL) method, and found it to be more accurate than urinalysis or the urine culture for prediction of UTI. The CDSS provides clinicians with this prediction within hours of ordering a culture and thereby enables them to hold off on prematurely prescribing antibiotics for UTI while awaiting the culture results.

Semi-Supervised Ensemble Learning Framework for Accelerating Power System Transient Stability Knowledge Base Generation

Machine learning approaches have exhibited high potential in power system transient stability assessment (TSA), yet their initial preparation stages of stability knowledge base generation (SKBG) based on time-domain simulations often undergo high computational costs. In fact, how to ease the heavy computational burden of SKBG without sacrificing the reliability is still a significant challenge. To address this problem, this paper develops a semi-supervised ensemble learning (SSEL) framework for reliable SKBG acceleration, without additional need for computing hardware upgrading. In particular, it performs detailed simulations for a minority of cases and fast simulations for the majority ones to reduce the total computation time. Considering the absence of stability status information of those fast simulated cases, an SSEL scheme is systematically designed to reliably label their stability status. Before implementing SSEL, two concise feature descriptors are first introduced to efficiently extract transient features from multiplex system trajectories. Then, all the cases are characterized in a unified feature space, whereby a series of semi-supervised support vector machines are trained in randomly formed subspaces. Afterward, these single machines are systematically combined to derive an enhanced SSEL model, which is able to make reliable and robust labeling decisions. Further, a backtrace strategy is carefully devised for SSEL, so as to maintain the high reliability of SKBG. Test results on the IEEE 39-bus system and the realistic GD Power Grid in South China illustrate the excellent performances of the proposed framework on SKBG acceleration.

Missing label imputation through inception-based semi-supervised ensemble learning

In classification tasks, unlabeled data bring the uncertainty in the learning process, which may result in the degradation of the performance. In this paper, we propose a novel semi-supervised inception neural network ensemble-based architecture to achieve missing label imputation. The main idea of the proposed architecture is to use smaller ensembles within a larger ensemble to involve diverse ways of missing label imputation and internal transformation of feature representation, towards enhancing the prediction accuracy. Following the process of imputing the missing labels of unlabeled data, the human-labeled data and the data with imputed labels are used together as a training set for the credible classifiers learning. Meanwhile, we discuss how this proposed approach is more effective as compared to the traditional ensemble learning approaches. Our proposed approach is evaluated on different well-known benchmark data sets, and the experimental results show the effectiveness of the proposed method. In addition, the approach is validated by statistical analysis using Wilcoxon signed rank test and the results indicate statistical significance of the performance improvement in comparison with other methods.

Semi-Supervised Ensemble Learning for Dealing with Inaccurate and Incomplete Supervision

In real-world tasks, obtaining a large set of noise-free data can be prohibitively expensive. Therefore, recent research tries to enable machine learning to work with weakly supervised datasets, such as inaccurate or incomplete data. However, the previous literature treats each type of weak supervision individually, although, in most cases, different types of weak supervision tend to occur simultaneously. Therefore, in this article, we present Smart MEnDR, a Classification Model that applies Ensemble Learning and Data-driven Rectification to deal with inaccurate and incomplete supervised datasets. The model first applies a preliminary phase of ensemble learning in which the noisy data points are detected while exploiting the unlabelled data. The phase employs a semi-supervised technique with maximum likelihood estimation to decide on the disagreement rate. Second, the proposed approach applies an iterative meta-learning step to tackle the problem of knowing which points should be made correct to improve the performance of the final classifier. To evaluate the proposed framework, we report the classification performance, noise detection, and the labelling accuracy of the proposed method against state-of-the-art techniques. The experimental results demonstrate the effectiveness of the proposed framework in detecting noise, providing correct labels, and attaining high classification performance.

Human action recognition method based on multi-view semi-supervised ensemble learning

Human action recognition method based on multi-view semi-supervised ensemble learning

Semi-supervised Ensemble Learning for Efficient Cancer Sample Classification from miRNA Gene Expression Data

Traditional classifiers often fail to produce desired classification accuracy because of inadequate training samples present in microRNA (miRNA) gene expression cancer datasets. In this context, we propose a novel semi-supervised ensemble learning (SSEL) strategy combining the (advantages of) semi-supervised learning and ensemble learning which is able to produce better results than the individual constituent classifiers. The proposed method is validated using eight publicly available miRNA gene expression datasets of pancreatic and colorectal cancers with respect to classification accuracy, precision, recall, macro $$F_{1}$$ -measure and kappa in comparison to six other state-of-the-art methods. The experimental results reveal that the proposed SSEL method significantly dominates other compared methods for cancer sample classification. The results of the statistical significance tests, receiver operating characteristic curve and area under curve justify the relevance of the better results in favor of the proposed method.

A reliable ensemble based approach to semi-supervised learning

Semi-supervised learning (SSL) methods attempt to achieve better classification of unseen data through the use of unlabeled data than can be achieved by learning from the available labeled data alone. Most SSL methods require the user to familiarize themselves with novel, complex concepts and to ensure the underlying assumptions made by these methods match the problem structure, or they risk a decrease in predictive performance. In this paper, we present the reliable semi-supervised ensemble learning (RESSEL) method, which exploits unlabeled data by using it to generate diverse classifiers through self-training and combines these classifiers into an ensemble for prediction. Our method functions as a wrapper around a supervised base classifier and refrains from introducing additional problem dependent assumptions. We conduct experiments on a number of commonly used data sets to prove its merit. The results show RESSEL improves significantly upon the supervised alternatives, provided that the base classifier which is used is able to produce adequate probability-based rankings. It is shown that RESSEL is reliable in that it delivers results comparable to supervised learning methods if this requirement is not met, while the method also broadens the range of good parameter values. Furthermore, RESSEL is demonstrated to outperform existing self-labeled wrapper approaches.

Data mining and machine learning approaches for prediction modelling of schistosomiasis disease vectors: Epidemic disease prediction modelling.

This research presents viable solutions for prediction modelling of schistosomiasis disease based on vector density. Novel training models proposed in this work aim to address various aspects of interest in the artificial intelligence applications domain. Topics discussed include data imputation, semi-supervised labelling and synthetic instance simulation when using sparse training data. Innovative semi-supervised ensemble learning paradigms are proposed focusing on labelling threshold selection and stringency of classification confidence levels. A regression-correlation combination (RCC) data imputation method is also introduced for handling of partially complete training data. Results presented in this work show data imputation precision improvement over benchmark value replacement using proposed RCC on 70% of test cases. Proposed novel incremental transductive models such as ITSVM have provided interesting findings based on threshold constraints outperforming standard SVM application on 21% of test cases and can be applied with alternative environment-based epidemic disease domains. The proposed incremental transductive ensemble approach model enables the combination of complimentary algorithms to provide labelling for unlabelled vector density instances. Liberal (LTA) and strict training approaches provided varied results with LTA outperforming Stacking ensemble on 29.1% of test cases. Proposed novel synthetic minority over-sampling technique (SMOTE) equilibrium approach has yielded subtle classification performance increases which can be further interrogated to assess classification performance and efficiency relationships with synthetic instance generation.

A Weighted Voting Ensemble Self-Labeled Algorithm for the Detection of Lung Abnormalities from X-Rays

During the last decades, intensive efforts have been devoted to the extraction of useful knowledge from large volumes of medical data employing advanced machine learning and data mining techniques. Advances in digital chest radiography have enabled research and medical centers to accumulate large repositories of classified (labeled) images and mostly of unclassified (unlabeled) images from human experts. Machine learning methods such as semi-supervised learning algorithms have been proposed as a new direction to address the problem of shortage of available labeled data, by exploiting the explicit classification information of labeled data with the information hidden in the unlabeled data. In the present work, we propose a new ensemble semi-supervised learning algorithm for the classification of lung abnormalities from chest X-rays based on a new weighted voting scheme. The proposed algorithm assigns a vector of weights on each component classifier of the ensemble based on its accuracy on each class. Our numerical experiments illustrate the efficiency of the proposed ensemble methodology against other state-of-the-art classification methods.

An ensemble semi-supervised learning method for predicting defaults in social lending

Social lending is made between peers, and with the risk that the investor can take direct damages from the borrower’s failure to repay, accurate default prediction for borrowers is important. The repayment result can be known after the end of the repayment period, and such data is limited. However, social loans are matched online in real time and large amounts of unlabeled data are being generated. In this paper, we propose a method to combine label propagation and transductive support vector machine (TSVM) with Dempster–Shafer theory for accurate default prediction of social lending using unlabeled data. In order to train a lot of data effectively, we ensemble semi-supervised learning methods with different characteristics. Label propagation is performed so that data having similar features are assigned to the same class and TSVM makes moving away data having different features. Dempster–Shafer fusion method allows accurate labeling by exploiting the merits of the two methods. Experiments are performed using the open data set from Lending Club. The accuracy of the proposed method is improved by about 10% against that of the model using only labeled data, and more accurate labeling can be performed through the proposed ensemble method.

On Ensemble SSL Algorithms for Credit Scoring Problem

Credit scoring is generally recognized as one of the most significant operational research techniques used in banking and finance, aiming to identify whether a credit consumer belongs to either a legitimate or a suspicious customer group. With the vigorous development of the Internet and the widespread adoption of electronic records, banks and financial institutions have accumulated large repositories of labeled and mostly unlabeled data. Semi-supervised learning constitutes an appropriate machine- learning methodology for extracting useful knowledge from both labeled and unlabeled data. In this work, we evaluate the performance of two ensemble semi-supervised learning algorithms for the credit scoring problem. Our numerical experiments indicate that the proposed algorithms outperform their component semi-supervised learning algorithms, illustrating that reliable and robust prediction models could be developed by the adaptation of ensemble techniques in the semi-supervised learning framework.

An Ensemble SSL Algorithm for Efficient Chest X-Ray Image Classification

A critical component in the computer-aided medical diagnosis of digital chest X-rays is the automatic detection of lung abnormalities, since the effective identification at an initial stage constitutes a significant and crucial factor in patient’s treatment. The vigorous advances in computer and digital technologies have ultimately led to the development of large repositories of labeled and unlabeled images. Due to the effort and expense involved in labeling data, training datasets are of a limited size, while in contrast, electronic medical record systems contain a significant number of unlabeled images. Semi-supervised learning algorithms have become a hot topic of research as an alternative to traditional classification methods, exploiting the explicit classification information of labeled data with the knowledge hidden in the unlabeled data for building powerful and effective classifiers. In the present work, we evaluate the performance of an ensemble semi-supervised learning algorithm for the classification of chest X-rays of tuberculosis. The efficacy of the presented algorithm is demonstrated by several experiments and confirmed by the statistical nonparametric tests, illustrating that reliable and robust prediction models could be developed utilizing a few labeled and many unlabeled data.

Multicore design space exploration via semi-supervised ensemble learning

Multicore design space exploration via semi-supervised ensemble learning

Processor Design Space Exploration via Statistical Sampling and Semi-Supervised Ensemble Learning

The design space exploration (DSE) has become a major challenge in microprocessors design due to the increasing complexity of microprocessor architecture and the extremely time-consuming software simulation technology. To more effectively and efficiently perform DSE, recently machine learning techniques are widely explored to build predictive models with a small set of simulations. However, most previous models are supervised models and the training samples are randomly selected. Thus they still suffered from high simulation cost or low prediction accuracy. In order to minimize the simulation overhead for DSE, this paper proposes an efficient DSE method which combines Latin hypercube sampling and semi-supervised ensemble learning technique. Latin hypercube sampling is first employed to select a small set of representative design points for simulation. Then a semi-supervised learning based AdaBoost model (SemiBoost) is proposed to predict the responses of the configurations that have not been simulated. We conduct extensive evaluations on the benchmarks of SPEC CPU2006 suite, and the experimental results demonstrate that the proposed SemiBoost model is superior to existing state-of-the-art models in terms of both efficiency and effectiveness.

SSEL-ADE: A semi-supervised ensemble learning framework for extracting adverse drug events from social media.

With the development of Web 2.0 technology, social media websites have become lucrative but under-explored data sources for extracting adverse drug events (ADEs), which is a serious health problem. Besides ADE, other semantic relation types (e.g., drug indication and beneficial effect) could hold between the drug and adverse event mentions, making ADE relation extraction - distinguishing ADE relationship from other relation types - necessary. However, conducting ADE relation extraction in social media environment is not a trivial task because of the expertise-dependent, time-consuming and costly annotation process, and the feature space's high-dimensionality attributed to intrinsic characteristics of social media data. This study aims to develop a framework for ADE relation extraction using patient-generated content in social media with better performance than that delivered by previous efforts. To achieve the objective, a general semi-supervised ensemble learning framework, SSEL-ADE, was developed. The framework exploited various lexical, semantic, and syntactic features, and integrated ensemble learning and semi-supervised learning. A series of experiments were conducted to verify the effectiveness of the proposed framework. Empirical results demonstrate the effectiveness of each component of SSEL-ADE and reveal that our proposed framework outperforms most of existing ADE relation extraction methods The SSEL-ADE can facilitate enhanced ADE relation extraction performance, thereby providing more reliable support for pharmacovigilance. Moreover, the proposed semi-supervised ensemble methods have the potential of being applied to effectively deal with other social media-based problems.