Semi-supervised Machine Learning Algorithm Research Articles

Identification of abnormal tribological regimes using a microphone and semi-supervised machine-learning algorithm

Functional surfaces in relative contact and motion are prone to wear and tear, resulting in loss of efficiency and performance of the workpieces/machines. Wear occurs in the form of adhesion, abrasion, scuffing, galling, and scoring between contacts. However, the rate of the wear phenomenon depends primarily on the physical properties and the surrounding environment. Monitoring the integrity of surfaces by offline inspections leads to significant wasted machine time. A potential alternate option to offline inspection currently practiced in industries is the analysis of sensors signatures capable of capturing the wear state and correlating it with the wear phenomenon, followed by in situ classification using a state-of-the-art machine learning (ML) algorithm. Though this technique is better than offline inspection, it possesses inherent disadvantages for training the ML models. Ideally, supervised training of ML models requires the datasets considered for the classification to be of equal weightage to avoid biasing. The collection of such a dataset is very cumbersome and expensive in practice, as in real industrial applications, the malfunction period is minimal compared to normal operation. Furthermore, classification models would not classify new wear phenomena from the normal regime if they are unfamiliar. As a promising alternative, in this work, we propose a methodology able to differentiate the abnormal regimes, i.e., wear phenomenon regimes, from the normal regime. This is carried out by familiarizing the ML algorithms only with the distribution of the acoustic emission (AE) signals captured using a microphone related to the normal regime. As a result, the ML algorithms would be able to detect whether some overlaps exist with the learnt distributions when a new, unseen signal arrives. To achieve this goal, a generative convolutional neural network (CNN) architecture based on variational auto encoder (VAE) is built and trained. During the validation procedure of the proposed CNN architectures, we were capable of identifying acoustics signals corresponding to the normal and abnormal wear regime with an accuracy of 97% and 80%. Hence, our approach shows very promising results for in situ and real-time condition monitoring or even wear prediction in tribological applications.

Bagging-based positive-unlabeled learning algorithm with Bayesian hyperparameter optimization for three-dimensional mineral potential mapping

Mineralization is a rare event. Hence, the geosciences datasets used for three-dimensional (3D) mineral potential mapping (MPM) are often imbalanced, consisting of scarce positive samples and abundant unlabeled data. Compared with selecting positive samples, it is challenging to select reliable negative samples in 3D MPM. However, the application of supervised machine learning algorithms in 3D MPM requires balanced positive and negative samples. Consequently, semi-supervised machine learning algorithms, which are only trained on positive samples, are widely used in 3D mineral prospecting. In this study, the bagging-based positive-unlabeled learning (PUL) algorithm that utilizes positive samples and unlabeled data in the training process was developed and applied to produce a 3D gold (Au) potential map in the Wulong Au district, China. This study employed Bayesian hyperparameter optimization to tune the hyperparameters of the bagging-based PUL algorithm. The performance of the bagging-based PUL algorithm was further compared with that of the widely used random forest, weights-of-evidence, one-class support vector machine, and continuous weighting approach in 3D MPM. The results demonstrated that the bagging-based PUL algorithm outperformed the aforementioned widely used predictive methods. The 3D mineral targets obtained by the bagging-based PUL algorithm can be beneficial for subsurface Au exploration in the Wulong Au district of China.

Earth Observation and Biodiversity Big Data for Forest Habitat Types Classification and Mapping

In the light of the “Biological Diversity” concept, habitats are cardinal pieces for biodiversity quantitative estimation at a local and global scale. In Europe EUNIS (European Nature Information System) is a system tool for habitat identification and assessment. Earth Observation (EO) data, which are acquired by satellite sensors, offer new opportunities for environmental sciences and they are revolutionizing the methodologies applied. These are providing unprecedented insights for habitat monitoring and for evaluating the Sustainable Development Goals (SDGs) indicators. This paper shows the results of a novel approach for a spatially explicit habitat mapping in Italy at a national scale, using a supervised machine learning model (SMLM), through the combination of vegetation plot database (as response variable), and both spectral and environmental predictors. The procedure integrates forest habitat data in Italy from the European Vegetation Archive (EVA), with Sentinel-2 imagery processing (vegetation indices time series, spectral indices, and single bands spectral signals) and environmental data variables (i.e., climatic and topographic), to parameterize a Random Forests (RF) classifier. The obtained results classify 24 forest habitats according to the EUNIS III level: 12 broadleaved deciduous (T1), 4 broadleaved evergreen (T2) and eight needleleaved forest habitats (T3), and achieved an overall accuracy of 87% at the EUNIS II level classes (T1, T2, T3), and an overall accuracy of 76.14% at the EUNIS III level. The highest overall accuracy value was obtained for the broadleaved evergreen forest equal to 91%, followed by 76% and 68% for needleleaved and broadleaved deciduous habitat forests, respectively. The results of the proposed methodology open the way to increase the EUNIS habitat categories to be mapped together with their geographical extent, and to test different semi-supervised machine learning algorithms and ensemble modelling methods.

Automated Power Lines Vegetation Monitoring Using High-Resolution Satellite Imagery

Vegetation Management is a significant preventive maintenance expense in many power transmission and distribution companies. Traditional Vegetation Management operational practices have proven ineffective and are rapidly becoming obsolete due to the lack of frequent inspection of vegetation and environmental states. The rise of satellite imagery data and machine learning provides an opportunity to close the loop with continuous data-driven vegetation monitoring. This paper proposes an automated framework for monitoring vegetation along power lines using high-resolution satellite imagery and a semi-supervised machine learning algorithm. The proposed satellite-based vegetation monitoring framework aims to reduce the cost and time of power line monitoring by partially replacing ground patrols and helicopter or drone inspection with satellite data analytics. It is implemented and demonstrated for a power distribution system operator (DSO) in the west of Norway. For further assessment, the satellite-based algorithm outcomes are compared with LiDAR survey data collected by helicopters. The results show the potential of the solution for reducing the monitoring costs for electric utilities.

Essential gene prediction using limited gene essentiality information-An integrative semi-supervised machine learning strategy.

Essential gene prediction helps to find minimal genes indispensable for the survival of any organism. Machine learning (ML) algorithms have been useful for the prediction of gene essentiality. However, currently available ML pipelines perform poorly for organisms with limited experimental data. The objective is the development of a new ML pipeline to help in the annotation of essential genes of less explored disease-causing organisms for which minimal experimental data is available. The proposed strategy combines unsupervised feature selection technique, dimension reduction using the Kamada-Kawai algorithm, and semi-supervised ML algorithm employing Laplacian Support Vector Machine (LapSVM) for prediction of essential and non-essential genes from genome-scale metabolic networks using very limited labeled dataset. A novel scoring technique, Semi-Supervised Model Selection Score, equivalent to area under the ROC curve (auROC), has been proposed for the selection of the best model when supervised performance metrics calculation is difficult due to lack of data. The unsupervised feature selection followed by dimension reduction helped to observe a distinct circular pattern in the clustering of essential and non-essential genes. LapSVM then created a curve that dissected this circle for the classification and prediction of essential genes with high accuracy (auROC > 0.85) even with 1% labeled data for model training. After successful validation of this ML pipeline on both Eukaryotes and Prokaryotes that show high accuracy even when the labeled dataset is very limited, this strategy is used for the prediction of essential genes of organisms with inadequate experimentally known data, such as Leishmania sp. Using a graph-based semi-supervised machine learning scheme, a novel integrative approach has been proposed for essential gene prediction that shows universality in application to both Prokaryotes and Eukaryotes with limited labeled data. The essential genes predicted using the pipeline provide an important lead for the prediction of gene essentiality and identification of novel therapeutic targets for antibiotic and vaccine development against disease-causing parasites.

Semi-Supervised Learning to Enhance Speech Signal for Mobile Communication

Machine learning algorithm to enhance the complex speech signal for mobile communication is one of the research problems in signal processing. The objective of this research paper is to develop a learning algorithm that improves the quality and intelligibility of voice signals that gets are corrupted by real world noise while they are transmitted through the channel. In this paper, we consider a semi-supervised machine learning algorithm for mobile phones that comes with system software to improve SNR of speech signal which is corrupted by manmade disturbance. Most of the disturbances are non-stationary where the effect of noise is non-uniform for all spectral components. In the projected algorithm training, the system is completed with a set of speech and noise data base. System parameters are derived during training process; these parameters are updated as per the disturbance present in the signal. These parameters are used to remove the noise present in speech signal. The obtained results show a substantial progress in SNR by 5–8% as compared to traditional methods.

Neurocognitive and functional heterogeneity in depressed youth.

Depression is a common psychiatric illness that often begins in youth, and is sometimes associated with cognitive deficits. However, there is significant variability in cognitive dysfunction, likely reflecting biological heterogeneity. We sought to identify neurocognitive subtypes and their neurofunctional signatures in a large cross-sectional sample of depressed youth. Participants were drawn from the Philadelphia Neurodevelopmental Cohort, including 712 youth with a lifetime history of a major depressive episode and 712 typically developing (TD) youth matched on age and sex. A subset (MDD n = 368, TD n = 200) also completed neuroimaging. Cognition was assessed with the Penn Computerized Neurocognitive Battery. A recently developed semi-supervised machine learning algorithm was used to delineate neurocognitive subtypes. Subtypes were evaluated for differences in both clinical psychopathology and brain activation during an n-back working memory fMRI task. We identified three neurocognitive subtypes in the depressed group. Subtype 1 was high-performing (high accuracy, moderate speed), Subtype 2 was cognitively impaired (low accuracy, slow speed), and Subtype 3 was impulsive (low accuracy, fast speed). While subtypes did not differ in clinical psychopathology, they diverged in their activation profiles in regions critical for executive function, which mirrored differences in cognition. Taken together, these data suggest disparate mechanisms of cognitive vulnerability and resilience in depressed youth, which may inform the identification of biomarkers for prognosis and treatment response.

Semi-Supervised Machine Learning Aided Anomaly Detection Method in Cellular Networks

The ever-increasing amount of data in cellular networks poses challenges for network operators to monitor the quality of experience (QoE). Traditional key quality indicators (KQIs)-based hard decision methods are difficult to undertake the task of QoE anomaly detection in the case of big data. To solve this problem, in this paper, we propose a KQIs-based QoE anomaly detection framework using semi-supervised machine learning algorithm, i.e., iterative positive sample aided one-class support vector machine (IPS-OCSVM). There are four steps for realizing the proposed method while the key step is combining machine learning with the network operator's expert knowledge using OCSVM. Our proposed IPS-OCSVM framework realizes QoE anomaly detection through soft decision and can easily fine-tune the anomaly detection ability on demand. Moreover, we prove that the fluctuation of KQIs thresholds based on expert knowledge has a limited impact on the result of anomaly detection. Finally, experiment results are given to confirm the proposed IPS-OCSVM framework for QoE anomaly detection in cellular networks.

Improved well log classification using semisupervised Gaussian mixture models and a new hyper-parameter selection strategy

Well log classification, the process of mapping well log measurements to lithofacies identified from core samples, is a common procedure in the oil and gas industry. Manually assigning lithofacies to the wire-line log measurements without core can be time consuming, and can also introduce a bias. Supervised machine learning algorithms are commonly used to automate this process, but they are prone to overfitting when the training data are scarce, which is common for well log classification problems. Semisupervised machine learning algorithms are designed for classification problems with minimal training data, and we adopt a semisupervised Gaussian mixture model (ssGMM) method to solve this problem. The dataset we consider for our study is from a machine learning competition held in 2016 and we simulate a semisupervised scenario by assuming only one out of the ten wells is the labeled data. We apply ssGMM to this well log dataset and compare its performance to the supervised method that was the winner of this competition, XGBoost. To try and improve the performance of both ssGMM and XGBoost, we also introduce a new hyper-parameter selection strategy that simultaneously uses the mean and standard deviation cross-validation scores, compared to the default procedure that only utilizes the mean cross-validation scores. Our results indicate that ssGMM is able to slightly outperform XGBoost in our semisupervised context, which supports the suggestion that semisupervised algorithms are more appropriate in low training data situations. We also show that our new hyper-parameter selection technique selects hyper-parameters for ssGMM that perform better on the testing data, but the performance is mixed for XGBoost.

Learning-Driven Detection and Mitigation of DDoS Attack in IoT via SDN-Cloud Architecture

The Internet-of-Things (IoT) network is growing big owing to its utility in smart applications. An IoT network is susceptible to security breaches, in majority due to the resource-constrained nature of IoT. Of the various breaches, the Distributed Denial-of-Service (DDoS) attack can snip off the network service to the users in various ways, such as consumption of server's resources, saturating link bandwidth, etc. These types of DDoS breaches can turn out to be a catastrophe in critical IoT use cases. This article delves into tackling the DDoS attack triggered by malicious wireless IoT on IoT servers. Our security scheme leverages the cloud and software-defined network (SDN) paradigm to mitigate the DDoS attack on IoT servers. We have proposed a novel mechanism named learning-driven detection mitigation (LEDEM) that detects DDoS using a semisupervised machine-learning algorithm and mitigates DDoS. We tested LEDEM in the testbed and emulated topology, and compared the results with state-of-the-art solutions. We achieved an improved accuracy rate of 96.28% in detecting DDoS attack.

A Framework for Efficient Healthcare Resources Utilization using Semi-supervised Machine Learning Algorithm

A Framework for Efficient Healthcare Resources Utilization using Semi-supervised Machine Learning Algorithm

An Autonomous Host-Based Intrusion Detection System for Android Mobile Devices

Intrusion Detection System (IDS) is crucial to protect smartphones from imminent security breaches and ensure user privacy. Android is the most popular mobile Operating System (OS), holding above 85% market share. The traffic generated by smartphones is expected to exceed the one generated by personal computers by 2021. Consequently, this prevalent mobile OS will stay one of the most attractive targets for potential attacks on fifth generation mobile networks (5G). Although Android malware detection has received considerable attention, offered solutions mostly rely on performing resource intensive analysis on a server, assuming a continuous connection between the device and the server, or on employing supervised Machine Learning (ML) algorithms for profiling the malware’s behaviour, which essentially require a training dataset consisting of thousands of examples from both benign and malicious profiles. However, in practice, collecting malicious examples is tedious since it entails infecting the device and collecting thousands of samples in order to characterise the malware’s behaviour and the labelling has to be done manually. In this paper, we propose a novel Host-based IDS (HIDS) incorporating statistical and semi-supervised ML algorithms. The advantage of our proposed IDS is two folds. First, it is wholly autonomous and runs on the mobile device, without needing any connection to a server. Second, it requires only benign examples for tuning, with potentially a few malicious ones. The evaluation results show that the proposed IDS achieves a very promising accuracy of above 0.9983, reaching up to 1.

Parametric Fault Diagnosis of Analog Circuits Based on a Semi-Supervised Algorithm

The parametric fault diagnosis of analog circuits is very crucial for condition-based maintenance (CBM) in prognosis and health management. In order to improve the diagnostic rate of parametric faults in engineering applications, a semi-supervised machine learning algorithm was used to classify the parametric fault. A lifting wavelet transform was used to extract fault features, a local preserving mapping algorithm was adopted to optimize the Fisher linear discriminant analysis, and a semi-supervised cooperative training algorithm was utilized for fault classification. In the proposed method, the fault values were randomly selected as training samples in a range of parametric fault intervals, for both optimizing the generalization of the model and improving the fault diagnosis rate. Furthermore, after semi-supervised dimensionality reduction and semi-supervised classification were applied, the diagnosis rate was slightly higher than the existing training model by fixing the value of the analyzed component.

IoT-based predictive maintenance for fleet management

In recent years, the Internet of Things (IoT) and big data have been hot topics. With all this data being produced, new applications such as predictive maintenance are possible. Consensus self-organized models approach (COSMO) is an example of a predictive maintenance system for a fleet of public transport buses, which attempts to diagnose faulty buses that deviate from the rest of the bus fleet. The present work proposes a novel IoT architecture for predictive maintenance and proposes a semi-supervised machine learning algorithm that attempts to improve the sensor selection performed in COSMO. With the help of the Société de Transport de l’Outaouais, a minimally viable prototype of the architecture has been deployed and J1939 sensor data have been acquired.

Online power quality disturbance detection by support vector machine in smart meter

Power quality assessment is an important performance measurement in smart grids. Utility companies are interested in power quality monitoring even in the low level distribution side such as smart meters. Addressing this issue, in this study, we propose segregation of the power disturbance from regular values using one-class support vector machine (OCSVM). To precisely detect the power disturbances of a voltage wave, some practical wavelet filters are applied. Considering the unlimited types of waveform abnormalities, OCSVM is picked as a semi-supervised machine learning algorithm which needs to be trained solely on a relatively large sample of normal data. This model is able to automatically detect the existence of any types of disturbances in real time, even unknown types which are not available in the training time. In the case of existence, the disturbances are further classified into different types such as sag, swell, transients and unbalanced. Being light weighted and fast, the proposed technique can be integrated into smart grid devices such as smart meter in order to perform a real-time disturbance monitoring. The continuous monitoring of power quality in smart meters will give helpful insight for quality power transmission and management.

Advances in Variational Inference.

Many modern unsupervised or semi-supervised machine learning algorithms rely on Bayesian probabilistic models. These models are usually intractable and thus require approximate inference. Variational inference (VI) lets us approximate a high-dimensional Bayesian posterior with a simpler variational distribution by solving an optimization problem. This approach has been successfully applied to various models and large-scale applications. In this review, we give an overview of recent trends in variational inference. We first introduce standard mean field variational inference, then review recent advances focusing on the following aspects: (a) scalable VI, which includes stochastic approximations, (b) generic VI, which extends the applicability of VI to a large class of otherwise intractable models, such as non-conjugate models, (c) accurate VI, which includes variational models beyond the mean field approximation or with atypical divergences, and (d) amortized VI, which implements the inference over local latent variables with inference networks. Finally, we provide a summary of promising future research directions.

Abstract WP213: Discovery of a Serum Protein Panel to Stratify Stroke and Stroke-Mimic Patients

Introduction: A rapid diagnostic tool that appropriately triages stroke and non-stroke patients would decrease patients’ time to treatment and therefore lower the risk of morbidity. Single blood protein biomarkers for stroke have not shown clinical utility. Our objective in this biomarker discovery study is to examine a wide array of proteins to identify a panel that stratifies stroke patients from stroke-mimicking patients. Hypothesis: Serum from ischemic stroke patients contains proteins identified by the SOMAscan biomarker discovery assay that can stratify stroke from non-stroke patients. Methods: Patients presenting to the emergency department with stroke-like symptoms had peripheral blood drawn upon arrival according to Stroke Alert protocols. Once informed consent was obtained, serum was analyzed with the SOMAscan assay to quantify the expression of 1305 proteins. These data were analyzed using the semi-supervised machine learning algorithm, random forest analysis (RFA), to determine a combination of proteins which best predicted the final patient discharge diagnosis. Results: Twenty-two stroke patients, 9 transient ischemic attack (TIA) patients, and 23 patients with other stroke-mimicking symptoms participated in the study. Normal age- and sex-matched serum controls were obtained commercially. RFA identified four panels encompassing a total of nine proteins to stratify patients between the stroke, TIA, other stroke mimic, and control groups. Applying these panels to the same training set of patient data, one panel of two proteins distinguished stroke from stroke-mimics with a sensitivity of 50% and specificity of 100%. A second panel of three proteins identified stroke from non-stroke patients with a sensitivity of 78% and specificity of 95%. A provisional patent application describing these panels is underway. Conclusion: Ischemic stroke patients and stroke-mimic patients exhibit a differential expression of serum proteins at the time of clinical presentation. The simultaneous analysis of two or three proteins in a biomarker panel exhibits high specificity and may be clinically useful as a stroke “rule in” tool for emergency personnel. This would facilitate triage and avoid inappropriate treatment of non-stroke patients.

Describing the performance of U.S. hospitals by applying big data analytics.

Public reporting of measures of hospital performance is an important component of quality improvement efforts in many countries. However, it can be challenging to provide an overall characterization of hospital performance because there are many measures of quality. In the United States, the Centers for Medicare and Medicaid Services reports over 100 measures that describe various domains of hospital quality, such as outcomes, the patient experience and whether established processes of care are followed. Although individual quality measures provide important insight, it is challenging to understand hospital performance as characterized by multiple quality measures. Accordingly, we developed a novel approach for characterizing hospital performance that highlights the similarities and differences between hospitals and identifies common patterns of hospital performance. Specifically, we built a semi-supervised machine learning algorithm and applied it to the publicly-available quality measures for 1,614 U.S. hospitals to graphically and quantitatively characterize hospital performance. In the resulting visualization, the varying density of hospitals demonstrates that there are key clusters of hospitals that share specific performance profiles, while there are other performance profiles that are rare. Several popular hospital rating systems aggregate some of the quality measures included in our study to produce a composite score; however, hospitals that were top-ranked by such systems were scattered across our visualization, indicating that these top-ranked hospitals actually excel in many different ways. Our application of a novel graph analytics method to data describing U.S. hospitals revealed nuanced differences in performance that are obscured in existing hospital rating systems.

Assisted Diagnosis Research Based on Improved Deep Autoencoder

Deep Autoencoder has the powerful ability to learn features from large number of unlabeled samples and a small number of labeled samples. In this work, we have improved the network structure of the general deep autoencoder and applied it to the disease auxiliary diagnosis. We have achieved a network by entering the specific indicators and predicting whether suffering from liver disease, the network using real physical examination data for training and verification. Compared with the traditional semi-supervised machine learning algorithm, deep autoencoder will get higher accuracy.

A Pattern Query Strategy Based on Semi-supervised Machine Learning in Distributed WSNs

Hardware of Distributed WSNs is limited, which turns into a big challenge. In recent years, there are some pattern-based query strategies have been proposed. But compared to the methods based on machine learning, these methods need high energy consumption in the calculation of distance. In this paper, we propose a pattern-based query strategy based on semi-supervised machine learning in wireless sensor network. The strategy uses historical time series data streams for o†ine machine learning. First of all, we flnd out the flxed length subsequences and screen out the suitable data sequences as training dataset and sample dataset. And then, because labeled data is less in practical application, we do label propagation with graph-based semi-supervised machine learning algorithm, Local and Global Consistency (LGC), to increase the number of labeled data sequences. Next, we use the supervised learning algorithm Support Vector Machine (SVM) to train a classifler. Finally, for a new data sequence, we can directly get its category through the classifler. Experimental results show that the performance of our strategy is almost better than other pattern-based query strategies on classiflcation accuracy and energy consumption.