Supervised Learning Approach Research Articles

Coherency-Based Supervised Learning Approach for Determining Optimal Post-Disturbance System Separation Strategies

Controlled islanding is the last remedial action to prevent cascading outages or blackouts in power systems. Conventional methods presented for controlled islanding strategy determination, particularly those calculating load shedding values using optimization methods, are not fast enough in online applications for modern power systems. In this paper, a novel learning-based approach is introduced for online coherency-based controlled islanding in transmission systems. The proposed approach presents a prediction and optimization model, which is faster than conventional optimization-based models in two ways. Firstly, the proposed approach uses a classification model to predict the splitting scheme in a short time following the occurrence of a disturbance, and secondly in the proposed approach, a simpler optimization problem with fewer variables is solved to find the load shedding amount required in each area. In the proposed load shedding approach, some candidate system partitioning schemes are calculated beforehand and therefore, the load shedding optimization problem is simplified significantly compared to similar optimization-based approaches. Note that appropriate features, which are used in this paper as the input of the classifier, are acquired by processing post-disturbance phase angle variations, which are measured across the network. The proposed approach is simulated on the 16-machine, 68-bus system, and its accuracy and efficacy have been demonstrated.

Dual-Stream Multiple Instance Learning for Depression Detection With Facial Expression Videos.

Depression is a common mental illness which has brought great harm to the individuals. With recent evidence that many objective physiological signals are associated with depression, automated detection of depression is urgent and important for the growing concern of mental illness. We investigate the problem of classifying depression by facial expressions, which may aid in online diagnosis and rehabilitation engineering of depression. In this work, We propose a weakly supervised learning approach employing multiple instance learning (MIL) on 150 videos data from 75 depressed and 75 healthy subjects. In addition, we present a novel MIL dual-stream aggregator that considers both the instance-level and the bag-level in order to emphasize the information with symptoms. Specifically, our method named ADDMIL uses max-pooling at the instance level to capture symptom information and further integrates the contribution of each instance at the bag level using attention weights. Our method achieves 74.7% accuracy and 74.5% recall on the collected dataset, which not only improves 10.1% accuracy and 9.8% recall over the baseline but also exceeds the best accuracy result of MIL-based method by 2.1%. Our work achieves results that are comparable to the state-of-the-art methods and demonstrates that multiple instance learning has great potential for depression classification. We present for the first time a weakly supervised learning approach in the detection of depression through raw facial expressions, which may provide a new framework for other psychiatric disorders detection methods.

Human Motion Segmentation via Velocity-Sensitive Dual-Side Auto-Encoder.

Human motion segmentation (HMS) aims to segment a long human action video into a bunch of short and meaningful action clips. Existing supervised learning approaches need a large amount of training data which may be costly in real-world scenario, while most unsupervised clustering methods cannot fully explore the temporal correlations among human motions and hard to achieve promising performances. In our paper, we design a novel unsupervised framework, called Velocity-Sensitive Dual-Side Auto-Encoder (VSDA), for HMS task. Specifically, a multi-neighbor auto-encoder (MNA) is proposed to extract informative temporal features, which fully explores the local temporal patterns of human motions. In addition, a long-short distance encoding (LSE) strategy is designed. It constrains the encoded representations of close (short-distance) frames becoming similar while the representations of far-away (long-distance) frames becoming distinctive. Similarly, this strategy is also deployed on the decoded outputs as the long-short distance decoding (LSD) module. The LSE/LSD guides the learning process explicitly and implicitly to achieve the dual-side structure. Moreover, we consider the energy variations during the human motion to propose the velocity-sensitive (VS) guidance mechanism for further model improvement. VSDA leverages the temporal characteristics of human motion and derives promising HMS performance. Comprehensive experiments on six real-world human motion datasets illustrate the effectiveness of our proposed model.

Self-FuseNet: Data Free Unsupervised Remote Sensing Image Super-Resolution

Real-world degradations deviate from ideal degradations, as most deep learning-based scenarios involve the ideal synthesis of low-resolution (LR) counterpart images by popularly used bicubic interpolation. Moreover, supervised learning approaches rely on many high-resolution (HR) and LR image pairings to reconstruct missing information based on their association, developed by complex long hours of deep neural network training. Additionally, the trained model's generalizability on various image datasets with various distributions is not guaranteed. To overcome this challenge, we proposed our novel Self-FuseNet, particularly for extremely poor-resolution satellite images. Also, the network exhibits strong generalization performance on additional datasets (both “ideal” and “nonideal” scenarios). The network is especially for those image datasets suffering from the following two significant limitations: 1) nonavailability of ground truth HR images; 2) limitation of a large count of the unpaired dataset for deep neural network training. The benefit of the proposed model is threefold: 1) it does not require any significant extensive training data, either paired or unpaired but only a single LR image without prior knowledge of its distribution; 2) it is a simple and effective model for super-resolving very poor-resolution images, saving computational resources and time; 3) using UNet, the processing of data are accelerated by the network's wide skip connections, allowing image reconstruction with fewer parameters. Rather than using an inverse approach, as common in most deep learning scenarios, we introduced a forward approach to super-resolve exceptionally LR remote sensing images. This demonstrates its supremacy over recently proposed state-of-the-art methods for unsupervised single real-world image blind super-resolution.

Development of two data bases with comments in Bulgarian language and application of supervised learning approaches on them for comparative sentiment analysis. А brief overview

The purpose of the current paper is to make an overview of the work done so far by the authors and make a summary of the results and reflections on the performed sentiment analysis on user comments in two different fields in Bulgarian language. As a starting point for the authors’ work is the development of two databases with users’ reviews and their preprocessing to become usable source of information for different types of analysis projects. As a result of the preprocessing is a revised Bulgarian language-driven algorithm for data preprocessing for Bulgarian language. The second part of the project is implemented into two steps: sentiment analysis using the supervised learning approaches developed for the two databases and a comparative sentiment analysis of the two databases, following their additional examination.

Deep Learning-Based Screening of Urothelial Carcinoma in Whole Slide Images of Liquid-Based Cytology Urine Specimens.

Urinary cytology is a useful, essential diagnostic method in routine urological clinical practice. Liquid-based cytology (LBC) for urothelial carcinoma screening is commonly used in the routine clinical cytodiagnosis because of its high cellular yields. Since conventional screening processes by cytoscreeners and cytopathologists using microscopes is limited in terms of human resources, it is important to integrate new deep learning methods that can automatically and rapidly diagnose a large amount of specimens without delay. The goal of this study was to investigate the use of deep learning models for the classification of urine LBC whole-slide images (WSIs) into neoplastic and non-neoplastic (negative). We trained deep learning models using 786 WSIs by transfer learning, fully supervised, and weakly supervised learning approaches. We evaluated the trained models on two test sets, one of which was representative of the clinical distribution of neoplastic cases, with a combined total of 750 WSIs, achieving an area under the curve for diagnosis in the range of 0.984-0.990 by the best model, demonstrating the promising potential use of our model for aiding urine cytodiagnostic processes.

Digital fingerprinting of microstructures

Finding efficient means of quantitatively describing material microstructure is a critical step towards harnessing data-centric machine learning approaches to understanding and predicting processing–microstructure–property relationships. Current quantitative descriptors of microstructure tend to consider only specific, narrow features such as grain size or phase fractions, but these metrics discard vast amounts of information. Since the gain in traction of machine learning and computer vision, more abstract methods for describing image data in a concise and quantitative manner have become available, but have yet to be fully exploited within materials science. The main aim of this paper is to investigate some of these methods as tools for constructing compressed numerical descriptions of microstructural image data, which are here referred to as “microstructural fingerprints”. A statistical framework is developed to combine some of these methods which includes some classical computer vision methods as special cases.The effectiveness of fingerprints, in this case, is assessed via a series of classification tasks, which take the fingerprints as input along with some label to describe the microstructure, and aim to predict the class label. The classification tasks are a simple way to assess the information content of differently constructed fingerprints, using readily available data. However, the potential applications for such fingerprints, in theory, extend far beyond this, provided a suitably labelled training dataset. For example, fingerprints constructed from image data labelled with some processing parameters or mechanical properties could be fed into regression-based tasks to predict properties from microstructure or predict the processing parameters required for a target microstructure. The ultimate purpose is to rapidly fingerprint sample images in the context of various high-throughput design/make/test scenarios. Such fingerprints would enable, for example, quantification of the disparity between microstructures for quality control, classifying microstructures, predicting materials properties from image data and identifying potential processing routes to engineer new materials with specific properties.Here, the approach is applied to two distinct datasets to illustrate various aspects of the fingerprints and some recommendations are made based on the findings. In particular, methods that leverage transfer learning with convolutional neural networks (CNNs), pretrained on the ImageNet dataset, are generally shown to outperform other methods. Additionally, dimensionality reduction of these CNN-based fingerprints is shown to have negligible impact on classification accuracy for the supervised learning approaches considered. In situations where there is a large dataset with only a handful of images labelled, graph-based label propagation to unlabelled data is shown to be favourable over discarding unlabelled data and performing supervised learning. In particular, label propagation by Poisson learning is shown to be highly effective at low label rates. The supplementary code is available on GitHub (White and Tarakanov, 2021).

Efficient Anticipatory Longitudinal Control of Electric Vehicles through Machine Learning-Based Prediction of Vehicle Speeds

Driving style and external factors such as traffic density have a significant influence on the vehicle energy demand especially in city driving. A longitudinal control approach for intelligent, connected vehicles in urban areas is proposed in this article to improve the efficiency of automated driving. The control approach incorporates information from Vehicle-2-Everything communication to anticipate the behavior of leading vehicles and to adapt the longitudinal control of the vehicle accordingly. A supervised learning approach is derived to train a neural prediction model based on a recurrent neural network for the speed trajectories of the ego and leading vehicles. For the development, analysis and evaluation of the proposed control approach, a co-simulation environment is presented that combines a generic vehicle model with a microscopic traffic simulation. This allows for the simulation of vehicles with different powertrains in complex urban traffic environment. The investigation shows that using V2X information improves the prediction of vehicle speeds significantly. The control approach can make use of this prediction to achieve a more anticipatory driving in urban areas which can reduce the energy consumption compared to a conventional Adaptive Cruise Control approach.

A hybrid unsupervised and supervised learning approach for postictal generalized EEG suppression detection.

Sudden unexpected death of epilepsy (SUDEP) is a catastrophic and fatal complication of epilepsy and is the primary cause of mortality in those who have uncontrolled seizures. While several multifactorial processes have been implicated including cardiac, respiratory, autonomic dysfunction leading to arrhythmia, hypoxia, and cessation of cerebral and brainstem function, the mechanisms underlying SUDEP are not completely understood. Postictal generalized electroencephalogram (EEG) suppression (PGES) is a potential risk marker for SUDEP, as studies have shown that prolonged PGES was significantly associated with a higher risk of SUDEP. Automated PGES detection techniques have been developed to efficiently obtain PGES durations for SUDEP risk assessment. However, real-world data recorded in epilepsy monitoring units (EMUs) may contain high-amplitude signals due to physiological artifacts, such as breathing, muscle, and movement artifacts, making it difficult to determine the end of PGES. In this paper, we present a hybrid approach that combines the benefits of unsupervised and supervised learning for PGES detection using multi-channel EEG recordings. A K-means clustering model is leveraged to group EEG recordings with similar artifact features. We introduce a new learning strategy for training a set of random forest (RF) models based on clustering results to improve PGES detection performance. Our approach achieved a 5-second tolerance-based detection accuracy of 64.92%, a 10-second tolerance-based detection accuracy of 79.85%, and an average predicted time distance of 8.26 seconds with 286 EEG recordings using leave-one-out (LOO) cross-validation. The results demonstrated that our hybrid approach provided better performance compared to other existing approaches.

Classification of climate-related insurance claims using gradient boosting

The aim of this paper is to implement, one of the most representative supervised learning approaches, the decision tree based ensemble method called gradient boosting for classifying the number of claims caused by storms in Greece using data from a major insurance company operating in Greece. Finally, a machine learning algorithm is used to for categorising the number of claims which have been occurred by a “storm event” into 3 categories: “no claims”, “1 claim”, “2 or more claims”.

Emati: a recommender system for biomedical literature based on supervised learning

The scientific literature continues to grow at an ever-increasing rate. Considering that thousands of new articles are published every week, it is obvious how challenging it is to keep up with newly published literature on a regular basis. Using a recommender system that improves the user experience in the online environment can be a solution to this problem. In the present study, we aimed to develop a web-based article recommender service, called Emati. Since the data are text-based by nature and we wanted our system to be independent of the number of users, a content-based approach has been adopted in this study. A supervised machine learning model has been proposed to generate article recommendations. Two different supervised learning approaches, namely the naïve Bayes model with Term Frequency-Inverse Document Frequency (TF-IDF) vectorizer and the state-of-the-art language model bidirectional encoder representations from transformers (BERT), have been implemented. In the first one, a list of documents is converted into TF-IDF-weighted features and fed into a classifier to distinguish relevant articles from irrelevant ones. Multinomial naïve Bayes algorithm is used as a classifier since, along with the class label, it also gives the probability that the input belongs to this class. The second approach is based on fine-tuning the pretrained state-of-the-art language model BERT for the text classification task. Emati provides a weekly updated list of article recommendations and presents it to the user, sorted by probability scores. New article recommendations are also sent to users' email addresses on a weekly basis. Additionally, Emati has a personalized search feature to search online services' (such as PubMed and arXiv) content and have the results sorted by the user's classifier. Database URL: https://emati.biotec.tu-dresden.de.

A prediction model of stock market trading actions using generative adversarial network and piecewise linear representation approaches.

Supervised learning applied stock prediction tasks and obtained satisfactory performance. The trading strategies are very complex and diverse but supervised learning is only learned and fitted by gold standard trading strategies. Supervised learning approaches often have over-fitting problems. To learn distribution of gold standard answers, the generative adversarial network (GAN) models can generate extra similar samples to improve performance. Therefore, the paper proposes a generative GAN-based frameworks with the piecewise linear representation (PLR) approach to learn three trading actions, namely buying, selling, and holding. The proposed framework consists of two parts: first, PLR approach uses to detect historical prices to form trading sequences with three actions, PLR can provide a guided trading strategy to discriminator of GAN. Second, the generator of GAN is used to generate/predict daily trading actions, and the discriminator is used to detect the real/fake trading actions from the PLR/generator of GAN. Experimental results indicate that the proposed GAN-based frameworks outperform the long short-term memory network.

Sensor-based jump detection and classification with machine learning in trampoline gymnastics

The task of the judge of difficulty in trampoline gymnastics is to check the elements and difficulty values entered on the competition cards and the difficulty of each element according to a numeric system. To do this, the judge must count all somersaults and twists for each jump during a routine and thus record the difficulty of the routine. This assessment can be automated with the help of inertial measurement units (IMUs) and facilitate the judges’ task during the competition. Currently, there is no known reliable method for the automated detection and recognition of the various elements to determine the difficulty of an exercise in trampoline gymnastics. Accordingly, a total of 2076 jumps and 50 different jump types were recorded over the course of several training sessions. In the first instance, 10 different jump types were used to train different machine learning (ML) models. Eight ML models were used for the automatic jump classification. Supervised learning approaches include a naive classifier, deep feedforward neural network, convolutional neural network, k‑nearest neighbors, Gaussian naive Bayes, support-vector classification, gradient boosting classifier, and stochastic gradient descent. When all classifiers were compared for accuracy, i.e., how many jumps were correctly detected by the ML model, the deep feedforward neural network and the convolutional neural network provided the best matches with 96.4 and 96.1%, respectively. The findings of this study will help to develop the automated classification of sensor-based data to support the judge and, simultaneously, for automated training logging.

Bridging optical and SAR satellite image time series via contrastive feature extraction for crop classification

Precise crop mapping is crucial for guiding agricultural production, forecasting crop yield, and ensuring food security. Integrating optical and synthetic aperture radar (SAR) satellite image time series (SITS) for crop classification is an essential and challenging task in remote sensing. Previously published studies generally employ a dual-branch network to learn optical and SAR features independently, while ignoring the complementarity and correlation between the two modalities. In this article, we propose a novel method to learn optical and SAR features for crop classification through cross-modal contrastive learning. Specifically, we develop an updated dual-branch network with partial weight-sharing of the two branches to reduce model complexity. Furthermore, we enforce the network to map features of different modalities from the same class to nearby locations in a latent space, while samples from distinct classes are far apart, thereby learning discriminative and modality-invariant features. We conducted a comprehensive evaluation of the proposed method on a large-scale crop classification dataset. Experimental results show that our method consistently outperforms traditional supervised learning approaches, no matter the training samples are adequate or not. Our findings demonstrate that unifying the representations of optical and SAR image time series enables the network to learn more competitive features and suppress inference noise.

Adaptive Beam Sweeping With Supervised Learning

Utilizing millimeter-wave (mmWave) frequencies for wireless communication in mobile systems is challenging since continuous tracking of the beam direction is needed. For the purpose, beam sweeping is performed periodically. Such approach can be sufficient in the initial deployment of the network when the number of users is small. However, a more efficient solution is needed when lots of users are connected to the network due to higher overhead consumption. We explore a supervised learning approach to adaptively perform beam sweeping, which has low implementation complexity and can improve cell capacity by reducing beam sweeping overhead. By formulating the beam tracking problem as a binary classification problem, we applied supervised learning methods to solve the formulated problem. The methods were tested on two scenarios: ray-tracing outdoor scenario and over-the-air (OTA) testing dataset from Ericsson. Both experimental results show that the proposed methods significantly increase cell throughput comparing with existing exhaustive sweeping and periodical sweeping strategies.

Supervised Learning Approach for Multi-Component Deghosting with a Focus on Shallow Water Data

In recent years, the use of machine learning (ML) has shown more and more successful applications, initially mainly in seismic interpretation (classification) but recently also in seismic processing challenges (regression) like denoising, interpolation and demultiple. In this paper, a supervised machine learning algorithm is proposed for receiver deghosting on multi-component data. A neural network is trained to map input pairs of hydrophone and geophone data into the deghosted up-going wavefield. This data-driven approach could offer a user-friendly alternative to existing deghosting methods, like the PZ-summation approach that is commonly used for multi-component streamer and ocean-bottom acquisitions. Although the PZ-summation approach offers an analytical solution to the deghosting problem, it requires some pre-processing steps that make it more time-consuming and user-intensive to run and QC. The advantage of the proposed ML approach is that it can be directly applied to multi-component data without any parameterization. However, the success of the supervised method strongly depends on whether a neural network can be trained easily with minimal user-interaction and can be applied to various types of (field) data without adapting the network for each individual case. To investigate if this is possible for the supervised multi-component receiver deghosting method, the selected neural network is trained on easy to model 1.5D data examples and applied to test data with increasing complexity. Results are very encouraging and evaluated to be at least on par in comparison to the existing PZ-summation deghosting method.

Predicting Integer Overflow Errors via Supervised Learning

An integer overflow error occurs when an integer operation in computer software evaluates a value out of the integer range. It can lead to a fatal system failure. The existing approaches to detecting integer overflow errors rely on data/control-flow analysis of the code or execution of the code with test cases. This paper presents a supervised learning approach to predicting whether each method in a given Java program has an integer overflow error by treating the source code as text. Built upon real-world programs, our Java dataset covers all integer data types and operations in Java, the methods for preventing integer overflow errors, and adversarial samples. We have evaluated six classification models, BERT, DistilBERT, codeBERT, Code2Vec, fastText, and NBSVM. They represent different text embedding techniques for dealing with source code. The experiment results show that BERT and its variants have outperformed other models. We have applied the resultant BERT model to eleven real-world projects, including JDK13.0 and ten top-ranked GitHub projects, and revealed 181 integer overflow errors. In addition, we have evaluated the classification models with a C/C++ dataset. The result is similar to that of the Java dataset.

Solid Waste Management Scenario in India and Illegal Dump Detection Using Deep Learning: An AI Approach towards the Sustainable Waste Management

The study is presented in four sections. The first section defines the municipal solid waste and solid waste management system. The second section illustrates the descriptive statistical analysis of waste generation patterns in India. The average waste generation in India was 160,038.9 tons per day in 2021; 95% of this total waste was collected and transported to the disposal sites. Based on scientific studies and observations, the per capita waste generation rate in 2018 was 0.490–0.626 g per day. In the last one and a half decades (1999–2000 to 2015–2016), Delhi and Bangalore have shown the highest percentage growth of 2075% and 1750%, respectively, in total waste generation among the highest population cities. The analysis of waste generation patterns concludes urbanization is a major factor that highly influences the waste generation rate. The third section describes the major issues in current solid waste management services. Some of these issues are the unavailability of web portals for citizens, no real-time monitoring of bins, collection vehicles and illegal dumping. These issues are identified based on the survey performed in a city and analysis of related research studies and scientific reports. We determined that illegal dumping is one of these major concerns and needs a technological solution. In the fourth section, we propose a multipath convolutional neural network (mp-CNN) to detect and localize the waste dumps on streets and roadsides. We constructed our dataset to train and test the proposed model, as no benchmark dataset is publicly available to obtain this objective. We applied the weakly supervised learning approach to training the model. In this approach, mp-CNN was trained according to the image class; in our case, it is two (waste and non-waste). In the testing phase, the model showed the performance evaluation matrices 97.82% of precision, 98.86% of recall, 98.34% of F1 score, 98.33% of accuracy, and 98.63% of AUROC for this binary classification. Due to the scarcity of benchmark datasets, waste localization results cannot be presented quantitatively. So, we performed a survey to compare the overlapping of the mask generated by the model with the region waste in the actual image. The average score for the generated mask obtained a score of 3.884 on a scale of 5. Based on the analysis of model performance evaluation parameters, precision-recall curve, receiver characteristic operator curve, and comparison of mask generated by the model over waste with corresponding actual images show that mp-CNN performs remarkably good in detection, classification, and localization of waste regions. Finally, two conceptual architectures in the context of developing countries are suggested to demonstrate the future practical applications of the mp-CNN model.

Mitigation Services on SDN for Distributed Denial of Service and Denial of Service Attacks Using Machine Learning Techniques

Software-Defined Networking (SDN) offers an innovative model over the separation of the data plane, control plane and management plane. This separation would result in more effective network management, including cost reductions for hardware and manpower, and the ability to deliver on-demand solutions using programmable SDN approaches. As network policy and on-demand services become more impartial, SDN is becoming more popular. However, there are safety risks associated with the SDN network due to malicious floods such as Distributed Denial of Service (DDoS) attacks and Denial of Service (DoS) attacks directed at the SDN Controller, OpenFlow Virtual Switch (OVS), and end nodes, which must be addressed. Because of these assaults, network throughput is reduced, resulting in a lapse in the availability of network services and a reduction in business operations. The main emphasis of this study is on the detection and mitigation of DDoS and DoS assaults in the SDN network, which is accomplished by the use of both unsupervised and supervised learning approaches. The use of the Dynamic Access Control List (DACL) allows for the performance of mitigation operations in the SDN network, which has been implemented using the mininet. The outcome of the experiment demonstrates that malicious (DDoS and DoS) flood is reduced as a consequence of the mitigation technique.

Supervised Machine Learning for Understanding and Improving the Computational Performance of Chemical Production Scheduling MIP Models

We adopt a supervised learning approach to predict runtimes of batch production scheduling mixed-integer programming (MIP) models with the aim of understanding what instance features make a model computationally expensive. We introduce novel features to characterize instance difficulty according to problem type. The developed machine learning models trained on runtime data obtained from a wide variety of instances show good predictive performances. Then, we discuss informative features and their effects on computational performance. Finally, based on the derived insights, we propose solution methods for improving the computational performance of batch scheduling MIP models.