Supervised Machine Learning Methods Research Articles

Comparison of unsupervised shallow and deep models for structural health monitoring

Recent failures of bridges worldwide highlight the need for robust structural health surveillance systems to provide alerts for early damage and prevent severe loss. Structural health monitoring (SHM) continues to evolve and gain wide adoption among asset managers to mitigate catastrophic failure risks of structural systems. With the prolific integration of SHM with artificial intelligence and internet of things systems becoming prevalent, several data-driven approaches have gained popularity in SHM. However, it is challenging to obtain ground-truth-labelled damage samples for supervised machine learning methods. A class of shallow machine learning methods known as unsupervised methods (e.g. the one-class support vector machine (OCSVM)), which only uses unlabelled data from a healthy state, has been shown to achieve promising results. However, shallow machine learning methods require laborious expert-crafted feature inputs for optimal performance. Inspired by the automatic feature-extraction ability of deep neural networks, unsupervised methods (deep learning models trained without supervisory ground truth labels) such as autoencoder (AE)-based methods were explored to detect structural damage. Results obtained on both real-world data from Sydney Harbour Bridge, one of Australia's iconic structures, and data collected from laboratory specimens demonstrate the effectiveness of the AE-based method over conventional shallow methods such as the OCSVM.

An Overview of Supervised Machine Learning Methods and Data Analysis for COVID-19 Detection.

Methods Our analysis and machine learning algorithm is based on most cited two clinical datasets from the literature: one from San Raffaele Hospital Milan Italia and the other from Hospital Israelita Albert Einstein São Paulo Brasilia. The datasets were processed to select the best features that most influence the target, and it turned out that almost all of them are blood parameters. EDA (Exploratory Data Analysis) methods were applied to the datasets, and a comparative study of supervised machine learning models was done, after which the support vector machine (SVM) was selected as the one with the best performance. Results SVM being the best performant is used as our proposed supervised machine learning algorithm. An accuracy of 99.29%, sensitivity of 92.79%, and specificity of 100% were obtained with the dataset from Kaggle (https://www.kaggle.com/einsteindata4u/covid19) after applying optimization to SVM. The same procedure and work were performed with the dataset taken from San Raffaele Hospital (https://zenodo.org/record/3886927#.YIluB5AzbMV). Once more, the SVM presented the best performance among other machine learning algorithms, and 92.86%, 93.55%, and 90.91% for accuracy, sensitivity, and specificity, respectively, were obtained. Conclusion The obtained results, when compared with others from the literature based on these same datasets, are superior, leading us to conclude that our proposed solution is reliable for the COVID-19 diagnosis.

LSI with Support Vector Machine for Text Categorization – a practical example with Python


 Artificial intelligence is becoming a powerful tool of modernity science, there is even a science consensus about how our society is turning to a data-driven society. Machine learning is a branch of Artificial intelligence that has the ability to learn from data and understand its behavers. Python programming language aiming the challenges of this new era is becoming one of the most popular languages for general programming and scientific computing. Keeping all this new era circumstances in mind, this article has as a goal to show one example of how to use one supervised machine learning method, Support Vector Machine, and to predict movie’s genre according to its description using the programming language of the moment, python. Firstly, Omdb official API was used to gather data about movies, then tuned Support Vector Machine model for Latent semantic indexing capable of predicting movies genres according to its plot was coded. The performance of the model occurred to be satisfactory considering the small dataset used and the occurrence of movies with hybrid genres. Testing the model with larger dataset and using multi-label classification models were purposed to improve the model.

Hydraulic Flow Unit Classification and Prediction Using Machine Learning Techniques: A Case Study from the Nam Con Son Basin, Offshore Vietnam

The test study area is the Miocene reservoir of Nam Con Son Basin, offshore Vietnam. In the study we used unsupervised learning to automatically cluster hydraulic flow units (HU) based on flow zone indicators (FZI) in a core plug dataset. Then we applied supervised learning to predict HU by combining core and well log data. We tested several machine learning algorithms. In the first phase, we derived hydraulic flow unit clustering of porosity and permeability of core data using unsupervised machine learning methods such as Ward’s, K mean, Self-Organize Map (SOM) and Fuzzy C mean (FCM). Then we applied supervised machine learning methods including Artificial Neural Networks (ANN), Support Vector Machines (SVM), Boosted Tree (BT) and Random Forest (RF). We combined both core and log data to predict HU logs for the full well section of the wells without core data. We used four wells with six logs (GR, DT, NPHI, LLD, LSS and RHOB) and 578 cores from the Miocene reservoir to train, validate and test the data. Our goal was to show that the correct combination of cores and well logs data would provide reservoir engineers with a tool for HU classification and estimation of permeability in a continuous geological profile. Our research showed that machine learning effectively boosts the prediction of permeability, reduces uncertainty in reservoir modeling, and improves project economics.

Learning an augmentation strategy for sparse datasets

The limited quantity of training data can hamper supervised machine learning methods, that generally need large amounts of data to avoid overfitting. Data augmentation has a long history of use with machine learning algorithms and is a straightforward method to overcome overfitting and improve model generalisation. However, data augmentation schemes are typically designed by hand and demand substantial domain knowledge to create suitable data transformations. This paper introduces a GAN based method that automatically learns an augmentation strategy appropriate for sparse datasets and can improve pixel-level semantic segmentation accuracy by filling the gaps in the training set. Our method can also be combined with other augmentation techniques to further improve performance. We evaluate the proposed method's feasibility on four datasets and three semantic segmentation models, leading to improvement in the mean intersection-over-union (mIoU) score of between 0.5 and 14 percentage points, under different circumstances.

Allosteric Effect of Nanobody Binding on Ligand-Specific Active States of the β2 Adrenergic Receptor.

Nanobody binding stabilizes G-protein-coupled receptors (GPCR) in a fully active state and modulates their affinity for bound ligands. However, the atomic-level basis for this allosteric regulation remains elusive. Here, we investigate the conformational changes induced by the binding of a nanobody (Nb80) on the active-like β2 adrenergic receptor (β2AR) via enhanced sampling molecular dynamics simulations. Dimensionality reduction analysis shows that Nb80 stabilizes structural features of the β2AR with an ∼14 Å outward movement of transmembrane helix 6 and a close proximity of transmembrane (TM) helices 5 and 7, and favors the fully active-like conformation of the receptor, independent of ligand binding, in contrast to the conditions under which no intracellular binding partner is bound, in which case the receptor is only stabilized in an intermediate-active state. This activation is supported by the residues located at hotspots located on TMs 5, 6, and 7, as shown by supervised machine learning methods. Besides, ligand-specific subtle differences in the conformations assumed by intracellular loop 2 and extracellular loop 2 are captured from the trajectories of various ligand-bound receptors in the presence of Nb80. Dynamic network analysis further reveals that Nb80 binding triggers tighter and stronger local communication networks between the Nb80 and the ligand-binding sites, primarily involving residues around ICL2 and the intracellular end of TM3, TM5, TM6, as well as ECL2, ECL3, and the extracellular ends of TM6 and TM7. In particular, we identify unique allosteric signal transmission mechanisms between the Nb80-binding site and the extracellular domains in conformations modulated by a full agonist, BI167107, and a G-protein-biased partial agonist, salmeterol, involving mainly TM1 and TM2, and TM5, respectively. Altogether, our results provide insights into the effect of intracellular binding partners on the GPCR activation mechanism, which should be taken into account in structure-based drug discovery.

Fire prediction with logistic regression on territory of Bosnia and Herzegovina

The Canadian Fire Weather Index system [1] has been used worldwide by many countries as classic approach in fire prediction. It represents system that account for the effects of fuel moisture and weather conditions on fire behaviour. It numerical outputs are based on calculation of four meteorological elements: air temperature, relative humidity, wind speed and precipitation in last 24h. In this paper meteorological data in combination with Canadian Fire Weather Index system (CFWI) components is used as input to predict fire occurrence using logistic regression model. As logistic regression is a supervised machine learning method it’s based on user input in the form of dataset. Dataset is collected using NASA GES DISC Giovanni web-based application in the form of daily area-averaged time series in period of 31.7.2010 to 31.7.2020, it’s analysed and pre-processed before it is used as input for logit model. CFWI components values are not imported but calculated in run-time based on pre-processed meteorological data. As a result of this research windows application was developed to assist fire managers and all those involved in studying the fire behaviour.

Performance Analysis of Different Machine Learning Algorithms for Identifying and Classifying the Failures of Traction Motors

This paper addresses electric motor fault diagnosis using supervised machine learning classification. A total of 15 distinct fault types are classified and multilabel strategies are used to classify concurrent faults. we explored, developed, and compared the performance of different types of binary (fault/non-fault), multi-class (fault type) and multi-label (single fault versus combination fault) classifiers. To evaluate the effectiveness of fault identification and classification, we used different supervised machine learning methods, including Random forest classification, support vector machine and neural network classification. Through experiment, we compared these methods over 4 classification regimes and finally summarize the most suitable machine learning algorithms for different aspects of health diagnosis in traction motors area.

Single Image De-Raining using Supervised CNN Model

Abstract: An image captured in rain reduces the visibility quality of image which affects the analytical task like detecting objects and classifying pictures. Hence, image de-raining became important in last few years. Since pictures taken in rain include rain streaks of all sizes, single image de-raining is becoming much difficult issue to solve, which may flow in different direction and the density of each rain streak is different. Rain streaks have a varied effect on various areas of picture, and hence it becomes important for removing rain streak from rainy pictures as rainy images tend to lose its high frequency information; previously many methods were proposed for this purpose but they failed to provide accurate results. Hence we have studied and implemented a supervised machine learning method using convolutional neural network (CNN) algorithm to get more accurate result of rain streak removal from an image captured during rain and in less elapsed time by preserving high rated information of image during removal of rain streak. Keywords: CNN, elapsed time, single image de-raining, supervised machine learning, rain streaks.

Integrating Optimized Multiscale Entropy Model with Machine Learning for the Localization of Epileptogenic Hemisphere in Temporal Lobe Epilepsy Using Resting-State fMRI.

The bottleneck associated with the validation of the parameters of the entropy model has limited the application of this model to modern functional imaging technologies such as the resting-state functional magnetic resonance imaging (rfMRI). In this study, an optimization algorithm that could choose the parameters of the multiscale entropy (MSE) model was developed, while the optimized effectiveness for localizing the epileptogenic hemisphere was validated through the classification rate with a supervised machine learning method. The rfMRI data of 20 mesial temporal lobe epilepsy patients with positive indicators (the indicators of epileptogenic hemisphere in clinic) in the hippocampal formation on either left or right hemisphere (equally divided into two groups) on the structural MRI were collected and preprocessed. Then, three parameters in the MSE model were statistically optimized by both receiver operating characteristic (ROC) curve and the area under the ROC curve value in the sensitivity analysis, and the intergroup significance of optimized entropy values was utilized to confirm the biomarked brain areas sensitive to the epileptogenic hemisphere. Finally, the optimized entropy values of these biomarked brain areas were regarded as the feature vectors input for a support vector machine to classify the epileptogenic hemisphere, and the classification effectiveness was cross-validated. Nine biomarked brain areas were confirmed by the optimized entropy values, including medial superior frontal gyrus and superior parietal gyrus (p < .01). The mean classification accuracy was greater than 90%. It can be concluded that combination of the optimized MSE model with the machine learning model can accurately confirm the epileptogenic hemisphere by rfMRI. With the powerful information interaction capabilities of 5G communication, the epilepsy side-fixing algorithm that requires computing power can be integrated into a cloud platform. The demand side only needs to upload patient data to the service platform to realize the preoperative assessment of epilepsy.

KNN-based gas-bearing prediction using local waveform similarity gas-indication attribute — An application to a tight sandstone reservoir

Gas-bearing prediction of tight sandstone reservoirs is significant but challenging due to the relationship between the gas-bearing property and its seismic response being nonlinear and complex. Although machine learning (ML) methods provide potential for solving the issue, the major challenge of ML applications to gas-bearing prediction is that of generating accurate and interpretable intelligent models with limited training sets. The k nearest neighbor ( kNN) method is a supervised ML method classifying an unlabeled sample according to its k neighboring labeled samples. We have introduced a kNN-based gas-bearing prediction method. The method can automatically extract a gas-sensitive attribute called the gas-indication local waveform similarity attribute (GLWSA) combining prestack seismic gathers with interpreted gas-bearing curves. GLWSA uses the local waveform similarity among the predicting samples and the gas-bearing training samples to indicate the existence of an exploitable gas reservoir. GLWSA has simple principles and an explicit geophysical meaning. We use a numerical model and field data to test the effectiveness of our method. The result demonstrates that GLWSA is good at characterizing the reservoir morphology and location qualitatively. When the method applies to the field data, we evaluate the performance with a blind well. The prediction result is consistent with the geologic law of the work area and indicates more details compared to the root-mean-square attribute.

Machine learning modeling of time-dependent corrosion rates of carbon steel in presence of corrosion inhibitors

We have employed supervised machine learning methods to model measurements of corrosion rates of carbon steel as a function of time when corrosion inhibitors are added in different dosage and dose-schedules. The experiments show that the time-profile of corrosion rates depend on the dose schedule, while the final rates depend mainly on the environment severity. We find that Random Forest was the best algorithm that predicted the entire time-profile of corrosion rates with the mean squared error ranging from 0.005 to 0.093. Sensitivity of corrosion rates to changes in the environmental variables are well-predicted by the trained Random Forest model.

Handling the impact of feature uncertainties on SVM: A robust approach based on Sobol sensitivity analysis

This paper addresses the problem of classification when target data are subject to feature uncertainties. A robust approach based on Sobol sensitivity analysis is proposed to improve the robustness of support vector machine (SVM) models. SVM is a supervised machine learning method for pattern recognition whose performance depends on the definition of its hyperparameters and the quality of data. The proposed approach analyzes the impact of the uncertainties on the predictive performance of SVM based on Sobol’ sensitivity analysis. Afterwards, a new parameter is introduced to improve the robustness of SVM to the impact of uncertainties. The efficiency of this approach is evaluated by applying it to six real-world datasets. The results are then discussed and analyzed.

Cardiovascular Disease Recognition Based on Heartbeat Segmentation and Selection Process

Assessment of heart sounds which are generated by the beating heart and the resultant blood flow through it provides a valuable tool for cardiovascular disease (CVD) diagnostics. The cardiac auscultation using the classical stethoscope phonological cardiogram is known as the most famous exam method to detect heart anomalies. This exam requires a qualified cardiologist, who relies on the cardiac cycle vibration sound (heart muscle contractions and valves closure) to detect abnormalities in the heart during the pumping action. Phonocardiogram (PCG) signal represents the recording of sounds and murmurs resulting from the heart auscultation, typically with a stethoscope, as a part of medical diagnosis. For the sake of helping physicians in a clinical environment, a range of artificial intelligence methods was proposed to automatically analyze PCG signal to help in the preliminary diagnosis of different heart diseases. The aim of this research paper is providing an accurate CVD recognition model based on unsupervised and supervised machine learning methods relayed on convolutional neural network (CNN). The proposed approach is evaluated on heart sound signals from the well-known, publicly available PASCAL and PhysioNet datasets. Experimental results show that the heart cycle segmentation and segment selection processes have a direct impact on the validation accuracy, sensitivity (TPR), precision (PPV), and specificity (TNR). Based on PASCAL dataset, we obtained encouraging classification results with overall accuracy 0.87, overall precision 0.81, and overall sensitivity 0.83. Concerning Micro classification results, we obtained Micro accuracy 0.91, Micro sensitivity 0.83, Micro precision 0.84, and Micro specificity 0.92. Using PhysioNet dataset, we achieved very good results: 0.97 accuracy, 0.946 sensitivity, 0.944 precision, and 0.946 specificity.

Exploring ensemble oversampling method for imbalanced keyword extraction learning in policy text based on three-way decisions and SMOTE

The e-government platform not only enables the government department to publish policy texts online, but also makes it easier for users to access the policy, especially for the convenience of understanding the policies by reading the keywords. For a given policy text, keywords take up only a small proportion, which can be seen as an unbalanced data set. Therefore, in this paper, we try to design automatic keyword extraction method of policy text with unbalanced data set. In order to achieve this goal, we firstly propose a new ensemble oversampling method to synthesize new data. In this case, we sample data from the training set by using Bagging method. During each sampling process, we train a logistic regression model to classify the training set. Based on the predicted probabilities, we utilize the classification confidence to divide training set into three regions by using three-way decisions (3WD). Then, we implement different strategies to synthesize new data. Besides, for keyword extraction of policy text, we conduct a series of experiments by using the classical supervised and unsupervised methods. In our experiment results, we can find that both in the public data sets and manual data sets, our sampling method can achieve better performance of F-measure and G-mean indexes, no matter what the supervised machine learning method is. This can also explain the advantage of 3WD. Different regions have different strategies to synthesize new data.

Performance Analysis of Two-Stage Iterative Ensemble Method over Random Oversampling Methods on Multiclass Imbalanced Datasets

Data imbalance occurring among multiclass datasets is very common in real-world applications. Existing studies reveal that various attempts were made in the past to overcome this multiclass imbalance problem, which is a severe issue related to the typical supervised machine learning methods such as classification and regression. But, still there exists a need to handle the imbalance problem efficiently as the datasets include both safe and unsafe minority samples. Most of the widely used oversampling techniques like SMOTE and its variants face challenges in replicating or generating the new data instances for balancing them across multiple classes, particularly when the imbalance is high and the number of rare samples count is too minimal thus leading the classifier to misclassify the data instances. To lessen this problem, we proposed a new data balancing method namely a two-stage iterative ensemble method to tackle the imbalance in multiclass environment. The proposed approach focuses on the rare minority sample’s influence on learning from imbalanced datasets and the main idea of the proposed approach is to balance the data without any change in class distribution before it gets trained by the learner such that it improves the learner’s learning process. Also, the proposed approach is compared against two widely used oversampling techniques and the results reveals that the proposed approach shows a much significant improvement in the learning process among the multiclass imbalanced data.

Predicting Overall Survival Time in Glioblastoma Patients Using Gradient Boosting Machines Algorithm and Recursive Feature Elimination Technique.

Simple SummaryDespite the highly aggressive nature of glioblastoma multiforme (GBM), survival time is in practice highly variable, and some of the patients remain stable for several years after treatment. The aim of this study was to develop a machine learning method that could precisely predict survival time of GBM patients. To do so, we integrated multi-modal MRI with non-supervised and supervised machines. We first identified compartments of the tumor then extracted their features. Then relevant useful features were selected by Random Forest-Recursive Feature Elimination (RF-RFE) to feed into Gradient Boosting Machine Algorithm with the aim of classifying GBM patients. By selecting the most relevant features, multi-modality MRI with tumor segmentation provided valuable independent and complete features to feed a machine learning model. Additionally, advanced machine-learning methods such as RF-RFE and GBoost are powerful tools for data mining. Hand-crafted feature-based methods have shown promising results, but there is no systematic way to determine survival-related hand-crafted features and existing methods mostly rely on experience.Despite advances in tumor treatment, the inconsistent response is a major challenge among glioblastoma multiform (GBM) that lead to different survival time. Our aim was to integrate multimodal MRI with non-supervised and supervised machine learning methods to predict GBM patients’ survival time. To this end, we identified different compartments of the tumor and extracted their features. Next, we applied Random Forest-Recursive Feature Elimination (RF-RFE) to identify the most relevant features to feed into a GBoost machine. This study included 29 GBM patients with known survival time. RF-RFE GBoost model was evaluated to assess the survival prediction performance using optimal features. Furthermore, overall survival (OS) was analyzed using univariate and multivariate Cox regression analyses, to evaluate the effect of ROIs and their features on survival. The results showed that a RF-RFE Gboost machine was able to predict survival time with 75% accuracy. The results also revealed that the rCBV in the low perfusion area was significantly different between groups and had the greatest effect size in terms of the rate of change of the response variable (survival time). In conclusion, not only integration of multi-modality MRI but also feature selection method can enhance the classifier performance.

Evaluation of Supervised Learning Models in Predicting Greenhouse Energy Demand and Production for Intelligent and Sustainable Operations

Plants need a specific environment to grow and reproduce in fine fettle. Nevertheless, climatic conditions are not stable and can impact their well-being and, consequently, harvest quality. Thus, greenhouse cultivation is one of the suitable agricultural techniques for creating and controlling the inside microclimate to be adequate for plant growth. The relevance of greenhouse control is widely recognized. The prediction of greenhouse variables using artificial intelligence methods is of great interest for intelligent control and the potential reduction in energetic and financial losses. However, the studies carried out in this context are still more or less limited and several machine learning methods have not been sufficiently exploited. The aim of this study is to predict the air conditioning electrical consumption and photovoltaic module electrical production at the smart Agro-Manufacturing Laboratory (SamLab) greenhouse, located in Albenga, north-western Italy. Different supervised machine learning methods were compared, namely, Artificial Neural Networks (ANNs), Gaussian Process Regression (GPR), Support Vector Machine (SVM) and Boosting trees. We evaluated the performance of the models based on three statistical indicators: the coefficient of correlation (R), the normalized root mean square error (nRMSE) and the normalized mean absolute error (nMAE). The results show good agreement between the measured and predicted values for all models, with a correlation coefficient R &gt; 0.9, considering the validation set. The good performance of the models affirms the importance of this approach and that it can be used to further improve greenhouse efficiency through its intelligent control.

A machine learning approach for single cell interphase cell cycle staging

The cell nucleus is a tightly regulated organelle and its architectural structure is dynamically orchestrated to maintain normal cell function. Indeed, fluctuations in nuclear size and shape are known to occur during the cell cycle and alterations in nuclear morphology are also hallmarks of many diseases including cancer. Regrettably, automated reliable tools for cell cycle staging at single cell level using in situ images are still limited. It is therefore urgent to establish accurate strategies combining bioimaging with high-content image analysis for a bona fide classification. In this study we developed a supervised machine learning method for interphase cell cycle staging of individual adherent cells using in situ fluorescence images of nuclei stained with DAPI. A Support Vector Machine (SVM) classifier operated over normalized nuclear features using more than 3500 DAPI stained nuclei. Molecular ground truth labels were obtained by automatic image processing using fluorescent ubiquitination-based cell cycle indicator (Fucci) technology. An average F1-Score of 87.7% was achieved with this framework. Furthermore, the method was validated on distinct cell types reaching recall values higher than 89%. Our method is a robust approach to identify cells in G1 or S/G2 at the individual level, with implications in research and clinical applications.

Normal-Weight Abdominal Obesity: A Risk Factor for Hypertension and Cardiometabolic Dysregulation

AbstractObjective:This study aimed to examine the associations of waist circumference with hypertension and cardiometabolic dysregulation among normal-weight adults.Methods:This cross-sectional study included 8795 normal-weight participants aged 20 to 79 years from the 2009–2018 US National Health and Nutrition Examination Survey. The demographic characteristics and cardiometabolic risk factors across waist circumference quartiles were summarized. We used adjusted multivariate logistic regression models, subgroup analysis, and restricted cubic spline to analyze the association between waist circumference and the prevalence of hypertension. Thereafter, we used the random forest supervised machine learning method, together with least absolute shrinkage and selection operator regression, to select hypertension-related features and created a predictive model based on regression analysis to identify hypertension in normal-weight individuals.Results:Waist circumference was positively correlated with hypertension in the non-adjusted, minimally adjusted, and fully adjusted models, with odds ratios (95% confidence interval) of 2.28 (2.14–2.44), 1.27 (1.12–1.44), and 1.27 (1.12–1.44), respectively. In the fully adjusted model, participants in the highest waist circumference quartile had a higher risk of hypertension relative to those in the lowest quartile, with an odds ratio (95% confidence interval) of 3.87 (1.59–10.34). Sensitivity analysis demonstrated the robustness of the association. In the testing set, the predictive model exhibited good performance, with an area under the curve of 0.803, sensitivity of 0.72, specificity of 0.76, and negative predictive value of 0.84.Conclusions:Measuring waist circumference may improve the evaluation of the risk of hypertension and help to manage cardiometabolic risk in normal-weight individuals.