Single Machine Learning Algorithm Research Articles

A supervised ensemble learning method for fault diagnosis in photovoltaic strings

This study proposes a fault diagnosis method based on the use of a machine learning (ML) technique called ensemble learning (EL) for photovoltaic (PV) systems. EL methods aim to obtain better generalizability and prediction accuracy than a single ML algorithm by combining the predictions of multiple algorithms. In this context, first the most relevant features are selected by using grid-search with cross-validation. Then each learning algorithm and the EL model that will combine them have been improved in terms of parameter optimization.Results show that, with the appropriate features and optimized parameters for each single learning algorithm and the EL model, the proposed method not only improves the classification performance but also has a strong generalization ability for PV system fault diagnosis.

Combination of Feature Selection and CatBoost for Prediction: The First Application to the Estimation of Aboveground Biomass

Increasing numbers of explanatory variables tend to result in information redundancy and “dimensional disaster” in the quantitative remote sensing of forest aboveground biomass (AGB). Feature selection of model factors is an effective method for improving the accuracy of AGB estimates. Machine learning algorithms are also widely used in AGB estimation, although little research has addressed the use of the categorical boosting algorithm (CatBoost) for AGB estimation. Both feature selection and regression for AGB estimation models are typically performed with the same machine learning algorithm, but there is no evidence to suggest that this is the best method. Therefore, the present study focuses on evaluating the performance of the CatBoost algorithm for AGB estimation and comparing the performance of different combinations of feature selection methods and machine learning algorithms. AGB estimation models of four forest types were developed based on Landsat OLI data using three feature selection methods (recursive feature elimination (RFE), variable selection using random forests (VSURF), and least absolute shrinkage and selection operator (LASSO)) and three machine learning algorithms (random forest regression (RFR), extreme gradient boosting (XGBoost), and categorical boosting (CatBoost)). Feature selection had a significant influence on AGB estimation. RFE preserved the most informative features for AGB estimation and was superior to VSURF and LASSO. In addition, CatBoost improved the accuracy of the AGB estimation models compared with RFR and XGBoost. AGB estimation models using RFE for feature selection and CatBoost as the regression algorithm achieved the highest accuracy, with root mean square errors (RMSEs) of 26.54 Mg/ha for coniferous forest, 24.67 Mg/ha for broad-leaved forest, 22.62 Mg/ha for mixed forests, and 25.77 Mg/ha for all forests. The combination of RFE and CatBoost had better performance than the VSURF–RFR combination in which random forests were used for both feature selection and regression, indicating that feature selection and regression performed by a single machine learning algorithm may not always ensure optimal AGB estimation. It is promising to extending the application of new machine learning algorithms and feature selection methods to improve the accuracy of AGB estimates.

A data fusion approach to optimize compositional stability of halide perovskites

Summary Search for resource-efficient materials in vast compositional spaces is an outstanding challenge in creating environmentally stable perovskite semiconductors. We demonstrate a physics-constrained sequential learning framework to subsequently identify the most stable alloyed organic-inorganic perovskites. We fuse data from high-throughput degradation tests and first-principle calculations of phase thermodynamics into an end-to-end Bayesian optimization algorithm using probabilistic constraints. By sampling just 1.8% of the discretized CsxMAyFA1−x−yPbI3 (MA, methylammonium; FA, formamidinium) compositional space, perovskites centered at Cs0.17MA0.03FA0.80PbI3 show minimal optical change under increased temperature, moisture, and illumination with >17-fold stability improvement over MAPbI3. The thin films have 3-fold improved stability compared with state-of-the-art multi-halide Cs0.05(MA0.17FA0.83)0.95Pb(I0.83Br0.17)3, translating into enhanced solar cell stability without compromising conversion efficiency. Synchrotron-based X-ray scattering validates the suppression of chemical decomposition and minority phase formation achieved using fewer elements and a maximum of 8% MA. We anticipate that this data fusion approach can be extended to guide materials discovery for a wide range of multinary systems.

Machine learning predicts lymph node metastasis of poorly differentiated-type intramucosal gastric cancer

To construct a machine learning algorithm model of lymph node metastasis (LNM) in patients with poorly differentiated-type intramucosal gastric cancer. 1169 patients with postoperative gastric cancer were divided into a training group and a test group at a ratio of 7:3. The model for lymph node metastasis was established with python machine learning. The Gbdt algorithm in the machine learning results finds that number of resected nodes, lymphovascular invasion and tumor size are the primary 3 factors that account for the weight of LNM. Effect of the LNM model of PDC gastric cancer patients in the training group: Among the 7 algorithm models, the highest accuracy rate was that of GBDT (0.955); The AUC values for the 7 algorithms were, from high to low, XGB (0.881), RF (0.802), GBDT (0.798), LR (0.778), XGB + LR (0.739), RF + LR (0.691) and GBDT + LR (0.626). Results of the LNM model of PDC gastric cancer patients in test group : Among the 7 algorithmic models, XGB had the highest accuracy rate (0.952); Among the 7 algorithms, the AUC values, from high to low, were GBDT (0.788), RF (0.765), XGB (0.762), LR (0.750), RF + LR (0.678), GBDT + LR (0.650) and XGB + LR (0.619). Single machine learning algorithm can predict LNM in poorly differentiated-type intramucosal gastric cancer, but fusion algorithm can not improve the effect of machine learning in predicting LNM.

Merging multiple satellite-based precipitation products and gauge observations using a novel double machine learning approach

This study proposed a novel double machine learning (DML) approach to merge multiple satellite-based precipitation products (SPPs) and gauge observations, and tested its reliability and validity over the Chinese mainland. The DML approach was mainly developed based on the classification model of random forest (RF) in combination with the regression models of the machine learning (ML) algorithms including RF, artificial neural network (ANN), support vector machine (SVM) and extreme learning machine (ELM). This led to four DML algorithms, i.e., RF-RF, RF-ANN, RF-SVM, and RF-LM. The performance of the DML algorithms were compared to the single machine learning (SML) algorithms developed based solely on the regression models of RF, ANN, SVM, and ELM, and to the liner merging methods including the inverse error variance weighting, the one-outlier-removed average, and the optimized weight average. In total, we produced twelve precipitation products including four of the DML algorithms, four of the SML algorithms, three of the liner merging methods, and another one generated via the gauge-only interpolation. The precipitation observations at 697 gauges were spatially and randomly divided into two parts (i.e., 70% and 30%), one was used for the training of the ML algorithms or for the interpolation, while the other for the performance evaluations. Results indicate that the DML algorithms outperform the other merging methods, the gauge-only interpolation, and the original SPPs over the Chinese mainland. The median Kling-Gupta efficiency (KGE) ranges 0.67–0.71 for the merged products of DML, which are obviously higher than the original SPPs (0.31–0.54), the linear merged product (0.54–0.55), gauge-only interpolated product (0.62), and the SML-based products (0.47–0.65). The DML-based products also exhibit better performances than the other products in detecting precipitation events with the threshold of 1 mm/day, and outperform the original SPPs regardless of the precipitation thresholds. Further analyses imply that: (i) the DML-based products could outperform the original SPPs even with a small training dataset size; (ii) the superiority of the DML approach to SML is mainly due to that the former can better capture the temporal dynamics of precipitation; (iii) the added values of the merged products of DML relative to the original SPPs and the gauge-only product vary with the sizes of the training dataset; and (iv) the ensemble of the DML algorithms could not further improve the accuracy of the precipitation estimates. This study not only provided an effective and robust tool for the fusion of multiple SPPs and gauge observations, but also, for the first time, compared the performance of various ML algorithms in merging satellite and gauge-based precipitation.

A particle swarm optimization based ensemble for vegetable crop disease recognition

Ensemble methods give better performance compared to a single machine learning algorithm. Vote is one of the best ensembles. Vote merges predictions from Simple Logistics and the Naive Bayes algorithms in the present work. The paper presents a new ensemble approach – Ensemble Particle Swarm Optimization (EnsPSO). The EnsPSO approach is a combination of (i) Vote, (ii) Correlation based Feature(s) Selection (CFS) method, (iii) PSO algorithm and (iv) random sampling method. The EnsPSO shows better performance results than Vote. The EnsPSO shows higher classification accuracy (96%) as compared to Vote (84%). The performance enhancement of EnsPSO is also proved using ten-fold cross validation on 3 standard datasets.

Hybrid randomised learning‐based probabilistic data‐driven method for fault‐induced delayed voltage recovery assessment of power systems

With a large number of inverter-interfaced renewable power generation, fault-induced delayed voltage recovery (FIDVR) events have become a serious threat to power system stability assessment. This study proposes a novel data-driven method based on probabilistic prediction, ensemble learning, and multi-objective optimisation programming (MOP) to rapidly predict the FIDVR severity index for real-time FIDVR assessment. Distinguished from the existing single machine learning (ML) algorithm data-driven method, the proposed method combines different randomised learning algorithms to acquire a more diversified ML outcome. The probabilistic prediction models the uncertainties existing in the prediction process, which quantifies the prediction confidence over a progressive observation window. Besides, the FIDVR can be evaluated through the time-adaptive framework to achieve the best FIDVR speed and accuracy with the MOP framework. The simulation results on the New England 10-machine 39-bus system display its preponderance over the single ML, and also demonstrate its better speed and accuracy performance in FIDVR assessment.

Design of an Accurate Machine Learning Algorithm to Predict the Binding Energies of Several Adsorbates on Multiple Sites of Metal Surfaces

AbstractIn the current work, we design a single unique machine learning (ML) algorithm capable of predicting the binding energies of several C, N and O‐based adsorbates and atomic hydrogen on different facets (100, 111, 211) of eleven transition‐metals considering an FCC bulk structure (Co, Rh, Ir, Ni, Pd, Pt, Ru, Os, Cu, Ag, Au) with high accuracy with respect to the reference DFT calculations. The selected properties/features are based on already available data or electronic properties easily obtained from DFT calculations of the free adsorbates and on the clean metal surfaces. The mean average error (MAE) for the training set is equal to 0.074 eV, while for the test set (data not used in training) is equal to 0.174 eV, thus having a very small error with respect to the reference DFT calculations. Further modifications of the present algorithm, which is based on an Extragradient boost (XGBoost) regressor in combination with a tree booster, with the addition of few more features might set the ground to develop ML algorithms able to predict the binding energies of adsorbates on more complex catalytic surfaces.

Super ensemble learning for daily streamflow forecasting: large-scale demonstration and comparison with multiple machine learning algorithms

Daily streamflow forecasting through data-driven approaches is traditionally performed using a single machine learning algorithm. Existing applications are mostly restricted to examination of few case studies, not allowing accurate assessment of the predictive performance of the algorithms involved. Here we propose super learning (a type of ensemble learning) by combining 10 machine learning algorithms. We apply the proposed algorithm in one-step ahead forecasting mode. For the application, we exploit a big dataset consisting of 10-year long time series of daily streamflow, precipitation and temperature from 511 basins. The super learner improves over the performance of the linear regression algorithm by 20.06%, outperforming the "hard to beat in practice" equal weight combiner. The latter improves over the performance of the linear regression algorithm by 19.21%. The best performing individual machine learning algorithm is neural networks, which improves over the performance of the linear regression algorithm by 16.73%, followed by extremely randomized trees (16.40%), XGBoost (15.92%), loess (15.36%), random forests (12.75%), polyMARS (12.36%), MARS (4.74%), lasso (0.11%) and support vector regression (-0.45%). Based on the obtained large-scale results, we propose super learning for daily streamflow forecasting.

A Robust Blood-based Signature of Cerebrospinal Fluid Aβ42 Status.

Early detection of Alzheimer's disease (AD) is of vital importance in the development of disease-modifying therapies. This necessitates the use of early pathological indicators of the disease such as amyloid abnormality to identify individuals at early disease stages where intervention is likely to be most effective. Recent evidence suggests that cerebrospinal fluid (CSF) amyloid β1-42 (Aβ42) level may indicate AD risk earlier compared to amyloid positron emission tomography (PET). However, the method of collecting CSF is invasive. Blood-based biomarkers indicative of CSF Aβ42 status may remedy this limitation as blood collection is minimally invasive and inexpensive. In this study, we show that APOE4 genotype and blood markers comprising EOT3, APOC1, CGA, and Aβ42 robustly predict CSF Aβ42 with high classification performance (0.84 AUC, 0.82 sensitivity, 0.62 specificity, 0.81 PPV and 0.64 NPV) using machine learning approach. Due to the method employed in the biomarker search, the identified biomarker signature maintained high performance in more than a single machine learning algorithm, indicating potential to generalize well. A minimally invasive and cost-effective solution to detecting amyloid abnormality such as proposed in this study may be used as a first step in a multi-stage diagnostic workup to facilitate enrichment of clinical trials and population-based screening.

Prediction of photovoltaic power output based on similar day analysis, genetic algorithm and extreme learning machine

Recently, many machine learning techniques have been successfully employed in photovoltaic (PV) power output prediction because of their strong non-linear regression capacities. However, single machine learning algorithm does not have stable prediction performance and sufficient generalization capability in the prediction of PV power output. In this work, a hybrid model (SDA-GA-ELM) based on extreme learning machine (ELM), genetic algorithm (GA) and customized similar day analysis (SDA) has been developed to predict hourly PV power output. In the SDA, Pearson correlation coefficient is employed to measure the similarity between different days based on five meteorological factors, and the data samples similar to those from the target forecast day are selected as the training set of ELM. This operation can effectively increase the number of useful samples and reduce the time consumption on training data. In the ELM, the optimal values of the hidden bias and the input weight are searched by GA to improve the prediction accuracy. The performance of the proposed forecast model is evaluated with coefficient of determination (R2), mean absolute error (MAE) and normalized root mean square error (nRMSE). The results show that the SDA-GA-ELM model has higher accuracy and stability in day-ahead PV power prediction.

Heat load prediction of residential buildings based on discrete wavelet transform and tree-based ensemble learning

Currently, the high energy consumption of district heating system represents a problem. Thus, accurate prediction of future heat load is the key to ensure energy saving and improve system operation efficiency. However, the accuracy of the prediction model based on single machine learning algorithms need to be improved, and the hybrid prediction model suffers from redundant inputs. The objective of this study was to obtain the most accurate prediction of the short-term heat load consumption. The discrete wavelet transform, two tree-based ensemble learning algorithms, extremely randomized trees regression and gradient boosting decision tree were used to establish hybrid models for predicting the heat load 1 h in advance. Pearson and least absolute shrinkage and selection operator (LASSO) methods were used to optimize the feature set, including system parameters, meteorological parameters and time steps. This resulted in four feature sets. In addition, the effect of the historical heat load on the performance of the prediction model was analysed. Actual operation data of an established heating system were collected to test the performance of the models. The results were then compared with those of the models established by artificial neural network and support vector regression. The results show that the historical heat load had a significant impact on the prediction accuracy of the heat load of the upcoming hour, and the extreme random tree model based on discrete wavelet transform showed a better prediction performance and lower model complexity in case of using the feature set selected by the LASSO method.

Type 2 Machine Learning: An Effective Hybrid Prediction Model for Early Type 2 Diabetes Detection

Importance: Diabetes is a chronic disease that can cause long term damage to various parts of the body. To prevent diabetic complications, different attempts integrating machine learning with medicine have been made for building models to predict whether a patient has diabetes or not, but predicting this disease still has room for improvement. Hybrid prediction model presents a novel method and mostly achieve a much better optimal outcome than single classical machine learning algorithms. Objective: To develop a high accuracy model for different onsets of type 2 diabetes prediction. In this way, the integration between clustering and classification techniques can be improved to help detecting diabetes at an earlier stage without deleting observations with missing values and also decrease insignificant features to get the most related features during data collection. Methods: We implement a noise reduction based technique using Kmeans clustering followed by running the Random forest and XGBoost classifiers to extract the unknown hidden features of the dataset and for more accurate results. Results: Prediction accuracy can be observed by benchmarking our model against up-to-date predictive models and common classification algorithms. With an accuracy of 97.53% by 10 fold cross validation, our T2ML model reaches a better accuracy compared with other experiments reported by other researchers in the literature and over various conventional classification algorithms.

Recognition of Bangla Handwritten Number Using Combination of PCA and FIS with the Aid of DWT

The structure of any Bangla numerical character is more complex compared to English numerical character. Two pairs of numerical character in Bangla resembles to be closed and they are: “one and nine” and “five and six”. We found that, handwritten Bangla numerical character cannot be recognized using single machine learning algorithm or discrete wavelet transform (DWT). Above phenomenon motivated us to use combination of DWT, Fuzzy Inference System (FIS) and Principal Component Analysis (PCA) to recognize numerical characters of Bangla in handwritten format. The four lowest spectral components of a preprocessed image are taken using DWT, which is considered as the feature vector to recognize the digits in first phase. The feature vector is then applied to FIS and PCA separately. The combined method provides recognition accuracy of 95.8% whereas application of individual method gives less rate of accuracy. Instead of storing the images itself in a folder, if we can store the feature vector of images achieved from DWT in tabular form. The records of table can be applied in FIS, PCA or other object detection algorithm. Although the technique used in the paper can detect objects with moderate rate of accuracy but can save huge storage against a benchmark database of images. If a tradeoff is made between storage requirements and accuracy of recognition, the model of the paper is preferable compared to other present state-of-art. Another finding of the paper is that, the spectral components of images acquired by DWT only matched with FIS and PCA for classification but do not match properly with unsupervised (K-mean clustering) and supervised (support vector machine) learning.

Application of ensemble learning techniques to model the atmospheric concentration of SO2.

In view of pollution prediction modeling, the study adopts homogenous (random forest, bagging, and additive regression) and heterogeneous (voting) ensemble classifiers to predict the atmospheric concentration of Sulphur dioxide. For model validation, results were compared against widely known single base classifiers such as support vector machine, multilayer perceptron, linear regression and regression tree using M5 algorithm. The prediction of Sulphur dioxide was based on atmospheric pollutants and meteorological parameters. While, the model performance was assessed by using four evaluation measures namely Correlation coefficient, mean absolute error, root mean squared error and relative absolute error. The results obtained suggest that 1) homogenous ensemble classifier random forest performs better than single base statistical and machine learning algorithms; 2) employing single base classifiers within bagging as base classifier improves their prediction accuracy; and 3) heterogeneous ensemble algorithm voting have the capability to match or perform better than homogenous classifiers (random forest and bagging). In general, it demonstrates that the performance of ensemble classifiers random forest, bagging and voting can outperform single base traditional statistical and machine learning algorithms such as linear regression, support vector machine for regression and multilayer perceptron to model the atmospheric concentration of sulphur dioxide.

A Novel Approach of Weighted Support Vector Machine with Applied Chance Theory for Forecasting Air Pollution Phenomenon in Egypt

The particulate matter air pollutant of diameter less than 10 micrometers (PM[Formula: see text]), a category of pollutants including solid and liquid particles, can be a health hazard for several reasons: it can harm lung tissues and throat, aggravate asthma and increase respiratory illness. Accurate prediction models of PM[Formula: see text] concentrations are essential for proper management, control, and making public warning strategies. Therefore, machine learning techniques have the capability to develop methods or tools that can be used to discover unseen patterns from given data to solve a particular task or problem. The chance theory has advanced concepts pertinent to treat cases where both randomness and fuzziness play simultaneous roles at one time. The main objective is to study the modification of a single machine learning algorithm — support vector machine (SVM) — applying the chance weight of the target variable, based on the chance theory, to the corresponding dataset point to be superior to the ensemble machine learning algorithms. The results of this study are outperforming of the SVM algorithms when modifying and combining with the right theory/technique, especially the chance theory over other modern ensemble learning algorithms.

Ensemble Methods with Voting Protocols Exhibit Superior Performance for Predicting Cancer Clinical Endpoints and Providing More Complete Coverage of Disease-Related Genes.

In genetic data modeling, the use of a limited number of samples for modeling and predicting, especially well below the attribute number, is difficult due to the enormous number of genes detected by a sequencing platform. In addition, many studies commonly use machine learning methods to evaluate genetic datasets to identify potential disease-related genes and drug targets, but to the best of our knowledge, the information associated with the selected gene set was not thoroughly elucidated in previous studies. To identify a relatively stable scheme for modeling limited samples in the gene datasets and reveal the information that they contain, the present study first evaluated the performance of a series of modeling approaches for predicting clinical endpoints of cancer and later integrated the results using various voting protocols. As a result, we proposed a relatively stable scheme that used a set of methods with an ensemble algorithm. Our findings indicated that the ensemble methodologies are more reliable for predicting cancer prognoses than single machine learning algorithms as well as for gene function evaluating. The ensemble methodologies provide a more complete coverage of relevant genes, which can facilitate the exploration of cancer mechanisms and the identification of potential drug targets.

Using Arbiter and Combiner Tree to Classify Contexts of Data

This paper reports on the use of ensemble learning to classify as either positive or negative the sentiment of Tweets. Tweets were chosen as Twitter is a popular tool and a public, human annotated dataset was made available as part of the SemEval 2013 competition. We report on a classification approach that contrasts single machine learning algorithms with a combination of algorithms in an ensemble learning approach. The single machine learning algorithms used were support vector machine (SVM) and Naive Bayes (NB), while the methods of ensemble learning include the arbiter tree and the combiner tree. Our system achieved an F-score using Tweets and SMS with the arbiter tree at 83.57% and 93.55%, respectively, which was better than base classifiers; meanwhile, the results from the combiner tree achieved lower scores than base classifiers.

An Operational Framework for Land Cover Classification in the Context of REDD+ Mechanisms. A Case Study from Costa Rica

REDD+ implementation requires robust, consistent, accurate and transparent national land cover historical data and monitoring systems. Satellite imagery is the only data source with enough periodicity to provide consistent land cover information in a cost-effective way. The main aim of this paper is the creation of an operational framework for monitoring land cover dynamics based on Landsat imagery and open-source software. The methodology integrates the entire land cover and land cover change mapping processes to produce a consistent series of Land Cover maps. The consistency of the time series is achieved through the application of a single trained machine learning algorithm to radiometrically normalized imagery using iteratively re-weighted multivariate alteration detection (IR-MAD) across all dates of the historical period. As a result, seven individual Land Cover maps of Costa Rica were produced from 1985/1986 to 2013/2014. Post-classification land cover change detection was performed to evaluate the land cover dynamics in Costa Rica. The validation of the land cover maps showed an overall accuracy of 87% for the 2013/2014 map, 93% for the 2000/2001 map and 89% for the 1985/1986 map. Land cover changes between forest and non-forest classes were validated for the period between 2001 and 2011, obtaining an overall accuracy of 86%. Forest age-classes were generated through a multi-temporal analysis of the maps. By linking deforestation dynamics with forest age, a more accurate discussion of the carbon emissions along the time series can be presented.

Indonesian Named-entity Recognition for 15 Classes Using Ensemble Supervised Learning

Here, we describe our effort in building Indonesian Named Entity Recognition (NER) for newspaper article with 15 classes which is larger number of class type compared to existing Indonesian NER. We employed supervised machine learning in the NER and conducted experiments to find the best attribute combination and the best algorithm with highest accuracy. We compared the attribute of word level, sentence level and document level. In the algorithm, we compared several single machine learning algorithms and also an ensembled one. Using 457 news articles, the best accuracy was achieved by using ensemble technique where the result of several machine learning algorithms were used as the feature for one machine learning algorithm.