Ensemble Of Machine Learning Models Research Articles

Approaches for a New Generation of Fast-Computing Catalyst Models

In the present study, three different approaches to model a SCRF disregarding the filtration effects were developed and tested against each other in terms of accuracy and calculation time. A high-dimensional, state-of-the-art, physicochemical axisuite$^{{\circledR }}$ model was used as reference and for the generation of training data. The examined approaches were a low-dimensional physicochemical model, a stack ensemble of machine learning models and a neural network. The training data, produced on a virtual test bench, consisted of 39 simulations of synthetic driving cycles which comply with the current RDE legislation. The test data includes simulations of a synthetic and a non-synthetic RDE cycle, a WLTC, and a NEDC. The chosen cycles differ significantly in terms of similarity to the training data, making it possible to examine the generalization capability of the models and the dependency on the data. The results clearly show the superiority of the neural network approach in terms of accuracy, training time, and simulation time. The neural network approach reached an average explained variance score of 0.98 followed very closely by the ensemble (0.96) and the low-dimensional approach (0.76). The runtime was compared based on the computational time required to simulate one real time second. The reference model needed 23 ms. The ensemble model was not able to achieve a major speedup. The low-dimensional model was five times faster and the neural network 450 times faster than the reference model.

Do Many Models Make Light Work? Evaluating Ensemble Solutions for Improved Rumor Detection

There have been many efforts to detect rumors using various machine learning (ML) models, but there is still a lack of understanding of their performance against different rumor topics and available features, resulting in a significant performance degrade against completely new and unseen (unknown) rumors. To address this issue, we investigate the relationship between ML models, features, and rumor topics to select the best rumor detection model under specific conditions using 13 different ML models. Our experiment results demonstrate that there is no clear winner among the ML models in all different rumor topics with respect to the detection performance. To overcome this problem, a possible way is to use an ensemble of ML models. Although previous work presents an improved detection of rumors using ensemble solutions (ES), their evaluation did not consider detecting unknown rumors. Further, they did not present nor evaluate the configuration of the ES to ensure that it indeed performs better than using a single ML model. Based on these observations, we propose to evaluate the use of an ES by examining their unknown rumor detection performance compared with single ML models but as well as different configurations of the ESes. Our experimental results using real-world datasets found that an ES of Random Forest, XGBoost and Multilayer perceptron overall produced the best F1 score of 0.79 for detecting unknown rumors, a significant improvement compared with a single best ML model which only achieved a 0.58 F1 score. We also showed that not all ESes are the same, with significantly degraded detection and large variations in performance when different ML models are used to construct the ES. Hence, it is infeasible to rely on any single ML model-based rumor detector. Finally, our solution also performed better than other recent detectors, such as eventAI and NileTMRG that performed similar to using a single ML model - making it a much more attractive solution to detect unknown rumors in practice.

Machine Learning Based Real-Time Monitoring of Long-Term Voltage Stability Using Voltage Stability Indices

This article presents a machine learning approach to predict the long-term voltage stability margin as represented by the Loadability Margin (LM). LM is an intuitive and easily understandable indicator of voltage stability. The unique feature of the proposed technique is the use of different Voltage Stability Indices (VSI) proposed in the literature as inputs to an ensemble of machine learning models which predict the LM. The VSIs used are carefully selected to include those based on different principles and computable using real time synchrophasor measurements. In addition, the paper presents a methodology to generate training data under different operational conditions and N-1 contingencies to train the machine learning models. The best machine learning algorithm and the categories of input VSIs are selected through a comparative study. These studies were conducted on the IEEE 14 bus system and IEEE 118 bus system and led to the selection of Random Forest Regression machine learning algorithm, and confirmed the accuracy and robustness of the proposed method. The system was implemented on real time PhasorSmart® synchrophasor application platform and validated using RTDS® real-time simulator. The impact of synchrophasor measurement errors on the proposed technique were also analyzed.

Machine Learning Based Epileptic Seizure Detection for Responsive Neurostimulator System Optimization

This paper proposes a novel method of identifying the time of epileptic seizure happening on patients by employing feature extraction and machine learning-based classification on Electroencephalogram (EEG) signal collected from a closed-loop interface implanted in the brain of patients. The closed-loop device was served as a neurostimulator which introduced stimuli to epilepsy patients when detecting the occurrence of seizure. A set of multiple time- and frequency-domain features are extracted from intracranial electroencephalography recordings of 7 subjects with epilepsy. Trained and tested on the extracted features, an ensemble of machine learning models with parameter tuning achieves an area under the curve (AUC) score of 0.99.

Recipe for Quantitative Trading with Machine Learning

On one hand, financial time series are multifractal, thus exhibiting non-Gaussian distribution, the presence of extreme values (outliers), and long-range dependent dynamics. On the other hand, machine learning (ML) models are processes relying heavily on statistical models and methodologies, but treated as black box models due to their inability to explicitly know the relations established between explanatory variables (input) and dependent variables (output). However, when forecasting market returns, or generating autonomous patterns, it is crucial to know the statistical properties of the time series produced by the ML model under consideration. We first present some results on using recurrent neural networks to forecast market returns. We then discuss forecasting stock price directions and propose an algorithm for simultaneously forecasting both future price levels and asset price directions. We also discuss autonomous pattern generation and focus on learning traditional shapes defined by stock analysts. Then, we present complex network analysis to predict stock price fluctuation patterns. At last, we take into consideration the statistical characteristics of financial time series and present a recipe for forecasting both market returns and their directions. We choose to reverse the causality and propose a solution consisting in deciding upon the framework by defining how the model should be specified before beginning to analyse the actual data. We should define the framework by properly formulating model hypotheses which make financial or economic sense, and then carefully determining the number of dependent variables in a regression model, or the number of factors and components in a stochastic model. Based on these observations, we use the multifractal formalism (MF) as a framework for testing the capacity of an ML model to reproduce some non-overlapping statistical properties of the time series. First we define a set of theoretical models with distinct statistical characteristics as well as a set of ML models with the ability to reproduce the properties of the models in the former set. We then train each individual ML model to replicate their respective theoretical model and use ensemble methods to combine these now calibrated ML models to form a meta-model. However, the weights of the latter are statistically computed as a function of the Hurst exponent. As a result, the meta-model is dynamically recombined based on the changing properties of the financial series over time. Once we have identified an ensemble of ML models with specific non-overlapping statistical properties, we can train each constituent model to learn a large number of patterns or technical indicators. We then devise a trading algorithm with strategies accounting for a specific level of Hurst exponent.

Feature reduction through manifold learning for a geospatial model of ambient soundscapes

Manifold learning is a type of nonlinear dimensionality reduction that helps identify a minimal combination of features to characterize data. This presentation discusses the use of diffusion maps, a type of manifold learning, on a dataset of 68 geospatial features which cover the contiguous United States. These geospatial features have been used previously to make predictions of ambient soundscapes through an ensemble of machine learning models. As the current model capabilities are expanded to predict the ambient soundscape outside of the contiguous United States, decisions must be made about which geospatial features are required for accurate models. In particular, some of these 68 features are not available or are expensive to obtain for regions outside of the contiguous United States. Diffusion maps can assist in identifying the features, or combinations of features, that best characterize the data space. [Work supported by a U.S. Army SBIR.]

Racial Disparity in the Genomic Basis of Radiosensitivity – An Exploration of Whole-Transcriptome Sequencing Data via a Machine-Learning Approach

There is emerging evidence for the existence of intrinsic biological differences in cancer responsiveness to therapeutic radiation between African-American (AA) men compared to non-Hispanic White (NHW) men (Spratt et al. [abstract ASTRO 2018]). The basis of this racial difference in radiosensitivity is likely based on variations in the expression of genes encoding for radiation response pathways. For example, decreased double strand break repair gene expression is associated increased radiosensitivity in somatic and cancer cell lines. We hypothesized that a racial difference in the expression levels of genes participating in radiation response pathways could be identified via a machine learning approach. We extracted the gene expression level data of 7,470 patients from the Genomic Data Commons Pan-Cancer database who had race identified as AA (n=802) or NHW (n=6,668). For each patient, the expression levels of 741 genes that are known to be involved in radiation response pathways were selected for subsequent analysis. An ensemble of five machine learning methods (support vector machine (SVM), linear discriminant analysis (LDA), gradient boosted machine (GBM), Bayesian generalized linear model (BGLM), and sample mean (SM)) was trained on 80% of the data to predict for race based on this 741-gene expression panel. Out-of-sample error was estimated using 5-fold cross validation. The trained ensemble model was used to predict on the remaining 20% of the data. Performance of the ensemble model was evaluated via area under the curve (AUC) of the receiver operating characteristic curve. The mean squared error for the SVM, LDA, GBM, BGLM, and SM methods were 0.071, 0.075, 0.081, 0.076, and 0.096 respectively. The ensemble model achieved a mean square error of 0.068. Prediction by the ensemble model yielded an AUC of 0.861 (95% CI 0.844-0.878). Expression levels of radiation response pathway genes can be used to accurately identify race via an ensemble of machine learning models. This supports the emerging evidence that race may be associated with radiosensitivity via intrinsic biologic differences in gene expression levels. Further studies are warranted to investigate whether these gene expression differences translate to clinically detectable variation in radiosensitivity and tumor control among different patient populations.

Assessment of an ensemble of machine learning models toward abnormality detection in chest radiographs.

Respiratory diseases account for a significant proportion of deaths and disabilities across the world. Chest X-ray (CXR) analysis remains a common diagnostic imaging modality for confirming intra-thoracic cardiopulmonary abnormalities. However, there remains an acute shortage of expert radiologists, particularly in under-resourced settings, resulting in severe interpretation delays. These issues can be mitigated by a computer-aided diagnostic (CADx) system to supplement decision-making and improve throughput while preserving and possibly improving the standard-of-care. Systems reported in the literature or popular media use handcrafted features and/or data-driven algorithms like deep learning (DL) to learn underlying data distributions. The remarkable success of convolutional neural networks (CNN) toward image recognition tasks has made them a promising choice for automated medical image analyses. However, CNNs suffer from high variance and may overfit due to their sensitivity to training data fluctuations. Ensemble learning helps to reduce this variance by combining predictions of multiple learning algorithms to construct complex, non-linear functions and improve robustness and generalization. This study aims to construct and assess the performance of an ensemble of machine learning (ML) models applied to the challenge of classifying normal and abnormal CXRs and significantly reducing the diagnostic load of radiologists and primary-care physicians.

Analytics Solution Helps Identify Rod-Pump Failure at the Wellhead

This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 192513, “Industrial Internet of Things Edge Analytics: Deploying Machine Learning at the Wellhead To Identify Rod-Pump Failure,” by Bartosz Boguslawski, Matthieu Boujonnier, Loryne Bissuel-Beauvais, Fahd Saghir, SPE, and Rajesh D. Sharma, SPE, Schneider Electric, prepared for the 2018 SPE Middle East Artificial Lift Conference and Exhibition, Manama, Bahrain, 28–29 November. The paper has not been peer reviewed. Industrial-Internet-of-things (IIOT) architecture provides an opportunity to improve asset uptime and maintenance of assets, reduce safety risks, and optimize operational costs. However, to turn data into meaningful insights, the industry must make full benefit of machine-learning (ML) models. This paper presents an analytics solution for identifying rod-pump failure capable of automated dynacard recognition at the wellhead that uses an ensemble of ML models. The proposed solution does not require Internet connectivity to generate alarms and meets confidentiality requirements. Rod-Pump-Control (RPC) Architecture Thanks to recent progress in microelectronics, the embedding of ML models in remote places with scarce connectivity, known as edge computing, is possible. Thanks to the insights generated at the oil field, onsite maintenance teams can apply immediate corrective responses, working efficiently and safely. RPC architecture enables automated control of sucker-rod pumps. A variable-speed drive (VSD) controls the pump by adjusting the speed of the motor to downhole conditions. The VSD is controlled by a remote terminal unit (RTU) that provides speed reference for improved rod-pump control. The RTU also collects sensor measurements. In the work covered in the paper, two sensors, namely the proximity sensor and the load cell, are of particular interest. The proximity sensor is mounted near the crank arm, while the load cell is a transducer mounted between the polished rod clamp and the bridle. The RTU sends the measurements to a touchscreen human/machine interface for a local user. It also communicates the measurements to a supervisory control and data-acquisition (SCADA) system. The communication between the RTU and the SCADA host is established through external wireless communication devices such as radios or cellular modems. Furthermore, in the ex-ample discussed in the paper, a local edge computing gateway is added to the architecture. Edge gateways have significant computing power and can run ML-based applications. The gateway retrieves the data from the RTU and performs onsite analytics, generates alarms, and communicates with the cloud sporadically. Using the measurements from the two sensors, the RPC produces a surface card (or dynacard), from which downhole pump conditions may be deduced, and consequently calculates a downhole card, from which downhole pump conditions may be inferred. The downhole card is, in essence, a translated surface card with oscillatory harmonics removed. An experienced operator is able to infer from the downhole card whether a pump is operating normally or has failed in some way.

Benchmarking deep learning models on large healthcare datasets

Deep learning models (aka Deep Neural Networks) have revolutionized many fields including computer vision, natural language processing, speech recognition, and is being increasingly used in clinical healthcare applications. However, few works exist which have benchmarked the performance of the deep learning models with respect to the state-of-the-art machine learning models and prognostic scoring systems on publicly available healthcare datasets. In this paper, we present the benchmarking results for several clinical prediction tasks such as mortality prediction, length of stay prediction, and ICD-9 code group prediction using Deep Learning models, ensemble of machine learning models (Super Learner algorithm), SAPS II and SOFA scores. We used the Medical Information Mart for Intensive Care III (MIMIC-III) (v1.4) publicly available dataset, which includes all patients admitted to an ICU at the Beth Israel Deaconess Medical Center from 2001 to 2012, for the benchmarking tasks. Our results show that deep learning models consistently outperform all the other approaches especially when the ‘raw’ clinical time series data is used as input features to the models.

Predicting artificially drained areas by means of a selective model ensemble

Farmers often install subsurface drainage systems to improve yields on wet soils, which has large impacts on the hydrological system. The present study uses an ensemble of machine learning models to map the extent of artificially drained areas in Denmark. The prediction is based on 745 field observations, of which one third is held out for evaluation, and 46 covariate layers. A library of 308 models is trained using 77 machine learning methods and four datasets containing either a combination of topographic variables, satellite imagery, soil properties and land use information or principal components based on these variables.A stepwise algorithm then selects models from the library, based on their predictions on a hillclimb dataset. The results show that models trained using principal components generally yielded a better performance than the models trained with the raw covariates. Furthermore, the best results were obtained when only a random fraction of the models was available for selection at each step. The covariates that were most important for the prediction of artificially drained areas mostly related to soil properties and topography. Overall, the ensemble predicted artificially drained areas with an accuracy of 76.5%. The study demonstrates machine learning as an accurate method for mapping artificially drained areas, which is likely to benefit both farmers and decision makers.