Artificial Neural Network Ensemble Research Articles

Machine learning for satellite-based sea-state prediction in an offshore windfarm

Accurate wave forecasts are essential for the safe and efficient maritime operations and, in particular, the maintenance of offshore wind farms. Here, machine learning and remote monitoring from satellites are integrated to provide uniquely detailed predictions of significant wave height (SWH). C-Band Synthetic Aperture Radar images from European Space Agency Sentinel-1 satellites were combined with wave-buoy data from around the UK, using the CEFAS Wavenet. A total of 240 images in wide swarth mode were collected, that represent significant wave height ranging from 0 to 4.7 m. Image properties related to sea surface roughness in dual-polarization mode, together with the wave buoy data, trained an ensemble of artificial neural networks. The trained networks were shown to provide an effective method for the estimation of the SWH, having an RMSE = 0.23 m for SWH<3 m, which is the region of interest for offshore wind energy applications. The methodology enables information on the spatial distribution of wave height in very high resolution to be obtained. Sea-state resolved from the satellite data, using the artificial neural network shows that windfarm infrastructure directly influences wave propagation. The overall variance of significant wave height throughout a wind farm was calculated, providing information on regions of interest with considerably different wave heights as compared to the nearest wave buoy. The new model will help towards improved downscaling of general sea state forecasts, locating hotspots of different wave height properties and correct prioritization of maintenance jobs to perform in wind turbines.

Forecasting point-of-consumption chlorine residual in refugee settlements using ensembles of artificial neural networks

Waterborne illnesses are a leading health concern in refugee and internally displaced person (IDP) settlements where waterborne pathogens often spread through household recontamination of stored water. Ensuring sufficient chlorine residual is important for protecting drinking water against recontamination and ensuring water remains safe up to the point-of-consumption. We used ensembles of artificial neural networks (ANNs) to probabilistically forecast the point-of-consumption free residual chlorine (FRC) concentration and to develop point-of-distribution FRC targets based on the risk of insufficient FRC at the point-of consumption. We built ANN ensemble models using data from three refugee settlements and found that the risk-based FRC targets generated by the ensemble models were consistent with an empirical water safety evaluation, indicating that the models accurately predicted the risk of low point-of-consumption FRC despite all ensemble forecasts being underdispersed even after post-processing. This demonstrates the usefulness of ANN ensembles for generating risk-based point-of-distribution FRC targets to ensure safe drinking water in humanitarian operations.

Artificial Neural Network classification of 4FGL sources

ABSTRACT The Fermi-LAT DR1 and DR2 4FGL catalogues feature more than 5000 gamma-ray sources of which about one fourth are not associated with already known objects, and approximately one third are associated with blazars of uncertain nature. We perform a three-category classification of the 4FGL DR1 and DR2 sources independently, using an ensemble of Artificial Neural Networks (ANNs) to characterize them based on the likelihood of being a Pulsar (PSR), a BL Lac type blazar (BLL) or a Flat Spectrum Radio Quasar (FSRQ). We identify candidate PSR, BLL, and FSRQ among the unassociated sources with approximate equipartition among the three categories and select 10 classification outliers as potentially interesting for follow-up studies.

A new ANN based crystal plasticity model for FCC materials and its application to non-monotonic strain paths

Machine learning (ML) methods are commonly used for pattern recognition in almost any field one could imagine. ML techniques can also offer a substantial improvement in computational time when compared to conventional numerical methods. In this research, a machine learning- and crystal plasticity-based framework is presented to predict stress–strain behaviour and texture evolution for a wide variety of materials within the face-centred cubic family (FCC). Firstly, the process of the framework design is described in detail. The proposed framework was designed to be built of ensemble of artificial neural networks (ANN) and a crystal-plasticity based algorithm. Next, the dataset constituent of crystal plasticity simulations was collected. The dataset consisted of examples of monotonic deformation cases, was prepared for training using mathematical transformations, and finally used to train ANNs used in the framework. Then, the ML framework was demonstrated to predict full stress–strain and texture evolution of different FCC single crystals under uniaxial tension, compression, simple shear, equibiaxial tension, tension–compression–tension, compression–tension–compression, and, finally, for some arbitrary non-monotonic loading cases. The proposed framework predicts the stress–strain response and texture evolution with a high degree of accuracy. The results demonstrated in this research show that the proposed machine learning- and crystal plasticity-based framework exhibits a tremendous computational advantage over conventional crystal plasticity model. Finally, one of the most important contributions of this work is to show the framework’s feasibility. The work demonstrates that machine learning methods can help predict complex strain paths without having to train machine learning models on the infinite set of possible non-monotonic loading scenarios.

Bootstrapped Ensemble of Artificial Neural Networks Technique for Quantifying Uncertainty in Prediction of Wind Energy Production

The accurate prediction of wind energy production is crucial for an affordable and reliable power supply to consumers. Prediction models are used as decision-aid tools for electric grid operators to dynamically balance the energy production provided by a pool of diverse sources in the energy mix. However, different sources of uncertainty affect the predictions, providing the decision-makers with non-accurate and possibly misleading information for grid operation. In this regard, this work aims to quantify the possible sources of uncertainty that affect the predictions of wind energy production provided by an ensemble of Artificial Neural Network (ANN) models. The proposed Bootstrap (BS) technique for uncertainty quantification relies on estimating Prediction Intervals (PIs) for a predefined confidence level. The capability of the proposed BS technique is verified, considering a 34 MW wind plant located in Italy. The obtained results show that the BS technique provides a more satisfactory quantification of the uncertainty of wind energy predictions than that of a technique adopted by the wind plant owner and the Mean-Variance Estimation (MVE) technique of literature. The PIs obtained by the BS technique are also analyzed in terms of different weather conditions experienced by the wind plant and time horizons of prediction.

EARN: an ensemble machine learning algorithm to predict driver genes in metastatic breast cancer

BackgroundToday, there are a lot of markers on the prognosis and diagnosis of complex diseases such as primary breast cancer. However, our understanding of the drivers that influence cancer aggression is limited.MethodsIn this work, we study somatic mutation data consists of 450 metastatic breast tumor samples from cBio Cancer Genomics Portal. We use four software tools to extract features from this data. Then, an ensemble classifier (EC) learning algorithm called EARN (Ensemble of Artificial Neural Network, Random Forest, and non-linear Support Vector Machine) is proposed to evaluate plausible driver genes for metastatic breast cancer (MBCA). The decision-making strategy for the proposed ensemble machine is based on the aggregation of the predicted scores obtained from individual learning classifiers to be prioritized homo sapiens genes annotated as protein-coding from NCBI.ResultsThis study is an attempt to focus on the findings in several aspects of MBCA prognosis and diagnosis. First, drivers and passengers predicted by SVM, ANN, RF, and EARN are introduced. Second, biological inferences of predictions are discussed based on gene set enrichment analysis. Third, statistical validation and comparison of all learning methods are performed by some evaluation metrics. Finally, the pathway enrichment analysis (PEA) using ReactomeFIVIz tool (FDR < 0.03) for the top 100 genes predicted by EARN leads us to propose a new gene set panel for MBCA. It includes HDAC3, ABAT, GRIN1, PLCB1, and KPNA2 as well as NCOR1, TBL1XR1, SIRT4, KRAS, CACNA1E, PRKCG, GPS2, SIN3A, ACTB, KDM6B, and PRMT1. Furthermore, we compare results for MBCA to other outputs regarding 983 primary tumor samples of breast invasive carcinoma (BRCA) obtained from the Cancer Genome Atlas (TCGA). The comparison between outputs shows that ROC-AUC reaches 99.24% using EARN for MBCA and 99.79% for BRCA. This statistical result is better than three individual classifiers in each case.ConclusionsThis research using an integrative approach assists precision oncologists to design compact targeted panels that eliminate the need for whole-genome/exome sequencing. The schematic representation of the proposed model is presented as the Graphic abstract.Graphic abstract

Rainfall forecasting in arid regions using an ensemble of artificial neural networks

Water rainfall prediction is one of the most difficult tasks in hydrology because rainfall events are extremely random. This research presents a comparative analysis of different models that predict rainfall in an arid region. The forecasting models comprise the feed-forward, general regression, recurrent, cascade, and Elman neural networks. The performance of the aforementioned models is assessed using three evaluation metrics, namely the correlation coefficient, coefficient of efficiency, and Willmott’s index of agreement. Furthermore, the statistical significance of the neural network models is evaluated using the Wilcoxon-Mann-Whitney test. Finally, the correspondence of the neural network model results compared to the observations is examined using the Taylor diagram. The findings reveal that the general neural network exhibits the best performance compared to other models using the tropical rainfall measuring mission dataset at Suez city in Egypt. The Egyptian water municipality is intended to benefit from the proposed model in monthly rainfall forecasting in this arid region. The precise modeling of rainfall is vital for managing water resources such as food production, water allocation, and drought management.

New hybrid data mining model for prediction of Salmonella presence in agricultural waters based on ensemble feature selection and machine learning algorithms

AbstractThis paper aims to create a new hybrid ensemble data mining model to predict the Salmonella presence in agricultural surface waters based on the combination of heterogeneous ensemble approach for feature selection, clustering, regression, and classification algorithms. The data set for this study was collected from six agricultural ponds in Central Florida consisting of 23 features with 540 instances (26 Salmonella positive and 514 Salmonella negative). The model consisted of three stages. Initially, a heterogeneous ensemble feature selection (HEFS) approach was applied to select top features. Then, the k‐means clustering algorithm was implemented to remove misclassified cases from the data set. Finally, classification and regression algorithms, including support vector machine (SVM), Naïve Bayes (NB), Artificial Neural Network (ANN), Random Forest (RF) with soft voting approach were applied to the preprocessed data set to predict the Salmonella presence in agricultural surface waters with the amount of test set (20%). These algorithms were combined in 10 different ensemble models through the soft voting approach. The performance of these hybrid ensemble models was also evaluated. The ensemble ANN + RF model achieved the highest performance and outperformed all other single and ensemble models based on Area under the ROC Curve (AUC) (0.98) and prediction accuracy (94.9%). The findings emphasize the validity of our hybrid ensemble model which encourages researchers to predict Salmonella presence in agricultural surface waters.

An Application of ANN Ensemble for Estimating of Precipitation Using Regional Climate Models

Climate change scenarios are used for predicting future precipitation. More detailed regional climate change scenarios are being used through dynamic downscale based on global circulation model results. There is a global tendency to utilize simulated precipitation data from downscaled regional climate models (RCMs) suitable for each country. In Korea, there are studies for improving the accuracy of climate change scenario precipitation forecasts compared with observed precipitation. In this study, the precipitation of five regional climate models and actual observed precipitation provided in Korea are applied to ANN (artificial neural network), which suggests ways to improve prediction accuracy for precipitation. The ANN ensemble of RCMs simulates the actual observed precipitation more accurately than the individual RCM. In particular, it is more effective inland than in coastal areas, where precipitation patterns are complex. Pearson correlation coefficient of ANN is high as 0.04 compared with MRA. It is expected that more detailed analysis will be possible if it is applied not only to four cities but also to other regions in Korea. If observed precipitation data are collected in sufficient quantity, the applicability of the ANN model will widen.

A novel approach for evaluating bone mineral density of hips based on Sobel gradient-based map of radiographs utilizing convolutional neural network

Osteoporosis, which is a common disorder associated with low bone mineral density (BMD), is one of the primary reasons for hip fracture. It not only limits mobility, but also makes the patient suffer from pain. Unlike traditional methods, which require both expensive equipment and long scanning times, this study aims to develop a novel technique employing a convolutional neural network (CNN) directly on radiographs of the hips to evaluate BMD. To construct the dataset, X-ray photographs of lower limbs and dual-energy X-ray absorptiometry (DXA) results of the hips of patients were collected. The core of this research is a deep learning-based model that was trained using the pre-processed X-rays images of 510 hips as the input data and the BMD values obtained from DXA as the standard reference. To improve performance quality, the radiographs of the hips were processed with a Sobel algorithm to extract the gradient magnitude maps, and an ensemble artificial neural network which analyses the outputs of CNN models corresponding to three Singh sites and biological parameters was utilized. The superior performance of the proposed method was confirmed by the high correlation coefficient of 0.8075 (p<0.0001) of the BMD measured by DXA in a total of 150 testing cases, with only 0.12 s required for applying the computing configuration to a single X-ray image.

ELSA: An efficient, adaptive Ensemble Learning-based Sampling Approach

The ongoing increase in computational capabilities has been successfully utilized for the engineering task of determining an optimal structural design, allowing for high-fidelity simulations of possible design parameter combinations. However, the more detailed these simulation models are, the more expensive a single performance prediction becomes, making the determination of the optimal design a very time-consuming task. In addition, with uncertainty modeling and propagation as an essential part of the robustness analysis of a design, the computational effort is yet again increased. By replacing the expensive simulation with a fast, capable metamodel, the structural design process can be significantly expedited. The prediction accuracy and, therefore, the usability of such a metamodel is strongly dependent on a high-quality set of data samples representing the relationship of input and output over the entire design space. Acquiring these samples is also an expensive process, therefore, it is important to select each sample in such a way as to maximize the information gain. Given the mostly limited or non-existing prior knowledge about what regions in the design space are of interest, i.e. show a highly nonlinear response, adaptive sampling strategies have been developed, which iteratively select new sample points automatically based on the existing data without further intervention by the user.In this contribution, we present a novel adaptive sampling strategy, which is constituted of a combination of an exploration and an exploitation criterion. The predictions’ variances of an ensemble of Artificial Neural Networks is used to detect regions of interest, whereas a Kernel Density Estimation of the existing samples allows for a detection and filling of those parts of the design space which exhibit a low sample density. The capability of the approach is shown for different test functions as well as an engineering example.

Prediction of Recurrence in Pyogenic Vertebral Osteomyelitis by Artificial Neural Network Using Time-series Data of C-Reactive Protein

Study Design. A retrospective cohort study. Objective. The aim of this study was to develop recurrence-prediction models of pyogenic vertebral osteomyelitis (PVO). Summary of Background Data. Prediction of recurrence in PVO is crucial to avoid additional prolonged antibiotic therapy and aggressive spinal surgery and to reduce mortality. However, prediction of PVO recurrence by previously identified, initial risk factors is limited in PVO patients who exceptionally require prolonged antibiotic therapy and experience various clinical events during the treatment. We hypothesized that time-series analysis of sequential C-reactive protein (CRP) routinely measured to estimate the response to the antibiotics in PVO patients could reflect such long treatment process and increase the power of the recurrence-prediction model. Methods. A retrospective study was performed to develop a PVO recurrence-prediction model, including initial risk factors and time-series data of CRP. Of 704 PVO patients, 493 and 211 were divided into training and test cohorts, respectively. Conventional stepwise logistic regression and artificial neural network (ANN) models were created from the training cohort, and the predictions of recurrence in the test cohort were compared. Results. Prediction models using initial risk factors showed poor sensitivity (4.7%) in both conventional logistic model and ANN models. However, baseline ANN models using time-series CRP data showed remarkably increased sensitivity (55.8%–60.5%). Ensemble ANN model using both initial risk factors and time-series CRP data showed additional benefit in prediction power. Conclusion. The recurrence-prediction models for PVO created only using the initial risk factors showed low sensitivity, regardless of statistical method. However, ANN models using time-series data of CRP values and their ensemble model showed considerably increased prediction power. Therefore, clinicians treating PVO patients should pay attention to the treatment response including changes of CRP levels to identify high-risk patients for recurrence, and further studies to develop recurrence-prediction model for PVO should focus on the treatment response rather than initial risk factors. Level of Evidence: 4

Procjena troškova izgradnje AB i prednapetih betonskih mostova primjenom strojnog učenja

Seven state-of-the-art machine learning techniques for estimation of construction costs of reinforced-concrete and prestressed concrete bridges are investigated in this paper, including artificial neural networks (ANN) and ensembles of ANNs, regression tree ensembles (random forests, boosted and bagged regression trees), support vector regression (SVR) method, and Gaussian process regression (GPR). A database of construction costs and design characteristics for 181 reinforced-concrete and prestressed-concrete bridges is created for model training and evaluation.

Pattern Recognition for Manufacturing Process Variation using Ensembled Artificial Neural Network

In quality control, monitoring and diagnosis of multivariate out of control condition is essential in today’s manufacturing industries. The simplest case involves two correlated variables; for instance, monitoring value of temperature and pressure in our environment. Monitoring refers to the identification of process condition either it is running in control or out of control. Diagnosis refers to the identification of source variables (X1 and X2) for out of control. In this study, an ensemble artificial neural network scheme was investigated in quality control of process in Hard Disk Drive manufacturing. This process was selected since it less reported in the literature. In the related point of view, this study should be useful in monitoring and diagnose the bivariate process pattern in Hard Disk Drive manufacturing process. The result of this study, suggested this scheme has a superior performance compared to the traditional artificial intelligence, namely single isolated Artificial Neural Network. In monitoring, ANN expected to be effective in rapid detection of out of control without false alarm. In diagnosis, this scheme was effective to be applied in identifying various types of process variation such as loading error, offsetting tool, and inconsistent pressure in clamping fixture. Whereby, diagnosis cannot be performed by traditional control chart. This study is useful for quality control practitioner, particularly in manufacturing industry.

Predictive analytics of bridge safety for intelligent transportation system using ensemble model

Bridges are the main links for many transportation systems. Many of the bridges built are subjected to deterioration but are still in use. Factors like heavy weights imposed by vehicles, pressure of water and depreciation effect the lifetime of the bridges. This may lead to various disasters endangering the lives of people. Hence continuous monitoring of bridges using real time bridge data is required to predict bridge safety and alert user passing by the bridge. Existing bridge safety monitoring systems are either Sensor based or uses specialized equipment to monitor safety using frequencies of vibration. Such systems are expensive and have low performance. Traditional prediction algorithms from data mining and regression modeling can help in correlation between input parameters and bridge condition using historical data. They have difficulty either in scaling of data or may not adapt to dynamic changes in input data. Mobile app based system that uses ensemble of both Artificial Neural Network (ANN) and Gaussian Process Regression (GPR) is proposed in this paper to predict bridge safety. This paper signifies on the structural health monitoring of the bridge and the potential advantage of integrating Artificial Intelligence based predictive analytics into IoT sensors to assess the bridge safety. Three different experiments based on ANN, GPR and ensemble of ANN,GPR are made. In the first step, ANN is trained and validated on two datasets with different scenario (Indian and U.S.A). Error reported from this best trained network is used by GPR for statistical analysis of the error distributions. This combined system based mobile app is evaluated using NRMSE, COD and cross entropy loss measures. Test results proved that, the third hyper-parameter values of ensemble model yielded a good result with less error rate than ANN and GPR predictive models.

Ensemble of artificial neural networks and K-nearest neighbour for classification of granite images

This study attempted to develop a granite quality classification model by comparing colour, texture, and ensemble of colour and texture. An average of 120 images was taken for each verities of grade A, grade B, grade C. A greyscale coexistence matrix was used for texture extraction and a colour histogram for colour extraction. Five textures and six colour features were extracted from each granite image for classification. To build the classification KNN, ANN and ensemble of KNN and ANN are examined. Based on the experiment ensemble of ANN and KNN model outperform good result by the combined texture and colour features using sequential forward feature selection (SFFS) methods. An average accuracy of 85.3%, 93.6%, and 95.8% is achieved for KNN, ANN and ensemble of KNN and ANN respectively. Granite fractures and vines of the images have a strong impact on the performance of the classifier.

Neural-Based Ensembles for Particulate Matter Forecasting

The air pollution caused by particulate matter (PM) has become a public health issue due to the risks to human life and the environment. The PM concentration in the air causes haze and affects the lungs and the heart, leading to reduced visibility, allergic reactions, pneumonia, asthma, cardiopulmonary diseases, lung cancer, and even death. In this context, the development of systems for monitoring, forecasting, and controlling emissions plays an important role. The literature about forecasting systems based on Artificial Neural Networks (ANNs) ensembles has been highlighted regarding statistical accuracy and efficiency. In this article, trainable and non-trainable combination methods are used for PM10 and PM2.5 (particles with an aerodynamic diameter less than 10 and 2.5 micrometers, respectively) time series forecasting for eight different locations, in Finland and Brazil, for different periods. Trainable ensembles based on ANNs, linear regression, and Copulas are compared with non-trainable combinations (mean and median), single ANNs, and linear statistical approaches. Different models are considered so far, including Autoregressive model (AR), Autoregressive and Moving Average Model (ARMA), Infinite Impulse Response Filters (IIR), Multilayer Perceptron (MLP), Radial Basis Function Networks (RBF), Extreme Learning Machines (ELM), Echo State Networks (ESN), and Adaptive Network Fuzzy Inference System (ANFIS). The use of ANNs ensembles, mainly combined with MLP, leads to a better one step ahead forecasting performance. The use of robust air pollution forecasting tools is prime to assist governments in managing air pollution issues like hospital collapse during adverse air quality situations. In this sense, our study is indirectly related to the following United Nations sustainable development goals: SDG 3 - good health and well-being and SDG 11 - sustainable cities and communities.

On the Value of River Network Information in Regional Frequency Analysis

AbstractRegional frequency analysis (RFA) is widely used in the design of hydraulic structures at locations where streamflow records are not available. RFA estimates depend on the precise delineation of homogenous regions for accurate information transfer. This study proposes new physiographical variables based on river network features and tests their potential to improve the accuracy of hydrological feature estimates. Information about river network types is used both in the definition of homogenous regions and in the estimation process. Data from 105 river basins in arid and semiarid regions of the United States were used in our analysis. Artificial neural network ensemble models and canonical correlation analysis were used to produce flood quantile estimates, which were validated through tenfold cross and jackknife validations. We conducted analysis for model performance based on statistical indices, such as the Nash–Sutcliffe efficiency, root-mean-square error, relative root-mean-square error, mean absolute error, and relative mean bias. Among various combinations of variables, a model with 10 variables produced the best performance. Further, 49, 36, and 20 river networks in the 105 basins were classified as dendritic, pinnate, and trellis networks, respectively. The model with river network classification for the homogenous regions appeared to provide a superior performance compared with a model without such classification. The results indicated that including our proposed combination of variables could improve the accuracy of RFA flood estimates with the classification of the network types. This finding has considerable implications for hydraulic structure design.

NowDeepN: An Ensemble of Deep Learning Models for Weather Nowcasting Based on Radar Products’ Values Prediction

One of the hottest topics in today’s meteorological research is Weather nowcasting, which is the weather forecast for a short time period such as one to six hours. Radar is an important data source used by operational meteorologists for issuing nowcasting warnings. With the main goal of helping meteorologists in analysing radar data for issuing nowcasting warnings, we propose NowDeepN, a supervised learning based regression model which uses an ensemble of deep artificial neural networks for predicting the values for radar products at a certain time moment. The values predicted by NowDeepN may be used by meteorologists in estimating the future development of potential severe phenomena and would replace the time consuming process of extrapolating the radar echoes. NowDeepN is intended to be a proof of concept for the effectiveness of learning from radar data relevant patterns that would be useful for predicting future values for radar products based on their historical values. For assessing the performance of NowDeepN, a set of experiments on real radar data provided by the Romanian National Meteorological Administration is conducted. The impact of a data cleaning step introduced for correcting the erroneous radar products’ values is investigated both from the computational and meteorological perspectives. The experimental results also indicate the relevance of the features considered in the supervised learning task, highlighting that the radar products’ values at a certain geographical location at a time moment may be predicted from the products’ values from a neighboring area of that location at previous time moments. An overall Normalized Root Mean Squared Error less than 4% was obtained for NowDeepN on the cleaned radar data. Compared to similar related work from the nowcasting literature, NowDeepN outperforms several approaches and this emphasizes the performance of our proposal.

Short term power dispatch using neural network based ensemble classifier

This work reports the application of ensemble artificial neural networks, a machine learning technique, in the solution of short term optimal power generation on the IEEE 30-bus test system. The study, carried out using MATLAB, is being reported. The motive for using ensemble artificial neural networks has been to take advantage of multiple parallel processors computing rather than the traditional serial computation. The Bootstraps are obtained through small bags in the Bagging algorithm and are combined by averaging. Employing ensemble neural networks reduces bias and variance in machine learning so that the overall error is reduced, and hence better prediction can be achieved. The results obtained from the proposed algorithm are compared with those from the other heuristic and conventional techniques to validate the effectiveness of the proposed methodology.