Neural Network Ensemble Approach Research Articles

PREDICTION INDONESIA COMPOSITE INDEX USING INTEGRATION DECOMPOSITION- NEURAL NETWORK ENSEMBLE DURING VUCA ERA

The Volatility, Uncertainty, Complexity, and Ambiguity (VUCA) era causes turmoil in the capital markets, stocks, commodities, etc. The impact is a decline in the Composite Stock Price Index (IHSG) in 2020. Therefore, future data is needed to inform investors and business people when making portfolio decisions. This paper develops a decomposition and Neural Network (NN) integration model to predict ICI during the VUCA era. The results are presented empirically to show the model's effectiveness in reducing prediction errors. First, the actual data is converted into three components; second, with the Neural Network Ensemble (NNE) approach where the initial step of decomposition results is trained using artificial NN with architecture, training data, and topology to produce individual networks; The output is selected using Principal Component Analysis (PCA) and becomes input to the ensemble model, then combined using a simple average and weighted average. The empirical results from ICI predictions illustrate: (1) decomposition has the potential to overcome data that is characterized by high volatility; (2) NNE is able to reduce errors (MSE≤0.100e-4, MAE≤0.01) compared to individual networks (MSE=0.0024 MAE=0.0376); (3) ensemble combinations using weighted averages (MSE≤3.00e-5,MAE≤0.002) are superior to simple averages (MSE≤5.00e-5,MAE≤0.01); (4) the integration carried out shows effectiveness in predicting ICI and provides better prediction results.

An ensemble neural network approach for space–time landslide predictive modelling

There is an urgent need for accurate and effective Landslide Early Warning Systems (LEWS). Most LEWS are currently based on temporally-aggregated measures of rainfall derived from either in-situ measurements or satellite-based rainfall estimates. Relying on a summary metric of precipitation may not capture the complexity of the rainfall signal and its dynamics in space and time in triggering landslides. Here, we present a proof-of-concept for constructing a LEWS based on an integrated spatio-temporal modelling framework. Our proposed methodology builds upon a recent approach that uses a daily rainfall time series instead of the traditional scalar aggregation. Specifically, we partition the study area into slope units and use a Gated Recurrent Unit (GRU) to process satellite-derived rainfall time series and combine the output features with a second neural network (NN) tasked with capturing the effect of terrain characteristics. To assess if our approach enhances accuracy, we applied it in Vietnam and benchmarked it against a modelling counterpart where we replaced the rainfall time series with the corresponding scalar representative of the cumulated precipitation. The corresponding duration was set at 14 days as it proved to produce the best performance. Our results show that our protocol leads to better performance in hindcasting landslides when making use of the rainfall as a continuous signal over time. While not tested here, our approach can be extended to rainfall obtained from weather forecasts, potentially leading to actual landslide forecasts.

Cloud detection from multi-angular polarimetric satellite measurements using a neural network ensemble approach

Abstract. This paper describes a neural network cloud masking scheme from PARASOL (Polarization and Anisotropy of Reflectances for Atmospheric Science coupled with Observations from a Lidar) multi-angle polarimetric measurements. The algorithm has been trained on synthetic measurements and has been applied to the processing of 1 year of PARASOL data. Comparisons of the retrieved cloud fraction with MODIS (Moderate Resolution Imaging Spectroradiometer) products show overall agreement in spatial and temporal patterns, but the PARASOL neural network (PARASOL-NN) retrieves lower cloud fractions. Comparisons with a goodness-of-fit mask from aerosol retrievals suggest that the NN cloud mask flags fewer clear pixels as cloudy than MODIS (∼ 3 % of the clear pixels versus ∼ 15 % by MODIS). On the other hand the NN classifies more pixels incorrectly as clear than MODIS (∼ 20 % by NN, versus ∼ 15 % by MODIS). Additionally, the NN and MODIS cloud mask have been applied to the aerosol retrievals from PARASOL using the Remote Sensing of Trace Gas and Aerosol Products (RemoTAP) algorithm. Validation with AERONET shows that the NN cloud mask performs comparably with MODIS in screening residual cloud contamination in retrieved aerosol properties. Our study demonstrates that cloud masking from multi-angle polarimeter (MAP) aerosol retrievals can be performed based on the MAP measurements themselves, making the retrievals independent of the availability of a cloud imager.

An Ensemble of Long Short-Term Memory Networks with an Attention Mechanism for Upper Limb Electromyography Signal Classification

Advancing cutting-edge techniques to accurately classify electromyography (EMG) signals are of paramount importance given their extensive implications and uses. While recent studies in the literature present promising findings, a significant potential still exists for substantial enhancement. Motivated by this need, our current paper introduces a novel ensemble neural network approach for time series classification, specifically focusing on the classification of upper limb EMG signals. Our proposed technique integrates long short-term memory networks (LSTM) and attention mechanisms, leveraging their capabilities to achieve accurate classification. We provide a thorough explanation of the architecture and methodology, considering the unique characteristics and challenges posed by EMG signals. Furthermore, we outline the preprocessing steps employed to transform raw EMG signals into a suitable format for classification. To evaluate the effectiveness of our proposed technique, we compare its performance with a baseline LSTM classifier. The obtained numerical results demonstrate the superiority of our method. Remarkably, the method we propose attains an average accuracy of 91.5%, with all motion classifications surpassing the 90% threshold.

M2CE: Multi‐convolutional neural network ensemble approach for improved multiclass classification of skin lesion

AbstractDue to inter‐class homogeneity and intra‐class variability, the classification of skin lesions in dermoscopy images has remained difficult. Although deep convolutional neural networks (DCNNs) have achieved satisfactory performance for binary skin cancer classification, multiclass skin lesion classification is still an open problem due to the limited training samples and class imbalance issues. To tackle these issues, in this article, we propose a multi‐CNN ensemble approach dubbed for multiclass skin lesion classification. The includes three individual CNN models, each helping in extracting different high‐level features from skin images and thereby yielding different prediction results. First, we design a lightweight CNN model to extract prominent features and train it from scratch, which primarily aims at avoiding the data scarcity problem. Then, we ensemble two different pre‐trained CNN models with the lightweight model to improve the performance and generalization capability. The proposed ensemble approach can effectively fuse the predictions of each individual CNN model using the averaging method. The approach is validated using a benchmark data set, HAM10000, which contains skin lesion images of seven different classes. The results demonstrate that the outperforms base CNN models and state‐of‐the‐art approaches without using any external data.

A trained neural network model accurately predicts Achilles tendon stress during walking and running based on shear wave propagation

Shear wave tensiometry is a noninvasive technique for measuring tendon loading during activity based on the speed of a shear wave traveling along the tendon. Shear wave speed has been shown to modulate with axial stress, but calibration is required to obtain absolute measures of tendon loading. However, the current technique only makes use of wave speed, whereas other characteristics of the wave (e.g., amplitude, frequency content) may also vary with tendon loading. It is possible that these data could be used in addition to wave speed to circumvent the need for calibration. Given the potential complex relationships to tendon loading, and the lack of an analytical model to guide the use of these data, it is sensible to use a machine learning approach. Here, we used an ensemble neural network approach to predict inverse dynamics estimates of Achilles tendon stress from shear wave tensiometry data collected in a prior study. Neural network-predicted stresses were highly correlated with stance phase inverse dynamics estimates for walking (R2 = 0.89 ± 0.06) and running (R2 = 0.87 ± 0.11) data reserved for neural network model testing and not included in model training. Additionally, error between neural network-predicted and inverse dynamics-estimated stress was reasonable (walking: RMSD = 11 ± 2% of peak load; running: 25 ± 14%). Results of this pilot analysis suggest that a machine learning approach could reduce the reliance of shear wave tensiometry on calibration and expand its usability in many settings.

An ensemble neural network approach to forecast Dengue outbreak based on climatic condition

Dengue fever is a virulent disease spreading over 100 tropical and subtropical countries in Africa, the Americas, and Asia. This arboviral disease affects around 400 million people globally, severely distressing the healthcare systems. The unavailability of a specific drug and ready-to-use vaccine makes the situation worse. Hence, policymakers must rely on early warning systems to control intervention-related decisions. Forecasts routinely provide critical information for dangerous epidemic events. However, the available forecasting models (e.g., weather-driven mechanistic, statistical time series, and machine learning models) lack a clear understanding of different components to improve prediction accuracy and often provide unstable and unreliable forecasts. This study proposes an ensemble wavelet neural network with exogenous factor(s) (XEWNet) model that can produce reliable estimates for dengue outbreak prediction for three geographical regions, namely San Juan, Iquitos, and Ahmedabad. The proposed XEWNet model is flexible and can easily incorporate exogenous climate variable(s) confirmed by statistical causality tests in its scalable framework. The proposed model is an integrated approach that uses wavelet transformation into an ensemble neural network framework that helps in generating more reliable long-term forecasts. The proposed XEWNet allows complex non-linear relationships between the dengue incidence cases and rainfall; however, mathematically interpretable, fast in execution, and easily comprehensible. The proposal’s competitiveness is measured using computational experiments based on various statistical metrics and several statistical comparison tests. In comparison with statistical, machine learning, and deep learning methods, our proposed XEWNet performs better in 75% of the cases for short-term and long-term forecasting of dengue incidence.

Fuzzified deep neural network ensemble approach for estimating cycle time range

Because of the high uncertainty associated with predicting the cycle time of a job in a complex manufacturing system, this task is a challenge for production planners. To replace an inaccurate cycle time forecast, the range of cycle time is estimated. To estimate the cycle time range of a job as precisely as possible, a fuzzified deep neural network (FDNN) ensemble approach is proposed in this paper. This approach involves the following steps. First, a deep neural network (DNN) is constructed to predict the cycle time of a job. The parameters of the DNN are then fuzzified to generate a fuzzy cycle time forecast that contains the actual cycle time. In contrast to existing methods that involve fuzzifying multiple parameters simultaneously, which is computationally intensive, the proposed methodology involves fuzzifying parameters independently to facilitate problem-solving. Subsequently, the cycle time ranges estimated by fuzzifying various parameters are aggregated through fuzzy intersection. The proposed FDNN ensemble approach was applied to a real case to validate its effectiveness. The experimental results indicated that the precision of cycle time range estimation was up to 38% higher with the FDNN ensemble approach than with the fuzzy backpropagation network approach.

CRBPDL: Identification of circRNA-RBP interaction sites using an ensemble neural network approach

Circular RNAs (circRNAs) are non-coding RNAs with a special circular structure produced formed by the reverse splicing mechanism. Increasing evidence shows that circular RNAs can directly bind to RNA-binding proteins (RBP) and play an important role in a variety of biological activities. The interactions between circRNAs and RBPs are key to comprehending the mechanism of posttranscriptional regulation. Accurately identifying binding sites is very useful for analyzing interactions. In past research, some predictors on the basis of machine learning (ML) have been presented, but prediction accuracy still needs to be ameliorated. Therefore, we present a novel calculation model, CRBPDL, which uses an Adaboost integrated deep hierarchical network to identify the binding sites of circular RNA-RBP. CRBPDL combines five different feature encoding schemes to encode the original RNA sequence, uses deep multiscale residual networks (MSRN) and bidirectional gating recurrent units (BiGRUs) to effectively learn high-level feature representations, it is sufficient to extract local and global context information at the same time. Additionally, a self-attention mechanism is employed to train the robustness of the CRBPDL. Ultimately, the Adaboost algorithm is applied to integrate deep learning (DL) model to improve prediction performance and reliability of the model. To verify the usefulness of CRBPDL, we compared the efficiency with state-of-the-art methods on 37 circular RNA data sets and 31 linear RNA data sets. Moreover, results display that CRBPDL is capable of performing universal, reliable, and robust. The code and data sets are obtainable at https://github.com/nmt315320/CRBPDL.git.

A neural network ensemble approach for GDP forecasting

We propose an ensemble learning methodology to forecast the future US GDP growth release. Our approach combines a Recurrent Neural Network (RNN) and a Dynamic Factor model accounting for time-variation in the mean with a Generalized Autoregressive Score (DFM-GAS). We show how our approach improves forecasts in the aftermath of the 2008-09 global financial crisis by reducing the forecast error for the one-quarter horizon. An exercise on the COVID-19 recession shows a good performance during the economic rebound. Eventually, we provide an interpretable machine learning routine based on integrated gradients to evaluate how the features of the model reflect the evolution of the business cycle.

Deep Ensemble Neural Network Approach for Federal Highway Administration Axle-Based Vehicle Classification Using Advanced Single Inductive Loops

The Federal Highway Administration (FHWA) vehicle classification scheme is designed to serve various transportation needs such as pavement design, emission estimation, and transportation planning. Many transportation agencies rely on Weigh-In-Motion and Automatic Vehicle Classification sites to collect these essential vehicle classification counts. However, the spatial coverage of these detection sites across the highway network is limited by high installation and maintenance costs. One cost-effective approach has been the use of single inductive loop sensors as an alternative to obtaining FHWA vehicle classification data. However, most data sets used to develop such models are skewed since many classes associated with larger truck configurations are less commonly observed in the roadway network. This makes it more difficult to accurately classify under-represented classes, even though many of these minority classes may have disproportionately adverse effects on pavement infrastructure and the environment. Therefore, previous models have been unable to adequately classify under-represented classes, and the overall performance of the models is often masked by excellent classification accuracy of majority classes, such as passenger vehicles and five-axle tractor-trailers. To resolve the challenge of imbalanced data sets in the FHWA vehicle classification, this paper constructed a bootstrap aggregating deep neural network model on a truck-focused data set using single inductive loop signatures. The proposed method significantly improved the model performance on several truck classes, especially minority classes such as Classes 7 and 11 which were overlooked in previous research. The model was tested on a distinct data set obtained from four spatially independent sites and achieved an accuracy of 0.87 and an average F1 score of 0.72.

High‐performance anomaly intrusion detection system with ensemble neural networks on reconfigurable hardware

SummaryIn this article, we propose an ensemble neural network approach so that multiple neuron network models can be deployed concurrently for detecting anomalies in a system. Additionally, a high‐performance architecture integrating the neural networks for reconfigurable hardware is also designed to exploit the parallelism of the technology. We carefully implement four different models for detecting SYN, DNS, UNP, and ICMP attacks with the same configuration of neural networks on the NetFPGA platform. The neural networks can be executed in the pipeline to improve performance. Experiments with the generated dataset are conducted to test both performance in terms of throughput and accuracy. Our experimental results show that the system can process packets with throughput up to 30.48Gbps and is able to recognize 99.98% attacking packets with only 0.98% false alarms.

A Neural Network Ensemble Approach for GDP Forecasting

We propose an ensemble learning methodology to forecast the future US GDP growth release. Our approach combines a Recurrent Neural Network (RNN) with a Dynamic Factor model accounting for time-variation in the mean with a Generalized Autoregressive Score (DFM-GAS). We show how this combination improves forecasts in the aftermath of the 2008-09 global financial crisis, as well as in the latest COVID-19 recession, by reducing the root mean squared error for the short-term forecast horizon. Thus, we evaluate the predictive ability of each component of the ensemble by considering variations in the mean, as the latter are potentially caused by recessions affecting the economy. For our scope, we employ a set of predictors encompassing a wide range of variables measured at different time frequencies. Thus, we provide dynamic coefficients for predictors after an interpretable machine learning routine to assess how the model reflects the evolution of the business cycle.

A neural network ensemble approach with jittered basin characteristics for regionalized low flow frequency analysis

Previous studies have shown that the predictive ability of artificial neural networks can be improved by combining multiple models. This study presents a jittering-based neural network ensemble approach to obtain improved regional low flow estimates at ungauged sites. The fundamental idea of the jittered ensemble is to add noises to the explanatory variables and thereby augments the training data sets to develop the network models based on different but associated training data. To evaluate the performance of the jittered ensemble, this study employs two different neural network architectures, a single-output neural network model and a multi-output neural network model. In addition, a jittered ensemble approach is coupled with the variable importance measuring algorithm to infer the relationship between basin characteristics and predicted low flow quantiles. Effectiveness of the proposed methods is demonstrated using selected basins in the northeastern United States. Results suggest that the jittering-based ensemble model is able to consistently outperform a single modeling approach. Also, improvement is achieved even using small sizes of ensembles although a sufficient sample size offers more reliable predictions. Finally, this study recognizes that the effects of the basin characteristics varied or remained constant based on the low flow quantiles considered.

Ensemble neural network approach detecting pain intensity from facial expressions.

This paper reports on research to design an ensemble deep learning framework that integrates fine-tuned, three-stream hybrid deep neural network (i.e., Ensemble Deep Learning Model, EDLM), employing Convolutional Neural Network (CNN) to extract facial image features, detect and accurately classify the pain. To develop the approach, the VGGFace is fine-tuned and integrated with Principal Component Analysis and employed to extract features in images from the Multimodal Intensity Pain database at the early phase of the model fusion. Subsequently, a late fusion, three layers hybrid CNN and recurrent neural network algorithm is developed with their outputs merged to produce image-classified features to classify pain levels. The EDLM model is then benchmarked by means of a single-stream deep learning model including several competing models based on deep learning methods. The results obtained indicate that the proposed framework is able to outperform the competing methods, applied in a multi-level pain detection database to produce a feature classification accuracy that exceeds 89 %, with a receiver operating characteristic of 93 %. To evaluate the generalization of the proposed EDLM model, the UNBC-McMaster Shoulder Pain dataset is used as a test dataset for all of the modelling experiments, which reveals the efficacy of the proposed method for pain classification from facial images. The study concludes that the proposed EDLM model can accurately classify pain and generate multi-class pain levels for potential applications in the medical informatics area, and should therefore, be explored further in expert systems for detecting and classifying the pain intensity of patients, and automatically evaluating the patients' pain level accurately.

Implementation of data intelligence models coupled with ensemble machine learning for prediction of water quality index.

In recent decades, various conventional techniques have been formulated around the world to evaluate the overall water quality (WQ) at particular locations. In the present study, back propagation neural network (BPNN) and adaptive neuro-fuzzy inference system (ANFIS), support vector regression (SVR), and one multilinear regression (MLR) are considered for the prediction of water quality index (WQI) at three stations, namely Nizamuddin, Palla, and Udi (Chambal), across the Yamuna River, India. The nonlinear ensemble technique was proposed using the neural network ensemble (NNE) approach to improve the performance accuracy of the single models. The observed WQ parameters were provided by the Central Pollution Control Board (CPCB) including dissolved oxygen (DO), pH, biological oxygen demand (BOD), ammonia (NH3), temperature (T), and WQI. The performance of the models was evaluated by various statistical indices. The obtained results indicated the feasibility of the developed data intelligence models for predicting the WQI at the three stations with the superior modelling results of the NNE. The results also showed that the minimum values for root mean square(RMS) varied between 0.1213 and 0.4107, 0.003 and 0.0367, and 0.002 and 0.0272 for Nizamuddin, Palla, and Udi (Chambal), respectively. ANFIS-M3, BPNN-M4, and BPNN-M3 improved the performance with regard to an absolute error by 41%, 4%, and 3%, over other models for Nizamuddin, Palla, and Udi (Chambal) stations, respectively. The predictive comparison demonstrated that NNE proved to be effective and can therefore serve as a reliable prediction approach. The inferences of this paper would be of interest to policymakers in terms of WQ for establishing sustainable management strategies of water resources.

PASSION: an ensemble neural network approach for identifying the binding sites of RBPs on circRNAs.

Different from traditional linear RNAs (containing 5' and 3' ends), circular RNAs (circRNAs) are a special type of RNAs that have a closed ring structure. Accumulating evidence has indicated that circRNAs can directly bind proteins and participate in a myriad of different biological processes. For identifying the interaction of circRNAs with 37 different types of circRNA-binding proteins (RBPs), we develop an ensemble neural network, termed PASSION, which is based on the concatenated artificial neural network (ANN) and hybrid deep neural network frameworks. Specifically, the input of the ANN is the optimal feature subset for each RBP, which has been selected from six types of feature encoding schemes through incremental feature selection and application of the XGBoost algorithm. In turn, the input of the hybrid deep neural network is a stacked codon-based scheme. Benchmarking experiments indicate that the ensemble neural network reaches the average best area under the curve (AUC) of 0.883 across the 37 circRNA datasets when compared with XGBoost, k-nearest neighbor, support vector machine, random forest, logistic regression and Naive Bayes. Moreover, each of the 37 RBP models is extensively tested by performing independent tests, with the varying sequence similarity thresholds of 0.8, 0.7, 0.6 and 0.5, respectively. The corresponding average AUC obtained are 0.883, 0.876, 0.868 and 0.883, respectively, highlighting the effectiveness and robustness of PASSION. Extensive benchmarking experiments demonstrate that PASSION achieves a competitive performance for identifying binding sites between circRNA and RBPs, when compared with several state-of-the-art methods. A user-friendly web server of PASSION is publicly accessible at http://flagship.erc.monash.edu/PASSION/. Supplementary data are available at Bioinformatics online.

Neural Network Ensemble Approach to Pushed Convoys Dispatching Problems

Abstract This paper investigates the use of neural networks (NNs) for the problem of assigning push boats to barge convoys in inland waterway transportation (IWT). Push boat–barge convoy assignmentsare part of the daily decision-making process done by dispatchers in IWT companiesforwhich a decision support tool does not exist. The aim of this paper is to develop a Neural Network Ensemble (NNE) model that will be able to assist in push boat–barge convoy assignments based on the push boat power.The primary objective of this paper is to derive an NNE model for calculation of push boat Shaft Powers (SHPs) by using less than 100% of the experimental data available. The NNE model is applied to a real-world case of more than one shipping company from the Republic of Serbia, which is encountered on the Danube River. The solution obtained from the NNE model is compared toreal-world full-scale speed/power measurements carried out on Serbian push boats, as well as with the results obtained from the previous NNE model. It is found that the model is highly accurate, with scope for further improvements.

Real-time Transportation Prediction Correction using Reconstruction Error in Deep Learning

In online complex systems such as transportation system, an important work is real-time traffic prediction. Due to the data shift, data model inconsistency, and sudden change of traffic patterns (like transportation accident), the prediction result derived from an offline-built model would be unreliable. Retraining the model is usually not time affordable for online prediction, especially when the prediction model is very complex and costs a lot of training time (for example, deep neural networks). A real-time prediction correction strategy would be of great value under this situation. Traditionally, the prediction correction usually relies on the prediction error in several previous time intervals. They assume that the error pattern is similar in the current time interval, so that it is time-delayed to some extent. In this article, we propose the prediction correction strategy using the reconstruction error in the deep neural network. The reconstruction error can reflect the model’s ability on feature representation and then determine the fitness of an input data to the model. We first build the relationship between reconstruction error and prediction error. From the perspective of the prediction interval, we demonstrate that the reconstruction error is in positive relation with the prediction interval. Thus the prediction result is more reliable when the reconstruction error is smaller. Then we propose two mechanisms of real-time prediction correction using the reconstruction error. The data driven prediction correction approach selects several training instances with similar reconstruction errors to the current instance and using their average prediction error in correcting the prediction result. The model-driven approach builds several component deep neural networks in training. The component training set for each network is selected according to the reconstruction error of training instances. For a predicting instance, it first computes the reconstruction error of the sample in each component network and then averages the results by the reconstruction error and prediction interval. The model-driven approach is actually a reconstruction error-based deep neural network ensemble approach. Finally, a series of experiments demonstrated that reconstruction error based prediction correction approaches are effective in several prediction problems in transportation including traffic flow prediction on road, traffic flow prediction in entrance and exit station and travel time prediction. Besides the high overall accuracy, our approach can also provide many observations of using the reconstruction error in transportation prediction.

Neural Network Ensembles for Sensor-Based Human Activity Recognition Within Smart Environments.

In this paper, we focus on data-driven approaches to human activity recognition (HAR). Data-driven approaches rely on good quality data during training, however, a shortage of high quality, large-scale, and accurately annotated HAR datasets exists for recognizing activities of daily living (ADLs) within smart environments. The contributions of this paper involve improving the quality of an openly available HAR dataset for the purpose of data-driven HAR and proposing a new ensemble of neural networks as a data-driven HAR classifier. Specifically, we propose a homogeneous ensemble neural network approach for the purpose of recognizing activities of daily living within a smart home setting. Four base models were generated and integrated using a support function fusion method which involved computing an output decision score for each base classifier. The contribution of this work also involved exploring several approaches to resolving conflicts between the base models. Experimental results demonstrated that distributing data at a class level greatly reduces the number of conflicts that occur between the base models, leading to an increased performance prior to the application of conflict resolution techniques. Overall, the best HAR performance of 80.39% was achieved through distributing data at a class level in conjunction with a conflict resolution approach, which involved calculating the difference between the highest and second highest predictions per conflicting model and awarding the final decision to the model with the highest differential value.