Failure Mode Detection of Reinforced Concrete Shear Walls Using Ensemble Deep Neural Networks

Accurate prediction of ore grade is essential for many basic mine operations, including mine planning and design, pit optimization, and ore grade control. Preference is given to the neural network over other interpolation techniques for ore grade estimation because of its ability to learn any linear or non-linear relationship between inputs and outputs. In many cases, ensembles of neural networks have been shown, both theoretically and empirically, to outperform a single network. The performance of an ensemble model largely depends on the accuracy and diversity of member networks. In this study, techniques of a genetic algorithm (GA) and k-means clustering are used for the ensemble neural network modeling of a lead–zinc deposit. Two types of ensemble neural network modeling are investigated, a resampling-based neural ensemble and a parameter-based neural ensemble. The k-means clustering is used for selecting diversified ensemble members. The GA is used for improving accuracy by calculating ensemble weights. Results are compared with average ensemble, weighted ensemble, best individual networks, and ordinary kriging models. It is observed that the developed method works fairly well for predicting zinc grades, but shows no significant improvement in predicting lead grades. It is also observed that, while a resampling-based neural ensemble model performs better than the parameter-based neural ensemble model for predicting lead grades, the parameter-based ensemble model performs better for predicting zinc grades.

In this study, a novel Neural Network (NN) ensemble model using Projection Pursuit Regression (PPR) and Least Squares Support Vector Regression (LS–SVR) is developed for financial forecasting. In the process of ensemble modeling, the first stage some important economic factors are selected by the PPR technology as input feature for NN. In the second stage, the initial data set is divided into different training sets by used Bagging and Boosting technology. In the third stage, these training sets are input to the different individual NN models, and then various single NN predictors are produced based on diversity principle. In the fourth stage, the Partial Least Square (PLS) technology is used to choosing the appropriate number of neural network ensemble members. In the final stage, LS–SVR is used for ensemble of the NN to prediction purpose. For testing purposes, this study compare the new ensemble model’s performance with some existing neural network ensemble approaches in terms of the Shanghai Stock Exchange index. Experimental results reveal that the predictions using the proposed approach are consistently better than those obtained using the other methods presented in this study in terms of the same measurements.KeywordsNeural NetworkProjection Pursuit RegressionLeast Squares Support Vector RegressionStock Market Forecasting

This paper investigates the use of neural networks (NNs) for the problem of assigning push boats to barge convoys in inland waterway transportation (IWT). Push boat–barge convoy assignmentsare part of the daily decision-making process done by dispatchers in IWT companiesforwhich a decision support tool does not exist. The aim of this paper is to develop a Neural Network Ensemble (NNE) model that will be able to assist in push boat–barge convoy assignments based on the push boat power.The primary objective of this paper is to derive an NNE model for calculation of push boat Shaft Powers (SHPs) by using less than 100% of the experimental data available. The NNE model is applied to a real-world case of more than one shipping company from the Republic of Serbia, which is encountered on the Danube River. The solution obtained from the NNE model is compared toreal-world full-scale speed/power measurements carried out on Serbian push boats, as well as with the results obtained from the previous NNE model. It is found that the model is highly accurate, with scope for further improvements.

Multi-model ensemble is an important fundamental technique of practical value for many artificial intelligence applications. However, the usage for multi-model ensemble has been limited when it is combined with deep neural networks to construct ensemble of deep neural networks. Due to the big time and computing resources required to train and to integrate multiple deep neural networks for the achievement of multi-model ensemble, the engineering application field where developing time and computing resources are usually restricted, has not yet widespreadly benefited from ensemble of deep neural networks. To alleviate this situation, we present a new multi-model ensemble approach entitled feature transferring based multi-model ensemble (FTBME), for ensemble of deep neural networks. Primarily, we propose a feature transferring based multi-model training strategy to more affordably find multiple extra models based on a given previously optimized deep neural network model. Sequentially, to develop better ensemble solutions, we design a more effective random greedy based ensemble selection strategy to filter out models non-positive to ensemble generalization. Finally, inspired by the idea of averaging parameter points, we propose to fuse the obtained models in weight space which eventually reduces the expense of ensemble at the testing stage to a single deep neural network model while retaining the generalization. These three advances constitute the resulting technique FTBME. We conducted extensive experiments using deep neural networks, from light weight to complex, on ImageNet, CIFAR-10 and CIFAR-100. Results show that, given a deep neural network model which has been well-optimized and reaching its limit, FTBME can obtain better generalization with minor extra training expense while maintaining the expense to a single model at ensemble testing. This promising property of FTBME make us believe that it could be leveraged to broaden the usage for ensemble of deep neural networks, alleviating the situation that the engineering application field has not yet widespreadly benefited from ensemble of deep neural networks.

In this paper, a novel nonlinear Radial Basis Function Neural Network (RBF-NN) ensemble model based on ν-Support Vector Machine (SVM) regression is presented for financial time series forecasting. In the process of ensemble modeling, the first stage the initial data set is divided into different training sets by used Bagging and Boosting technology. In the second stage, these training sets are input to the different individual RBF-NN models, and then various single RBF-NN predictors are produced based on diversity principle. In the third stage, the Partial Least Square (PLS) technology is used to choosing the appropriate number of neural network ensemble members. In the final stage, ν-Support Vector Machine (SVM) regression is used for ensemble of the RBF-NN to prediction purpose. For testing purposes, this paper compare the new ensemble model's performance with some existing neural network ensemble approaches in terms of two financial time series: S & P 500 and Nikkei 225. Experimental results reveal that the predictions using the proposed approach are consistently better than those obtained using the other methods presented in this study in terms of the same measurements. Those results show that the proposed nonlinear ensemble technique provides a promising alternative to financial time series prediction.

In this paper, a novel hybrid Radial Basis Function Neural Network (RBF-NN) ensemble model using Wavelet Support Vector Machine Regression (W-SVR) is developed for rainfall forecasting. In the process of ensemble modeling, the first stage the initial data set is divided into different training sets by used Bagging and Boosting technology. In the second stage, these training sets are input to the different individual RBF-NN models, and then various single RBF-NN predictors are produced based on diversity principle. In the third stage, the Partial Least Square (PLS) technology is used to choosing the appropriate number of neural network ensemble members. In the final stage, W-SVR is used for ensemble of the RBF-NN to prediction purpose. For testing purposes, this study compare the new ensemble model's performance with some existing neural network ensemble approaches in terms of monthly rainfall forecasting on Guangxi, China. Experimental results reveal that the predictions using the proposed approach are consistently better than those obtained using the other methods presented in this study in terms of the same measurements. Those results show that the proposed hybrid ensemble technique provides a promising alternative to rainfall prediction.

Due to now outdated construction technology, houses which have not been retrofitted since construction typically fail to meet modern energy performance levels. However, identifying at a city scale which houses could benefit the most from retrofit solutions is currently a labour intensive process. In this paper, a system that uses a vehicle mounted camera to capture pictures of residential buildings and then performs semantic segmentation to differentiate components of captured buildings is presented. An ensemble of U-Net semantic segmentation models are trained to identify walls, roofs, chimneys, windows and doors from building facade images and differentiate between window and door instances which are partially visible or obscured. Results show that the ensemble of U-Net models achieved high accuracy in identifying walls, roofs and chimneys, moderate accuracy in identifying windows and low accuracy in identifying doors and instances of windows and doors which were partially visible or obscured. When U-Net models were retrained to identify doors or windows, irrespective of partially visible and obscured instances, a significant rise in door and window identification accuracy was observed. It is believed that a larger training dataset would produce significantly improved results across all classes. The results presented here prove the operational feasibility in the first part of a process to combine this model with high-resolution thermography and GPS for automating building retrofitting evaluations.

Degradation process prediction for rotational machinery based on hybrid intelligent model

PurposeThe purpose of this paper is to present a deep ensemble neural network model for the detection of forest fires in aerial vehicle videos.Design/methodology/approachPresented deep ensemble models include four convolutional neural networks (CNNs): a faster region-based CNN (Faster R-CNN), a simple one-stage object detector (RetinaNet) and two different versions of the you only look once (Yolo) models. The presented method generates its output by fusing the outputs of these different deep learning (DL) models.FindingsThe presented fusing approach significantly improves the detection accuracy of fire incidents in the input data.Research limitations/implicationsThe computational complexity of the proposed method which is based on combining four different DL models is relatively higher than that of using each of these models individually. On the other hand, however, the performance of the proposed approach is considerably higher than that of any of the four DL models.Practical implicationsThe simulation results show that using an ensemble model is quite useful for the precise detection of forest fires in real time through aerial vehicle videos or images.Social implicationsBy this method, forest fires can be detected more efficiently and precisely. Because forests are crucial breathing resources of the earth and a shelter for many living creatures, the social impact of the method can be considered to be very high.Originality/valueThis study fuses the outputs of different DL models into an ensemble model. Hence, the ensemble model provides more potent and beneficial results than any of the single models.

An ensemble of deep neural networks for kidney ultrasound image classification

Introduction: The Coronavirus has spread across the globe and infected millions of people, having devastating effect on the global public health and economies. A fast diagnostic system should be implemented to mitigate the impact of the virus and save lives. In this study, we propose a decision tree-based ensemble model using two mixtures of discriminative experts (MoE) to classify COVID-19 and non-COVID-19 lung infections on chest X-ray images. The Epistocracy algorithm, a hyper-heuristic evolutionary method, is employed to optimize the neural networks used in this work. Using this approach can help detect COVID-19 cases and accelerate treatment of those who need it the most. Data: we collected 2,500 chest X-ray images from Henry Ford Health System consisting of 1,250 Covid images and 1,250 non-Covid images. The input images have been cropped and resized to 224 by 224 pixels. Out of 2,500 images, we left out 500 images containing 250 Covid and 250 non-Covid for testing. The rest, 2,000 images, were used 80% for training and 20% for validation. Methods and Results: To improve the accuracy of the proposed model, first we divided our 2,000 images into 5 different clusters using K-Means clustering algorithm with VGG16 feature extractor to help build strong discriminative expert models to be used in our proposed classifier. We trained VGG16, VGG19, InceptionV3, InceptionResNetV2, MobileNetV2, EfficientNetB7, Xception, and DenseNet201 to classify each cluster into Covid and non-Covid cases. The best result was obtained from VGG16 as a base model with a deep neural network as a head model optimized by Epistocracy algorithm. Then we built a mixture of transfer learning-based experts consisting of 5 different VGG16 models supervised by InceptionV3 as a gating network. Finally, we built a decision tree-based ensemble model to determine the classification of the data using two different MoEs with highest accuracies. As a result, for initial clusters c1, c2, c3, c4, and c5 we obtained validation accuracy of 92.50%, 86.30%, 86.51%, 85.34%, and 93.62% respectively. The first MoE had 93.75% accuracy on validation, and the second MoE had 94.25%. The final ensemble model on average obtained 94% accuracy on the testing dataset. More specifically, we got 96% accuracy on Covid images and 92% accuracy on non-Covid. Conclusion: we showed that an ensemble model consisting of two mixtures of cluster-based discriminative convolutional neural network experts can be used to detect Covid from non-Covid with high accuracy, and Epistocracy algorithm can be effectively used to optimize the hyper-parameters of the proposed models. Citation Format: Seyed Ziae Mousavi Mojab, Seyedmohammad Shams, Farshad Fotouhi, Hamid Soltanian-Zadeh. EpistoNet: An ensemble of deep convolutional neural networks using mixture of discriminative experts for detecting COVID-19 on chest X-ray images [abstract]. In: Proceedings of the AACR Virtual Meeting: COVID-19 and Cancer; 2021 Feb 3-5. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(6_Suppl):Abstract nr P05.

Various criteria are proposed to select the number of hidden neurons in artificial neural network (ANN) models and based on the criterion evolved an intelligent ensemble neural network model is proposed to predict wind speed in renewable energy applications. The intelligent ensemble neural model based wind speed forecasting is designed by averaging the forecasted values from multiple neural network models which includes multilayer perceptron (MLP), multilayer adaptive linear neuron (Madaline), back propagation neural network (BPN), and probabilistic neural network (PNN) so as to obtain better accuracy in wind speed prediction with minimum error. The random selection of hidden neurons numbers in artificial neural network results in overfitting or underfitting problem. This paper aims to avoid the occurrence of overfitting and underfitting problems. The selection of number of hidden neurons is done in this paper employing 102 criteria; these evolved criteria are verified by the computed various error values. The proposed criteria for fixing hidden neurons are validated employing the convergence theorem. The proposed intelligent ensemble neural model is applied for wind speed prediction application considering the real time wind data collected from the nearby locations. The obtained simulation results substantiate that the proposed ensemble model reduces the error value to minimum and enhances the accuracy. The computed results prove the effectiveness of the proposed ensemble neural network (ENN) model with respect to the considered error factors in comparison with that of the earlier models available in the literature.

Purpose – The aim of this research study is to develop a risk-based framework that can quantify maintainability to forecast future maintainability of a building at early stages as a decision tool to minimize increase of maintenance cost. Design/methodology/approach – A survey-based approach was used to explore the risk factors in the domain of maintainability risks under tropical environmental conditions. The research derived ten risk factors based on 58 identified causes related to maintainability issues as common to high-rise buildings in tropical conditions. Impact of these risk factors was evaluated using an indicator referred to as the “maintenance score (MS)” which was derived from the “whole-life maintenance cost” involved in maintaining the expected “performance” level of the building. Further, an ensemble neural network (ENN) model was developed to model the MS for evaluating maintainability risks in high-rise buildings. Findings – Results showed that predictions from the model were highly compatible and in the same order when compared with calculations based on actual past data. It further showed that, maintainability of buildings could be improved if the building was designed, constructed and managed properly by controlling their maintainability risks. Originality/value – The ENN model was used to analyze maintainability of a high-rise building. Thus, it provides a useful tool for designers, clients, facilities managers/maintenance managers and users to analyze maintainability risks of buildings at early stages.

This study aims to classify cybersickness (CS) caused by virtual reality (VR) immersion through a machine-deep-ensemble learning model. The heart rate variability and respiratory signal parameters of 20 subjects were measured, while watching a VR video for ∼5 minutes. After the experiment, the subjects were examined for CS and questioned to determine their CS states. Based on the results, we constructed a machine-deep-ensemble learning model that could identify and classify VR immersion CS among subjects. The ensemble model comprised four stacked machine learning models (support vector machine [SVM], k-nearest neighbor [KNN], random forest, and AdaBoost), which were used to derive prediction data, and then, classified the prediction data using a convolution neural network. This model was a multiclass classification model, allowing us to classify subjects' CS into three states (neutral, non-CS, and CS). The accuracy of SVM, KNN, random forest, and AdaBoost was 94.23 percent, 92.44 percent, 93.20 percent, and 90.33 percent, respectively, and the ensemble model could classify the three states with an accuracy of 96.48 percent. This implied that the ensemble model has a higher classification performance than when each model is used individually. Our results confirm that CS caused by VR immersion can be detected as physiological signal data with high accuracy. Moreover, our proposed model can determine the presence or absence of CS as well as the neutral state. Clinical Trial Registration Number: 20-2021-1.

Chunking means splitting the sentences into tokens and then grouping them in a meaningful way. When it comes to high-performance chunking systems, transformer models have proved to be the state of the art benchmarks. To perform chunking as a task it requires a large-scale high quality annotated corpus where each token is attached with a particular tag similar as that of Named Entity Recognition Tasks. Later these tags are used in conjunction with pointer frameworks to find the final chunk. To solve this for a specific domain problem, it becomes a highly costly affair in terms of time and resources to manually annotate and produce a large-high-quality training set. When the domain is specific and diverse, then cold starting becomes even more difficult because of the expected large number of manually annotated queries to cover all aspects. To overcome the problem, we applied a grammar-based text generation mechanism where instead of annotating a sentence we annotate using grammar templates. We defined various templates corresponding to different grammar rules. To create a sentence we used these templates along with the rules where symbol or terminal values were chosen from the domain data catalog. It helped us to create a large number of annotated queries. These annotated queries were used for training the machine learning model using an ensemble transformer-based deep neural network model [24.] We found that grammar-based annotation was useful to solve domain-based chunks in input query sentences without any manual annotation where it was found to achieve a classification F1 score of 96.97% in classifying the tokens for the out of template queries.