Ensemble Learning Theory Research Articles

Ensemble learning-based method for maritime background subtraction in open sea environments

Maritime autonomous surface ships employ computer vision to detect open sea environments for various applications, including safety surveillance, collision prevention, and autonomous navigation, all of which have grown increasingly important as maritime traffic has increased and automated systems have become more prevalent. Background subtraction (BS) may be used for real-time change detection in unseen contexts to obtain foreground information of interest without prior knowledge and training. However, because marine settings are dynamic, classic BS methods are vulnerable to noise (e.g., reflections and ship wakes) on the ocean surface. As a result, developing effective background models is challenging, resulting in considerable false foreground information. This study proposes a novel maritime BS method based on ensemble learning theory that integrates heterogeneous BS methods. Maritime noise filtering is also included to increase the background model’s ability to cope with complex marine situations. Experimental comparisons conducted on the Maritime BS Benchmark dataset showed that the proposed method had the highest real-time detection accuracy. The proposed method may also be used to improve autonomous ships’ situational awareness abilities in open waters, enhancing maritime transportation security.

Automatic generation of fuzzy inference systems for multivariate time series forecasting

For the time series forecasting process, the model must learn the variables' temporal dynamics to obtain consistent future values. Although its forecast may depend only on past values in some time series, there are numerous applications in which multivariate forecasting is necessary for a more accurate result. Therefore a model that deals with this type of problem is needed. In addition, explainability is an essential feature for choosing the forecast model since it is possible to extract insights and understand which variables were most important for the result generated. A Fuzzy Inference System, named AutoMFIS, is a model for multivariate time series forecasting with good accuracy and interpretability. However, despite the favorable characteristics, this system has limitations that make this technique not practical in problems with high-dimensionality databases. This article proposes an extension to the base model AutoMFIS for time series forecasting with high dimensionality data, named e-AutoMFIS. Based on the Ensemble learning theory, this new methodology applies distributed learning for automatic fuzzy rules generation and rule filtering to keep the system semantically interpretable. The proposed model is evaluated in different case studies, in which the results are compared in terms of accuracy along with other methods in the literature. In addition, in each selected problem, the interpretability aspect is also assessed.

Aerodynamic Parameter Identification of Projectile Based on Improved Extreme Learning Machine and Ensemble Learning Theory

The firing accuracy of the projectile has a positive relation with aerodynamic parameters. Due to the complex dynamic characteristics of projectiles, there is an overfitting risk when a single extreme learning machine (ELM) is used to identify the aerodynamic parameters of the projectile, and the identification results oscillate transonic region. To obtain the aerodynamic parameters of the projectile accurately, an aerodynamic parameter identification model based on ensemble learning theory and ELM optimized by improved particle swarm optimization is proposed. The improved particle swarm optimization algorithm (IPSO) with an adaptive update strategy is used to optimize the weight and threshold of ELM. Combined with the ensemble learning theory, the improved ELM neural network is regarded as a weak learner to generate a strong learner. The structural parameters of the strong learner were continuously optimized through training, and an aerodynamic parameter identification model of projectile based on ensemble learning theory is obtained. The simulation results show that the introduction of the IPSO and ensemble learning theory enables the model to exhibit excellent generalization ability. The proposed identification model can accurately describe the variation of aerodynamic parameters with the Mach number.

Short-term load forecasting based on EEMD-Adaboost-BP

ABSTRACT In order to realize short-time load forecasting, an Adaboost-BP method with a weight update mechanism is proposed based on ensemble learning theory. Firstly, the original historical load power is decomposed into a set of sub-series with diverse characteristics via using ensemble empirical mode decomposition. Then, BP neural network is performed as a weak learner to predict the load power of test samples. At the same time, the prediction results are used to update the weight of the weak learner and test sample and then construct a strong learner to obtain the final prediction results. According to the analysis results of the characteristics of each sub-series, the load forecasting model is established. The result of analysing the calculation example shows that the proposed prediction model outperforms all other algorithms in accuracy, which has high engineering application value.

Cross-Modal Transfer Learning From EEG to Functional Near-Infrared Spectroscopy for Classification Task in Brain-Computer Interface System.

The brain-computer interface (BCI) based on functional near-infrared spectroscopy (fNIRS) has received more and more attention due to its vast application potential in emotion recognition. However, the relatively insufficient investigation of the feature extraction algorithms limits its use in practice. In this article, to improve the performance of fNIRS-based BCI, we proposed a method named R-CSP-E, which introduces EEG signals when computing fNIRS signals’ features based on transfer learning and ensemble learning theory. In detail, we used the Independent Component Analysis (ICA) algorithm for the correspondence between the sources of the two signals. We then introduced the EEG signals when computing the spatial filter based on a modified Common Spatial Pattern (CSP) algorithm. Experimental results on public datasets show that the proposed method in this paper outperforms traditional methods without transfer. In general, the mean classification accuracy can be increased by up to 5%. To our knowledge, it is an innovation that we tried to apply transfer learning between EEG and fNIRS. Our study’s findings not only prove the potential of the transfer learning algorithm in cross-model brain-computer interface, but also offer a new and innovative perspective to research the hybrid brain-computer interface.

Prediction of Ecosystem Service Function of Grain for Green Project Based on Ensemble Learning

The Grain for Green Project (GGP) was implemented over 20 years ago as one of six major forestry projects in China, and its scope of implementation is still expanding. However, it is still unclear how many ecosystem services (ESs) the project will produce in the future. The GGP’s large-scale ecological monitoring officially started in 2015 and there is a lack of early monitoring data, making it challenging to predict the future ecological benefits. Therefore, this paper proposes a method to predict future ESs by using ecological monitoring data. First, a new ensemble learning system, auto-XGBoost-ET-DT, is developed based on ensemble learning theory. Using the GGP’s known ESs in 2015, 2017, and 2019, the missing ESs of the past decade have been evaluated via reverse regression. Data from 2020 to 2022 within a convolution neural network and the coupling coordination degree model have been used to analyze the coupling between the prediction results and economic development. The results show that the growth distributions of ESs in three years were as follows: soil consolidation, 3.70–6.34%; forest nutrient retention, 2.72–.71%; water conservation, 2.52–6.09%; carbon fixation and oxygen release, 3.00–4.64%; and dust retention, 3.82–5.75%. The coupling coordination degree of the ESs and economic development has been improved in 97% of counties in 2020 compared with 2019. The results verify a feasible ES prediction method and provide a basis for the progressive implementation of the GGP.

Pipeline image diagnosis algorithm based on neural immune ensemble learning

In order to provide direction guidance for the heat loss of steam pipeline and ensure the safe and energy-saving operation of steam pipe network, it is very important to realize rapidly and accurate diagnosis of high-temperature area of the pipeline. This paper is aimed at the problem of the complicated working environment of the steam pipe, which causes the infrared image obtained have strong background interference. A neural immune ensemble learning algorithm, based on the ensemble learning theory and the coordination between nervous system and immune system, is proposed to diagnose abnormal high temperature region in infrared steam pipeline image. Experimental results show that the proposed algorithm can effectively diagnose the high temperature region of steam pipeline. Compared with other image classification algorithms, the results show that the neural immune ensemble learning algorithm has higher diagnostic accuracy and can meet the needs of actual engineering applications.

DAMER: a novel diagnosis aggregation method with evidential reasoning rule for bearing fault diagnosis

Ensemble learning method has shown its superiority in bearing fault diagnosis based on the condition based monitoring. Nevertheless, features extracted from the monitoring signals of bearing systems often contain interrelated and redundant components, leading to poor performances of the base classifiers in the ensemble. Moreover, the current ensemble methods rely on voting strategies to aggregate the diagnostic predictions of these base classifiers without considering their reliabilities and weights simultaneously. To address the aforementioned issues, we propose a novel Diagnosis Aggregation Method with Evidential Reasoning rule, i.e., DAMER, for bearing fault diagnosis. In this method, a semi-random subspace approach using a structured sparsity learning model is developed to decrease the negative effect of interrelated and redundant features, and in the meanwhile to generate accurate and diverse base classifiers. Furthermore, an adaptive evidential reasoning rule (ER rule) incorporating with ensemble learning theory is utilized to aggregate the diagnostic predictions of the base classifiers by taking both their weights and reliabilities into account. To validate the proposed DAMER, an empirical study is conducted on Case Western Reserve University bearing vibration dataset, and the experimental results verify the effectiveness of the proposed DAMER as well as its superiority over commonly used ensemble methods. The performances of feature subsets from multiple domains and the aggregation capability of the adaptive ER rule were also investigated. Results illustrate that DAMER can be utilized as an effective method for bearing fault diagnosis.

Neoliberalism in the North American University

This paper uses observations from empirical articles and personal experiences of the authors to explore issues associated with the rise of neoliberalism and academic capitalism in the contemporary public university. It frames these issues as stemming from conflicting ontologies between academicians who adhere to the differentiated individual ontology and university administrators who favor the undifferentiated individual ontology. To overcome the disconnect, a differentiated relational ontology that adheres to principles of Mary Parker Follett and Alfred North Whitehead is proposed. The driving force behind this ontology can be highlighted through a communicated crisis, and a specific application of Follett’s differentiated relational ontology is Ensemble Learning Theory (ELT). A potential limitation of this study is generalizability, because the focus is centered on North American public universities and anecdotes are used to characterize a broader educational problem. This evolution is pertinent to academicians and administrators because the ontological impasse experienced in North American public universities threatens their existence as institutions, and has a broader and potentially negative impact on the quality of educational focus and output.

Selective ensemble learning and its applications in fully polarimetric SAR image classification

Polarimetric SAR (PolSAR) can comprehensively describe the differences of targets combining the polarimetric scattering power and the relative phase information among the polarimetric channels. For the traditional image classification methods, a single classifier cannot fully solve the classification problem, in which certain inevitable bias between the classification results and the actual condition exists. To improve the classification performance, ensemble learning theory has been introduced into PolSAR classification, in which a number of different learners are integrated to get the final results. However, a larger number of learners may also lead to some negative effects. The concept of selective ensemble learning (SEL) is then introduced, in which some better learners are selected from a group of individual learners according to the selection strategy and then integrated to obtain a generalized classifier. Based on this idea, a SEL-based PolSAR image classification method is proposed, in which the representative features are extracted based on the characteristics of PolSAR image meanwhile high-performing learners are selected using genetic algorithm optimization. In order to verify the effectiveness and application of the proposed SEL-based PolSAR image classification method, the tests are implemented using UAVSAR L-band PolSAR data, besides the other two newly acquired experimental data of Chinese airborne and spaceborne system. The quantitative and qualitative experimental results confirm that the proposed method has excellent performance in classification.

A novel ensemble learning method for crash prediction using road geometric alignments and traffic data

As an important part of traffic safety analysis, crash prediction models using road geometric alignments and traffic data (CPM-GAs) have been regarded as the most classic way and can be used in stages of road safety evaluation and road operating and management. To improve the predictive performance of tradition CPM-GAs and avoid the overfitting problem of machine learning algorithms, a framework of CPM-GA based on ensemble learning theory and a new ensemble rule for connecting traditional models and machine learning models were proposed in this study. Results of the ensemble learning CPM-GA show that (1) classification and regression tree (CART) is recommended for important variable selection procedure before applying support vector machine (SVM), (2) machine learning models outperformed traditional models significantly in aspects of model fitting and prediction accuracy but are unstable in the sensitivity tests, (3) the new proposed ensemble method of traditional model and machine learning model can effectively improve the accuracy of traditional CPM-GAs by 10%–16% and reduce the variance of machine learning CPM-GAs by 12%–36% simultaneously. Finally, the ensemble method presented in this article may shed light on more research of crash prediction models based on ensemble learning theory.

A case for folletian interventions in public universities

PurposeThe purpose of this paper is to illustrate the ideological impasses between educationally minded faculty and neoliberal oriented university administrators. To bridge and benefit from these two perspectives, Follettian integration is introduced. Specifically, the ensemble learning theory (ELT) and entrepreneurship centers are illustrated as Follettian interventions and their reasons for success are discussed.Design/methodology/approachThis paper is theoretical, but provides ethnographic anecdotes of the problems occurring during the rise of neoliberalism and academic capitalism in the public university. The successful use of the ELT and entrepreneurship centers is likewise explored anecdotally.FindingsThis paper illustrates the benefits of utilizing the ELT and entrepreneurship centers in two different university settings.Research limitations/implicationsWhile the sample sizes of this paper are small, the anecdotal examples provide the basis for reasoning by analogy.Practical implicationsThis work illustrates two possible Follettian interventions that serve as a guide to assist university administrators and faculty to find common ground and better serve students and university communities.Originality/valueThe rise of academic capitalism and neoliberalism has devalued education and resulted in poorer educational outcomes and a modern generation with less intellectual capital. This is one of the first papers to utilize Mary Parker Follett’s theories of education and apply them to the impending identity crisis of the public university. The result is a win-win for both neoliberal administrators and faculty in the face of an impending identity crisis for the public university.

Fast SAR target recognition based on random convolution features and ensemble extreme learning machines

Deep convolution neural network has demonstrated excellent performance in target detection and recognition tasks. However, few training samples and optimization design of deep models are two main problems to be solved when applied to SAR target recognition. This paper proposes an algorithm for SAR target recognition by combination of two dimensional random convolution features and ensemble extreme learning machines. Firstly, two dimensional random convolution kernels with different widths are generated, and convolution and pooling operations are performed in input image to extract random convolution feature vectors. Secondly, random samplings based on ensemble learning theory are done for extracted feature vectors to improve generalization performance of classifier and reduce training time, and base classifiers are then trained by extreme learning machines (ELM). Finally, majority vote method is adopted to combine the classification results of base classifiers. SAR target recognition experiments based on MSTAR database were performed, and experimental results demonstrate that, training time has dropped by ten times due to fast training capability of ELM, and the proposed algorithm achieves comparable classification performance with deep-learning-based methods which use data augmentation and multiple convolution layers. The proposed algorithm has the advantages of easy implementation and fewer adjustable parameters, and improves classifier's generalization performance through adoption of ensemble learning ideas.

Ensemble predictive model for more accurate soil organic carbon spectroscopic estimation

A myriad of signal pre-processing strategies and multivariate calibration techniques has been explored in attempt to improve the spectroscopic prediction of soil organic carbon (SOC) over the last few decades. Therefore, to come up with a novel, more powerful, and accurate predictive approach to beat the rank becomes a challenging task. However, there may be a way, so that combine several individual predictions into a single final one (according to ensemble learning theory). As this approach performs best when combining in nature different predictive algorithms that are calibrated with structurally different predictor variables, we tested predictors of two different kinds: 1) reflectance values (or transforms) at each wavelength and 2) absorption feature parameters. Consequently we applied four different calibration techniques, two per each type of predictors: a) partial least squares regression and support vector machines for type 1, and b) multiple linear regression and random forest for type 2. The weights to be assigned to individual predictions within the ensemble model (constructed as a weighted average) were determined by an automated procedure that ensured the best solution among all possible was selected. The approach was tested at soil samples taken from surface horizon of four sites differing in the prevailing soil units. By employing the ensemble predictive model the prediction accuracy of SOC improved at all four sites. The coefficient of determination in cross-validation (R2cv) increased from 0.849, 0.611, 0.811 and 0.644 (the best individual predictions) to 0.864, 0.650, 0.824 and 0.698 for Site 1, 2, 3 and 4, respectively. Generally, the ensemble model affected the final prediction so that the maximal deviations of predicted vs. observed values of the individual predictions were reduced, and thus the correlation cloud became thinner as desired.

Regional frequency analysis at ungauged sites using a two-stage resampling generalized ensemble framework

Regional frequency analysis (RFA) deals with the estimation of hydrological characteristics at sites where little or no data is available. Recently, machine learning applications to RFA have received considerable attention in terms of their flexibility in modeling as well as superior generalization ability compared to conventional approaches. The proper application of machine learning techniques, however, requires good understanding of the available information about system's dynamics, i.e. system variables. This paper presents two contributions to the literature. First, novel ensemble architecture, using a unique two-stage resampling approach, is proposed. The objective of the proposed ensemble model is to promote diversity within the individual learners and reduce over fitting. Second, the application of the proposed model is demonstrated in RFA case study to obtain improved regional flood quantile estimates at ungauged sites. A jackknife validation procedure is used for the evaluation of the model's performance. The method is applied to data from the province of Quebec, Canada. The model showed similar performance and generalization compared to major ensemble models in the literature, which were investigated in previous studies using the same data set. The proposed model confirmed the diversity requirement in ensemble modeling and, in the same time, validated the proposed model adherence to ensemble learning theory.

A Novel Distributed Online Anomaly Detection Method in Resource-Constrained Wireless Sensor Networks

In this paper, a novel distributed online anomaly detection method in resource-constrained WSNs was proposed. Firstly, the spatiotemporal correlation existing in the sensed data was exploited and a series of single anomaly detectors were built in each distributed deployment sensor node based on ensemble learning theory. Secondly, these trained detectors were broadcasted to the member sensor nodes in the cluster, combining with its trained detector, and the initial ensemble detector was built. Thirdly, considering resources-constrained WSNs, ensemble pruning based on biogeographical based optimization (BBO) was employed in the cluster head node to obtain an optimized subset of ensemble members. Further, the pruned ensemble detector coded by the state matrix was broadcasted to each member sensor nodes for the distributed online global anomaly detection. Finally, the experiments operated on a real WSN dataset demonstrated the effectiveness of the proposed method.

A Novel Emotion Recognition Method Based on Ensemble Learning and Rough Set Theory

Emotion recognition is a very hot topic, which is related with computer science, psychology, artificial intelligence, etc. It is always performed on facial or audio information with classical method such as ANN, fuzzy set, SVM, HMM, etc. Ensemble learning theory is a novelty in machine learning and ensemble method is proved an effective pattern recognition method. In this paper, a novel ensemble learning method is proposed, which is based on selective ensemble feature selection and rough set theory. This method can meet the tradeoff between accuracy and diversity of base classifiers. Moreover, the proposed method is taken as an emotion recognition method and proved to be effective according to the simulation experiments.