Fast Ensemble Research Articles

Data- & compute-efficient deviance mining via active learning and fast ensembles

Detecting deviant traces in business process logs is crucial for modern organizations, given the harmful impact of deviant behaviours (e.g., attacks or faults). However, training a Deviance Prediction Model (DPM) by solely using supervised learning methods is impractical in scenarios where only few examples are labelled. To address this challenge, we propose an Active-Learning-based approach that leverages multiple DPMs and a temporal ensembling method that can train and merge them in a few training epochs. Our method needs expert supervision only for a few unlabelled traces exhibiting high prediction uncertainty. Tests on real data (of either complete or ongoing process instances) confirm the effectiveness of the proposed approach.

FASTEN: Fast Ensemble Learning for Improved Adversarial Robustness

FASTEN: Fast Ensemble Learning for Improved Adversarial Robustness

Three's a crowd: Fast ensemble perception of first impressions of trustworthiness

Trustworthiness impressions are fundamental social judgements with far-reaching consequences in many aspects of society, including criminal justice, leadership selection and partner preferences. Thus far, most research has focused on facial characteristics that make a face individually appear more or less trustworthy. However, in everyday life, faces are not always perceived in isolation but are often encountered in crowds. It has been proposed that we deal with the large amount of facial information in a group by extracting summary statistics of the crowd, a phenomenon called ensemble perception. Prior research showed that ensemble perception occurs for various facial features, such as emotional expression, facial identity, and attractiveness. Here, we investigated whether observers can integrate the level of trustworthiness from multiple faces to extract an average impression of the crowd. Across four studies, participants were presented with crowds of faces and were asked to report their average level of trustworthiness with an adjustment (Experiment 1) and a rating task (Experiments 2 and 3). Participants were able to extract an ensemble perception of trustworthiness impressions from multiple faces. Moreover, observers were able to form a summary statistic of trustworthiness impressions from a group of faces as quickly as 250 ms (Experiment 4). Taken together, these results demonstrate that ensemble perception can occur at the level of impressions of trustworthiness. Thus, these critical social judgements not only occur for individual faces but are also integrated into a unique ensemble impression of crowds. Our findings contribute to the development of a more ecological approach to the study of trust impressions, since they provide an understanding of trustworthiness judgements not only on an individual level, but on a much broader social group level. Furthermore, our results drive forward new theory because they demonstrate for the first time that ensemble representations cover a broad range of phenomena than previously recognized, including complex high-level facial trait judgements such as trustworthiness impressions.

FDEIR: Content-Based Image Retrieval using Fast Demeanor Ensemble Features

The demand for CBIR-based systems is increasing day-by-day with the increase of its applications. Nowadays, various digital information systems like Medicine, Digital Libraries, Biodiversity information systems, Fingerprint Identification, crime prevention, and many more are in trend, and all these systems need accuracy. Several Automated Systems were earlier developed using features to improve the systems' accuracy, but it introduces complexity and speed. A fast Demeanor Ensemble features-based approach is proposed to cope with this issue, which also deals with accuracy, speeds up the system, and reduces its complexity. Different shape-based features are used to form 3-set composite features, followed by the selection phase where the ALO algorithm is utilized and reduces system complexity. The accuracy of the proposed system is estimated using distance-based matching methods. Correl-1000 dataset is used to analyze the system's performance based on different metrics, and the results show the proposed system's ability.

Fast Ensemble Empirical Mode Decomposition Using the Savitzky Golay filter

Empirical mode decomposition (EMD) is a powerful algorithm proposed to analysis of nonlinear and non-stationary signals. The phenomenon of mode mixing is one of the major disadvantages of the EMD. The Ensemble EMD (EEMD) was introduced to eliminate the mode-mixing effect. The principle of EEMD is to add additional white noise into the signal with many trials. The noise in each trial is different; and the added noise can be completely cancelled out on average, if the number of trials is very high. The number of trials is a high computational load. The improvement on computational efficiency of EEMD is therefore required. In this paper, an improvement on the computing time of the EEMD was proposed by replacing white noise with white noise filtered using Savitzky-Golay (SG) filter. Numerical simulations were performed to demonstrate that such replacement has effectively reduced the number of trials to obtain a noise-free reconstructed signal.

Snapshot boosting: a fast ensemble framework for deep neural networks

Boosting has been proven to be effective in improving the generalization of machine learning models in many fields. It is capable of getting high-diversity base learners and getting an accurate ensemble model by combining a sufficient number of weak learners. However, it is rarely used in deep learning due to the high training budget of the neural network. Another method named snapshot ensemble can significantly reduce the training budget, but it is hard to balance the tradeoff between training costs and diversity.Inspired by the ideas of snapshot ensemble and boosting, we propose a method named snapshot boosting.A series of operations are performed to get many base models with high diversity and accuracy,such as the use of the validation set, the boosting-based training framework, and the effectiveensemble strategy. Last, we evaluate our method on the computer vision (CV) and the natural languageprocessing (NLP) tasks, and the results show that snapshot boosting can get a more balanced trade-offbetween training expenses and ensemble accuracy than other well-known ensemble methods.

FastEE: Fast Ensembles of Elastic Distances for time series classification

In recent years, many new ensemble-based time series classification (TSC) algorithms have been proposed. Each of them is significantly more accurate than their predecessors. The Hierarchical Vote Collective of Transformation-based Ensembles (HIVE-COTE) is currently the most accurate TSC algorithm when assessed on the UCR repository. It is a meta-ensemble of 5 state-of-the-art ensemble-based classifiers. The time complexity of HIVE-COTE—particularly for training—is prohibitive for most datasets. There is thus a critical need to speed up the classifiers that compose HIVE-COTE. This paper focuses on speeding up one of its components: Ensembles of Elastic Distances (EE), which is the classifier that leverages on the decades of research into the development of time-dedicated measures. Training EE can be prohibitive for many datasets. For example, it takes a month on the ElectricDevices dataset with 9000 instances. This is because EE needs to cross-validate the hyper-parameters used for the 11 similarity measures it encompasses. In this work, Fast Ensembles of Elastic Distances is proposed to train EE faster. There are two versions to it. The exact version makes it possible to train EE 10 times faster. The approximate version is 40 times faster than EE without significantly impacting the classification accuracy. This translates to being able to train EE on ElectricDevices in 13 h.

A Comprehensive Fault Diagnosis Method for Rolling Bearings Based on Refined Composite Multiscale Dispersion Entropy and Fast Ensemble Empirical Mode Decomposition.

This study presents a comprehensive fault diagnosis method for rolling bearings. The method includes two parts: the fault detection and the fault classification. In the stage of fault detection, a threshold based on refined composite multiscale dispersion entropy (RCMDE) at a local maximum scale is defined to judge the health state of rolling bearings. If the bearing is in fault, a generalized multi-scale feature extraction method is developed to fully extract fault information by combining fast ensemble empirical mode decomposition (FEEMD) and RCMDE. Firstly, the fault vibration signals are decomposed into a set of intrinsic mode functions (IMFs) by FEEMD. Secondly, the RCMDE value of multiple IMFs is calculated to generate a candidate feature pool. Then, the maximum-relevance and minimum-redundancy (mRMR) approach is employed to select the sensitive features from the candidate feature pool to construct the final feature vectors, and the final feature vectors are fed into random forest (RF) classifier to identify different fault working conditions. Finally, experiments and comparative research are carried out to verify the performance of the proposed method. The results show that the proposed method can detect faults effectively. Meanwhile, it has a more robust and excellent ability to identify different fault types and severity compared with other conventional approaches.

A hybrid wind speed forecasting model based on a decomposition method and an improved regularized extreme learning machine

Abstract Accurate wind speed forecasting plays a critical role in wind farm planning for economic operation of power grid. New forecasting methods which lead to better performances are urgently demanded for the wind energy management as there is no one model that is qualified in all cases. In this paper, a novel hybrid model is developed by integrating Fast Ensemble Empirical Mode Decomposition (FEEMD) and Regularized Extreme Learning Machine (RELM) optimized by Backtracking Search Algorithm (BSA) to realize the accurate short-term wind speed forecasting. This hybrid model can be partitioned into three phases: data preprocessing, training RELM by BSA, and aggregated calculation. Experiments have been conducted under different forecasting horizons and seasons with the proposed method as well as five benchmark models (RBF, GRNN, RELM, FEEMD-RELM, and BSA-RELM), respectively. Experimental results show the proposed model reduces RMSE by about 60% on average for all four cases than single RELM and it is superior to all the involved models in short-term wind speed forecasting.

Analysis and Forecasting of the Carbon Price in China’s Regional Carbon Markets Based on Fast Ensemble Empirical Mode Decomposition, Phase Space Reconstruction, and an Improved Extreme Learning Machine

With the development of the carbon market in China, research on the carbon price has received more and more attention in related fields. However, due to its nonlinearity and instability, the carbon price is undoubtedly difficult to predict using a single model. This paper proposes a new hybrid model for carbon price forecasting that combines fast ensemble empirical mode decomposition, sample entropy, phase space reconstruction, a partial autocorrelation function, and an extreme learning machine that has been improved by particle swarm optimization. The original carbon price series is decomposed using the fast ensemble empirical mode decomposition and sample entropy methods, which eliminate noise interference. Then, the phase space reconstruction and partial autocorrelation function methods are combined to determine the input and output variables in the forecasting models. An extreme learning machine optimized by particle swarm optimization was employed to forecast carbon prices. An empirical study based on carbon prices in three typical regional carbon markets in China found that this new hybrid model performed better than other comparable models.

Customised ensemble methodologies for deep learning: Boosted Residual Networks and related approaches

This paper introduces a family of new customised methodologies for ensembles, called Boosted Residual Networks, which builds a boosted ensemble of residual networks by growing the member network at each round of boosting. The proposed approach combines recent developments in residual networks—a method for creating very deep networks by including a shortcut layer between different groups of layers—with Deep Incremental Boosting, a methodology to train fast ensembles of networks of increasing depth through the use of boosting. Additionally, we explore a simpler variant of Boosted Residual Networks based on bagging, called Bagged Residual Networks. We then analyse how the recent developments in ensemble distillation can improve our results. We demonstrate that the synergy of residual networks and Deep Incremental Boosting has better potential than simply boosting a residual network of fixed structure or using the equivalent Deep Incremental Boosting without the shortcut layers, by permitting the creation of models with better generalisation in significantly less time.

Fast ensemble empirical mode decomposition model for forecasting crude oil and condensates

Crude oil and condensates supply and demand strives to be main authority of the sustenance of almost all country’s economy. The sudden rise in the oil price has forced the government to forecast the supply and demand of crude oil and condensates in order to make sure that the amount of crude oil meets the supply and demand of the country. Accurate forecasts can save cost, foresee scarcity of demand, and help in budgeting profit. In addition, predicting crude oil and condensate data is frequently proven to be a demanding task considering the various intricacies of oil data pattern. The main objective of this study was to forecast crude oil and condensates demand data in Malaysia using Fast Ensemble Empirical Mode Decomposition (FEEMD) model. The forecasting process using FEEMD model was performed in order to achieve the most desirable forecast accuracy of the crude oil and condensates data. The FEEMD model is an extension of the Empirical Mode Decomposition (EMD) model whereby white noise signal was added to the existing signal in the sifting process. The effectiveness of the proposed forecasting method was compared to other traditional models of ARIMA, ARIMAX and GARCH. The results revealed that the proposed FEEMD method for forecasting crude oil and condensates data was very promising as it achieved good forecast accuracy.

A Hybrid GA-FEEMD for Forecasting Crude Oil Prices

Forecasting crude oil prices is essential but usually involves a difficult process. In this paper, we proposed a hybrid Genetic Algorithm and Fast Ensemble Empirical Mode Decomposition (GA-FEEMD) for forecasting crude oil price time series data. The proposed GA-FEEMD basically involves three steps. Firstly, we decomposed the original crude oil price time series data into two Intrinsic Mode Functions (IMFs) using FEEMD algorithm. Then, we forecasted the second IMF which basically is the intrinsic trend of the crude oil prices. Then, we applied Genetic Algorithm (GA) to obtain the stopping criterion in the FEEMD process. The hybrid GA-FEEMD forecasting model was compared with ARIMA and Artificial Neural Network methods. The results showed that the proposed GA-FEEMD model improved the forecasting accuracy of the crude oil price time series data.

Fast EEMD Based AM-Correntropy Matrix and Its Application on Roller Bearing Fault Diagnosis

Roller bearing plays a significant role in industrial sectors. To improve the ability of roller bearing fault diagnosis under multi-rotating situation, this paper proposes a novel roller bearing fault characteristic: the Amplitude Modulation (AM) based correntropy extracted from the Intrinsic Mode Functions (IMFs), which are decomposed by Fast Ensemble Empirical mode decomposition (FEEMD) and employ Least Square Support Vector Machine (LSSVM) to implement intelligent fault identification. Firstly, the roller bearing vibration acceleration signal is decomposed by FEEMD to extract IMFs. Secondly, IMF correntropy matrix (IMFCM) as the fault feature matrix is calculated from the AM-correntropy model of the primary vibration signal and IMFs. Furthermore, depending on LSSVM, the fault identification results of the roller bearing are obtained. Through the bearing identification experiments in stationary rotating conditions, it was verified that IMFCM generates more stable and higher diagnosis accuracy than conventional fault features such as energy moment, fuzzy entropy, and spectral kurtosis. Additionally, it proves that IMFCM has more diagnosis robustness than conventional fault features under cross-mixed roller bearing operating conditions. The diagnosis accuracy was more than 84% for the cross-mixed operating condition, which is much higher than the traditional features. In conclusion, it was proven that FEEMD-IMFCM-LSSVM is a reliable technology for roller bearing fault diagnosis under the constant or multi-positioned operating conditions, and as such, it possesses potential prospects for a broad application of uses.

Fast Ensemble Forecast of Storm Surge along the Coast of China

ABSTRACT Ding, X. L.; Chen, Y. P.; Pan, Y., and Reeve, D., 2016. Fast Ensemble Forecast of Storm Surge along the Coast of China. In: Vila-Concejo, A.; Bruce, E.; Kennedy, D.M., and McCarroll, R.J. (eds.), Proceedings of the 14th International Coastal Symposium (Sydney, Australia). Journal of Coastal Research, Special Issue, No. 75, pp. 1077–1081. Coconut Creek (Florida), ISSN 0749-0208. The uncertainties in typhoon wind field forecasts may introduce significant errors in storm surge forecasts. The common method to tackle this problem is based on ensemble forecasting of a typhoon wind field by using different initial and/or boundary conditions in the adopted weather forecast model. However, this method demands very high computational costs and therefore may not always be acceptable for operational use. In order to improve time efficiency, this paper introduces a new method which mainly relies on the forecast results from different weather forecast centres. With the bias modification, the control typhoon fo...

Wind speed forecasting approach using secondary decomposition algorithm and Elman neural networks

Wind speed forecasting technology is important in the field of wind power. However, the wind speed signals are always nonlinear and non-stationary so that it is difficult to predict them accurately. Aims at this challenge, a new hybrid approach has been proposed for the wind speed high-accuracy predictions based on the Secondary Decomposition Algorithm (SDA) and the Elman neural networks. The proposed SDA combines the Wavelet Packet Decomposition (WPD) and the Fast Ensemble Empirical Mode Decomposition (FEEMD), which includes twice decomposing processes as: (a) the WPD decomposes the original wind speed into the appropriate components and the detailed components; and (b) the FEEMD further decomposes the WPD generating detailed components into a number of wind speed Intrinsic Mode Functions (IMFs). The experimental results in five real forecasting cases show that: (a) the proposed hybrid WPD-FEEMD-Elman model has satisfactory performance in the multi-step wind speed predictions; and (b) the hybrid WPD-FEEMD-Elman model has improved the forecasting performance of the hybrid WPD-Elman model and the standard Elman neural networks considerably.

New wind speed forecasting approaches using fast ensemble empirical model decomposition, genetic algorithm, Mind Evolutionary Algorithm and Artificial Neural Networks

Wind speed high-precision prediction is one of the most important technical aspects to protect the safety of wind power utilization. In this study, two new hybrid methods [FEEMD-MEA-MLP/FEEMD-GA-MLP] are proposed for the wind speed accurate multi-step predictions by combining FEEMD (Fast Ensemble Empirical Mode Decomposition), MEA (Mind Evolutionary Algorithm), GA (Genetic Algorithm) and MLP (Multi Layer Perceptron) neural networks. In these two hybrid methods, the FEEMD algorithm is adopted to decompose the original wind speed series into a number of sub-layers and the MLP neural networks optimized by the MEA algorithm and the GA algorithm are built to predict the decomposed wind speed sub-layers, respectively. The innovation of the study is to investigate the promoted percentages of the MLP neural networks by the FEEMD decomposition and the MEA/GA optimization, respectively. The involved forecasting models in the performance comparison in the study include the hybrid FEEMD-MEA-MLP, the hybrid FEEMD -GA-MLP, the hybrid FEEMD-MLP, the hybrid MEA-MLP, the hybrid GA-MLP and the single MLP. Two experimental results show that: (a) among all the involved methods, the hybrid FEEMD-MEA-MLP model has the best forecasting performance; (b) the FEEMD algorithm promotes the performance of the MLP neural networks significantly while the MEA/GA algorithms do not improve the performance of the MLP neural networks significantly; and (c) the hybrid FEEMD-MEA-MLP method and the hybrid FEEMD-GA-MLP method are both effective in the wind speed high-precision predictions.

Four wind speed multi-step forecasting models using extreme learning machines and signal decomposing algorithms

Realization of accurate wind speed forecasting is important to guarantee the safety of wind power utilization. In this paper, a new hybrid forecasting architecture is proposed to realize the wind speed accurate forecasting. In this architecture, four different hybrid models are presented by combining four signal decomposing algorithms (e.g., Wavelet Decomposition/Wavelet Packet Decomposition/Empirical Mode Decomposition/Fast Ensemble Empirical Mode Decomposition) and Extreme Learning Machines. The originality of the study is to investigate the promoted percentages of the Extreme Learning Machines by those mainstream signal decomposing algorithms in the multiple step wind speed forecasting. The results of two forecasting experiments indicate that: (1) the method of Extreme Learning Machines is suitable for the wind speed forecasting; (2) by utilizing the decomposing algorithms, all the proposed hybrid algorithms have better performance than the single Extreme Learning Machines; (3) in the comparisons of the decomposing algorithms in the proposed hybrid architecture, the Fast Ensemble Empirical Mode Decomposition has the best performance in the three-step forecasting results while the Wavelet Packet Decomposition has the best performance in the one and two step forecasting results. At the same time, the Wavelet Packet Decomposition and the Fast Ensemble Empirical Mode Decomposition are better than the Wavelet Decomposition and the Empirical Mode Decomposition in all the step predictions, respectively; and (4) the proposed algorithms are effective in the wind speed accurate predictions.

Comparison of new hybrid FEEMD-MLP, FEEMD-ANFIS, Wavelet Packet-MLP and Wavelet Packet-ANFIS for wind speed predictions

The technology of wind speed prediction is important to guarantee the safety of wind power utilization. Compared to the single algorithms, the hybrid ones always have better performance in the wind speed predictions. In this paper, three most important decomposing algorithms [Wavelet Decomposition – WD/Wavelet Packet Decomposition – WPD/Empirical Mode Decomposition – EMD] and a latest decomposing algorithm [Fast Ensemble Empirical Mode Decomposition – FEEMD] are all adopted to realize the wind speed high-precision predictions with two representative networks [MLP Neural Network/ANFIS Neural Network]. Based on the hybrid forecasting framework, two new wind speed forecasting methods [FEEMD-MLP and FEEMD-ANFIS] are proposed. Additionally, a series of performance comparison is provided, which includes EMD-MLP, FEEMD-MLP, EDM-ANFIS, FEEMD-ANFIS, WD-MLP, WD-ANFIS, WPD-MLP and WPD-ANFIS. The aim of the study is to investigate the decomposing and forecasting performance of the different hybrid models. Two experimental results show that: (1) Due to the inclusion of the decomposing algorithms, the hybrid ANN algorithms have better performance than their corresponding single ANN algorithms; (2) the proposed new FEEMD-MLP hybrid model has the best performance in the three-step predictions while the WPD-MLP hybrid model has the best performance in the one-step predictions; (3) among the decomposing algorithms, the FEEMD and WPD have better performance than the EMD and WD, respectively; (4) in the forecasting neural networks, the MLP has better performance than the ANFIS; and (5) all of the proposed hybrid algorithms are suitable for the wind speed predictions.

Using Peano–Hilbert space filling curves for fast bidimensional ensemble EMD realization

Empirical mode decomposition (EMD) is a fully unsupervised and data-driven approach to the class of nonlinear and non-stationary signals. A new approach is proposed, namely PHEEMD, to image analysis by using Peano–Hilbert space filling curves to transform 2D data (image) into 1D data, followed by ensemble EMD (EEMD) analysis, i.e., a more robust realization of EMD based on white noise excitation. Tests’ results have shown that PHEEMD exhibits a substantially reduced computational cost compared to other 2D-EMD approaches, preserving, simultaneously, the information lying at the EMD domain; hence, new perspectives for its use in low computational power devices, like portable applications, are feasible.