Ensemble Filter Research Articles

The Efficiency of Aggregation Methods in Ensemble Filter Feature Selection Models

Ensemble feature selection is recommended as it proves to produce a more stable subset of features and a better classification accuracy when compared to the individual feature selection methods. In this approach, the output of feature selection methods, called base selectors, are combined using some aggregation methods. For filter feature selection methods, a list aggregation method is needed to aggregate output ranked lists into a single list, and since many list aggregation methods have been proposed the decision on which method to use to build the optimum ensemble model is a de facto question.
 In this paper, we investigate the efficiency of four aggregation methods, namely; Min, Median, Arithmetic Mean, and Geometric Mean. The performance of aggregation methods is evaluated using five datasets from different scientific fields with a variant number of instances and features. Besides, the classifies used in the evaluation are selected from three different classes, Trees, Rules, and Bayes.
 The experimental results show that 11 out of the 15 best performance results are corresponding to ensemble models. And out of the 11 best performance ensemble models, the most efficient aggregation methods are Median (5/11), followed by Arithmetic Mean (3/11) and Min (3/11). Also, results show that as the number of features increased, the efficient aggregation method changes from Min to Median to Arithmetic Mean. This may suggest that for a very high number of features the efficient aggregation method is the Arithmetic Mean. And generally, there is no aggregation method that is the best for all cases.

Machine learning techniques to construct patched analog ensembles for data assimilation

Using generative models from the machine learning literature to create artificial ensemble members for use within data assimilation schemes has been introduced in Grooms (2021) [1] as constructed analog ensemble optimal interpolation (cAnEnOI). Specifically, we study general and variational autoencoders for the machine learning component of this method, and combine the ideas of constructed analogs and ensemble optimal interpolation in the data assimilation piece. To extend the scalability of cAnEnOI for use in data assimilation on complex dynamical models, we propose using patching schemes to divide the global spatial domain into digestible chunks. Using patches makes training the generative models possible and has the added benefit of being able to exploit parallelism during the generative step. Testing this new algorithm on a 1D toy model, we find that larger patch sizes make it harder to train an accurate generative model (i.e. a model whose reconstruction error is small), while conversely the data assimilation performance improves at larger patch sizes. There is thus a sweet spot where the patch size is large enough to enable good data assimilation performance, but not so large that it becomes difficult to train an accurate generative model. In our tests the new patched cAnEnOI method outperforms the original (unpatched) cAnEnOI, as well as the ensemble square root filter results from [1].

Ensemble Based Filter Feature Selection with Harmonize Particle Swarm Optimization and Support Vector Machine for Optimal Cancer Classification

Explosive increase of dataset features may intensify the complexity of medical data analysis in deciding necessary treatment for the patient. In most cases, the accuracy of diagnosis system is vitally impacted by the data dimensionality and classifier parameters. Since these two processes are dependent, conducting them independently could deteriorate the accuracy performance. Filter algorithm is used to eliminate irrelevant features based on ranking. However, independent filter still incapable to consider features dependency and resulting in imbalance selection of significant features which consequently degrade the classification performance. In order to mitigate this problem, ensemble of multi filters algorithm such as Information Gain (IG), Gain Ratio (GR), Chi-squared (CS) and Relief-F (RF) are utilized as it can considers the intercorrelation between features. The proper kernel parameters settings may also influence the classification performance. Hence, a harmonize classification technique using Particle Swarm Optimization (PSO) and Support Vector Machine (SVM) is employed to optimize the searching of optimal significant features and kernel parameters synchronously without degrading the accuracy. Therefore, an ensemble filter feature selection with harmonize classification of PSO and SVM (Ensemble-PSO-SVM) are proposed in this research. The effectiveness of the proposed method is examined on standard Breast Cancer and Lymphography datasets. Experimental results showed that the proposed method successfully signify the classifier accuracy performance with optimal significant features compared to other existing methods such as PSO-SVM and classical SVM. Hence, the proposed method can be used as an alternative method for determining the optimal solution in handling high dimensional data.

The maximum likelihood ensemble filter for computational flame and fluid dynamics

Abstract The numerical solution of partial differential equations that govern fluid dynamics with turbulence and combustion is challenging due to the multiscale nature of the dynamical system and the need to resolve small-scale physical features. In addition, the uncertainties in the dynamical system, including those in the physical models and parameters, initial and boundary conditions and numerical methods, impact the computational fluid dynamics (CFD) prediction of turbulence and chemical reactions. To improve the CFD prediction, this study focuses on the development and application of a maximum likelihood ensemble filter (MLEF), an ensemble-based data assimilation (DA), for flows featuring combustion and/or turbulence. MLEF finds the optimal analysis and its uncertainty by maximizing the posterior probability density function. The novelty of the study lies in the combination of advanced DA and CFD methods for a new comprehensive application to predict engineering fluid dynamics. The study combines important aspects, including an ensemble-based DA with analysis and uncertainty estimation, an augmented control vector that simultaneously adjusts initial conditions and model empirical parameters and an application of DA to CFD modeling of combustion and flows with complex geometry. The DA performance is validated by a turbulent Couette flow. The new CFD–DA system is then applied to solve the time-evolving shear-layer mixing with methane-air combustion and the turbulent flow over a bluff-body geometry. Results demonstrate the improvement of estimates of model parameters and the uncertainty reduction in initial conditions (ICs) for CFD modeling of flames and flows by the MLEF method.

Data assimilation challenges posed by nonlinear operators: A comparative study of ensemble and variational filters and smoothers

AbstractThe ensemble Kalman Filter (EnKF) and the 4D variational method (4DVar) are the most commonly used filters and smoothers in atmospheric science. These methods typically approximate prior densities using a Gaussian and solve a linear system of equations for the posterior mean and covariance. Therefore, strongly nonlinear model dynamics and measurement operators can lead to bias in posterior estimates. To improve the performance in nonlinear regimes, minimization of the 4DVar cost function typically follows multiple sets of iterations, known as an “outer loop”, which helps reduce bias caused by linear assumptions. Alternatively, “iterative ensemble methods” follow a similar strategy of periodically re-linearizing model and measurement operators. These methods come with different, possibly more appropriate, assumptions for drawing samples from the posterior density, but have seen little attention in numerical weather prediction (NWP) communities. Lastly, particle filters (PFs) present a purely Bayesian filtering approach for state estimation, which avoids many of the assumptions made by the above methods. Several strategies for applying localized PFs for NWP have been proposed very recently. The current study investigates intrinsic limitations of current data assimilation methodology for applications that require nonlinear measurement operators. In doing so, it targets a specific problem that is relevant to the assimilation of remotely-sensed measurements, such as radar reflectivity and all-sky radiances, which pose challenges for Gaussian-based data assimilation systems. This comparison includes multiple data assimilation approaches designed recently for nonlinear/non-Gaussian applications, as well as those currently used for NWP.

Assimilation of the Maximum Vertical Velocity Converted From Total Lightning Data Through the EnSRF Method

AbstractA total lightning data assimilation (LDA) scheme is developed at the cloud‐resolving scale in this study. The LDA scheme assimilates total lightning data through the ensemble square root filter (EnSRF) method based on the relationship between the maximum vertical velocity and the instantaneous flash rate. To verify the effect of LDA, three assimilation experiments based on a convective activity on July 6, 2019 are performed. The results show that all LDA experiments can improve the forecast. LDA improves the water vapor field and provides a warm and moist environment in which convection can develop. LDA can also increase the convergence at low levels and the divergence at upper levels, which leads to intense convection development. However, the LDA experiment that does not consider vertical localization (LDA_no) shows excessive adjustments of the model variables, which leads to overestimation of precipitation. The LDA experiment which adjusts the state variables in the charge zone (LDA_CZ) underestimates the forecast. The LDA experiment using a global group filter (GGF) localization function (LDA_GGF) is optimal in theory and produces a better forecast. The results of LDA_GGF show that the interaction between the cold pool and the strong low‐level vertical wind shear in front of the cold pool is beneficial for triggering new convection in front of the convection, which leads to a rapid shifting in convective activity. In addition, the LDA_GGF scheme is evaluated further for a rainfall process on August 1, 2020 and obtains similar forecast improvements for composite reflectivity and accumulated precipitation.

Adaptive Region Proposal With Channel Regularization for Robust Object Tracking

In this paper, we propose an adaptive region proposal scheme with feature channel regularization to facilitate robust object tracking. We consider tracking as a linear regression problem and an ensemble of correlation filters is trained on-line to distinguish the foreground target from the background. Further, we integrate adaptively learned region proposals into an enhanced two-stream tracking framework based on correlation filters. For the tracking stream, we learn two-stage cascade correlation filters on deep convolutional features to ensure competitive tracking performance. For the detection stream, we employ adaptive region proposals, which are effective in recovering target objects from tracking failures caused by heavy occlusion or out-of-view movement. In contrast to traditional tracking-by-detection methods using random samples or sliding windows, we perform target re-detection over adaptively learned region proposals. Since region proposals naturally take the objectness information into account, we show that the proposed adaptive region proposals can handle the challenging scale estimation problem as well. In addition, we observe the channel redundancy and noisy of feature representation, especially for the convolutional features. Thus, we apply a channel regularization to the correlation filter learning. Extensive experimental validations on OTB, VOT and UAV-123 datasets demonstrate that the proposed method performs favorably against state-of-the-art tracking algorithms.

Texture defect classification with multiple pooling and filter ensemble based on deep neural network

Fabric quality control is one of the most important phases of production in order to ensure high-quality standards in the fabric production sector. For this reason, the development of successful automatic quality control systems has been a very important research subject. In this study, we propose a Multiple Pooling and Filter approach based on a Deep Neural Network (MPF-DNN) for the classification of texture defects. This model consists of three basic stages: preprocessing, feature extraction, and classification. In the preprocessing stage, the texture images were first divided into n × n equal parts. Then, median filtering and pooling processes were applied to each piece prior to performing image merging. In the proposed pre-treatment stage, it is aimed to clarify fabric errors and increase performance. For the feature extraction stage, deep features were extracted from the texture images using the pretrained ResNet101 model based on the transfer learning approach. Finally, classification and testing procedures were conducted on the obtained deep-effective properties using the SVM method. The multiclass TILDA dataset was used in order to test the proposed model. In experimental work, the MPF-DNN model for all four classes achieved a significant overall accuracy score of 95.82%. In the results obtained from extensive experimental studies, it was observed that the proposed MPF-DNN model was more successful than previous studies that used pretrained deep architectures.

Hyperspectral Image Classification via Discriminant Gabor Ensemble Filter.

For a broad range of applications, hyperspectral image (HSI) classification is a hot topic in remote sensing, and convolutional neural network (CNN)-based methods are drawing increasing attention. However, to train millions of parameters in CNN requires a large number of labeled training samples, which are difficult to collect. A conventional Gabor filter can effectively extract spatial information with different scales and orientations without training, but it may be missing some important discriminative information. In this article, we propose the Gabor ensemble filter (GEF), a new convolutional filter to extract deep features for HSI with fewer trainable parameters. GEF filters each input channel by some fixed Gabor filters and learnable filters simultaneously, then reduces the dimensions by some learnable 1×1 filters to generate the output channels. The fixed Gabor filters can extract common features with different scales and orientations, while the learnable filters can learn some complementary features that Gabor filters cannot extract. Based on GEF, we design a network architecture for HSI classification, which extracts deep features and can learn from limited training samples. In order to simultaneously learn more discriminative features and an end-to-end system, we propose to introduce the local discriminant structure for cross-entropy loss by combining the triplet hard loss. Results of experiments on three HSI datasets show that the proposed method has significantly higher classification accuracy than other state-of-the-art methods. Moreover, the proposed method is speedy for both training and testing.

A Computationally Efficient Ensemble Filtering Scheme for Quantitative Volcanic Ash Forecasts

AbstractA method of assimilating satellite observations in quantitative ensemble forecasting models of airborne volcanic ash is presented in this study. The method employs many trial dispersion model simulations that are generated by both deterministic and random perturbations of the source term and use of an ensemble of numerical weather prediction model fields. An ensemble filter is then applied to the trial simulations, which are either selected or rejected by the filter based on their degree of agreement with observations within a specified time window. The observations may be in the form of quantitative satellite retrieved mass load fields or qualitative ash detection fields, which means that useful results can be obtained even when retrievals are not available in real time provided that the ash boundaries can be identified. The filtering process is repeated several times with different random realizations of the source term to reduce sampling error and minimize filter degeneracy, a phenomenon that plagues all ensemble filter models. The selected members are then propagated forward in time beyond the observational time window to form the forecast ensemble. We show, using several eruption case studies, that forecast ensembles constructed in this way are generally superior in skill to reference forecasts that do not assimilate observations.

Towards Automated Eye Diagnosis: An Improved Retinal Vessel Segmentation Framework Using Ensemble Block Matching 3D Filter.

Automated detection of vision threatening eye disease based on high resolution retinal fundus images requires accurate segmentation of the blood vessels. In this regard, detection and segmentation of finer vessels, which are obscured by a considerable degree of noise and poor illumination, is particularly challenging. These noises include (systematic) additive noise and multiplicative (speckle) noise, which arise due to various practical limitations of the fundus imaging systems. To address this inherent issue, we present an efficient unsupervised vessel segmentation strategy as a step towards accurate classification of eye diseases from the noisy fundus images. To that end, an ensemble block matching 3D (BM3D) speckle filter is proposed for removal of unwanted noise leading to improved detection. The BM3D-speckle filter, despite its ability to recover finer details (i.e., vessels in fundus images), yields a pattern of checkerboard artifacts in the aftermath of multiplicative (speckle) noise removal. These artifacts are generally ignored in the case of satellite images; however, in the case of fundus images, these artifacts have a degenerating effect on the segmentation or detection of fine vessels. To counter that, an ensemble of BM3D-speckle filter is proposed to suppress these artifacts while further sharpening the recovered vessels. This is subsequently used to devise an improved unsupervised segmentation strategy that can detect fine vessels even in the presence of dominant noise and yields an overall much improved accuracy. Testing was carried out on three publicly available databases namely Structured Analysis of the Retina (STARE), Digital Retinal Images for Vessel Extraction (DRIVE) and CHASE_DB1. We have achieved a sensitivity of 82.88, 81.41 and 82.03 on DRIVE, SATARE, and CHASE_DB1, respectively. The accuracy is also boosted to 95.41, 95.70 and 95.61 on DRIVE, SATARE, and CHASE_DB1, respectively. The performance of the proposed methods on images with pathologies was observed to be more convincing than the performance of similar state-of-the-art methods.

A deep learning‐based ensemble filter for nonlinear data assimilation

AbstractThis article presents a novel deep learning‐based ensemble conditional mean filter (DL‐EnCMF) for nonlinear data assimilation. The filter's key component is the approximation of the conditional expectation (CE) using deep neural networks (DNNs). We implement the DL‐EnCMF for tracking the states of the Lorenz‐63 system. Numerical results show that the DL‐EnCMF outperforms the ensemble Kalman filter (EnKF)—a common technique for data assimilation.

On the continuous time limit of ensemble square root filters

We provide a continuous time limit analysis for the class of Ensemble Square Root Filter algorithms with deterministic model perturbations. In the particular linear case, we specify general conditions on the model perturbations implying convergence of the empirical mean and covariance matrix towards their respective counterparts of the Kalman-Bucy Filter. As a second main result we identify additional assumptions for the convergence of the whole ensemble towards solutions of the Ensemble Kalman-Bucy filtering equations introduced in [1]. The latter result can be generalized to nonlinear Lipschitz-continuous model operators. A striking implication of our results is the fact that the limiting equations for the ensemble members are universal for a large class of Ensemble Square Root Filters. This yields a mathematically rigorous justification for the analysis of these algorithms with the help of the Ensemble Kalman-Bucy Filter. [1] de Wiljes, Jana, Reich, Sebastian, Stannat, Wilhelm, Long-Time Stability and Accuracy of the Ensemble Kalman-Bucy Filter for Fully Observed Processes and Small Measurement Noise. SIAM Journal on Applied Dynamical Systems, Vol. 17, No. 2, 1152-1181, 2018

A recursive ensemble-based feature selection for multi-output models to discover patterns among the soil nutrients

Finding the reduced and the relevant subsets of the predictors is inevitable when it comes to predictive modelling. If the datasets involved are heterogeneous and heteroscedastic in nature such as the soil samples, the task turns out to be trickier and such scenarios demand ensemble-based feature selection approach. The proposed algorithm uses an ensemble of filter (RReliefF), wrapper (Adaptive Plus –l and Minus –r), and embedded (Neighbourhood Component Analysis) approaches and applies it to the datasets in a heterogeneous and homogeneous manner. The Adaptive Plus –l and Minus –r is an experiment done on the Plus –l and Minus –r wrapper method to enhance the performance of the algorithm. The proposed combination rule of the ensemble filters out the irrelevant predictors for each response variable. Further, this ensemble is recursively implemented using the floating set of predictors to estimate the optimal subsets for multiple response variables at one go. Each step in recursion ensures that only the best subset of features (in terms of length and weights) among the current and previous iterations is retained. The Akaikes Information Criteria and the predictors weights calculated, assessed the efficiency of the resultant predictor sets of the recursive ensemble. The adjusted R2, Median Absolute Deviation and Root Mean Square Error on the unseen datasets confirmed the suitability of the same for predictive modelling in ecological domains.

Exploring the direct influence of control parameters of experimental facility for fuel cells based on improved generalized regression neural network

The exploration of the control parameters for experimental facility in terms of their coupling effects can provide direct guidance for experimental research of fuel cells. In this study, a model based on an improved generalized regression neural network (GRNN) is proposed to study the coupling effect of control parameters that can be set and adjusted directly during the experiment. In addition, the expectation-maximization algorithm and the ensemble filter algorithm are introduced to preprocess data; the adaptive learning factor and the differential evolution strategy are introduced to improve the GRNN. Moreover, a moment-independent global sensitivity analysis is introduced to apply for sensitivity analysis by the dimensional reduction technique and principle of maximum entropy. The results indicated that the improved model can accurately predict performance in a dynamic environment; the relative error was <1.7%, the prediction errors were, respectively, reduced by 35.61%, 19.78%, 60.99%, and 37.55% as compared with the GRNN model, demonstrating the effectiveness of the improvement strategy; the back pressure and inlet temperatures, dew point, and cell temperatures were classified as sensitive factors; when the anodic back pressure was approximately 1 atm, the cathodic back pressure had little influence, the performance conforms to the maximum level.

Sequential Geoacoustic Inversion Using an Improved Kalman Particle Filter

Geoacoustic inversion is an efficient method to study the physical properties and structure of ocean bottom while sequential geoacoustic inversion is a challenging task due to the complexity and non-linearity of the underwater environment. In this paper, an ensemble Kalman Particle filter is described to address the sequential geoacoustic inversion problem of range-dependent environment in shallow water. This filter combines the advantages of Particle filter and ensemble Kalman filter so its ability of tracking dynamical geoacoustic parameters is improved. The proposed filtering method is demonstrated with simulated data in a changing oceanic environment and outperforms Particle filter and ensemble Kalman filter. This method is also tested in sea-trial data collected from a shallow-water experiment in the East China Sea. The inverted sound speed in sediment is consistent with in situ measurement and the error between transmission loss predicted by inverted parameters, and the experimental transmission loss is small.

Improving parameter and state estimation of a hydrological model with the ensemble square root filter

Data assimilation techniques are widely used in hydrology and water resources management to improve model forecast uncertainty by assimilating observations. The big challenge in practical applications is how to describe model uncertainties correctly to avoid the occurrence of spurious covariance during data assimilation. In this study, the ensemble square root filter (EnSRF) is used to estimate parameters and states of a groundwater model in Guantao, China, which updates ensemble means and perturbations separately and avoids the need to perturb observations. The uncertainty in parameters and states decreased with time while assimilating observations. However, incorrect updates of parameters and states were obtained, which could not be corrected by assimilating further observations improving the representation of the hydrological system. To compensate for this effect and reduce other sampling errors introduced during assimilation, localization and two covariance-tuning methods (inflation factor and damping factor) are explored in the study. The results show that alternative scenarios with proper localization length or a large inflation factor or a small damping factor produce better model estimates and improve the filter performance. The scenario with a damping factor of 0.05 shows a distinct gain in model predictive capability. The damping factor method is superior to the inflation factor method and preferable in real field applications. The scenario combining the damping factor with localization further improved the filter performance. The performance of the EnSRF with respect to different amounts of measurement error is also analysed. Even though the increase of observation error can increase the error covariance, a corresponding filter improvement is not observed as in that case, observations are less informative.

Analog ensemble data assimilation and a method for constructing analogs with variational autoencoders

AbstractIt is proposed to use analogs of the forecast mean to generate an ensemble of perturbations for use in ensemble optimal interpolation (EnOI) or ensemble variational (EnVar) methods. A new method of constructing analogs using variational autoencoders (VAEs; a machine learning method) is proposed. The resulting analog methods using analogs from a catalog (AnEnOI), and using constructed analogs (cAnEnOI), are tested in the context of a multiscale Lorenz‐‘96 model, with standard EnOI and an ensemble square root filter for comparison. The use of analogs from a modestly‐sized catalog is shown to improve the performance of EnOI, with limited marginal improvements resulting from increases in the catalog size. The method using constructed analogs (cAnEnOI) is found to perform as well as a full ensemble square root filter, and to be robust over a wide range of tuning parameters.

A Novel Class Noise Detection Method for High-Dimensional Data in Industrial Informatics

The data in industrial informatics may be high-dimensional and mislabeled. Irrelevant or noisy features pose a significant challenge to the detection of high-dimensional mislabeling. The traditional method usually adopts a two-step solution, first finding the relevant subspace and then using it for mislabeling detection. This two-step method struggles to provide the optimal mislabeling detection performance, since it separates the procedures of feature selection and label error detection. To solve this problem, in this article, we integrate the two steps and propose a sequential ensemble noise filter (SENF). In the SENF, relevant features are selected and used to generate a noise score for each instance. Continuously, these noise scores guide feature selection in the regression learning. Thus, the SENF falls in the scope of sequential ensemble learning. We evaluate our approach on several benchmark datasets with high dimensionality and much label noise. It is shown that the SENF is significantly better than other existing label noise detection methods.

Assimilating Observations with Spatially Correlated Errors Using a Serial Ensemble Filter with a Multiscale Approach

AbstractThe serial ensemble square root filter (EnSRF) typically assumes observation errors to be uncorrelated when assimilating the observations one at a time. This assumption causes the filter solution to be suboptimal when the observation errors are spatially correlated. Using the Lorenz-96 model, this study evaluates the suboptimality due to mischaracterization of observation error spatial correlations. Neglecting spatial correlations in observation errors results in mismatches between the specified and true observation error variances in spectral space, which cannot be resolved by inflating the overall observation error variance. As a remedy, a multiscale observation (MSO) method is proposed to decompose the observations into multiple scale components and assimilate each component with separately adjusted spectral error variance. Experimental results using the Lorenz-96 model show that the serial EnSRF, with the help from the MSO method, can produce solutions that approach the solution from the EnSRF with correctly specified observation error correlations as the number of scale components increases. The MSO method is further tested in a two-layer quasigeostrophic (QG) model framework. In this case, the MSO method is combined with the multiscale localization (MSL) method to allow the use of different localization radii when updating the model state at different scales. The combined method (MSOL) improves the serial EnSRF performance when assimilating observations with spatially correlated errors. With adjusted observation error spectral variances and localization radii, the combined MSOL method provides the best solution in terms of analysis accuracy and filter consistency. Prospects and challenges are also discussed for the implementation of the MSO method for more complex models and observing networks.