Resampling Algorithm Research Articles

On the performance evaluation of a hierarchical-structure prototype product using inconsistent prior information and limited test data

Performance evaluation or parameter estimation for a hierarchical structure prototype product is an imperative yet difficult task. The challenge comes from both inaccurate prior knowledge and insufficient data acquisition at its early usage period. In this paper, we propose an innovative Bayesian inference framework for the uncertainty quantification of the performance of a newly developed product with hierarchical structure characteristics. We focus particularly on the presence of inconsistent prior distributions on one quantity. An adaptive Bayesian melding method is investigated to aggregate multi-source information in a flexible way. We realize this method via a modified sampling importance re-sampling algorithm. This algorithm is designed to substitute for the MCMC methods and is computationally efficient. We use a practical engineering case to validate our proposal.

Feature Selection and Rule Generation Integrated Learning for Takagi-Sugeno-Kang Fuzzy System and its Application in Medical Data Classification

The rule-based fuzzy systems have successfully applied for numerous medical data classification problems. However, structuring the concise and interpretable fuzzy rules with good classification performance is still a big challenge. To address this issue, a novel feature selection and rule generation integrated learning for Takagi-Sugeno-Kang fuzzy system (called FSRG-IL-TSK) in this paper. FSRG-IL-TSK represents feature selection, structure identification and parameter learning into a Bayesian model, and uses the sequential importance resampling (SIR) algorithm to obtain the optimal parameters simultaneously, including the optimal features for each fuzzy rule, number of rules, and antecedent/consequent parameter of rules. Due to an integrated learning mechanism, it can select a small set of useful features and obtain a small number of rules. The effectiveness and advantages of FSRG-IL-TSK are validated experimentally on real-world medical data classification tasks.

Predicting Class-Imbalanced Business Risk Using Resampling, Regularization, and Model Emsembling Algorithms

We aim at developing and improving the imbalanced business risk modeling via jointly using proper evaluation criteria, resampling, cross-validation, classifier regularization, and ensembling techniques. Area Under the Receiver Operating Characteristic Curve (AUC of ROC) is used for model comparison based on 10-fold cross validation. Two undersampling strategies including random undersampling (RUS) and cluster centroid undersampling (CCUS), as well as two oversampling methods including random oversampling (ROS) and Synthetic Minority Oversampling Technique (SMOTE), are applied. Three highly interpretable classifiers, including logistic regression without regularization (LR), L1-regularized LR (L1LR), and decision tree (DT) are implemented. Two ensembling techniques, including Bagging and Boosting, are applied on the DT classifier for further model improvement. The results show that, Boosting on DT by using the oversampled data containing 50% positives via SMOTE is the optimal model and it can achieve AUC, recall, and F1 score valued 0.8633, 0.9260, and 0.8907, respectively.

Improved Parallel Resampling Methods for Particle Filtering

Particle filter techniques are common methods used to estimate the evolving state of nonlinear, non-Gaussian time-variant systems by utilizing a periodic sequence of noisy measurements. The accuracy of particle filter methods has often been shown to be superior to other state estimation techniques, such as the extended Kalman filter (EKF), for many applications. Unfortunately, the high computational cost and highly nondeterministic runtime behavior of particle filters often preclude their use in hard, real-time environments, where filter response must meet the strict timing requirements of the application. Particle filter algorithms are composed of three main stages: prediction, update, and resampling. General purpose graphics processing units (GPGPUs) have been successfully employed in previous research to accelerate the computation of both the prediction and update stages by exploiting their natural fine-grain parallelism. This research focuses on accelerating the resampling stage for (GPGPU) execution, which has been much more difficult to parallelize due to it's apparent inherent sequentially. This paper introduces a novel GPGPU implementation of the systematic and stratified resampling algorithms that exploit the monotonically increasing nature of the prefix-sum and the evolutionary nature of the particle weighting process to allow the re-indexing portion of the algorithms to occur in a two-phase, multi-threaded manner. This resulting measured factor of performance improvement for the systematic and stratified algorithms was 15$\times$ and 32$\times$ , respectively, over the serial implementations.

Bidirectional self-adaptive resampling in internet of things big data learning

This paper focuses on the problem of low learning algorithm accuracy caused by serious imbalance of big data in Internet of Things, and proposes a bidirectional self-adaptive resampling algorithm for imbalanced big data. Based on the sizes of data sets and imbalance ratios inputted by the user, the algorithm will process the data using a combination of oversampling for minority class and distribution sensitive undersampling for majority class. This paper proposes a new distribution-sensitive resampling algorithm. According to the distribution of samples, the majority and minority samples are divided into different categories, and different processing methods are adopted for the samples with different distribution characteristics The algorithm makes the sample set after resampling keep the same characteristics with the original data set as much as possible. The algorithm emphasizes the importance of boundary samples, that is, the samples at the boundary of majority classes and minority classes are more important than other samples for learning algorithm. The boundary minority samples will be copied, and the boundary majority samples will be reserved. Real-world application is introduced in the end, which shows that compared with the existing imbalanced data resampling algorithms, this algorithm improves the accuracy of learning algorithm, especially for the accuracy and recall rate of minority class.

Assimilation of passive microwave AMSR-2 satellite observations in a snowpack evolution model over northeastern Canada

Abstract. Over northeastern Canada, the amount of water stored in a snowpack, estimated by its snow water equivalent (SWE) amount, is a key variable for hydrological applications. The limited number of weather stations driving snowpack models over large and remote northern areas generates great uncertainty in SWE evolution. A data assimilation (DA) scheme was developed to improve SWE estimates by updating meteorological forcing data and snowpack states with passive microwave (PMW) satellite observations and without using any surface-based data. In this DA experiment, a particle filter with a Sequential Importance Resampling algorithm (SIR) was applied and an inflation technique of the observation error matrix was developed to avoid ensemble degeneracy. Advanced Microwave Scanning Radiometer 2 (AMSR-2) brightness temperature (TB) observations were assimilated into a chain of models composed of the Crocus multilayer snowpack model and radiative transfer models. The microwave snow emission model (Dense Media Radiative Transfer – Multi-Layer model, DMRT-ML), the vegetation transmissivity model (ω-τopt), and atmospheric and soil radiative transfer models were calibrated to simulate the contributions from the snowpack, the vegetation, and the soil, respectively, at the top of the atmosphere. DA experiments were performed for 12 stations where daily continuous SWE measurements were acquired over 4 winters (2012–2016). Best SWE estimates are obtained with the assimilation of the TBs at 11, 19, and 37 GHz in vertical polarizations. The overall SWE bias is reduced by 68 % compared to the original SWE simulations, from 23.7 kg m−2 without assimilation to 7.5 kg m−2 with the assimilation of the three frequencies. The overall SWE relative percentage of error (RPE) is 14.1 % (19 % without assimilation) for sites with a fraction of forest cover below 75 %, which is in the range of accuracy needed for hydrological applications. This research opens the way for global applications to improve SWE estimates over large and remote areas, even when vegetation contributions are up to 50 % of the PMW signal.

Rainfall disaggregation for hydrological modeling: is there a need for spatial consistence?

Abstract. In this study, the influence of disaggregated rainfall products with different degrees of spatial consistence on rainfall–runoff modeling results is analyzed for three mesoscale catchments in Lower Saxony, Germany. For the disaggregation of daily rainfall time series into hourly values, a multiplicative random cascade model is applied. The disaggregation is applied on a station by station basis without consideration of surrounding stations; hence subsequent steps are then required to implement spatial consistence. Spatial consistence is represented here by three bivariate spatial rainfall characteristics that complement each other. A resampling algorithm and a parallelization approach are evaluated against the disaggregated time series without any subsequent steps. With respect to rainfall, clear differences between these three approaches can be identified regarding bivariate spatial rainfall characteristics, areal rainfall intensities and extreme values. The resampled time series lead to the best agreement with the observed ones. Using these different rainfall products as input to hydrological modeling, we hypothesize that derived runoff statistics – with emphasis on seasonal extreme values – are subject to similar differences as well. However, an impact on the extreme values' statistics of the hydrological simulations forced by different rainfall approaches cannot be detected. Several modifications of the study design using rainfall–runoff models with and without parameter calibration or using different rain gauge densities lead to similar results in runoff statistics. Only if the spatially highly resolved rainfall–runoff WaSiM model is applied instead of the semi-distributed HBV-IWW model can slight differences regarding the seasonal peak flows be identified. Hence, the hypothesis formulated before is rejected in this case study. These findings suggest that (i) simple model structures might compensate for deficiencies in spatial representativeness through parameterization and (ii) highly resolved hydrological models benefit from improved spatial modeling of rainfall.

On bootstrap testing inference in cure rate models

ABSTRACTSurvival models deal with the time until the occurrence of an event of interest. However, in some situations the event may not occur in part of the studied population. The fraction of the population that will never experience the event of interest is generally called cure rate. Models that consider this fact (cure rate models) have been extensively studied in the literature. Hypothesis testing on the parameters of these models can be performed based on likelihood ratio, gradient, score or Wald statistics. Critical values of these tests are obtained through approximations that are valid in large samples and may result in size distortion in small or moderate sample sizes. In this sense, this paper proposes bootstrap corrections to the four mentioned tests and bootstrap Bartlett correction for the likelihood ratio statistic in the Weibull promotion time model. Besides, we present an algorithm for bootstrap resampling when the data presents cure fraction and right censoring time (random and non-informative). Simulation studies are conducted to compare the finite sample performances of the corrected tests. The numerical evidence favours the corrected tests we propose. We also present an application in an actual data set.

A holistic comparison of the different resampling algorithms for particle filter based prognosis using lithium ion batteries as a case study

Prognostic health management (PHM) is a critical and essential aspect of any robust maintenance program in the manufacturing industry for early failure detection and prediction of the remaining useful life (RUL) for the entire system or for a component (sub-system) whose condition is being monitored in real-time. In recent years, a lot of research has been done on developing better performing prognostic algorithms for RUL prediction with the “particle filter (PF)” framework being the most widely used amongst them. To address the problems of particle degeneracy and particle impoverishment, several adaptations of standard particle filters have been proposed by improvising the resampling strategies. However, the efficacy of these algorithms is assessed only under specific conditions involving relatively clean degradation data (low noise), large training data sets and limited degradation patterns (mostly linear or “almost” linear). The purpose of this study is to make a comparison of four most frequently used resampling strategies: Multinomial resampling, Stratified resampling, Systematic resampling and Residual Systematic resampling for lithium-ion battery RUL prediction. They are similar in terms of operation but differ only in the way the ordered sequence of random numbers is generated for resampling thus enabling a standardized comparison in terms of computational complexity of O(N). The robustness of these resampling techniques is tested by adding 50 dB of noise to the measurement data and by considering three different time instants at different stages of the device lifecycle for prediction with different amount of training data. We use the mean squared deviation (MSD), relative accuracy (RA), execution time and the α – λ plot as the performance metrics for comparing the effectiveness of the different resampling techniques. Our analysis shows that the residual systematic resampling algorithm is the most preferred approach considering the reasonable accuracy and short computational time.

Optimization of Particle CBMeMBer Filters for Hardware Implementation

It is a promising solution for real-time multitarget tracking to implement particle cardinality balanced multitarget multi-Bernoulli (CBMeMBer) filters in hardware platforms. However, this solution is difficult to materialize since there is a contradiction between the time-varying number of Bernoulli intensity components in CBMeMBer filters and the limited number of particles in hardware platforms. Moreover, real-time hardware implementation requires a resampling procedure that is suitable for parallel processing, while the existing parallel resampling algorithms oversimplify this procedure, resulting in estimation performance degradation. In this paper, we propose an optimization algorithm of particle allocation to overcome the above-mentioned contradiction, and a parallel resampling algorithm to improve the estimation performance. Numerical experiments demonstrate the effectiveness of the proposed algorithms in multitarget tracking.

Optimization of the Spectrum of Digital Diagnostic Signals to Improve the Analysis of Harmonic Parameters Using Resampling Algorithms

The paper discusses the influence of resampling distortions on the quality of spectral resolution optimization in diagnostic signals digitally recorded for objects in a steady state. Analysis of harmonic parameters and detection of foreign frequencies, which are most often interpreted as fault results, may be problematic because of the spectral leakage effect. The reason is that some frequencies which are not actually present in the signal can be observed in the DFT (discrete Fourier transform) vector. In addition, actual existing frequencies may be distorted. The use of window functions most often reduces the effect of spectral leakage, but also increases harmonic distortion. When the signal contains only the fundamental frequency and harmonics, it is possible to adjust its spectral resolution to eliminate any distortion for regular frequencies. If the algorithms which perform the analysis of the diagnostic signal require a fixed number samples, high quality resampling should be used.

Leveraging sentiment analysis at the aspects level to predict ratings of reviews

Online reviews are an important asset for users who are deciding to buy a product, see a movie, or go to a restaurant and for managers who are making business decisions. The reviews from e-commerce websites are usually attached to ratings, which facilitates learning from the reviews by users. However, many reviews that spread across forums or social media are written in plain text, which is not rated, and these reviews are called non-rated reviews in this paper. From the perspective of sentiment analysis at the aspects level, this study develops a predictive framework for calculating ratings for non-rated reviews. The idea behind the framework began with an observation: the sentiment of an aspect is determined by its context; the rating of the review depends on the sentiment of the aspects and the number of positive and negative aspects in the review. Viewing term pairs that co-occur with aspects as their context, we conceived of a variant of a Conditional Random Field model, called SentiCRF, for generating term pairs and calculating their sentiment scores from a training set. Then, we developed a cumulative logit model that uses aspects and their sentiments in a review to predict the ratings of the review. In addition, we met the challenge of class imbalance when calculating the sentiment scores of term pairs. We also conceived of a heuristic re-sampling algorithm to tackle class imbalance. Experiments were conducted on the Yelp dataset, and their results demonstrate that the predictive framework is feasible and effective at predicting the ratings of reviews.

Deep Learning-Based Classification and Reconstruction of Residential Scenes From Large-Scale Point Clouds

The reconstruction of urban buildings from large-scale airborne laser scanning point clouds is an important research topic in the geoscience field. Large-scale urban scenes usually contain a large number of object categories and many overlapped or closely neighboring objects, which poses great challenges for classifying and modeling buildings from these data sets. In this paper, we propose a deep reinforcement learning framework that integrates a 3-D convolutional neural network, a deep Q-network, and a residual recurrent neural network for the efficient semantic parsing of large-scale 3-D point clouds. The proposed framework provides an end-to-end automatic processing method that maps the raw point cloud to the classification results of the given categories. After obtaining the building classes, we utilize an edge-aware resampling algorithm to consolidate the point set with noise-free normals and clean preservation of sharp features. Finally, 2.5-D dual contouring, which is a data-driven approach, is introduced to generate urban building models from the consolidated point clouds. Our method can generate lightweight building models with arbitrarily shaped roofs while preserving the verticality of connecting walls.

An improved particle filtering-based approach for health prediction and prognosis of nonlinear systems

Health monitoring of nonlinear systems is broadly concerned with the system health tracking and its prediction to future time horizons. Estimation and prediction schemes constitute as principle components of any health monitoring technique. Particle filter (PF) represents a powerful tool for performing state and parameter estimation as well as prediction of nonlinear dynamical systems. Estimation of the system parameters along with the states can yield an up-to-date and reliable model that can be used for long-term prediction problems through utilization of particle filters. This feature enables one to deal with uncertainty issues in the resulting prediction step as the time horizon is extended. Towards this end, this paper presents an improved method to achieve uncertainty management for long-term prediction of nonlinear systems by using particle filters. In our proposed approach, an observation forecasting scheme is developed to extend the system observation profiles (as time-series) to future time horizon. Particles are then propagated to future time instants according to a resampling algorithm instead of considering constant weights for the particles propagation in the prediction step. The uncertainty in the long-term prediction of the system states and parameters are managed by utilizing dynamic linear models for development of an observation forecasting scheme. This task is addressed through an outer adjustment loop for adaptively changing the sliding observation injection window based on the Mahalanobis distance criterion. Our proposed approach is then applied to predicting the health condition as well as the remaining useful life (RUL) of a gas turbine engine that is affected by degradations in the system health parameters. Extensive simulation and case studies are conducted to demonstrate and illustrate the capabilities and performance characteristics of our proposed and developed schemes.

Rotor-Current-Based Fault Diagnosis for DFIG Wind Turbine Drivetrain Gearboxes Using Frequency Analysis and a Deep Classifier

Fault diagnosis of drivetrain gearboxes is a prominent challenge in wind turbine condition monitoring. Many machine learning algorithms have been applied to gearbox fault diagnosis. However, many of the current machine learning algorithms did not provide satisfactory fault diagnosis results due to their shallow architectures. Recently, a class of machine learning models with deep architectures called deep learning has received more attention, because it can learn high-level features of inputs. This paper proposes a new fault diagnosis method for the drivetrain gearboxes of the wind turbines equipped with doubly-fed induction generators (DFIGs) using DFIG rotor current signal analysis. In the proposed method, the instantaneous fundamental frequency of the rotor current signal is first estimated to obtain the instantaneous shaft rotating frequency. Then, the Hilbert transform is used to demodulate the rotor current signal to obtain its envelope, and the resultant envelope signal contains fault characteristic frequencies that are in proportion to the varying DFIG shaft rotating frequency. Next, an angular resampling algorithm is designed to resample the nonstationary envelope signal to be stationary based on the estimated instantaneous shaft rotating frequency. After that, the power spectral density analysis is performed on the resampled envelope signal for the gearbox fault detection. Finally, a classifier with a deep architecture that consists of a stacked autoencoder and a support vector machine is proposed for gearbox fault classification using extracted fault features. Experimental results obtained from a DFIG wind turbine drivetrain test rig are provided to verify the effectiveness of the proposed method.

An improved unscented particle filter approach for lithium-ion battery remaining useful life prediction

Lithium-ion rechargeable batteries are widely used as power sources for mobile phones, laptops and electric cars, and gradually extended to military communication, navigation, aviation, aerospace and other fields. Accurate remaining useful life (RUL) prediction of lithium-ion battery plays an important role in avoiding serious security and economic consequences caused by failure to supply required power levels. Thus, the RUL prediction for lithium-ion battery has become a critical task in engineering practices. With its superiority in handling nonlinear and non-Gaussian system behaviors, the particle filtering (PF) technique is widely used in the remaining life prediction. However, the choice of importance function and the degradation of diversity in sampling particles limit the estimation accuracy. This paper presents an improved PF algorithm, that is, the unscented particle filter (UPF) based on linear optimizing combination resampling (U-LOCR-PF) to improve the prediction accuracy. In one aspect, the unscented Kalman filter (UKF) is used to generate a proposal distribution as an importance function for particle filtering. In the other aspect, the linear optimizing combination resampling (LOCR) algorithm is used to overcome the particle diversity deficiency. It should be noted that the step coefficient K can affect the performance of LOCR algorithm, and the fuzzy inference system is applied to determine the value of step coefficient K. According to the analysis results, it can be seen that the proposed prognostic method shows higher accuracy in the RUL prediction of lithium-ion battery, compared with the existing PF-based and UPF-based prognostic methods.

An Enhanced Mobile Device-Based Navigation Model

This article formulated, simulated and evaluated an enhanced model for a non-linear/non-Gaussian integrated Global Positioning System (GPS) and Inertial Navigation System (INS) mobile-based device navigation system using the Particle Filter (PF). This was with a view to enhancing the accuracy and minimize the delay experienced in the existing system which relies on linear data. An android driven Infinix X5 mobile device with GPS and INS-based sensors was used to implement the model formulated, standard Bayesian estimators were used to generate non-linear datasets with Gaussian/non-Gaussian white noises for mobile device based INS/GPS sensors. A mathematical model was formulated using Sampling Importance-weight Resampling (SIR) algorithm of the PF. The conceptual model was developed using Simulink and the design specification was done using Unified Modeling Language (UML). The model was simulated with MATLAB and the simulation results obtained were evaluated using standard metrics, and benchmarked with existing model. The overall results showed that the proposed model performed better than existing ones in term of accuracy. However, the model did not impact on delay reduction.

Adaptive particle filter for localization problem in service robotics

In this paper we present a statistical approach to the likelihood computation and adaptive resampling algorithm for particle filters using low cost ultrasonic sensors in the context of service robotics. This increases the efficiency of the particle filter in the Monte Carlo Localization problem by means of preventing sample impoverishment and ensuring it converges towards the most likely particle and simultaneously keeping less likely ones by systematic resampling. Proposed algorithms were developed in the ROS framework, simulation was done in Gazebo environment. Experiments using a differential drive mobile platform with 4 ultrasonic sensors in the office environment show that our approach provides strong improvement over particle filters with fixed sample sizes.

Computer-aided diagnosis of clinically significant prostate cancer from MRI images using sparse autoencoder and random forest classifier

A novel method to diagnose clinically significant prostate cancer (PCa) using Multi-parametric Magnetic Resonance Imaging (mpMRI) biomarkers in a highly imbalanced dataset is investigated in this paper. Transaxial T2 Weighted (T2W), Apparent Diffusion Coefficient (ADC) and high B-Value (BVAL) Diffusion-Weighted (DW) images obtained from PROSTATEx 2016 challenge dataset publicly available in TCIA (The Cancer Imaging Archive) is used for this study. High-level features are extracted using a single layer Sparse Autoencoder (SAE). Synthetic Minority Oversampling Technique (SMOTE), Weka Resample algorithm and Adaptive Synthetic (ADASYN) sampling approach are explored to solve the class-imbalance problem. The performance of various classifiers are also investigated and it was found that the data augmented using ADASYN followed by classification using random forest classifier achieved the best performance. It achieved an area under ROC curve of 0.979. It also reached a Cohen's kappa score of 0.873, an accuracy of 93.65% and F-Measure of 0.94 in distinguishing clinically significant PCa from indolent PCa.

Semi-Independent Resampling for Particle Filtering

Among Sequential Monte Carlo (SMC) methods,Sampling Importance Resampling (SIR) algorithms are based on Importance Sampling (IS) and on some resampling-based)rejuvenation algorithm which aims at fighting against weight degeneracy. However %whichever the resampling technique used this mechanism tends to be insufficient when applied to informative or high-dimensional models. In this paper we revisit the rejuvenation mechanism and propose a class of parameterized SIR-based solutions which enable to adjust the tradeoff between computational cost and statistical performances.