Sequential Inference Research Articles

Bayesian treed response surface models

Tree‐based regression and classification, popularized in the 1980s with the advent of the classification and regression trees (CART) has seen a recent resurgence in popularity alongside a boom in modern computing power. The new methodologies take advantage of simulation‐based inference, and ensemble methods, to produce higher fidelity response surfaces with competitive out‐of‐sample predictive performance while retaining many of the attractive features of classic trees: thrifty divide‐and‐conquer nonparametric inference, variable selection and sensitivity analysis, and nonstationary modeling features. In this paper, we review recent advances in Bayesian modeling for trees, from simple Bayesian CART models, treed Gaussian process, sequential inference via dynamic trees, to ensemble modeling via Bayesian additive regression trees (BART). We outline open source R packages supporting these methods and illustrate their use.This article is categorized under: Algorithmic Development > Statistics Algorithmic Development > Structure Discovery Application Areas > Data Mining Software Tools

Sequential Fuzzy Diagnosis Method for Motor Roller Bearing in Variable Operating Conditions Based on Vibration Analysis

A novel intelligent fault diagnosis method for motor roller bearings which operate under unsteady rotating speed and load is proposed in this paper. The pseudo Wigner-Ville distribution (PWVD) and the relative crossing information (RCI) methods are used for extracting the feature spectra from the non-stationary vibration signal measured for condition diagnosis. The RCI is used to automatically extract the feature spectrum from the time-frequency distribution of the vibration signal. The extracted feature spectrum is instantaneous, and not correlated with the rotation speed and load. By using the ant colony optimization (ACO) clustering algorithm, the synthesizing symptom parameters (SSP) for condition diagnosis are obtained. The experimental results shows that the diagnostic sensitivity of the SSP is higher than original symptom parameter (SP), and the SSP can sensitively reflect the characteristics of the feature spectrum for precise condition diagnosis. Finally, a fuzzy diagnosis method based on sequential inference and possibility theory is also proposed, by which the conditions of the machine can be identified sequentially as well.

Efficient learning via simulation: A marginalized resample-move approach

In state–space models, parameter learning is practically difficult and is still an open issue. This paper proposes an efficient simulation-based parameter learning method. First, the approach breaks up the interdependence of the hidden states and the static parameters by marginalizing out the states using a particle filter. Second, it applies a Bayesian resample-move approach to this marginalized system. The methodology is generic and needs little design effort. Different from batch estimation methods, it provides posterior quantities necessary for full sequential inference and recursive model monitoring. The algorithm is implemented both on simulated data in a linear Gaussian model for illustration and comparison and on real data in a Lévy jump stochastic volatility model and a structural credit risk model.

Bayesian statistics with a smile: A resampling–sampling perspective

In this paper we develop a simulation-based approach to sequential inference in Bayesian statistics. Our resampling–sampling perspective provides draws from posterior distributions of interest by exploiting the sequential nature of Bayes theorem. Predictive inferences are a direct byproduct of our analysis as are marginal likelihoods for model assessment. We illustrate our approach in a hierarchical normal-means model and in a sequential version of Bayesian lasso. This approach provides a simple yet powerful framework for the construction of alternative posterior sampling strategies for a variety of commonly used models.

Inference for a class of partially observed point process models

This paper presents a simulation-based framework for sequential inference from partially and discretely observed point process models with static parameters. Taking on a Bayesian perspective for the static parameters, we build upon sequential Monte Carlo methods, investigating the problems of performing sequential filtering and smoothing in complex examples, where current methods often fail. We consider various approaches for approximating posterior distributions using SMC. Our approaches, with some theoretical discussion are illustrated on a doubly stochastic point process applied in the context of finance.

Closed Sequential and Multistage Inference on Binary Responses With or Without Replacement

We consider closed sequential or multistage sampling, with or without replacement, from a lot of N items, where each item can be identified as defective (in error, tainted, etc.) or not. The goal is to make inference on the proportion, π, of defectives in the lot, or equivalently on the number of defectives in the lot D = Nπ. It is shown that exact inference on π using closed (bounded) sequential or multistage procedures with general prespecified elimination boundaries is completely tractable and not at all inconvenient using modern statistical software. We give relevant theory and demonstrate functions for this purpose written in R (R Development Core Team 2005, available as online supplementary material). Applicability of the methodology is illustrated in three examples: (1) sharpening of Wald's (1947) sequential probability ratio test used in industrial acceptance sampling, (2) two-stage sampling for auditing Medicare or Medicaid health care providers, and (3) risk-limited sequential procedures for election audits.

An Intelligent Diagnosis Method for Rotating Machinery Using Least Squares Mapping and a Fuzzy Neural Network

This study proposes a new condition diagnosis method for rotating machinery developed using least squares mapping (LSM) and a fuzzy neural network. The non-dimensional symptom parameters (NSPs) in the time domain are defined to reflect the features of the vibration signals measured in each state. A sensitive evaluation method for selecting good symptom parameters using detection index (DI) is also proposed for detecting and distinguishing faults in rotating machinery. In order to raise the diagnosis sensitivity of the symptom parameters the synthetic symptom parameters (SSPs) are obtained by LSM. Moreover, possibility theory and the Dempster & Shafer theory (DST) are used to process the ambiguous relationship between symptoms and fault types. Finally, a sequential diagnosis method, using sequential inference and a fuzzy neural network realized by the partially-linearized neural network (PLNN), is also proposed, by which the conditions of rotating machinery can be identified sequentially. Practical examples of fault diagnosis for a roller bearing are shown to verify that the method is effective.

Sequential Monte Carlo-based fidelity selection in dynamic-data-driven adaptive multi-scale simulations

In a simulation-based planning and control framework, timely monitoring, analysis, and control is important not to disrupt a dynamically changing system. To meet this temporal requirement, a dynamic-data-driven adaptive multi-scale simulation (DDDAMS) paradigm was proposed earlier, where the fidelity of a complex simulation model adapts to available computational resources by incorporating dynamic data into the executing model, which then steers the measurement process for selective data update. In this work, a sequential Monte Carlo method (sequential Bayesian inference technique) is proposed and embedded into the simulation to enable its ideal fidelity selection given massive datasets under the DDDAMS framework. As dynamic information becomes available, the proposed method makes efficient inferences to determine the sources of abnormality in the system (a shop floor in this paper). A parallelisation framework is also discussed to further reduce the number of data accesses while maintaining the accuracy of parameter estimates. A prototype DDDAMS involving the proposed algorithm has been implemented successfully for preventive maintenance scheduling and part routing scheduling in a semiconductor manufacturing supply chain, reducing the average waiting time of batches and increasing the machine utilisation significantly.

Fuzzy Diagnosis Method for Rotating Machinery in Variable Rotating Speed

In order to effectively diagnose faults for rotating machinery in the variable rotating speed, a novel diagnosis method is proposed based on time-frequency analysis techniques, the automatic feature extraction method, and fuzzy inference. The diagnosis sensitivities of three time-frequency analysis methods, namely, the short-time Fourier transform (STFT), wavelet analysis (WA), and the pseudo-Wigner-Ville distribution (PWVD), are investigated for condition diagnosis of rotating machinery. In the case of the bearing diagnosis, the diagnosis sensitivity of the PWVD was found to be highest. An extraction method for instantaneous feature spectrum is proposed using the relative crossing information (RCI), by which the feature spectrum from time-frequency distribution can be automatically extracted by a computer in order to identify among the conditions of a machine. The symptom parameters are also defined in the frequency domain using the feature spectrum extracted by the RCI. The synthetic symptom parameters can be obtained by the least squares mapping (LSM) technique to increase the diagnosis sensitivity of the symptom parameters. Based on the above studies, a fuzzy diagnosis method using sequential inference and possibility theory was also proposed, by which the conditions of machinery can be well identified sequentially. Practical examples of diagnosis for a roller bearing are given in order to verify the effectiveness of the approaches proposed in this paper.

Feature Extraction Method Based on Pseudo-Wigner-Ville Distribution for Rotational Machinery in Variable Operating Conditions

In the case of fault diagnosis for roller bearings, the conventional diagnosis approaches by using the time interval of energy impacts in time-frequency distribution or the pass-frequencies are based on the assumption that machinery operates under a constant rotational speed. However, when the rotational speed varies in the broader range, the pass-frequencies vary with the change of rotational speed and bearing faults cannot be identified by the interval of impacts. Researches related to automatic diagnosis for rotational machinery in variable operating conditions were quite few. A novel automatic feature extraction method is proposed based on a pseudo-Wigner-Ville distribution (PWVD) and an extraction of symptom parameter (SP). An extraction method for instantaneous feature spectrum is presented using the relative crossing information (RCI) and sequential inference approach, by which the feature spectrum from time-frequency distribution can be automatically, sequentially extracted. The SPs are considered in the frequency domain using the extracted feature spectrum to identify among the conditions of a machine. A method to obtain the synthetic symptom parameter is also proposed by the least squares mapping (LSM) technique for increasing the diagnosis sensitivity of SP. Practical examples of diagnosis for bearings are given in order to verify the effectiveness of the proposed method. The verification results show that the features of bearing faults, such as the outer-race, inner-race and roller element defects have been effectively extracted, and the proposed method can be used for condition diagnosis of a machine under the variable rotational speed.

Detection and Tracking of Coordinated Groups

In this paper, we describe models and algorithms for detection and tracking of group and individual targets. We develop two novel group dynamical models within a continuous time setting using stochastic differential equations (SDE) that aim to mimic behavioural properties of groups. We also describe a possible way of modeling interactions between closely spaced targets using repulsive forces. These can be combined with a group structure transition model to create realistic evolving group models. We use a Markov chain Monte Carlo (MCMC)-particles algorithm to perform sequential inference. Computer simulations demonstrate the ability of the algorithm to detect and track targets within groups as well as to infer the correct group structure over time. The group tracking model is also applied to two sets of real ground moving target indicator (GMTI) radar data with group targets. The results show significant improvement in tracking accuracy over tracking without group models.

Efficient Learning via Simulation: A Marginalized Resample-Move Approach

In state-space models, parameter learning is practically difficult and is still an open issue. This paper proposes an efficient simulation-based parameter learning method. First, the approach breaks up the interdependence of the hidden states and the static parameters by marginalizing out the states using a particle filter. Second, it applies a Bayesian resample-move approach to this marginalized system. The methodology is generic and needs little design effort. Different from batch estimation methods, it provides posterior quantities necessary for full sequential inference and recursive model monitoring. The algorithm is implemented both on simulated data in a linear Gaussian model for illustration and comparison and on real data in a Levy jump stochastic volatility model and a structural credit risk model.

Studentized Dynamical System for Robust Object Tracking

This paper describes a studentized dynamical system (SDS) for robust target tracking using a subspace representation. Dynamical systems (DS) provide a powerful framework for the probabilistic modeling of temporal sequences. Visual tracking problems are often cast as a sequential inference problem within the DS framework and a compact way to model the observation distributions (i.e., object appearances) is through probabilistic principal component analysis (PPCA). PPCA is a classic Gaussian based subspace representation method and a popular tool for appearance modeling. Although Gaussian density has theoretically nice properties, resulting in models that are always tractable, they are also severely limited in practical settings. One of the central issues in the use of PPCA for target appearance modeling is that it is very sensitive to outliers. The Gaussian density has a very light tail, while real world data with outliers exhibit heavy tails. Recently, more heavy-tailed distributions (e.g., Student's t-distribution) have been introduced to increase the robustness of the original PPCA. We propose to augment the traditional target tracking DS by adding a set of auxiliary latent variables to adjust the shape of the observation distribution. We show that by carefully choosing the probability density of these auxiliary latent variables, a more robust observation distribution can be obtained with tails heavier than Gaussian. Numerical experiments verify that the proposed SDS has a better capability to handle considerable amount of outlier noise and an improved tracking performance over DS with a Gaussian based observation model.

Implementation of Dynamic Bayesian Decision Making by Intracellular Kinetics

Decision making in a noisy and dynamically changing environment is a fundamental task for a cell. To choose appropriate decisions over time, a cell must be equipped with intracellular kinetics that can conduct dynamic and efficient decision making. By using the theory of sequential inference, I demonstrate that dynamic Bayesian decision making can be implemented by an intracellular kinetics with a dual positive feedback structure. I also show that the combination of linear instantaneous and nonlinear stationary sensitivities to the input dominantly contributes to decision making efficiency, and that the state-dependent sensitivity change further suppresses noisy response. The statistical principles underlying these two factors are further clarified to be a log-likelihood-dependent quantification of the input information and uncertainty-dependent sensitivity control.

Vehicle Trajectory Estimation Using Spatio-Temporal MCMC

This paper presents an algorithm for modeling and tracking vehicles in video sequences within one integrated framework. Most of the solutions are based on sequential methods that make inference according to current information. In contrast, we propose a deferred logical inference method that makes a decision according to a sequence of observations, thus processing a spatio-temporal search on the whole trajectory. One of the drawbacks of deferred logical inference methods is that the solution space of hypotheses grows exponentially related to the depth of observation. Our approach takes into account both the kinematic model of the vehicle and a driver behavior model in order to reduce the space of the solutions. The resulting proposed state model explains the trajectory with only 11 parameters. The solution space is then sampled with a Markov Chain Monte Carlo (MCMC) that uses a model-driven proposal distribution in order to control random walk behavior. We demonstrate our method on real video sequences from which we have ground truth provided by a RTK GPS (Real-Time Kinematic GPS). Experimental results show that the proposed algorithm outperforms a sequential inference solution (particle filter).

Video Tracking Using Dual-Tree Wavelet Polar Matching and Rao-Blackwellised Particle Filter

We describe a video tracking application using the dual-tree Polar Matching Algorithm. We develop the dynamical and observation models in a probabilistic setting and study the empirical probability distribution of the Polar Matching output. We model the visible and occluded target statistics using Beta distributions. This is incorporated into a Track-Before-Detect (TBD) solution for the overall observation likelihood of each video frame and provides a principled derivation of the observation likelihood. Due to the nonlinear nature of the problem, we design a Rao-Blackwellised Particle Filter (RBPF) for the sequential inference. Computer simulations demonstrate the ability of the algorithm to track a simulated video moving target in an urban environment with complete and partial occlusions.

DIWSAN: Distributed Intelligent Wireless Sensor and Actuator Network for Heterogeneous Environment

In a WSAN (Wireless Sensor and Actuator Network), most resources, including sensors and actuators, are designed for certain applications in a dedicated environment. Many researchers have proposed to use of gateways to infer and annotate heterogeneous data; however, such centralized methods produce a bottlenecking network and computation overhead on the gateways that causes longer response time in activity processing, worsening performance. This work proposes two distribution inference mechanisms: regionalized and sequential inference mechanisms to reduce the response time in activity processing. Finally, experimental results for the proposed inference mechanisms are presented, and it shows that our mechanisms outperform the traditional centralized inference mechanism.

Intelligent diagnosis methods for plant machinery

This paper reports several intelligent diagnostic approaches based on artificial neural network and fuzzy algorithm for plant machinery, such as the diagnosis method using the wavelet transform, rough sets, and fuzzy neural network; the diagnosis method based on the sequential inference and fuzzy neural network; the diagnosis approach by the possibility theory and certainty factor model; and the diagnosis method on the basis of the adaptive filtering technique and fuzzy neural network. These intelligent diagnostic methods have been successfully applied to condition diagnosis in different types of practical plant machinery.

Sequential Inference for Nonlinear Models using Slice Variables

This paper develops particle-based methods for sequential inference in nonlinear models. Sequential inference is notoriously difficult in nonlinear state space models. To overcome this, we use auxiliary state variables to slice out nonlinearities where appropriate. This induces a Fixed-dimension conditional sufficient statistics and, given these, we adapt existing particle learning algorithms to update posterior beliefs about states and parameters. We provide three illustrations. First, a dynamic exponential model with Gaussian errors. Second, a stochastic growth model with nonlinear state evolution and t-distributed errors. Finally, a bivariate radar tracking problem which was originally analyzed in the nonlinear Monte Carlo Filtering literature. In all cases, we illustrate the efficiency of our methodology.

A PENALTY-LOGIC SIMPLE-TRANSITION MODEL FOR STRUCTURED SEQUENCES

We study the problem of learning to infer hidden-state sequences of processes whose states and observations are propositionally or relationally factored. Unfortunately, standard exact inference techniques such as Viterbi and graphical model inference exhibit exponential complexity for these processes. The main motivation behind our work is to identify a restricted space of models, which facilitate efficient inference, yet are expressive enough to remain useful in many applications. In particular, we present the penalty-logic simple-transition model, which utilizes a very simple-transition structure where the transition cost between any two states is constant. While not appropriate for all complex processes, we argue that it is often rich enough in many applications of interest, and when it is applicable there can be inference and learning advantages compared to more general models. In particular, we show that sequential inference for this model, that is, finding a minimum-cost state sequence, efficiently reduces to a single-state minimization (SSM) problem. We then show how to define atemporal-cost models in terms of penalty logic, or weighted logical constraints, and how to use this representation for practically efficient SSM computation. We present a method for learning the weights of our model from labeled training data based on Perceptron updates. Finally, we give experiments in both propositional and relational video-interpretation domains showing advantages compared to more general models.