Constrained Expectation-maximization Research Articles

Maneuvering extended object tracking based on constrained expectation maximization

This paper addresses the problem of maneuvering extended object tracking, in which the object extension (OE) and the turn rate are identified simultaneously. Due to its non-linearity with respect to the turn rate, the state transition is converted into a linear form of a newly defined hyper-parametric vector by parameter substitution. For the hyper-parametric equality constraints (HECs) introduced in the transformation, a constrained expectation conditional maximization algorithm is designed. The HECs are projected onto the conditional expectation function for regularization, so as to realize the iterative identification of multi-parameter (i.e., OE and turn rate). The transformation and CECM optimization bring the advantage of avoiding nonlinear model approximation, which is important for the convergence and accuracy of state estimation. Finally, simulation results demonstrate the superiority of the proposed method in terms of both estimation accuracy and identification effectiveness.

Semi-Supervised Affinity Propagation with Soft Instance-Level Constraints.

Soft-constraint semi-supervised affinity propagation (SCSSAP) adds supervision to the affinity propagation (AP) clustering algorithm without strictly enforcing instance-level constraints. Constraint violations lead to an adjustment of the AP similarity matrix at every iteration of the proposed algorithm and to addition of a penalty to the objective function. This formulation is particularly advantageous in the presence of noisy labels or noisy constraints since the penalty parameter of SCSSAP can be tuned to express our confidence in instance-level constraints. When the constraints are noiseless, SCSSAP outperforms unsupervised AP and performs at least as well as the previously proposed semi-supervised AP and constrained expectation maximization. In the presence of label and constraint noise, SCSSAP results in a more accurate clustering than either of the aforementioned established algorithms. Finally, we present an extension of SCSSAP which incorporates metric learning in the optimization objective and can further improve the performance of clustering.

Internet traffic clustering with side information

Internet traffic classification is a critical and essential functionality for network management and security systems. Due to the limitations of traditional port-based and payload-based classification approaches, the past several years have seen extensive research on utilizing machine learning techniques to classify Internet traffic based on packet and flow level characteristics. For the purpose of learning from unlabeled traffic data, some classic clustering methods have been applied in previous studies but the reported accuracy results are unsatisfactory. In this paper, we propose a semi-supervised approach for accurate Internet traffic clustering, which is motivated by the observation of widely existing partial equivalence relationships among Internet traffic flows. In particular, we formulate the problem using a Gaussian Mixture Model (GMM) with set-based equivalence constraint and propose a constrained Expectation Maximization (EM) algorithm for clustering. Experiments with real-world packet traces show that the proposed approach can significantly improve the quality of resultant traffic clusters.

LOS and NLOS Classification for Underwater Acoustic Localization

The low sound speed in water makes propagation delay (PD)-based range estimation attractive for underwater acoustic localization (UWAL). However, due to the long channel impulse response and the existence of reflectors, PD-based UWAL suffers from significant degradation when PD measurements of nonline-of-sight (NLOS) communication links are falsely identified as line-of-sight (LOS). In this paper, we utilize expected variation of PD measurements due to mobility of nodes and present an algorithm to classify the former into LOS and NLOS links. First, by comparing signal strength-based and PD-based range measurements, we identify object-related NLOS (ONLOS) links, where signals are reflected from objects with high reflection loss, for example, ships hull, docks, rocks and so on. In the second step, excluding PD measurements related to ONLOS links, we use a constrained expectation-maximization algorithm to classify PD measurements into two classes: LOS and sea-related NLOS (SNLOS), and to estimate the statistical parameters of each class. Since our classifier relies on models for the underwater acoustic channel, which are often simplified, alongside simulation results, we validate the performance of our classifier based on measurements from three sea trials. Both our simulation and sea trial results demonstrate a high detection rate of ONLOS links, and accurate classification of PD measurements into LOS and SNLOS.

Constrained expectation-maximization algorithm for stochastic inertial error modeling: study of feasibility

Stochastic modeling is a challenging task for low-cost sensors whose errors can have complex spectral structures. This makes the tuning process of the INS/GNSS Kalman filter often sensitive and difficult. For example, first-order Gauss–Markov processes are very often used in inertial sensor models. But the estimation of their parameters is a non-trivial task if the error structure is mixed with other types of noises. Such an estimation is often attempted by computing and analyzing Allan variance plots. This contribution demonstrates solving situations when the estimation of error parameters by graphical interpretation is rather difficult. The novel strategy performs direct estimation of these parameters by means of the expectation-maximization (EM) algorithm. The algorithm results are first analyzed with a critical and practical point of view using simulations with typically encountered error signals. These simulations show that the EM algorithm seems to perform better than the Allan variance and offers a procedure to estimate first-order Gauss–Markov processes mixed with other types of noises. At the same time, the conducted tests revealed limits of this approach that are related to the convergence and stability issues. Suggestions are given to circumvent or mitigate these problems when complexity of error structure is ‘reasonable’. This work also highlights the fact that the suggested approach via EM algorithm and the Allan variance may not be able to estimate the parameters of complex error models reasonably well and shows the need for new estimation procedures to be developed in this context. Finally, an empirical scenario is presented to support the former findings. There, the positive effect of using the more sophisticated EM-based error modeling on a filtered trajectory is highlighted.

Real-Time Motion Segmentation of Sparse Feature Points at Any Speed

We present a real-time incremental approach to motion segmentation operating on sparse feature points. In contrast to previous work, the algorithm allows for a variable number of image frames to affect the segmentation process, thus enabling an arbitrary number of objects traveling at different relative speeds to be detected. Feature points are detected and tracked throughout an image sequence, and the features are grouped using a spatially constrained expectation-maximization (EM) algorithm that models the interactions between neighboring features using the Markov assumption. The primary parameter used by the algorithm is the amount of evidence that must accumulate before features are grouped. A statistical goodness-of-fit test monitors the change in the motion parameters of a group over time in order to automatically update the reference frame. Experimental results on a number of challenging image sequences demonstrate the effectiveness and computational efficiency of the technique.

A Bayesian framework for noise covariance estimation using the facet model

In image processing literature, thus far, researchers have assumed the perturbation in the data to be white (or uncorrelated) having a covariance matrix sigma2I, i.e., assumption of equal variance for all the data samples and that no correlation exists between the data samples. However, there have been very few attempts to estimate noise characteristics under the assumption that there is a correlation between data samples. In this work, we propose a new and a novel approach for the simultaneous Bayesian estimation of the unknown colored or correlated noise (population) covariance matrix and the hyperparameters of the covariance model using the well-known facet model. We also estimate the facet model coefficients. We use the facet model because of its simple, yet elegant, mathematical formulation. We use the generalized inverted Wishart density as the prior model for the noise covariance matrix. We place a structure on the covariance matrix using the parameters of a correlation filter. These hyperparameters are estimated by a new extension of the expectation-maximization algorithm called the generalized constrained expectation maximization algorithm that we developed.

A Constrained EM Algorithm for Independent Component Analysis

We introduce a novel way of performing independent component analysis using a constrained version of the expectation-maximization (EM) algorithm. The source distributions are modeled as D one-dimensional mixtures of gaussians. The observed data are modeled as linear mixtures of the sources with additive, isotropic noise. This generative model is fit to the data using constrained EM. The simpler "soft-switching" approach is introduced, which uses only one parameter to decide on the sub- or supergaussian nature of the sources. We explain how our approach relates to independent factor analysis.