Maximum Likelihood Estimation Problem Research Articles

Maximum Marginal Likelihood Estimation of Phase Connections in Power Distribution Systems

Accurate phase connectivity information is essential for advanced monitoring and control applications in power distribution systems. The existing data-driven approaches for phase identification lack precise physical interpretation and theoretical performance guarantee. Their performance generally deteriorates as the complexity of the network, the number of phase connections, and the level of load balance increase. In this paper, by linearizing the three-phase power flow manifold, we develop a physical model, which links the phase connections to the smart meter measurements. The phase identification problem is first formulated as a maximum likelihood estimation problem and then reformulated as a maximum marginal likelihood estimation problem. We prove that the correct phase connection achieves the highest log likelihood values for both problems. An efficient solution method is proposed by decomposing the original problem into subproblems with a binary least-squares formulation. The numerical tests on a comprehensive set of distribution circuits show that our proposed method yields very high accuracy on both radial and meshed distribution circuits with a combination of single-phase, two-phase, and three-phase loads. The proposed algorithm is robust with respect to inaccurate feeder models and incomplete measurements. It also outperforms the existing methods on complex circuits.

Identification of Markov Jump Autoregressive Processes from Large Noisy Data Sets

This paper introduces a novel methodology for the identification of switching dynamics for switched autoregressive linear models. Switching behavior is assumed to follow a Markov model. The system’s outputs are contaminated by possibly large values of measurement noise. Although the procedure provided can handle other noise distributions, for simplicity, it is assumed that the distribution is Normal with unknown variance. Given noisy input-output data, we aim at identifying switched system coefficients, parameters of the noise distribution, dynamics of switching and probability transition matrix of Markovian model. System dynamics are estimated using previous results which exploit algebraic constraints that system trajectories have to satisfy. Switching dynamics are computed with solving a maximum likelihood estimation problem. The efficiency of proposed approach is shown with several academic examples. Although the noise to output ratio can be high, the method is shown to be effective in the situations where a large number of measurements is available.

Semidefinite Programming Methods for Alleviating Clock Synchronization Bias and Sensor Position Errors in TDOA Localization

This paper investigates the problem of source localization using signal time-difference-of-arrival (TDOA) measurements in the presence of clock synchronization bias and sensor position errors. Our existing work has developed a unified solution for TDOA localization in the presence of sensor position errors but clock synchronization bias was not considered. Clock synchronization bias is a more complex problem often encountered in practical localization networks. This paper further generalizes this framework to include clock synchronization bias. The proposed technique employs multiple calibration emitters to simultaneously alleviate both the sensor position errors and clock synchronization bias. The maximum likelihood estimator (MLE) for this problem is optimal, but too complex to be applied in practice. We develop a semidefinite programming (SDP) based localization algorithm to effectively solve the MLE problem. This SDP algorithm can reach the Cramer-Rao lower bound when sensor position errors are not unrealistically large.

Efficient Convex Solution for 3-D Localization in MIMO Radars Using Delay and Angle Measurements

In this letter, an efficient estimator for 3-D target localization in distributed multiple-input multiple-output (MIMO) radars using time delay (TD) and angle of arrival (AOA) measurements is proposed. First, an approximately equivalent maximum likelihood (ML) estimation problem is formulated. Then, the aforementioned ML problem is recast into a convex optimization problem for which we derive a semi closed-form solution that eventually boils down to finding the roots of certain polynomials. Using numerical simulations, we demonstrate that the proposed estimator reaches the Cramer-Rao lower bound (CRLB) up to relatively high Gaussian measurement noise levels. Furthermore, the proposed method is shown to outperform the state-of-the-art algorithms under the above-mentioned noise model.

Efficient Likelihood Ratio Confidence Intervals using Constrained Optimization

Using constrained optimization, we develop a simple, efficient approach (applicable in both unconstrained and constrained maximum-likelihood estimation problems) to computing profile-likelihood confidence intervals. In contrast to Wald-type or score-based inference, the likelihood ratio confidence intervals use all the information encoded in the likelihood function concerning the parameters, which leads to improved statistical properties. In addition, the method does no suffer from the computational burdens inherent in the bootstrap. In an application to Rust's (1987) bus-engine replacement problem, our approach does better than either the Wald or the bootstrap methods, delivering very accurate estimates of the confidence intervals quickly and efficiently. An extensive Monte Carlo study reveals that in small samples, only likelihood ratio confidence intervals yield reasonable coverage properties, while at the same time discriminating implausible values.

Scalable Gaussian Process Computations Using Hierarchical Matrices

We present a kernel-independent method that applies hierarchical matrices to the problem of maximum likelihood estimation for Gaussian processes. The proposed approximation provides natural and scalable stochastic estimators for its gradient and Hessian, as well as the expected Fisher information matrix, that are computable in quasilinear complexity for a large range of models. To accomplish this, we (i) choose a specific hierarchical approximation for covariance matrices that enables the computation of their exact derivatives and (ii) use a stabilized form of the Hutchinson stochastic trace estimator. Since both the observed and expected information matrices can be computed in quasilinear complexity, covariance matrices for maximum likelihood estimators (MLEs) can also be estimated efficiently. In this study, we demonstrate the scalability of the method, show how details of its implementation effect numerical accuracy and computational effort, and validate that the resulting MLEs and confidence intervals based on the inverse Fisher information matrix faithfully approach those obtained by the exact likelihood. Supplementary materials for this article are available online.

Efficient Joint Moving Target and Antenna Localization in Distributed MIMO Radars

In this paper, a novel solution for the problem of joint moving target and antenna localization in the distributed multiple-input multiple-output (MIMO) radar systems is proposed. The localization problem in the presence of antenna location uncertainty is formulated as a maximum likelihood (ML) estimation problem, which is then recast into convex form by defining some auxiliary variables and applying semidefinite relation (SDR) technique. Next, an algebraic closed-form estimator is proposed to jointly estimate the target and the antennas error terms and refine their uncertain values. The proposed method is shown analytically and verified by the numerical simulations to be an efficient estimator, which can achieve the Cramer–Rao lower bound (CRLB) performance for both the target and antenna estimations under small error conditions. Furthermore, the results of numerical simulations demonstrate that the proposed estimator outperforms the state-of-the-art methods presented for moving target localization in MIMO radars.

Monopulse forward‐looking imaging algorithm based on Levenberg–Marquardt optimisation

With precise angle measurement, traditional monopulse techniques for forward-looking imaging can acquire exact angle information of one single target within a radar beam. However, when multiple targets exist in a beam, it is difficult to resolve them. To address this problem, a novel monopulse forward-looking imaging algorithm based on Levenberg–Marquardt (LM) optimisation is proposed. The core idea of this algorithm is to solve maximum likelihood estimation problem based on LM optimisation (ML-LM) to obtain the direction of arrival (DOA) estimation of unresolved targets. First, the echo model of two targets within a forward-looking radar beam is established, then the imaging framework of the proposed algorithm is introduced. Finally, the advantages of ML-LM are illustrated based on a series of evaluation criterion, including angle measurement error for various values of signal-to-noise ratio (SNR), angle interval adaptability and computational efficiency. The simulation results show that two targets within a forward-looking radar beam can be resolved and relocated accurately utilising the proposed algorithm. Meanwhile, the comparison with other algorithms shows it has higher DOA estimation accuracy, less computational complexity and a wider range of angle interval adaptability.

Automated fusion of forest airborne and terrestrial point clouds through canopy density analysis

Airborne laser scanning (ALS) and terrestrial laser scanning (TLS) systems are effective ways to capture the 3D information of forests from complementary perspectives. Registration of the two sources of point clouds is necessary for various forestry applications. Since the forest point clouds show irregular and natural point distributions, standard registration methods working on geometric keypoints (e.g., points, lines, and planes) are likely to fail. Hence, we propose a novel method to register the ALS and TLS forest point clouds through density analysis of the crowns. The proposed method extracts mode-based keypoints by the mean shift method and aligns them by maximum likelihood estimation. Firstly, the differences in the point densities of the ALS and TLS crowns are minimized to produce analogous modes, which represent the local maxima of the underlying probability density function (PDF). The mode-based keypoints are then aligned through the coherent point drift (CPD) algorithm, which is independent of the descriptor similarities and considers the alignment as a maximum likelihood estimation problem. The sets of keypoints derived from the two data sources need not be equal. Finally, the recovered transformation is applied to the original point clouds and refined through the standard iterative closest point (ICP) algorithm. In contrast to some of the existing methods, the proposed method avoids the geometric description of the forest point clouds. Furthermore, additional information such as tree diameter or height is not required to evaluate the similarities. The experiments in this study were conducted in a Scandinavian boreal forest, located in Evo, Finland. The proposed method was tested on four datasets (ALS data: a circle with a diameter of 60 m, multi-scan TLS data: 32 × 32 m) with heterogeneous tree species and structures. The results showed that the proposed probabilistic-based method obtains a good performance with a 3D distance residual of 0.069 m, and improved the accuracy of the registration when compared with the existing methods.

Maximum likelihood estimation in Gaussian models under total positivity

We analyze the problem of maximum likelihood estimation for Gaussian distributions that are multivariate totally positive of order two ($\mathrm{MTP}_{2}$). By exploiting connections to phylogenetics and single-linkage clustering, we give a simple proof that the maximum likelihood estimator (MLE) for such distributions exists based on $n\geq2$ observations, irrespective of the underlying dimension. Slawski and Hein [Linear Algebra Appl. 473 (2015) 145–179], who first proved this result, also provided empirical evidence showing that the $\mathrm{MTP}_{2}$ constraint serves as an implicit regularizer and leads to sparsity in the estimated inverse covariance matrix, determining what we name the ML graph. We show that we can find an upper bound for the ML graph by adding edges corresponding to correlations in excess of those explained by the maximum weight spanning forest of the correlation matrix. Moreover, we provide globally convergent coordinate descent algorithms for calculating the MLE under the $\mathrm{MTP}_{2}$ constraint which are structurally similar to iterative proportional scaling. We conclude the paper with a discussion of signed $\mathrm{MTP}_{2}$ distributions.

An efficient kriging modeling method for high-dimensional design problems based on maximal information coefficient

Kriging, one of the most popular surrogate models, is widely used in computationally expensive optimization problems to improve the design efficiency. However, due to the “curse-of-dimensionality,” the time for generating the kriging model increases exponentially as the dimension of the problem grows. When it comes to the cases that the kriging model needs to be frequently constructed, such as sequential sampling for kriging modeling or global optimization based on kriging model, the increased modeling time should be taken into consideration. To overcome this challenge, we propose a novel kriging modeling method which combines kriging with maximal information coefficient (MIC). Taking the features of the optimized hyper-parameters into consideration, MIC is utilized for estimating the relative magnitude of hyper-parameters. Then this knowledge of hyper-parameters is incorporated into the maximum likelihood estimation problem to reduce the dimensionality. In this way, the high dimensional optimization can be transformed into a one-dimensional optimization, which can significantly improve the modeling efficiency. Five representative numerical examples from 20-D to 80-D and an industrial example with 35 variables are used to show the effectiveness of the proposed method. Results show that compared with the conventional kriging, the modeling time of the proposed method can be ignored, while the loss of accuracy is acceptable. For the problems with more than 40 variables, the proposed method can even obtain a more accurate kriging model with given computational resources. Besides, the proposed method is also compared with KPLS (kriging combined with the partial least squares method), another state-of-the-art kriging modeling method for high-dimensional problems. Results show that the proposed method is more competitive than KPLS, which means the proposed method is an efficient kriging modeling method for high-dimensional problems.

Word Embedding as Maximum A Posteriori Estimation

The GloVe word embedding model relies on solving a global optimization problem, which can be reformulated as a maximum likelihood estimation problem. In this paper, we propose to generalize this approach to word embedding by considering parametrized variants of the GloVe model and incorporating priors on these parameters. To demonstrate the usefulness of this approach, we consider a word embedding model in which each context word is associated with a corresponding variance, intuitively encoding how informative it is. Using our framework, we can then learn these variances together with the resulting word vectors in a unified way. We experimentally show that the resulting word embedding models outperform GloVe, as well as many popular alternatives.

New perspective on the benefits of the gene–environment independence in case–control studies

AbstractWe study the benefit of exploiting the gene–environment independence (GEI) assumption for inferring the joint effect of genotype and environmental exposure on disease risk in a case–control study. By transforming the problem into a constrained maximum likelihood estimation problem we derive the asymptotic distribution of the maximum likelihood estimator (MLE) under the GEI assumption (MLE‐GEI) in a closed form. Our approach uncovers a transparent explanation of the efficiency gained by exploiting the GEI assumption in more general settings, thus bridging an important gap in the existing literature. Moreover, we propose an easy‐to‐implement numerical algorithm for estimating the model parameters in practice. Finally, we conduct simulation studies to compare the proposed method with the traditional prospective logistic regression method and the case‐only estimator. The Canadian Journal of Statistics 47: 473–486; 2019 © 2019 Statistical Society of Canada

Robust Non-Rigid Feature Matching for Image Registration Using Geometry Preserving.

In this paper, a robust non-rigid feature matching approach for image registration with geometry constraints is proposed. The non-rigid feature matching approach is formulated as a maximum likelihood (ML) estimation problem. The feature points of one image are represented by Gaussian mixture model (GMM) centroids, and are fitted to the feature points of the other image by moving coherently to encode the global structure. To preserve the local geometry of these feature points, two local structure descriptors of the connectivity matrix and Laplacian coordinate are constructed. The expectation maximization (EM) algorithm is applied to solve this ML problem. Experimental results demonstrate that the proposed approach has better performance than current state-of-the-art methods.

Topology-aware non-rigid point set registration via global–local topology preservation

We propose a new topology-aware point set registration algorithm which can cope with multi-part articulated and non-rigid deformations. Point set registration is formulated as a maximum likelihood (ML) estimation problem where two topologically complementary constraints are jointly optimized in a probabilistic framework. The first is coherent point drift that keeps the overall spatial connectivity and associativity by moving the point set collectively and coherently. The second is local linear embedding that preserves the local topological structure during registration. Hence, the new algorithm is called global–local topology preservation (GLTP). Without any pre-segmentation and correspondence initialization, GLTP is particularly useful and effective in dealing with complex shape matching with non-coherent and non-rigid local deformations at different parts of a point set. We have derived the expectation maximization algorithm for the ML optimization constrained with both regularization terms. Experimental results on a large set of 2D and 3D examples show the advantages and robustness of GLTP over existing algorithms in the presence of outliers, noise and missing data, especially in the case of articulated non-rigid transformations.

Reconstructing signed networks via Ising dynamics.

Revealing unknown network structure from observed data is a fundamental inverse problem in network science. Current reconstruction approaches were mainly proposed to infer the unsigned networks. However, many social relationships, such as friends and foes, can be represented as signed social networks that contain positive and negative links. To the best of our knowledge, the method of reconstructing signed networks has not yet been developed. To this purpose, we develop a statistical inference approach to fully reconstruct the signed network structure (positive links, negative links, and nonexistent links) based on the Ising dynamics. By the theoretical analysis, we show that our approach can transfer the problem of maximum likelihood estimation into the problem of solving linear systems of equations, where the solution of the linear system of equations uncovers the neighbors and the signs of links of each node. The experimental results on both synthetic and empirical networks validate the reliability and efficiency of our method. Our study moves the first step toward reconstructing signed networks.

Joint clock synchronization and position estimation in time of arrival–based passive positioning systems

A widely used scheme for target localization is to measure the time of arrival of a wireless signal emitted by a tag, which requires the clocks of the anchors (receivers at known locations) to be accurately synchronized. Conventional systems rely on transmissions from a timing reference node at a known location for clock synchronization and therefore are susceptible to reference node failure. In this article, we propose a novel localization scheme which jointly estimates anchor clock offsets and target positions. The system does not require timing reference nodes and is completely passive (non-intrusive). The positioning algorithm is formulated as a maximum likelihood estimation problem, which is solved efficiently using an iterative linear least square method. The Cramér–Rao lower bound of positioning error is also analyzed. It is shown that the performance of the proposed scheme improves with the number of targets in the system and approaches that of a system with perfectly synchronized anchors.

Fusing Multiple Multiband Images

High-resolution hyperspectral images are in great demand but hard to acquire due to several existing fundamental and technical limitations. A practical way around this is to fuse multiple multiband images of the same scene with complementary spatial and spectral resolutions. We propose an algorithm for fusing an arbitrary number of coregistered multiband, i.e., panchromatic, multispectral, or hyperspectral, images through estimating the endmember and their abundances in the fused image. To this end, we use the forward observation and linear mixture models and formulate an appropriate maximum-likelihood estimation problem. Then, we regularize the problem via a vector total-variation penalty and the non-negativity/sum-to-one constraints on the endmember abundances and solve it using the alternating direction method of multipliers. The regularization facilitates exploiting the prior knowledge that natural images are mostly composed of piecewise smooth regions with limited abrupt changes, i.e., edges, as well as coping with potential ill-posedness of the fusion problem. Experiments with multiband images constructed from real-world hyperspectral images reveal the superior performance of the proposed algorithm in comparison with the state-of-the-art algorithms, which need to be used in tandem to fuse more than two multiband images.

A Fast, Decentralized Covariance Selection-Based Approach to Detect Cyber Attacks in Smart Grids

Recent studies have shown that an attacker can compromise some of the power grid measurements to mislead the conventional state estimators (SEs), since the manipulated measurements can pass the SE residue tests. Statistical structure learning-based approaches have been recently introduced as a powerful tool to detect some of the most complicated cyber attacks. However, the expensive computational complexity of the learning process limits the applicability of these approaches for real time cyber attack detection. This paper proposes a fast and decentralized approach for cyber attack detection based on a maximum likelihood (ML) estimation which exploits the near chordal sparsity of power grids to establish an efficient framework to solve the associated ML estimation problem. The proposed detection method is then decomposed to several local ML estimation problems; this would ensure privacy and reduce the complexity of the underlying problem. The simulation studies validate the efficiency of the proposed method in detecting truly complicated stealthy false data injection attacks.

Some Notes about Inference for the Lognormal Diffusion Process with Exogenous Factors

Different versions of the lognormal diffusion process with exogenous factors have been used in recent years to model and study the behavior of phenomena following a given growth curve. In each case considered, the estimation of the model has been addressed, generally by maximum likelihood (ML), as has been the study of several characteristics associated with the type of curve considered. For this process, a unified version of the ML estimation problem is presented, including how to obtain estimation errors and asymptotic confidence intervals for parametric functions when no explicit expression is available for the estimators of the parameters of the model. The Gompertz-type diffusion process is used here to illustrate the application of the methodology.