Maximum Likelihood Estimation Problem Research Articles

Source shape estimation for neutron imaging systems using convolutional neural networks.

Neutron imaging systems are important diagnostic tools for characterizing the physics of inertial confinement fusion reactions at the National Ignition Facility (NIF). In particular, neutron images give diagnostic information on the size, symmetry, and shape of the fusion hot spot and surrounding cold fuel. Images are formed via collection of neutron flux from the source using a system of aperture arrays and scintillator-based detectors. Currently, reconstruction of fusion source geometry from the collected neutron images is accomplished by solving a computationally intensive maximum likelihood estimation problem via expectation maximization. In contrast, it is often useful to have simple representations of the overall source geometry that can be computed quickly. In this work, we develop convolutional neural networks (CNNs) to reconstruct the outer contours of simple source geometries. We compare the performance of the CNN for penumbral and pinhole data and provide experimental demonstrations of our methods on both non-noisy and noisy data.

Expectation-Maximization enables Phylogenetic Dating under a Categorical Rate Model.

Dating phylogenetic trees to obtain branch lengths in time unit is essential for many downstream applications but has remained challenging. Dating requires inferring substitution rates that can change across the tree. While we can assume to have information about a small subset of nodes from the fossil record or sampling times (for fast-evolving organisms), inferring the ages of the other nodes essentially requires extrapolation and interpolation. Assuming a distribution of branch rates, we can formulate dating as a constrained maximum likelihood (ML) estimation problem. While ML dating methods exist, their accuracy degrades in the face of model misspecification where the assumed parametric statistical distribution of branch rates vastly differs from the true distribution. Notably, most existing methods assume rigid, often unimodal, branch rate distributions. A second challenge is that the likelihood function involves an integral over the continuous domain of the rates and often leads to difficult non-convex optimization problems. To tackle these two challenges, we propose a new method called Molecular Dating using Categorical-models (MD-Cat). MD-Cat uses a categorical model of rates inspired by non-parametric statistics and can approximate a large family of models by discretizing the rate distribution into k categories. Under this model, we can use the Expectation- Maximization (EM) algorithm to co-estimate rate categories and branch lengths in time units. Our model has fewer assumptions about the true distribution of branch rates than parametric models such as Gamma or LogNormal distribution. Our results on two simulated and real datasets of Angiosperms and HIV and a wide selection of rate distributions show that MD-Cat is often more accurate than the alternatives, especially on datasets with exponential or multimodal rate distributions.

Extensions of Some Statistical Concepts to the Complex Domain

This paper deals with the extension of principal component analysis, canonical correlation analysis, the Cramer–Rao inequality, and a few other statistical concepts in the real domain to the corresponding complex domain. Optimizations of Hermitian forms under a linear constraint, a bilinear form under Hermitian-form constraints, and similar maxima/minima problems in the complex domain are discussed. Some vector/matrix differential operators are developed to handle the above types of problems. These operators in the complex domain and the optimization problems in the complex domain are believed to be new and novel. These operators will also be useful in maximum likelihood estimation problems, which will be illustrated in the concluding remarks. Detailed steps are given in the derivations so that the methods are easily accessible to everyone.

Staring-imaging satellite pointing estimation based on sequential ISAR images

Pointing estimation for spacecraft using Inverse Synthetic Aperture Radar (ISAR) images plays a significant role in space situational awareness and surveillance. However, feature extraction and cross-range scaling of ISAR images create bottlenecks that limit performances of current estimation methods. Especially, the emergence of staring imaging satellites, characterized by complex kinematic behaviors, presents a novel challenge to this task. To address these issues, this article proposes a pointing estimation method based on Convolutional Neural Networks (CNNs) and a numerical optimization algorithm. A satellite’s main axis, which is extracted from ISAR images by a proposed Semantic Axis Region Regression Net (SARRN), is chosen for investigation in this article due to its unique structure. Specifically, considering the kinematic characteristic of the staring satellite, an ISAR imaging model is established to bridge the target pointing and the extracted axes. Based on the imaging model, pointing estimation and cross-range scaling can be described as a maximum likelihood estimation problem, and an iterative optimization algorithm modified by using the strategy of random sampling-consistency check and weighted least squares is proposed to solve this problem. Finally, the pointing of targets and the cross-range scaling factors of ISAR images are obtained. Simulation experiments based on actual satellite orbital parameters verify the effectiveness of the proposed method. This work can improve the performance of satellite reconnaissance warning, while accurate cross-range scaling can provide a basis for subsequent data processes such as 3D reconstruction and attitude estimation.

Parameter estimation with reluctant quantum walks: a maximum likelihood approach

The parametric maximum likelihood estimation problem is addressed in the context of quantum walk theory for quantum walks on the lattice of integers. A coin action is presented, with the real parameter θ to be estimated identified with the angular argument of an orthogonal reshuffling matrix. We provide analytic results for the probability distribution for a quantum walker to be displaced by d units from its initial position after k steps. For k large, we show that the likelihood is sharply peaked at a displacement determined by the ratio d/k which is correlated with the reshuffling parameter θ. We suggest that this ‘reluctant walker’ behaviour provides the framework for maximum likelihood estimation analysis, allowing for robust parameter estimation of θ via return probabilities of closed evolution loops and quantum measurements of the position of quantum walker with ‘reluctance index’ r = d/k.

3DMNDT: 3D Multi-View Registration Method Based on the Normal Distributions Transform

The normal distributions transform (NDT) is an effective paradigm for point set registration. This method was initially designed for pair-wise registration and suffers from the accumulated error problem when directly applied to multi-view registration. Under the framework of point-to-cluster correspondence, this paper proposes a novel multi-view registration method named 3D multi-view registration based on the normal distributions transform (3DMNDT), which integrates the k-means clustering and Lie algebra optimizer to achieve multi-view registration. More specifically, the multi-view registration is cast into the maximum likelihood estimation problem. Firstly, k-means clustering is utilized to divide all data points into different clusters, where one normal distribution is computed to locally model the probability of measuring a data point in each cluster. Subsequently, the multi-view registration problem is formulated by the NDT-based likelihood function. To maximize this likelihood function, the Lie algebra optimizer is introduced and developed to optimize each rigid transformation sequentially. 3DMNDT implements data point clustering, NDT computing, and rigid transformation optimization alternately until the desired registration results are obtained. Experimental results tested on benchmark data sets illustrate that 3DMNDT can achieve state-of-the-art performance for multi-view registration. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper is motivated by solving the problem of registering multiple point sets. The normal distributions transform (NDT) is a well-known pair-wise registration method widely applied in the robotic domain. This paper extends the original NDT and proposes a novel registration method to simultaneously align more than two point sets. The multi-view registration is cast into the maximum likelihood estimation problem. Subsequently, the k-means clustering and Lie algebra optimizer are integrated to estimate registration parameters. Experimental results demonstrate its superior performance on the accuracy, efficiency, and robustness for multi-view registration of point sets.

A Bayesian Approach Toward Robust Multidimensional Ellipsoid-Specific Fitting.

This work presents a novel and effective method for fitting multidimensional ellipsoids (i.e., ellipsoids embedded in [Formula: see text]) to scattered data in the contamination of noise and outliers. Unlike conventional algebraic or geometric fitting paradigms that assume each measurement point is a noisy version of its nearest point on the ellipsoid, we approach the problem as a Bayesian parameter estimate process and maximize the posterior probability of a certain ellipsoidal solution given the data. We establish a more robust correlation between these points based on the predictive distribution within the Bayesian framework, i.e., considering each model point as a potential source for generating each measurement. Concretely, we incorporate a uniform prior distribution to constrain the search for primitive parameters within an ellipsoidal domain, ensuring ellipsoid-specific results regardless of inputs. We then establish the connection between measurement point and model data via Bayes' rule to enhance the method's robustness against noise. Due to independent of spatial dimensions, the proposed method not only delivers high-quality fittings to challenging elongated ellipsoids but also generalizes well to multidimensional spaces. To address outlier disturbances, often overlooked by previous approaches, we further introduce a uniform distribution on top of the predictive distribution to significantly enhance the algorithm's robustness against outliers. Thanks to the uniform prior, our maximum a posterior probability coincides with a more tractable maximum likelihood estimation problem, which is subsequently solved by a numerically stable Expectation Maximization (EM) framework. Moreover, we introduce an ε-accelerated technique to expedite the convergence of EM considerably. We also investigate the relationship between our algorithm and conventional least-squares-based ones, during which we theoretically prove our method's superior robustness. To the best of our knowledge, this is the first comprehensive method capable of performing multidimensional ellipsoid-specific fitting within the Bayesian optimization paradigm under diverse disturbances. We evaluate it across lower and higher dimensional spaces in the presence of heavy noise, outliers, and substantial variations in axis ratios. Also, we apply it to a wide range of practical applications such as microscopy cell counting, 3D reconstruction, geometric shape approximation, and magnetometer calibration tasks. In all these test contexts, our method consistently delivers flexible, robust, ellipsoid-specific performance, and achieves the state-of-the-art results.

Distributed Inference Over Linear Models Using Alternating Gaussian Belief Propagation

We consider the problem of maximum likelihood estimation in linear models represented by factor graphs and solved via the Gaussian belief propagation algorithm. Motivated by massive internet of things (IoT) networks and edge computing, we set the above problem in a clustered scenario, where the factor graph is divided into clusters and assigned for processing in a distributed fashion across a number of edge computing nodes. For these scenarios, we show that an alternating Gaussian belief propagation (AGBP) algorithm that alternates between inter-and intra-cluster iterations, demonstrates superior performance in terms of convergence properties compared to the existing solutions in the literature. We present a comprehensive framework and introduce appropriate metrics to analyse AGBP algorithm across a wide range of linear models characterised by symmetric and non-symmetric, square, and rectangular matrices. We extend the analysis to the case of dynamic linear models by introducing dynamic arrival of new data over time. Using a combination of analytical and extensive numerical results, we show the efficiency and scalability of AGBP algorithm, making it a suitable solution for large-scale inference in massive IoT networks.

An efficient gradient-enhanced kriging modeling method assisted by fast kriging for high-dimension problems

PurposeThe modeling cost of the gradient-enhanced kriging (GEK) method is prohibitive for high-dimensional problems. This study aims to develop an efficient modeling strategy for the GEK method.Design/methodology/approachA two-step tuning strategy is proposed for the construction of the GEK model. First, an auxiliary kriging is built efficiently. Then, the hyperparameter of the kriging model is served as a good initial guess to that of the GEK model, and a local optimal search is further used to explore the search space of hyperparameter to guarantee the accuracy of the GEK model. In the construction of the auxiliary kriging, the maximal information coefficient is adopted to estimate the relative magnitude of the hyperparameter, which is used to transform the high-dimension maximum likelihood estimation problem into a one-dimensional optimization. The tuning problem of the auxiliary kriging becomes independent of the dimension. Therefore, the modeling efficiency can be improved significantly.FindingsThe performance of the proposed method is studied with analytic problems ranging from 10D to 50D and an 18D aerodynamic airfoil example. It is further compared with two efficient GEK modeling methods. The empirical experiments show that the proposed model can significantly improve the modeling efficiency without sacrificing accuracy compared with other efficient modeling methods.Originality/valueThis paper developed an efficient modeling strategy for GEK and demonstrated the effectiveness of the proposed method in modeling high-dimension problems.

MmWave 6D Radio Localization With a Snapshot Observation From a Single BS

Accurate and ubiquitous localization is crucial for a variety of applications such as logistics, navigation, intelligent transport, monitoring, control, and also for the benefit of communications. Exploiting millimeter-wave (mmWave) signals in 5G and Beyond 5G systems can provide accurate localization with limited infrastructure. We consider the single base station (BS) localization problem and extend it to 3D position and 3D orientation estimation of an unsynchronized multi-antenna user equipment (UE), using downlink multiple-input multiple-output orthogonal frequency-division multiplexing (MIMO-OFDM) signals. Through a Fisher information analysis, we show that the problem is often identifiable, provided that there is at least one multipath component in addition to the line-of-sight (LoS), even if the position of corresponding incidence point (IP) is a priori unknown. Subsequently, we pose a maximum likelihood (ML) estimation problem, to jointly estimate the 3D position and 3D orientation of the UE as well as several nuisance parameters (the UE clock offset and the positions of IPs corresponding to the multipath). The ML problem is a high-dimensional nonconvex optimization problem over a product of Euclidean and non-Euclidean manifolds. To avoid complex exhaustive search procedures, we propose a geometric initial estimate of all parameters, which reduces the problem to a 1-dimensional search over a finite interval. Numerical results show the efficiency of the proposed ad-hoc estimation, whose gap to the Cramér-Rao bound (CRB) is tightened using the ML estimation.

Estimation of recursive route choice models with incomplete trip observations

This work concerns the estimation of recursive route choice models in the situation that the trip observations are incomplete, i.e., there are unconnected links (or nodes) in the observations. A direct approach to handle this issue could be intractable because enumerating all paths between unconnected links (or nodes) in a real network is typically not possible. We exploit an expectation–maximization (EM) method that allows dealing with the missing-data issue by alternatively performing two steps of sampling the missing segments in the observations and solving maximum likelihood estimation problems. Moreover, observing that the EM method could be expensive, we propose a new estimation method based on the idea that the choice probabilities of unconnected link observations can be exactly computed by solving systems of linear equations. We further design a new algorithm, called decomposition–composition (DC), that helps reduce the number of systems of linear equations to be solved and speed up the estimation. We compare our proposed algorithms with some standard baselines using a dataset from a real network, and show that the DC algorithm outperforms the other approaches in recovering missing information in the observations. Our methods work with most of the recursive route choice models proposed in the literature, including the recursive logit, nested recursive logit, or discounted recursive models.

Offline GNSS/Camera Extrinsic Calibration Using RTK and Fiducials

Accurate robotic state estimation often requires precise knowledge of inter-sensor offsets. In this paper we present a solution for inter-sensor calibration of systems that employ a combination of GNSS and visual-inertial sensors. RTK-GNSS and fiducial measurements are utilized to produce highly precise estimates. We present an offline batch estimation approach that utilizes a continuous-time spline on the Lie group SE(3). This approach simultaneously estimates the vehicle trajectory and sensor spatio-temporal offsets by choosing optimal spline control points and calibration parameters to solve a maximum likelihood estimation problem. Our calibration method is validated in both simulation and hardware, with comparison to an online extended Kalman filter. Hardware experiments are conducted in both a motion capture environment and outdoors. Results show that our method outperforms online methods and approaches millimeter-level accuracy.

A posteriori control densities: Imitation learning from partial observations

This paper treats a special case of the Imitation from Observations (IfO) problem. IfO is a generalisation of Imitation Learning from state-only demonstrations. Our treatment of IfO considers the case of feature-only demonstrations. This means that the full state is inaccessible for inference, and imitation must occur on the basis of a limited set of features. We refer to this setting as Imitation from Partial Observations (IfPO). This scenario has the advantage of allowing to address a wider variety of demonstrations, as well as solving the problem of heteromorphic student and teacher. We set out for policy learning methods that extract an executable state-feedback policy, directly from those features, which in the literature is known as Behavioural Cloning. In this theoretical work, we formalize the rational inference model of the student decision maker, devoted to imitation, as a controlled Hidden Markov Model. The IfPO problem is then reformulated as a Maximum Likelihood Estimation problem and treated using Expectation-Maximization. We name the resulting fixed point iterations A Posteriori Control Densities. We compare the presented approach to existing methods in the field and identify potential directions for further development, such as an extension to unknown transition and emission models.

改进蚁群优化算法的最大似然DOA估计方法

In order to alleviate the huge amount of calculation when using the continuous domain ant colony optimization (ACO) algorithm to handle the maximum likelihood (ML) estimation problem, we propose a maximum likelihood direction of arrival (ML-DOA) estimation method based on modified ant colony optimization (MACO) algorithm. Firstly, MACO algorithm adopts the elite reverse learning strategy to obtain a better initial solution group. Secondly, the optimization method of ant colony is conducted by combining global cross-neighborhood search and local search of Gaussian kernel function to expand the algorithm search space and accelerate the convergence speed. Finally, the nonlinear global optimal solution of ML estimation method is obtained. Simulation results show that, compared with ML estimation methods based on particle swarm optimization (PSO) algorithm and ACO algorithm, the convergence speed of the ML-MACO algorithm is 4 times faster than that of the ML-ACO algorithm, the computational load is 1/3 that of the ML-ACO algorithm, the resolution success rate is higher than the ML-PSO algorithm and the ML-ACO algorithm, and the estimation error is less than the ML-PSO algorithm and the ML-ACO algorithm. The ML-MACO algorithm maintains the excellent estimation performance of the ML algorithm with lower computational effort, better convergence performance, and higher estimation accuracy.

Deep Learning for Estimation and Pilot Signal Design in Few-Bit Massive MIMO Systems

Estimation in few-bit MIMO systems is challenging, since the received signals are nonlinearly distorted by the low-resolution ADCs. In this paper, we propose a deep learning framework for channel estimation, data detection, and pilot signal design to address the nonlinearity in such systems. The proposed channel estimation and data detection networks are model-driven and have special structures that take advantage of domain knowledge in the few-bit quantization process. While the first data detection network, B-DetNet, is based on a linearized model obtained from the Bussgang decomposition, the channel estimation network and the second data detection network, FBM-CENet and FBM-DetNet respectively, rely on the original quantized system model. To develop FBM-CENet and FBM-DetNet, the maximum-likelihood channel estimation and data detection problems are reformulated to overcome the indeterminant gradient issue. An important feature of the proposed FBM-CENet structure is that the pilot matrix is integrated into the weight matrices of its channel estimator. Thus, training the proposed FBM-CENet enables a joint optimization of both the channel estimator at the base station and the pilot signal transmitted from the users. Simulation results show significant performance gains in estimation accuracy by the proposed deep learning framework.

Majorization-Minimization Method for Elliptic Localization in the Absence of Transmitter Position

This paper investigates the problem of elliptic localization in the absence of transmitter position. An efficient iterative method is developed to jointly evaluate the target and transmitter positions. Using the measurement information from the indirect paths reflected from the target and the direct paths between the transmitter and receivers, a non-convex maximum likelihood estimation (MLE) problem is formulated. Owing to the non-convex nature of the issue, we apply the majorization-minimization (MM) principle to address the MLE problem, which iteratively minimizes a convex surrogate function instead of the original objective function. Moreover, the proposed MM method is further extended to tackle a general scenario where both multiple unknown transmitters and receiver position errors are considered. Finally, numerical simulations demonstrate that the proposed MM method outperforms the state-of-the-art methods.

Three-Dimensional Cooperative Positioning for Internet of Things Provenance

A large number of Internet of Things (IoT) devices have been interconnected for information collection and exchange. The data are only meaningful if it is captured at the expected location (i.e., the IoT devices or sensors are not removed accidentally or intentionally). This article presents a new algorithm, which cooperatively locates multiple IoT devices deployed in a 3-D space based on pairwise Euclidean distance measurements. When the distance measurement noises are negligible, a new feasibility problem of rank-3 variables is formulated. We solve the problem using the difference-of-convex (DC) programming to preserve the rank-3 constraints, rather than relaxing the constraints, using semidefinite relaxation (SDR). When the distance measurements are corrupted by additive noises and nonlight-of-sight (NLOS) propagation, a maximum-likelihood estimation (MLE) problem is formulated and transformed to a DC program solved with the rank-3 constraints preserved. Simulation results indicate that the proposed approach can achieve satisfactory accuracy results with a low complexity and strong robustness to the irregular topology, poor connectivity, and measurement errors, as compared to existing SDR-based alternatives.

Estimators for Space-Time Block-Coded ARTM CPM in Aeronautical Mobile Telemetry

The use of space-time coded ARTM continuous phase modulation to solve the two-antenna problem in aeronautical mobile telemetry requires estimates of the parameters that define the propagation environment. The maximum likelihood estimator problem is defined and used to motivate reduced-complexity estimators suitable for use in a real system. A modified gradient descent algorithm performs the search required to find the delay parameters. An “inner” phase-lock loop operating with an “outer” frequency-lock loop computes decision-directed estimates of the frequency offset. Computer simulations were used to assess the impact on bit error rate performance introduced by the estimators. The simulation results show that the combined joint estimator for the delays, channel gains, and frequency offset imposes a 1.15-dB loss in performance.

Point Cloud Registration Leveraging Structural Regularity in Manhattan World

Point cloud registration is an essential technique for many tasks including autonomous navigation, augmented/virtual reality and scene reconstruction. In man-made environments, abundant plane features with structural regularities are commonly seen, which is actually beneficial for point cloud registration. This paper presents an accurate and robust registration method that formulates the registration problem as a maximum likelihood estimation (MLE) problem where the useful structure information is also leveraged. To align two point sets efficiently, one point set is first represented as a Gaussian mixture models (GMM) tree, and then the Manhattan frame (MF) with dominant planes is estimated and the degenerated nodes corresponding to the dominant planes in the GMM tree are clustered to embody the structure information. The proposed MLE problem is solved by an Expectation Maximization (EM) framework. In the E step, the probabilistic data association between the GMM and another point set is first estimated and then refined by aligning the MF and planes. In the M step, an optimization problem using Mahalanobis distance is solved. Experimental results on both indoor room and hybrid man-made environments show that the proposed method not only achieves a good balance between accuracy and speed, but also provides robust and accurate performance in sequence registration process. The source code is available on our Github <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .

Point set registration via rigid transformation consensus

Point set registration is a fundamental problem in many domains. This paper proposes a novel pair-wise registration algorithm based on the rigid transformation consensus. It starts by building a point correspondence set, which contains both inliers and outliers. Due to non-overlapping regions, it associates each point correspondence with a latent variable and formulates pair-wise registration as a maximum likelihood estimation problem, which is optimized by the expectation-maximum algorithm. Since all inliers follows the consensus of one similar rigid transformation, each correspondence is assigned a posterior probability to indicate whether it is inlier or outlier. To obtain the desired result, it requires to alternatively implement the establishment of point correspondence and maximum likelihood estimation. Given initial rigid transformation, the proposed algorithm is able to obtain a desired registration result for the pair-wise registration. Experiments tested on public available data sets illustrate its superior performance on accuracy and efficiency over previous algorithms.