Constrained Cramer-Rao Bound Research Articles

Maximum Likelihood and IRLS Based Moving Source Localization with Distributed Sensors

In this article, we consider the problem of estimating the location and velocity of a moving source using a distributed passive radar sensor network. We first derive the maximum likelihood estimator (MLE) using direct sensor observations, when the source signal is unknown and modeled as a deterministic process. Since the MLE obtains the source location and velocity estimates through a search process over the parameter space and is quite computationally intensive, we also developed an efficient algorithm to solve the problem using a two-step approach. The first step finds the time difference of arrival (TDOA) and frequency difference of arrival (FDOA) estimates for each sensor with respect to a reference sensor by using a two-dimensional fast Fourier transform and interpolation, while the second step employs an iterative reweighted least square (IRLS) approach with a varying weighting matrix to determine the source location and velocity. To benchmark the performance of the proposed methods, a constrained Cramer–Rao bound (CRB) for the considered source localization problem is derived. Numerical results show that the IRLS approach has a lower signal-to-noise ratio threshold phenomenon compared with several recent TDOA/FDOA-based methods, especially when the source is considerably farther away from some sensors than others, creating a larger disparity in the quality of sensors observations.

Cramér-Rao Bound Under Norm Constraint

The constrained Cramer-Rao bound (CCRB) is a benchmark for constrained parameter estimation. However, the CCRB unbiasedness conditions are too strict and thus, the CCRB may not be a lower bound for estimators under constraints. The recently developed Lehmann-unbiased-CCRB (LU-CCRB) was shown to be a lower bound for the commonly used constrained maximum likelihood (CML) estimator performance in cases where the CCRB is not. In constrained parameter estimation, the estimator is usually required to satisfy the constraints. However, the LU-CCRB is a lower bound for Lehmann-unbiased estimators that do not necessarily satisfy the constraints. In this letter, we consider the norm constraint and derive a novel bound, called norm-constrained CCRB (NC-CCRB), which is a lower bound on the mean-squared-error matrix trace of Lehmann-unbiased estimators that satisfy the norm constraint. The NC-CCRB is shown to be tighter than the LU-CCRB. In the simulations, we consider a linear estimation problem under norm constraint in which the proposed NC-CCRB better predicts the performance of the CML estimator than the CCRB trace and the LU-CCRB.

Towards better estimability of electrode-specific state of health: Decoding the cell expansion

Li-ion batteries are prone to adverse physical and chemical mechanisms that can degrade their performance over time. For this reason, identifying each electrode's capacity and utilization window is important for the safe operation of the battery, unfortunately the standard capacity estimation method cannot provide this. In this work, we introduce electrode-specific State of Health (eSOH) related parameters, including individual electrode capacity and utilization window. We explore the identifiability of the parameters using terminal voltage alone and voltage plus cell expansion measurements. The analysis here is based on the constrained Cramer-Rao Bound (CRB) formulation, which provides the error bounds for the parameters. The model utilizes the voltage/expansion functions of lithium stoichiometry for the individual electrodes based on the underlying physics of phase transitions. It is shown that slope changes in voltage and expansion that correspond to phase transitions in the electrodes enhance the estimation. As a result, with the addition of the expansion, the parameters are estimable without the need to discharge the battery to a high Depth of Discharge (>70%), which rarely happens in automotive applications. This makes eSOH estimation feasible for a wider range of real-world driving scenarios.

Cram$\acute{\text{e}}$r–Rao Bound for Constrained Parameter Estimation Using Lehmann-Unbiasedness

The constrained Cramer-Rao bound (CCRB) is a lower bound on the mean-squared-error (MSE) of estimators that satisfy some unbiasedness conditions. Although the CCRB unbiasedness conditions are satisfied asymptotically by the constrained maximum likelihood (CML) estimator, in the non-asymptotic region these conditions are usually too strict and the commonly used estimators, such as the CML estimator, do not satisfy them. Therefore, the CCRB may not be a lower bound on the MSE matrix of such estimators. In this paper, we propose a new definition for unbiasedness under constraints, denoted by C-unbiasedness, which is based on using Lehmann-unbiasedness with a weighted MSE (WMSE) risk and taking into account the parametric constraints. In addition to C-unbiasedness, a Cramer-Rao-type bound on the WMSE of C-unbiased estimators, denoted as Lehmann-unbiased CCRB (LU-CCRB), is derived. This bound is a scalar bound that depends on the chosen weighted combination of estimation errors. It is shown that C-unbiasedness is less restrictive than the CCRB unbiasedness conditions. Thus, the set of estimators that satisfy the CCRB unbiasedness conditions is a subset of the set of C-unbiased estimators and the proposed LU-CCRB may be an informative lower bound in cases where the corresponding CCRB is not. In the simulations, we examine linear and nonlinear estimation problems under nonlinear constraints in which the CML estimator is shown to be C-unbiased and the LU-CCRB is an informative lower bound on the WMSE, while the corresponding CCRB on the WMSE is not a lower bound and is not informative in the non-asymptotic region.

Matched, mismatched, and robust scatter matrix estimation and hypothesis testing in complex t-distributed data

Scatter matrix estimation and hypothesis testing are fundamental inference problems in a wide variety of signal processing applications. In this paper, we investigate and compare the matched, mismatched, and robust approaches to solve these problems in the context of the complex elliptically symmetric (CES) distributions. The matched approach is when the estimation and detection algorithms are tailored on the correct data distribution, whereas the mismatched approach refers to the case when the scatter matrix estimator and the decision rule are derived under a model assumption that is not correct. The robust approach aims at providing good estimation and detection performance, even if suboptimal, over a large set of possible data models, irrespective of the actual data distribution. Specifically, due to its central importance in both the statistical and engineering applications, we assume for the input data a complex t-distribution. We analyze scatter matrix estimators derived under the three different approaches and compare their mean square error (MSE) with the constrained Cramer-Rao bound (CCRB) and the constrained misspecified Cramer-Rao bound (CMCRB). In addition, the detection performance and false alarm rate (FAR) of the various detection algorithms are compared with that of the clairvoyant optimum detector.

MIMO Radar Waveform Design with Prior Information of Targets in the Presence of Signal-Dependent Noise

Multiple-input multiple-output (MIMO) radar waveforms design with specified properties has a number of superiority over its phased-array counterpart, such as clutter suppression and interference mitigation. In this paper, we consider the problem of waveform optimization with prior information on targets of interest to improve the parameter estimation performance of MIMO radar in the presence of signal-dependent noise, which is based on the constrained Cramer-Rao bound (CRB). The waveform covariance matrix (WCM) is designed to minimize the trace of the constrained CRB such that the parameter estimation performance can be improved. In order to solve the resultant nonlinear optimization problem, a novel diagonal loading (DL) based method is proposed to relax this optimization issue as a semidefinite programming (SDP) one, which can be solved very efficiently. Following that, an optimal solution to the initial issue can be obtained via the least squares (LS) fitting of the solution acquired by the relaxed one. The effectiveness of the proposed method is verified by numerical examples, as compared to the uncorrelated waveforms.

On the Performance Bound of Sparse Estimation With Sensing Matrix Perturbation

This paper focusses on the sparse estimation in the situation where both the the sensing matrix and the measurement vector are corrupted by additive Gaussian noises. The performance bound of sparse estimation is analyzed and discussed in depth. Two types of lower bounds, the constrained Cram\'{e}r-Rao bound (CCRB) and the Hammersley-Chapman-Robbins bound (HCRB), are discussed. It is shown that the situation with sensing matrix perturbation is more complex than the one with only measurement noise. For the CCRB, its closed-form expression is deduced. It demonstrates a gap between the maximal and nonmaximal support cases. It is also revealed that a gap lies between the CCRB and the MSE of the oracle pseudoinverse estimator, but it approaches zero asymptotically when the problem dimensions tend to infinity. For a tighter bound, the HCRB, despite of the difficulty in obtaining a simple expression for general sensing matrix, a closed-form expression in the unit sensing matrix case is derived for a qualitative study of the performance bound. It is shown that the gap between the maximal and nonmaximal cases is eliminated for the HCRB. Numerical simulations are performed to verify the theoretical results in this paper.

Constrained Cramér–Rao Bound on Robust Principal Component Analysis

We investigate the behavior of the mean-square error (MSE) of low-rank and sparse matrix decomposition, in particular the special case of the robust principal component analysis (RPCA), and its generalization matrix completion and correction (MCC). We derive a constrained Cramer-Rao bound (CRB) for any locally unbiased estimator of the low-rank matrix and of the sparse matrix. We analyze the typical behavior of the constrained CRB for MCC where a subset of entries of the underlying matrix are randomly observed, some of which are grossly corrupted. We obtain approximated constrained CRBs by using a concentration of measure argument. We design an alternating minimization procedure to compute the maximum-likelihood estimator (MLE) for the low-rank matrix and the sparse matrix, assuming knowledge of the rank and the sparsity level. For relatively small rank and sparsity level, we demonstrate numerically that the performance of the MLE approaches the constrained CRB when the signal-to-noise-ratio is high. We discuss the implications of these bounds and compare them with the empirical performance of the accelerated proximal gradient algorithm as well as other existing bounds in the literature.

Lower Bounds on the Mean-Squared Error of Low-Rank Matrix Reconstruction

We investigate the behavior of the mean-square error (MSE) of low-rank matrix reconstruction and its special case, matrix completion. We first derive the constrained Cramer-Rao bound (CRB) on the MSE matrix of any locally unbiased estimator, and then analyze the behavior of the constrained CRB when a subset of entries of the underlying matrix is randomly observed. We design an alternating minimization procedure to compute the maximum likelihood estimator (MLE) for the low-rank matrix, and demonstrate through numerical simulations that the performance of the MLE approaches the constrained CRB when the signal-to-noise ratio is high. Applying a Chapman-Robbins type Barankin bound allows us to derive lower bounds on the worst-case scalar MSE. We demonstrate that the worst-case scalar MSE is infinite even if the model is identifiable. However, the infinite scalar MSE is achieved only on a set of low-rank matrices with measure zero. We discuss the implications of these bounds and compare them with the empirical performance of the matrix LASSO estimator and the existing bounds in the literature.

A New Derivation of Constrained Cramér–Rao Bound Via Norm Minimization

The constrained Cramer-Rao bound (CCRB) is widely used to evaluate the estimation performance for deterministic parameters with parametric constraints. An alternative derivation of the CCRB is presented from the perspective of solving a norm minimization problem under a set of linear constraints. The parametric constraints are embedded in the linear constraints by constructing a proper basis to represent the estimation error. This derivation avoids sophisticated matrix manipulations and has a clear physical meaning that the parametric constraints cut down the bases used to represent the estimation error and hence reduce the minimum norm of the error, and the bound can be achieved if and only if the error is in the subspace spanned by the bases. The result is applicable to biased estimators and singular Fisher information matrices.

The Cramér-Rao Bound for Estimating a Sparse Parameter Vector

The goal of this contribution is to characterize the best achievable mean-squared error (MSE) in estimating a sparse deterministic parameter from measurements corrupted by Gaussian noise. To this end, an appropriate definition of bias in the sparse setting is developed, and the constrained Cramer-Rao bound (CRB) is obtained. This bound is shown to equal the CRB of an estimator with knowledge of the support set, for almost all feasible parameter values. Consequently, in the unbiased case, our bound is identical to the MSE of the oracle estimator. Combined with the fact that the CRB is achieved at high signal-to-noise ratios signal-to-noise ratio (SNRs) by the maximum likelihood technique, our result provides a new interpretation for the common practice of using the oracle estimator as a gold standard against which practical approaches are compared.

On the Constrained CramÉr–Rao Bound With a Singular Fisher Information Matrix

The problem considered in this letter is to bound the performance of estimators of a deterministic parameter which satisfies given constraints. Specifically, previous work on the constrained Cramer-Rao bound (CRB) is generalized to include singular Fisher information matrices and biased estimators. A necessary and sufficient condition for the existence of a finite CRB is obtained. A closed form for an estimator achieving the CRB, if one exists, is also provided, as well as a necessary and sufficient condition for the existence of such an estimator. It is shown that biased estimators achieving the CRB can be constructed in situations for which no unbiased technique exists.

Maximum-Likelihood Estimation, the CramÉr–Rao Bound, and the Method of Scoring With Parameter Constraints

Maximum-likelihood (ML) estimation is a popular approach to solving many signal processing problems. Many of these problems cannot be solved analytically and so numerical techniques such as the method of scoring are applied. However, in many scenarios, it is desirable to modify the ML problem with the inclusion of additional side information. Often this side information is in the form of parametric constraints, which the ML estimate (MLE) must now satisfy. We unify the asymptotic constrained ML (CML) theory with the constrained Cramer-Rao bound (CCRB) theory by showing the CML estimate (CMLE) is asymptotically efficient with respect to the CCRB. We also generalize the classical method of scoring using the CCRB to include the constraints, satisfying the constraints after each iterate. Convergence properties and examples verify the usefulness of the constrained scoring approach. As a particular example, an alternative and more general CMLE is developed for the complex parameter linear model with linear constraints. A novel proof of the efficiency of this estimator is provided using the CCRB.

The Constrained CramÉr–Rao Bound From the Perspective of Fitting a Model

Stoica and Ng (1998) presented a simple expression for the constrained Cramer-Rao bound (CCRB) when the constraints are given by a differentiable function of the parameter to be estimated. This letter considers the parallel case in developing the CCRB when the parameters are locally fitted to a lower-dimensional parametric model, i.e., the parameters are locally assumed to be functions of a distinct reduced parameter vector. We employ classical elements of CRB theory on the locally fitted model to present a very simple derivation of the CCRB, conditions for attaining the bound, and a regularity condition. Examples illustrate the key ideas.