Deterministic Parameter Vector Research Articles

Sensitivity analysis in seismic reliability of an urban self-anchored suspension bridge

Seismic resistance and disaster prevention for self-anchored suspension bridge (SASB) are important, but the seismic reliability of this complex structure is time-consuming or low-accuracy based on existing methods, let alone the relative sensitivity analysis. A sampling-based hybrid methodology of sensitivity analysis in seismic reliability, combining subset simulation (SuS), explicit time domain method (ETDM), BP neural network (BPNN) and Pearson’s Linear Correlation Coefficient (PLCC), is proposed herein. The separated treatment of low and high variabilities of structural and earthquake parameters based on ETDM with BPNN accelerates the computing efficiency. With the application to the SASB, sensitivities of structural parameters to seismic reliability are obtained, where three largest ones are ambient temperature, Young’s modulus and density of girder. The failure probability of this bridge is mostly smaller than 1.0 × 10-3, but it takes only about 10 ~ 15 min to obtain single failure probability with a deterministic parameter vector based on the proposed method, which is in lieu of hundreds of days based on Monte Carlo simulation. Hence, it is verified a sound approach with great accuracy and efficiency used in engineering structures.

Joint Dimension Assignment and Compression for Deterministic Parameter Vector Estimation in Distributed Multisensor Networks

This article considers distributed estimation of an unknown deterministic parameter vector in a bandwidth constrained multisensor network with a fusion center (FC). Due to the stringent bandwidth requirements, each sensor compresses its observation as a low-dimensional vector via a linear transformation. Then, the FC linearly combines all received compressed data to estimate the deterministic parameter vector based on the best linear unbiased estimator. The problem of interest is to jointly design the dimension assignment (i.e., the compression dimension of each sensor) and the corresponding compression matrix when the total number of compression dimensions is given. Such a joint design problem is formulated as an optimization problem with rank and linear matrix equality constraints, which is shown to be NP-hard for the first time. In addition, penalty decomposition (PD), successive quadratic upper-bound minimization method of multipliers (SQUM-M), and SQUM-M-block coordinate descent (SQUM-M-BCD) are proposed to solve it approximately. Furthermore, we show that any accumulation point of the sequence generated by the PD satisfies the Karush-Kuhn-Tucker conditions of the equivalent formulation of the joint design problem; the SQUM-M admits the same convergence property as the PD under some conditions. Numerical experiments corroborate the merits of PD and SQUM-M-BCD as compared with existing strategies for the heterogeneous scenario, and further illustrate the effectiveness of the proposed algorithms for the correlated noise case.

Compact CRB for delay, Doppler, and phase estimation – application to GNSS SPP and RTK performance characterisation

The derivation of tight estimation lower bounds is a key tool to design and assess the performance of new estimators. In this contribution, first, the authors derive a new compact Cramér–Rao bound (CRB) for the conditional signal model, where the deterministic parameter's vector includes a real positive amplitude and the signal phase. Then, the resulting CRB is particularised to the delay, Doppler, phase, and amplitude estimation for band-limited narrowband signals, which are found in a plethora of applications, making such CRB a key tool of broad interest. This new CRB expression is particularly easy to evaluate because it only depends on the signal samples, then being straightforward to evaluate independently of the particular baseband signal considered. They exploit this CRB to properly characterise the achievable performance of satellite-based navigation systems and the so-called real-time kinematics (RTK) solution. To the best of the authors’ knowledge, this is the first time these techniques are theoretically characterised from the baseband delay/phase estimation processing to position computation, in terms of the CRB and maximum-likelihood estimation.

Closed-Form Solution for Optimal Compression Matrix Design in Distributed Estimation

In this paper, we consider a distributed estimation of a deterministic parameter vector in a bandwidth constrained wireless sensor network. Due to the stringent bandwidth limitation, each sensor compresses its observation to one message transmitted to the fusion center. We assume that the observation noise is uncorrelated across the sensors and focus on a homogeneous case in which all the sensors possess identical noise covariance matrix. Our aim is to design the compression matrix so that the estimation error attains a universal lower bound. When the noise covariance matrix is a scaled identity matrix, we provide a closed-form expression of the optimal compression matrix. Meanwhile, for a general noise covariance matrix, the optimal compression matrix can be attained explicitly as well, if the dimension of the parameter estimated is less than or equal to 4. Furthermore, as a byproduct, two hyperplane-based vector quantization problems can be solved completely and partly, respectively. Performance analysis and simulations demonstrate the merits of our constructed optimal compression matrices when compared with the existing compression strategy.

The Constrained Misspecified Cramér–Rao Bound

The aim of this letter is to provide a constrained version of the misspecified Cramer–Rao bound (MCRB). Specifically, the MCRB is a lower bound on the error covariance matrix of any unbiased (in a proper sense) estimator of a deterministic parameter vector under misspecified models, i.e., when the true and the assumed data distributions are different. Here, we aim at finding an expression of the MCRB for estimation problems involving continuously differentiable equality constraints. Our proof generalizes the derivation of the classical constrained CRB (CCRB) by showing that the constrained MCRB (CMCRB) can be obtained by exploiting the building blocks of its unconstrained counterpart and a basis of the null space of the constraint’s Jacobian matrix. The conditions for the existence of the CMCRB are also discussed.

The Misspecified Cramer-Rao Bound and Its Application to Scatter Matrix Estimation in Complex Elliptically Symmetric Distributions

This paper focuses on the application of recent results on lower bounds under misspecified models to the estimation of the scatter matrix of complex elliptically symmetric (CES) distributed random vectors. Buildings upon the original works of Q. H. Vuong [Cramer-Rao Bounds for Misspecified Models, Div. of the Humanities and Social Sci., California Inst. of Technol., Pasadena, CA, USA, Working Paper 652, Oct. 1986] and Richmond–Horowitz [“Parameter Bounds on Estimation Accuracy Under Model Misspecification,” IEEE Trans. Signal Process., vol. 63, no. 9, pp. 2263–2278, May 2015], a lower bound, named misspecified Cramer–Rao bound (MCRB), for the error covariance matrix of any unbiased (in a proper sense) estimator of a deterministic parameter vector under misspecified models, is introduced. Then, we show how to apply these results to the problem of estimating the scatter matrix of CES distributed data under data mismodeling. In particular, the performance of the maximum likelihood (ML) estimator are compared, under mismatched conditions, with the MCRB and with the classical CRB in some relevant study cases.

Maximum Likelihood Estimation From Sign Measurements With Sensing Matrix Perturbation

The problem of estimating an unknown deterministic parameter vector from sign measurements with a perturbed sensing matrix is studied in this paper. We analyze the best achievable mean square error (MSE) performance by exploring the corresponding Cram\'{e}r-Rao Lower Bound (CRLB). To estimate the parameter, the maximum likelihood (ML) estimator is utilized and its consistency is proved. We show that the perturbation on the sensing matrix exacerbates the performance of ML estimator in most cases. However, suitable perturbation may improve the performance in some special cases. Then we reformulate the original ML estimation problem as a convex optimization problem, which can be solved efficiently. Furthermore, theoretical analysis implies that the perturbation-ignored estimation is a scaled version with the same direction of the ML estimation. Finally, numerical simulations are performed to validate our theoretical analysis.

MSE Bounds With Affine Bias Dominating the CramÉr–Rao Bound

In continuation to an earlier work, we further develop bounds on the mean-squared error (MSE) when estimating a deterministic parameter vector thetas <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> in a given estimation problem, as well as estimators that achieve the optimal performance. The traditional Cramer-Rao (CR) type bounds provide benchmarks on the variance of any estimator of thetas <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> under suitable regularity conditions, while requiring <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</i> <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">priori</i> specification of a desired bias gradient. To circumvent the need to choose the bias, which is impractical in many applications, it was suggested in our earlier work to directly treat the MSE, which is the sum of the variance and the squared-norm of the bias. While previously we developed MSE bounds assuming a <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">linear</i> bias vector, here we study, in the same spirit, <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">affine</i> bias vectors. We demonstrate through several examples that allowing for an affine transformation can often improve the performance significantly over a linear approach. Using convex optimization tools we show that in many cases we can choose an affine bias that results in an MSE bound that is smaller than the unbiased CR bound for all values of thetas <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> . Furthermore, we explicitly construct estimators that achieve these bounds in cases where an efficient estimator exists, by performing an affine transformation of the standard maximum likelihood (ML) estimator. This leads to estimators that result in a smaller MSE than ML for all possible values of thetas <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> .

MV-PURE Estimator: Minimum-Variance Pseudo-Unbiased Reduced-Rank Estimator for Linearly Constrained Ill-Conditioned Inverse Problems

This paper proposes a novel estimator named minimum-variance pseudo-unbiased reduced-rank estimator (MV- PURE) for the linear regression model, designed specially for the case where the model matrix is ill-conditioned and the unknown deterministic parameter vector to be estimated is subjected to known linear constraints. As a natural generalization of the Gauss-Markov (BLUE) estimator, the MV-PURE estimator is a solution of the following hierarchical nonconvex constrained optimization problem directly related to the mean square error expression. In the first-stage optimization, under a rank constraint, we minimize simultaneously all unitarily invariant norms of an operator applied to the unknown parameter vector in view of suppressing bias of the proposed estimator. Then, in the second-stage optimization, among all pseudo-unbiased reduced-rank estimators defined as the solutions of the first-stage optimization, we find the one achieving minimum variance. We derive a closed algebraic form of the MV-PURE estimator and show that well-known estimators-the Gauss-Markov (BLUE) estimator, the generalized Marquardt's reduced-rank estimator, and the minimum-variance conditionally unbiased affine estimator subject to linear restrictions-are all special cases of the MV-PURE estimator. We demonstrate the effectiveness of the proposed estimator in a numerical example, where we employ the MV-PURE estimator to the ill-conditioned problem of reconstructing a 2-D image subjected to linear constraints from blurred, noisy observation. This example demonstrates that the MV-PURE estimator outperforms all aforementioned estimators, as it achieves smaller mean square error for all values of signal-to-noise ratio.

Linear Regression With Gaussian Model Uncertainty: Algorithms and Bounds

In this paper, we consider the problem of estimating an unknown deterministic parameter vector in a linear regression model with random Gaussian uncertainty in the mixing matrix. We prove that the maximum-likelihood (ML) estimator is a (de)regularized least squares estimator and develop three alternative approaches for finding the regularization parameter that maximizes the likelihood. We analyze the performance using the Cramer-Rao bound (CRB) on the mean squared error, and show that the degradation in performance due the uncertainty is not as severe as may be expected. Next, we address the problem again assuming that the variances of the noise and the elements in the model matrix are unknown and derive the associated CRB and ML estimator. We compare our methods to known results on linear regression in the error in variables (EIV) model. We discuss the similarity between these two competing approaches, and provide a thorough comparison that sheds light on their theoretical and practical differences.

A pre-test like estimator dominating the least-squares method

We develop a pre-test type estimator of a deterministic parameter vector β in a linear Gaussian regression model. In contrast to conventional pre-test strategies, that do not dominate the least-squares (LS) method in terms of mean-squared error (MSE), our technique is shown to dominate LS when the effective dimension is greater than or equal to 4. Our estimator is based on a simple and intuitive approach in which we first determine the linear minimum MSE (MMSE) estimate that minimizes the MSE. Since the unknown vector β is deterministic, the MSE, and consequently the MMSE solution, will depend in general on β and therefore cannot be implemented. Instead, we propose applying the linear MMSE strategy with the LS substituted for the true value of β to obtain a new estimate. We then use the current estimate in conjunction with the linear MMSE solution to generate another estimate and continue iterating until convergence. As we show, the limit is a pre-test type method which is zero when the norm of the data is small, and is otherwise a non-linear shrinkage of LS.

Blind Minimax Estimation

We consider the linear regression problem of estimating an unknown, deterministic parameter vector based on measurements corrupted by colored Gaussian noise. We present and analyze blind minimax estimators (BMEs), which consist of a bounded parameter set minimax estimator, whose parameter set is itself estimated from measurements. Thus, one does not require any prior assumption or knowledge, and the proposed estimator can be applied to any linear regression problem. We demonstrate analytically that the BMEs strictly dominate the least-squares estimator, i.e., they achieve lower mean-squared error for any value of the parameter vector. Both Stein's estimator and its positive-part correction can be derived within the blind minimax framework. Furthermore, our approach can be readily extended to a wider class of estimation problems than Stein's estimator, which is defined only for white noise and non-transformed measurements. We show through simulations that the BMEs generally outperform previous extensions of Stein's technique.

Uniformly Improving the CramÉr-Rao Bound and Maximum-Likelihood Estimation

An important aspect of estimation theory is characterizing the best achievable performance in a given estimation problem, as well as determining estimators that achieve the optimal performance. The traditional Cramer-Rao type bounds provide benchmarks on the variance of any estimator of a deterministic parameter vector under suitable regularity conditions, while requiring a-priori specification of a desired bias gradient. In applications, it is often not clear how to choose the required bias. A direct measure of the estimation error that takes both the variance and the bias into account is the mean squared error (MSE), which is the sum of the variance and the squared-norm of the bias. Here, we develop bounds on the MSE in estimating a deterministic parameter vector x0 over all bias vectors that are linear in x0, which includes the traditional unbiased estimation as a special case. In some settings, it is possible to minimize the MSE over all linear bias vectors. More generally, direct minimization is not possible since the optimal solution depends on the unknown x0. Nonetheless, we show that in many cases, we can find bias vectors that result in an MSE bound that is smaller than the Cramer-Rao lower bound (CRLB) for all values of x0. Furthermore, we explicitly construct estimators that achieve these bounds in cases where an efficient estimator exists, by performing a simple linear transformation on the standard maximum likelihood (ML) estimator. This leads to estimators that result in a smaller MSE than the ML approach for all possible values of x0

Maximum likelihood estimation in linear models with a Gaussian model matrix

We consider the problem of estimating an unknown deterministic parameter vector in a linear model with a Gaussian model matrix. We derive the maximum likelihood (ML) estimator for this problem and show that it can be found using a simple line-search over a unimodal function that can be efficiently evaluated. We then discuss the similarity between the ML, the total least squares (TLS), the regularized TLS, and the expected least squares estimators.

Comparing between estimation approaches: admissible and dominating linear estimators

We treat the problem of evaluating the performance of linear estimators for estimating a deterministic parameter vector x in a linear regression model, with the mean-squared error (MSE) as the performance measure. Since the MSE depends on the unknown vector x, a direct comparison between estimators is a difficult problem. Here, we consider a framework for examining the MSE of different linear estimation approaches based on the concepts of admissible and dominating estimators. We develop a general procedure for determining whether or not a linear estimator is MSE admissible, and for constructing an estimator strictly dominating a given inadmissible method so that its MSE is smaller for all x. In particular, we show that both problems can be addressed in a unified manner for arbitrary constraint sets on x by considering a certain convex optimization problem. We then demonstrate the details of our method for the case in which x is constrained to an ellipsoidal set and for unrestricted choices of x. As a by-product of our results, we derive a closed-form solution for the minimax MSE estimator on an ellipsoid, which is valid for arbitrary model parameters, as long as the signal-to-noise-ratio exceeds a certain threshold.

Minimax estimation of deterministic parameters in linear models with a random model matrix

We consider the problem of estimating an unknown deterministic parameter vector in a linear model with a random model matrix, with known second-order statistics. We first seek the linear estimator that minimizes the worst-case mean-squared error (MSE) across all parameter vectors whose (possibly weighted) norm is bounded above. We show that the minimax MSE estimator can be found by solving a semidefinite programming problem and develop necessary and sufficient optimality conditions on the minimax MSE estimator. Using these conditions, we derive closed-form expressions for the minimax MSE estimator in some special cases. We then demonstrate, through examples, that the minimax MSE estimator can improve the performance over both a Baysian approach and a least-squares method. We then consider the case in which the norm of the parameter vector is also bounded below. Since the minimax MSE approach cannot account for a nonzero lower bound, we consider, in this case, a minimax regret method in which we seek the estimator that minimizes the worst-case difference between the MSE attainable using a linear estimator that does not know the parameter vector, and the optimal MSE attained using a linear estimator that knows the parameter vector. For analytical tractability, we restrict our attention to the scalar case and develop a closed-form expression for the minimax regret estimator.

Minimax MSE estimation of deterministic parameters with noise covariance uncertainties

In this paper, a minimax mean-squared error (MSE) estimator is developed for estimating an unknown deterministic parameter vector in a linear model, subject to noise covariance uncertainties. The estimator is designed to minimize the worst-case MSE across all norm-bounded parameter vectors, and all noise covariance matrices, in a given region of uncertainty. The minimax estimator is shown to have the same form as the estimator that minimizes the worst-case MSE over all norm-bounded vectors for a least-favorable choice of the noise covariance matrix. An example demonstrating the performance advantage of the minimax MSE approach over the least-squares and weighted least-squares methods is presented.

Maximum set estimators with bounded estimation error

We consider the linear regression problem of estimating a deterministic parameter vector x from observations y = Hx + w, where H is known, and w is additive noise. We seek an estimator whose estimation error does not exceed a given maximum error for as wide a range of conditions as possible. The maximum error is a design choice and is generally lower than the error provided by the well-known least-squares (LS) estimator. We develop estimators guaranteeing the required error for as large a parameter set as possible and for as large a noise level as possible. We discuss methods for finding these estimators and demonstrate that in many cases, the proposed estimators outperform the LS estimator.