General Estimation Problem Research Articles

A new method for the approximation of certain non-linear systems

Abstract A common problem in systems and control theory is to provide an approximation to non-linear systems. We provide a novel approach as a general solution to this problem originally conceived by Gamkrelidze. We consider and solve a general approximation problem which provides the fundamentals for various switching-type systems thus encompassing a wide range of systems theory problems.

A practical modeling of circuit breakers with unknown statuses in generalized state estimators

This paper proposes a new practical approach for including circuit breakers with unknown statuses in the formulation of the generalized state estimation problem. Unlike other proposals, only one single operating constraint is proposed for representing two possible and mutually exclusive breaker statuses: closed or open. This is achieved by using the concept of complementarity conditions based on the apparent power flow and complex voltages across the breaker terminals. This complementarity constraint is then transformed into an equality constraint by using the Fischer-Burmeister merit function, and it is directly included into the state estimation problem to simultaneously co-estimate the operating state of the entire network together with the breaker's unknown operating statuses. The proposal's effectiveness is demonstrated in the IEEE 24-bus benchmark system with different substation configurations and circuit breakers with unknown statuses.

Joint optimal measurement for locating two incoherent optical point sources near the Rayleigh distance

Simultaneously optimizing the estimation of the centroid and separation of two incoherent optical point sources is constrained by a tradeoff relation determined by an incompatibility coefficient. At the Rayleigh distance, the incompatibility coefficient vanishes and thus the tradeoff relation no longer restricts the simultaneous optimization of measurement for a joint estimation. We construct such a joint optimal measurement by an elaborated analysis on the operator algebra of the symmetric logarithmic derivative. Our work not only confirms the existence of a joint optimal measurement for this specific imaging model, but also gives a promising method to characterize the condition on measurement compatibility for general multiparameter estimation problems.

Discussion of “Multivariate dynamic modeling for Bayesian forecasting of business revenue”

Discussion of “Multivariate dynamic modeling for Bayesian forecasting of business revenue”

Privacy-Preserving in-bed pose monitoring: A fusion and reconstruction study

Background and objectives:Recently, in-bed human pose estimation has attracted the interest of researchers due to its relevance to a wide range of healthcare applications. Compared to the general problem of human pose estimation, in-bed pose estimation has several inherent challenges, the most prominent being frequent and severe occlusions caused by bedding. In this paper we explore the effective use of images from multiple non-visual and privacy-preserving modalities such as depth, long-wave infrared (LWIR) and pressure maps for the task of in-bed pose estimation in two settings. First, we explore the effective fusion of information from different imaging modalities for better pose estimation. Secondly, we propose a framework that can estimate in-bed pose estimation when visible images are unavailable, and demonstrate the applicability of fusion methods to scenarios where only LWIR images are available. Method:We analyze and demonstrate the effect of fusing features from multiple modalities. For this purpose, we consider four different techniques: (1) Addition, (2) Concatenation, (3) Fusion via learned modal weights, and 4) End-to-end fully trainable approach; with a state-of-the-art pose estimation model. We also evaluate the effect of reconstructing a data-rich modality (i.e., visible modality) from a privacy-preserving modality with data scarcity (i.e., long-wavelength infrared) for in-bed human pose estimation. For reconstruction, we use a conditional generative adversarial network. Results:We conduct experiments on a publicly available dataset for feature fusion and visible image reconstruction. We conduct ablative studies across different design decisions of our framework. This includes selecting features with different levels of granularity, using different fusion techniques, and varying model parameters. Through extensive evaluations, we demonstrate that our method produces on par or better results compared to the state-of-the-art. Conclusion:The insights from this research offer stepping stones towards robust automated privacy-preserving systems that utilize multimodal feature fusion to support the assessment and diagnosis of medical conditions.

Adaptive multitask clustering algorithm based on distributed diffusion least-mean-square estimation

Distributed estimation with multitask clustering in wireless sensor networks has always been a challenging and attractive topic. When the multitask estimation problem becomes complex and general, such as more complex parameters and more general networks, many multitask clustering algorithms may experience significant performance degradation. To overcome these situations, a highly robust multitask clustering algorithm is proposed in this paper, which can be applied to more general and complex multitask estimation problems. The proposed algorithm is considered as a hybrid system. One subsystem is used to provide a reference estimation, which is further used to perform multitask clustering. Based on the clustering network, the other subsystem is performed to get a distributed adaptive estimation. In addition, each node is considered as a relaying node, so it can obtain the relaying information across the network for enhancing the performance. The convergence performance of the proposed algorithms is analyzed in the mean and mean-square senses. Simulation results demonstrate the robustness and accuracy of the proposed algorithms as compared to other multitask clustering algorithms.

The generalized orthogonal Procrustes problem in the high noise regime

AbstractWe consider the problem of estimating a cloud of points from numerous noisy observations of that cloud after unknown rotations and possibly reflections. This is an instance of the general problem of estimation under group action, originally inspired by applications in three-dimensional imaging and computer vision. We focus on a regime where the noise level is larger than the magnitude of the signal, so much so that the rotations cannot be estimated reliably. We propose a simple and efficient procedure based on invariant polynomials (effectively: the Gram matrices) to recover the signal, and we assess it against fundamental limits of the problem that we derive. We show our approach adapts to the noise level and is statistically optimal (up to constants) for both the low and high noise regimes. In studying the variance of our estimator, we encounter the question of the sensivity of a type of thin Cholesky factorization, for which we provide an improved bound which may be of independent interest.

Nonparametric Estimation of Continuously Parametrized Families of Probability Density Functions—Computational Aspects

We consider a rather general problem of nonparametric estimation of an uncountable set of probability density functions (p.d.f.’s) of the form: f ( x ; r ) , where r is a non-random real variable and ranges from R 1 to R 2 . We put emphasis on the algorithmic aspects of this problem, since they are crucial for exploratory analysis of big data that are needed for the estimation. A specialized learning algorithm, based on the 2D FFT, is proposed and tested on observations that allow for estimate p.d.f.’s of a jet engine temperatures as a function of its rotation speed. We also derive theoretical results concerning the convergence of the estimation procedure that contains hints on selecting parameters of the estimation algorithm.

Unbiased Least-Squares Modelling

In this paper we analyze the bias in a general linear least-squares parameter estimation problem, when it is caused by deterministic variables that have not been included in the model. We propose a method to substantially reduce this bias, under the hypothesis that some a-priori information on the magnitude of the modelled and unmodelled components of the model is known. We call this method Unbiased Least-Squares (ULS) parameter estimation and present here its essential properties and some numerical results on an applied example.

Witnessing entanglement in experiments with correlated noise

The purpose of an entanglement witness experiment is to certify the creation of an entangled state from a finite number of trials. The statistical confidence of such an experiment is typically expressed as the number of observed standard deviations of witness violations. This method implicitly assumes that the noise is well-behaved so that the central limit theorem applies. In this work, we propose two methods to analyze witness experiments where the states can be subject to arbitrarily correlated noise. Our first method is a rejection experiment, in which we certify the creation of entanglement by rejecting the hypothesis that the experiment can only produce separable states. We quantify the statistical confidence by a p-value, which can be interpreted as the likelihood that the observed data is consistent with the hypothesis that only separable states can be produced. Hence a small p-value implies large confidence in the witnessed entanglement. The method applies to general witness experiments and can also be used to witness genuine multipartite entanglement. Our second method is an estimation experiment, in which we estimate and construct confidence intervals for the average witness value. This confidence interval is statistically rigorous in the presence of correlated noise. The method applies to general estimation problems, including fidelity estimation. To account for systematic measurement and random setting generation errors, our model takes into account device imperfections and we show how this affects both methods of statistical analysis. Finally, we illustrate the use of our methods with detailed examples based on a simulation of NV centers.

MML Is Not Consistent for Neyman-Scott

Strict Minimum Message Length (SMML) is an information-theoretic statistical inference method widely cited (but only with informal arguments) as providing estimations that are consistent for general estimation problems. It is, however, almost invariably intractable to compute, for which reason only approximations of it (known as MML algorithms) are ever used in practice. Using novel techniques that allow for the first time direct, non-approximated analysis of SMML solutions, we investigate the Neyman-Scott estimation problem, an oft-cited showcase for the consistency of MML, and show that even with a natural choice of prior neither SMML nor its popular approximations are consistent for it, thereby providing a counterexample to the general claim. This is the first known explicit construction of an SMML solution for a natural, high-dimensional problem.

Performance guarantees for a variational “multi-space” decoder

Model-order reduction methods tackle the following general approximation problem: find an “easily-computable” but accurate approximation $\hat {\boldsymbol {h}}$ of some target solution h⋆. In order to achieve this goal, standard methodologies combine two main ingredients: (i) a set of partial observations of h⋆ and (ii) some “simple” prior model on the set of target solutions. The most common prior models encountered in the literature assume that the target solution h⋆ is “close” to some low-dimensional subspace. Recently, triggered by the work by Binev et al. (SIAM/ASA Journal on Uncertainty Quantification 5(1), 1–29, 2017), several contributions have shown that refined prior models (based on a set of embedded approximation subspaces) may lead to enhanced approximation performance. In this paper, we focus on a particular decoder exploiting such a “multi-space” information and evaluating $\hat {\boldsymbol {h}}$ as the solution of a constrained optimization problem. To date, no theoretical results have been derived to support the good empirical performance of this decoder. The goal of the present paper is to fill this gap. More specifically, we provide a mathematical characterization of the approximation performance achievable by this variational “multi-space” decoder and emphasize that, in some specific setups, it has provably better recovery guarantees than its standard “single-space” counterpart. We also discuss the similarities and differences between this decoder and the one proposed in Binev et al. (SIAM/ASA Journal on Uncertainty Quantification 5(1), 1–29, 2017).

Channel Estimation: Unified View of Optimal Performance and Pilot Sequences

Channel estimation is of paramount importance in most communication systems in order to optimize the data rate/energy consumption tradeoff. In modern systems, the possibly large number of transmit/receive antennas and subcarriers makes this task difficult. Designing pilot sequences of reasonable size yielding good performance is thus critical. Classically, the number of pilots is reduced by viewing the channel as a random vector and assuming knowledge of its distribution. In practice, this requires estimating the channel covariance matrix, which can be computationally costly and not adapted to scenarios with high mobility. In this paper, an alternative view is considered, in which the channel is a function of unknown deterministic parameters. In this setting, the problem of designing optimal pilot sequences of smallest possible size is studied for any parametric channel model. To do so, the Cram{\'e}r-Rao bound (CRB) for this general channel estimation problem is given, highlighting its key dependency on the introduced variation space. Then, the minimal size of pilot sequences and minimal value of the CRB are determined. Moreover, a general strategy to build optimal minimal length power constrained pilots sequences is given, based on an estimation of the variation space. The theoretical results are finally illustrated in a massive MIMO system context. They conveniently allow to retrieve well known previous results, but also to exhibit minimal length optimal pilot sequences for a new strategy based on a nonlinear physical model.

Framework of Channel Estimation for Hybrid Analog-and-Digital Processing Enabled Massive MIMO Communications

We investigate a general channel estimation problem in the massive multiple-input multiple-output system which employs the hybrid analog/digital precoding structure with limited radio-frequency (RF) chains. By properly designing RF combiners and performing multiple trainings, the proposed channel estimation can approach the performance of fully-digital estimations depending on the degree of channel spatial correlation and the number of RF chains. Dealing with the hybrid channel estimation, the optimal combiner is theoretically derived by relaxing the constant-magnitude constraint in a specific single-training scenario, which is then extended to the design of combiners for multiple trainings by sequential and alternating methods. Further, we develop a technique to generate the phase-only RF combiners based on the corresponding unconstrained ones to satisfy the constant-magnitude constraints. The performance of the proposed hybrid channel estimation scheme is examined by simulations under both nonparametric and spatial channel models. The simulation results demonstrate that the estimated channel state information can approach the performance of fully-digital estimations in terms of both mean square error and spectral efficiency. Moreover, a practical spatial channel covariance estimation method is proposed and its effectiveness in hybrid channel estimation is verified by simulations.

A perspective on CLMS as a deficient length augmented CLMS: Dealing with second order noncircularity

Mean and mean square performance analyses of the strictly linear complex least mean square (CLMS) algorithm are addressed for widely linear estimation (WLE) of second order noncircular (improper) Gaussian inputs, for which both the covariance and pseudo-covariance matrices contain nonzero off-diagonal elements. A detailed performance analysis of standard CLMS in this ‘suboptimal’ context is not trivial but is important for practical applications. To this end, we here consider the strictly linear CLMS as a ‘deficient length’ version of the widely linear augmented CLMS (ACLMS) algorithm, which is second order optimal for improper inputs. Rigorous performance analysis is provided, which also statistically quantifies the suboptimality of CLMS in both the transient and steady state stages. The recently introduced approximate uncorrelating transform (AUT) is employed to derive closed-form expressions for the mean square stability and the steady-state performance of CLMS. In addition, since WLE with second order noncircular inputs is a general linear estimation problem in the complex domain C, these results provide a generalised framework from which current statistical descriptions of CLMS for strictly linear estimation (SLE) can be deduced as special cases. Simulations in system identification settings validate the findings.

Achieving Cramer–Rao Lower Bounds in Sensor Network Estimation

Achievable bounds for parametric estimation using sensor networks are known for some specific problems. In this article, achievable bounds for the general parametric estimation problem with fixed and varying parameters in the network and for various noise conditions on sensor nodes are studied. The general CramerRao lower bounds (CRLBs) of parameter estimates using sensor array network are derived. Procedures for reaching these bounds can be designed based on the obtained results. The accuracy of derived relationships is confirmed by an example. It is shown that it is possible to develop algorithms of moderate complexity and communication cost achieving the CRLB.

A mathematical model of modelling – epistemology and natural sciences

Here, a theoretical framework is proposed which enables us to formalize the process of mathematical modelling and to discuss the recurrent question whether a chosen model is appropriate, accurate or comprehensive with respect to a certain application. Thereto, episte-mological problems in philosophy are separated from modelling questions in the natural sciences. The background of the investigations lies in life-science applications, their inherent qualitative and quantitative uncertainties and their oftentimes unclear hierarchy of the mechanisms under consideration. The resulting description of mathematical modelling as a general approximation problem is still sufficiently rich to study model families, inheritance of model properties, model identification, minimal models and the optimality of models under different objective functions.

Intervehicle Range Estimation From Periodic Broadcasts

Dedicated short-range communications enables vehicular communication using periodic broadcast messages. We propose to use these periodic broadcasts to perform intervehicle ranging. Motivated by this scenario, we study the general problem of range estimation between pairs of moving vehicles using periodic broadcasts. Each vehicle has its own independent and unsynchronized clock, which can exhibit significant drift between consecutive periodic broadcast transmissions. Consequently, both the clock offsets and drifts need to be taken into account in addition to the vehicle motion to accurately estimate the vehicle ranges. We develop a range estimation algorithm using local polynomial smoothing of the vehicle motion. The proposed algorithm can be applied to networks with arbitrary number of vehicles and requires no additional message exchanges apart from the periodic broadcasts. We experimentally validate our algorithm and show that its performance is close to that of unicast round-trip time ranging. In particular, we are able to achieve submeter ranging accuracies in vehicular scenarios. Our scheme requires additional timestamp information to be transmitted as part of the broadcast messages, and we develop a novel timestamp compression algorithm to minimize the resulting overhead.

Weiss–Weinstein Bound on Multiple Change-Points Estimation

In the context of multiple change-points estimation, performance analysis of estimators such as the maximum likelihood is often difficult to assess since the regularity assumptions are not met. Focusing on the estimators variance, one can, however, use lower bounds on the mean square error. In this paper, we derive the so-called Weiss–Weinstein bound (WWB) that is known to be an efficient tool in signal processing to obtain a fair overview of the estimation behavior. Contrary to several works about performance analysis in the change-point literature, our study is adapted to multiple changes. First, useful formulas are given for a general estimation problem whatever the considered distribution of the data. Second, closed-form expressions are given in the cases of Gaussian observations with changes in the mean and/or the variance, and changes in the mean rate of a Poisson distribution. Furthermore, a semidefinite programming formulation of the minimization procedure is given in order to compute the tightest WWB. Specifically, it consists of finding the unique minimum volume covering the set constituted by hyperellipsoid elements that are generated using the derived candidate WWB matrices w.r.t. the so-called Loewner partial ordering. Finally, simulation results are provided to show the good behavior of the proposed bound.

Recovering low-rank and sparse matrix based on the truncated nuclear norm

Recovering the low-rank, sparse components of a given matrix is a challenging problem that arises in many real applications. Existing traditional approaches aimed at solving this problem are usually recast as a general approximation problem of a low-rank matrix. These approaches are based on the nuclear norm of the matrix, and thus in practice the rank may not be well approximated. This paper presents a new approach to solve this problem that is based on a new norm of a matrix, called the truncated nuclear norm (TNN). An efficient iterative scheme developed under the linearized alternating direction method multiple framework is proposed, where two novel iterative algorithms are designed to recover the sparse and low-rank components of matrix. More importantly, the convergence of the linearized alternating direction method multiple on our matrix recovering model is discussed and proved mathematically. To validate the effectiveness of the proposed methods, a series of comparative trials are performed on a variety of synthetic data sets. More specifically, the new methods are used to deal with problems associated with background subtraction (foreground object detection), and removing shadows and peculiarities from images of faces. Our experimental results illustrate that our new frameworks are more effective and accurate when compared with other methods.