Rational Covariance Extension Problem Research Articles

A Generalized Multidimensional Circulant Rational Covariance and Cepstral Extension Problem

Abstract We propose a general scenario to estimate the spectral density of an homogeneous random field from its moments. More precisely, we consider a multidimensional rational covariance and cepstral extension problem. The latter is usually solved by searching the spectral density maximizing the entropy rate while matching the moments. The generality of our mathematical formulation can be seen from the employed entropic index as well as the definition of cepstral coefficients. We characterize the solution in the circulant case. Finally, we apply our theory to a 2-d system identification problem.

Multidimensional Rational Covariance Extension with Approximate Covariance Matching

In our companion paper "Multidimensional rational covariance extension with applications to spectral estimation and image compression" we discussed the multidimensional rational covariance extension problem (RCEP), which has important applications in image processing, and spectral estimation in radar, sonar, and medical imaging. This is an inverse problem where a power spectrum with a rational absolutely continuous part is reconstructed from a finite set of moments. However, in most applications these moments are determined from observed data and are therefore only approximate, and RCEP may not have a solution. In this paper we extend the results to handle approximate covariance matching. We consider two problems, one with a soft constraint and the other one with a hard constraint, and show that they are connected via a homeomorphism. We also demonstrate that the problems are well-posed and illustrate the theory by examples in spectral estimation and texture generation.

Multidimensional Rational Covariance Extension with Applications to Spectral Estimation and Image Compression

The rational covariance extension problem (RCEP) is an important problem in systems and control occurring in such diverse fields as control, estimation, system identification, and signal and image processing, leading to many fundamental theoretical questions. In fact, this inverse problem is a key component in many identification and signal processing techniques and plays a fundamental role in prediction, analysis, and modeling of systems and signals. It is well-known that the RCEP can be reformulated as a (truncated) trigonometric moment problem subject to a rationality condition. In this paper we consider the more general multidimensional trigonometric moment problem with a similar rationality constraint. This generalization creates many interesting new mathematical questions and also provides new insights into the original one-dimensional problem. A key concept in this approach is the complete smooth parametrization of all solutions, allowing solutions to be tuned to satisfy additional design specifications without violating the complexity constraints. As an illustration of the potential of this approach we apply our results to multidimensional spectral estimation and image compression. This is just a first step in this direction, and we expect that more elaborate tuning strategies will enhance our procedures in the future.

Moment-based Dirac Mixture Approximation of Circular Densities

Given a circular probability density function, called the true probability density function, the goal is to find a Dirac mixture approximation based on some circular moments of the true density. When keeping the locations of the Dirac points fixed, but almost arbitrarily located, we are applying recent results on the circulant rational covariance extension problem to the problem of calculating the weights. For the case of simultaneously calculating optimal locations, additional constraints have to be deduced from the given density. For that purpose, a distance measure for the deviation of the Dirac mixture approximation from the true density is derived, which then is minimized while considering the moment conditions as constraints. The method is based on progressive numerical minimization, converges quickly and gives well- distributed Dirac mixtures that fulfill the constraints, i.e., have the desired circular moments.

The Circulant Rational Covariance Extension Problem: The Complete Solution

The rational covariance extension problem to determine a rational spectral density given a finite number of covariance lags can be seen as a matrix completion problem to construct an infinite-dimensional positive-definite Toeplitz matrix the northwest corner of which is given. The circulant rational covariance extension problem considered in this paper is a modification of this problem to partial stochastic realization of periodic stationary process, which are better represented on the discrete unit circle rather than on the discrete real line . The corresponding matrix completion problem then amounts to completing a finite-dimensional Toeplitz matrix that is circulant. Another important motivation for this problem is that it provides a natural approximation, involving only computations based on the fast Fourier transform, for the ordinary rational covariance extension problem, potentially leading to an efficient numerical procedure for the latter. The circulant rational covariance extension problem is an inverse problem with infinitely many solutions in general, each corresponding to a bilateral ARMA representation of the underlying periodic process. In this paper, we present a complete smooth parameterization of all solutions and convex optimization procedures for determining them. A procedure to determine which solution that best matches additional data in the form of logarithmic moments is also presented.

Rational Covariance Extension for Boundary Data and Positive Real Lemma with Positive Semidefinite Matrix Solution

AbstractWe consider the rational covariance extension problem with boundary data in terms of the positive real lemma and block discrete-time Schwarz form. The solution of the positive real function has zeros, poles and spectral zeros on the stability boundary. We use the positive real lemma for the positivity characterization, where the matrix P of independent variable is positive semidefinite.

On the Nevanlinna-Pick interpolation problem: Analysis of the McMillan degree of the solutions

In this paper we investigate some aspect of the Nevanlinna-Pick and Schur interpolation problem formulated for Schur-functions considered on the right-half plane of C . We consider the well established parametrization of the solution Q = T Θ ( S ) ≔ ( S Θ 12 + Θ 22 ) - 1 ( S Θ 11 + Θ 21 ) (see e.g. [J.A. Ball, I. Gohberg, L. Rodman, Realization and interpolation of rational matrix functions, Operator Theory: Adv. Appl. 33 (1988) 1–72; H. Dym, J-Contractive matrix functions, reproducing kernel Hilbert spaces and interpolation, CBMS Regional Conference Series in Mathematics, No. 71, Amer. Math. Soc., Providence, RI, 1989]), where the J-inner function Θ is completely determined by the interpolation data and S is an arbitrary Schur function. We then compare the relations between the realizations of Q and S induced by Θ. We show in particular that S generates a solution with a low McMillan degree if and only if S satisfies some interpolation conditions formulated on the left-half plane of C . This analysis can be considered to be partially complementary to the results of A. Lindquist, C. Byrnes et al. on Carathéodory functions, [C. Byrnes, A. Lindquist, On the partial stochastic realization problem, IEEE Trans. Automatic Control 42 (1997) 1049–1070; C. Byrnes, A. Lindquist, S.V. Gusev, A.S. Mateev, A complete parametrization of all positive rational extension of a covariance sequence, IEEE Trans. Automatic Control 40 (1995) 1841–1857; C. Byrnes, A. Lindquist, S.V. Gusev, A convex optimization approach to the rational covariance extension problem, TRIA/MAT, 1997].

On the solutions of the rational covariance extension problem corresponding to pseudopolynomials having boundary zeros

In this note, we study the rational covariance extension problem with degree bound when the chosen pseudopolynomial of degree at most n has zeros on the boundary of the unit circle and derive some new theoretical results for this special case. In particular, a necessary and sufficient condition for a solution to be bounded (i.e., has no poles on the unit circle) is established. Our approach is based on convex optimization, similar in spirit to the recent development of a theory of generalized interpolation with a complexity constraint. However, the two treatments do not proceed in the same way and there are important differences between them which we discuss herein. An implication of our results is that bounded solutions can be computed via methods that have been developed for pseudopolynomials which are free of zeros on the boundary, extending the utility of those methods. Numerical examples are provided for illustration.

New results on the rational covariance extension problem with degree constraint

A new theory for the rational covariance extension problem (with degree constraint), or simply the RCEP, has recently emerged with applications in high-resolution spectral estimation, speech synthesis and possibly new applications in time series analysis and system identification. This paper establishes some new theoretical results on the RCEP via an alternative analysis. In one result we show the bijective correspondence between denominator polynomials of non-strictly-positive solutions of the RCEP and the minimizers of a class of (strictly) convex functionals, associated with non-strictly-positive pseudopolynomials, defined on a subset of a finite dimensional space. This result leads to an alternative constructive proof of a theorem of Georgiou on complete parametrization of all solutions of the RCEP and a new geometric proof, with an extension to non-real interpolators, of a homeomorphism derived previously by Blomqvist, Fanizza and Nagamune. We then generalize this homeomorphism to also allow for variation in the covariance data. Our contribution in the generalization is allowing for non-strictly-positive pseudopolynomials. For the special case of strictly positive pseudopolynomials, a stronger property of diffeomorphism (in the context of the Nevanlinna–Pick interpolation with degree constraint) has been shown earlier by Byrnes and Lindquist. The results of this paper have direct analogues to Nevanlinna–Pick interpolation with degree constraint which is of interest to the robust control community.

On spectral analysis using models with pre-specified zeros

The fundamental theory of Lindquist and co-workers on the rational covariance extension problem provides a very elegant framework for ARMA spectral estimation. Here the choice of zeros is completely arbitrary, and can be used to tune the estimator. An alternative approach to ARMA model estimation with pre-specified zeros is to use a prediction error method based on generalizing autoregressive (AR) modeling using orthogonal rational filters . Here the motivation is to reduce the number of parameters needed to obtain useful approximate models of stochastic processes by suitable choice of zeros, without increasing the computational complexity.The objective of this contribution is to discuss similarities and differences between these two approaches to spectral estimation.

SIGEST

Previous article Next article Full AccessSIGESThttps://doi.org/10.1137/SIREAD000043000004000643000001BibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstractSince its inception in March 1999, the SIGEST section of SIAM Review has highlighted excellent papers of broad interest from SIAM's specialized journals, making SIAM readers aware of outstanding work whose content and roots span multiple areas. This issue's SIGEST selection takes that (short) tradition one step further: Christopher I. Byrnes, Sergei V. Gusev, and Anders Lindquist present "From Finite Covariance Windows to Modeling Filters: A Convex Optimization Approach," an expanded, generalized version of their paper "A Convex Optimization Approach to the Rational Covariance Extension Problem," which originally appeared in SIAM Journal on Control and Optimization, volume 37 (1998). The SIGEST paper begins with a careful survey of the rational covariance extension problem, tracing its historical developments from the early twentieth-century work of Carathéodory, Toeplitz, and Schur to the present. Connections and applications abound: potential theory, stochastic realization, spectral estimation, trigonometric moments, circuit and systems theory, signal and speech processing, robust control, and mobile communication. The authors have gone out of their way to relate mathematical developments to practical consequences---and to bring theory to life through well-chosen examples. In one instance among many, the authors clearly explain why the easily computable maximum entropy solution cannot capture valleys in the speech spectrum and hence produces "flat" synthesized speech. The scientific contributions of the paper are substantial, including new theoretical results as well as an efficient Newton-based interior method. Of particular interest is the formulation of appropriate spaces leading to an optimization problem that solves the rational covariance extension problem with degree constraints. Appearance of a barrier-like integral analogous to barrier terms in interior methods leads to existence of interior minimizers, a well-posed optimization problem, and a systems-theoretic consequence. We are grateful indeed to the authors for their efforts in giving us an impressive SIGEST paper of wide scope and fascinating content. Previous article Next article FiguresRelatedReferencesCited ByDetails Volume 43, Issue 4| 2001SIAM Review History Published online:04 August 2006 Article & Publication DataArticle DOI:10.1137/SIREAD000043000004000643000001Article page range:pp. 643-643ISSN (print):0036-1445ISSN (online):1095-7200Publisher:Society for Industrial and Applied Mathematics

From Finite Covariance Windows to Modeling Filters: A Convex Optimization Approach

The trigonometric moment problem is a classical moment problem with numerous applications in mathematics, physics, and engineering. The rational covariance extension problem is a constrained version of this problem, with the constraints arising from the physical realizability of the corresponding solutions. Although the maximum entropy method gives one well-known solution, in several applications a wider class of solutions is desired. In a seminal paper, Georgiou derived an existence result for a broad class of models. In this paper, we review the history of this problem, going back to Carath{éodory, as well as applications to stochastic systems and signal processing. In particular, we present a convex optimization problem for solving the rational covariance extension problem with degree constraint. Given a partial covariance sequence and the desired zeros of the shaping filter, the poles are uniquely determined from the unique minimum of the corresponding optimization problem. In this way we obtain an algorithm for solving the covariance extension problem, as well as a constructive proof of Georgiou's existence result and his conjecture, a generalized version of which we have recently resolved using geometric methods. We also survey recent related results on constrained Nevanlinna--Pick interpolation in the context of a variational formulation of the general moment problem.

A Convex Optimization Approach to the Rational Covariance Extension Problem

In this paper we present a convex optimization problem for solving the rational covariance extension problem. Given a partial covariance sequence and the desired zeros of the modeling filter, the poles are uniquely determined from the unique minimum of the corresponding optimization problem. In this way we obtain an algorithm for solving the covariance extension problem, as well as a constructive proof of Georgiou's seminal existence result and his conjecture, a stronger version of which we have resolved in [Byrnes et al., IEEE Trans. Automat. Control, AC-40 (1995), pp. 1841--1857].

On the partial stochastic realization problem

We describe a complete parameterization of the solutions to the partial stochastic realization problem in terms of a nonstandard matrix Riccati equation. Our analysis of this covariance extension equation (CEE) is based on a complete parameterization of all strictly positive real solutions to the rational covariance extension problem, answering a conjecture due to Georgiou (1987) in the affirmative. We also compute the dimension of partial stochastic realizations in terms of the rank of the unique positive semidefinite solution to the CEE, yielding some insights into the structure of solutions to the minimal partial stochastic realization problem. By combining this parameterization with some of the classical approaches in partial realization theory, we are able to derive new existence and robustness results concerning the degrees of minimal stochastic partial realizations. As a corollary to these results, we note that, in sharp contrast with the deterministic case, there is no generic value of the degree of a minimal stochastic realization of partial covariance sequences of fixed length.

Canonical correlation analysis, approximate covariance extension, and identification of stationary time series

In this paper we analyze a class of state space identification algorithms for time-series, based on canonical correlation analysis, in the light of recent results on stochastic systems theory. In principle, these so called “subspace methods” can be described as covariance estimation followed by stochastic realization. The methods offer the major advantage of converting the nonlinear parameter estimation phase in traditional ARMA models identification into the solution of a Riccati equation but introduce at the same time some nontrivial mathematical problems related to positivity. The reason for this is that an essential part of the problem is equivalent to the well-known rational covariance extension problem. Therefore, the usual deterministic arguments based on factorization of a Hankel matrix are not valid for generic data, something that is habitually overlooked in the literature. We demonstrate that there is no guarantee that several popular identification procedures based on the same principle will not fail to produce a positive extension, unless some rather stringent assumptions are made which, in general, are not explicitly reported. In this paper the statistical problem of stochastic modeling from estimated covariances is phrased in the geometric language of stochastic realization theory. We review the basic ideas of stochastic realization theory in the context of identification, discuss the concept of stochastic balancing and of stochastic model reduction by principal subsystem truncation. The model reduction method of Desai and Pal (1982) [A realization approach to stochastic model reduction. Proc. 1 st Decision and Control Conf., pp. 1105–1112.], based on truncated balanced stochastic realizations, is partially justified, showing that the reduced system structure has a positive covariance sequence but is in general not balanced. As a byproduct of this analysis we obtain a theorem prescribing conditions under which the ‘subspace identification’ methods produce bona fide stochastic systems.