Linear Fixed Point Equations Research Articles

Optimal Oracle Inequalities for Projected Fixed-Point Equations, with Applications to Policy Evaluation

Linear fixed-point equations in Hilbert spaces arise in a variety of settings, including reinforcement learning, and computational methods for solving differential and integral equations. We study methods that use a collection of random observations to compute approximate solutions by searching over a known low-dimensional subspace of the Hilbert space. First, we prove an instance-dependent upper bound on the mean-squared error for a linear stochastic approximation scheme that exploits Polyak–Ruppert averaging. This bound consists of two terms: an approximation error term with an instance-dependent approximation factor and a statistical error term that captures the instance-specific complexity of the noise when projected onto the low-dimensional subspace. Using information-theoretic methods, we also establish lower bounds showing that both of these terms cannot be improved, again in an instance-dependent sense. A concrete consequence of our characterization is that the optimal approximation factor in this problem can be much larger than a universal constant. We show how our results precisely characterize the error of a class of temporal difference learning methods for the policy evaluation problem with linear function approximation, establishing their optimality. Funding: This work was partially supported by grants from the Office of Naval Research [Grant DOD-ONRN00014-18-1-2640] and the National Science Foundation (NSF) [NSF-IIS Grant 1909365, NSF-DMS Grant 2015454, and NSF-CCF Grant 1955450] to M. J. Wainwright. Part of this work was performed when A. Pananjady was visiting the Simons Institute for the Theory of Computing, where he was supported by a Swiss Re Research Fellowship. Supplemental Material: The online supplementary file is available at https://doi.org/10.1287/moor.2022.1341 .

The combinatorics of Green’s functions in planar field theories

The aim of this work is to outline in some detail the use of combinatorial algebra in planar quantum field theory. Particular emphasis is given to the relations between the different types of planar Green's functions. The key object is a Hopf algebra which is naturally defined on non-commuting sources, and the fact that its genuine unshuffle coproduct splits into left- and right unshuffle half-coproduts. The latter give rise to the notion of unshuffle bialgebra. This setting allows to describe the relation between planar full and connected Green's functions by solving a simple linear fixed point equation. A modification of this linear fixed point equation gives rise to the relation between planar connected and one-particle irreducible Green's functions. The graphical calculus that arises from this approach also leads to a new understanding of functional calculus in planar QFT, whose rules for differentiation with respect to non-commuting sources can be translated into the language of growth operations on planar rooted trees. We also include a brief outline of our approach in the framework of non-planar theories.

Cumulants, free cumulants and half-shuffles.

Free cumulants were introduced as the proper analogue of classical cumulants in the theory of free probability. There is a mix of similarities and differences, when one considers the two families of cumulants. Whereas the combinatorics of classical cumulants is well expressed in terms of set partitions, that of free cumulants is described and often introduced in terms of non-crossing set partitions. The formal series approach to classical and free cumulants also largely differs. The purpose of this study is to put forward a different approach to these phenomena. Namely, we show that cumulants, whether classical or free, can be understood in terms of the algebra and combinatorics underlying commutative as well as non-commutative (half-)shuffles and (half-) unshuffles. As a corollary, cumulants and free cumulants can be characterized through linear fixed point equations. We study the exponential solutions of these linear fixed point equations, which display well the commutative, respectively non-commutative, character of classical and free cumulants.

Error Bounds for Approximations from Projected Linear Equations

We consider linear fixed point equations and their approximations by projection on a low dimensional subspace. We derive new bounds on the approximation error of the solution, which are expressed in terms of low dimensional matrices and can be computed by simulation. When the fixed point mapping is a contraction, as is typically the case in Markov decision processes (MDP), one of our bounds is always sharper than the standard contraction-based bounds, and another one is often sharper. The former bound is also tight in a worst-case sense. Our bounds also apply to the noncontraction case, including policy evaluation in MDP with nonstandard projections that enhance exploration. There are no error bounds currently available for this case to our knowledge.

Mean value iterations for nonlinear elliptic Cauchy problems

We investigate the Cauchy problem for a class of nonlinear elliptic operators with $C^\infty$-coefficients at a regular set $\Omega \subset R^n$. The Cauchy data are given at a manifold $\Gamma \subset \partial\Omega$ and our goal is to reconstruct the trace of the $H^1(\Omega)$ solution of a nonlinear elliptic equation at $\partial \Omega / \Gamma$. We propose two iterative methods based on the segmenting Mann iteration applied to fixed point equations, which are closely related to the original problem. The first approach consists in obtaining a corresponding linear Cauchy problem and analyzing a linear fixed point equation; a convergence proof is given and convergence rates are obtained. On the second approach a nonlinear fixed point equation is considered and a fully nonlinear iterative method is investigated; some preliminary convergence results are proven and a numerical analysis is provided.

Acceleration of iteration methods for interval fixed point problems

We consider the fixed point equation x = F ( x ) with a continuous and inclusion isotone interval function F : R n → R n . The iteration x k+1= F ( x k) converges monotonically ( x k+1⊆ x k ) to a fixed point of F if x 1⊆ x 0 . We prove a theorem on an accelerated monotone iteration and apply it to systems of linear and nonlinear equations. For linear fixed point equations ( x = Ax + b ), we also present a modified single step method.

Error Estimates for the Approximate Solution of Linear Fixed Point Equations

AbstractThe defects which are caused by the solutions of approximate equations in the original equation provide an effective tool for the asymptotically exact estimation of approximation errors. Using these defects may be justified without uniformity restrictions on the underlying mesh sequences. As an example, asymptotically exact error estimates are investigated for the numerical solution of a one‐dimensional linear diffusion‐convection equation by means of finite elements.

Ein Einschliessungsverfahren für Lösungen Fehlerbehafteter Linearer Gleichungen

In a partially ordered space, the method xn+1 = L+xn+ − N+xn- − L−y+ + N− yn- + r, yn+1 = N+y+ − L+yn- − N−xn+ + L−x− + t of successive approximation is developed in order to enclose the solutions of a set of linear fixed point equations monotonously. The method works if only the inequalities x0 ≤ y0, x0 ≤ x1, y1 ≤ y0 related to the starting elements are satisfied. In finite-dimensional spaces suitable starting vectors can be computed if a sufficiently good approximation for the fixed points is known.

Exact Renormalization in the Statistical Mechanics of Fluids

Wegner's paradigm for constructing renormalization groups in statistical mechanics is applied to fluids. The coordinates of the fraction ($1\ensuremath{-}y$) of the molecules are integrated over and a system of interactions describing the remaining fraction $y$ is constructed with the aid of the McMillan-Mayer solution theory. Renormalized Ursell functions are defined in terms of which the group operations are shown to be linear. The group operations commute with a scaling operation and it is found that a series of renormalization groups depending on a parameter $x$ can be constructed by combining the operation of integrating over coordinates with the scaling operation. By comparing the renormalized cluster functions with the form of the critical cluster functions given by scaling it is shown that a fixed point of one of the series of groups corresponds to the critical point. The parameter $x$ turns out to be the scaling exponent for the chemical potential. The question of the universality of critical phenomena is expressed as the question of the discreteness or continuity of the spectrum of values of the parameter $x$ for which the linear fixed-point equations have a nontrivial solution. It is suggested that this question can be answered definitively only when a precise formulation is made of the function space in which the sequence of renormalized Ursell functions must lie.