Infinite-dimensional Linear Programming Research Articles

Peak estimation of rational systems using convex optimization

This paper presents algorithms that upper-bound the peak value of a state function along trajectories of a continuous-time system with rational dynamics. The finite-dimensional but nonconvex peak estimation problem is cast as a convex infinite-dimensional linear program in occupation measures. This infinite-dimensional program is then truncated into finite-dimensions using the moment-Sum-of-Squares (SOS) hierarchy of semidefinite programs. Prior work on treating rational dynamics using the moment-SOS approach involves clearing dynamics to common denominators or adding lifting variables to handle reciprocal terms under new equality constraints. Our solution method uses a sum-of-rational method based on absolute continuity of measures. The Moment-SOS truncations of our program possess lower computational complexity and (empirically demonstrated) higher accuracy of upper bounds on example systems as compared to prior approaches.

Stokes, Gibbs, and Volume Computation of Semi-Algebraic Sets

We consider the problem of computing the Lebesgue volume of compact basic semi-algebraic sets. In full generality, it can be approximated as closely as desired by a converging hierarchy of upper bounds obtained by applying the Moment-SOS (sums of squares) methodology to a certain infinite-dimensional linear program (LP). At each step one solves a semidefinite relaxation of the LP which involves pseudo-moments up to a certain degree. Its dual computes a polynomial of same degree which approximates from above the discontinuous indicator function of the set, hence with a typical Gibbs phenomenon which results in a slow convergence of the associated numerical scheme. Drastic improvements have been observed by introducing in the initial LP additional linear moment constraints obtained from a certain application of Stokes’ theorem for integration on the set. However and so far there was no rationale to explain this behavior. We provide a refined version of this extended LP formulation. When the set is the smooth super-level set of a single polynomial, we show that the dual of this refined LP has an optimal solution which is a continuous function. Therefore in this dual one now approximates a continuous function by a polynomial, hence with no Gibbs phenomenon, which explains and improves the already observed drastic acceleration of the convergence of the hierarchy. Interestingly, the technique of proof involves recent results on Poisson’s partial differential equation (PDE).

LP based upper and lower bounds for Cesàro and Abel limits of the optimal values in problems of control of stochastic discrete time systems

In this paper, we study asymptotic properties of problems of control of stochastic discrete time systems (also known as Markov decision processes) with time averaging and time discounting optimality criteria, and we establish that the Cesàro and Abel limits of the optimal values in such problems can be evaluated with the help of a certain infinite-dimensional linear programming problem and its dual.

Converging Approximations of Attractors via Almost Lyapunov Functions and Semidefinite Programming

In this paper we combine two existing approaches for approximating global attractors. One of them approximates the global attractors arbitrarily well by sublevel sets related to solutions of infinite dimensional linear programming problems. A downside there is that these sets are not necessarily positively invariant. On the contrary, the second method provides supersets of the global attractor which are positively invariant. Their method on the other hand has the disadvantage that the underlying optimization problem is not computationally tractable without the use of heuristics – and incorporating them comes at the price of losing guaranteed convergence. In this paper we marry both approaches by combining their techniques and we get converging outer approximations of the global attractor consisting of positively invariant sets based on convex optimization via sum-of-squares techniques. The method is easy to use and illustrated by numerical examples.

On Linear Programming for Constrained and Unconstrained Average-Cost Markov Decision Processes with Countable Action Spaces and Strictly Unbounded Costs

We consider the linear programming approach for constrained and unconstrained Markov decision processes (MDPs) under the long-run average-cost criterion, where the class of MDPs in our study have Borel state spaces and discrete countable action spaces. Under a strict unboundedness condition on the one-stage costs and a recently introduced majorization condition on the state transition stochastic kernel, we study infinite-dimensional linear programs for the average-cost MDPs and prove the absence of a duality gap and other optimality results. Our results do not require a lower-semicontinuous MDP model. Thus, they can be applied to countable action space MDPs where the dynamics and one-stage costs are discontinuous in the state variable. Our proofs make use of the continuity property of Borel measurable functions asserted by Lusin’s theorem.

Converging outer approximations to global attractors using semidefinite programming

This paper develops a method for obtaining guaranteed outer approximations for global attractors of continuous and discrete time nonlinear dynamical systems. The method is based on a hierarchy of semidefinite programming problems of increasing size with guaranteed convergence to the global attractor. The approach taken follows an established line of reasoning, where we first characterize the global attractor via an infinite dimensional linear programming problem (LP) in the space of Borel measures. The dual to this LP is in the space of continuous functions and its feasible solutions provide guaranteed outer approximations to the global attractor. For systems with polynomial dynamics, a hierarchy of finite-dimensional sum-of-squares tightenings of the dual LP provides a sequence of outer approximations to the global attractor with guaranteed convergence in the sense of volume discrepancy tending to zero. The method is very simple to use and based purely on convex optimization. Numerical examples with the code available online demonstrate the method.

Linear programming estimates for Cesàro and Abel limits of optimal values in optimal control problems

<p style='text-indent:20px;'>We consider infinite horizon optimal control problems with time averaging and time discounting criteria and give estimates for the Cesàro and Abel limits of their optimal values in the case when they depend on the initial conditions. We establish that these limits are bounded from above by the optimal value of a certain infinite dimensional (ID) linear programming (LP) problem and that they are bounded from below by the optimal value of the corresponding dual problem. (These estimates imply, in particular, that the Cesàro and Abel limits exist and are equal to each other if there is no duality gap). In addition, we obtain IDLP-based optimality conditions for the long run average optimal control problem, and we illustrate these conditions by an example.</p>

Exploiting Sparsity for Semi-Algebraic Set Volume Computation

We provide a systematic deterministic numerical scheme to approximate the volume (i.e., the Lebesgue measure) of a basic semi-algebraic set whose description follows a correlative sparsity pattern. As in previous works (without sparsity), the underlying strategy is to consider an infinite-dimensional linear program on measures whose optimal value is the volume of the set. This is a particular instance of a generalized moment problem which in turn can be approximated as closely as desired by solving a hierarchy of semidefinite relaxations of increasing size. The novelty with respect to previous work is that by exploiting the sparsity pattern we can provide a sparse formulation for which the associated semidefinite relaxations are of much smaller size. In addition, we can decompose the sparse relaxations into completely decoupled subproblems of smaller size, and in some cases computations can be done in parallel. To the best of our knowledge, it is the first contribution that exploits correlative sparsity for volume computation of semi-algebraic sets which are possibly high-dimensional and/or non-convex and/or non-connected.

Convex Computation of Extremal Invariant Measures of Nonlinear Dynamical Systems and Markov Processes

We propose a convex-optimization-based framework for computation of invariant measures of polynomial dynamical systems and Markov processes, in discrete and continuous time. The set of all invariant measures is characterized as the feasible set of an infinite-dimensional linear program (LP). The objective functional of this LP is then used to single out a specific measure (or a class of measures) extremal with respect to the selected functional such as physical measures, ergodic measures, atomic measures (corresponding to, e.g., periodic orbits) or measures absolutely continuous w.r.t. to a given measure. The infinite-dimensional LP is then approximated using a standard hierarchy of finite-dimensional semidefinite programming problems, the solutions of which are truncated moment sequences, which are then used to reconstruct the measure. In particular, we show how to approximate the support of the measure as well as how to construct a sequence of weakly converging absolutely continuous approximations. As a by-product, we present a simple method to certify the nonexistence of an invariant measure, which is an important question in the theory of Markov processes. The presented framework, where a convex functional is minimized or maximized among all invariant measures, can be seen as a generalization of and a computational method to carry out the so-called ergodic optimization, where linear functionals are optimized over the set of invariant measures. Finally, we also describe how the presented framework can be adapted to compute eigenmeasures of the Perron–Frobenius operator.

Hybrid [formula omitted]-performance analysis and control of linear time-varying impulsive and switched positive systems

Recent works have shown that the L1 and L∞-gains are natural performance criteria for linear positive systems as they can be exactly characterized by linear programs. Those performance measures have also been extended to linear positive impulsive and switched systems through the concept of hybrid L1×ℓ1-gain. For LTI positive systems, the L∞-gain is known to coincide with the L1-gain of the transposed system and, as a consequence, one can use linear copositive Lyapunov functions for characterizing the L∞-gain of LTI positive systems. Unfortunately, this does not hold in the time-varying setting and one cannot characterize the hybrid L∞×ℓ∞-gain of a linear positive impulsive system in terms of the hybrid L1×ℓ1-gain of the transposed system. To solve this, an approach based on the use of linear copositive max-separable Lyapunov functions is proposed. We first prove very general necessary and sufficient conditions characterizing the exponential stability and the L∞×ℓ∞- and L1×ℓ1-gains using linear max-separable copositive and linear sum-separable copositive Lyapunov functions. These two results are then connected together using operator theoretic results and the notion of adjoint system. Results characterizing the stability and the hybrid L∞×ℓ∞-gain of linear positive impulsive systems under arbitrary, constant, minimum, and range dwell-time constraints are then derived from the previously obtained general results. These conditions are then exploited to yield constructive convex stabilization conditions via state-feedback. By reformulating linear positive switched systems as impulsive systems with multiple jump maps, stability and stabilization conditions are also obtained for linear positive switched systems. It is notably proven that the obtained conditions generalize existing ones of the literature. As all the results are stated as infinite-dimensional linear programs, sum of squares programming is used to turn those optimization problems into sufficient tractable finite-dimensional semidefinite programs. Interestingly, the relaxation becomes necessary if we allow the degrees of the polynomials to be arbitrarily large. Several particular cases of the approach such as LTV positive systems and periodic positive systems are also discussed for completeness. Examples are given for illustration.

Nonlinear optimal control: a numerical scheme based on occupation measures and interval analysis

This paper presents an approximation scheme for optimal control problems using finite-dimensional linear programs and interval analysis. This is done in two parts. Following Vinter approach (SIAM J Control Optim 31(2):518–538, 1993) and using occupation measures, the optimal control problem is written into a linear programming problem of infinite-dimension (weak formulation). Thanks to Interval arithmetic, we provide a relaxation of this infinite-dimensional linear programming problem by a finite dimensional linear programming problem. A proof that the optimal value of the finite dimensional linear programming problem is a lower bound to the optimal value of the control problem is given. Moreover, according to the fineness of the discretization and the size of the chosen test function family, obtained optimal values of each finite dimensional linear programming problem form a sequence of lower bounds which converges to the optimal value of the initial optimal control problem. Examples will illustrate the principle of the methodology.

Computing Controlled Invariant Sets from Data Using Convex Optimization

This work presents a data-driven method for approximation of the maximum positively invariant (MPI) set and the maximum controlled invariant (MCI) set for nonlinear dynamical systems. The method only requires the knowledge of a finite collection of one-step transitions of the discrete-time dynamics, without the requirement of segments of trajectories or the control inputs that effected the transitions to be available. The approach uses a novel characterization of the MPI and MCI sets as the solution to an infinite-dimensional linear programming (LP) problem in the space of continuous functions, with the optimum being attained by a (Lipschitz) continuous function under mild assumptions. The infinite-dimensional LP is then approximated by restricting the decision variable to a finite-dimensional subspace and by imposing the non-negativity constraint of this LP only on the available data samples. This leads to a single finite-dimensional LP that can be easily solved using off-the-shelf solvers. We analyze the convergence rate and sample complexity, proving probabilistic as well as hard guarantees on the volume error of the approximations. The approach is very general, requiring minimal underlying assumptions. In particular, the dynamics is not required to be polynomial or even continuous (forgoing some of the theoretical results). Detailed numerical examples up to state-space dimension ten with code available online demonstrate the method.

On uniqueness of solutions to viscous HJB equations with a subquadratic nonlinearity in the gradient

Uniqueness of positive solutions to viscous Hamilton–Jacobi–Bellman (HJB) equations of the form − Δ u ( x ) + 1 γ | Du ( x ) | γ = f ( x ) − λ , with f a coercive function and λ a constant, in the subquadratic case, that is, γ ∈ ( 1 , 2 ) , appears to be an open problem. Barles and Meireles [Comm. Partial Differential Equations 41 (2016)] show uniqueness in the case that f ( x ) ≈ | x | β and | Df ( x ) | ≲ | x | ( β − 1 ) + for some β > 0 , essentially matching earlier results of Ichihara, who considered more general Hamiltonians but with better regularity for f. Without enforcing this assumption, to our knowledge, there are no results on uniqueness in the literature. In this short article, we show that the equation has a unique positive solution for any locally Lipschitz continuous, coercive f which satisfies | Df ( x ) | ≤ κ ( 1 + | f ( x ) | 2 − 1 / γ ) for some positive constant κ. Since 2 − 1 γ > 1 , this assumption imposes very mild restrictions on the growth of the potential f. We also show that this solution fully characterizes optimality for the associated ergodic problem. Our method involves the study of an infinite dimensional linear program for elliptic equations for measures, and is very different from earlier approaches. It also applies to the larger class of Hamiltonians studied by Ichihara, and we show that it is well suited to provide optimality results for the associated ergodic control problem, even in a pathwise sense, and without resorting to the parabolic problem.

Formula omitted]-to-[formula omitted] analysis of linear positive impulsive systems with application to the [formula omitted]-to-[formula omitted] interval observation of linear impulsive and switched systems

Sufficient conditions characterizing the asymptotic stability and the hybrid L1×ℓ1-gain of linear positive impulsive systems under minimum and range dwell-time constraints are obtained. These conditions are stated as infinite-dimensional linear programming problems that can be solved using sum of squares programming, a relaxation that is known to be asymptotically exact in the present case. These conditions are then adapted to formulate constructive and convex sufficient conditions for the existence of L1×ℓ1-to-L1×ℓ1 interval observers for linear impulsive and switched systems. Suitable observer gains can be extracted from the (suboptimal) solution of the infinite-dimensional optimization problem where the L1×ℓ1-gain of the system mapping the disturbances to the weighted observation errors is minimized. Some examples on impulsive and switched systems are given for illustration.

Necessary and Sufficient Conditions for Qualitative Properties of Infinite Dimensional Linear Programming Problems

Necessary and sufficient conditions for qualitative properties of infinite dimensional linear programing problems such as solvability, duality, and complementary slackness conditions are studied in this article. As illustrations for the results, we investigate the parametric version of Gale’s example.

New alternative convex conditions on exponential stability and stabilisation of switched positive linear systems with dwell time

This study is concerned with dwell time stability and stabilisation problems of switched positive linear systems (SPLSs). The dwell time refers to minimum dwell time and constant dwell time. Several stability conditions for primal and transpose SPLSs with dwell time are presented, and the relation between these conditions is illustrated. Some of these conditions are given in terms of infinite-dimensional linear programming (LP), which cannot be solved directly. Then, by utilising the piecewise linear approach, new alternative convex conditions are formulated in terms of finite-dimensional LP. Compared to the existing literature, results with lower or at least the same conservatism can be obtained under the new conditions for the same discretised order. An algorithm is given to reduce the computational cost. Meanwhile, it is proved that there exists a relation between these convex and non-convex conditions if the discretised order is sufficiently large. By utilising the transpose conditions, alternative convex conditions on stabilisation of SPLSs with dwell time are also presented. The controller gain matrices can be computed by solving a set of LP directly. Finally, the correctness and superiority of the results are verified by numerical examples.

A land of monotone plenty

A fundamental concept in optimal transport is c-cyclical monotonicity: it allows to link the optimality of transport plans to the geometry of their support sets. Recently, related concepts have been successfully applied in the multi-marginal version of the transport problem as well as in the martingale transport problem which arises from model-independent finance. We establish a unifying concept of c-monotonicity / finitistic optimality which describes the geometric structure of optimizers to infinite-dimensional linear programming problems. This allows us to strengthen known results in martingale optimal transport and the infinitely marginal case of the optimal transport problem. If the optimization problem can be formulated as a multi-marginal transport problem our contribution is parallel to a recent result of Zaev.

LP Formulations of Discrete Time Long-Run Average Optimal Control Problems: The NonErgodic Case

We formulate and study the infinite dimensional linear programming (LP) problem associated with the deterministic discrete time long-run average criterion optimal control problem. Along with its du...

Optimal Transport Over Deterministic Discrete-Time Nonlinear Systems Using Stochastic Feedback Laws

This letter considers the relaxed version of the transport problem for general nonlinear control systems, where the objective is to design time-varying feedback laws that transport a given initial probability measure to a target probability measure under the action of the closed-loop system. To make the problem analytically tractable, we consider control laws that are stochastic, i.e., the control laws are maps from the state space of the control system to the space of probability measures on the set of admissible control inputs. Under some controllability assumptions on the control system as defined on the state space, we show that the transport problem, considered as a controllability problem for the lifted control system on the space of probability measures, is well-posed for a large class of initial and target measures. We use this to prove the well-posedness of a fixed-endpoint optimal control problem defined on the space of probability measures, where along with the terminal constraints, the goal is to optimize an objective functional along the trajectory of the control system. This optimization problem can be posed as an infinite-dimensional linear programming problem. This formulation facilitates numerical solutions of the transport problem for low-dimensional control systems, as we show in two numerical examples.

Linear Programming Formulation of Long-Run Average Optimal Control Problem

We formulate and study the infinite-dimensional linear programming problem associated with the deterministic long-run average cost control problem. Along with its dual, it allows one to characterize the optimal value of this control problem. The novelty of our approach is that we focus on the general case wherein the optimal value may depend on the initial condition of the system.