Sequence Of Optimal Control Problems Research Articles

Maximum Principle for Stochastic Control of SDEs with Measurable Drifts

In this paper, we consider stochastic optimal control of systems driven by stochastic differential equations with irregular drift coefficient. We establish a necessary and sufficient stochastic maximum principle. To achieve this, we first derive an explicit representation of the first variation process (in the Sobolev sense) of the controlled diffusion. Since the drift coefficient is not smooth, the representation is given in terms of the local time of the state process. Then we construct a sequence of optimal control problems with smooth coefficients by an approximation argument. Finally, we use Ekeland’s variational principle to obtain an approximating adjoint process from which we derive the maximum principle by passing to the limit. The work is notably motivated by the optimal consumption problem of investors paying wealth tax.

Temporal logic guided safe model-based reinforcement learning: A hybrid systems approach

This paper studies the problem of synthesizing control policies for uncertain continuous-time nonlinear systems from linear temporal logic (LTL) specifications using model-based reinforcement learning (MBRL). Rather than taking an abstraction-based approach, we view the interaction between the LTL formula’s corresponding Büchi automaton and the nonlinear system as a hybrid automaton whose discrete dynamics match exactly those of the Büchi automaton. To find satisfying control policies, we pose a sequence of optimal control problems associated with states in the accepting run of the automaton and leverage control barrier functions (CBFs) to prevent specification violation. Since solving many optimal control problems for a nonlinear system is computationally intractable, we take a learning-based approach in which the value function of each problem is learned online in real-time. Specifically, we propose a novel off-policy MBRL algorithm that allows one to simultaneously learn the uncertain dynamics of the system and the value function of each optimal control problem online while adhering to CBF-based safety constraints. Unlike related approaches, the MBRL method presented herein decouples convergence, stability, and safety, allowing each aspect to be studied independently, leading to stronger safety guarantees than those developed in related works. Numerical results are presented to validate the efficacy of the proposed method.

Retracted] Local Null‐Controllability for Some Quasi‐Linear Phase‐Field Systems with Neumann Boundary Conditions by one Control Force

In this article, we study the local null‐controllability for some quasi‐linear phase‐field systems with homogeneous Neumann boundary conditions and an arbitrary located internal controller in the frame of classical solutions. In order to minimize the number of control forces, we prove the Carleman inequality for the associated linear system. By constructing a sequence of optimal control problems and an iteration method based on the parabolic regularity, we find a qualified control in Hölder space for the linear system. Based on the theory of Kakutani’s fixed point theorem, we prove that the quasi‐linear system is local null‐controllable when the initial datum is small and smooth enough.

An Approximation of Solutions for the Problem with Quasistatic Contact in the Case of Dry Friction

In this paper, we discuss the question of finding an optimal control for the solutions of the problem with dry friction quasistatic contact, in the case that the friction law is modeled by a nonlocal version of Coulomb’s law. In order to get the necessary optimality conditions, we use some regularization techniques, and this leads us to a problem of control for an inequality of the variational type. The optimal control problem consists, in our case, of minimizing a sequence of optimal control problems, where the control variable is given by a Neumann-type boundary condition. The state system is represented by a limit of a sequence, whose terms are obtained from the discretization, in time with finite difference and space with the finite element method of a regularized quasistatic contact problem with Coulomb friction. The purpose of this optimal control problem is that the traction force (the control variable) acting on one side of the boundary (the Neumann boundary condition) of the elastic body produces a displacement field (the state system solution) close enough to the imposed displacement field, and the traction force from the boundary remains small enough.

An Augmented Lagrangian for Optimal Control of DAE Systems: Algorithm and Properties

This article proposes a relax-and-discretize approach for optimal control of continuous-time differential algebraic systems. It works by relaxing the algebraic equations and penalizing the violation into the objective function using the augmented Lagrangian, which converts the original problem into a sequence of optimal control problems (OCPs) of ordinary differential equations (ODEs). The relax-and-discretize approach brings about flexibility, by allowing the OCPs of ODEs to be solved by the method of choice, such as direct or indirect methods. Conditions are developed for global, local, and suboptimal convergence in terms of the solution of the underlying OCPs. The method is applied to an illustrative example.

On the Turnpike Property and the Receding-Horizon Method for Linear-Quadratic Optimal Control Problems

Optimal control problems with a very large time horizon can be tackled with the receding-horizon control (RHC) method, which consists in solving a sequence of optimal control problems with small prediction horizon. The main result of this article is the proof of the exponential convergence (with respect to the prediction horizon) of the control generated by the RHC method toward the exact solution of the problem. The result is established for a class of infinite-dimensional linear-quadratic optimal control problems with time-independent dynamics and integral cost. Such problems satisfy the turnpike property: the optimal trajectory remains most of the time very close to the solution to the associated static optimization problem. Specific terminal cost functions, derived from the Lagrange multiplier associated with the static optimization problem, are employed in the implementation of the RHC method.

Fisher information calculation in a complex ecological model: An optimal control-based approach

A new approach to the calculation of Fisher Information in a complex dynamic system is proposed in this work. The numerical strategy involves the successive solution of a series of optimal control problems to achieve the simultaneous estimation of the time cycle, as a fundamental parameter in the calculation of Fisher Information, and a set of Fisher Information average values, as well as the dynamic optimization of the system. A sequence of optimal control problems are solved first for estimating the time cycle, which is required to calculate the Fisher Information of the system. Then, the last optimization problem is solved to obtain not only the optimal profiles for the state and control variables, but also a set of time averaged Fisher Information values; such averaged values can be used to identify changes on the dynamic order of the system. Two illustrative examples are studied: a simple predator-prey Lotka-Volterra model is solved to explain the application of the numerical strategy for the estimation of the time cycle and time averaged Fisher Information values, and a proposed thirteen compartment socio-economic-ecological model, solved after the simple model to show the effectiveness of the approach to detect regime changes in a more complex system. The results of the simple model show a good approximation to the estimation of the time cycle after local minimum and maximum optimal points are obtained, since the system shows a regular cyclic behavior. For the larger model, the results confirm the effectiveness of the proposed optimal control based approach in more complex systems and the benefits of using Fisher Information average values to analyze the behavior of the system, while also identifying the policies that optimize the mass interactions among the compartments.

Optimality conditions for reflecting boundary control problems

We consider a control problem with reflecting boundary and obtain necessary optimality conditions in the form of the maximum Pontryagin principle. To derive these results we transform the constrained problem in an unconstrained one or we use penalization techniques of Morreau-Yosida type to approach the original problem by a sequence of optimal control problems with Lipschitz dynamics. Then nonsmooth analysis theory is used to study the convergence of the penalization in order to obtain optimality conditions.

Model Predictive Control of Impulsive Dynamical Systems

We propose an MPC framework to address impulsive dynamical systems, i.e. systems in which the state trajectories are permitted to have discontinuities (jumps) in response to impulsive controls. We describe how previous results on sampled-data trajectories, invariance of sets, optimal control, and stability in the context of impulsive systems can be used and adapted to a sampled-data MPC framework. We provide sufficient conditions on the design parameters of the optimal control problems that guarantee: (i) feasibility of the sequence of optimal control problems, and (ii) stability of the controlled system.

An optimal control problem involving impulsive integrodifferential systems

In this article, we consider a class of optimal control problems involving dynamical systems described by impulsive integrodifferential equations. First, we approximate the integral kernel of the integral equation by a finite expansion of the shifted Chebyshev polynomial. Through this process, the optimal control problem is approximated by a sequence of optimal control problems involving only impulsive ordinary differential equations. Each of them can be viewed as a nonlinear optimization problem. For each of these approximated problems, the gradient formula of the cost functional can be derived and hence can be solved by many efficient optimization techniques. Consequently, the optimal control software, MISER, is applicable for the purpose. Then, we present some convergence results showing the relationship between the sequence of the optimal controls of the approximated problems and that of the original problem. Finally, a numerical example is presented to illustrate the efficiency of the proposed method.

Inequality path constraints in optimal control: a finite iteration ε-convergent scheme based on pointwise discretization

This paper presents a new result in the analysis and implementation of path constraints in optimal control problems (OCPs). The scheme uses the well-known concept of discretizing path constraints on a finite number of points, yielding a set of interior-time point constraints replacing the original path constraints. The approach replaces the original OCP by a sequence of OCPs which is shown to converge in a finite number of steps to the solution of the original path constrained problem with ε-accuracy. Numerical results, verifying the theoretical analysis, are presented. The method is shown to be effective and promising for future applications, particularly in control vector parameterization implementations.

Approximate controllability for the semilinear heat equation in R N involving gradient terms

We prove the approximate controllability of the semilinear heat equation in R N , when the nonlinear term is globally Lipschitz and depends both on the state u and its spatial gradientu. The approximate controllability is viewed as the limit of a sequence of optimal control problems. In order to avoid the difficulties related to the lack of compactness of the Sobolev embeddings, we work with the similarity variables and use weighted Sobolev spaces.

On $S$-Homogenization of an Optimal Control Problem with Control and State Constraints

We study the limiting behavior of an optimal control problem for a linear elliptic equation subject to control and state constraints. Each constituent of the mathematical description of such an optimal control problem may depend on a small parameter \epsilon . We study the limit of this problem when \epsilon \rightarrow 0 in the framework of variational S -convergence which generalizes the concept of \Gamma -convergence. We also introduce the notion of G* -convergence generalizing the concept of G -convergence to operators with constraints. We show convergence of the sequence of optimal control problems and identify its limit. We then apply the theory to an elliptic problem on a perforated domain.

Lie Algebraic Obstructions to $\Gamma$-Convergence of Optimal Control Problems

We investigate the possibility of describing the of a sequence of optimal control problems $({\cal P})_{\piccolo (b_n)}$, each of which is characterized by the presence of a time dependent vector valued coefficient $b_n=(b_{\piccolo n_1},\ldots,b_{\piccolo n_M})$. The notion of is intended in the sense of $\Gamma$-convergence, which, roughly speaking, prescribes the convergence of both the minimizers and the infimum values. Due to the type of growth involved in each problem $({\cal P})_{\piccolo (b_n)}$ the (weak) limit of the functions $(\bunq,\ldots,\bMnq)$---beside the limit (b1, . . . ,bM) of the $(\bun,\ldots,\bMn)$---is crucial for the description of the limit problem. Of course, since the bn are L2 maps, the limit of the $(b_{\piccolo n_1}^2,\ldots,b_{\piccolo n_M}^2)$ may well be a (vector valued) measure $\mu=(\mu_1,\ldots,\mu_M)$. It happens that when the problems $({\cal P})_{\piccolo (b_n)}$ enjoy a certain commutativity property, then the pair $(b,\mu)$ is sufficient to characterize the limit problem. This is no longer true when the commutativity property is not in force. Indeed, we construct two sequences of problems $({\cal P})_{\piccolo (b_n)}$ and $({\cal P})_{\piccolo (\tilde{b}_n)}$ which are equal except for the coefficient $b_n(\cdot)$ and $\tilde{b}_n(\cdot)$, respectively. Moreover, both the sequences $(b_n,b^2_n)$ and $(\tilde{b}_n,\tilde{b}^2_n)$ converge to the same pair $(b,\mu)$. However, the infimum values of the problems $({\cal P})_{\piccolo (b_n)}$ tend to a value which is different from the limit of the infimum values of the $({\cal P})_{\piccolo (\tilde{b}_n)}$. This means that the mere information contained in the pair $(b,\mu)$ is not sufficient to characterize the limit problem. We overcome this drawback by embedding the problems in a more general setting where limit problems can be characterized by triples of functions $(B_0,B,\gamma)$ with $B_0 \geq 0$.

Finite-Time Optimal Control of Polynomial Systems Using Successive Suboptimal Approximations

It is shown in this paper that the finite-time optimal control of polynomial systems can be obtained by solving a sequence of optimal control problems for the linearized problem. The paper provides proof of convergence as well as illustration of the procedure by two examples.

Approximate Controllability for the Semilinear Heat Equation Involving Gradient Terms

This paper deals with the approximate controllability of the semilinear heat equation, when the nonlinear term depends on both the state y and its spatial gradient ∇y and the control acts on any nonempty open subset of the domain. Our proof relies on the fact that the nonlinearity is globally Lipschitz with respect to (y, ∇y). The approximate controllability is viewed as the limit of a sequence of optimal control problems. Another key ingredient is a unique continuation property proved by Fabre (Ref. 1) in the context of linear heat equations. Finally, we prove that approximate controllability can be obtained simultaneously with exact controllability over finite-dimensional subspaces.

Homogenization of Optimal Control Problems for Functional Differential Equations

The paper deals with the variational convergence of a sequence of optimal control problems for functional differential state equations with deviating argument. Variational limit problems are found under various conditions of convergence of the input data. It is shown that, upon sufficiently weak assumptions on convergence of the argument deviations, the limit problem can assume a form different from that of the whole sequence. In particular, it can be either an optimal control problem for an integro-differential equation or a purely variational problem. Conditions are found under which the limit problem preserves the form of the original sequence.

Algorithms for Unconstrained Optimization Problems via Control Theory

Existing algorithms for solving unconstrained optimization problems are generally only optimal in the short term. It is desirable to have algorithms which are long-term optimal. To achieve this, the problem of computing the minimum point of an unconstrained function is formulated as a sequence of optimal control problems. Some qualitative results are obtained from the optimal control analysis. These qualitative results are then used to construct a theoretical iterative method and a new continuous-time method for computing the minimum point of a nonlinear unconstrained function. New iterative algorithms which approximate the theoretical iterative method and the proposed continuous-time method are then established. For convergence analysis, it is useful to note that the numerical solution of an unconstrained optimization problem is none other than an inverse Lyapunov function problem. Convergence conditions for the proposed continuous-time method and iterative algorithms are established by using the Lyapunov function theorem.

An extended penalty function approach to the numerical solution of constrained optimal control problems

This paper presents the extended penalty function method for solving constrained optimal control problems. Here, equality and inequality constraints on the state and control variables are considered. Using the extended penalty function method, the original constrained optimal control problem is transformed into a sequence of optimal control problems without inequality constraints. This is accomplished by adding to the cost functional a penalty term that takes on large values when the inequality constraints are violated and small values when the constraints are satisfied. Also presented is a continuation method for solving the sequence of differential-algebraic boundary value problems arising from the transformed optimal control problems. The effectiveness of the approach is demonstrated via examples.

Sensitivity analysis of distributed-parameter optimal control problems for nonlinear parabolic equations

A sequence of optimal control problems for systems described by nonlinear parabolic equations is considered. It is proved that, under the Γ-convergence of objective functionals, the parabolicG-convergence of operators in the state equations, and the Kuratowski convergence of control constraint sets, a convergent sequence of optimal pairs has a limit which is an optimal pair for the limit control problem. The convergence of minimal values is also obtained.