Optimal Control Problem Research Articles

Loss control regions in optimal control problems

This paper addresses optimal control problems with loss control regions. In that context the state space is partitioned into disjoint subsets, referred to as regions, which are classified into two types: control regions and loss control regions. When the state belongs to a control region, the control is permanent (i.e. the control value is authorized to be modified at any time). On the contrary, when the state belongs to a loss control region, the control must remain constant as long as the state belongs to this region. The objective of this paper is twofold. First, we reformulate the above setting into a hybrid optimal control problem (with spatially heterogeneous dynamics) involving moreover a regionally switching parameter, and we prove a corresponding hybrid maximum principle: hence first-order necessary optimality conditions in a Pontryagin form are obtained. Second, this paper proposes a two-steps numerical scheme to solve optimal control problems with loss control regions. It is based on a direct numerical method (applied to a regularized problem) which initializes an indirect numerical method (applied to the original problem and based on the aforementioned necessary optimality conditions). This numerical approach is applied to several illustrative examples.

Stochastic maximum principle for partially observed optimal control problem of McKean–Vlasov FBSDEs with Teugels martingales

Abstract In this paper, we study the stochastic maximum principle for a partially observed optimal control problem of forward-backward stochastic differential equations (FBSDEs for short) of McKean–Vlasov type driven by both a family of Teugels martingales and an independent Brownian motion. The coefficients of the system and the cost functional depending on the state of the solution process as well as its probability law and the control variable. We establish partially observed necessary conditions of optimality for this system under assumption that the control domain is supposed to be convex. Our main result is based on Girsavov’s theorem and the derivatives with respect to probability law. As an application of the general theory, a partially observed linear-quadratic control problem of McKean–Vlasov type is studied in terms of stochastic filtering.

Singular backward stochastic Volterra integral equations in infinite dimensional spaces

In this paper, the notion of singular backward stochastic Volterra integral equations (singular BSVIEs for short) in infinite dimensional space is introduced, and the corresponding well-posedness is carefully established. A class of singularity conditions are proposed, which not only cover that of fractional kernel, Volterra Heston model kernel, completely monotone kernels, to mention a few, but also happen to be used in the forward stochastic Volterra integral with new conclusions arising. Motivated by mathematical physics problem such as the viscoelasticity/thermoviscoelasticity of materials, heat conduction in materials with memory, optimal control problems of abstract stochastic Volterra integral equations (including fractional stochastic evolution equations and stochastic evolutionary integral equations) are presented. At last, our BSVIEs are surprisingly used in maximum principle of controlled stochastic delay evolution equations. One advantage of this new standpoint is that the final cost functional can naturally depend on the past state.

Modified wavelets technique for multitype 2D fractional optimal control problems

Modified wavelets technique for multitype 2D fractional optimal control problems

Optimal control of strength allocation strategies generation with complex constraints

This study addresses the satellite swarm strength allocation strategies generation problem with complex constraints. The satellite swarm confrontation attrition process is described by an improved Lanchester equation and multi-phase confrontation patterns. On the basis of them, a satellite swarm strength allocation strategies generation optimal control model is first presented. Unlike the general optimization model, this model considers three constraints such as sunlight angle, supplement number, and confrontation time to make it suitable for describing the space confrontation process. Then, the multi-interval orthogonal collocation (MIOC) method is proposed to generate the strength allocation strategies from the constructed model. This method not only solves optimal control problems but also solves discontinuities caused by multi-phase confrontation patterns. Finally, a commencement solution exploration approach is proposed to explore the appropriate initial value for the MIOC method. The aim of it is to avoid initial value-sensitive problems and to make convergence fast. The effectiveness of the MIOC method is verified through three numerical examples in comparison with the static optimization methods.

How do time delays influence dynamics and controls of a generalized SEAIR model?

Given the critical role of time delays in epidemic modeling, this paper delves into the dynamics and finite-time optimal stabilization of a novel epidemic system characterized by such delays. Our findings reveal that time delays significantly influence both the system’s dynamics and the formulation of an optimal control strategy. Specifically, the system’s endemic equilibrium point remains locally asymptotically stable under mild conditions for small time delays. However, exceeding critical delay thresholds induces Hopf bifurcation. To closely regulate the system’s state towards the equilibrium point and minimize control costs, we propose an optimal control problem. We derive the explicit form of the optimal control strategy employing Pontryagin’s Maximum Principle. Finally, numerical simulations further confirm the theoretical results obtained in this paper.

Approximation of Chattering Arcs in Optimal Control Problems Governed by Mono-Input Affine Control Systems

Approximation of Chattering Arcs in Optimal Control Problems Governed by Mono-Input Affine Control Systems

Optimal control, well-posedness and sensitivity analysis for a class of generalized evolutionary systems

In this paper, we are concerned with a generalized evolution dynamical system, called fractional differential variational-hemivariational inequality (FDVHVI, for short), which is composed of a nonlinear fractional evolution inclusion and a time-dependent mixed variational-hemivariational inequality in the framework of Banach spaces. The objective of this paper is four fold. The first one is to investigate the nonemptiness as well as the compactness of the mild solutions set to the FDVHVI. The second aim is to study the optimal control problems described by the FDVHVI. The third goal is to establish the well-posedness results of the FDVHVI, including the existence, uniqueness, and stability. Furthermore, the sensitivity analysis of a perturbed problem associated to the FDVHVI with respect to the initial state and the two parameters is also obtained. Finally, a comprehensive fractional model is given to illustrate the validity of our main results.

Optimal potential shaping on SE(3) via neural ordinary differential equations on Lie groups

This work presents a novel approach for the optimization of dynamic systems on finite-dimensional Lie groups. We rephrase dynamic systems as so-called neural ordinary differential equations (neural ODEs), and formulate the optimization problem on Lie groups. A gradient descent optimization algorithm is presented to tackle the optimization numerically. Our algorithm is scalable, and applicable to any finite-dimensional Lie group, including matrix Lie groups. By representing the system at the Lie algebra level, we reduce the computational cost of the gradient computation. In an extensive example, optimal potential energy shaping for control of a rigid body is treated. The optimal control problem is phrased as an optimization of a neural ODE on the Lie group SE(3), and the controller is iteratively optimized. The final controller is validated on a state-regulation task.

Method for adapting radioacoustic sounding systems of the atmosphere

Stations of radioacoustic sounding (RAS) of the atmosphere are a promising means of obtaining information about the altitudinal distribution of meteorological parameters in the Earth's atmosphere used in the process of solving current scientific and applied tasks to ensure aircraft flights, weather forecasting, etc. However, the effectiveness of the existing radio-acoustic means is insufficient, and there are practical needs for the development of appropriate prospective structures and algorithms, which will be implemented during the construction of specific stations designed to solve actual applied tasks. The article presents the synthesis of the RAS systems algorithm adaptation performed by changing the frequency of the sounding radio signal to fulfill the Bragg condition when the emitted acoustic pulse signal moves along the sounding path from the standpoint of optimal control theory. Since in the problem of synthesizing the algorithm for controlling the frequency of a radio signal, the disturbing and leading to violations of the Bragg condition during the propagation of an acoustic packet along the sounding path, as well as the process causing measurement errors, are considered as random, this problem is considered as a problem of stochastic optimal control. It is shown that in accordance with the separation theorem, known from the theory of optimal stochastic control, the method of controlling the frequency of a sounding radio signal should include sequentially peformed operations of forming estimates of the information parameter of the scattered radio signal, optimal linear filtering of the obtained estimates, and deterministic control of the frequency of the sounding radio signal.

Optimal control of sweeping processes in unmanned surface vehicle and nanoparticle modelling

ABSTRACT This paper addresses novel applications to practical modelling of the newly developed theory of necessary optimality conditions in controlled sweeping/Moreau processes with free time and pointwise control and state constraints. Problems of this type appear, in particular, in dynamical models dealing with unmanned surface vehicles (USVs) and nanoparticles. We formulate optimal control problems for a general class of such dynamical systems and show that the developed necessary optimality conditions for constrained free-time controlled sweeping processes lead us to designing efficient procedures to solve practical models of this class. Moreover, the paper contains numerical calculations of optimal solutions to marine USVs and nanoparticle models in specific situations. Overall, this study contributes to the advancement of optimal control theory for constrained sweeping processes and its practical applications in the fields of marine USVs and nanoparticle modelling.

Neural network approach to portfolio optimization with leverage constraints: a case study on high inflation investment

Motivated by the current global high inflation scenario, we aim to discover a dynamic multi-period allocation strategy to optimally outperform a passive benchmark while adhering to a bounded leverage limit. We formulate an optimal control problem to outperform a benchmark portfolio throughout the investment horizon. To obtain strategies under the bounded leverage constraint among other realistic constraints, we propose a novel leverage-feasible neural network (LFNN) to represent the control, which converts the original constrained optimization problem into an unconstrained optimization problem that is computationally feasible with gradient descent, without dynamic programming. We establish mathematically that the LFNN approximation can yield a solution that is arbitrarily close to the solution of the original optimal control problem with bounded leverage. We further validate the performance of the LFNN empirically by deriving a closed-form solution under jump-diffusion asset price models and show that a shallow LFNN model achieves comparable results on synthetic data. In the case study, we apply the LFNN approach to a four-asset investment scenario with bootstrap-resampled asset returns from the filtered high inflation regimes. The LFNN strategy is shown to consistently outperform the passive benchmark strategy by about 200 bps (median annualized return), with a greater than 90% probability of outperforming the benchmark at the end of the investment horizon.

Data-driven adaptive optimal control for discrete-time linear time-invariant systems

In this paper, the problem of data-driven quadratic optimal control is investigated for discrete-time linear systems without the exact knowledge of system dynamics. A value iteration-based algorithm is proposed to obtain the optimal feedback gain. The main idea of the proposed approach is to obtain two sequences for the approximation of the unique positive definite solution of the corresponding algebraic Riccati matrix equation and the optimal feedback gain by using the collected data of the system states and control inputs. The proposed algorithm could be activated by an arbitrary bounded control input and a simple initial value of the Lyapunov matrix. An important feature of this algorithm is that an original stabilising feedback gain is not required. Finally, the effectiveness of the proposed approach is verified by two examples.

Adaptive Finite Element Method for an Elliptic Optimal Control Problem with Integral State Constraints

Adaptive Finite Element Method for an Elliptic Optimal Control Problem with Integral State Constraints

Dynamic Exception Points for Fair Liver Allocation

There are disparities in access to livers based on transplant patients’ height, which disproportionately affects Hispanics, Asians, and women (across all ethnicities), because short patients can receive transplants from a smaller pool of available deceased donors for medical reasons. Reduced likelihood of transplantation leads to higher mortality rates and longer waiting times. We analyze fairness within the current U.S. liver allocation system where patients receive priority dynamically, based on their model for end-stage liver disease (MELD) scores, which reflect the severity of liver disease. We propose a simple adjustment, providing additional (exception) points based on height and MELD score, that can be easily implemented in practice, which materially reduces the disparity without sacrificing overall efficiency. We model the liver allocation system as a multiclass fluid model of overloaded queues with heterogeneous servers. We impose explicit equity constraints for all static patient classes, that is, height. We characterize the optimal solution under the objective of minimizing pretransplant mortality. The discretized version of the optimal policy is numerically solved using estimates from clinical data and a detailed simulation study demonstrates its effectiveness. The optimal policy, called the equity adjusted mortality risk policy, advocates ranking patients based on their short-term mortality risk adjusted for equity among height classes. Interpretation of the shadow prices of equity constraints in the optimal control problem as MELD exception points is novel in the transplant context since they can be seamlessly mapped into the existing system. Our simulations show that for women, the disparity can be almost completely eliminated. Hispanics and Asians greatly benefit from receiving these MELD exception points also. Our work provides a remedy to reduce the disparities in access to liver transplantation within the MELD-based allocation. Our approach can help the on-going analysis of the continuous distribution model for livers because it also considers aspects of candidate biology, notably height and body surface area. Funding: M. Akan was supported by the National Science Foundation [Grant CMMI-1334194] and the Carnegie Mellon University (CMU) [Onetto Fellowship in Operations Management]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/serv.2023.0092 .

Iterative Chebyshev approximation method for optimal control problems

We present a novel numerical approach for solving nonlinear constrained optimal control problems (NCOCPs). Instead of directly solving the NCOCPs, we start by linearizing the constraints and dynamic system, which results in a sequence of sub-problems. For each sub-problem, we use finite number of Chebyshev polynomials to estimate the control and state vectors. To eliminate the errors at non-collocation points caused by conventional collocation methods, we additionally estimate the coefficient functions involved in the linear constraints and dynamic system by Chebyshev polynomials. By leveraging the characteristics of Chebyshev polynomials, the approximate sub-problem is changed into an equivalent nonlinear optimization problem with linear equality constraints. Consequently, any feasible point of the approximate sub-problem will satisfy the constraints and dynamic system throughout the entire time scale. To validate the efficacy of the new method, we solve three examples and assess the accuracy of the method through the computation of its approximation error. Numerical results obtained show that our approach achieves lower approximation error when compared to the Chebyshev pseudo-spectral method. The proposed method is particularly suitable for scenarios that require high-precision approximation, such as aerospace and precision instrument production.

How can the preferences of policy makers be operationalised in optimum control problems with macroeconometric models? A case study for Slovenian fiscal policies

How can the preferences of policy makers be operationalised in optimum control problems with macroeconometric models? A case study for Slovenian fiscal policies

Dynamic analysis and control strategies of the SEIHR rumor diffusion model in online social networks

With the prevalence of online social networks, rumors spread rampantly online. The versatility and complexity of human attitudes make dissemination more challenging to portray, so it is meaningful to develop a more comprehensive diffusion model. To this end, an improved SEIHR model considering three attitudes towards rumors is proposed. Then we deduce the stability of the dynamics model and use two scenarios to validate its accuracy. Meanwhile, rumor diffusion generates extensive social threats. To mitigate such threats, we introduce three immunization strategies and forge bonds among their basic reproduction numbers. Besides, we develop three synergistic optimal controls that vary with the time and degree to impede rumor diffusion. The analysis of the optimal control problem is conducted by Pontryagin's maximum principle. Next, experiments are conducted to verify the efficacy of the SEIHR model and control strategies. Our simulations show that adding the exposed and the hibernated undermines the impact of rumors and puts off terminal time. Additionally, immunization strategies effectively curb rumor diffusion. Moreover, the combination of three synergistic optimal controls leads to the smallest rumor scale and the least cumulative total costs.

Deep reinforcement learning for optimal control of induction welding process

Optimizing induction welding (IW) process parameters for the application of joining thermoplastic composites is challenging as it requires achieving complex spatiotemporal thermal characteristics along the weld-line to obtain desired weld quality. We formulate an optimal control problem which captures these requirements and seeks to optimize the IW coil speed using a fast-acting dynamic IW process model. We develop a novel Deep Reinforcement Learning (DRL) framework to solve this computationally challenging control problem and demonstrate via simulation study that the learned DRL feedback control policy results in better spatiotemporal thermal characteristics as compared to the current state-of-the-art.

Finite Elements with Switch Detection for numerical optimal control of nonsmooth dynamical systems with set-valued heaviside step functions

This paper develops high-accuracy methods for the numerical solution of optimal control problems subject to nonsmooth differential equations with set-valued Heaviside step functions. An important subclass of these systems are Filippov systems. By writing the Heaviside step function as the solution map of a linear program and using its optimality conditions, the initial nonsmooth system is rewritten into an equivalent Dynamic Complementarity System (DCS). The Finite Elements with Switch Detection (FESD) method (Nurkanović et al., 2024) was originally developed for Filippov systems transformed via Stewart’s reformulation into DCS (Stewart, 1990). This paper extends it to the above mentioned class of nonsmooth systems. The key ideas are to start with a standard Runge–Kutta method for the DCS and to let the integration step sizes to be degrees of freedom. Then, additional conditions are introduced to allow implicit but accurate switch detection and to remove possible spurious degrees of freedom when no switches occur. The theoretical properties of the FESD method are studied. The motivation for these developments is to obtain a computationally tractable formulation of nonsmooth optimal control problems. Numerical simulations and optimal control examples are used to illustrate the favorable properties of the proposed approach. All methods introduced in this paper are implemented in the open-source software package nosnoc (Nurkanović and Diehl, 2022).