Formulation Of Control Problem Research Articles

Desensitized Optimal Control

Desensitized optimal control (DOC) enables the formulation of optimal control problems that incorporate the optimal reference solution’s sensitivities to state perturbations into the optimization process. Mathematically, this is achieved through the introduction of Lagrange multiplierlike quantities that act as additional state variables and capture the desired sensitivity information. By penalizing user-specified sensitivities, the reference solution becomes easier to control under feedback. Thus, the DOC method can be viewed as a method to improve nonlinear robustness. By converting fixed parameters appearing in the problem formulation to state variables with trivial dynamics, the DOC approach can be used to improve robustness to parameter uncertainties. The paper also analyzes the relationship between sensitivities and the covariance matrix and compares the benefits and limitations of covariance shaping versus DOC. A simple Zermelo-type boat path optimization problem with uncertainties in the water current is analyzed to demonstrate the DOC approach.

Reducing Tyre Wear Emissions of Automated Articulated Vehicles through Trajectory Planning.

Effective emission control technologies and eco-friendly propulsion systems have been developed to decrease exhaust particle emissions. However, more work must be conducted on non-exhaust traffic-related sources such as tyre wear. The advent of automated vehicles (AVs) enables researchers and automotive manufacturers to consider ways to further decrease tyre wear, as vehicles will be controlled by the system rather than by the driver. In this direction, this work presents the formulation of an optimal control problem for the trajectory optimisation of automated articulated vehicles for tyre wear minimisation. The optimum velocity profile is sought for a predefined road path from a specific starting point to a final one to minimise tyre wear in fixed time cases. Specific boundaries and constraints are applied to the problem to ensure the vehicle's stability and the feasibility of the solution. According to the results, a small increase in the journey time leads to a significant decrease in the mass loss due to tyre wear. The employment of articulated vehicles with low powertrain capabilities leads to greater tyre wear, while excessive increases in powertrain capabilities are not required. The conclusions pave the way for AV researchers and manufacturers to consider tyre wear in their control modules and come closer to the zero-emission goal.

Data-Driven Stochastic Optimal Control With Safety Constraints Using Linear Transfer Operators

We provide a data-driven framework for optimal control of a continuous-time control-affine stochastic dynamical system. The proposed framework relies on the linear operator theory involving Perron-Frobenius (P-F) and Koopman operators. Our first result involving the P-F operator provides a convex formulation to the optimal control problem in the dual space of densities. This convex formulation of the stochastic optimal control problem leads to an infinite-dimensional convex program. The finite-dimensional approximation of the convex program is obtained using a data-driven approximation of the linear operators. We provide rigorous error bounds and convergence rate for the data-driven approximation of the linear operators and comment on the convergence rate of the optimal control w.r.t. data size. Our second results demonstrate using the Koopman operator, dual to the P-F operator, for the stochastic optimal control design. We show that the Hamilton Jacobi Bellman (HJB) equation can be expressed using the Koopman operator. We provide an iterative procedure along the lines of a popular policy iteration algorithm based on the data-driven approximation of the Koopman operator for solving the HJB equation. The two formulations, namely the convex formulation involving the P-F operator and the Koopman-based formulation using the HJB equation, can be viewed as dual to each other where the duality follows due to the dual nature of the P-F and Koopman operators. Finally, we present examples to demonstrate the efficacy of the developed framework and verify the convergence rates for the operator and optimal control numerically.

Reduced Desensitization Formulation for Optimal Control Problems

Solutions to nonlinear optimal control problems (OCPs) exhibit dispersions under model uncertainties and it is desirable to generate optimal solutions that exhibit less sensitivity to model uncertainties. We propose a novel solution desensitization method dubbed “Reduced Desensitization Formulation (RDF)” by leveraging non-uniqueness of the solution of the costate differential equations when a hybrid indirect-direct optimization method is used. A key property of the RDF method is a significant reduction in the number of differential equations needed for generating desensitized solutions. This feature facilitates the generation of open-loop desensitized trajectories and makes the methodology applicable to OCPs with a larger number of uncertain parameters. To demonstrate the utility of the RDF method, three important classes of trajectory optimization problems are considered with uncertainty in the thrust magnitude of the propulsion system: (1) minimum-fuel low-thrust interplanetary rendezvous maneuvers, (2) low-thrust orbit-raising maneuvers, and (3) minimum-fuel high-thrust rocket-landing problems. For the considered problems with bang-bang control profiles, an analysis is presented on the change in the number of control switches between sensitive and desensitized optimal solutions. Numerical results demonstrate desensitization of the considered performance indices with respect to the thrust magnitude of the propulsion system.

Optimal control for reachability of Markov jump switching Boolean control networks subject to output trackability

This article aims at designing the finite horizon optimal control sequence (FHOCS) for ensuring the reachability of Markov jump switching Boolean control networks (MJSBCNs) subject to output trackability. The key step in solving such a problem is to construct a Luenberger-like observer (LLO). Our approach includes a few steps as follows. We firstly propose the formulation of the optimal control problem (OCP) of maximising the probability of finite-time reachability subject to output trackability. Then, by converting the dynamics of MJSBCNs into an equivalent expectation system, an LLO is established for MJSBCNs. In this setup, a Luenberger-like observation matrix is constructed, which is an effect tool to solve the finite-time reachability problem. Finally, an LLO method is developed to find the FHOCS by maximising the T -step Luenberger-like observation matrix. In a biological example, we show the effectiveness of our developed methods.

Determining Fast Battery Charging Profiles Using an Equivalent Circuit Model and a Direct Optimal Control Approach

This work used an electrical equivalent circuit model combined with a temperature model and computational optimal control methods to determine minimum time charging profiles for a lithium–ion battery. To effectively address the problem, an optimal control problem formulation and direct solution approach were adopted. The results showed that, in most cases studied, the solution to the battery’s fast-charging problem resembled the constant current–constant voltage (CC-CV) charging protocol, with the advantage being that our proposed approach optimally determined the switching time between the CC and CV phases, as well as the final time of the charging process. Considering path constraints related to the terminal voltage and temperature gradient between the cell core and case, the results also showed that additional rules could be incorporated into the protocol to protect the battery against under/over voltage-related damage and high-temperature differences between the core and its case. This work addressed several challenges and knowledge gaps, including emulating the CC-CV protocol using a multi-phase optimal control approach and direct collocation methods, and improving it by including efficiency and degradation terms in the objective function and safety constraints. To the authors’ knowledge, this is the first time the CC-CV protocol has been represented as the solution to a multi-phase optimal control problem.

Determination of Fast Battery-Charging Profiles Using an Electrochemical Model and a Direct Optimal Control Approach

This paper describes an approach to determine a fast-charging profile for a lithium-ion battery by utilising a simplified single-particle electrochemical model and direct collocation methods for optimal control. An optimal control problem formulation and a direct solution approach were adopted to address the problem effectively. The results shows that, in some cases, the optimal current profile resembles the current profile in the Constant Current–Constant Voltage charging protocol. Several challenges and knowledge gaps were addressed in this work, including a reformulation of the optimal control problem that utilises direct methods as an alternative to overcome the limitations of indirect methods employed in similar studies. The proposed formulation considers the minimum-time optimal control case, trade-offs between the total charging time, the maximisation of the lithium bulk concentration, and energy efficiency, along with inequality constraints and other factors not previously considered in the literature, which can be helpful in practical applications.

Optimization over the Pareto front of nonconvex multi-objective optimal control problems

Simultaneous optimization of multiple objective functions results in a set of trade-off, or Pareto, solutions. Choosing a, in some sense, best solution in this set is in general a challenging task: In the case of three or more objectives the Pareto front is usually difficult to view, if not impossible, and even in the case of just two objectives constructing the whole Pareto front so as to visually inspect it might be very costly. Therefore, optimization over the Pareto (or efficient) set has been an active area of research. Although there is a wealth of literature involving finite dimensional optimization problems in this area, there is a lack of problem formulation and numerical methods for optimal control problems, except for the convex case. In this paper, we formulate the problem of optimizing over the Pareto front of nonconvex constrained and time-delayed optimal control problems as a bi-level optimization problem. Motivated by existing solution differentiability results, we propose an algorithm incorporating (i) the Chebyshev scalarization, (ii) a concept of the essential interval of weights, and (iii) the simple but effective bisection method, for optimal control problems with two objectives. We illustrate the working of the algorithm on two example problems involving an electric circuit and treatment of tuberculosis and discuss future lines of research for new computational methods.

Data-driven discovery of sparse dynamical model of cardiovascular system for model predictive control

Cardiovascular diseases remain the leading cause of death globally. In recent years, vagal nerve stimulation (VNS) has shown promising results in the treatment of a number of cardiovascular diseases. In this approach, mild electrical pulses are sent to the brain via the vagus nerve. This open-loop neurostimulation, however, leads to various side effects due to physiological and inter-patient variability and therefore a closed-loop delivery strategy of electrical pulses that accounts for this variability is desired. In this context, we envision data-driven sparse dynamical model parameterized by patient-specific data as appropriate for use in closed loop controller design. In this work, we build a dynamical model for mean arterial pressure and heart rate using the method sparse identification of nonlinear dynamics (SINDy). As a proxy for real datasets or measurements from a patient, we simulate a mechanistic model from the literature and then discover a data-driven model for predicting mean arterial pressure and heart rate in response to neural stimulus. This discovered model is then used to design a controller to be implemented in closed-loop via model predictive control. We observe that this data-driven model is interpretable, consistent with experiments, provides insights on the sensitivity of different stimulation locations and simplifies the formulation of the optimal control problem. Noting the set-point tracking performance of this closed-loop model-based controller that uses this discovered model, we conclude that the model is adequate in capturing the dynamics of a highly nonlinear cardiovascular system for the purpose of optimal predictive controller design.

Adaptive Synthesized Control for Solving the Optimal Control Problem

The development of artificial intelligence systems assumes that a machine can independently generate an algorithm of actions or a control system to solve the tasks. To do this, the machine must have a formal description of the problem and possess computational methods for solving it. This article deals with the problem of optimal control, which is the main task in the development of control systems, insofar as all systems being developed must be optimal from the point of view of a certain criterion. However, there are certain difficulties in implementing the resulting optimal control modes. This paper considers an extended formulation of the optimal control problem, which implies the creation of such systems that would have the necessary properties for its practical implementation. To solve it, an adaptive synthesized optimal control approach based on the use of numerical methods of machine learning is proposed. Such control moves the control object, optimally changing the position of the stable equilibrium point in the presence of some initial position uncertainty. As a result, from all possible synthesized controls, one is chosen that is less sensitive to changes in the initial state. As an example, the optimal control problem of a quadcopter with complex phase constraints is considered. To solve this problem, according to the proposed approach, the control synthesis problem is firstly solved to obtain a stable equilibrium point in the state space using a machine learning method of symbolic regression. After that, optimal positions of the stable equilibrium point are searched using a particle swarm optimization algorithm using the source functional from the initial optimal control problem statement. It is shown that such an approach allows for generating the control system automatically by computer, basing this on the formal statement of the problem and then directly implementing it onboard as far as the stabilization system has already been introduced.

Stochastic optimal control of mesostructure of supramolecular assemblies using dissipative particle dynamics and dynamic programming with experimental validation

The self-assembly process, where molecules form complex structures through interaction forces, has broad applications in various fields. However, controlling the dynamics of self-assembling molecules presents challenges, such as kinetic trapping, and the stochastic and nonlinear nature of governing equations. Recent studies have used Markov decision processes to control self-assemblies, but this approach may not be suitable for more complex systems and may result in disassembly or irreversible changes. To address these limitations, a novel integrated framework for the systematic development of control schemes for self-assembling systems is proposed. The framework uses a dissipative particle dynamics model to accurately capture self-assembly and generate a high-fidelity representation of relevant nanostructures. The probabilities of transitioning between states are calculated, which enables the formulation of a stochastic optimal control problem. To solve this problem, dynamic programming is employed. The framework is applied to a case study involving dynamic binary complexes with multiple unwanted metastable states, which serves to experimentally demonstrate the efficacy of the controller in driving the system towards desired morphologies. Finally, the closed-loop simulation results highlight that the proposed framework can drive the system of thermosensitive DBCs towards target morphologies while avoiding kinetic traps.

Automatic decomposition of large-scale industrial processes for distributed MPC on the Shell–Yokogawa Platform for Advanced Control and Estimation (PACE)

The kernel of industrial advanced process control (APC) lies in the formulation and solution of model predictive control (MPC) problems, which specify the controller moves according to the solution of an optimal control problem at each sampling time. A significant challenge is the online computation for large-scale industrial systems. As the state-of-the-art APC technology, the Shell–Yokogawa Platform for Advanced Control and Estimation (PACE) has adopted a systematic framework of handling dynamic optimization of large-scale systems, where an automatic decomposition procedure generates subsystems for distributed MPC. The decomposition is implemented on network representations of the MPC models that capture interactions among process variables, with community detection used to maximize the statistical significance of the subnetworks with preferred internal interconnections. This paper introduces the fundamentals of such a decomposition approach and this functionality in PACE, followed by a case study on a crude distillation process to showcase its industrial application.

A space–time variational method for optimal control problems: well-posedness, stability and numerical solution

We consider an optimal control problem constrained by a parabolic partial differential equation with Robin boundary conditions. We use a space–time variational formulation in Lebesgue–Bochner spaces yielding a boundedly invertible solution operator. The abstract formulation of the optimal control problem yields the Lagrange function and Karush–Kuhn–Tucker conditions in a natural manner. This results in space–time variational formulations of the adjoint and gradient equation in Lebesgue–Bochner spaces, which are proven to be boundedly invertible. Necessary and sufficient optimality conditions are formulated and the optimality system is shown to be boundedly invertible. Next, we introduce a conforming uniformly stable simultaneous space–time (tensorproduct) discretization of the optimality system in these Lebesgue–Bochner spaces. Using finite elements of appropriate orders in space and time for trial and test spaces, this setting is known to be equivalent to a Crank–Nicolson time-stepping scheme for parabolic problems. Comparisons with existing methods are detailed. We show numerical comparisons with time-stepping methods. The space–time method shows good stability properties and requires fewer degrees of freedom in time to reach the same accuracy.

Optimal Tracking in Switched Systems With Free Final Time and Fixed Mode Sequence Using Approximate Dynamic Programming.

Optimal tracking in switched systems with fixed mode sequence and free final time is studied in this article. In the optimal control problem formulation, the switching times and the final time are treated as parameters. For solving the optimal control problem, approximate dynamic programming (ADP) is used. The ADP solution uses an inner loop to converge to the optimal policy at each time step. In order to decrease the computational burden of the solution, a new method is introduced, which uses evolving suboptimal policies (not the optimal policies), to learn the optimal solution. The effectiveness of the proposed solutions is evaluated through numerical simulations.

A statistical moment-based spectral approach to the chance-constrained stochastic optimal control of epidemic models

This paper presents a spectral approach to the uncertainty management in epidemic models through the formulation of chance-constrained stochastic optimal control problems. Specifically, a statistical moment-based polynomial expansion is used to calculate surrogate models of the stochastic state variables of the problem that allow for the efficient computation of their main statistics as well as their marginal and joint probability density functions at each time instant, which enable the uncertainty management in the epidemic model. Moreover, the surrogate models are employed to perform the corresponding sensitivity and risk analyses. The proposed methodology provides the designers of the optimal control policies with the capability to increase the predictability of the outcomes by adding suitable chance constraints to the epidemic model and formulating a proper cost functional. The chance-constrained optimal control of a COVID-19 epidemic model is considered in order to illustrate the practical application of the proposed methodology.

Stochastic Framework for Optimal Control of Planetary Reentry Trajectories Under Multilevel Uncertainties

We present a novel stochastic optimal control framework that accounts for various types of uncertainties, with application to reentry trajectory planning. The formulation of the optimal trajectory control problem is presented in the context of an indirect method where a functional objective associated with the terminal vehicle speed is to be minimized. Uncertain input parameters in the optimal trajectory control model, including aerodynamic parameters and initial and terminal conditions, are modeled as aleatory random variables, while the statistical parameters of these aleatory distributions are themselves random variables. The parametric and model uncertainties are simultaneously propagated through an extended polynomial chaos expansion (EPCE) formalism. Several metrics are described to evaluate response statistics and presented as insightful tools for robust decision making. Specifically, the response probability density function (PDF) reflecting influence of both epistemic and aleatory uncertainties is obtained. By sampling over the random variables representing model error, an ensemble of response PDFs is generated and the associated failure probability is estimated as a random variable with its own polynomial chaos expansion. Besides, the sensitivity index functions of response PDF with respect to the statistical parameters are evaluated. Coupling parametric and model uncertainties within the EPCE framework leads to a robust and efficient paradigm for multilevel uncertainty propagation and PDF characterization in general optimal control problems.

Existence of solutions for an optimal control problem in forestry management

Physical, biological, and economic factors remain inexhaustible sources when determining silvicultural management strategies that maximize the net benefit of timber volume. The objective of this paper is to formulate an optimal control problem to maximize the net benefit of timber volume and to demonstrate the existence of optimal solutions to this problem. The optimal control theory will be used, through the formulation of a bioeconomic optimal control problem subjected to a system of three ordinary differential equations that describe the interactions between the living biomass, the intrinsic growth of the biomass, and the burned area. Four control variables are associated with these differential equations, which are reforestation, felling, thinning, and fire prevention. Assuming adequate conditions on the control variables, the existence of optimal solutions to the optimal control problem is obtained using Filippov’s theorem. In conclusion, the optimal control problem is well formulated and solutions exist. This will allow us to define forest management strategies that optimize the economic gain of the process, taking into consideration the physical, biological, and economic phenomena involved in the dynamics of forest plantations.

A Cartesian-Based Trajectory Optimization with Jerk Constraints for a Robot

To address the time-optimal trajectory planning (TOTP) problem with joint jerk constraints in a Cartesian coordinate system, we propose a time-optimal path-parameterization (TOPP) algorithm based on nonlinear optimization. The key insight of our approach is the presentation of a comprehensive and effective iterative optimization framework for solving the optimal control problem (OCP) formulation of the TOTP problem in the (s,s˙)-phase plane. In particular, we identify two major difficulties: establishing TOPP in Cartesian space satisfying third-order constraints in joint space, and finding an efficient computational solution to TOPP, which includes nonlinear constraints. Experimental results demonstrate that the proposed method is an effective solution for time-optimal trajectory planning with joint jerk limits, and can be applied to a wide range of robotic systems.

Moment-Based Model Predictive Control of Autonomous Systems

Great efforts have been devoted to the intelligent control of autonomous systems. Yet, most of existing methods fail to effectively handle the uncertainty of their environment and models. Uncertain locations of dynamic obstacles pose a major challenge for their optimal control and safety, while their linearization or simplified system models reduce their actual performance. To address them, this paper presents a new model predictive control framework with finite samples and a Gaussian model, resulting in a chance-constrained program. Its nominal model is combined with a Gaussian process. Its residual model uncertainty is learned. The resulting method addresses an efficiently solvable approximate formulation of a stochastic optimal control problem by using approximations for efficient computation. There is no perfect distribution knowledge of a dynamic obstacle's location uncertainty. Only finite samples from sensors or past data are available for moment estimation. We use the uncertainty propagation of a system's state and obstacles' locations to derive a general collision avoidance condition under tight concentration bounds on the error of the estimated moments. Thus, this condition is suitable for different obstacles, e.g., bounding box and ellipsoid obstacles. We provide proved guarantees on the satisfaction of the chance-constraints corresponding to the nominal yet unknown moments. Simulation examples of a vehicle's control are used to show that the proposed method can well realize autonomous control and obstacle avoidance of a vehicle, when it operates in an uncertain environment with moving obstacles. It outperforms the existing moment methods in both performance and computational time.

Mean field games with branching

Mean field games are concerned with the limit of large-population stochastic differential games where the agents interact through their empirical distribution. In the classical setting, the number of players is large but fixed throughout the game. However, in various applications, such as population dynamics or economic growth, the number of players can vary across time and this may lead to different Nash equilibria. In order to account for this evolution, we introduce a branching mechanism in the population of agents and obtain a variant of the original mean field game problem. As a first step, we study a simple model using a PDE approach to illustrate the main differences with the classical setting. We prove existence of a solution and show that it provides an approximate Nash-equilibrium for large population games. We also present a numerical example for a linear–quadratic model. Then we study the problem in a general setting by a probabilistic approach. It is based upon the relaxed formulation of stochastic control problems which allows us to obtain a general existence result.