Formulation Of Optimal Control Problem Research Articles

Desensitized Optimal Control

Desensitized optimal control (DOC) enables the formulation of optimal control problems that incorporate the optimal reference solution’s sensitivities to state perturbations into the optimization process. Mathematically, this is achieved through the introduction of Lagrange multiplierlike quantities that act as additional state variables and capture the desired sensitivity information. By penalizing user-specified sensitivities, the reference solution becomes easier to control under feedback. Thus, the DOC method can be viewed as a method to improve nonlinear robustness. By converting fixed parameters appearing in the problem formulation to state variables with trivial dynamics, the DOC approach can be used to improve robustness to parameter uncertainties. The paper also analyzes the relationship between sensitivities and the covariance matrix and compares the benefits and limitations of covariance shaping versus DOC. A simple Zermelo-type boat path optimization problem with uncertainties in the water current is analyzed to demonstrate the DOC approach.

Approximate propagation of normal distributions for stochastic optimal control of nonsmooth systems

We present a method for the approximate propagation of mean and covariance of a probability distribution through ordinary differential equations (ODE) with discontinuous right-hand side. For piecewise affine systems, a normalization of the propagated probability distribution at every time step allows us to analytically compute the expectation integrals of the mean and covariance dynamics while explicitly taking into account the discontinuity. This leads to a natural smoothing of the discontinuity such that for relevant levels of uncertainty the resulting ODE can be integrated directly with standard schemes and it is neither necessary to prespecify the switching sequence nor to use a switch detection method. We then show how this result can be employed in the more general case of piecewise smooth functions based on a structure preserving linearization scheme. The resulting dynamics can be straightforwardly used within standard formulations of stochastic optimal control problems with chance constraints.

Data-Driven Stochastic Optimal Control With Safety Constraints Using Linear Transfer Operators

We provide a data-driven framework for optimal control of a continuous-time control-affine stochastic dynamical system. The proposed framework relies on the linear operator theory involving Perron-Frobenius (P-F) and Koopman operators. Our first result involving the P-F operator provides a convex formulation to the optimal control problem in the dual space of densities. This convex formulation of the stochastic optimal control problem leads to an infinite-dimensional convex program. The finite-dimensional approximation of the convex program is obtained using a data-driven approximation of the linear operators. We provide rigorous error bounds and convergence rate for the data-driven approximation of the linear operators and comment on the convergence rate of the optimal control w.r.t. data size. Our second results demonstrate using the Koopman operator, dual to the P-F operator, for the stochastic optimal control design. We show that the Hamilton Jacobi Bellman (HJB) equation can be expressed using the Koopman operator. We provide an iterative procedure along the lines of a popular policy iteration algorithm based on the data-driven approximation of the Koopman operator for solving the HJB equation. The two formulations, namely the convex formulation involving the P-F operator and the Koopman-based formulation using the HJB equation, can be viewed as dual to each other where the duality follows due to the dual nature of the P-F and Koopman operators. Finally, we present examples to demonstrate the efficacy of the developed framework and verify the convergence rates for the operator and optimal control numerically.

Determining Fast Battery Charging Profiles Using an Equivalent Circuit Model and a Direct Optimal Control Approach

This work used an electrical equivalent circuit model combined with a temperature model and computational optimal control methods to determine minimum time charging profiles for a lithium–ion battery. To effectively address the problem, an optimal control problem formulation and direct solution approach were adopted. The results showed that, in most cases studied, the solution to the battery’s fast-charging problem resembled the constant current–constant voltage (CC-CV) charging protocol, with the advantage being that our proposed approach optimally determined the switching time between the CC and CV phases, as well as the final time of the charging process. Considering path constraints related to the terminal voltage and temperature gradient between the cell core and case, the results also showed that additional rules could be incorporated into the protocol to protect the battery against under/over voltage-related damage and high-temperature differences between the core and its case. This work addressed several challenges and knowledge gaps, including emulating the CC-CV protocol using a multi-phase optimal control approach and direct collocation methods, and improving it by including efficiency and degradation terms in the objective function and safety constraints. To the authors’ knowledge, this is the first time the CC-CV protocol has been represented as the solution to a multi-phase optimal control problem.

Determination of Fast Battery-Charging Profiles Using an Electrochemical Model and a Direct Optimal Control Approach

This paper describes an approach to determine a fast-charging profile for a lithium-ion battery by utilising a simplified single-particle electrochemical model and direct collocation methods for optimal control. An optimal control problem formulation and a direct solution approach were adopted to address the problem effectively. The results shows that, in some cases, the optimal current profile resembles the current profile in the Constant Current–Constant Voltage charging protocol. Several challenges and knowledge gaps were addressed in this work, including a reformulation of the optimal control problem that utilises direct methods as an alternative to overcome the limitations of indirect methods employed in similar studies. The proposed formulation considers the minimum-time optimal control case, trade-offs between the total charging time, the maximisation of the lithium bulk concentration, and energy efficiency, along with inequality constraints and other factors not previously considered in the literature, which can be helpful in practical applications.

Stochastic optimal control of mesostructure of supramolecular assemblies using dissipative particle dynamics and dynamic programming with experimental validation

The self-assembly process, where molecules form complex structures through interaction forces, has broad applications in various fields. However, controlling the dynamics of self-assembling molecules presents challenges, such as kinetic trapping, and the stochastic and nonlinear nature of governing equations. Recent studies have used Markov decision processes to control self-assemblies, but this approach may not be suitable for more complex systems and may result in disassembly or irreversible changes. To address these limitations, a novel integrated framework for the systematic development of control schemes for self-assembling systems is proposed. The framework uses a dissipative particle dynamics model to accurately capture self-assembly and generate a high-fidelity representation of relevant nanostructures. The probabilities of transitioning between states are calculated, which enables the formulation of a stochastic optimal control problem. To solve this problem, dynamic programming is employed. The framework is applied to a case study involving dynamic binary complexes with multiple unwanted metastable states, which serves to experimentally demonstrate the efficacy of the controller in driving the system towards desired morphologies. Finally, the closed-loop simulation results highlight that the proposed framework can drive the system of thermosensitive DBCs towards target morphologies while avoiding kinetic traps.

Optimal Tracking in Switched Systems With Free Final Time and Fixed Mode Sequence Using Approximate Dynamic Programming.

Optimal tracking in switched systems with fixed mode sequence and free final time is studied in this article. In the optimal control problem formulation, the switching times and the final time are treated as parameters. For solving the optimal control problem, approximate dynamic programming (ADP) is used. The ADP solution uses an inner loop to converge to the optimal policy at each time step. In order to decrease the computational burden of the solution, a new method is introduced, which uses evolving suboptimal policies (not the optimal policies), to learn the optimal solution. The effectiveness of the proposed solutions is evaluated through numerical simulations.

A statistical moment-based spectral approach to the chance-constrained stochastic optimal control of epidemic models

This paper presents a spectral approach to the uncertainty management in epidemic models through the formulation of chance-constrained stochastic optimal control problems. Specifically, a statistical moment-based polynomial expansion is used to calculate surrogate models of the stochastic state variables of the problem that allow for the efficient computation of their main statistics as well as their marginal and joint probability density functions at each time instant, which enable the uncertainty management in the epidemic model. Moreover, the surrogate models are employed to perform the corresponding sensitivity and risk analyses. The proposed methodology provides the designers of the optimal control policies with the capability to increase the predictability of the outcomes by adding suitable chance constraints to the epidemic model and formulating a proper cost functional. The chance-constrained optimal control of a COVID-19 epidemic model is considered in order to illustrate the practical application of the proposed methodology.

Risk-averse stochastic optimal control: An efficiently computable statistical upper bound

In this paper, we discuss an application of the Stochastic Dual Dynamic Programming (SDDP) type algorithm to nested risk-averse formulations of Stochastic Optimal Control (SOC) problems. We propose a construction of a statistical upper bound for the optimal value of risk-averse SOC problems. This outlines an approach to a solution of a long standing problem in that area of research. The bound holds for a large class of convex and monotone conditional risk mappings. Finally, we show the validity of the statistical upper bound to solve a real-life stochastic hydro-thermal planning problem.

Existence of solutions for an optimal control problem in forestry management

Physical, biological, and economic factors remain inexhaustible sources when determining silvicultural management strategies that maximize the net benefit of timber volume. The objective of this paper is to formulate an optimal control problem to maximize the net benefit of timber volume and to demonstrate the existence of optimal solutions to this problem. The optimal control theory will be used, through the formulation of a bioeconomic optimal control problem subjected to a system of three ordinary differential equations that describe the interactions between the living biomass, the intrinsic growth of the biomass, and the burned area. Four control variables are associated with these differential equations, which are reforestation, felling, thinning, and fire prevention. Assuming adequate conditions on the control variables, the existence of optimal solutions to the optimal control problem is obtained using Filippov’s theorem. In conclusion, the optimal control problem is well formulated and solutions exist. This will allow us to define forest management strategies that optimize the economic gain of the process, taking into consideration the physical, biological, and economic phenomena involved in the dynamics of forest plantations.

A Cartesian-Based Trajectory Optimization with Jerk Constraints for a Robot

To address the time-optimal trajectory planning (TOTP) problem with joint jerk constraints in a Cartesian coordinate system, we propose a time-optimal path-parameterization (TOPP) algorithm based on nonlinear optimization. The key insight of our approach is the presentation of a comprehensive and effective iterative optimization framework for solving the optimal control problem (OCP) formulation of the TOTP problem in the (s,s˙)-phase plane. In particular, we identify two major difficulties: establishing TOPP in Cartesian space satisfying third-order constraints in joint space, and finding an efficient computational solution to TOPP, which includes nonlinear constraints. Experimental results demonstrate that the proposed method is an effective solution for time-optimal trajectory planning with joint jerk limits, and can be applied to a wide range of robotic systems.

Moment-Based Model Predictive Control of Autonomous Systems

Great efforts have been devoted to the intelligent control of autonomous systems. Yet, most of existing methods fail to effectively handle the uncertainty of their environment and models. Uncertain locations of dynamic obstacles pose a major challenge for their optimal control and safety, while their linearization or simplified system models reduce their actual performance. To address them, this paper presents a new model predictive control framework with finite samples and a Gaussian model, resulting in a chance-constrained program. Its nominal model is combined with a Gaussian process. Its residual model uncertainty is learned. The resulting method addresses an efficiently solvable approximate formulation of a stochastic optimal control problem by using approximations for efficient computation. There is no perfect distribution knowledge of a dynamic obstacle's location uncertainty. Only finite samples from sensors or past data are available for moment estimation. We use the uncertainty propagation of a system's state and obstacles' locations to derive a general collision avoidance condition under tight concentration bounds on the error of the estimated moments. Thus, this condition is suitable for different obstacles, e.g., bounding box and ellipsoid obstacles. We provide proved guarantees on the satisfaction of the chance-constraints corresponding to the nominal yet unknown moments. Simulation examples of a vehicle's control are used to show that the proposed method can well realize autonomous control and obstacle avoidance of a vehicle, when it operates in an uncertain environment with moving obstacles. It outperforms the existing moment methods in both performance and computational time.

Model Predictive Control for Energy-Efficient Yaw-Stabilizing Torque Vectoring in Electric Vehicles With Four In-Wheel Motors

This paper considers the problem of stabilizing and energy-efficient torque vectoring for electric vehicles with four independent in-wheel motors. In electric vehicles with four in-wheel motors, four electric motors are separately attached to the four wheels without an extra drive shaft. The mechanical and structural nature enables reduction of energy loss during power transmission and securing extra interior space. In addition, independent control of wheel torques can provide better yaw motion stability and improved energy efficiency. This paper proposes two model predictive control (MPC) methods for stability-constrained energy-efficient torque vectoring of four in-wheel motor electric vehicles. For the adaptive weighting factors of multiple objective functions of reference tracking and energy saving, we use exponential functions that vary with the lateral motion and steering input. Depending on the optimal control problem formulation with different dynamical system equations and constraints, the associated predictive controller can be represented as either a linear parameter-varying MPC (LPV-MPC) or nonlinear MPC (NMPC). For LPV-MPC, longitudinal and lateral motions are decoupled, whereas the coupled dynamics of the two-track model are exploited in NMPC. For comparisons and demonstrations of LPV-MPC and NMPC in the MPC of torque vectoring, three driving scenarios are simulated with a high-fidelity vehicle simulation solution, CarMaker (IPG Automotive). In comparison with the built-in IPG driver implemented in CarMaker, we demonstrate fuel efficiency improvements of over 2–3 % on average with guaranteed yaw stability.

Multiobjective strict dissipativity via a weighted sum approach

We consider nonlinear model predictive control (MPC) with multiple competing cost functions. This leads to the formulation of multiobjective optimal control problems (MO OCPs). Since the design of MPC algorithms for directly solving multiobjective problems is rather complicated, particularly if terminal conditions shall be avoided, we use an indirect approach via a weighted sum formulation for solving these MO OCPs. This way, for each set of weights we obtain an optimal control problem with a single objective. In economic MPC it is known that strict dissipativity is the key assumption for concluding performance and stability results. We thus investigate under which conditions a convex combination of strictly dissipative stage costs is strictly dissipative again. We first give conditions for problems with linear dynamics and then move on to consider fully nonlinear optimal control problems. We derive both necessary and sufficient conditions on the individual cost functions and on the weights to conclude strict dissipativity and illustrate our findings with numerical examples.

Battery lifetime extension through optimal design and control of traction and heating systems in hybrid drivetrains

In this paper, we investigate the optimal design and control of an integrated energy and thermal management system (IETMS) of a battery-assisted trolley bus that is subject to minimum battery lifetime requirements. Therefore, we jointly optimize both the control trajectories of the traction and the heating systems and the design of the thermal energy buffer. We further analyze the resulting Pareto fronts which characterize the trade-off between the battery lifetime and the energy consumption. This holistic approach fills a gap in the literature published, namely the formulation of an optimization problem that combines component sizing with a battery-health-aware IETMS. While the model derived allows the formulation of a convex optimal control problem for a given vehicle design, the combined design and control problem is non-convex. Therefore, we perform grid searches within a reasonable subset of the design space and show that the problem is smooth, has only one stationary point, and that the solver converges to the optimal solution even for the simultaneous, non-convex problem formulation. We further present a case study showing that, if an IETMS is used, a realistic bus service life without battery replacement can be achieved with a reduction of energy consumption of up to 7% on some driving missions, compared to a heuristic heating strategy. If the design of the thermal system is co-optimized, battery lifetime can be extended further by up to 15% without affecting the amount of energy consumed. In summary, our study reveals a potential to make electric transportation more efficient in terms of both energy and costs based on a holistic consideration of battery-health-aware IETMS with optimized component dimensioning.

Risk Aware Minimum Principle for Optimal Control of Stochastic Differential Equations

We present a probabilistic formulation of risk aware optimal control problems for stochastic differential equations. Risk awareness is in our framework captured by objective functions in which the risk neutral expectation is replaced by a risk function, a nonlinear functional of random variables that accounts for the controller's risk preferences. We state and prove a risk aware minimum principle that gives necessary and sufficient conditions for optimality of generalized control processes taking values on probability measures defined on a given action space. We show that going from the risk neutral to the risk aware case, the minimum principle is modified by the introduction of one additional real-valued stochastic process that acts as a risk adjustment factor. This adjustment process is explicitly given as the expectation, conditional on the filtration at the given time, of an appropriately defined functional derivative of the risk function evaluated at the random total cost. The control model we employ differs from standard relaxed controls, and we elaborate on the differences, and benefits and drawbacks, of the control types; we further give conditions under which the generalized control can be realized using a strict control process. We present an application of the results for a portfolio allocation problem and show that the risk awareness of the objective function gives rise to a risk premium term that is characterized by the risk adjustment process described above. This suggests uses of our results in e.g. pricing of risk modeled by generic risk functions in financial applications.

Equivalent Formulations of Optimal Control Problems with Maximum Cost and Applications.

We revisit the optimal control problem with maximum cost with the objective to provide different equivalent reformulations suitable to numerical methods. We propose two reformulations in terms of extended Mayer problems with state constraints, and another one in terms of a differential inclusion with upper-semi-continuous right member without state constraint. For the latter we also propose a scheme that approximates from below the optimal value. These approaches are illustrated and discussed in several examples.

Maintenance scheduling optimisation of Reverse Osmosis Networks (RONs) via a multistage Optimal Control reformulation

State-of-the-art approaches for membrane cleaning scheduling have focused on the Mixed-Integer Nonlinear Programming (MINLP) formulation so far, a strategy leading to a combinatorial problem that does not capture accurately the dynamic behaviour of the system. In this work, the Reverse Osmosis (RO) cleaning scheduling problem is solved using a novel approach based on the Multistage Integer Nonlinear Optimal Control Problem (MSINOCP) formulation. The approach produces an automated solution for the membrane cleaning scheduling, which also obviates the need for any form of combinatorial optimisation.Two different simulations, for 26 and 52 periods of operation (each period with a duration of one week), are carried out to illustrate the application of the proposed framework and the total cost is 1.17 and 2.48107€, respectively. The RO network configuration considers 2 stages, each with 3 individual RO modules. The results show evidently that the new proposed solution framework can solve successfully this type of problems, even for large scale configurations, long time horizons and arbitrary realistic complexity of the underlying dynamic model of the RO process considered.

Stochastic optimal well control in subsurface reservoirs using reinforcement learning

We present a case study of model-free reinforcement learning (RL) framework to solve stochastic optimal control for a predefined parameter uncertainty distribution and partially observable system. We focus on robust optimal well control problem which is a subject of intensive research activities in the field of subsurface reservoir management. For this problem, the system is partially observed since the data is only available at well locations. Furthermore, the model parameters are highly uncertain due to sparsity of available field data. In principle, RL algorithms are capable of learning optimal action policies – a map from states to actions – to maximize a numerical reward signal. In deep RL, this mapping from state to action is parameterized using a deep neural network. In the RL formulation of the robust optimal well control problem, the states are represented by saturation and pressure values at well locations while the actions represent the valve openings controlling the flow through wells. The numerical reward refers to the total sweep efficiency and the uncertain model parameter is the subsurface permeability field. The model parameter uncertainties are handled by introducing a domain randomization scheme that exploits cluster analysis on its uncertainty distribution. We present numerical results using two state-of-the-art RL algorithms, proximal policy optimization (PPO) and advantage actor–critic (A2C), on two subsurface flow test cases representing two distinct uncertainty distributions of permeability field. The results were benchmarked against optimization results obtained using differential evolution algorithm. Furthermore, we demonstrate the robustness of the proposed use of RL by evaluating the learned control policy on unseen samples drawn from the parameter uncertainty distribution that were not used during the training process.

Weight metamorphosis of varifolds and the LDDMM-Fisher-Rao metric

This paper introduces and studies a metamorphosis framework for geometric measures known as varifolds, which extends the diffeomorphic registration model for objects such as curves, surfaces and measures by complementing diffeomorphic deformations with a transformation process on the varifold weights. We consider two classes of cost functionals to penalize those combined transformations, in particular the LDDMM-Fisher-Rao energy which, as we show, leads to a well-defined Riemannian metric on the space of varifolds with existence of corresponding geodesics. We further introduce relaxed formulations of the respective optimal control problems, study their well-posedness and derive optimality conditions for the solutions. From these, we propose a numerical approach to compute optimal metamorphoses between discrete varifolds and illustrate the interest of this model in the situation of partially missing data.