Formulation Of Control Problem Research Articles

Stochastic Framework for Optimal Control of Planetary Reentry Trajectories Under Multilevel Uncertainties

We present a novel stochastic optimal control framework that accounts for various types of uncertainties, with application to reentry trajectory planning. The formulation of the optimal trajectory control problem is presented in the context of an indirect method where a functional objective associated with the terminal vehicle speed is to be minimized. Uncertain input parameters in the optimal trajectory control model, including aerodynamic parameters and initial and terminal conditions, are modeled as aleatory random variables, while the statistical parameters of these aleatory distributions are themselves random variables. The parametric and model uncertainties are simultaneously propagated through an extended polynomial chaos expansion (EPCE) formalism. Several metrics are described to evaluate response statistics and presented as insightful tools for robust decision making. Specifically, the response probability density function (PDF) reflecting influence of both epistemic and aleatory uncertainties is obtained. By sampling over the random variables representing model error, an ensemble of response PDFs is generated and the associated failure probability is estimated as a random variable with its own polynomial chaos expansion. Besides, the sensitivity index functions of response PDF with respect to the statistical parameters are evaluated. Coupling parametric and model uncertainties within the EPCE framework leads to a robust and efficient paradigm for multilevel uncertainty propagation and PDF characterization in general optimal control problems.

Existence of solutions for an optimal control problem in forestry management

Physical, biological, and economic factors remain inexhaustible sources when determining silvicultural management strategies that maximize the net benefit of timber volume. The objective of this paper is to formulate an optimal control problem to maximize the net benefit of timber volume and to demonstrate the existence of optimal solutions to this problem. The optimal control theory will be used, through the formulation of a bioeconomic optimal control problem subjected to a system of three ordinary differential equations that describe the interactions between the living biomass, the intrinsic growth of the biomass, and the burned area. Four control variables are associated with these differential equations, which are reforestation, felling, thinning, and fire prevention. Assuming adequate conditions on the control variables, the existence of optimal solutions to the optimal control problem is obtained using Filippov’s theorem. In conclusion, the optimal control problem is well formulated and solutions exist. This will allow us to define forest management strategies that optimize the economic gain of the process, taking into consideration the physical, biological, and economic phenomena involved in the dynamics of forest plantations.

A Cartesian-Based Trajectory Optimization with Jerk Constraints for a Robot

To address the time-optimal trajectory planning (TOTP) problem with joint jerk constraints in a Cartesian coordinate system, we propose a time-optimal path-parameterization (TOPP) algorithm based on nonlinear optimization. The key insight of our approach is the presentation of a comprehensive and effective iterative optimization framework for solving the optimal control problem (OCP) formulation of the TOTP problem in the (s,s˙)-phase plane. In particular, we identify two major difficulties: establishing TOPP in Cartesian space satisfying third-order constraints in joint space, and finding an efficient computational solution to TOPP, which includes nonlinear constraints. Experimental results demonstrate that the proposed method is an effective solution for time-optimal trajectory planning with joint jerk limits, and can be applied to a wide range of robotic systems.

Moment-Based Model Predictive Control of Autonomous Systems

Great efforts have been devoted to the intelligent control of autonomous systems. Yet, most of existing methods fail to effectively handle the uncertainty of their environment and models. Uncertain locations of dynamic obstacles pose a major challenge for their optimal control and safety, while their linearization or simplified system models reduce their actual performance. To address them, this paper presents a new model predictive control framework with finite samples and a Gaussian model, resulting in a chance-constrained program. Its nominal model is combined with a Gaussian process. Its residual model uncertainty is learned. The resulting method addresses an efficiently solvable approximate formulation of a stochastic optimal control problem by using approximations for efficient computation. There is no perfect distribution knowledge of a dynamic obstacle's location uncertainty. Only finite samples from sensors or past data are available for moment estimation. We use the uncertainty propagation of a system's state and obstacles' locations to derive a general collision avoidance condition under tight concentration bounds on the error of the estimated moments. Thus, this condition is suitable for different obstacles, e.g., bounding box and ellipsoid obstacles. We provide proved guarantees on the satisfaction of the chance-constraints corresponding to the nominal yet unknown moments. Simulation examples of a vehicle's control are used to show that the proposed method can well realize autonomous control and obstacle avoidance of a vehicle, when it operates in an uncertain environment with moving obstacles. It outperforms the existing moment methods in both performance and computational time.

Mean field games with branching

Mean field games are concerned with the limit of large-population stochastic differential games where the agents interact through their empirical distribution. In the classical setting, the number of players is large but fixed throughout the game. However, in various applications, such as population dynamics or economic growth, the number of players can vary across time and this may lead to different Nash equilibria. In order to account for this evolution, we introduce a branching mechanism in the population of agents and obtain a variant of the original mean field game problem. As a first step, we study a simple model using a PDE approach to illustrate the main differences with the classical setting. We prove existence of a solution and show that it provides an approximate Nash-equilibrium for large population games. We also present a numerical example for a linear–quadratic model. Then we study the problem in a general setting by a probabilistic approach. It is based upon the relaxed formulation of stochastic control problems which allows us to obtain a general existence result.

Modern methods for solving the optimal control problem in economics

The modern formulation of the optimal control problem is given, following which a survey of the methods currently applied in solving the optimal control problem in economics is conducted, with a focus given to numerical methods. The most important problems in the application of numerical methods in solving the optimal control problem are given and explained. The article then lists and explains the most common computational methods of solving the optimal control problem that are being applied in todays economic sphere, how far these methods go in terms of achieving their objectives and providing solutions, the difficulties in implementing these methods, and the associated limitations. The latest developments in software and programs that are beginning to be used by mainly economists in solving some of the most common optimal control problems are listed, and their advantages and limitations are explained. Lastly, a general analysis of some of the next-generation methods and the future of the optimal control problem in general is made.

Data-driven optimal control via linear transfer operators: A convex approach

This paper is concerned with the data-driven optimal control of nonlinear systems. We present a convex formulation of the optimal control problem with a discounted cost function. We consider optimal control problems with both positive and negative discount factors. The convex approach relies on lifting nonlinear system dynamics in the space of densities using the linear Perron–Frobenius operator. This lifting leads to an infinite-dimensional convex optimization formulation of the optimal control problem. The data-driven approximation of the optimization problem relies on the approximation of the Koopman operator and its dual: the Perron–Frobenius operator, using a polynomial basis function. We write the approximate finite-dimensional optimization problem as a polynomial optimization which is then solved efficiently using a sum-of-squares-based optimization framework. Simulation results demonstrate the efficacy of the developed data-driven optimal control framework.

A Distributed Mixed-Integer Framework to Stochastic Optimal Microgrid Control

This article deals with distributed control of microgrids composed of storages, generators, renewable energy sources, and critical and controllable loads. We consider a stochastic formulation of the optimal control problem associated with the microgrid that appropriately takes into account the unpredictable nature of the power generated by renewables. The resulting problem is a mixed-integer linear program and is NP-hard and nonconvex. Moreover, the peculiarity of the considered framework is that no central unit can be used to perform the optimization, but rather the units must cooperate with each other by means of neighboring communication. To solve the problem, we resort to a distributed methodology based on a primal decomposition approach. The resulting algorithm is able to compute high-quality feasible solutions to a two-stage stochastic optimization problem, for which we also provide a theoretical upper bound on the constraint violation. Finally, a Monte Carlo numerical computation on a scenario with a large number of devices shows the efficacy of the proposed distributed control approach. The numerical experiments are performed on realistic scenarios obtained from Generative Adversarial Networks (GANs) trained an open-source historical dataset of the EU.

A Resource Allocation Algorithm for Formation Control of Connected Vehicles

A formation control problem is considered for a network of connected vehicles communicating over a 5G C-V2X communication system with limited network resources. To solve this problem, we propose a new combined control - resource allocation scheme to make the vehicles achieve the targeted formation. We concentrate on the formulation of the formation control problem under the constraints imposed by the communication system. We state the resource allocation problem and present an optimization algorithm to select the transmitting agents. The proposed algorithm allows to assign the network resources in a centralized way to ensure the convergence towards the desired formation while preserving the network connectivity.

Model Predictive Control for Energy-Efficient Yaw-Stabilizing Torque Vectoring in Electric Vehicles With Four In-Wheel Motors

This paper considers the problem of stabilizing and energy-efficient torque vectoring for electric vehicles with four independent in-wheel motors. In electric vehicles with four in-wheel motors, four electric motors are separately attached to the four wheels without an extra drive shaft. The mechanical and structural nature enables reduction of energy loss during power transmission and securing extra interior space. In addition, independent control of wheel torques can provide better yaw motion stability and improved energy efficiency. This paper proposes two model predictive control (MPC) methods for stability-constrained energy-efficient torque vectoring of four in-wheel motor electric vehicles. For the adaptive weighting factors of multiple objective functions of reference tracking and energy saving, we use exponential functions that vary with the lateral motion and steering input. Depending on the optimal control problem formulation with different dynamical system equations and constraints, the associated predictive controller can be represented as either a linear parameter-varying MPC (LPV-MPC) or nonlinear MPC (NMPC). For LPV-MPC, longitudinal and lateral motions are decoupled, whereas the coupled dynamics of the two-track model are exploited in NMPC. For comparisons and demonstrations of LPV-MPC and NMPC in the MPC of torque vectoring, three driving scenarios are simulated with a high-fidelity vehicle simulation solution, CarMaker (IPG Automotive). In comparison with the built-in IPG driver implemented in CarMaker, we demonstrate fuel efficiency improvements of over 2–3 % on average with guaranteed yaw stability.

Tuning of subspace predictive controls

Data-driven predictive control has recently gained increasing attention, as it makes it possible to design constrained controls directly from a set of data, without requiring an intermediate identification step. In this paper, we focus on a Subspace Predictive Control (SPC) scheme, with the aim of clarifying the sensitivity of the final closed-loop performance to its main hyperparameters, namely the length of the past horizon and the regularization penalties. Moreover, by delving deep into the structural properties of the control problem formulation, we provide a set of guidelines for the choice of such hyperparameters. The effectiveness of the resulting overall tuning strategy is assessed on two benchmark examples.

Multiobjective strict dissipativity via a weighted sum approach

We consider nonlinear model predictive control (MPC) with multiple competing cost functions. This leads to the formulation of multiobjective optimal control problems (MO OCPs). Since the design of MPC algorithms for directly solving multiobjective problems is rather complicated, particularly if terminal conditions shall be avoided, we use an indirect approach via a weighted sum formulation for solving these MO OCPs. This way, for each set of weights we obtain an optimal control problem with a single objective. In economic MPC it is known that strict dissipativity is the key assumption for concluding performance and stability results. We thus investigate under which conditions a convex combination of strictly dissipative stage costs is strictly dissipative again. We first give conditions for problems with linear dynamics and then move on to consider fully nonlinear optimal control problems. We derive both necessary and sufficient conditions on the individual cost functions and on the weights to conclude strict dissipativity and illustrate our findings with numerical examples.

Optimal Joint Power and Rate Adaptation for Awareness and Congestion Control in Vehicular Networks

To support inter-vehicular applications, vehicles broadcast beacons with information about their state. Congestion may occur when the load on the channel due to beaconing can prevent the transmission of other types of messages. In this paper, we propose a distributed algorithm for optimal joint adaptation of transmit power and beaconing rate for congestion and awareness control. Our approach is based on a network utility formulation of the congestion control problem, which allows us to induce a desired fairness notion and set different priorities for vehicles. We formulate the general problem but, since it is not convex, we assume Rayleigh fading and derive an algorithm, called PRAIOS, that solves the optimization problem in a decentralized way with convergence guarantees. Our results, validated with realistic simulations in both static and dynamic scenarios and compared with other proposals, show that PRAIOS quickly converges to close to optimal allocations, while keeping the maximum beaconing load at the desired level. They also show that it can control the load when the fading is not Rayleigh. Applications can dynamically set their requirements as constraints, that are enforced by the algorithm while complying with the maximum load, which allows seamless integration of operational requirements into the control framework.

Battery lifetime extension through optimal design and control of traction and heating systems in hybrid drivetrains

In this paper, we investigate the optimal design and control of an integrated energy and thermal management system (IETMS) of a battery-assisted trolley bus that is subject to minimum battery lifetime requirements. Therefore, we jointly optimize both the control trajectories of the traction and the heating systems and the design of the thermal energy buffer. We further analyze the resulting Pareto fronts which characterize the trade-off between the battery lifetime and the energy consumption. This holistic approach fills a gap in the literature published, namely the formulation of an optimization problem that combines component sizing with a battery-health-aware IETMS. While the model derived allows the formulation of a convex optimal control problem for a given vehicle design, the combined design and control problem is non-convex. Therefore, we perform grid searches within a reasonable subset of the design space and show that the problem is smooth, has only one stationary point, and that the solver converges to the optimal solution even for the simultaneous, non-convex problem formulation. We further present a case study showing that, if an IETMS is used, a realistic bus service life without battery replacement can be achieved with a reduction of energy consumption of up to 7% on some driving missions, compared to a heuristic heating strategy. If the design of the thermal system is co-optimized, battery lifetime can be extended further by up to 15% without affecting the amount of energy consumed. In summary, our study reveals a potential to make electric transportation more efficient in terms of both energy and costs based on a holistic consideration of battery-health-aware IETMS with optimized component dimensioning.

Risk Aware Minimum Principle for Optimal Control of Stochastic Differential Equations

We present a probabilistic formulation of risk aware optimal control problems for stochastic differential equations. Risk awareness is in our framework captured by objective functions in which the risk neutral expectation is replaced by a risk function, a nonlinear functional of random variables that accounts for the controller's risk preferences. We state and prove a risk aware minimum principle that gives necessary and sufficient conditions for optimality of generalized control processes taking values on probability measures defined on a given action space. We show that going from the risk neutral to the risk aware case, the minimum principle is modified by the introduction of one additional real-valued stochastic process that acts as a risk adjustment factor. This adjustment process is explicitly given as the expectation, conditional on the filtration at the given time, of an appropriately defined functional derivative of the risk function evaluated at the random total cost. The control model we employ differs from standard relaxed controls, and we elaborate on the differences, and benefits and drawbacks, of the control types; we further give conditions under which the generalized control can be realized using a strict control process. We present an application of the results for a portfolio allocation problem and show that the risk awareness of the objective function gives rise to a risk premium term that is characterized by the risk adjustment process described above. This suggests uses of our results in e.g. pricing of risk modeled by generic risk functions in financial applications.

An operator theoretic approach to linear ensemble control

In this paper, we revisit the original least squares formulation of the ensemble control problem. Based on new considerations, we are able to resolve the longstanding problem of formulating general, and easily verifiable conditions for an ensemble to be controllable in the least-squares sense. The key is to take a purely operator theoretic approach from the very beginning, i.e. to study the ensemble control problem by virtue of the associated Fredholm integral equation. By establishing a direct connection to a recently introduced moment-based approach, the theoretical question of least-squares ensemble controllability is eventually settled. In the second part, we take the very same integral operator theoretic approach to consider the equally longstanding problem of synthesizing inputs that realize a desired steering between two ensemble states. By means of a suitable discretization of the integral operator, we obtain a computational procedure to synthesize control signals that steer ensembles in both a robust and minimum energy fashion. This yields a unified framework for the ensemble control of linear systems.

Maintenance scheduling optimisation of Reverse Osmosis Networks (RONs) via a multistage Optimal Control reformulation

State-of-the-art approaches for membrane cleaning scheduling have focused on the Mixed-Integer Nonlinear Programming (MINLP) formulation so far, a strategy leading to a combinatorial problem that does not capture accurately the dynamic behaviour of the system. In this work, the Reverse Osmosis (RO) cleaning scheduling problem is solved using a novel approach based on the Multistage Integer Nonlinear Optimal Control Problem (MSINOCP) formulation. The approach produces an automated solution for the membrane cleaning scheduling, which also obviates the need for any form of combinatorial optimisation.Two different simulations, for 26 and 52 periods of operation (each period with a duration of one week), are carried out to illustrate the application of the proposed framework and the total cost is 1.17 and 2.48107€, respectively. The RO network configuration considers 2 stages, each with 3 individual RO modules. The results show evidently that the new proposed solution framework can solve successfully this type of problems, even for large scale configurations, long time horizons and arbitrary realistic complexity of the underlying dynamic model of the RO process considered.

Space-time formulation, discretization, and computational performance studies for phase-field fracture optimal control problems

The purpose of this work is the development of space-time discretization schemes for phase-field optimal control problems. Specifically in the optimal control minimization problem, a tracking-type cost functional is minimized to steer the crack via the phase-field variable into a desired pattern. To achieve such optimal solutions, Neumann type boundary conditions need to be determined. First, a time discretization of the forward problem is derived using a discontinuous Galerkin formulation. Here, a challenge is to include regularization terms and the crack irreversibility constraint. The optimal control setting is formulated by means of the Lagrangian approach from which the primal part, adjoint, tangent and adjoint Hessian are derived. Herein the overall Newton algorithm is based on a reduced approach by eliminating the state constraint, namely the displacement and phase-field unknowns, but keeping the control variable as the only unknown. From the low-order discontinuous Galerkin discretization, adjoint time-stepping schemes are finally obtained. Both our formulation and algorithmic developments are substantiated and illustrated with six numerical experiments.

Stochastic optimal well control in subsurface reservoirs using reinforcement learning

We present a case study of model-free reinforcement learning (RL) framework to solve stochastic optimal control for a predefined parameter uncertainty distribution and partially observable system. We focus on robust optimal well control problem which is a subject of intensive research activities in the field of subsurface reservoir management. For this problem, the system is partially observed since the data is only available at well locations. Furthermore, the model parameters are highly uncertain due to sparsity of available field data. In principle, RL algorithms are capable of learning optimal action policies – a map from states to actions – to maximize a numerical reward signal. In deep RL, this mapping from state to action is parameterized using a deep neural network. In the RL formulation of the robust optimal well control problem, the states are represented by saturation and pressure values at well locations while the actions represent the valve openings controlling the flow through wells. The numerical reward refers to the total sweep efficiency and the uncertain model parameter is the subsurface permeability field. The model parameter uncertainties are handled by introducing a domain randomization scheme that exploits cluster analysis on its uncertainty distribution. We present numerical results using two state-of-the-art RL algorithms, proximal policy optimization (PPO) and advantage actor–critic (A2C), on two subsurface flow test cases representing two distinct uncertainty distributions of permeability field. The results were benchmarked against optimization results obtained using differential evolution algorithm. Furthermore, we demonstrate the robustness of the proposed use of RL by evaluating the learned control policy on unseen samples drawn from the parameter uncertainty distribution that were not used during the training process.

Unerroric of control of mutual compliance of the efficiency of hydrogen engines of unmanned vehicles in the conditions of mass production

The issues related to the reliability of hydrogen engines of unmanned vehicles and increasing the efficiency of using hydrogen as fuel when using the method of its production during the decomposition of hydrogen-containing molecules of liquid-phase organic compounds in a plasma discharge under the action of intense ultrasonic exposure are considered. Experiments have shown that as a result of decomposition in the acoustoplasma discharge of liquid hydrocarbons, solid-phase carbon-containing products are formed, chemical transformations occur in the liquid phase and hydrogen-containing combustible gas is formed. Hydrogen-containing gas can be used as fuel immediately after synthesis, i.e. it does not require separation, since in addition to hydrogen it contains only impurities of CO2 and water vapor. The purpose of the study is to formalize the basic conditions for tightening the control of mutual compliance with the efficiency of hydrogen engines of the same series in the conditions of their mass production. Methods of mathematical statistics and hardware-software modeling were used in the study. The term “unerroric of quality mutual compliance control” is introduced to describe a set of hardware and software tools for such control. The principle of in-depth testing of the technical condition of such engines of one series is described in a multidimensional formulation of the quality control problem for three of their operating parameters at once. The conditions for increasing the mutual correspondence of the measured values of such parameters in the conditions of serial production of hydrogen engines are formalized.