Deterministic Optimal Control Research Articles

Deterministic Optimal Control on Riemannian Manifolds Under Probability Knowledge of the Initial Condition

Deterministic Optimal Control on Riemannian Manifolds Under Probability Knowledge of the Initial Condition

Detecting the most probable transition pathway based on optimal control theory

Many natural systems exhibit transitions when external environmental conditions (such as complex noise) spark a shift to a new and sometimes quite different state. Therefore, detecting the most probable transition pathway between metastable states of a stochastic dynamical system is a significant topic. The most probable transition pathway can be treated as the minimizer of the associated Onsager-Machlup action functional. We convert this variational problem for computing the most probable transition pathway into a deterministic optimal control problem. One traditional approach for an optimal control problem is via Pontryagin's Maximum Principle, but it is challenging in high dimensional systems. In this paper, we devise a method to detect the most probable transition pathway for stochastic dynamical systems, by combining Pontryagin's Maximum Principle with a successive approximation scheme and a nested neural network technique. We validate our method with three stochastic dynamical systems, including a double well system, a Maier-Stein system, and a Nutrient-Phytoplankton-Zooplankton system. Specifically, in order to illustrate its effectiveness, we test the error and convergence of our method with an upper bound of training loss. Our work contributes to a better understanding of transition phenomena in complex systems under random fluctuations.

Optimal control of deterministic and stochastic Eco-epidemic food adulteration model

In this paper, we develop a deterministic and stochastic mathematical model to study the effect of contaminated foods on human health by spreading media awareness. A predator–prey SI model with media-induced response function is considered. In the deterministic case, the model has three equilibria: trivial, disease-free, and endemic. While the endemic equilibria exist for low media impact provided R0>1. Here, the trivial equilibrium is always unstable. However, the disease-free equilibrium exhibits local and global asymptotically stable behavior provided that R0<1. The conditions for the local behavior of endemic equilibrium are also determined along with the existence of Hopf bifurcation. Further, the impact of external interference is studied for the stochastic model. The stochastic model has a unique global positive solution which is ultimately bounded. The stochastic model also demonstrates asymptotic behavior around disease-free and endemic equilibria. Additionally, optimal control theory has been employed to explore deterministic and stochastic optimal control systems in depth. Finally, the analytical results are validated using some numerical simulations.

Robust trajectory optimization of re-entry flight with prescribed endpoint region via sparse grid ensemble pseudospectral optimal control

This paper introduces a sampling-based computational framework with a pseudospectral transcription, designed to investigate performance-robustness tradeoffs for constrained re-entry flights with a non-Gaussian uncertainty propagation. Ensemble of trajectories which are generated with sparse grids with Conjugate Unscented Transformation have been integrated into the nonlinear programming via a vectorized and parallelized formulation. A cross-range maximization problem is formulated to assess the influences of uncertainties in entry interface conditions and model parametric uncertainties. A comprehensive analysis, facilitated by rapid gradient-based optimization convergence and warm-starting, highlights the significant effects of initial speed uncertainties and drag coefficient on problem feasibility, offering insights into optimal control behavior under different degrees and combinations of uncertainties. Proper scaling of controls and the objective function proves pivotal for achieving convergence. Furthermore, a novel phenomenon called “contact arcs” is discovered, composed of contact points that form an arc at the edge of the path constraint in the constrained ensemble optimal control as a counterpart to the boundary arcs in deterministic optimal control problems. Simulation results show excellent agreement with the derived optimality conditions.

Optimal Scheduling in General Multi-Queue System by Combining Simulation and Neural Network Techniques

The problem of optimal scheduling in a system with parallel queues and a single server has been extensively studied in queueing theory. However, such systems have mostly been analysed by assuming homogeneous attributes of arrival and service processes, or Markov queueing models were usually assumed in heterogeneous cases. The calculation of the optimal scheduling policy in such a queueing system with switching costs and arbitrary inter-arrival and service time distributions is not a trivial task. In this paper, we propose to combine simulation and neural network techniques to solve this problem. The scheduling in this system is performed by means of a neural network informing the controller at a service completion epoch on a queue index which has to be serviced next. We adapt the simulated annealing algorithm to optimize the weights and the biases of the multi-layer neural network initially trained on some arbitrary heuristic control policy with the aim to minimize the average cost function which in turn can be calculated only via simulation. To verify the quality of the obtained optimal solutions, the optimal scheduling policy was calculated by solving a Markov decision problem formulated for the corresponding Markovian counterpart. The results of numerical analysis show the effectiveness of this approach to find the optimal deterministic control policy for the routing, scheduling or resource allocation in general queueing systems. Moreover, a comparison of the results obtained for different distributions illustrates statistical insensitivity of the optimal scheduling policy to the shape of inter-arrival and service time distributions for the same first moments.

Stochastic discontinuous Galerkin methods for robust deterministic control of convection-diffusion equations with uncertain coefficients

We investigate a numerical behavior of robust deterministic optimal control problem subject to a convection-diffusion equation containing uncertain inputs. Stochastic Galerkin approach, turning the original optimization problem containing uncertainties into a large system of deterministic problems, is applied to discretize the stochastic domain, while a discontinuous Galerkin method is preferred for the spatial discretization due to its better convergence behavior for optimization problems governed by convection dominated PDEs. Error analysis is done for the state and adjoint variables in the energy norm, while the estimate of deterministic control is obtained in the L2-norm. Large matrix system emerging from the stochastic Galerkin method is addressed by the low-rank version of GMRES method, which reduces both the computational complexity and the memory requirements by employing Kronecker-product structure of the obtained linear system. Benchmark examples with and without control constraints are presented to illustrate the efficiency of the proposed methodology.

Trajectory Optimization of Chance-Constrained Nonlinear Stochastic Systems for Motion Planning Under Uncertainty

In this article, we present generalized polynomial chaos-based sequential convex programming (gPC-SCP) to compute a suboptimal solution for a continuous-time chance-constrained stochastic nonlinear optimal control (SNOC) problem. The approach enables motion planning for robotic systems under uncertainty. The gPC-SCP method involves two steps. The first step is to derive a surrogate problem of deterministic nonlinear optimal control (DNOC) with convex constraints by using gPC expansion and the distributionally robust convex subset of the chance constraints. The second step is to solve the DNOC problem using sequential convex programming for trajectory generation and control. We prove that in the unconstrained case, the optimal value of the DNOC converges to that of SNOC asymptotically and that any feasible solution of the constrained DNOC is a feasible solution of the chance-constrained SNOC. We also present the predictor–corrector extension (gPC-SCP <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\text{PC}$</tex-math></inline-formula> ) for real-time motion trajectory generation in the presence of stochastic uncertainty. In the gPC-SCP <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\text{PC}$</tex-math></inline-formula> method, we first predict the uncertainty using the gPC method and then optimize the motion plan to accommodate the uncertainty. We empirically demonstrate the efficacy of the gPC-SCP and the gPC-SCP <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\text{PC}$</tex-math></inline-formula> methods for the following two test cases: first, collision checking under uncertainty in actuation and physical parameters and second, collision checking with stochastic obstacle model for 3DOF and 6DOF robotic systems. We validate the effectiveness of the gPC-SCP method on the 3DOF robotic spacecraft testbed.

Uncertain optimal control problem with the first hitting time objective and application to a portfolio selection model

This paper discusses an uncertain time optimal control problem by considering time efficiency, which is to optimize the objective function about the first hitting time subject to uncertain differential equations. According to the definition of the α-path, the uncertain time optimal control problem is transformed into an equivalent deterministic optimal control problem. Two kinds of time optimal control models are presented where optimistic value and reaching index are chosen as the optimality criteria, respectively. Applying the proposed uncertain optimal control model to a portfolio selection problem, we obtain the uncertainty distribution of the first hitting time (the investors’ first profit time). Meanwhile, sufficient conditions of the optimal control strategy of such models are provided. Numerical simulations are provided which reveal the change for our optimal control strategy.

Deterministic optimal control for discrete-time systems with multiplicative noises and random coefficients

This paper studies a linear-quadratic optimal control problem for discrete-time systems with multiplicative noise and random coefficients. Motivated by fiscal policy planning problems, where deterministic policies are required, this paper aims to find a deterministic optimal controller. The challenge is to seek a solution to forward and backward stochastic difference equations with structural inconsistency caused by random and deterministic terms. We derive necessary and sufficient conditions to solve this optimal control problem by employing a decoupling method. We also present an explicit expression of the optimal controller based on coupled stochastic Riccati-type difference equations.

HIV/AIDS and TB co-infection deterministic model bifurcation and optimal control analysis

HIV/AIDS-TB co-infection is a worldwide common health problem. In this paper a compartmental model with optimal control strategies on the spread of HIV/AIDS-TB co-infection has been developed and analyzed with preventive measure dependent behavioral changes in the population and treatment of infected (and infectious) classes. And the impact of the co-infection and single infection treatment on the spreading dynamics of HIV/AIDS-TB co-infection and single infection were assessed. The two sub-models were analyzed mathematically when each of the two diseases was considered separately and the TB-only system shows backward bifurcation. Optimal control strategy analysis is applied to the full model with the aim to evaluate the impacts of control measures. Various combinations of controls are also analyzed and presented. The numerical simulation section shows that an optimal combination of preventive and therapeutic controls will suppress the occurrence of HIV, TB and HIV/AIDS-TB co-infection. Moreover, the therapeutic controls were found to be more effective than preventive controls for the infected individuals, preventive controls in turn were more effective than therapeutic controls in uninfected individuals.

Least-square Fitting on the Circle using Optimal Control

Filtering of circular data from noisy measurements is well known to be a hard problem because of the ambiguity of the wrapped phase and further complicated by the constraint that filtered states need to be on the circle. Probabilistic models focus on specifying the dynamics of the random phase and then estimate these states recursively using Bayes’ formula, while deterministic approaches normally define a cost function containing the error between the states and measurements and then minimize it over all allowed state paths. In this paper, we construct a deterministic filter on the circle by minimizing the least square error based on Pontryagin's minimum principle, where the optimal state trajectory is described by a bilinear differential equation with deterministic optimal control input. The effectiveness of the proposed filter is shown through numerical experiments.

Optimal control for stochastic differential equations and related Kolmogorov equations

&lt;p style='text-indent:20px;'&gt;This paper concerns a stochastic optimal control problem with feedback Markov inputs. The problem is reduced to a deterministic optimal control problem for a Kolmogorov equation where the control for the deterministic problem is of open-loop type. The existence of an optimal control is proved for the deterministic control problem in a particular case. A maximum principle and some first order necessary optimality conditions are derived. Some examples and comments are discussed.&lt;/p&gt;

An Adaptive Multi-Level Max-Plus Method for Deterministic Optimal Control Problems

We introduce a new numerical method to approximate the solution of a finite horizon deterministic optimal control problem. We exploit two Hamilton-Jacobi-Bellman PDE, arising by considering the dynamics in forward and backward time. This allows us to compute a neighborhood of the set of optimal trajectories, in order to reduce the search space. The solutions of both PDE are successively approximated by max-plus linear combinations of appropriate basis functions, using a hierarchy of finer and finer grids. We show that the sequence of approximate value functions obtained in this way does converge to the viscosity solution of the HJB equation in a neighborhood of optimal trajectories. Then, under certain regularity assumptions, we show that the number of arithmetic operations needed to compute an approximate optimal solution of a d-dimensional problem, up to a precision ε, is bounded by O(Cd|logε|), for some constant C > 1, whereas ordinary grid-based methods have a complexity in O(1/εad) for some constant a > 0.

Optimal programmed control in energy minimisation problem

This paper is devoted to the construction of optimal programmed control of the motion of a material point in a medium with a random coefficient of resistance in a homogeneous field of gravity of the Earth. The control objective is to minimise the mathematical expectation of the total mechanical energy of the material point. The problem is solved by two methods proposed by the author in previous works. Both methods assume approximation of the initial stochastic problem by deterministic optimal control problems. In the first case, the transition to approximate deterministic problems is carried out by replacing the parameters representing continuous random variables by discrete-type random variables converging in distribution to the original continuous ones (the method of distribution discretisation). The meaning of such a replacement is that the resulting approximate problem can be considered no longer as a stochastic, but as a deterministic optimal control problem, and can be solved, accordingly, with the help of known standard methods. In the second case, the initial stochastic dynamic system as a result of averaging its equations is replaced first by an infinite system and then, after zeroing all moments of sufficiently high order, by a finite system of equations for mixed moments of the solution and a random parameter (method of moments). The control functions and the optimal values of the quality functional obtained by the two methods were found to be almost identical, and both methods showed good convergence.

Robust Motion Planning of the Powered Descent of a Space Vehicle

The motion planning of powered descent problems has often been treated in the deterministic optimal control framework, which provides efficient theoretical and numerical tools. However, future applications require robustness, usually obtained by introducing stochastic components in the dynamics to model uncertainties. After stating the robust motion planning problem, this paper proposes a deterministic approximation which avoids the computational difficulties of stochastic optimal control. The approach consists of guiding the mean while reducing the covariance, the dynamics of these two quantities being approximated thanks to statistical linearization. In addition, since feedback control is necessary to control covariance, two techniques are provided to deal with actuator limits when the control is stochastic.

Solution to Forward–Backward Stochastic Differential Equations With Random Coefficients and Application to Deterministic Optimal Control

We address forward–backward stochastic differential equations (FBSDEs) with random coefficients. Differently from previous approaches, we consider FBSDEs with coefficients that are random and correlated with forward and backward variables. In addition, expectation terms appear in the forward equation, and this makes challenging to solve coupled FBSDEs, e.g., those coming from corporations’ fiscal policy planning. Using a discretization approach, an analytic solution to the FBSDEs is given in terms of coupled stochastic Riccati-type differential equations. We also apply our results to optimal deterministic control problem for Itô stochastic systems with both random and deterministic coefficients. Our results show that the discretization approach is a powerful tool to decouple complex FBSDEs.

Deterministic and stochastic optimal control models for plant growth using locust fertilizer

Deterministic and stochastic optimal control models for plant growth using locust fertilizer

Linear programming based optimality conditions and approximate solution of a deterministic periodic optimal control problem in discrete time

This paper is focused on a discrete time periodic optimal control problem. We establish linear programming based optimality conditions and present a method for the construction of near optimal controls. A numerical example is included to demonstrate the results.

Reliability-based optimal control design for seismic-excited structures: A hybrid IS-MTLBO pseudo-double loop method

This study addressed a reliability-based optimal control design (RBOCD) approach for seismic-excited structures. For this aim, a hybrid importance sampling-modified teaching-learning based optimization (IS-MTLBO) pseudo-double loop method is developed. Uncertainty sources such as structural model, stochastic model of the ground acceleration, structural response measurements in sensors, and designed/applied forces in actuators are considered in this study. Numerical studies are carried out on a 10-story building subjected to earthquake excited. A deterministic optimal control design (DOCD) approach based on the MTLBO algorithm is firstly conducted for the structure. Then, a sensitivity analysis using the Borgonovo sensitivity method is carried out to identify and measure the effectiveness of the mentioned uncertainty sources on the seismic responses. Finally, the IS-MTLBO pseudo-double loop approach as an RBOCD approach is implemented for the studied structure. The average reductions of the seismic responses in terms of maximum displacement, acceleration, and drift of floors are obtained as 18.95%, 27.7%, and 24.65% for the RBOCD approach, while, the corresponding reduction are 7.30%, 19.45%, and 14.05% for the DOCD approach. It is shown that although the designed controller based on the DOCD approach satisfies the limitation considered for the maximum floor drift in the nominal model, it is not able to satisfy it in the presence of uncertainties. In other words, the RBOCD approach represents a robust control design against uncertainties, while the DOCD approach gives an uncertainty-sensitive control scheme. Considering four well-known earthquakes, the failure probabilities of the structure controlled by both DOCD and RBOCD approaches are compared. The results indicate the failure probability of the structure controlled by the RBOCD approach is much less than those given using the DOCD approach.

An approximate stochastic optimal control framework to simulate nonlinear neuro-musculoskeletal models in the presence of noise.

Optimal control simulations have shown that both musculoskeletal dynamics and physiological noise are important determinants of movement. However, due to the limited efficiency of available computational tools, deterministic simulations of movement focus on accurately modelling the musculoskeletal system while neglecting physiological noise, and stochastic simulations account for noise while simplifying the dynamics. We took advantage of recent approaches where stochastic optimal control problems are approximated using deterministic optimal control problems, which can be solved efficiently using direct collocation. We were thus able to extend predictions of stochastic optimal control as a theory of motor coordination to include muscle coordination and movement patterns emerging from non-linear musculoskeletal dynamics. In stochastic optimal control simulations of human standing balance, we demonstrated that the inclusion of muscle dynamics can predict muscle co-contraction as minimal effort strategy that complements sensorimotor feedback control in the presence of sensory noise. In simulations of reaching, we demonstrated that nonlinear multi-segment musculoskeletal dynamics enables complex perturbed and unperturbed reach trajectories under a variety of task conditions to be predicted. In both behaviors, we demonstrated how interactions between task constraint, sensory noise, and the intrinsic properties of muscle influence optimal muscle coordination patterns, including muscle co-contraction, and the resulting movement trajectories. Our approach enables a true minimum effort solution to be identified as task constraints, such as movement accuracy, can be explicitly imposed, rather than being approximated using penalty terms in the cost function. Our approximate stochastic optimal control framework predicts complex features, not captured by previous simulation approaches, providing a generalizable and valuable tool to study how musculoskeletal dynamics and physiological noise may alter neural control of movement in both healthy and pathological movements.