Principle For Optimal Control Research Articles

Dynamic programming principle for optimal control of uncertain random differential equations and its application to optimal portfolio selection

This study aimed to examine an uncertain stochastic optimal control problem premised on an uncertain stochastic process. The proposed approach is used to solve an optimal portfolio selection problem. This paper’s research is relevant because it outlines the procedure for solving optimal control problems in uncertain random environments. We implement Bellman’s principle of optimality method in dynamic programming to derive the principle of optimality. Then the resulting Hamilton-Jacobi-Bellman equation (the equation of optimality in uncertain stochastic optimal control) is used to solve a proposed portfolio selection problem. The results of this study show that the dynamic programming principle for optimal control of uncertain stochastic differential equations can be applied in optimal portfolio selection. Also, the study results indicate that the optimal fraction of investment is independent of wealth. The main conclusion of this study is that, in Itô-Liu financial markets, the dynamic programming principle for optimal control of uncertain stochastic differential equations can be applied in solving the optimal portfolio selection problem.

Design of sliding mode controller for servo feed system based on generalized extended state observer with reinforcement learning

Nonlinear friction, system uncertainty, and external disturbances have a significant impact on the performance of high-precision servo feed systems. In order to achieve higher tracking accuracy, a sliding mode controller based on a generalized extended state observer with the double critic deep deterministic policy gradient algorithm is designed. Firstly, a flexible two mass drive model (FTMDM) is established for the two-axis differential micro-feed system (TDMS). Next, a generalized extended state observer (GESO) is designed to estimate matched interference and mismatched interference. And it is proved that the observation error of GESO is bounded. A sliding mode controller based on GESO is further proposed. The stability of the controller has been proven through Lyapunov theory, and the error is bounded and converges to zero in finite time. The tuning process of controller parameters is simplified by using quadratic optimal control principle. Furthermore, a double critic deep deterministic policy gradient algorithm (DCDDPG) is proposed to achieve dynamic optimization of parameters about GESO. The simulation results show that GESO with DCDDPG can reduce the observation error of step signal and sinusoidal signal, and improve the observation accuracy of nonlinear friction significantly. Finally, experimental results show that the proposed control method achieves more accurate position tracking performance on TDMS.

Dynamic Analysis and Optimal Control of a Fractional Order Fishery Model with Refuge and Protected Area

In this paper, a mathematical analysis of fractional order fishery model with stage structure for predator is carried out under the background of prey refuge and protected area. First, it is demonstrated that the solution exists and is unique. The paper aims to analyze predator-prey dynamics in a fishery model through the application of fractional derivatives. It is worth emphasizing that we explicitly examine how fractional derivatives affect the dynamics of the model. The existence of each equilibrium point and the stability of the system at the equilibrium point are proved. The theoretical results are proved by numerical simulation. Alternatively, allocate harvesting efforts within an improved model aimed at maximizing economic benefits and ecologically sustainable development. The ideal solution is obtained by applying Pontryagin’s optimal control principle. A large number of numerical simulations show that the optimal control scheme can realize the sustainable development of the ecosystem.

A hierarchical frequency stability control strategy for distributed energy resources based on ADMM

With the large-scale distributed energy resources (DERs) interfaced by power electronic converters connected to the power grid, the traditional synchronous generation is gradually replaced, and the inertia and primary frequency regulation capability of the power system is weakened, which seriously threatens the frequency security of the power grid. In view of the above problems, a hierarchical control strategy by coordinating the distributed wind turbine (WT) and photovoltaic (PV) is presented to improve the transient frequency (TF) and steady-state frequency (SSF). For the system control level, the optimal frequency regulation coefficients of multiple WTs and PVs clustering systems are determined by the optimal control principle with minimum frequency regulation energy, while considering the constraints of TF and SSF. For the WT/PV control level, the frequency regulation coefficients of WTs and PVs are solved by the model predictive controller (MPC) and alternating direction method of multipliers (ADMM), with the minimum control cost of WT and PV. It achieves the optimal allocation of frequency regulation power and improves the computing efficiency. Finally, the validity of the proposed method is tested in the modified IEEE 10-machine 39-bus system.

Применение принципа Понтрягина для оптимального отключения энергетического ядерного реактора

The theory of optimal control is based on two approaches, the dynamic programming method (Bellman equation) and Pontryagin maximum principle. Pontryagin maximum principle is applied in the physics of nuclear reactors when optimizing various transient processes. The mathematical justification of this theory is based on the elements of convex analysis, which is not used by physicists and engineers, so the physical issues are less studied in scientific literature. The subject of the study is a nuclear power reactor of the WWER type. The problem of minimizing its shutdown time, bypassing the iodine pit is solved and it is possible to start up the reactor at any moment after its shutdown. Analytical and numerical methods are used. The paper considers an example of applying the maximum principle for optimal control of the process of shutting down a nuclear reactor bypassing the iodine pit. A physical mathematical model of the Pontryagin principle is formulated. The process of optimal control of reactor shutdown for large and small reactivity margins is justified and calculated. Pontryagin principle does not contain an algorithm to find an optimization process; the stages of the process must be selected based on physical considerations, but these stages must satisfy the specified principle. Based on the Pontryagin principle, the results of the study make it possible to draw up a step-by-step action plan when shutting down a WWER-type reactor with any value of the reactivity margin and its switching is possible at any time after the transition process, which avoids downtime. The proposed plan can be used both in mathematical modeling of transient processes in a reactor and in reactor control systems to improve its controllability and, consequently, to improve safety.

Realistic 3D human saccades generated by a 6-DOF biomimetic robotic eye under optimal control.

We recently developed a biomimetic robotic eye with six independent tendons, each controlled by their own rotatory motor, and with insertions on the eye ball that faithfully mimic the biomechanics of the human eye. We constructed an accurate physical computational model of this system, and learned to control its nonlinear dynamics by optimising a cost that penalised saccade inaccuracy, movement duration, and total energy expenditure of the motors. To speed up the calculations, the physical simulator was approximated by a recurrent neural network (NARX). We showed that the system can produce realistic eye movements that closely resemble human saccades in all directions: their nonlinear main-sequence dynamics (amplitude-peak eye velocity and duration relationships), cross-coupling of the horizontal and vertical movement components leading to approximately straight saccade trajectories, and the 3D kinematics that restrict 3D eye orientations to a plane (Listing's law). Interestingly, the control algorithm had organised the motors into appropriate agonist-antagonist muscle pairs, and the motor signals for the eye resembled the well-known pulse-step characteristics that have been reported for monkey motoneuronal activity. We here fully analyse the eye-movement properties produced by the computational model across the entire oculomotor range and the underlying control signals. We argue that our system may shed new light on the neural control signals and their couplings within the final neural pathways of the primate oculomotor system, and that an optimal control principle may account for a wide variety of oculomotor behaviours. The generated data are publicly available at https://data.ru.nl/collections/di/dcn/DSC_626870_0003_600.

Research on an Intelligent Vehicle Trajectory Tracking Method Based on Optimal Control Theory

This study aims to explore an intelligent vehicle trajectory tracking control method based on optimal control theory. Considering the limitations of existing control strategies in dealing with signal delays and communication lags, a control strategy combining an anthropomorphic forward-looking reference path and longitudinal velocity closure is proposed to improve the accuracy and stability of intelligent vehicle trajectory tracking. Firstly, according to the vehicle dynamic error tracking model, a linear quadratic regulator (LQR) transverse controller is designed based on the optimal control principle, and a feedforward control strategy is added to reduce the system steady-state error. Secondly, an anthropomorphic look-ahead prediction model is established to mimic human driving behavior to compensate for the signal lag. The double proportional–integral–derivative (DPID) control algorithm is used to track the longitudinal speed reference value. Finally, a joint simulation is conducted based on MatLab/Simulink2021b and CarSim2019.0 software, and the effectiveness of the control strategy proposed in this paper is verified by constructing a semi-physical experimental platform and carrying out a hardware-in-the-loop test. The simulation and test results show that the control strategy can significantly improve the accuracy and stability of vehicle path tracking, which provides a new idea for future intelligent vehicle control system design.

Enhancing stance robustness and jump height in bipedal muscle-actuated systems: a bioinspired morphological development approach.

Recognizing humans' unmatched robustness, adaptability, and learning abilities across anthropomorphic movements compared to robots, we find inspiration in the simultaneous development of both morphology and cognition observed in humans. We utilize optimal control principles to train a muscle-actuated human model for both balance and squat jump tasks in simulation. Morphological development is introduced through abrupt transitions from a 4 year-old to a 12 year-old morphology, ultimately shifting to an adult morphology. We create two versions of the 4 year-old and 12 year-old models- one emulating human ontogenetic development and another uniformly scaling segment lengths and related parameters. Our results show that both morphological development strategies outperform the non-development path, showcasing enhanced robustness to perturbations in the balance task and increased jump height in the squat jump task. Our findings challenge existing research as they reveal that starting with initial robot designs that do not inherently facilitate learning and incorporating abrupt changes in their morphology can still lead to improved results, provided these morphological adaptations draw inspiration from biological principles.

Derivation of Dirac equation from the stochastic optimal control principles of quantum mechanics.

In this paper, we present a stochastic approach to relativistic quantum mechanics. We formulate the three fundamental principles of this theory and derive the Dirac equations based on them. This approach enables us to bring more insight into the nature of Dirac's spinors. Furthermore, we provide a physical interpretation of the stochastic optimal control theory of quantum mechanics.

The maximum principle for optimal control of mean‐field FBSDE driving by Teugels martingales with terminal state constraints

AbstractThis article studies the problem of optimal control with state constraints for mean‐field type stochastic systems, which is governed by a fully coupled forward‐backward stochastic differential equation with Teugels martingales. In this system, the coefficients contain not only the state processes but also its expectation value, and the cost function is of mean‐field type as well. We use an equivalent backward formulation to deal with the terminal state constraint, and then we obtain a stochastic maximum principle by Ekeland's variational principle. In addition, we discuss a stochastic linear‐quadratic control problem with state constraints.

Deep L1 Stochastic Optimal Control Policies for Planetary Soft Landing

In this paper, a novel deep-learning-based solution is introduced to the Powered Descent Guidance problem, grounded in principles of nonlinear Stochastic Optimal Control (SOC) and Feynman–Kac theory. Our algorithm solves the problem by framing it as an L1-SOC problem for minimum fuel consumption. Additionally, it can handle practically useful control constraints and nonlinear dynamics and enforces state constraints as soft constraints. This is achieved by building off of recent work on deep forward-backward stochastic differential equations and differentiable neural-network layers for nonconvex optimization based on stochastic search. In contrast to previous approaches, our algorithm does not require convexification of the constraints or linearization of the dynamics and is empirically shown to be robust to stochastic disturbances and the initial conditions of the spacecraft. After training offline, our control policy can be activated once the spacecraft is within a prespecified radius of the landing zone and at a prespecified altitude, in other words, the base of an inverted cone with the tip at the landing zone. We demonstrate empirically that our controller can successfully and safely land all trajectories initialized in the vicinity of the base of this cone as well as with randomization in the starting velocities and spacecraft mass while minimizing fuel consumption.

The Global Maximum Principle for Optimal Control of Partially Observed Stochastic Systems Driven by Fractional Brownian Motion

The Global Maximum Principle for Optimal Control of Partially Observed Stochastic Systems Driven by Fractional Brownian Motion

Separation Principle for Partially-Observed Linear-Quadratic Optimal Control for Mean-Field Type Stochastic Systems

Separation Principle for Partially-Observed Linear-Quadratic Optimal Control for Mean-Field Type Stochastic Systems

Optimized path planning and scheduling strategies for connected and automated vehicles at single-lane roundabouts.

This paper focuses on the cooperative driving challenges of connected and automated vehicles (CAVs) at single-lane roundabouts. First, a geometric path planning method is proposed for CAVs navigating a single-lane roundabout. Based on this method, a vehicle roundabout model is established. Four potential traffic scenarios for CAVs are established, and the optimal arrival times at conflict points are analyzed. By correlating the optimal arrival times at conflict points with the optimal entry times into the roundabout, the multi-vehicle coordination problem in complex intersections is simplified to a speed control issue during entry. Utilizing the principles of optimal control and Pontryagin minimization, two speed optimization strategies are proposed. Finally, MATLAB is employed for simulation analysis. The results indicate that the control strategy proposed in this paper enables the system to clearly identify potential conflicts between vehicles and implement an optimal control strategy, ensuring that vehicles can navigate the roundabout efficiently in terms of time and fuel without collisions. Additionally, the minimum time interval is established at 0.2 seconds to completely prevent vehicle collisions. In this study, the fusion problem involving two vehicles at a single conflict point is further expanded to encompass multiple vehicles at multiple conflict points. Thus, the efficient scheduling of multiple vehicles in single-lane roundabouts is realized.

Optimal robust reinsurance contracts with investment strategy under variance premium principle

This paper investigates the optimal robust proportional reinsurance contracts with investment in a liquid financial market under variance premium principle in a principal-agent framework. The surplus process of the insurer (agent) is assumed to follow an approximating compound Poisson risk process. Both the insurer and reinsurer (principal) are allowed to invest in a risk-free asset and a risky asset whose price process is governed by the constant elasticity of variance model. The insurer and reinsurer aim to maximize the expected exponential utility of terminal wealth. The reinsurer is ambiguity-averse and has deterministic ambiguity aversion preferences against the diffusion risk caused by the financial market and the approximated diffusion risk which comes from the claim process. The reinsurance price is described by the reinsurer's safety loading which can be decided by the optimal strategies of the insurer and the reinsurer. By utilizing stochastic optimal control principle and HJB (or HJBI) equations, the optimal (robust) proportional reinsurance-investment strategies and the corresponding value functions are obtained explicitly. Numerical examples are provided.

A maximum principle for progressive optimal control of mean‐field forward–backward stochastic system involving random jumps and impulse controls

AbstractIn this paper, we study an optimal control problem of a mean‐field forward–backward stochastic system with random jumps in progressive structure, where both regular and singular controls are considered in our formula. In virtue of the variational technology, the related stochastic maximum principle (SMP) has been obtained, and it is essentially different from that in the classical predictable structure. Specifically, there are three parts in our SMP, that is, continuous part, jump part, and impulse part, and they are, respectively, used to characterize the characteristics of the optimal controls at continuous time, jump time, and impulse time. This shows that the progressive structure can more accurately describe the characteristics of the optimal control at the jump time. We also give two linear–quadratic examples to show the significance of our results.

On pointwise second‐order maximum principle for optimal stochastic controls of general mean‐field type

AbstractIn this paper, we establish a second‐order stochastic maximum principle for optimal stochastic control of stochastic differential equations of general mean‐field type. The coefficients of the system are nonlinear and depend on the state process as well as of its probability law. The control variable is allowed to enter into both drift and diffusion terms. We establish a set of second‐order necessary conditions for the optimal control in integral form. The control domain is assumed to be convex. The proof of our main result is based on the first‐ and second‐order derivatives with respect to the probability law and by using a convex perturbation with some appropriate estimates.

Maximum Principle for Optimal Control of Stochastic Evolution Equations with Recursive Utilities

Maximum Principle for Optimal Control of Stochastic Evolution Equations with Recursive Utilities

The effects of public health measures on severe dengue cases: An optimal control approach

Dengue fever is the most important viral mosquito-borne disease worldwide, with approximately 3.9 billion people at risk of acquiring dengue infection. Measures against mosquito bite combined with vector control programs to reduce mosquito population have been used in endemic countries for several years. Most recently, vaccines have become an important ally to prevent and control disease transmission. Economic costs of dengue control programs vary from region to region and therefore designing an optimal control strategy must be evaluated at different epidemiological contexts. Using a multi-strain vector-host mathematical model, we investigate the impact of different control measures to reduce dengue prevalence. A detailed sensitivity analysis to identify the key parameters influencing disease transmission is followed by an exploratory analysis of the possible solutions for the optimal control problem considering preventive measures to avoid mosquito bites, reduce mosquito population and vaccinate human hosts. The proposed cost functional includes a weighted sum of several efforts (not necessarily quantified as economic costs) for the controls which are evaluated alone and combined. The control system is analyzed using the Pontryagin‘s Principle for optimal control where different strategies are compared. Our results have shown that the simultaneous use of intervention measures are highly effective to reduce disease cases, however, the use of a single control measure can be as effective as the use of two or more controls combined. A careful evaluation of the epidemiological scenario is advised before designing strategies for disease prevention and control, allowing an optimal allocation of the public health resources.

The Global Maximum Principle for Progressive Optimal Control of Partially Observed Forward-Backward Stochastic Systems with Random Jumps

This paper is concerned with a partially observed progressive optimal control problem of forward-backward stochastic differential equations with random jumps, where the control domain is not necessarily convex, and the control variable enters into all the coefficients. In our model, the observation equation is not only driven by a Brownian motion but also a Poisson random measure, which also has correlated noises with the state equation. The partially observed global maximum principle is proved. To show its applications, a partially observed linear quadratic progressive optimal control problem of forward-backward stochastic differential equations with random jumps is investigated by the maximum principle and stochastic filtering. State estimate feedback representation of the optimal control is given in a more explicit form by introducing some ordinary differential equations.