Optimal Control Problem Research Articles

Integral reinforcement learning-based event-triggered optimal tracking control for modular robot manipulators via non-zero-sum game

Abstract Under an event-triggered mechanism, a non-zero-sum (NZS) game optimal tracking control method for modular robot manipulator (MRM) systems with input constraints is proposed using the adaptive dynamic programming (ADP) method based on integral reinforcement learning (IRL). First, a dynamic model of the MRM system is developed based on joint torque feedback technology, consisting of an n-joint subsystem related to interconnected dynamic coupling (IDC). Second, we design a robust compensation controller to handle the known model term and an optimal compensation controller to deal with the uncertainty term caused by the IDC and friction, respectively. In addition, a nonlinear disturbance observer is established to dispose of the negative effects caused by the uncertain sensor output disturbance. Third, based on differential game theory, we transform the optimal tracking control problem of the MRM system into an n-player NZS game problem. Then, the IRL-based ADP method is adopted, which relaxes the need for system partial unknown dynamic information, and only a critic neural network is used to solve the coupled Hamilton–Jacobi equation, so as to obtain the optimal control policy. Then, using Lyapunov theory, the tracking error of the MRM system is demonstrated to be uniformly ultimately bounded. Finally, the effectiveness and superiority of the proposed algorithm are verified through experiments.

Finite element error analysis of affine optimal control problems

This paper is concerned with error estimates for the numerical approximation for affine optimal control problems subject to semilinear elliptic PDEs. To investigate the error estimates, we focus on local minimizers that satisfy certain local growth conditions. The local growth conditions we consider in this paper appeared recently in the context of solution stability and contain the joint growth of the first and second variation of the objective functional. These growth conditions are especially meaningful for affine control constrained optimal control problems because the first variation can satisfy a local growth, which is not the case for unconstrained problems. The main results of this paper are the achievement of error estimates for the numerical approximations generated by a finite element scheme with piecewise constant controls or a variational discretization scheme. Even though the growth conditions considered are weaker than those appearing in the recent literature on finite element error estimates for affine problems, this paper substantially improves the existing error estimates for both the optimal controls and the states when a Hölder-type growth is assumed.

Infinite Horizon Mean-Field Linear Quadratic Optimal Control Problems with Jumps and the Related Hamiltonian Systems

Infinite Horizon Mean-Field Linear Quadratic Optimal Control Problems with Jumps and the Related Hamiltonian Systems

Stackelberg exact controllability of a class of nonlocal parabolic equations

This paper deals with a multi-objective control problem for a class of nonlocal parabolic equations, where the non-locality is expressed through an integral kernel. We present the Stackelberg strategy that combines the concepts of controllability to trajectories with optimal control. The strategy involves two controls: a main control (the leader) and a secondary control (the follower). The leader solves a controllability to trajectories problem which consists to drive the state of the system to a prescribed target at a final time while the follower solves an optimal control problem which consists to minimize a given cost functional. The paper considers two cases: in the first case, both the leader and the follower act in the interior of the domain, and in the second case, the leader acts in the interior of the domain and the follower acts on a small part of the boundary. These results are applied to both linear and nonlinear nonlocal parabolic systems.

Control strategy for trading off solar power and control input while rendezvous and docking

This paper presents a novel strategy for decoupling the attitude and orbit equations of a CubeSat for rendezvous and docking with an uncontrollable cooperative target. For computing a safe trajectory, the proposed control algorithm takes into account the solar energy received by the CubeSat. The CubeSat is equipped with a thruster for orbit maneuvers, magnetic coils for attitude control, and four fixed single-sided rectangular solar panels. The coupled dynamics arising from the dynamical model of the spacecraft is solved using a novel weighted vector-based approach. This paper presents a linearized nonlinear optimal control problem arising in small spacecrafts while trying to maximize solar energy input in the rendezvous and docking process. An intermediate orbit is defined and used to divide the problem into two different optimal control problems: rendezvous optimization and docking optimization problems. A geometrical approach based on the shape of the chaser and the target is contemplated for collision avoidance while docking. The proposed controller design is modified whenever the sunlight is obstructed by the Earth, and the maneuvering controls are redesigned accordingly for optimal rendezvous and docking. Numerical simulations have been carried out to show the efficacy of the proposed concept for steering the trade-off between solar energy and control input during the rendezvous and docking of CubeSat with a tumbling target.

Linear quadratic optimal control problems of infinite‐dimensional mean‐field type with jumps

AbstractIn this paper, we formulate and investigate a framework for the theory of the linear quadratic optimal control problem (LQ problem) for infinite‐dimensional mean‐field stochastic evolution systems with jumps. We ensure the well‐posedness of the investigated problems by establishing the existence, uniqueness, and a priori estimates for mild solutions to general infinite‐dimensional mean‐field forward stochastic evolution equations (MF‐SEE) and mean‐field backward stochastic evolution equations (MF‐BSEE) with jumps. Leveraging the Yosida approximation theory, we establish a dual theory between MF‐SEE and MF‐BSEE with jumps, overcoming the challenge posed by the inapplicability of Itô's formula in the context of mild solutions. Our main results regarding the existence and uniqueness of the optimal control, along with corresponding dual characterizations and state feedback representations, are obtained through convex analysis techniques, our established dual theory, and decoupling methods.

Advanced Optimal Tracking Control With Stability Guarantee via Novel Value Learning Formulation.

In this article, to solve the optimal tracking control problem (OTCP) for discrete-time (DT) nonlinear systems, general value iteration (GVI) scheme and online value iteration (VI) algorithms with novel value function are discussed. First, the disadvantage of the traditional value function for the OTCP is presented and the novel value function is introduced. Second, we analyze the monotonicity and convergence of GVI and establish the admissibility condition of GVI to evaluate the admissibility of the current iterative control. Note that a novel approach is introduced to analyze the admissibility. Third, based on the attraction domain, improved control policies with online VI can be obtained by judging the location of the current tracking error and reference point. Finally, the stability of the online VI-based control system is guaranteed. Besides, we provide two simulation examples to show the performance of the proposed methods.

Two-grid Method of Expanded Mixed Finite Element Approximations for Parabolic Integro-differential Optimal Control Problems

Two-grid Method of Expanded Mixed Finite Element Approximations for Parabolic Integro-differential Optimal Control Problems

Convex Hull Relaxation of Optimal Control Problems With General Nonconvex Control Constraints

Convex Hull Relaxation of Optimal Control Problems With General Nonconvex Control Constraints

AONN-2: An adjoint-oriented neural network method for PDE-constrained shape optimization

PDE-constrained shape optimization has been playing an important role in a large variety of engineering applications. Traditional mesh-dependent shape optimization methods often encounter challenges due to mesh deformation. To address this issue, we present a new adjoint-oriented neural network method, AONN-2, for PDE-constrained shape optimization problems. This method extends the capabilities of the original AONN method [1], which is developed for efficiently solving parametric optimal control problems. AONN-2 inherits from AONN the direct-adjoint looping (DAL) framework for computing the extremum of an objective functional and the involved neural network methods for solving complicated PDEs. Furthermore, AONN-2 expands the application scope to shape optimization by taking advantage of the shape derivatives to optimize the shape represented by discrete boundary points. AONN-2 is a fully mesh-free shape optimization approach, naturally sidestepping issues related to mesh deformation, with no needs for maintaining mesh quality and additional mesh corrections. A series of experimental results are presented, highlighting the flexibility, robustness, and accuracy of AONN-2.

The energy-balance method for optimal control in renewable energy applications

A theoretical method is presented, called the energy-balance method, for maximising the energy extracted from a renewable energy converter in terms of determination of an optimal control. The method applies to control systems specified by linear graphs, and graph-theoretic techniques are employed. The method simplifies a number of optimal control problems by essentially expressing the performance objective — maximising energy extraction — in terms of an equivalent objective involving fewer variables, thereby reducing the complexity of the optimisation. As illustrated, in certain cases the optimal control problem may be reduced to one solvable by elementary calculus techniques. The theory is illustrated with examples from solar, wave and wind applications.

On a Non-Convex Lagrange Optimal Control Problem

Abstract In this paper, we are concerned with an iterative differential inclusion governed by the time-dependent maximal monotone operator with perturbation. The approach to solve our problem is based on the Yosida approximation technique. The theoretical result is applied to prove an existence result for a Lagrange optimal control problem without assumptions concerning convexity.

Stochastic intervention control of mean-field jump system with noisy observation via L-derivatives with application to finance

In this paper, we investigate stochastic optimal intervention control of mean-field nonlinear random Poisson-jump-system with related noisy process. We derive the necessary conditions of optimality for partially observed optimal intervention control problems of mean-field type. The coefficients depend on the state of the solution process as well as of its probability distribution and the control variable. The proof of our main result is obtained by applying L-derivatives in the sense of Lions. In our control model, there are two models of jumps for the state process, the inaccessible ones which come from the random Poission process and the predictable ones which come from the intervention control. Finally, we apply our result to study conditional mean-variance portfolio selection problem with interventions, where the foreign exchange interventions are intended to contain excessive fluctuations in foreign exchange rates and to stabilize them.

Indirect method for solving non-linear optimal control of a non-rectilinear motion of a rocket with variable mass

In this paper, an optimal trajectory of the rocket angle with a variable mass will be calculated by considering the aerodynamic forces, the acceleration of gravity and moves with a non-rectilinear motion from a initial state to a final state with a known altitude. The aim is to optimize the lateral offset of the rocket. For this, we formulate an optimal control problem where the rocket angle is the control. In order to solve the problem, let applied Shooting method's based on the Pontryagin's maximum principle, and study the precision and a duration time. Finally, we validate the results by using MATLAB software.

Mathematical Modeling of Infectious Disease and Prey-Predator Interaction with Optimal Control

In this paper, the impact of viral illnesses on the predator-prey relationship with an optimal control analysis is studied. An ecoepidemiological model of four compartments, namely, susceptible prey, susceptible predator, infected prey, and infected predator populations, in the interaction of the prey-predator system is formulated. The fundamental tenet of our ecoepidemiology model is that sick predators do not engage in predation. It is confirmed that the system’s solution exists, is positive, and is bounded. The system’s equilibrium points are determined and computed. Lyapunov functions and a linearizing form are used for local and global stability analysis, respectively. The next generation matrix approach is used to calculate the threshold value for diseased predators and prey at the disease-free equilibrium point. Optimal treatment options for vulnerable and infected populations are established by applying optimal control theory to the ecoepidemiology model of a prey-predator system. MATLAB software is utilized to obtain numerical simulations that validate the analytical outcomes. The optimal control problem simulations demonstrate that the number of infected populations in a given prey-predator system can be decreased by implementing control measures.

ANALYSIS AND OPTIMAL CONTROL OF TUBERCULOSIS DISEASE SPREAD MODEL WITH VACCINATION AND CASE FINDING CONTROL (CASE STUDY: SURABAYA CITY)

Tuberculosis is a contagious disease that infects humans. It is caused by the bacterium Mycobacterium tuberculosis (M.tb). It is the largest infectious disease in the world and has become a major global health problem. Therefore, efforts are being made to control the spread of tuberculosis disease through vaccination and case finding, with the aim of reducing the population of latently infected and actively infected individuals. This study discusses the mathematical model of tuberculosis disease spread, disease-free and endemic equilibrium points, and stability analysis around the equilibrium points. Then, using Pontryagin's minimum principle, the optimal control problem is solved numerically by the 4th-order Runge-Kutta method. Based on the analysis and simulation results, the system is asymptotically stable around the disease-free and endemic equilibrium points. Furthermore, optimal control in the form of vaccination of susceptible individuals is required to further suppress the rate of change of susceptible individuals into latent individuals, while control in the form of case finding on latently infected individuals is required until the 9th year to minimize the population size, while on actively infected individuals, it is required until the 8th year to minimize the population size. Providing optimal control resulted in a 100% increase in the susceptible population and a 100% reduction in the latent and infected populations.

MATHEMATICAL MODEL OF REPELLENT EFFECT IN DENGUE TRANSMISSION

Dengue is a disease caused by the dengue virus, transmitted through the bite of an infected female Aedes aegypti. Dengue virus is a member of the genus Flavivirus, family Flaviviridae. Indonesia is one of the countries with the most dengue cases in Southeast Asia. Therefore, dengue transmission must be controlled to reduce the increase in dengue cases. One of the controls is by using repellents. Repellent is one of the human protection strategies to avoid mosquito bites used by spraying or smearing. This study models dengue transmission by reviewing the effect and control of repellent. A mathematical model of repellent effect and control in dengue transmission uses a SIR compartment model. The SIR model is modified by involving mosquitoes and the human population. Repellent is used in both susceptible humans, infected humans, and recovered humans. Numerical and analytical simulations are conducted to analyze the behavior of each compartment of the mosquito and human populations in dengue transmission. Analytical results show that the factors affecting the spread of infection are the transmission rate of the dengue virus and the loss of human-repellent protection. The transmission rate of dengue virus in the interval increases the infected human by 2.73%, while the rate of loss of human repellent protection in the interval increases the infected human by 0.03%. Optimal control is used to minimize the number of infected humans who do not use repellent. The results of numerical simulations on the optimal control problem show that an increase in the proportion of healthy humans who have campaign effect and use repellent regularly in the range of 14.67% can reduce infected individuals by 0.647%.

An Optimal Control Problem of SECR Corruption Dynamic Model

In this paper, the SECR model of corruption dynamics was proposed and will be analyzed, including non-negativity and boundedness of the model, corruption cases-free equilibrium point, and local stability. In order to reduce the number of corruption cases, the SECR model is extended with two controls strategies, i.e, u1 : preventive campaign for corruption by the KPK institution, and u2 : giving punishment for corrupted individual. Furthermore, it is investigated how control strategies should be applied to prevent corruption cases optimally. The model will be simulated in three scenarios, i.e. with full control u1 and u2 simultaneously, only u1 control, and only u2 control. Then, each of these simulations results will be compared with the results of the simulation model without control. The optimal control problem will be solved using Pontryagin’s Minimum Principle and Runge Kutta order 4th method for numerical simulation with Matlab. The simulation results show that the first scenario give the optimal solution in reducing the number of exposed and corrupted subpopulations with minimum controls effort and short time period. Based on these results, it can be concluded that in order to prevent and reduce the number of corruption cases, the control strategy giving punishment for the corrupted individuals must be also followed by prevention campaign corruption in society by the KPK, where both these control strategies must be carried out together to give the optimal results.

Inferring Human Control Intent Using Inverse LQR with Output Penalty Versus Gain Penalty: Better Fit But Similar Intent

Abstract To identify the underlying mechanisms of human motor control, parametric models are utilized. One approach of employing these models is inferring the control intent (estimating motor control strategy). A well-accepted assumption is that human motor control is optimal, thus the intent is inferred by solving an inverse optimal control problem. Linear quadratic regulator (LQR) is a well-established optimal controller and its inverse problem (ILQR) has been used in the literature to infer the control intent of one subject. This implementation used a cost function with gain penalty; minimizing the error between LQR gain and a preliminary estimated gain. We hypothesize that relying on an estimated gain may limit ILQR optimization capability. In this study, we derive an ILQR optimization with output penalty; minimizing the error between the model output and the measured output. We conducted the test on 30 healthy subjects who sat on a robotic seat capable of rotation. The task utilized a physical human-robot interaction with a perturbation torque as input and lower and upper body angles as output. Our method significantly improved the goodness of fit compared to the gain-penalty ILQR. Moreover, the dominant inferred intent was not statistically different between the two methods. To our knowledge, this work is the first that infers motor control intent for a sample of healthy subjects. This is a step closer to investigating control intent differences between healthy subjects and subjects with altered motor control, e.g., low back pain.

Finite Elements with Switch Detection for direct optimal control of nonsmooth systems

This paper introduces Finite Elements with Switch Detection (FESD), a numerical discretization method for nonsmooth differential equations. We consider the Filippov convexification of these systems and a transformation into dynamic complementarity systems introduced by Stewart (Numer Math 58(1):299–328, 1990). FESD is based on solving nonlinear complementarity problems and can automatically detect nonsmooth events in time. If standard time-stepping Runge–Kutta (RK) methods are naively applied to a nonsmooth ODE, the accuracy is at best of order one. In FESD, we let the integrator step size be a degree of freedom. Additional complementarity conditions, which we call cross complementarities, enable exact switch detection, hence FESD can recover the high order accuracy that the RK methods enjoy for smooth ODE. Additional conditions called step equilibration allow the step size to change only when switches occur and thus avoid spurious degrees of freedom. Convergence results for the FESD method are derived, local uniqueness of the solution and convergence of numerical sensitivities are proven. The efficacy of FESD is demonstrated in several simulation and optimal control examples. In an optimal control problem benchmark with FESD, we achieve up to five orders of magnitude more accurate solutions than a standard time-stepping approach for the same computational time.