Discrete Optimal Control Research Articles

A Weak Maximum Principle for Discrete Optimal Control Problems with Mixed Constraints

A Weak Maximum Principle for Discrete Optimal Control Problems with Mixed Constraints

The Performance of Symbolic Limited Optimal Discrete Controller Synthesis in the Control and Path Planning of the Quadcopter

The Performance of Symbolic Limited Optimal Discrete Controller Synthesis in the Control and Path Planning of the Quadcopter

Differential games in the supply chain of innovative products with consumer purchase regret induced returns.

Innovative products entering the market will cause dynamic changes in market demand, and consumers' purchase regret and their return behavior make the market environment more and more complex, which in turn affects the dynamic decision-making in the supply chain. In this paper, under the situation of discrete decision time, combining with the objective reality, we make discrete modification to the classical Bass diffusion model (Bass model), construct a manufacturer-led, retailer-followed supply chain differential game model, analyze the optimal decision-making of the manufacturer and the retailer by combining with the theory of discrete optimal control, and then verify the conclusions by numerical simulation. The results show that: when retailers purchase directly from the manufacturer and sell in the market, the optimal pricing of the innovative product can make the supply chain as a whole, realizing Pareto optimality; consumer's purchase regret will increase the amount of returns, which will lead to the decrease of product sales and the profits of the manufacturer and the retailer; when the innovative product accounts for a different share of the market, the impact of purchase regret on the wholesale price and the retail price are also different. Therefore, manufacturers need to have an extensive comprehension of the market to minimize the negative effects of consumer regret and returns, and to formulate a reasonable pricing strategy for their products to gain as much profit as possible.

The turnpike property for high‐dimensional interacting agent systems in discrete time

AbstractWe investigate the interior turnpike phenomenon for discrete‐time multi‐agent optimal control problems. While for continuous systems the turnpike property has been established, we focus here on first‐order discretizations of such systems. It is shown that the resulting time‐discrete system inherits the turnpike property with estimates of the same type as in the continuous case. In particular, we prove that the discrete time optimal control problem is strictly dissipative and the cheap control assumption holds.

Sparse randomized policies for Markov decision processes based on Tsallis divergence regularization

This work investigates a somewhat different point of view on Markov decision processes by reinterpreting them as a randomized shortest paths problem on a bipartite graph, therefore establishing bridges with entropy-regularized reinforcement learning. The graph structure contains the set of states as “left” nodes and the set of actions as “right” nodes. In that context, the action-to-state transition probabilities are provided by the environment whereas the state-to-action probabilities correspond to the (stochastic) policy to be found. The randomized shortest paths formalism (minimizing expected cost to the goal state subject to (Shannon or Tsallis) relative entropy regularization) is then readily applied to this bipartite structure, providing a possibly sparse stochastic policy interpolating between a least-cost and a purely random policy. The algorithm computing the policy is closely related to the dual linear programming formulation of the Markov decision processes to which the relative entropy regularization term, multiplied by a scaling factor balancing exploitation and exploration (the temperature), is added. It is derived from well-known techniques of discrete optimal control, relying on costates (Lagrange parameters) backward computation. In summary, the proposed algorithm allows the design of optimal stochastic – but still sparse – policies, ranging from a purely rational to a random behavior, depending on the temperature parameter.

UAV Swarm Cooperative Dynamic Target Search: A MAPPO-Based Discrete Optimal Control Method

Unmanned aerial vehicles (UAVs) are commonly employed in pursuit and rescue missions, where the target’s trajectory is unknown. Traditional methods, such as evolutionary algorithms and ant colony optimization, can generate a search route in a given scenario. However, when the scene changes, the solution needs to be recalculated. In contrast, more advanced deep reinforcement learning methods can train an agent that can be directly applied to a similar task without recalculation. Nevertheless, there are several challenges when the agent learns how to search for unknown dynamic targets. In this search task, the rewards are random and sparse, which makes learning difficult. In addition, because of the need for the agent to adapt to various scenario settings, interactions required between the agent and the environment are more comparable to typical reinforcement learning tasks. These challenges increase the difficulty of training agents. To address these issues, we propose the OC-MAPPO method, which combines optimal control (OC) and Multi-Agent Proximal Policy Optimization (MAPPO) with GPU parallelization. The optimal control model provides the agent with continuous and stable rewards. Through parallelized models, the agent can interact with the environment and collect data more rapidly. Experimental results demonstrate that the proposed method can help the agent learn faster, and the algorithm demonstrated a 26.97% increase in the success rate compared to genetic algorithms.

The Stability of Solutions of the Variable-Order Fractional Optimal Control Model for the COVID-19 Epidemic in Discrete Time

This article introduces a discrete-time fractional variable order over a SEIQR model, incorporated for COVID-19. Initially, we establish the well-possedness of solution. Further, the disease-free and the endemic equilibrium points are determined. Moreover, the local asymptotic stability of the model is analyzed. We develop a novel discrete fractional optimal control problem tailored for COVID-19, utilizing a discrete mathematical model featuring a variable order fractional derivative. Finally, we validate the reliability of these analytical findings through numerical simulations and offer insights from a biological perspective.

Discrete recoil control system modelling and optimal recoil damping of deepwater drilling riser systems

This study addresses discrete dynamic modelling and the optimal recoil control problem of deepwater drilling riser systems subject to the friction resistance of fluid discharge and platform heave motion. First, a discrete-time exosystem model for friction resistance of drilling fluid discharge is developed; then, a discrete-time dynamic model of platform heave motion is established. Third, a finite-time discrete optimal recoil controller with feedforward compensation mechanisms was designed. The optimal recoil controller gains can be obtained by iteratively solving discrete equations. The simulation results show that under the discrete feedforward and feedback optimal recoil controller and pure feedback optimal recoil controller, the recoil movements of the riser system are significantly restrained. Moreover, the former is more effective than the latter at resisting the riser recoil response.

Error analysis for the finite element approximations of Dirichlet parabolic boundary control problem with measure data

This article investigates the convergence analysis of finite element method for Dirichlet boundary control problem governed by parabolic equation with measure data. The presence of measure data in the source term and Dirichlet boundary control lead the solution of the state equation to have low regularity, which creates a challenging task for the error analysis. We prove the existence, uniqueness and regularity results of the solution to the control problem. For the discretization of the state and adjoint variables, we employ continuous piecewise linear polynomials while the control variable is approximated by piecewise constant functions. The backward Euler technique provides the foundation for the temporal discretization. For the completely discrete optimal control problem, a priori error bounds for the state variable in the L2(0,T;L2(Ω))-norm and for the control variable in the L2(0,T;L2(∂Ω))-norm are derived. Numerical experiment is performed to validate our theoretical analysis.

Obtaining the Most Likely Path in Stochastic Hidden Input Automata by Using Limited Optimal Discrete Control

Obtaining the Most Likely Path in Stochastic Hidden Input Automata by Using Limited Optimal Discrete Control

Gender differences in cycling motions: On objective functions for urban cycling

AbstractSocietal challenges such as climate change and the energy crisis are putting bicycles at the centre of many people's mobility considerations ‐ but not all. In addition to the need for improved infrastructure, stability and comfort when cycling also play a major role. Simulations offer the possibility to investigate the influence of biometric differences on comfort and stability more cost‐effectively than the measurement methods currently used for bike fitting. Bicycle dynamics and multibody simulation of cycling motions have been the subject of research for a long time. Often data‐driven models are used that follow pre‐established measurement data. Moreover, optimal control simulations for competitive sports are available, where for example, the travel distance during a given time is maximised. However, such models are usually based on the biometric data of an average 18–25 year old male, while the influence of gender differences on cycling motions are rarely explored. Yielding towards closing this gender data gap, we use a discrete mechanics and optimal control framework (DMOCC), which benefits from its structure preserving formulation and has been successfully used for biomechanical applications before. The implemented multibody model of a leg performing a cycling motion can be adapted to individual 3D scans via the geometry parameters and the bounds on the joint angles and torques, providing the possibility to investigate the influence of biometric diversity on the resulting motions. In this first approach, we discuss several possibilities to formulate appropriate objective functions for cycling. The final aim of this study is to supplement a given bike frame by software chosen adaptations so that it optimally fits to individual biometric conditions, thus increasing comfort, the sense of safety and performance, which ultimately enables greater participation in the mobility transition for women, children, and seniors.

Safe Trajectory Generation for Wheel-Leg Hybrid Mechanism Using Discrete Mechanics and Optimal Control

Abstract The wheel-legged robot inherits the merit of both the wheeled robot and the legged robot, which can not only adapt to the complex terrain but also maintain the driving efficiency on the flat road. This article presents an optimization-based approach that leverage ideas from computational geometric mechanics to generate safe and high-quality wheel-leg hybrid motions among obstacles. The formulation of the proposed motion optimization problem incorporates the Lagrange–d’Alembert principle as the robot’s dynamic constraints and an efficient closed-form formulation of collision-free constraints. By discretizing the variational mechanics principle directly, rather than its corresponding forced Euler–Lagrange equation, the continuous trajectory optimization problem is transformed into a nonlinear programming (NLP) problem. Numerical simulations and several real-world experiments are conducted on a wheel-legged robot to demonstrate the effectiveness of the proposed trajectory generation approach.

Process optimization of microbial fermentation with parameter uncertainties via distributionally robust discrete control

There are some uncertain kinetic parameters in microbial fermentation system because of the unclear intracellular metabolic mechanisms. Considering the affection of these uncertain parameters on system performance, dynamic process optimization of the fermentation system can be modeled as a distributionally robust discrete control problem under moment uncertainty, which aims to maximize the mean productivity by optimizing the discrete-valued dilution rate function. Based on duality theory, the established min–max discrete optimal control problem is first transformed into a single level minimization problem, which is then discretized into a large-scale parameter optimization problem with semi-infinite constraint via time transformation and control parameterization. A new two-step estimation of distribution algorithm is developed to solve the obtained large-scale optimization problem. Numerical results show the feasibility and effectiveness of the proposed solution approach together with the superiority of the obtained control strategy considering parameter uncertainties.

Altitude control of quadcopter with symbolic limited optimal discrete control

Altitude control of quadcopter with symbolic limited optimal discrete control

Discrete Integral Optimal Controller for Quadrotor Attitude Stabilization: Experimental Results

The Unmanned Aerial Vehicle (UAV) attitude stabilization problem has been dealt with in many previous works through applying a vast range of philosophies of control strategies. In this paper, a discrete controller based on a Linear Quadratic Regulator (LQR) plus integral action is synthesized to stabilize the attitude and altitude of a quadrotor helicopter. This kind of control strategy allows us to reduce the energy consumption rate, and the desired UAV behavior is properly achieved. Experimental tests are conducted with external disturbances such as crosswinds deliberately added to affect the performance of the aerial vehicle. This provides experimental evidence that the integral part considered in the proposed control strategy contributes to improving the performance of the vehicle under external disturbances. In fact, a comparative analysis of potential and kinetic energy consumption is developed between the Optimal Integral Controller (OIC) and a Proportional Integral Derivative Controller (PID), allowing us to determine the level of improvement of the closed-loop system when the discrete Integral Optimal Controller is applied.

Numerical analysis of a family of simultaneous distributed-boundary mixed elliptic optimal control problems and their asymptotic behaviour through a commutative diagram and error estimates

In this paper, we consider a family of simultaneous distributed-boundary optimal control problems (Pα) on the internal energy and the heat flux for a system governed by a mixed elliptic variational equality with a parameter α>0 (the heat transfer coefficient on a portion of the boundary of the domain) and a simultaneous distributed-boundary optimal control problem (P) governed also by an elliptic variational equality with a Dirichlet boundary condition on the same portion of the boundary. We formulate discrete approximations Phα and Ph of the optimal control problems Pα and (P) respectively, for each h>0 and for each α>0, through the finite element method with Lagrange’s triangles of type 1 with parameter h (the longest side of the triangles). The goal of this paper is to study the convergence of this family of discrete simultaneous distributed-boundary mixed elliptic optimal control problems Phα when the parameters α goes to infinity and the parameter h goes to zero simultaneously. We prove the convergence of the family of discrete problems Phα to the discrete problem Ph when α→+∞, for each h>0, in adequate functional spaces. We study the convergence of the discrete problems Phα and Ph, for each α>0, when h→0+ obtaining a commutative diagram which relates the continuous and discrete simultaneous distributed-boundary mixed elliptic optimal control problems Phα,Pα,Ph and (P) by taking the limits h→0+ and α→+∞ respectively. We also study the double convergence of Phα to (P) when (h,α)→(0+,+∞) which represents the diagonal convergence in the above commutative diagram.

Discrete optimal quadratic AGC based cost functional minimization for interconnected power systems

The increasing complexity and difficulty of the Automatic generation control (AGC) problem has resulted from the increasing scale of interconnected power networks and changing daily demands. The primary goals of AGC are to control frequency variations at nominal levels and tie-line power variances at planned levels. To effectively deal with AGC control difficulties, this study introduces Discrete Optimal Quadratic Automatic Generation Control (OQAGC). One advantages of this method is the differentiation of quadratic cost function results into linear terms while minimizing control actions and minimizing state deviations. This developed control method leads to a simple and easy discrete control law that can be implemented for both linear and nonlinear systems. For optimizing the controller, this research work utilized an optimum control theorem using Lagrangian multipliers, while the functional minimization technique is used for systematically selecting the state and control weighting matrices in discrete form for N control regions (where N is the number of interconnected power systems). The discrete cost function needs are derived using this technique in terms of area control errors, integral area control errors, and control energy expenditure. Four interconnected power systems were analyzed with/without disturbances and area control errors, each with one thermal, hydro, and gas-generating unit. A two-area multi-source power system with renewable energy in control area 2 is analyzed for the performance of the proposed controller with generation rate constraints (GRCs). The functional minimization technique simplifies and eases the choosing of weighting matrices. Furthermore, the simulation findings suggest that the developed discrete optimum quadratic AGC control-based cost functional minimization approach enhances power system dynamics in terms of stability, steady-state performance, and the closed-loop control system's robustness to input load disturbances. As a result, the newly developed OQAGC approach demonstrates the significance of the discrete LQR controller for N multi-area power systems.

Steering exact penalty DCA for nonsmooth DC optimisation problems with equality and inequality constraints

We propose and study a version of the DCA (Difference-of-Convex functions Algorithm) using the penalty function for solving nonsmooth DC optimisation problems with nonsmooth DC equality and inequality constraints. The method employs an adaptive penalty updating strategy to improve its performance. This strategy is based on the so-called steering exact penalty methodology and relies on solving some auxiliary convex subproblems to determine a suitable value of the penalty parameter. We present a detailed convergence analysis of the method and illustrate its practical performance by applying the method to two nonsmooth discrete optimal control problem.

Discrete mathematical modeling and optimal control of the marital status: Islamic polygamous marriage model case

In this paper, we discuss a discrete mathematical model of Islamic polygamy and the social position of Muslims. In eleven compartments we explain the social situation and give an explanation of the marital status of each Males and females in Islamic societies that allow polygamy. In order to controlling and reducing the number of virgins men and women, divorced men and women we implement two control variables. The first control characterizes the benefits of an awareness campaign to educate virgin men and women about the benefits marriage to the individual and society, and the second control is about the legal procedures, administrative complexities and the grave financial and social implications of divorce. After that, we applied the optimal control theory to describe such an optimal strategies and finally a numerical simulation was performed to verify the theoretical analysis using a progressive-regressive discrete schema that converges following a convenient test related to the Forward–Backward Sweep Method (FBSM).

Hurwicz criterion based optimal control model for uncertain descriptor systems with an application to industrial management

In this work, optimal control models subject to discrete and continuous uncertain descriptor systems are both investigated under Hurwicz criterion. At first, by employing dynamic programming method, a recurrence equation is obtained to simplify the discrete optimal control model, and a bang-bang control problem is settled according to this equation. Then, for the continuous optimal control model, an equation of optimality is proposed and testified with the help of uncertainty theory and the principle of optimality, revealing the internal connection between optimal control and optimal value. Finally, a numerical example and an industrial management model are provided to illustrate the validness of above results.