Bellman's Dynamic Programming Method Research Articles

Maximum Principle in Autonomous Multi-Object Safe Trajectory Optimization

The following article presents the task of optimizing the control of an autonomous object within a group of other passing objects using Pontryagin’s bounded maximum principle. The basis of this principle is a multidimensional nonlinear model of the control process, with state constraints reflecting the motion of passing objects. The analytical synthesis of optimal multi-object control became the basis for the algorithm for determining the optimal and safe object trajectory. Simulation tests of the algorithm on the example of real navigation situations with various numbers of objects illustrate their safe trajectories in changing environmental conditions. The optimal object trajectory obtained using Pontryagin’s maximum principle was compared with the trajectory calculated using the Bellman dynamic programming method. The analysis of the research allowed for the formulation of valuable conclusions and a plan for further research in the field of autonomous vehicle control optimization. The maximum principle algorithm allows one to take into account a larger number of objects whose data are derived from ARPA anti-collision radar systems.

Planning the Optimal Reference Flight Path of an Aircraft Using a Terrain Map

In connection with the development of aviation technology, the expansion of the functionality of modern aircraft and the complication of the tasks being solved, additional stringent requirements are imposed on the accuracy of air navigation. Along with high accuracy, navigation autonomy is also required. The autonomy of aircraft navigation implies the navigation of an aircraft without the use of active radar facilities. In this article, to calculate the optimal reference flight path of an aircraft in a horizontal plane, the Bellman dynamic programming method is proposed using a digital terrain map. A technique, algorithms have been developed, and a numerical experiment has been carried out using fragments of a digital terrain map. High accuracy and autonomy of navigation are necessary, among other things, to ensure stealth (invisibility) of aircraft by ground-based radar facilities of a potential enemy in low-altitude flight mode. The best stealth of an aircraft flight in a low-altitude flight mode is achieved not only by flying around, but also by avoiding obstacles, due to the shielding properties of the earth’s surface. Autonomy of navigation provides periodic correction of the reference flight path during long-term flight, and high accuracy prevents aircraft collisions with the earth’s surface in low-altitude flight mode. The purpose of this work is to select the optimal reference trajectory (route) of an aircraft flight in the low-altitude flight mode. This article discusses a method developed based on the principle of dynamic programming for calculating the reference flight path in the horizontal plane depending on the DEM and the implementation of the algorithm based on the proposed method with a specific numerical example. The method makes it possible to calculate the optimal flight route, which provides the greatest secrecy of the aircraft in the low-altitude flight mode using the screening properties of the earth’s surface irregularities.

Maintaining Symmetry in Optimal and Safe Control of the Ship to Avoid Collisions at Sea

The aim of this study was to make a novel symmetry analysis in relation to the importance of optimizing the ship’s trajectory and safety in situations at sea where there is a risk of collision with other ships. To achieve this, the state constraints in the optimization were formulated as ship domains generated by the neural network. In addition, the use of the Bellman dynamic programming method enabled the effective optimization of the ship’s safe control. The above assumptions were confirmed by the calculations of the optimal and safe ship traffic paths for the two valid agree with COLREGs states of visibility at sea and for different densities of the dynamic programming grid. Practical conclusions from the research were formulated, and a plan for further research on methods of ensuring safety in navigation was outlined.

Проектирование схемы системы поддержки принятия решений на основе онтологии перекомпонуемого производства

The purpose of the work is to design a scheme of the DSS of a recomposable manufacturing using an ontological approach and mathematical modeling. The decomposition of the production process in the recomposed manufacturing is carried out, the main influencing factors are identified. The conceptual scheme and hierarchy of classes of the constructed ontology of the subject area are given. In order to take into account the priority of influencing factors and identify the controlling influences affecting the speed of reconfiguration of the recomposed manufacturing, the Ishikawa cause-and-effect diagram was developed. A formalized description of the process of choosing the optimal way to reconfigure the system is proposed. A DSS scheme for the reconfigurable manufacturing has been developed, based on the consideration of temporal and spatial factors of the reconfiguration. A recomposition time structure is proposed, which allows optimizing several factors at the same time. An example of optimizing the recomposition time of a recomposable manufacturing area using the Bellman dynamic programming method and graph theory is demonstrated.

Optimal Control of Automatic Manipulator for Elimination of Galvanic Line Load Oscillation

This paper provides an analysis of the current state-of-the-art technologies in the field of auto-operators used in production during the electroplating process. General schemes of operations are presented, benefits and drawbacks of each scheme are discussed. The paper discusses an increase in the operating efficiency of the auto-operator in transient conditions (braking and acceleration) by reducing suspension oscillations and provides an example of a similar problem from other industries. In addition to the classification of the auto-operators, three main ways and control methods of the auto-operator of a galvanic line are presented. The main ways of eliminating oscillations during the movement of the auto-operator, as well as the rationale for the choice of adaptive (optimal) control, based on and comparing the basic control algorithms of the robot manipulator, are discussed. The comparative analysis of algorithms used to determine the optimal control has been carried out. Application field of each optimal control method described, moreover advantages and disadvantages as well as implementation methods described. Bellman dynamic programming method was chosen to eliminate oscillations of the suspension with details during the auto-operator transient conditions, the chosen method takes into account all necessary conditions to achieve the desired result.

Optimization of the flight trajectory of a non-manoeuvrable aircraft to minimize fuel consumption by the dynamic programming method

Abstract Using Bellman's dynamic programming method, a fuel-consumption optimum flight trajectory of a typical mid-range aircraft is constructed for a prescribed range. The optimum trajectory was calculated for the full model of motion of the aircraft along a trajectory in a vertical plane. Optimization was realized simultaneously for the whole trajectory without decomposing the process into individual steps. The method made it possible to take into account all restrictions imposed both by technical peculiarities of the aircraft and also by other requirements – safety and comfort of the passengers.

Investigating a Nonlinear Optimal Control Problem with Discounting

In this article, we consider one-dimensional optimization models that are relevant for mining applications. The maximand functional is interpreted as discounted profit. Several alternative formulations of the optimization problem are considered on finite and infinite horizons. Optimal solutions are constructed in analytical form. The optimal control is determined in the form of a time function (a program) and in the form of a function of the phase coordinate (feedback -- the hyperbolic tangent law). The theoretical basis for our results is provided by Pontryagin's maximum principle and Bellman's dynamic programming method. The theoretical results provide a foundation for computer experiments with prototype and real-life data. The article is an expanded version of [5]. The discussion is accompanied by graphs.

Quasicanonical structure of optimal control in constrained discrete systems

This paper considers discrete processes governed by difference rather than differential equations for the state transformation. The basic question asked is if and when Hamiltonian canonical structures are possible in optimal discrete systems. Considering constrained discrete control, general optimization algorithms are derived that constitute suitable theoretical and computational tools when evaluating extremum properties of constrained physical models. The mathematical basis of the general theory is the Bellman method of dynamic programming (DP) and its extension in the form of the so-called Carathéodory-Boltyanski (CB) stage criterion which allows a variation of the terminal state that is otherwise fixed in the Bellman's method. Two relatively unknown, powerful optimization algorithms are obtained: an unconventional discrete formalism of optimization based on a Hamiltonian for multistage systems with unconstrained intervals of holdup time, and the time interval constrained extension of the formalism. These results are general; namely, one arrives at: the discrete canonical Hamilton equations, maximum principles, and (at the continuous limit of processes with free intervals of time) the classical Hamilton-Jacobi theory along with all basic results of variational calculus. Vast spectrum of applications of the theory is briefly discussed.

Certain Hypotheses in Optimal Control Theory and the Relationship of the Maximum Principle with the Dynamic Programming Method

Certain hypotheses concerning optimal control are refined and the relationship between the Pontryagin maximum principle and Bellman's dynamic programming method in the context of these hypotheses is discussed.

On Feedback Laws for Robotic Systems

Abstract : A variety of control problems arising from robotics applications can be restated as optimal control problems of minimum-time state transfer in the presence of state-space constraints and constraints of incomplete-state information. The traditional approaches to solving such problems are Pontryagin's Maximum Principle, in the case of open-loop control, and Bellman's Dynamic Programming method.

Canonicial Transformations and the Hamilton‐Jacobi Theorem in the Optimum Control Theory

AbstractThis paper deals with the problem to establish a profound relationship between Pontryagin's maximum principle and Bellman's dynamic programming method via the canonical transformations of the variables, as it is a case in classical mechanics. A rigorous form of the Hamilton‐Jacobi theorem is proved for optimal control systems. Further it is shown that the controlled systems may be treated by a partial differential equation of the Hamilton‐Jacobi type, which, however, is fundamentally different of the Bellman functional equation. The solution of this equation is a function of the generalized momenta, which appear in Pontryagin's theory, and the time. In some cases this equation can be advantageously used in comparison with the Bellman equation. An example is solved to illustrate the presented theory.