Conventional Dynamic Programming Method Research Articles

Optimization of speed trajectory for electric wheel loaders: Battery lifetime extension

The electrification of wheel loaders is considered a leading trend due to its advantage of zero‑carbon emissions. However, the inevitable phenomenon of battery degradation has led to increased battery usage and maintenance costs. This study first extends the battery lifetime by optimizing the speed trajectory based on the typical loading cycle of the wheel loader. The optimal control problem is formulated by systematically modeling the wheel loader's powertrain and using a precise semi-empirical battery aging model. To reduce computational costs, the modified optimal control problem includes a weighted penalty on travel time. Unlike the conventional dynamic programming method that takes substantial computation time, a combined algorithm of dynamic programming and Brent's method (DP-BM) is first introduced to provide a numerical solution to the optimization problem with a reduced computational burden. Simulation results demonstrate that the optimized trajectory can decrease the average power consumption of the battery and reduce the number of full equivalent cycles, resulting in a 4.48% improvement in the average battery lifetime compared to the typical trajectory. Modeling errors are analyzed to make sure the results are reliable. Furthermore, the proposed approach significantly reduces computation time compared to the conventional dynamic programming method, with an average reduction of 95%. The total cost for each year is finally summarized, and the results indicate that the optimized trajectory offers a distinct advantage in terms of cost-effectiveness.

Closed-form solution to the dynamic programming for a heavy-duty parallel hybrid vehicle energy management

Dynamic programming (DP) is frequently used to obtain the optimal solution to the hybrid electric vehicle (HEV) energy management. The trade-off between the accuracy and the computational effort is the biggest problem for the DP method. The closed-form solution to the DP is proposed to solve this problem. Firstly, the affine linear model of the engine fuel rate is obtained based on engine test data. The piecewise linear approximation of the motor power demand is obtained considering the different energy flows in the charging and discharging stages of the battery. Then, the second-order Taylor expansion for the cost matrix at each time and state grid point is introduced to get the closed-form solution of the optimal torque split. The results show that this method can greatly reduce the computing burden by 93% while ensuring near-optimal fuel economy compared with the conventional DP method.

Data-driven energy management and velocity prediction for four-wheel-independent-driving electric vehicles

This paper proposes an online energy management and optimization method for four-wheel-independent-driving electric vehicles via stochastic model predictive control (SMPC), where velocity is predicted as the foundation to ensure feasibility and efficiency. By utilizing operating data of real-world electric vehicles from a big data platform, a data-driven Markov chain method is adopted to achieve vehicle velocity prediction in an accurate and reliable way. On top of the proposed method, real-time updates of the sample space and online substitution of the velocity-acceleration (V-A) state space can be realized, which mitigates problems of prediction interruption resulting from deficiency of sample state. Simulation results based on a constructed Hardware-in-Loop system indicate effectiveness of velocity prediction with root-mean-square error under 1.3 km/h. In the perspective of the energy conservation, the SMPC method can decrease energy consumption by 7.92% compared with traditional Rule-based methods, which is close to the optimization result of a conventional dynamic programming method. Further simulation and test results demonstrate that the proposed data-driven method is capable of realizing online accurate velocity prediction and energy management for real-world vehicles. • An energy management optimization method for four-wheel-independent-driving vehicles. • A data-driven Markov chain with is adopted for vehicle velocity prediction. • The stochastic model prediction control method is employed for torque allocation. • The proposed energy optimization method is verified through the Hardware-in-Loop test.

Fast alignment and comparison of RNA structures

To recognize remote relationships between RNA molecules, one must be able to align structures without regard to sequence similarity. We have implemented a method, which is swift [O(n(2))], sensitive and tolerant of large gaps and insertions. Molecules are broken into overlapping fragments, which are characterized by their memberships in a probabilistic classification based on local geometry and H-bonding descriptors. This leads to a probabilistic similarity measure that is used in a conventional dynamic programming method. Examples are given of database searching, the detection of structural similarities, which would not be found using sequence based methods, and comparisons with a previously published approach. Source code (C and perl) and binaries for linux are freely available at www.zbh.uni-hamburg.de/fries.

Hydropower Optimization for the Lower Seyhan Basin System in Turkey using dynamic programming

Dynamic programming with successive approximation has been used in the past for optimizing multi-reservoir water resources systems. In this study, a State Incremental Dynamic Programming (SIDP) model is developed for energy optimization of multi-reservoir systems. A random file access method is used to generate initial and intermediate data and cope with the curse of dimensionality of dynamic programming. The conventional dynamic programming method is used for each single reservoir to find the initial trajectory of the reservoirs. Then, the computer program developed in the study is applied to the multipurpose-multi-reservoir system in lower Seyhan basin, which comprises six reservoirs, some serial and some parallel. Extended historical flows are used to first maximize firm energy in the critical period, and then total energy over the entire period of flow records. The program is run with 50-year long segments (20 flow scenarios) of the synthetic flow data generated by using the hec-4 generalized computer program to account for the stochastic nature of streamflows. A 20% approximate increase in total energy is obtained by using the developed model for the lower seyhan basin system as compared to that calculated previously by conventional methods.

Hydropower Optimization for the Lower Seyhan System in Turkey using Dynamic Programming

Dynamic programming with successive approximation has been used in the past for optimizing multi-reservoir water resources systems. In this study, the State Incremental Dynamic Programming (SIDP) model is developed for energy optimization of multi-reservoir systems. A random file access method is used for reaching initial and intermediate data to cope with the curse of dimensionality of dynamic programming. A conventional dynamic programming method is used for each single reservoir to find the initial trajectory of the reservoirs. Then, the computer program developed in the study is applied to the multipurpose-multi-reservoir system in Lower Seyhan Basin, which has six reservoirs, some of which are serial and some parallel. First, extended historical flows were used to maximize firm energy in the critical period, and then total energy in the total flows. The program was run with 50-year long segments (20 flow scenarios) of the synthetic flow data generated by using the HEC-4 generalized computer program to take into account the stochastic nature of stream flows. An increment of approximately 20 percent in total energy was obtained by using the model for the Lower Seyhan System, as compared to that calculated previously by conventional methods.

An inverse dynamics-based dynamic programming method for optimal control problems of linear systems

In solving optimal control problems, the conventional dynamic programming method often requires interpolations to determine the optimal control law. As a consequence, interpolation errors often degenerate the accuracy of the conventional dynamic programming method. In view of this problem, this paper introduces an inverse dynamics-based dynamic programming method to eliminate the interpolation requirement for systems with linear dynamics. Simulation results show that the proposed approach provides more reliable solutions than the conventional dynamic programming method. Copyright © 1998 John Wiley & Sons, Ltd.

An inverse dynamics‐based dynamic programming method for optimal control problems of linear systems

In solving optimal control problems, the conventional dynamic programming method often requires interpolations to determine the optimal control law. As a consequence, interpolation errors often degenerate the accuracy of the conventional dynamic programming method. In view of this problem, this paper introduces an inverse dynamics-based dynamic programming method to eliminate the interpolation requirement for systems with linear dynamics. Simulation results show that the proposed approach provides more reliable solutions than the conventional dynamic programming method. Copyright © 1998 John Wiley & Sons, Ltd.

Path constrained time-optimal trajectory planning for X-Y robots via an efficient dynamic programming method

By exploiting an important property of the time optimal control law structure, this paper introduces a new dynamic programming approach for X-Y robot minimum time path following problem. Compared with the conventional dynamic programming method, the proposed approach significantly reduces the computational cost. The efficiency and effectiveness of the proposed approach are demonstrated via computer simulations. In addition, via experimental studies, this paper also addresses practical difficulties in implementing the minimum-time control law.

An inverse dynamic-based dynamic programming method for optimal point-to-point trajectory planning of robotic manipulators

Abstract This paper introduces an inverse dynamic-based dynamic programming (IDBDP) method for solving optimal point-to-point robot trajectory planning problems. Compared with the conventional dynamic programming method, the proposed method offers several advantages. First, it eliminates the interpolation requirement. Second, the proposed method requires only inverse dynamic computations. The requirement to integrate equations of motion is thus avoided. As a result, the IDBDP method is computationally more efficient than the conventional dynamic programming method. The reliability and efficiency of the IDBDP method have been tested via computer simulations

Optimal alignment between groups of sequences and its application to multiple sequence alignment.

Four algorithms, A-D, were developed to align two groups of biological sequences. Algorithm A is equivalent to the conventional dynamic programming method widely used for aligning ordinary sequences, whereas algorithms B-D are designed to evaluate the cost for a deletion/insertion more accurately when internal gaps are present in either or both groups of sequences. Rigorous optimization of the 'sum of pairs' (SP) score is achieved by algorithm D, whose average performance is close to O(MNL2), where M and N are numbers of sequences included in the two groups and L is the mean length of the sequences. Algorithm B uses some approximations to cope with profile-based operations, whereas algorithm C is a simpler variant of algorithm D. These group-to-group alignment algorithms were applied to multiple sequence alignment with two iterative strategies: a progressive method based on a given binary tree and a randomized grouping--realignment method. The advantages and disadvantages of the four algorithms are discussed on the basis of the results of examinations of several protein families.

Fuzzy dynamic programming: an application to unit commitment

A new approach using fuzzy dynamic programming is proposed for the unit commitment of a power system. A characteristic feature of the approach is that the errors in the forecast hourly loads can be taken into account by using fuzzy set notations, making the approach superior to the conventional dynamic programming method which assumes that the hourly loads are exactly known and there exist no errors in the forecast loads. To reach an optimal commitment strategy under the fuzzy environment, a fuzzy dynamic programming model in which the hourly loads, the cost, and system security are all expressed in fuzzy set notations is developed. The effectiveness of the approach is demonstrated by the unit commitment of the Taiwan power system, which contains six nuclear units, 48 thermal units, and 44 hydro units. >

Synthesis of Digital Control Systems for Nuclear Reactors, (I)

The present work is an attempt to synthesize an optimal closed-loop control system for controlling the transition from one power level to another in nuclear reactors. This first report covers a study on optimal solutions, undertaken with a view to gaining a better understanding of the basic characteristics possessed by such solutions, and to providing useful guides to the synthesis of closed loop control systems. The reactor system model adopted here is a three-dimensional nonlinear discrete-time system of about twenty sampling intervals, which makes it impracticable to apply the conventional dynamic programming method, on account of the excessive demands on computer time and core memories. To overcome this difficulty, a method termed “regionwise dynamic programming” is adopted, to handle the nonlinear multi-stage discrete time system. Some discussions are presented concerning the accuracies obtained and the range of applicability of the present method. The numerical solutions solved by this proposed method can be considered almost exact and physically reasonable. These solutions shall be utilized in a forthcoming paper for synthesizing control systems.

Synthesis of Digital Control Systems for Nuclear Reactors, (I)

The present work is an attempt to synthesize an optimal closed-loop control system for controlling the transition from one power level to another in nuclear reactors. This first report covers a study on optimal solutions, undertaken with a view to gaining a better understanding of the basic characteristics possessed by such solutions, and to providing useful guides to the synthesis of closed loop control systems. The reactor system model adopted here is a three-dimensional nonlinear discrete-time system of about twenty sampling intervals, which makes it impracticable to apply the conventional dynamic programming method, on account of the excessive demands on computer time and core memories. To overcome this difficulty, a method termed “regionwise dynamic programming” is adopted, to handle the nonlinear multi-stage discrete time system. Some discussions are presented concerning the accuracies obtained and the range of applicability of the present method. The numerical solutions solved by this proposed method can be considered almost exact and physically reasonable. These solutions shall be utilized in a forthcoming paper for synthesizing control systems.

Dynamic programming and the Hamilton-Jacobi method of classical mechanics

The conventional dynamic programming method for analytically solving a variational problem requires the determination of a particular solution, the optimal value function or return function, of the fundamental partial differential equation. Associated with it is another function, the optimal policy function. At each point, this function yields the value of the slope of the optimal curve to that point (or from that point, depending on the method of solution). The optimal curve itself can then be found by integration. In this paper, dynamic programming concepts and principles are used to develop two alternatives to the conventional method of solution. In the first method, a particular solution of two simultaneous partial differential equations is used to generate optimal curves by differentiations and solution of simultaneous equations. In the second method, any solution of the fundamental equation containing an appropriate number of arbitrary constants is sought. It is shown how such a function yields directly, by differentiations and solution of simultaneous equations, the optimal curve for a given problem. While the derivations to follow are new, the results are equivalent to those of a method due to Hamilton and its modification due to Jacobi.