Application Of Dynamic Programming Research Articles

Research and application of the flatness target curve discrete dynamic programming based on two-dimensional decision making

The flatness target curve (FTC) is a fundamental process model in cold rolling flatness control, which directly affects the quality control effect of the strip. The accurate and efficient setting of the flatness target curve is the key to ensuring continuous rolling and is a common problem faced by various production companies. This paper establishes a discrete dynamic planning (DDP) setting method for FTC with two-dimensional decision-making. To quantitatively analyze the roles of the coefficients in the FTC, the preprocessing process is established using standardized processing methods and statistical analysis methods. The flatness target curve discrete dynamic programming (FTC-DDP) is based on Bellman optimization theory with a dual structure of local and system decision-making. The decision-making functions are established based on the effect of each coefficient on FTC geometric characteristics. The concepts of “curvature factor” and “wedge factor” are innovatively proposed to obtain the decision-making value functions. The regression analysis establishes the solution equations of even number coefficients, odd number coefficients, and gain coefficients, respectively. Roll bending and tilting roll analyze the FTC’s ability to correct flatness defects. Meanwhile, utilizing a 1450 mm five-stand six-roll cold rolling mill, it is verified that FTC-DDP can quickly and accurately set the FTC, which provides an effective solution for obtaining high-precision and high-quality cold-rolled strips. Application results show that FTC-DDP can reduce the difficulty of setting up FTC by more than 70 %, reduce the setup time to milliseconds, and improve the setup accuracy by more than 50 %.

Various Applications of Dynamic Programming (D.P.) in the Iraqi Economy

The main objective of this research is to conduct some applications of dynamic programming in the Iraqi economy in order to determine the optimal solution, such as distributing investments among some economic sectors or finding the shortest distance between provinces using the Bellman-Ford algorithm. This includes providing some hypothetical examples of other dynamic programming methods such as the traveling salesman problem or the captain's ship model, as it is sometimes called. Dynamic programming revolves around cost reduction and maximizing returns, also known as optimality. Practical studies have been conducted on the shortest distance between provinces of Middle Euphrates in Iraq, in addition to applying the knapsack method to maximize returns in some economic sectors in Iraq for the year 2007.

A review of dynamic optimization in aquaculture production economics

AbstractDynamic optimization (DO) has been applied to aquaculture industry to determine optimal management strategies of aquaculture production systems. Optimal control of feeding, optimal stocking successions, and harvesting time among other factors has been studied. This work shows a review of the application of optimal control theory (OCT) and dynamic programming in economics and management of aquaculture. The Pontriagyn's maximum principle and the Bellman's optimality are the most commonly used mechanisms to solve control problems that optimize the producer's benefit or cost. Recently, model‐based predictive control has also been applied. Such tools have been used in aspects involving the design, planning, and monitoring of variables relevant to the optimal management of the culture system. It was found that in aquaculture there is low scientific productivity in the application of DO in bioeconomic models. In this review, an example of OCT applied to the control of feeding of farmed tilapia using a bioeconomic model is shown. DO represents a useful tool for optimal decision‐making and this review discusses the implications of the use of DO in aquaculture and recommendations for its use in the future.

Automated design of dynamic programming schemes for RNA folding with pseudoknots

Although RNA secondary structure prediction is a textbook application of dynamic programming (DP) and routine task in RNA structure analysis, it remains challenging whenever pseudoknots come into play. Since the prediction of pseudoknotted structures by minimizing (realistically modelled) energy is NP-hard, specialized algorithms have been proposed for restricted conformation classes that capture the most frequently observed configurations. To achieve good performance, these methods rely on specific and carefully hand-crafted DP schemes. In contrast, we generalize and fully automatize the design of DP pseudoknot prediction algorithms. For this purpose, we formalize the problem of designing DP algorithms for an (infinite) class of conformations, modeled by (a finite number of) fatgraphs, and automatically build DP schemes minimizing their algorithmic complexity. We propose an algorithm for the problem, based on the tree-decomposition of a well-chosen representative structure, which we simplify and reinterpret as a DP scheme. The algorithm is fixed-parameter tractable for the treewidth tw of the fatgraph, and its output represents a O(ntw+1)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\\mathcal {O}}(n^{tw+1})$$\\end{document} algorithm (and even possibly O(ntw)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\\mathcal {O}}(n^{tw})$$\\end{document} in simple energy models) for predicting the MFE folding of an RNA of length n. We demonstrate, for the most common pseudoknot classes, that our automatically generated algorithms achieve the same complexities as reported in the literature for hand-crafted schemes. Our framework supports general energy models, partition function computations, recursive substructures and partial folding, and could pave the way for algebraic dynamic programming beyond the context-free case.

Near-Optimal Scheduling for Crowdsourced Transit System With Skip-Stop Tactic

Efficient bus scheduling is a crucial component for the improvement of public transit services. Without systematic optimization and efficient shift arrangement, the bus scheduling system may suffer from poor vehicle loading rate or crowd onboard, resulting in wasted energy and passenger dissatisfaction. In this paper, we consider a crowdsourced bus service system (on a fixed route) that receives user requests as input and computes the scheduling of buses with flexible departure time and skip-stop to minimize the travel time of users. We first show that the general problem of computing the optimal scheduling is NP-hard. On the other hand, for the case when skip-stop is not adopted, we propose the Optimized Departure Time (ODT) algorithm that computes optimal scheduling. Our algorithm is built on an innovative reduction of the problem to some variants of the k-clustering problem and an efficient application of dynamic programming. On top of ODT, we further improve the effectiveness of the solution by utilizing the power of skip-stop tactic, named ODTS. Our experimental results demonstrate that ODT and ODTS dramatically outperform existing algorithms for the bus scheduling problem in terms of effectiveness and efficiency. Moreover, the solutions given by ODTS are very close to the optimum.

An application of dynamic programming to local adaptation decision-making

Adaptation decision-making in mountain regions necessitates dealing with uncertainties which are driven by the complex topography and the potential interconnections of stochastic events. Such events can lead to amplifying consequences for the exposed communities located at different elevations. In this study, we present a stylized application of stochastic dynamic programming for local adaptation decision-making for a small alpine community exposed to debris flows and floods. We assume that local decision-makers and planners aim at maximizing specific objectives by choosing from a feasible set of adaptation measures and under given constraints on these actions. Our results show that stochastic dynamic programming is a promising tool to address the underlying problem faced by local planners when evaluating the feasibility and effectiveness of adaptation measures. Furthermore, stochastic dynamic programming has some advantages compared to deterministic approaches which assume full knowledge of the system of interest in a world dominated by randomness. We provide an estimation of a best option and an appropriate metric to benchmark adaptation effectiveness for long time horizons. We show how multiple constraints, risk preferences, time horizons and decision periods all influence the decision-making and the overall success of adaptation responses over time.

Probability-Based Rule Curves for Multi-Purpose Reservoir System in the Seine River Basin, France

Multiple reservoir operation is of paramount importance due to tradeoffs in water supply and their cost functions. Understanding this complexity is important for optimizing water supply and increasing synergies gained from the joint operation. Therefore, this study aimed to develop a conceptual framework for addressing the effects of climate change on water security under the operating rules of the multiple reservoir system in northern France. A dynamic programming approach (DP) was employed to find the cost–benefit analysis that best fit with the objectives of reservoir operation, while the space rule was applied to balance the available space in each reservoir of a parallel system. A finite-horizon optimal regulation was then adopted for determining daily reservoir storage based on probability-based rule curves. The results indicated that the predicted inflow during the drawdown–refill cycle period to the Marne and Pannecière reservoirs would be the largest and lowest, respectively. The proposed upper rule curves during high-flow conditions suggested that the release from Aube reservoir should be postponed from July to August until September. At 50- and 100-year return periods, quite a high release rate from Seine and Marne reservoirs was observed during the dry season. A decrease in future water supply from Pannecière reservoir was found during summer, while the withdrawal in November could cause excessive water in the Seine tributary and Paris City. Under low-flow conditions in all return periods, the proposed lower rule curves recommended that the reservoir storage should go below the current operating rule, with a clear difference in July (the largest in Marne and the smallest in Pannecière) and almost no difference in November. Moreover, the web-based support system IRMaRA was developed for revising operating rules of four main reservoirs located in the Seine River Basin. The novelty of this modeling framework would contribute to the practice of deriving optimal operating rules for a multi-reservoir system by the probability-based rule curve method. Based on the evaluation of the effects of applying the estimated reservoir storage capacity under different return periods, both less overflow and water shortage represented by different levels of quantity and severity can be expected compared to the existing target storage at specified control points. Finally, the obtained finding revealed that the application of dynamic programming for reservoir optimization would help in developing a robust operating policy for tackling the effects of climate change.

On the application of modified dynamic programming in operational and organizational control

The methodology of application of dynamic programming and its modification in operational and organizational control is developed. To reduce dimensionality, computational difficulties, comparison of the methods and maximum use of all resources of computational means the principle of complexity is used. Computational procedures of dynamic programming are implemented in such a way that access to external computer memory is carried out as rarely as possible and at the same time all the output information is used effectively.

An application of approximate dynamic programming in multi-period multi-product advertising budgeting

<p style='text-indent:20px;'>Advertising has always been considered a key part of marketing strategy and played a prominent role in the success or failure of products. This paper investigates a multi-product and multi-period advertising budget allocation, determining the amount of advertising budget for each product through the time horizon. Imperative factors including life cycle stage, <inline-formula><tex-math id="M1">\begin{document}$ BCG $\end{document}</tex-math></inline-formula> matrix class, competitors' reactions, and budget constraints affect the joint chain of decisions for all products to maximize the total profits. To do so, we define a stochastic sequential resource allocation problem and use an approximate dynamic programming (<inline-formula><tex-math id="M2">\begin{document}$ ADP $\end{document}</tex-math></inline-formula>) algorithm to alleviate the huge size of the problem and multi-dimensional uncertainties of the environment. These uncertainties are the reactions of competitors based on the current status of the market and our decisions, as well as the stochastic effectiveness (rewards) of the taken action. We apply an approximate value iteration (<inline-formula><tex-math id="M3">\begin{document}$ AVI $\end{document}</tex-math></inline-formula>) algorithm on a numerical example and compare the results with four different policies to highlight our managerial contributions. In the end, the validity of our proposed approach is assessed against a genetic algorithm. To do so, we simplify the environment by fixing the competitor's reaction and considering a deterministic environment.</p>

DETERMINATION OF CONTROL FACTORS AFFECTING THE PROBABILITY OF A SUDDEN OUTBURST OF COAL AND GAS IN A BREAKAGE FACE

Purpose. To establish priority (control) factors affecting the probability of a sudden outburst of coal and gas in a breakage face, which will allow making optimal technological decisions when mining outburst-prone coal seams. Methods. An integrated method that involves the use of energy-force models to assess the likelihood of sudden outbursts of coal and gas; application of simulation modelling to determine priority control factors; application of an algorithm for enumeration of discrete values for each parameter, to find the average value of the increment of potential energy in a unit volume of coal in the working area of the longwall machine; application of dynamic programming to substantiate technological parameters (seam thickness, web width cutter-loader, length of a breakage face, etc.) mining of outburst-prone coal seams. Results. A mathematical model has been developed for the excitation of a sudden outburst of coal and gas, based on the energy theory of the occurrence of a sudden outburst of coal and gas. According to this model, the main sources of energy leading to the destruction of the seam are the energy of elastic deformation of the coal seam, the potential and kinetic energy of rocks, the internal energy of free and desorbed gas. On steeply falling seams, additional energy sources are the gravitational energy (positional energy) of the coal seam. The developed model allows establishing priority control factors and study the patterns of their change. Based on the setting of priority control factors, it is possible to choose the optimal technological schemes for mining outburst-prone coal seams. Scientific novelty. The scientific novelty consists in determining the priority control factors that affect the possibility of a sudden outburst of coal and gas, as well as establishing the pattern of their change. For this, a simulation algorithm was developed, which made it possible to establish the essential parameters of outburst-prone coal seams. Based on the algorithm for estimating the probability of a sudden outburst of coal and gas, the optimal technological parameters for mining of outburst-prone coal seams (web width, length of a breakage face, rate of advance) were determined. Practical implication. It consists in creating a package of applications for the selection of treatment equipment in accordance with certain technological solutions. On the basis of the data obtained, the optimal technological schemes for the mining of outburst-prone coal seams are substantiated, and mechanization means are selected based on the application of a package of applied programs.

New Results for Weakly Compatible (WC) and R-Weakly Commuting (RWC) Mappings with an Applicationin Dynamic Programming

The aim of this paper is to obtain some new results about common fixed points. Our results use weaker conditions than those previously used. We have relaxed the conditions for commutating pair mappings and compatible mappings of the type (A), which were introduced in 1976. The theorems are enriched by using the concept of WC and various types of weakly commuting pairs of maps in metric spaces. To discuss the existence and uniqueness of the common solutions, we have obtained an application to the functional equations in dynamic programming.

Fixed-Point Theorems in Fuzzy Normed Linear Spaces for Contractive Mappings with Applications to Dynamic-Programming

The aim of this paper is to provide new ways of dealing with dynamic programming using a context of newly proven results about fixed-point problems in linear spaces endowed with a fuzzy norm. In our paper, the general framework is set to fuzzy normed linear spaces as they are defined by Nădăban and Dzitac. When completeness is required, we will use the George and Veeramani (G-V) setup, which, for our purposes, we consider to be more suitable than Grabiec-completeness. As an important result of our work, we give an original proof for a version of Banach’s fixed-point principle on this particular setup of fuzzy normed spaces, a variant of Jungck’s fixed-point theorem in the same setup, and they are proved in G-V-complete fuzzy normed spaces, paving the way for future developments in various fields within this framework, where our application of dynamic programming makes a proper example. As the uniqueness of almost every dynamic programming problem is necessary, the fixed-point theorems represent an important tool in achieving that goal.

Model predictive control-based energy management strategy with vehicle speed prediction for hybrid electric vehicles

The speed of a hybrid electric vehicle is a critical factor that affects its energy management performance. In this study, we focus on the importance of solving the problem of inaccurate speed prediction in the energy management strategy (EMS) and application of dynamic programming (DP) needs to know the entire driving cycle. A gated recurrent unit neural network (GRU-NN) speed predictive model based on machine learning is developed by using the model predictive control (MPC) framework and solved in the prediction domain by employing DP. The neural network is trained on the training set, which is a collection of standard driving cycles. The results are compared with other two types of speed predictive models to verify the effects of different parameters of different speed predictive models on the state of charge and fuel consumption under Urban Dynamometer Driving Schedule driving cycle. Simulation shows that MPC based on the GRU-NN speed predictive model can effectively improve the fuel economy of hybrid electric vehicles, with a 94.14% fuel economy, which proves its application potential. Finally, the GRU-NN speed predictive model is applied under the Real-World Driving Cycle, whose fuel consumption has a fuel economy of 91.95% compared with that of the original rule-based EMS.

Optimal regimes of heavy oil transportation through a heated pipeline

Heavy oil with a high pour point and high viscosity is transported in a heated oil pipeline with many oil pumping and heating stations. Electricity and fuel costs account for a large share of total pumping cost. Minimizing the total operating cost of pumps and heating furnaces used to pump oil requires optimization of their hydrodynamic parameters, and, above all, the distribution of pressure and temperature along the pipeline. The application of graph theory and dynamic programming to determine the problem of the global optimum of energy consumption is given in a heated oil pipeline with many oil pumping and heating stations. The problem of mass and heat transport was solved numerically. The results of the optimization problem were applied to a digital section of a heated trunk interstate oil pipeline 450 km long. The results of thermohydraulic calculations were consistent with the actual data of the Supervisory Control and Data Acquisition (SCADA) system. Energy saving is 38.9 % in the optimal regime.

Dynamic Programming Based Rapid Energy Management of Hybrid Electric Vehicles with Constraints on Smooth Driving, Battery State-of-Charge and Battery State-of-Health

Dynamic programming (DP) is currently the reference optimal energy management approach for hybrid electric vehicles (HEVs). However, several research concerns arise regarding the effective application of DP for optimal HEV control problems which involve a significant number of control variables, state variables and optimization constraints. This paper deals with an optimal control problem for a full parallel P2 HEV with constraints on battery state-of-charge (SOC), battery lifetime in terms of state-of-health (SOH), and smooth driving in terms of the frequencies of internal combustion engine (ICE) activations and gear shifts over time. The DP formulation for the considered HEV control problem is outlined, yet its practical application is demonstrated as unfeasible due to a lack of computational power and memory in current desktop computers. To overcome this drawback, a computationally efficient version of DP is proposed which is named Slope-weighted Rapid Dynamic Programming (SRDP). Computational advantage is achieved by SRDP in considering only the most efficient HEV powertrain operating points rather than the full set of control variable values at each time instant of the drive cycle. A benchmark study simulating various drive cycles demonstrates that the introduced SRDP can achieve compliance with imposed control constraints on battery SOC, battery SOH and smooth driving. At the same time, SRDP can achieve up to 78% computational time saving compared with a baseline DP approach considering the Worldwide Harmonized Light Vehicle Test Procedure (WLTP). On the other hand, the increase in the fuel consumption estimated by SRDP is limited within 3.3% compared with the baseline DP approach if the US06 Supplemental Federal Test Procedure is considered. SRDP could thus be exploited to efficiently explore the large design space associated to HEV powertrains.

An Improved Strategy for Task Scheduling in the Parallel Computational Alignment of Multiple Sequences.

Task scheduling in parallel multiple sequence alignment (MSA) through improved dynamic programming optimization speeds up alignment processing. The increased importance of multiple matching sequences also needs the utilization of parallel processor systems. This dynamic algorithm proposes improved task scheduling in case of parallel MSA. Specifically, the alignment of several tertiary structured proteins is computationally complex than simple word-based MSA. Parallel task processing is computationally more efficient for protein-structured based superposition. The basic condition for the application of dynamic programming is also fulfilled, because the task scheduling problem has multiple possible solutions or options. Search space reduction for speedy processing of this algorithm is carried out through greedy strategy. Performance in terms of better results is ensured through computationally expensive recursive and iterative greedy approaches. Any optimal scheduling schemes show better performance in heterogeneous resources using CPU or GPU.

To the application of two-stage dynamic programming in the problem of sequential visiting of megalopolises

It is considered the routing problem solution in which the task set is the union of two subsets (zones); one of subsets should be serviced earlier than the second. It is supposed that each task is connected with the visit to a megalopolis (nonempty finite set) and fulfilment of works. The regular succession of task fulfilment should satisfy to precedence conditions realized in each of zones. The cost functions assume a task list dependence; cost aggregation is supposed additive. For solution, the two-stage dynamic programming procedure is used.

РОЗРОБКА ТА РЕАЛІЗАЦІЯ МОДЕЛІ ДЛЯ ОБҐРУНТУВАННЯ ОПТИМАЛЬНИХ ТЕХНОЛОГІЧНИХ СХЕМ ВІДПРАЦЮВАННЯ ВИКИДОНЕБЕЗПЕЧНИХ ВУГІЛЬНИХ ПЛАСТІВ

Purpose. The purpose of the work is to develop a new approach to substantiate the optimal technological schemes for the mining of outburst-prone coal seams, which will ensure the minimum likelihood of sudden outbursts of coal and gas in the breakage face. Methodology. The integrated approach has been applied, involving the use of graphs, network models, optimization algorithms to find optimal technological solutions; application of energy-force models to assess the likelihood of sudden outbursts of coal and gas; application of dynamic programming to substantiate technological parameters (seam thickness, cutter-loader web width, length of a breakage face, etc.) for mining of outburst-prone coal seams. Results. In the course of the work, a new mathematical model has been obtained and implemented, which allows assessing the risks of a sudden outburst of coal and gas during the mining of coal seams. Based on the application of the model, an algorithm for substantiating technical and technological solutions has been developed. Technical solutions provide for the choice of means of mechanization of a breakage face, a mining schedule out a breakage face, measures for degassing a coal seam, while technological solutions provide for the justification of the length of a breakage face, the cutter-loader web width, and the speed of movement of the breakage face line. Scientific novelty. The originality lies in the creation of the mathematical model that allows assessing the likelihood of a sudden outburst of coal or gas, which made it possible to create a new approach based on the use of graphs and network models, as well as to determine the optimal technological solutions for mining coal seams. Practical significance. The practical value of the work lies in the creation of an application package for the selection of purification facility in accordance with certain technological solutions. The general algorithm for justifying technological schemes for mining outburst-prone coal seams involves the following sequence of actions: setting the initial parameters of a rock mass and breaking them into discrete values; for possible routes of the graph of technological solutions, the mathematical expectation and dispersion are determined; the probability of sudden outbursts of coal and gas for the studied parameters is determined; on the basis of the calculated probability, dependences of the occurrence of a coal or gas outburst are obtained in accordance with the values of the parameters (face length, the speed of movement of the breakage face line, etc.); on the basis of the data obtained, the optimal technological schemes for mining outburst-prone coal seams are substantiated; based on the application of a package of applied programs, mechanization means are selected. The scope of the above approach is not limited to the existing means of mechanization, mining and geological conditions of coal seams, existing technological schemes, and others.

Adaptive dynamic programming and deep reinforcement learning for the control of an unmanned surface vehicle: Experimental results

This paper presents a low-level controller for an unmanned surface vehicle based on adaptive dynamic programming and deep reinforcement learning. This approach uses a single deep neural network capable of self-learning a policy, and driving the surge speed and yaw dynamics of a vessel. A simulation of the vehicle mathematical model was used to train the neural network with the model-based backpropagation through time algorithm, capable of dealing with continuous action-spaces. The path-following control scenario is additionally addressed by combining the proposed low-level controller and a line-of-sight based guidance law with time-varying look-ahead distance. Simulation and real-world experimental results are presented to validate the control capabilities of the proposed approach and contribute to the diversity of validated applications of adaptive dynamic programming based control strategies. Results show the controller is capable of self-learning the policy to drive the surge speed and yaw dynamics, and has an improved performance in comparison to a standard controller.

Analysis and Application of Dynamic Programming

Dynamic programming is a part that is not well understood in the introductory learning of algorithms, but it is also a part worth learning. It has been successfully applied in many fields, such as gene sequencing, human flow control, and hydropower resource allocation. This article systematically explains the principle of dynamic programming. At the same time, through the comparison with other algorithms, we can deeply understand the nature of dynamic programming and its advantages and disadvantages in solving problems compared with other algorithms. Finally, it analyzes the problem-solving methods and steps of dynamic programming based on relevant application examples.