Discrete Dynamic Programming Research Articles

Research and application of the flatness target curve discrete dynamic programming based on two-dimensional decision making

The flatness target curve (FTC) is a fundamental process model in cold rolling flatness control, which directly affects the quality control effect of the strip. The accurate and efficient setting of the flatness target curve is the key to ensuring continuous rolling and is a common problem faced by various production companies. This paper establishes a discrete dynamic planning (DDP) setting method for FTC with two-dimensional decision-making. To quantitatively analyze the roles of the coefficients in the FTC, the preprocessing process is established using standardized processing methods and statistical analysis methods. The flatness target curve discrete dynamic programming (FTC-DDP) is based on Bellman optimization theory with a dual structure of local and system decision-making. The decision-making functions are established based on the effect of each coefficient on FTC geometric characteristics. The concepts of “curvature factor” and “wedge factor” are innovatively proposed to obtain the decision-making value functions. The regression analysis establishes the solution equations of even number coefficients, odd number coefficients, and gain coefficients, respectively. Roll bending and tilting roll analyze the FTC’s ability to correct flatness defects. Meanwhile, utilizing a 1450 mm five-stand six-roll cold rolling mill, it is verified that FTC-DDP can quickly and accurately set the FTC, which provides an effective solution for obtaining high-precision and high-quality cold-rolled strips. Application results show that FTC-DDP can reduce the difficulty of setting up FTC by more than 70 %, reduce the setup time to milliseconds, and improve the setup accuracy by more than 50 %.

Energy Management Systems’ Modeling and Optimization in Hybrid Electric Vehicles

Optimization studies for the energy management systems of hybrid electric powertrains have critical importance as an effective measure for vehicle manufacturers to reduce greenhouse gas emissions and fuel consumption due to increasingly stringent emission regulations in the automotive industry, strict fuel economy legislation, continuously rising oil prices, and increasing consumer awareness of global warming and environmental pollution. In this study, firstly, the mathematical model of the powertrain and the rule-based energy management system of the vehicle with a power-split hybrid electric vehicle configuration are developed in the Matlab/Simulink environment and verified with real test data from the vehicle dynamometer for the UDDS drive cycle. In this way, a realistic virtual test platform has been developed where the simulation results of the energy management systems based on discrete dynamic programming and Pontryagin’s minimum principle optimization can be used to train the artificial neural network-based energy management algorithms for hybrid electric vehicles. The average fuel consumption in relation to the break specific fuel consumption of the internal combustion engine and the total electrical energy consumption of the battery in relation to the operating efficiency of the electrical machines, obtained by comparing the simulation results at the initial battery charging conditions of the vehicle using different driving cycles, will be analyzed and the advantages of the different energy management techniques used will be evaluated.

SeaTE: Subjective ex ante Treatment Effect of Health on Retirement

The paper studies the effect of health on work among older workers by eliciting two- and four-year-ahead subjective probabilities of working under alternative health states. These measures predict work outcomes. Person-specific differences in working probabilities across health states can be interpreted as Subjective ex ante Treatment Effects (SeaTEs) in a potential outcomes framework and as taste for work within a discrete choice dynamic programming framework. There is substantial heterogeneity in expectations of work conditional on health. The paper shows how heterogeneity in taste for work correlated with health can bias regression estimates of the effect of health on retirement. (JEL D84, I12, J14, J22, J26)

Dynamic pricing using flexible heterogeneous sales response models

We combine nonparametric price response modeling and dynamic pricing. In particular, we model sales response for fast-moving consumer goods sold by a physical retailer using a Bayesian semiparametric approach and incorporate the price of the previous period as well as further time-dependent covariates. All nonlinear effects including the one-period lagged price dynamics are modeled via P-splines, and embedding the semiparametric model into a Hierarchical Bayesian framework enables the estimation of nonlinear heterogeneous (i.e., store-specific) immediate and lagged price effects. The nonlinear heterogeneous model specification is used for price optimization and allows the derivation of optimal price paths of brands for individual stores of retailers. In an empirical study, we demonstrate that our proposed model can provide higher expected profits compared to competing benchmark models, while at the same time not seriously suffering from boundary problems for optimized prices and sales quantities. Optimal price policies for brands are determined by a discrete dynamic programming algorithm.

Accept or reject a ride? This is the problem

PurposeOnline ride-hailing platforms match drivers with passengers by receiving ride requests from passengers and forwarding them to the nearest driver. In this context, the low acceptance rate of offers by drivers leads to friction in the process of driver and passenger matching. What policies by the platform may increase the acceptance rate and by how much? What factors influence drivers' decisions to accept or reject offers and how much? Are drivers more likely to turn down a ride offer because they know that by rejecting it, they can quickly receive another offer, or do they reject offers due to the availability of outside options? This paper aims to answer such questions using a novel dataset from Tapsi, a ride-hailing platform located in Iran.Design/methodology/approachThe authors specify a structural discrete dynamic programming model to evaluate how drivers decide whether to accept or reject a ride offer. Using this model, the authors quantitatively measure the effect of different policies that increase the acceptance rate. In this model, drivers compare the value of each ride offer with the value of outside options and the value of waiting for better offers before making a decision. The authors use the simulated method of moments (SMM) method to match the dynamic model with the data from Tapsi and estimate the model's parameters.FindingsThe authors find that the low driver acceptance rate is mainly due to the availability of a variety of outside options. Therefore, even hiding information from or imposing fines on drivers who reject ride offers cannot motivate drivers to accept more offers and does not affect drivers' welfare by a large amount. The results show that by hiding the information, the average acceptance rate increases by about 1.81 percentage point; while, it is 4.5 percentage points if there were no outside options. Moreover, results show that the imposition of a 10-min delay penalty increases acceptance rate by only 0.07 percentage points.Originality/valueTo answer the questions of the paper, the authors use a novel and new dataset from a ride-hailing company, Tapsi, located in a Middle East country, Iran and specify a structural discrete dynamic programming model to evaluate how drivers decide whether to accept or reject a ride offer. Using this model, the authors quantitatively measure the effect of different policies that could potentially increase the acceptance rate.

Local regularity estimates for general discrete dynamic programming equations

We obtain an analytic proof for asymptotic Hölder estimate and Harnack's inequality for solutions to a discrete dynamic programming equation. The results also generalize to functions satisfying Pucci-type inequalities for discrete extremal operators. Thus the results cover a quite general class of equations.

Discrete LQR and ILQR methods based on high order Runge-Kutta discretizations

In this paper, discrete linear quadratic regulator (DLQR) and iterative linear quadratic regulator (ILQR) methods based on high-order Runge-Kutta (RK) discretizations are proposed for solving linear and nonlinear quadratic optimal control problems respectively. As discovered in Hager (2000) [11], direct approaches with RK discretization are equivalent with indirect approaches based on symplectic partitioned Runge-Kutta (SPRK) integration. In this paper, we will reconstruct this equivalence by the analogue of continuous and discrete dynamic programming. Then, based on the equivalence, we discuss the issue that the internal-stage controls produced by direct approaches may have lower order accuracy than the RK method used. We propose order conditions for internal-stage controls and then demonstrate that third or fourth order explicit RK methods cannot avoid the order reduction phenomenon. To overcome this obstacle, we calculate node control instead of internal-stage controls in DLQR and ILQR methods. Another advantage of our methods is high computational efficiency which comes from the usage of feedback technique. A simple numerical example will illustrate the validity of ILQR in solving nonlinear optimal control problems with high-order RK discretizations. In this paper, we also demonstrate that ILQR is essentially a quasi-Newton method with linear convergence rate.

A Singular Stochastic Control Approach for Optimal Pairs Trading with Proportional Transaction Costs

Optimal trading strategies for pairs trading have been studied by models that try to find either optimal shares of stocks by assuming no transaction costs or optimal timing of trading fixed numbers of shares of stocks with transaction costs. To find optimal strategies that determine optimally both trade times and number of shares in a pairs trading process, we use a singular stochastic control approach to study an optimal pairs trading problem with proportional transaction costs. Assuming a cointegrated relationship for a pair of stock log-prices, we consider a portfolio optimization problem that involves dynamic trading strategies with proportional transaction costs. We show that the value function of the control problem is the unique viscosity solution of a nonlinear quasi-variational inequality, which is equivalent to a free boundary problem for the singular stochastic control value function. We then develop a discrete time dynamic programming algorithm to compute the transaction regions, and show the convergence of the discretization scheme. We illustrate our approach with numerical examples and discuss the impact of different parameters on transaction regions. We study the out-of-sample performance in an empirical study that consists of six pairs of U.S. stocks selected from different industry sectors, and demonstrate the efficiency of the optimal strategy.

Multi-objective water-sediment optimal operation of cascade reservoirs in the Yellow River Basin

Sediment deposition in sediment-laden rivers seriously restricts the normal operation of reservoirs and the socioeconomic development of the basin. Studying the water–sediment joint operation of cascade reservoirs is important for extending the life of reservoirs. The objective of this paper is to provide a sound scientific approach to optimize the water–sediment operation of cascade reservoirs considering multiple objectives. An innovative time-dependent objective function was proposed, in which the sediment release efficiency equations suitable for different sediment inflow scenarios were introduced for sediment flushing calculation. The Sanmenxia (SMX) and Xiaolangdi (XLD) Reservoirs on the Yellow River were selected as case studies to test the proposed method. First, two typical data sets, including high and low flow magnitudes, were taken as model input scenarios. Second, the multiple-objective joint optimal operation model was established. Discrete differential dynamic programming (DDDP) was used to solve the operation model. Finally, the operation results based on different weights were discussed. Results indicate that, for sediment-laden rivers, such as the Yellow River, the proposed model can achieve the maximization of the hydropower generation and sediment flushing benefits for the high flow and low flow series. And the proposed model can guarantee the flood control, ice flood control, water supply, and ecological objectives. For the SMX-XLD cascade reservoirs, when the water resources are abundant, the hydropower generation reservoirs increase significantly. And in flood season, the weight for sediment flushing can be increased appropriately to improve the comprehensive benefits of the reservoir in sediment-laden rivers. In addition, the time-dependent objective function can provide a reference for the multi-objective optimal operation of cascade reservoirs. The optimal operation model provides a model basis for the water–sediment-energy joint operation of large-scale reservoir group in the sediment-laden basins.

Efficiency of cluster schemes implementation for discrete dynamic programming for systems of support of dispatching technological processes in transport systems

Within the framework of a computationally complex canonical scheduling problem, formulated by an optimization model for one-processor servicing of a finite deterministic flow of objects, a scheme of computational process of an algorithm of discrete dynamic programming in cluster implementation is considered. Variants of balancing of computational subtasks over network cluster array are investigated, purposed to reduce the volume and intensity of intranetwork interaction. It has been established that for practical improvement of efficiency of cluster algorithm, it is required not to increase the uniformity of distribution of subtasks among the cluster nodes, but to minimize the network traffic between the cluster nodes. Balancing options are proposed that allow to significantly increase localization of data in network computing. Experimental results are analytically confirmed, showing the scaling limits of implementation of discrete dynamic programming algorithms on a cluster architecture. The method for choosing the number of computational nodes and dimension of the problem being solved, which provide a threefold reduction in overhead costs for network exchange, is shown. The results obtained make it possible to objectively substantiate the choice of methodological and algorithmic approaches when choosing computer tools developing architectural and technological solutions for dispatching systems support in inland water transport.

An integrated energy-efficient train operation approach based on the space-time-speed network methodology

Reduction on the traction energy and increasing of the reused regenerative energy are two main ways for saving energy in metro systems, which are related to the driving strategy as well as the train timetable. To minimize the systematic energy, this paper proposes an integrated energy-efficient train operation method in which the driving strategy and the train timetable are jointly optimized. Firstly, the models of calculating the traction energy and the reuse of the regenerate energy are introduced with the constraints of the train operation. Then, the systematical optimization model is formulated by taking the net energy (i.e., the difference between the traction energy and the reused regenerate energy) as the objective function. Based on the Space–-Time-Speed network methodology, the optimization model is transformed into a discrete decision problem. Next, two algorithms are used to solve the problem. The dynamic programming algorithm is used to obtain the global optimal solution, and the discrete differential dynamic programming algorithm is applied to get the approximate optimal solution to reduce the computing time. Finally, two numeral examples are conducted to illustrate the effectiveness of the proposed method on energy saving. The method can reduce the net energy consumption by up to 25.0% compared to the result without optimization and by up to 8.7% compared to the result by using the two-stage method.

Method for high-dimensional hydropower system operations coupling random sampling with feasible region identification

Optimal operations of high-dimensional hydropower systems (HDHSs) with dozens or hundreds of plants face a serious challenge called “Curse of dimensionality” because the computational time and storage memory grow exponentially with the plant number. This study develops an efficient method to alleviate the dimensionality challenge. A random sampling (RS) technique is exploited to select representative water level states at each period by considering discrepant weights for different solution ranges. The RS can dynamically adjust sampling size and times with the selected samples. Moreover, acceptability and reliability parameters are introduced into the RS optimization to ensure a reasonable reliability level of the results. A feasible region identification (FRI) method is well designed to narrow the search range by transforming various operation constraints into equivalent water level limits and integrating them with water level range got from hundreds of simulation calculations. Furthermore, this method can use the water levels obtained in last iteration to dynamically update feasible regions during the calculation process. Three case studies involving a large-scale hydropower system with 21 plants are presented to verify the validity, efficiency, and sensitivity of the proposed method. Compared to dynamic programming (DP), discrete differential dynamic programming (DDDP) and progressive optimality algorithm (POA), the method requires only 0.85%, 1.27% and 3.14% of the computational effort to produce almost the same results. Moreover, a sensitivity analysis indicates that the reliability parameter in this method has a larger impact on the computational time than other parameters.

Robust operation interval of a large-scale hydro-photovoltaic power system to cope with emergencies

The joint operation of multiple renewable energies has become a promising approach to promote the penetration of renewables into power systems, where multiple uncertainties are unavoidably involved. Uncertainties caused by emergencies can significantly affect power system operation. However, the traditional single operation trajectory, which only considers pre-scribed uncertainties, is not enough to cope with emergencies. This study aims to propose a robust operation interval to deal with uncertainties caused by emergencies. First, a multi-objective model is developed to derive the robust operation interval considering the economy (power generation), flexibility (robust operation interval width) as well as reliability (portion of feasible solutions). Second, a two-layered nested framework is used by coupling non-dominated sorting genetic algorithm II and discrete differential dynamic programming in a hierarchical structure to improve calculation efficiency. Finally, the stochastic simulations are used to validate the effectiveness of the robust operation interval. Results for a case study using China’s Longyangxia hydro–photovoltaic power plant indicated that the proposed method could derive the robust operation interval effectively. The multi-objective optimization revealed that the interval’s economy was in conflict with both the flexibility and reliability. The robust operation interval reduced 4.32% loss of annual power generation by adjusting 0.14% annual reservoir discharge compared with the pre-scribed trajectory in a plague emergency, and decreased 8.99% loss of quarterly power generation by adjusting 1.95% quarterly discharge in an earthquake emergency. The proposed robust operation interval effectively deals with uncertainties caused by emergencies.

An optimization model for joint power generation dispatching of the cascade hydropower stations on the lower reaches of Jinsha River and the Yangtze River

The Xiluodu, Xiangjiaba, Three Gorges and Gezhouba cascade hydropower stations, which located at the lower reaches of Jinsha River and the upper reaches of Yangtze River, plays a very important role in the electrical power system in China. To optimize and solve the joint power generation problem of the cascade system with highly complex constraints, hydraulic relationship and multiple dispatching objectives, this research focus on: (1) a mid-and-long term power generation optimization model for the cascade system is established, considering hydraulic and electricty connections, as well as the flood control, water supply and ecology objectives; (2) an improved LSSDC-DDDP hybrid optimization framework, based on the theory of large-scale system decomposition coordination (LSSDC) and the discrete differential dynamic programming (DDDP), is designed to well solving the proposed optimization model. In the optimization framework, a relative coefficient is used to calculate the coordination factor among subsystems (a single hydropower station); meanwhile, an adaptive corridor and a bias corridor are adopted to enhance convergence speed of DDDP. The optimization model is applied in the dispatching calculation of the cascade system using the 2015-2017 actual runoff conditions, respectively. The results indicate that joint optimal dispatching of the cascade system can theoretically increase its yearly total power generation benefit about 2.73 billion yuan on average, simultaneously satisfying the requirements of downstream water supply and improving the total firm output guarantee rate of the system.

Dynamic programming with state-dependent discounting

This paper extends the core results of discrete time infinite horizon dynamic programming to the case of state-dependent discounting. We obtain a condition on the discount factor process under which all of the standard optimality results can be recovered. We also show that the condition cannot be significantly weakened. Our framework is general enough to handle complications such as recursive preferences and unbounded rewards. Economic and financial applications are discussed.

Solving High-Dimensional Dynamic Portfolio Choice Models with Hierarchical B-Splines on Sparse Grids

Discrete time dynamic programming to solve dynamic portfolio choice models has three immanent issues: firstly, the curse of dimensionality prohibits more than a handful of continuous states. Secondly, in higher dimensions, even regular sparse grid discretizations need too many grid points for sufficiently accurate approximations of the value function. Thirdly, the models usually require continuous control variables, and hence gradient-based optimization with smooth approximations of the value function is necessary to obtain accurate solutions to the optimization problem. For the first time, we enable accurate and fast numerical solutions with gradient-based optimization while still allowing for spatial adaptivity using hierarchical B-splines on sparse grids. When compared to the standard linear bases on sparse grids or finite difference approximations of the gradient, our approach saves an order of magnitude in total computational complexity for a representative dynamic portfolio choice model with varying state space dimensionality, stochastic sample space, and choice variables.

On History-Dependent Optimization Models: A Unified Framework to Analyze Models with Habits, Satiation and Optimal Growth

In this paper, we first provide a framework for history-dependent utility models. We further consider discrete infinite horizon dynamic optimization programs in which the instantaneous payoff presents such historydependence. Habits are a particular case of history-dependence. An issue about models with habits is their lack of general framework. Our framework allows to study habit models that are either additive or multiplicative or neither, as well as satiation models. Moreover, with this unified setting one can treat the usual optimal growth models with or without habit formation and with or without satiation effects. As the way the history dependence is formalized allows us to use dynamic programming tools. We show that the value function is the unique fixed point of the Bellman operator. Such history-dependent modelizations have their motivations and applications in many areas among which decision theory, psychology, behavioral and environmental economics.

Stabilization on Periodic Impulse Control Systems

This paper studies the stabilization for a kind of linear and impulse control system in finite-dimensional space where impulse instants appear periodically. We present several characterizations on the stabilization; show how to design feedback laws, and provide locations for impulse instants to ensure the stabilization. In the proofs of these results, we set up a discrete LQ problem, derive a discrete dynamic programming principle, build up a variant of Riccati's equation, apply repeatedly the Kalman controllability decomposition, and use a controllability result built up in [S. Qin and G. Wang, J. Differential Equations, 263 (2017), pp. 6456--6493].

ТЕХНОЛОГІЯ МОДЕЛЮВАННЯ ІНТЕГРАЦІЙНИХ ПРОЦЕСІВ ДЛЯ ПІДВИЩЕННЯ РІВНЯ БЕЗПЕКИ ДОРОЖНЬОГО РУХУ ТРАНСПОРТНИХ ЗАСОБІВ

The article is devoted to the development of technologies for modeling integration processes that synergistically affect the level of road safety of vehicles in terms of risk and uncertainty of non-stationary environmental factors. The mathematical description of the given sphere and information space of interaction of road users in the zones of the greatest probability of commission of road accidents is formalized that is shown by statistics of supervision. The essence, peculiarity and specificity of situational modes of dynamic, continuous interaction between road users, drivers and environmental factors in the space-time discrete cells of the electronic map of the critical zone on the basis of the frequency of previous accidents are substantiated. Analytical means of formation of transversal trajectories of safe movement without accidents and catastrophes are offered, which are guaranteed by the corresponding on-board information controlled complexes (BICC) of vehicles. Active targeted management within the system of navigation and control of mobile objects is implemented with a bias on the criteria to increase the level of road safety directly in the current critical area of the road network. A method of discrete dynamic programming of processes of synthesis and realization of controlled motion on a guaranteed safe transverse trajectory has been developed, which is locally applied only situationally within the transient mode of evasion from approach to threats with shock contacts. The technology of parameterization of predicted mutual maneuvering functions in the conditions of individual restrictions on the local space-time zone of own motion for a pair of vehicles that are really close to each other is formalized. BICC information and analytical tools ensure the reliability of estimates of the interval of endurance according to the prejudiced piecewise continuous transverse curve, which also provides local points of avoidance of collisions or the passage of forbidden local microzones. The only methods of modeling different vehicles by each BICC actively synthesize paired microphase spatial divisions of their own trajectories, which, due to the presence of risks, record the absence of traffic accidents with loss of safety within nonstationary flows due to the coordinated multiple heterogeneous interaction under similar conditions of field approximations. KEY WORDS: UNPRECEDENTED SAFETY, TRAFFIC, GUARANTEED MANAGEMENT, COLLISION PREVENTION, REASONABLE DYNAMICS.

Power and Resourse Efficient Envoronmentally Safe Technology for Processing Dumps of Technogenic Waste From Ore-Dressing and Processing Enterprises


 
 
 This research proposes a systematic approach for the analysis of volumes, physicochemical, granulometric, lithologic and thermal characteristics of waste from ore-dressing and processing enterprises stored in the dumps (tailing dumps) of ore-dressing and processing plants to assess the economic potential of its use in the system of complex power and resource efficient environmentally safe processing including palletizing machines, conveyor indurating machines and ore–thermal furnaces. The obtained results allow the authors to formulate the basic engineering, technological, economic and environmental requirements for complex chemical and power engineering systems of processing technogenic waste from ore-dressing and processing plants, these results make it also possible to define the degree of variability for the characteristics of the waste lots from various dumps. The paper describes the developed intensional and mathematical formulations for the multiscale problem of optimizing chemical and power engineering processes of technogenic raw materials processing in a complex chemical and power engineering system as a problem for discrete dynamic programming. The distinctive feature of this problem is to take into account the spatio-temporal multistage processing in a moving multilayer mass of pelletized raw material, the intensity of the process of internal moisture transfer and the variables for the control flow of the heat carrier gas. It allows increasing power efficiency by intensifying heat and mass transfer processes of multilayer drying, calcination and sintering. The criterion of the efficiency is the minimum cost of electric and thermal energy spent on processing. The obtained results were used to calculate power efficient environmentally safe processing of technogenic waste from ore-dressing and processing enterprises dumps. It was defined that heat and mass transfer processes are intensified, power consumption is reduced and the quality of the finished product is increased in the conditions of optimal power and resource efficient operation for the processing system.
 
 
 
 Keywords: tecnhogenic waste, waste processing, ore-dressing and processing plant, power and resource efficiency, optimization, system analysis, environmentally safety