Multidimensional Dynamic Programming Research Articles

Two-stage robust optimization of power cost minimization problem in gunbarrel natural gas networks by approximate dynamic programming

In short-term operation of natural gas network, the impact of demand uncertainty is not negligible. To address this issue we propose a two-stage robust model for power cost minimization problem in gunbarrel natural gas networks. The demands between pipelines and compressor stations are uncertain with a budget parameter, since it is unlikely that all the uncertain demands reach the maximal deviation simultaneously. During solving the two-stage robust model we encounter a bilevel problem which is challenging to solve. We formulate it as a multi-dimensional dynamic programming problem and propose approximate dynamic programming methods to accelerate the calculation. Numerical results based on real network in China show that we obtain a speed gain of 7 times faster in average without compromising optimality compared with original dynamic programming algorithm. Numerical results also verify the advantage of robust model compared with deterministic model when facing uncertainties. These findings offer short-term operation methods for gunbarrel natural gas network management to handle with uncertainties.

Research on Energy Management Strategy of Fuel Cell Vehicle Based on Multi-Dimensional Dynamic Programming

The powertrain of a fuel cell vehicle typically consists of two energy sources: a proton electrolyte membrane fuel cell (PEMFC) stack and a battery package. In this paper, multi-dimensional dynamic programming (MDDP) is used to solve the energy management strategy (EMS) of fuel cell hybrid powertrain. This study built a fuel cell hybrid powertrain model, in which the battery model is built based on the Thevenin equivalent circuit. In order to improve the calculating efficiency and maintain the accuracy of the algorithm, the state variables in each stage are divided into primary and secondary. In the reverse solution process, the corresponding relationship between the multi state variables grid and the optimal cumulative function has been changed from three-dimensional to two-dimensional. The EMS based on MDDP is applied to component sizing of a commercial vehicle. Simulations were conducted using MATLAB under the C-WTVC working condition. By analyzing the fuel economy and system durability, the optimal component combination of comprehensive performance is obtained. Compared with the EMS based on dynamic programming (DP), the proposed method effectively improves the calculation accuracy: the hydrogen consumption can be reduced by 3.10%, and the durability of the fuel cell and battery can be improved by 1.08% and 0.13%, respectively.

Improved multi-dimensional dynamic programming energy management strategy for a vehicle power-split hybrid powertrain

Dynamic programming is a widely used algorithm to solve the optimal control problem for hybrid electric vehicle in the whole driving scenario, but the problems of interpolation error and dimensional disaster have not been completely solved. This study proposes an adaptive adjustment method to solve the interpolation error problem, in which the solution without theoretical error can be obtained at the cost of a tiny driving cycle accuracy. Furthermore, a dynamic equivalent consumption factor calculation method is proposed to analyze the change of energy storage quality in the battery pack and the instantaneous equivalent consumption of the motors. The effectiveness of the improved algorithm is verified on a passenger car with hybrid powertrain using planetary gear mechanism. The results show that the established multi-dimensional dynamic programming algorithm has a relatively stable energy-saving ability. The maximum gap of fuel consumption rate is 3.2% in the four test driving cycles. Besides, there is only a maximum difference of 0.29‱ between the original driving cycles and the new cycles generated by the adaptive adjustment method. And obvious dynamic equivalent consumption factor changes of 15.7% in WLTC-class3, 10.3% in CLTC-P, 11.8% in FTP75 and 11.0% in NEDC are observed through the proposed calculation method.

Efficient Multidimensional Dynamic Programming-Based Energy Management Strategy for Global Composite Operating Cost Minimization for Fuel Cell Trams

Energy management is key to improve the performance of hybrid system. At present, the global optimization-based energy management strategy (EMS) is considered to have the best optimization effect under a known operating condition. However, the traditional global optimization-based EMS is often very time-consuming, which brings great inconvenience to debug and obtain the optimal result. Besides, in the literature, most works still introduce only the fuel consumption into the objective function while failing to consider the durability of fuel cells. In order to make up for these shortcomings, based on the modeling of hybrid system, this article first proposes a global composite operating cost minimization strategy (CMS) both considering fuel consumption and depreciation of purchase cost of fuel cells. Then, this article proposes computationally efficient multidimensional dynamic programming (EMDP) to resolve the optimal control trajectory of the studied issue. Finally, based on the hardware-in-the-loop platform, this article carries out comparative tests of the proposed EMDP algorithm and CMS strategy with traditional ones. Results show that the EMDP algorithm can effectively speed up the computation compared with the traditional dynamic programming. Besides, the proposed CMS strategy would lead to lower operating cost mainly by reducing the degradation cost of fuel cells so that the lifetime of fuel cells could also be extended.

Real-time schedule of integrated heat and power system: A multi-dimensional stochastic approximate dynamic programming approach

This paper proposes a multi-dimensional approximate dynamic programming (ADP) algorithm for the real-time schedule of integrated heat and power system (IHPS) with battery and heat storage tank (HST). The multi-time period optimization problem is reformulated under the Markov Decision Process. The high dimensional state variables are aggregated into the state of charge (SOC) of battery and overall available heat (OAH) of HST to reduce the computation of value function approximation (VFA) while ensuring approximate accuracy. Under sufficient training by uncertainty scenarios of wind power, electricity price, electrical and heat load, the approximate value function (AVF) can derive empirical knowledge and help IHPS make decisions to cope with uncertainties. The proposed ADP algorithm can efficiently take advantage of multi-energy integration and provide a near-optimal operation strategy to ensure the economy of IHPS by recursively solving the Bellman’s equation. Simulation results compared with existing methods validate the superiority of the proposed algorithm.

Individualized Pricing for a Cloud Provider Hosting Interactive Applications

We consider a cloud provider that hosts interactive applications, such as mobile apps and online games. Depending on the traffic of users for an application, the provider commits a subset of its resources (hardware capacity) to serve the application. The provider must choose a dynamic pricing mechanism to indirectly select the applications hosted and maximize revenue. We model the provider’s pricing problem as a large-scale stochastic dynamic program. To approach this problem, we propose a tractable approach to enable decomposing the multidimensional stochastic dynamic program into single-dimensional subproblems. We then extend the proposed framework to define an individualized dynamic pricing mechanism for the cloud provider. We present novel upper bounds on the optimal revenue to evaluate the performance of our pricing mechanism. The computational results show that a contract-based model of selling interactive cloud services achieves significantly greater revenue than the prevalent alternative and that our pricing scheme attains near-optimal revenue.

Optimization of Year-End Water Level of Multi-Year Regulating Reservoir in Cascade Hydropower System Considering the Inflow Frequency Difference

In order to analyze the year-end water level of multi-year regulating reservoir of the cascade hydropower system, this paper studied the joint operation optimization model of cascade reservoirs and its solving method based on multi-dimensional dynamic programming, and analyzed the power generation impact factors of cascade system that contains multi-year regulating reservoir. In particular, taking the seven reservoirs in the middle and lower reaches of Yalong River as an example, the optimal year-end water levels of multi-year regulating reservoir under the multi-year average situation and different inflow frequencies situation were studied. Based on the optimal calculation results of multi-dimensional dynamic programming, the inflow frequency difference considered operation rule of year-end water level of Lianghekou reservoir was extracted using the least square principle. The simulation results showed that, compared with the fixed year-end water level in multi-year, the extracted rule can improve the cascade power generation by more than 400 million kWh in an average year, representing an increase of 0.4%. This result means that the extracted rule can give full play to the regulation performance of multi-year regulating reservoir and improve the conversion efficiency of hydropower resources in cascade system. This is of great significance to the practical operation of cascade reservoirs system that contains multi-year regulating reservoir.

The Optimal Resource Self-configuration Method of Cognitive Network for Survivability Enhancement

In view of that general lack of intelligence and flexibility of the existing network resource allocation methods in the case of time-varying environments and diversified requirements, an efficient self-configuration method is put forward to optimize the allocation of resources and improve the survivability of system. First of all, the utility function of consumption domain is introduced as an indicator to pre-arrange the priority of user’s QoS, as a result, the utility maximization of the system under resource constraint is obtained. Then, based on this definition, a multidimensional dynamic programming framework is proposed to define and describe the self-configuration process, and the problem model is constructed under certain constraints. Furthermore, the adaptive adjustment and configuration of resources are implemented by determining the priority sequence of user services, finding the optimal resource configuration scheme, and optimizing the time configuration window. Finally, The simulation results show that the proposed method is superior to the traditional resource allocation scheme in terms of system reliability, connectivity, broadband utilization, average response time and transmission rate, which improves the system’s ability to adapt to the environment intelligently and survivability effectively.

Research and Application of Optimal Water Resources Allocation Based on Improved Multi-dimensional Dynamic Programming

To solve the problem of the multi-dimensional dynamic programming model which including three state variables and three decision variables in each stage, we have been using discrete differential dynamic programming (DDDP) algorithm. In this paper, considering the particularity of each variable of the objective function is independent in the same stage and the variables of the constraint condition is increasing step by step, we propose a one-dimensional dynamic programming recursive optimization method to obtain the initial solution of the DDDP algorithm.

Fast value iteration: an application of Legendre-Fenchel duality to a class of deterministic dynamic programming problems in discrete time

ABSTRACTWe propose an algorithm, which we call ‘Fast Value Iteration’ (FVI), to compute the value function of a deterministic infinite-horizon dynamic programming problem in discrete time. FVI is an efficient algorithm applicable to a class of multidimensional dynamic programming problems with concave return (or convex cost) functions and linear constraints. In this algorithm, a sequence of functions is generated starting from the zero function by repeatedly applying a simple algebraic rule involving the Legendre-Fenchel transform of the return function. The resulting sequence is guaranteed to converge, and the Legendre-Fenchel transform of the limiting function coincides with the value function.

Real-time stochastic optimal scheduling of large-scale electric vehicles: A multidimensional approximate dynamic programming approach

This paper studies the real-time charging optimization (RTCO) of large-scale electric vehicles (EVs), which is a multistage and multidimensional stochastic resource allocation problem. In order to handle the complex RTCO of large-scale EVs, a multidimensional approximate dynamic programming (ADP-RTCO) is designed for the sequent optimal decision makings. The hierarchy of ADP-RTCO contains two layers. In the upper layer, RTCO of large-scale EVs is formulated as a multidimensional energy storage problem by classifying EVs into several virtual EV clusters (EVCs). Then, temporal difference learning based policy iteration method is used to obtain the value function approximation for each EVC. In the lower layer, priority based reallocation algorithm is employed to obtain the detailed charging power for connected EVs. ADP-RTCO is designed with on-line learning ability to enhance its adaptability and robustness to ambiguous environment. Comprehensive simulation results demonstrate the optimality and robustness of ADP-RTCO in both cost-saving and load-flattening problems. Furthermore, ADP-RTCO can obtain higher-quality solution compared with other algorithms and is applicable for RTCO of large-scale EVs under uncertainties due to its high computation efficiency.

Individualized Pricing for a Cloud Provider Hosting Interactive Applications

We consider a cloud provider which hosts interactive applications such as mobile apps and online games. Depending on the traffic of users for an application, the provider commits a subset of its resources (hardware capacity) to serve the application. The provider must choose a dynamic pricing mechanism to indirectly select the applications hosted and maximize revenue. We model the provider's pricing problem as a large-scale stochastic dynamic program. To approach this problem, we propose a tractable approach to enable decomposing the multi-dimensional stochastic dynamic program into single-dimensional subproblems. We then extend the proposed framework to define an individualized dynamic pricing mechanism for the cloud provider. We present novel upper bounds on the optimal revenue to evaluate the performance of our pricing mechanism. The computational results show that a contract-based model of selling interactive cloud services achieves significantly greater revenue than the prevalent alternative, and that our pricing scheme attains near-optimal revenue.

Dynamic optimal decision making for manufacturers with limited attention based on sparse dynamic programming

In a fully competitive industry, the market demand is changing rapidly. Thus, it is important for manufacturers to manage their inventory effectively as well as to determine the best order quantity and optimal production strategy. In this paper, our concern is how shall a manufacturer with limited attention determine his optimal order quantity and optimal production strategy in an environment when many factors are volatile, such as the price of raw materials (respectively, finished goods) and attrition rate of inventory of raw materials (respectively, finished product). Under this environment, it is observed, according to various empirical studies, that decision makers tend to focus their attention on factors with major changes. Taking all these into account, our problem is formulated as a discrete-time stochastic dynamic programming. We propose a general approach based on the sparse dynamic programming method to solve this multidimensional dynamic programming problem. From the numerical examples solved using the proposed method, it is interesting to observe that decision makers with limited attention do not adjust their final decision when the volatility is small.

Studying Operation Rules of Cascade Reservoirs Based on Multi-Dimensional Dynamics Programming

Although many optimization models and methods are applied to the optimization of reservoir operation at present, the optimal operation decision that is made through these models and methods is just a retrospective review. Due to the limitation of hydrological prediction accuracy, it is practical and feasible to obtain the suboptimal or satisfactory solution by the established operation rules in the actual reservoir operation, especially for the mid- and long-term operation. In order to obtain the optimized sample data with global optimality; and make the extracted operation rules more reasonable and reliable, this paper presents the multi-dimensional dynamic programming model of the optimal joint operation of cascade reservoirs and provides the corresponding recursive equation and the specific solving steps. Taking Li Xianjiang cascade reservoirs as a case study, seven uncertain problems in the whole operation period of the cascade reservoirs are summarized after a detailed analysis to the obtained optimal sample data, and two sub-models are put forward to solve these uncertain problems. Finally, by dividing the whole operation period into four characteristic sections, this paper extracts the operation rules of each reservoir for each section respectively. When compared the simulation results of the extracted operation rules with the conventional joint operation method; the result indicates that the power generation of the obtained rules has a certain degree of improvement both in inspection years and typical years (i.e., wet year; normal year and dry year). So, the rationality and effectiveness of the extracted operation rules are verified by the comparative analysis.

Two Dimension Reduction Methods for Multi-Dimensional Dynamic Programming and Its Application in Cascade Reservoirs Operation Optimization

An efficient reservoir operation technique plays a very important role in improving the water resources and energy efficiency of reservoirs. In order to effectively avoid or alleviate the “curse of dimensionality” of Multi-dimensional Dynamic Programming (MDP) in the application of cascade reservoirs operation optimization (CROO) and keep a global convergence at the same time, two dimension reduction methods are proposed in this paper. One is a hybrid algorithm of MDP and a Progressive Optimality Algorithm (POA), named MDP-POA, which combines the global convergence of MDP and the strong local search ability of POA. MDP-POA first takes the global optimal trajectory of MDP in a low discrete degree as the initial trajectory of the POA, and then implements further optimization to the obtained initial trajectory by the POA with a high discrete degree, so as to avoid the “curse of dimensionality” of MDP in high discrete degree and the dependency of the POA for the initial trajectory. The other is an improved MDP (IMDP), which first constructs a corridor by the optimal trajectory of MDP in a lower discrete degree, and then implements further optimization in the corridor by MDP with a relatively high discrete degree, so as to avoid a large number of unnecessary calculations, and shorten the run-time effectively. In a case study, the results of MDP-POA, IMDP, and MDP are compared and analyzed from the aspects of power generation and run-time. The analysis indicates that the proposed MDP-POA and IMDP both have a good application effect and are worthy of further promotion.

Energy-Aware Data Allocation With Hybrid Memory for Mobile Cloud Systems

Resource scheduling is one of the most important issues in mobile cloud computing due to the constraints in memory, CPU, and bandwidth. High energy consumption and low performance of memory accesses have become overwhelming obstacles for chip multiprocessor (CMP) systems used in cloud systems. In order to address the daunting “memory wall” problem, hybrid on-chip memory architecture has been widely investigated recently. Due to its advantages in size, real-time predictability, power, and software controllability, scratchpad memory (SPM) is a promising technique to replace the hardware cache and bridge the processor–memory gap for CMP systems. In this paper, we present a novel hybrid on-chip SPM that consists of a static random access memory (RAM), a magnetic RAM (MRAM), and a zero-capacitor RAM for CMP systems by fully taking advantages of the benefits of each type of memory. To reduce memory access latency, energy consumption, and the number of write operations to MRAM, we also propose a novel multidimensional dynamic programming data allocation (MDPDA) algorithm to strategically allocate data blocks to each memory. Experimental results show that the proposed MDPDA algorithm can efficiently reduce the memory access cost and extend the lifetime of MRAM.

Solving Multi-Dimensional Dynamic Programming Problems Using Stochastic Grids and Nearest-Neighbor Interpolation

We propose two modifications to the method of endogenous grid points that greatly decreases the computational time for life cycle models with many exogenous state variables. First, we use simulated stochastic grids on the exogenous state variables. Second, when we interpolate to find the continuation value of the model, we split the interpolation step into two: We use nearest-neighbor interpolation over the exogenous state variables, and multilinear interpolation over the endogenous state variables. We evaluate the numerical accuracy and computational efficiency of the algorithm by solving a standard consumption/savings life-cycle model with an arbitrary number of exogenous state variables. The model with eight exogenous state variables is solved in around eight minutes on a standard desktop computer. We then use a more realistic income process estimated by Guvenen et al (2015) to demonstrate the usefulness of the algorithm. We demonstrate that the consumption dynamics differ compared to agents facing a more traditional income process.

Multidimensional Dynamic Programming Algorithm forN-Level Batching with Hierarchical Clustering Structure

This study focuses on theN-level batching problem with a hierarchical clustering structure. Clustering is the task of grouping a set of item types in such a way that item types in the same cluster are more similar (in some sense or another) to each other than to those in other clusters. In hierarchical clustering structure, more and more different item types are clustered together as the level of the hierarchy increases.N-level batching is the process by which items with different types are grouped into several batches passed from level 1 to levelNsequentially for given hierarchical clustering structure such that batches in each level should satisfy the maximum and minimum batch size requirements of the level. We consider two types of processing costs of the batches: unit processing cost and batch processing cost. We formulate theN-level batching problem with a hierarchical clustering structure as a nonlinear integer programming model with the objective of minimizing the total processing cost. To solve the problem optimally, we propose a multidimensional dynamic programming algorithm with an example.

Multi-Dimensional Dynamic Programming Algorithm for Input Lot Formation in a Semiconductor Wafer Fabrication Facility

Multi-Dimensional Dynamic Programming Algorithm for Input Lot Formation in a Semiconductor Wafer Fabrication Facility

Research and Application of Parallel Normal Cloud Mutation Shuffled Frog Leaping Algorithm in Cascade Reservoirs Optimal Operation

In order to improve the premature convergence problem of traditional shuffled frog leaping algorithm (SFLA), this paper proposed a normal cloud mutation shuffled frog leaping algorithm (NCM-SFLA) by mixing the cloud model algorithm (NCM) with SFLA algorithm, NCM is used to overcome the shortage of SFLA which is easy to fall into local optimal solution. The proposed NCM-SFLA has a good parallel characteristic, and the parallel computing can be implemented easily in multi core environment. In case study, this paper takes the Li Xianjiang cascade reservoirs in China as an instance to solve the cascade reservoirs operation optimization problem by the proposed NCM-SFLA. The results show that, compared with the Multi- dimensional Dynamic Programming (MDP), NCM-SFLA has the better global search ability and faster convergence speed, and the corresponding parallel computing can effectively shorten the run-time of NCM-SFLA. Therefore, the feasibility and rationality of the proposed NCM-SFLA and its parallel computing are effectively proved by the case study results.