Dynamic Programming Procedure Research Articles

Identifying and minimizing critical driving range thresholds for electric vehicles in intercity networks

It is well known that range anxiety faced by electric vehicle (EV) drivers nowadays can be effectively mitigated by either increasing the driving range of vehicles or increasing the distribution density of charging stations. To enact an effective research and development plan, vehicle manufacturers should understand well the relationship between the EV adoption and usage rates and the performance level of battery and energy management technologies. On the other hand, the underlying relationship between EV adoption and usage rates and the number and distribution of charging stations is also critical for charging infrastructure investors in determining their construction plans. This paper is dedicated to analytically identifying two critical driving range thresholds and further reducing these driving range thresholds as much as possible by optimally deploying charging station locations in intercity highway networks. These critical thresholds represent the turning points of individual routing convenience conditions or satisfaction levels for long-distance trips, implying that the inconvenient travel restrictions may be largely overcome if the driving range of EVs goes beyond these thresholds. For this purpose, we develop two integer programming models and dynamic programming algorithms for the driving range threshold identification problems, and extend the modeling framework to accommodate two relevant driving range threshold minimization problems as well, which are solved by a branching algorithm encapsulating the aforementioned dynamic programming procedures. The output of this research offers a set of new analytical tools for EV manufacturers to make strategic decisions on extending the driving range of their products and for charging infrastructure investors to make planning decisions on selecting charging station locations.

Improved dynamic programming algorithms for unconstrained two-dimensional guillotine cutting

In the unconstrained two-dimensional cutting problem (U2DCP), we are given a large rectangular sheet to be cut in order to extract small rectangular pieces, with no limits on the number (demand) of desired pieces. We face the variant with guillotine constraint, requiring to cut any rectangle in two parts through vertical/horizontal cuts with end points on the rectangle boundaries.For a given U2DCP instance, the dynamic programming approach can be used either to optimally solve it, or to obtain a full matrix of upper bounds suitable for the constrained variant of the problem where limits exist on the piece demands. The elements of the full matrix are also usable as partial solutions to build lower bounds for the non-guillotine variant.In this paper, we propose two major improvements to a dynamic programming procedure previously shown to be capable of solving very large size instances. First, we introduce a new option for one of the three conditions used for the anti-redundancy strategies on cut coordinates. Second, following the effort of the Operations Research community to exploit the feature of modern CPUs containing multi-core processors, we provide a parallelization scheme.An extended computational campaign is presented. We compare the upgraded procedure with its previous version on a single thread and with the currently state-of-the-art algorithm for multi-thread platforms, outperforming both in terms of execution time on average by a factor of 1.7 and 12, respectively, or for some problem instances up to 4.5 and 50, respectively. Moreover, the new procedure can solve two very large instances previously unsolved, as well as the new huge instances proposed in this paper.

Detailed Activity-Based Earthwork Scheduling Model to Aid during the Planning Stage of Road Construction

Earthwork scheduling (during the planning phase of road construction) is an important task that directly affects the cost and time of a project. However, the current scheduling methods are not performed at a detailed level and carry forward gaps from the actual schedule in the construction stage, causing problems, including time delays and additional costs, during the construction stage and thereby leading to the modification and supplementation of existing plans. Many studies related to scheduling have focused on process optimization or automation; therefore, the gaps between the planning and construction stages have not been studied well. These gaps must be determined in advance to solve the fundamental problem of earthwork scheduling in road construction. Therefore, this study proposes a detailed activity-based scheduling model for earthwork at the planning stage to minimize the gap between the planned schedule and the actual process at the construction stage. The proposed model comprises a detailed activity-based database of earthwork in road construction (DADER), which enables the subdivision of the existing earthwork schedule plan, and a dynamic programming (DP) procedure, which enables the combination of activities. The earthwork schedule at the case site, planned for only three activities, is subdivided using DADER into 36 activities. Additionally, the DP procedure is used to derive a combination of activity alternatives that minimizes the cost among the conditions that satisfied the input target duration. The model is verified through an expert survey using a 7-point Likert scale. Results show that the model has cost efficiency (4.19), onsite applicability (4.70), and task efficiency (4.48). The findings indicate that performing scheduling during the planning stage of earthwork projects can help reduce additional work caused by the gap between the site and the plan. This study will help improve the productivity of road construction projects by providing correct process optimization and automation research data.

Optimal truncated sequential test for exponential distribution

In order to save on test costs, the optimal truncated sequential test (OTST) for the parameter of the exponential distribution is studied. According to Bayes decision theory, dynamic programming methods and procedures are established to solve the OTST. Through comparison with the test plans provided by international standard IEC 61124 and Russian national standard GOST R 27.402, the results show that the OTSTs solved by our new method can control the error probabilities strictly and save more synthetical expected test time (SETT). The OTSTs are also compared with the near-optimal ones solved by the sample space sorting method (SSSM). The results show that the test plans solved by these two methods are almost consistent. However, our new method is much faster in computation, especially when the maximum sample size (MSS) of a truncated sequential test increases. It can save 74.23% of the computational time compared with SSSM when the MSS equals 15. At last, the MSS of OTST is optimized. By optimizing the MSS, the corresponding OTST can save about 5% of the SETT compared to the one with the minimum MSS.

An actor-critic algorithm with policy gradients to solve the job shop scheduling problem using deep double recurrent agents

In this work, we present a method that applies Deep Reinforcement Learning, an approximate dynamic programming procedure using deep neural networks, to the job shop scheduling problem (JSSP). The aim is to show that a greedy-like heuristic trained on a subset of problems, can effectively generalize to some extent to unseen instances, and be competitive compared to other methods. We model the JSSP as a Markov Decision Process and we exploit the efficacy of reinforcement learning to solve the problem. We adopt an actor-critic scheme based on policy gradients, specifically the Proximal Policy Gradient method, where the action taken by the agent is influenced by policy considerations on the state-value function. The procedures take into account the challenging nature of JSSP, where the state and the action space change for every instance and after each decision. To tackle this variability, we introduced a novel model based on two incident Long-Short Term Memory networks, followed by an encoding model, different in structure for both the actor and the critic. Experiments show the algorithm reaches good solutions in a short time, proving that is possible to generate new greedy heuristics just from learning-based methodologies. We compared our algorithms against several established heuristics, an adaptive method, a commercial solver based on branch and cut, and another approach based on Deep Reinforcement Learning, proving the validity of the proposed method in terms of time and makespan. The model can generalize, to some extent, to larger problems originating from a different distribution.

Optimization of Vehicle Trajectories Considering Uncertainty in Actuated Traffic Signal Timings

This paper introduces a robust green light optimal speed advisory (GLOSA) system for fixed and actuated traffic signals considering a probability distribution. These distributions represent the domain of possible switching times from the signal phasing and timing (SPaT) messages. The system finds the least-cost (minimum fuel consumption) vehicle trajectory using a computationally efficient A <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^\ast$</tex-math> </inline-formula> algorithm incorporated within a dynamic programming (DP) procedure to minimize the vehicle’s total fuel consumption. Constraints are introduced to ensure that vehicles do not collide with other vehicles, run red indications, or exceed a maximum vehicular jerk for passenger comfort. Results of simulation scenarios are evaluated against empirical comparable trajectories of uninformed drivers to compute fuel consumption savings. The proposed approach produced significant fuel savings compared to an uninformed driver behavior amounting to 37% on average for deterministic SPaT and 30% for stochastic SPaT data. A sensitivity analysis was performed to understand how the degree of uncertainty in SPaT predictions affects the optimal trajectory’s fuel consumption. The results present the required levels of confidence in these predictions to achieve savings in fuel consumption. Specifically, the study demonstrates that the proposed system can be within 85% of the maximum savings if the timing error is ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\pm$</tex-math> </inline-formula> 3.3 seconds) at a 95% confidence level. Results also emphasize the importance of more reliable SPaT predictions as the time to green decreases relative to the time the vehicle is expected to reach the intersection given its current speed.

A bottleneck routing problem with a system of priority tasks

We consider a minimax routing problem related to visiting megacities under precedence conditions and cost functions with task list dependence. It is supposed that some megacity system requiring visiting above all is selected. For solving, an approach with decomposition into a set of two minimax routing problems is proposed. A two-step widely understood dynamic programming procedure realizing an optimal composition solution is constructed. The above-mentioned optimality is established by theoretical methods. Application of the results obtained is possible under investigation of multi-stage processes connected with regular allocation of resources. Another variant of application concerns the particular case of one-element megacities (i.e., cities) and may be related to the issues of aviation logistics under organization of flights using one tool (airplane or helicopter) under system of tasks on the realization of passing cargo transportation with prioritization of visits realized above all.

ToMExO: A probabilistic tree-structured model for cancer progression.

Identifying the interrelations among cancer driver genes and the patterns in which the driver genes get mutated is critical for understanding cancer. In this paper, we study cross-sectional data from cohorts of tumors to identify the cancer-type (or subtype) specific process in which the cancer driver genes accumulate critical mutations. We model this mutation accumulation process using a tree, where each node includes a driver gene or a set of driver genes. A mutation in each node enables its children to have a chance of mutating. This model simultaneously explains the mutual exclusivity patterns observed in mutations in specific cancer genes (by its nodes) and the temporal order of events (by its edges). We introduce a computationally efficient dynamic programming procedure for calculating the likelihood of our noisy datasets and use it to build our Markov Chain Monte Carlo (MCMC) inference algorithm, ToMExO. Together with a set of engineered MCMC moves, our fast likelihood calculations enable us to work with datasets with hundreds of genes and thousands of tumors, which cannot be dealt with using available cancer progression analysis methods. We demonstrate our method's performance on several synthetic datasets covering various scenarios for cancer progression dynamics. Then, a comparison against two state-of-the-art methods on a moderate-size biological dataset shows the merits of our algorithm in identifying significant and valid patterns. Finally, we present our analyses of several large biological datasets, including colorectal cancer, glioblastoma, and pancreatic cancer. In all the analyses, we validate the results using a set of method-independent metrics testing the causality and significance of the relations identified by ToMExO or competing methods.

Iterated Maximum Large Neighborhood Search for the Traveling Salesman Problem with Time Windows and its Time-dependent Version

This article introduces a new algorithm for finding feasible or makespan-optimal solutions of Traveling Salesman Problems with Time Windows (TSPTWs) and Time-Dependent TSPTWs (TDTSPTWs). The algorithm starts from a sequence of visits of the customers involved in the problem, uses destroy and repair operations to iteratively improve this sequence, and applies perturbations to diversify search. For the destroy phase, customers are removed from the current sequence of visits as long as a parameter called the insertion-width is not too high. For the repair phase, the customers removed are reinserted for the best based on a dynamic programming procedure whose complexity is only linear in the number of customers. For the perturbation phase, some customers are randomly shifted in the sequence of visits. The algorithm obtained is called Iterated Maximum Large Neighborhood Search (ImaxLNS). On seven standard TSPTW benchmarks, it returns the best-known solution for each instance in less than one second on average. On two TDTSPTW benchmarks related to urban logistics, it provides new feasible solutions and best solutions. On a TDTSPTW benchmark related to Earth observing satellites, it solves most of the instances in less than a second.

BINOMIAL REAL OPTIONS MODEL WITH DYNAMIC PROGRAMMING APPLIED TO THE EVALUATION OF RAILWAY INFRASTRUCTURE PROJECTS IN BRAZIL

This paper presents a methodology for evaluating public investments in railway infrastructure based on the theory of real options, using the binomial model combined with dynamic programming procedures and Monte Carlo simulation. This modelling was proposed in order to overcome the inefficiencies in the evaluation process of projects conducted by the Brazilian public agencies, as pointed out by World Bank studies. According to these studies, there is evidence that the various managerial and regulatory instances of the Brazilian government have been limited to applying classical analysis techniques, considering only the discounted cash flow indicators, instead of paying more attention to risk issues, as well as to the possibilities of managerial flexibility. The proposed analytical procedures are recommended to support infrastructure investment decisions that can be transferred to the private sector or to guide the formation of public-private partnerships (PPPs).

To the application of two-stage dynamic programming in the problem of sequential visiting of megalopolises

It is considered the routing problem solution in which the task set is the union of two subsets (zones); one of subsets should be serviced earlier than the second. It is supposed that each task is connected with the visit to a megalopolis (nonempty finite set) and fulfilment of works. The regular succession of task fulfilment should satisfy to precedence conditions realized in each of zones. The cost functions assume a task list dependence; cost aggregation is supposed additive. For solution, the two-stage dynamic programming procedure is used.

Optimal Battery Dispatch Using Finite-Input Set Non-Linear Model Predictive Control: Algorithm Development and Case Study

The advancement in battery manufacturing has played a significant role in the use of batteries as a cost-effective energy storage system. This paper proposes an optimal charging and discharging strategy for the battery energy storage system deployed for economic dispatch and supply/demand balancing services in the presence of intermittent renewables such as solar photovoltaic systems. A decision-making strategy for battery charge/discharge operations in a discrete-time rolling horizon framework is developed as a finite-input set non-linear model predictive control instances and a dynamic programming procedure is proposed for its real-time implementation. The proposed scheme is tested on controllable loads and a photovoltaic generation scenario in the premises of a sports centre, as a part of a pilot demonstration of the inteGRIDy EU-funded project. The test results confirm that the implemented stacking of the battery and optimal decision-making algorithm can enhance net saving in the electricity bill of the sports centre, and lead to corresponding CO2 reductions.

Quantile Markov Decision Processes.

The goal of a traditional Markov decision process (MDP) is to maximize expected cumulative reward over a defined horizon (possibly infinite). In many applications, however, a decision maker may be interested in optimizing a specific quantile of the cumulative reward instead of its expectation. In this paper we consider the problem of optimizing the quantiles of the cumulative rewards of a Markov decision process (MDP), which we refer to as a quantile Markov decision process (QMDP). We provide analytical results characterizing the optimal QMDP value function and present a dynamic programming-based algorithm to solve for the optimal policy. The algorithm also extends to the MDP problem with a conditional value-at-risk (CVaR) objective. We illustrate the practical relevance of our model by evaluating it on an HIV treatment initiation problem, where patients aim to balance the potential benefits and risks of the treatment.

A new dynamic shape adjustment and placement algorithm for 3D yard allocation problem with time dimension

This paper studies a yard allocation problem at a container terminal, namely the 3D yard allocation problem with time dimension (3DYAPT), that determines the container storage locations in a given storage block to satisfy requirements from batches of arrived containers. The objective is to minimize the two-dimensional area of the storage block occupied for temporarily storing the containers within a given planning horizon (time dimension). The 3DYAPT is challenging and proved to be strongly NP-hard since it requires dynamically adjusting the shape of the allocated area when placing containers from the same request. We formulate the 3DYAPT as an integer linear programming model and develop a simulated annealing-based dynamic shape adjustment and placement algorithm (SA-DSAP). The simulated annealing-based algorithm comprises a novel dynamic programming procedure with several speed-up techniques that sequentially computes the storage space solution given a particular sequence of requests. Extensive computational experiments are conducted, showing that SA-DSAP is capable of finding optimal solutions very efficiently for nearly all small instances. For large instances, we also find that SA-DSAP produces significantly better heuristic solutions than the existing algorithm from the literature.

Time-Critical Influence Maximization in Social Networks with Time-Delayed Diffusion Process

Influence maximization is a problem of finding a small set of highly influential users in a social network such that the spread of influence under certain propagation models is maximized. In this paper, we consider time-critical influence maximization, in which one wants to maximize influence spread within a given deadline. Since timing is considered in the optimization, we also extend the Independent Cascade (IC) model to incorporate the time delay aspect of influence diffusion in social networks. We show that time-critical influence maximization under the time-delayed IC model maintains desired properties such as submodularity, which allows a greedy algorithm to achieve an approximation ratio of 1-1/e, to circumvent the NP-hardness of the problem. To overcome the inefficiency of the approximation algorithm, we design two heuristic algorithms: the first one is based on a dynamic programming procedure that computes exact influence in tree structures, while the second one converts the problem to one in the original IC model and then applies existing fast heuristics to it. Our simulation results demonstrate that our heuristics achieve the same level of influence spread as the greedy algorithm while running a few orders of magnitude faster, and they also outperform existing algorithms that disregard the deadline constraint and delays in diffusion.

Algorithm for Solving Problem of Designing Regional Logistics Infrastructure

The paper considers an algorithm for solving the problem of finding the optimal location of key objects of transport and warehouse infrastructures within the framework of a methodological approach to designing logistics infrastructure in the territory of the region of the countries. The methodological approach includes three stages. At the first stage, areas are determined where it is advisable to locate key objects of the regional logistics infrastructure. Further, using the models developed by the authors, the linking of warehouse infrastructure objects on the ground has been carried out and, taking into account the designed warehouse network, the optimal dislocation of transport infrastructure objects has been determined. To find the optimal locations for the objects for regional logistics infrastructure facilities, the authors propose an algorithm that is applicable both for building warehouse and transport infrastructures due to the similarity of the models. The algorithm is based on the method of constructing a sequence of plans. At the initial stage, the final expansion is constructed for the set of plans under consideration. For a given set, a minorant has been determined for the cost function associated with the placement and maintenance of infrastructure facilities, the movement of goods, and the haul of an empty vehicle. After that, an iterative algorithm has been formed that determines the sequence of optima of the minorant on a sequence of nested sets. At the first step, an element of the set of plans has been found that minimizes the minorant, at the next step, the found element is excluded from the set under consideration, and a new optimum is sought on the remaining set for which the minorant takes the minimum value. To eliminate multiple plans, it is advisable to use dynamic programming procedures. The limits of applicability of the method for constructing a sequence of plans are determined by the ability to construct an extension of the set of plans for placing objects, select a minorant on it, and build an algorithm for ordering optima.

Generalizations, formulations and subgradient based heuristic with dynamic programming procedure for target set selection problems

Target set selection problems model influence processes through networks. Usually, these problems seek a set of individuals to be initially influenced by some content and will help to propagate the content to other individuals of the network. This work considers generalizations for the minimization and maximization versions of these problems. We propose binary linear models and Lagrangian relaxations that, for general cases, offer strictly better bounds than the linear relaxations. To efficiently solve our relaxations we design dynamic programming algorithms and embedded them in subgradient methods to find the best possible bounds. We also use the subgradient methods to construct feasible heuristic solutions and, to assess the effectiveness of our approaches, we run computational experiments on instances generated from real-world datasets. The experiments indicate that our heuristics obtain almost optimal solutions with good bounds in a very short time and, in some cases, they guarantee optimality for the constructed solutions.

Production dynamics in case of organizational learning

Evidences indicate that around 50% of productivity increase originates from organizational learning and learning by doing. The speed of implementing productivity knowledge increase is a key component of competitiveness and for the case when the implementation is instantaneous, an optimal control model is developed. The analysis of this model indicates that, under certain circumstances, the optimal production level does not change over time, and utilizing this property, a closed form solution is provided to determine the optimal production level. When the implementation process happens step by step, a discrete time model is offered, and we found that using the same assumptions, the production dynamics will be decreasing. To determine the optimal production volumes, a backward dynamic programming procedure is presented.

Computation of expected shortfall by fast detection of worst scenarios

We consider multi-step algorithms for the computation of the historical expected shortfall. At each step of the algorithms, we use Monte Carlo simulations to reduce the number of historical scenarios that potentially belong to the set of worst-case scenarios. We show how this can be optimized by either solving a simple deterministic dynamic programming algorithm or in an adaptive way by using a stochastic dynamic programming procedure under a given prior. We prove -error bounds and numerical tests are performed.

A method for efficiently routing order pickers in the leaf warehouse

Warehouse order picking is a critical operation for every supply chain because of its direct influence on customer satisfaction and the high time investment that is usually required for completing it. Hence, improving order picking efficiency can enhance both customer satisfaction and warehouse throughput. The most critical step in order picking is the travelling along the aisles of the warehouse, which accounts for a major share of the total warehouse operating cost. To minimize this cost for the special case of a warehouse with a leaf layout, the paper at hand proposes an efficient optimal order picker routing policy based on an Eulerian graph and a dynamic programming procedure. It further develops four simple routing heuristics, referred to as leaf S-shape, leaf return, leaf midpoint, and leaf largest gap, for guiding the order picker through the leaf warehouse. The performance of the routing heuristics is compared to the exact algorithm proposed in this paper in a systematic numerical experiment. Finally, we present some managerial insights for managing order picking in the leaf warehouse.