Dynamic Programming Approach Research Articles

Energy‐efficient operation of medium‐speed maglev through integrated traction and train control

AbstractIn contrast to the wheel‐track trains, where the motor characteristics are considered a constant value, the motor characteristics of the medium‐speed maglev (MSM) trains are the dependent variable of the position. This article studies the integration of traction power control and train control for the MSM to minimize energy consumption. First, an innovative integrated energy‐efficient optimization model for MSM train control is constructed, considering the characteristics of the linear motors. Then, a multi‐level dynamic programming (DP) approach, which includes the train operation simulation with the linear motor, is developed to solve the optimization problem. Furthermore, a speed‐up strategy for the DP approach is proposed by a pre‐calculated target train speed band (TTSB). The results of numerical experiments show that the DP approach yields a more practical train speed profile. In contrast, the DP approach with the TTSB strategy can achieve a better trade‐off between solution accuracy and computational efficiency.

Active set prediction for nonlinear model predictive control on a shrinking horizon based on the principle of optimality

SummaryWe provide insights into the structure of the set of active constraints arising for optimal solutions to nonlinear model predictive control problems along a shrinking horizon. The principle of optimality combined with a particular order of the constraints allows the prediction of the future active sets without solving the corresponding optimization problem. By describing the development of optimal active sets along a shrinking horizon, we state an important relationship for transferring ideas such as dynamic programming approaches from the linear to the nonlinear case. We further use the information about active and inactive constraints to rearrange and remove constraints of the original nonlinear program as described in previous work and thus simplify the problem. Numerical experiments show for the problem class treated here that the inherent robustness coming with the regional characteristic of the active sets with respect to the state space makes this approach useful also if uncertainties are present.

Time‐inconsistent contract theory

AbstractThis paper investigates the moral hazard problem in finite horizon with both continuous and lump‐sum payments, involving a time‐inconsistent sophisticated agent and a standard utility maximizer principal: Building upon the so‐called dynamic programming approach in Cvitanić et al. (2018) and the recently available results in Hernández and Possamaï (2023), we present a methodology that covers the previous contracting problem. Our main contribution consists of a characterization of the moral hazard problem faced by the principal. In particular, it shows that under relatively mild technical conditions on the data of the problem, the supremum of the principal's expected utility over a smaller restricted family of contracts is equal to the supremum over all feasible contracts. Nevertheless, this characterization yields, as far as we know, a novel class of control problems that involve the control of a forward Volterra equation via Volterra‐type controls, and infinite‐dimensional stochastic target constraints. Despite the inherent challenges associated with such a problem, we study the solution under three different specifications of utility functions for both the agent and the principal, and draw qualitative implications from the form of the optimal contract. The general case remains the subject of future research. We illustrate some of our results in the context of a project selection contracting problem between an investor and a time‐inconsistent manager.

Robust reinsurance contract and investment with delay under mean-variance framework

We consider a (non) zero-sum differential game between two insurers and determine a robust reinsurance contract from joint interests of the insurers and the reinsurer under the framework of Stackelberg differential game, that is, a hybrid stochastic differential game framework is introduced in this article. We assume that each party can invest in a financial market consisting of a risk-free asset and a risky asset whose price evolution follows a stochastic volatility model, and the reinsurer charges the premium according to both the expected value premium principle and the variance premium principle. Since the performance-related capital inflow or outflow feature is introduced, the insurers and reinsurer all aim to maximize their own time-inconsistent mean-variance criterion with delay. Using the dynamic programming approach, we derive the equilibrium investment strategy, the robust equilibrium reinsurance contract, and the corresponding equilibrium value functions under two premium principles with certain conditions, respectively. Some interesting properties of the equilibrium investment strategy are proved explicitly, and several numerical examples and sensitivity analysis are presented to demonstrate the effects of model parameters on the equilibrium reinsurance contract.

Developing auto process mapping technique for powder bed fusion using an electron beam

Powder bed fusion using an electron beam offers promise for manufacturing intricate metal parts. However, process optimization for defect-free parts proves costly and time-consuming. Many studies have investigated process optimization and defect prediction methods, but automating process optimization remains a significant challenge. This study developed and validated software to automatically determine i + 1-th trial conditions based on the results of the i-th trial experiment. Two algorithms were implemented and evaluated:—a dynamic programming approach and a selecting boundary conditions approach. The latter method considerably reduced the time required to determine the next conditions compared to the former approach. Considering a process mapping experiment requiring real-time trial condition determination during the build, we chose the selecting boundary conditions approach. The selecting boundary conditions approach was used to conduct a process mapping experiment to validate the software for constructing a process map using machine learning. The model and hyperparameters were optimized using sequential model-based global optimization with a tree-structured Parzen estimator. The process map underwent four updates using the developed software to determine i + 1-th trial conditions and construct a process map from the results of the i-th trial experiment.

A simulation-based approximate dynamic programming approach to dynamic and stochastic resource-constrained multi-project scheduling problem

We consider the dynamic and stochastic resource-constrained multi-project scheduling problem which allows for the random arrival of projects and stochastic task durations. Completing projects generates rewards, which are reduced by a tardiness cost in the case of late completion. Multiple types of resource are available, and projects consume different amounts of these resources when under processing. The problem is modelled as an infinite-horizon discrete-time Markov decision process and seeks to maximise the expected discounted long-run profit. We use an approximate dynamic programming algorithm (ADP) with a linear approximation model which can be used for online decision making. Our approximation model uses project elements that are easily accessible by a decision-maker, with the model coefficients obtained offline via a combination of Monte Carlo simulation and least squares estimation. Our numerical study shows that ADP often statistically significantly outperforms the optimal reactive baseline algorithm (ORBA). In experiments on smaller problems however, both typically perform suboptimally compared to the optimal scheduler obtained by stochastic dynamic programming. ADP has an advantage over ORBA and dynamic programming in that ADP can be applied to larger problems. We also show that ADP generally produces statistically significantly higher profits than common algorithms used in practice, such as a rule-based algorithm and a reactive genetic algorithm.

Model-free frequency regulation in islanded microgrids: An event-triggered adaptive dynamic programming approach

This paper is concerned with frequency regulation in islanded microgrids via an event-triggered adaptive dynamic programming method. Comparatively to the existing works, we relax the typical limitations on the system, namely, exactly known models, constant load impedances, and single operation mode. The adopted frequency regulation scheme can reach frequency synchronization within the acceptable bounds in the presence of the model uncertainties, operation mode switching, and time-varying loads with no need for any prior information on the system parameters. By employing an event-triggered scheme, the secondary controller is updated only at the aperiodic triggering instants which contributes to less transmission cost. Besides, the use of measurements can help the adaptive weights to be adjusted online in the approximation process, which allows the proposed controller to adapt to the disturbances in real time. Finally, tests including the islanding process and load variation are performed in MATLAB/SimPowerSystem to show the effectiveness of the proposed method.

Approximate dynamic programming approach to efficient metro train timetabling and passenger flow control strategy with stop-skipping

Urban metro systems continuously face high travel demand during rush hours, which brings excessive energy waste and high risk to passengers. In order to alleviate passenger congestion, improve train service levels and reduce energy consumption, a nonlinear dynamic programming (DP) model of efficient metro train timetabling and passenger flow control strategy with stop-skipping is presented, which consists of state transition equations concerning train traffic and passenger load. To overcome the curse of dimensionality, the formulated nonlinear DP problem is transformed into a discrete Markov decision process, and a novel approximate dynamic programming (ADP) approach is designed based on the lookahead policy and linear parametric value function approximation. Finally, the effectiveness of this method is verified by three groups of numerical experiments. Compared with Particle Swarm Optimization (PSO) and Simulated Annealing (SA), the designed ADP approach could obtain high-quality solutions quickly, which makes it applicable to the practical implementation of metro operations.

Decomposition and approximate dynamic programming approach to optimization of train timetable and skip-stop plan for metro networks

Carefully coordinating train timetables of different operating lines can help reduce transfer delays, which in turn reduces station crowding and improves overall service quality. This paper explores the optimization to train timetable and skip-stop plans that aims to minimize the total passenger waiting time and station crowding. The problem is formulated as a mixed-integer non-linear programming model. To effectively address the complexity of the model, a decomposition and approximate dynamic programming approach is designed to convert the original network-level problem into a series of small-scale subproblems, one for each operating line, to be solved quickly in a distributed manner. The effectiveness and practicability of the model and algorithm are demonstrated on two case networks: a small-scale synthetic network of three metro lines and a real-world network based on Beijing metro. The computational results illustrate that the proposed strategy to generate train timetables and skip-stop plans can effectively reduce passenger waiting time and station crowing. The proposed decomposition and approximate dynamic programming approach is also shown to perform more efficiently than traditional heuristic algorithms, such as genetic algorithm and simulated annealing algorithm for large-scale networks.

An Extended McKean–Vlasov Dynamic Programming Approach to Robust Equilibrium Controls Under Ambiguous Covariance Matrix

An Extended McKean–Vlasov Dynamic Programming Approach to Robust Equilibrium Controls Under Ambiguous Covariance Matrix

Utilizing Dynamic Programming for Optional Shortest Route Determination

Fuel prices have seen a significant increase. This encourages individuals to save costs on their fuel expenditure by finding the shortest route to their destination. The shorter the route, the less fuel will be released. This study answers the challenge of determining the shortest route between BTN Lembah Furia Sentani in Jayapura Regency to the Department of Mathematics of Cenderawasih University in Papua Province. Many roads can be passed from BTN Lembah Furia Sentani in Jayapura Regency, to the Department of Mathematics of Cenderawasih University in Papua Province. As is well known, that the streets in Jayapura City have many alternative roads with different characters. The existing road characteristics include density, physical condition of the road and the size of the road width. With the existing road character, it can be assumed that each road has an average travel length. This average value is used as the cost of the road. To solve this problem, a dynamic programming approach is used to identify optimal routes while minimizing costs. The dynamic program utilizes analyzed and calculated data to yield the best outcomes. The results indicate that the quickest path from BTN Lembah Furia Sentani to the Mathematics Department at Cenderawasih University is via Komba Netar Bridge, GKI Petrus Waena Church, covering a distance of 25.2 kilometers

Optimal VIX-linked structure for the target benefit pension plan

AbstractIn this paper, we study the optimal VIX-linked target benefit (TB) pension design. By applying the dynamic programming approach, we show the optimal risk-sharing structure for the benefit payment exhibits a linear form that consists of three components: (1) a model-robust performance adjustment, (2) a counter-cyclical volatility adjustment that depends on the VIX index, and (3) a TB level that is partially indexed to the cost-of-living adjustment. Differences between our results and the previous literature are highlighted via both theoretical derivations and numerical illustrations.

A dynamic programming approach to multi-objective logic synthesis of quantum circuits

A dynamic programming approach to multi-objective logic synthesis of quantum circuits

A New Dynamic Programming Approach for Spanning Trees with Chain Constraints and Beyond

Short spanning trees subject to additional constraints are important building blocks in various approximation algorithms, and moreover, they capture interesting problem settings on their own. Especially in the context of the traveling salesman problem (TSP), new techniques for finding spanning trees with well-defined properties have been crucial in recent progress. We consider the problem of finding a spanning tree subject to constraints on the edges in a family of cuts forming a laminar family of small width. Our main contribution is a new dynamic programming approach in which the value of a table entry does not only depend on the values of previous table entries, as is usually the case, but also on a specific representative solution saved together with each table entry. This allows for handling a broad range of constraint types. In combination with other techniques—including negatively correlated rounding and a polyhedral approach that, in the problems we consider, allows for avoiding potential losses in the objective through the randomized rounding—we obtain several new results. We first present a quasi-polynomial time algorithm for the minimum chain-constrained spanning tree problem with an essentially optimal guarantee. More precisely, each chain constraint is violated by a factor of at most [Formula: see text], and the cost is no larger than that of an optimal solution not violating any chain constraint. The best previous procedure is a bicriteria approximation violating each chain constraint by up to a constant factor and losing another factor in the objective. Moreover, our approach can naturally handle lower bounds on the chain constraints, and it can be extended to constraints on cuts forming a laminar family of constant width. Furthermore, we show how our approach can also handle parity constraints (or, more precisely, a proxy thereof) as used in the context of (path) TSP and one of its generalizations and discuss implications in this context. Funding: This project received funding through the Swiss National Science Foundation [Grants 200021_184622 and P500PT_206742], the European Research Council under the European Union’s Horizon 2020 research and innovation program [Grant 817750], and the Deutsche Forschungsgemeinschaft (German Research Foundation) under Germany’s Excellence Strategy – EXC 2047/1 [Grant 390685813].

The dispersive art gallery problem

We introduce a new variant of the art gallery problem that comes from safety issues. In this variant we are not interested in guard sets of smallest cardinality, but in guard sets with largest possible distances between these guards. To the best of our knowledge, this variant has not been considered before. We call it the Dispersive Art Gallery Problem. In particular, in the dispersive art gallery problem we are given a polygon P and a real number ℓ, and want to decide whether P has a guard set such that every pair of guards in this set is at least a distance of ℓ apart.In this paper, we study the vertex guard variant of this problem for the class of polyominoes. We consider rectangular visibility and distances as geodesics in the L1-metric. Our results are as follows. We give a (simple) thin polyomino such that every guard set has minimum pairwise distances of at most 3. On the positive side, we describe an algorithm that computes guard sets for simple polyominoes that match this upper bound, i.e., the algorithm constructs worst-case optimal solutions. We also study the computational complexity of computing guard sets that maximize the smallest distance between all pairs of guards within the guard sets. We prove that deciding whether there exists a guard set realizing a minimum pairwise distance for all pairs of guards of at least 5 in a given polyomino is NP-complete.We also present an optimal dynamic programming approach that computes a guard set that maximizes the minimum pairwise distance between guards in tree-shaped polyominoes, i.e., computes optimal solutions. Because the shapes constructed in the NP-hardness reduction are thin as well (but have holes), this result completes the case for thin polyominoes.

An approximate dynamic programming approach to network-based scheduling of chemotherapy treatment sessions

A solution approach is proposed for the interday problem of assigning chemotherapy sessions at a network of treatment centres with the goal of increasing the cost-efficiency of system-wide capacity use. This network-based scheduling procedure is subject to the condition that both the first and last sessions of a patient's treatment protocol are administered at the same centre the patient is referred to by their oncologist. All intermediate sessions may be administered at other centres. It provides a systematic way of identifying effective multi-appointment scheduling policies that exploit the total capacity of a networked system, allowing patients to be treated at centres other than their home centre. The problem is modelled as a Markov decision process which is then solved approximately using techniques of approximate dynamic programming. The benefits of the approach are evaluated and compared through simulation with the existing manual scheduling procedures at two treatment centres in Santiago, Chile. The results suggest that the approach would obtain a 20% reduction in operating costs for the whole system and cut existing first-session waiting times by half. A key conclusion, however, is that a network-based scheduling procedure brings no real benefits if it is not implemented in conjunction with a proactive assignment policy like the one proposed in this paper.

Robust job-sequencing with an uncertain flexible maintenance activity

In this study, the problem of scheduling a set of jobs and one uncertain maintenance activity on a single machine, with the objective of minimizing the makespan is addressed. The maintenance activity has a given duration and must be executed within a given time window. Furthermore, duration and time window of the maintenance are uncertain, and can take different values which can be described by different scenarios. The problem is to determine a job sequence which performs well, in terms of makespan, independently on the possible variation of the data concerning the maintenance. A robust scheduling approach is used for the problem, in which four different measures of robustness are considered, namely, maximum absolute regret, maximum relative regret, worst-case scenario, and ordered weighted averaging. Complexity and approximation results are presented. In particular, we show that, for all the four robustness criteria, the problem is strongly NP-hard. A number of special cases are explored, and an exact pseudopolynomial algorithm based on dynamic programming is devised when the number of scenarios is fixed. Two Mixed Integer Programming (MIP) models are also presented for the general problem. Several computational experiments have been conducted to evaluate the efficiency and effectiveness of the MIP models and of the dynamic programming approach.

Exact approaches for the unconstrained two-dimensional cutting problem with defects

This paper studies the unconstrained two-dimensional cutting problem with defects, which requires cutting a set of rectangular item types from a rectangular sheet with defects. The available number of each type is not limited. Two versions of the problem are studied, based on whether the guillotine cut constraint is required. Due to equipment limitations, the guillotine cutting restrictions are essential in specific industries, requiring cutting from one side to the other with each cut and splitting the sheet into two separate sheets. An integer model is proposed for the non-guillotine cut version, and an exact dynamic programming (DP) approach is proposed for the guillotine cut version. To reduce the number of vertical and horizontal positions, we extend the normal patterns, raster points, and meet-in-the-middle patterns to consider the effect of defects. The experimental results on standard benchmarks show that the proposed model can solve most of the instances in literature in a short time. For the guillotine cut version, our approach can significantly reduce the number of cut positions and improve the efficiency of the DP approach compared to the previous exact approach that considers all possible positions. Our approach solves the large instances with more than 300 item types in a relatively short time.

Medium-Term Hydrothermal Scheduling of the Infiernillo Reservoir Using Stochastic Dual Dynamic Programming (SDDP): A Case Study in Mexico

This article aims to obtain and evaluate medium-term operating policies for the hydrothermal scheduling problem by using the stochastic dual dynamic programming (SDDP) approach. To this end, to feed the mathematical model and build the probability distribution functions that best fit each month of the actual inflow volume, monthly inflow data recorded from 1938 to 2018 for the Infiernillo reservoir located in Mexico were employed. Moreover, we simulated inflow volume scenarios using the Monte Carlo method for each month of a one-year planning period. The SDDP approach to solving the optimization problem consisted of the simulation of one forward scenario per iteration and the stabilization of the total operating cost as a convergence criterion, which results in an operating policy. We then assessed its quality by estimating the one-sided optimality gap. It is worth mentioning that the best operation policy required scenario trees of up to 17,000 inflow realizations per stage. Additionally, to study the evolution of the expected value along the planning horizon of the main variables involved in the medium-term hydrothermal scheduling problem, we simulated the best operation policy over 10,000 inflow scenarios. Finally, to show the practical value of the proposed approach, we report its computational complexity.

A cloud-based eco-driving solution for autonomous hybrid electric bus rapid transit in cooperative vehicle-infrastructure systems: A dynamic programming approach

Efficient public transportation has always intrigued extensive research. Aiming to improve the commuting efficiency and fuel economy of the autonomous hybrid electric buses in the Bus Rapid Transit (BRT), a cloud-based eco-driving solution adopting dynamic programming and model predictive control is proposed in this paper. This solution contains an upper-level cloud-based scheduling strategy and a lower-level onboard predictive energy management, which is conceived to function in a Cyber-physical system of the cooperative vehicle-infrastructure system. The scheduling model carefully considered coupled spatiotemporal constraints for the driving of autonomous BRT buses, including traffic lights, traffic regulations, stations, and ride comfort. The onboard energy management leverages the pre-planned scheduling information to achieve near-optimal fuel economy. The eco-driving solution is examined in three scenarios with intersections, stations, and ramps. Simulation results show that the proposed method can deal with different spatiotemporal limits along the route, with virtually no non-essential stops and sudden acceleration or braking, and achieves 97%–98% energy-saving potential compared with the baseline performance.