Tuning approximate dynamic programming policies for ambulance redeployment via direct search

Abstract

Open AccessOpen Access licenseAboutSectionsView PDF ToolsAdd to favoritesDownload CitationsTrack Citations ShareShare onFacebookTwitterLinked InEmail Go to SectionOpen AccessOpen Access license HomeStochastic SystemsVol. 3, No. 2 Tuning Approximate Dynamic Programming Policies for Ambulance Redeployment via Direct SearchMatthew S. Maxwell, Shane G. Henderson, Huseyin TopalogluMatthew S. Maxwell, Shane G. Henderson, Huseyin TopalogluPublished Online:15 Nov 2013https://doi.org/10.1287/10-SSY020AbstractIn this paper we consider approximate dynamic programming methods for ambulance redeployment. We first demonstrate through simple examples how typical value function fitting techniques, such as approximate policy iteration and linear programming, may not be able to locate a high-quality policy even when the value function approximation architecture is rich enough to provide the optimal policy. To make up for this potential shortcoming, we show how to use direct search methods to tune the parameters in a value function approximation architecture so as to obtain high-quality policies. Direct search is computationally intensive. We therefore use a post-decision state dynamic programming formulation of ambulance redeployment that, together with direct search, requires far less computation with no noticeable performance loss. We provide further theoretical support for the post-decision state formulation of the ambulance-deployment problem by showing that this formulation can be obtained through a limiting argument on the original dynamic programming formulation. Back to Top Next FiguresReferencesRelatedInformationCited byInpatient Overflow: An Approximate Dynamic Programming ApproachJ. G. Dai, Pengyi Shi16 May 2019 | Manufacturing & Service Operations Management, Vol. 21, No. 4A Unified Framework for Optimization Under UncertaintyWarren B. Powell4 November 2016The Minimum Expected Penalty Relocation Problem for the Computation of Compliance Tables for Ambulance VehiclesThije van Barneveld21 April 2016 | INFORMS Journal on Computing, Vol. 28, No. 2Discrete Optimization Models for Homeland Security and Disaster ManagementLaura Albert McLay26 October 2015Simulation-Based Approximate Policy Iteration with Generalized Logistic FunctionsAntoine Sauré, Jonathan Patrick, Martin L. Puterman28 September 2015 | INFORMS Journal on Computing, Vol. 27, No. 3Technical Note—Trading Off Quick versus Slow Actions in Optimal SearchSteven M. Shechter, Farhad Ghassemi, Yasin Gocgun, Martin L. Puterman6 March 2015 | Operations Research, Vol. 63, No. 2A Bound on the Performance of an Optimal Ambulance Redeployment PolicyMatthew S. Maxwell, Eric Cao Ni, Chaoxu Tong, Shane G. Henderson, Huseyin Topaloglu, Susan R. Hunter16 July 2014 | Operations Research, Vol. 62, No. 5 Volume 3, Issue 2December 2013Pages 322-633 Article Information Metrics Information Received:November 01, 2010Published Online:November 15, 2013 Copyright © 2013, The author(s)Cite asMatthew S. Maxwell, Shane G. Henderson, Huseyin Topaloglu (2013) Tuning Approximate Dynamic Programming Policies for Ambulance Redeployment via Direct Search. Stochastic Systems 3(2):322-361. https://doi.org/10.1287/10-SSY020 PDF download