Reward function design method to achieve system-level objectives in ambulance diversion problem

Abstract

This study addresses a reward function design problem in multi-agent systems, using ambulance diversion (AD) problem in emergency medical services (EMS) as an application. While Emergency Departments (EDs) may declare AD to alleviate overcrowding, excessive use of this strategy by individual EDs can have negative consequences, such as limited access and delayed care for emergency patients. To eliminate negative consequences caused by individual decisions, we propose a reward function design method to align individual agents’ decisions with socially optimal strategies. This method ensures that an equilibrium status is achieved under socially optimal operation by removing incentives for deviations from socially optimal behavior while maximizing system welfare without extra investment. The proposed method offers an exact solution for designing reward function of a stochastic game model with finite state and action spaces. We conduct numerical experiments for a 2-ED system and demonstrate that the designed reward function properly guides the EDs toward a socially optimal AD strategy.