Online contextual learning with perishable resources allocation

Abstract

We formulate a novel class of online matching problems with learning. In these problems, randomly arriving customers must be matched to perishable resources so as to maximize a total expected reward. The matching accounts for variations in rewards among different customer–resource pairings. It also accounts for the perishability of the resources. Our work is motivated by a healthcare application, but it can be easily extended to other service applications. Our work belongs to the online resource allocation streams in service systems. We propose the first online algorithm for contextual learning and resource allocation with perishable resources. Our algorithm explores and exploits in distinct interweaving phases. We prove that our algorithm achieves an expected regret per period that increases sub-linearly with the number of planning cycles.