Markov decision processes in service facilities holding perishable inventory

Abstract

In this article, we consider a single server queueing system with finite waiting space N (including one customer in service) and an inventory is attached with the maximum capacity S. The arrival of customer at the system is according to independent Poisson Processes with rate λ through a single channel. The service time is exponentially distributed with mean 1/μ and the item in stock has exponential life time with perishable rate γ(>0). When we place the order due to the demand of the customers, we assume that the lead time of procurement of item is exponentially distributed with parameter δ. Our object is to make a decision at each state of the system to operate the server by minimizing the entire service cost. The problem is modelled as a Markov decision problem by using the value iteration algorithm to obtain the minimal average cost of the service. The unique equilibrium probability distributions {p(q, i)} is also obtained by using Matrix geometric form in which the two dimensional state space contains infinite queue length and finite capacity of inventory. Numerical examples are provided to obtain the optimal average cost.