An intelligent open trading system for on-demand delivery facilitated by deep Q network based reinforcement learning

Abstract

On-demand delivery in urban areas has been growing rapidly in recent years. Nevertheless, on-demand delivery networks lack an efficient, sustainable, and environmentally friendly operative strategy. An open trading system equipped with on-line auctions provides an opportunity for increasing the efficiency of on-demand delivery systems. Reinforcement learning techniques that automate decision-making can facilitate the implementation of such complex and dynamic systems. This paper presents an on-line auction-based request trading platform embedded within an open trading system as a new scheme for carriers and shippers to trade on-demand delivery requests. The system is developed based on a multi-agent model, composed of carriers, shippers, and the on-line platform as autonomous agents. Deep Q network enabled reinforcement learning is used in the decision-making processes for the agents to optimise their behaviour in a dynamic environment. Numerical experiments conducted on the Melbourne metropolitan network demonstrate the effectiveness of the open trading system, which can provide benefits for all stakeholders involved in the on-demand delivery market as well as the entire system. The reinforcement learning enabled platform can gain more profit when there are more learning carriers. The results indicate that the intelligent open trading system with on-line auctions is a promising city logistics solution.