ABSTRACT In this study, an online reinforcement learning-based approach and a reinforcement learning with prior knowledge approach are proposed to enhance decision intelligence in inventory management systems for handling nonstationary stochastic market demands in e-commerce environment with crowdsourcing resources. The proposed inventory control policies are designed to solve a multi-period inventory problem with the objectives of optimising inventory-related costs and service levels in the absence of prior information on demand patterns. An experimental analysis reveals that the proposed reinforcement learning-based inventory control policies achieve cost savings and higher service levels across various settings of cost ratios and lead times.