ABSTRACT Cloud manufacturing is an emerging manufacturing paradigm that delivers on-demand manufacturing services via an Internet platform. Multi-task scheduling is an indispensable technique for accomplishing this goal, enabling the simultaneous handling of multiple customers’ complex requirements by scheduling centralized managed manufacturing services. Despite the development of various meta-heuristic algorithms for cloud manufacturing multi-task scheduling (CMMS) problems to improve scheduling efficiency, their sophisticated design procedures and laborious parameter tuning pose challenges in dynamic, large-scale manufacturing environments. Deep reinforcement learning (DRL) has recently been applied to the CMMS problem due to its efficiency and flexibility. Nevertheless, the current body of this research is still nascent, with most existing DRL-based frameworks focusing on learning constructive heuristics. This study proposes a new DRL-based framework for learning improvement heuristics to solve the CMMS problem. Three state-of-the-art DRL technologies are selected to systematically investigate their potential in addressing the CMMS problem under unpredictable service availability. A comprehensive comparison of these DRL technologies regarding effectiveness and efficiency, robustness and adaptability, and scalability is conducted. Experimental results demonstrate the effectiveness of the proposed DRL-based framework and indicate that PPO is superior to the other two technologies in solving the CMMS problem.