Dynamic uncertainty factors such as equipment faults are common in practically implemented cloud manufacturing (CMfg) environments, often causing the manufacturing service to be invalidated. In that case, efficient reconfiguration of the original service composition under practical constraints is critical; however, existing research scarcely focuses on it. This paper proposes a dynamic service composition reconfiguration model to bridge the gap by considering practical constraints (DSCRPC) in a real-life cloud manufacturing environment. Based on the constraints considered in this study, the DSCRPC model redefines three objectives: time (T*), cost (C*), and product service quality (Q*S*). To optimize the DSCRPC model, this study developed an adaptive multi-population multi-objective whale optimization algorithm (AMPOWOA) based on the Pareto strategy. The algorithm adopts four balancing strategies and adaptively optimizes and adjusts the key parameters under various balancing strategies through well-designed reinforcement learning models. Finally, we conduct numerical experiments and actual application case tests to compare the performances of AMPOWOA and other algorithms (MOWOA, MOHHO, NSGA-II). The results show that DSCRPC can continuously tackle the cloud manufacturing service composition (CMSC) reconfiguration issue with constraints until an order is completed. Moreover, AMPOWOA is superior to the other algorithms optimizing the DSCRPC model. This significantly enhances the robustness of service composition reconfiguration in real-life CMfg.