Carbon productivity is a better measure of a region's sustainable economic growth since it considers economic development and carbon emissions at the same time. It is crucial to improve carbon productivity through the co-agglomeration of manufacturing and producer services for China. To better understand the effect and mechanism of co-agglomeration, this study uses panel data of 282 Chinese cities from 2003 to 2017 to examine the effect of manufacturing and producer services co-agglomeration on carbon productivity. Further, moderating effects models and panel threshold models are used to investigate the mechanisms of co-agglomeration effect on carbon productivity. The results suggest that the influence of co-agglomeration on carbon productivity is U-shaped with an inflection point of 3.63, indicating that a high level of co-agglomeration is required for carbon productivity to increase. Compared with other cities, cities in the eastern region or with a large scale have lower inflection points. The upgrading of industrial structure, production efficiency, and the production chain has a positive moderating effect on the influence of co-agglomeration on carbon productivity, while upgrading of non-production chains does not. When the levels of capital misallocation and labor misallocation reach the threshold of 1.115 and 1.557 respectively, industrial co-agglomeration has a promoting effect on carbon productivity. This research offers valuable insights for optimizing the urban industrial layout to improve carbon productivity.