Over the past two decades, the industrial sector of China has experienced rapid development, which has correspondingly led to a significant increase in water resource consumption. To better understand the dynamics of industrial water use, and formulate appropriate water resource conservation and management policies, it is necessary to evaluate the evolution of industrial water use efficiency and its influencing factors in China. Given the high sensitivity and accuracy of the stochastic frontier analysis (SFA) model for efficiency assessment, the Tobit model is more suitable for regression analyses of truncated data. This study employed the SFA–Tobit panel model to evaluate the industrial water use efficiency of provinces in China from 2003 to 2021. The results indicate that national industrial water use efficiency improved from 0.41 to 0.65 during the study period. All provinces showed significant improvements, with developed provinces exhibiting higher industrial water use efficiency than undeveloped provinces. Regionally, the eastern areas demonstrated superior industrial water use efficiency compared to the western regions, with the central regions having the lowest overall water use efficiency. Moreover, the efficiency gap between regions has been narrowing. The national industrial water-saving potential is estimated at 31.306 billion cubic meters, with Jiangsu province having the highest saving potential at 3.709 billion cubic meters. In comparison, Beijing has the lowest at just 32,000 cubic meters. The Tobit regression results reveal that economic development and technological progress positively contribute to increased industrial water use efficiency. In contrast, water use intensity, openness, and urbanization levels negatively impacted the improvement of industrial water use efficiency. Therefore, it is necessary to increase investment in technological innovation, strictly control industrial water intensity, appropriately balance import and export trade with urbanization levels, and promote sustainable economic development. This study can provide effective support for the subsequent green transformation of China’s industry.