Public transportation is often considered as a green travel mode to alleviate the negative externalities such as traffic congestion and haze pollution generated from transport. However, is prioritizing urban public transportation actually conducive to haze emission reduction? In this study, considering special emphasis on the cumulative effect of haze, a dynamic panel model is constructed to analyze and quantify the impact of public transportation on haze pollution by using the data of 284 cities in China, and the heterogeneity of the impact in cities with different pollution levels is examined. Several interesting findings are derived from the empirical results. First, the development of urban public transportation can significantly alleviate urban haze pollution. Second, the haze reduction effect of public transportation in cities with different pollution levels is non-universal. Comparatively speaking, the haze reduction effect of public transportation in lightly polluted cities is more evident than that in heavily polluted cities. Therefore, in order to reduce haze pollution in a more effective manner, China should continue to promote urban public transportation priority strategy. Moreover, the government should also formulate differentiated traffic development strategies to effectively alleviate the urban traffic burdens.