The port of Shanghai, as the world’s largest container port, has been experiencing rapid development in recent years, with increasing cargo throughput capacity. The combustion of diesel fuels used by internal and external port-related container trucks and in-port machineries can release various pollutants, causing air pollution. The terminals are close to the residential area, and the emissions are concentrated, which is worth paying attention to. This study aims to synthetically assess the port-related emissions and their environmental impacts. We firstly constructed an emission inventory of air pollutants in the port of Shanghai and then used the WRF-CMAQ model to estimate the influence of port-related source emissions on air quality. The results show that the annual emissions of SO2, NOX, CO, VOCS, PM, PM10, PM2.5, CO2, BC and OC caused by cargo-handling equipment were 21.88 t, 1811.22 t, 1741.72 t, 222.76 t, 61.52 t, 61.42 t, 58.41 t, 141,805.40 t, 26.80 t and 10.07 t in 2015. The emissions of NOX, CO, VOCS, PM10 and PM2.5 caused by external port-related container trucks were 18,002.92 t, 5308.0 t, 1134.57 t, 711.12 t and 640.58 t. The exhaust of external port-related container trucks was much larger than that of cargo-handling equipment, so the impact on air quality was also higher than that of the machinery. The peak annual average concentrations of PM2.5 and NOX contributed by the port-related sources were 1.75 μg/m3 and 49.21 μg/m3, respectively, which accounted for 3.08% and 36.7%, respectively, of the simulated ambient concentrations by all the anthropogenic emissions in Shanghai. Our results imply that the emission control policy to reduce the combined port-related emissions, especially for the cargo-delivery transportation phase from port to city, is key for large coastal port cities such as Shanghai.