Climate change can significantly affect building energy use and associated greenhouse gas (GHG) emissions in urban areas, as fossil fuels remain a significant energy source. Green roofs can offer multiple benefits to the urban environment, but their effects on GHG mitigation have not been fully investigated, especially under climate change. This study assessed green roofs’ contribution to GHG mitigation by saving building energy and absorbing CO2 under the present (2017–2019) and future (2049–2051) climate scenarios (SSP2-45 and SSP5-85) in Shanghai, China, at the city and township scale. A Geographic Information System (GIS)-based spatial statistical method was developed based on climate change modeling and building energy simulation. The results suggested that installing green roofs can effectively save building energy regardless of building type, yet the amount of savings can vary depending on the weather conditions within the city. The contribution analysis indicated that most saved building energy was attributed to the Heating, Ventilation, and Cooling (HVAC) system, with more energy saved under warmer climate scenarios in the future, particularly during the summer months. More energy was saved from shopping malls on an annual and monthly scale, regardless of the climate scenarios and weather zones. Finally, a case study indicated installing green roofs on all five types of buildings (office, hotel, hospital, shopping mall, apartment) of less than 50 m in height can reduce 8.28% of the CO2 emitted during the building operation stage in the entire city under the present climate scenario. The annual CO2 reduction varied with the location of townships, ranging from 2.18% to 13.78%, depending on the composition of building types and local weather conditions in Shanghai. This study offered policymakers a reference on the environmental benefits and investment values of installing green roofs in large cities.