Pumping water in high-rise buildings has been overlooked in energy calculations for urban water supply, despite being a major contributor. Using data for two commonly used pumping systems and extrapolating results to a megacity in China, we show that over one third of energy for water supply is associated with around one tenth of a city’s population. Buildings can achieve a 45% reduction in pumping energy by replacing traditional break tank systems with pressurized booster systems. The latter capitalize on pressure supplied by the water distribution network, whereas the former lose energy by storing water at atmospheric pressure. Electricity saved by replacing 25% of break tank systems with pressurized booster systems is 11% of high-rise pumping energy in the case city and reduces annual emissions by 8600 tCO2e. Controlled replacement represents a realistic way of reducing emissions associated with water supply as China’s urban population grows and living density increases.