To guarantee clean heating of buildings in cold regions, an off-peak electricity driven chemisorption heat pump employing SrCl2/sulfurized expanded graphite composite sorbent is developed. However, the heat output temperature of chemisorption heat pump fluctuates greatly, up to 30 °C, which seriously restricts its popularization and application. In this paper, the above-mentioned problems are solved from two aspects of sorption bed structure design and system operation strategy. The sorption bed utilized for thermal energy storage is especially composed of several unit reactors, and the asynchronous start-stop control method of unit reactors is innovatively proposed. By controlling their start and stop time, the temperature fluctuation range of hot water that the heat pump outputs is significantly reduced. Simultaneously, to improve the energy efficiency of heat pump, small temperature difference fan coil units are adopted indoors. The results indicate that when ten reactors operate asynchronously, the output temperature of hot water fluctuates by as low as 1.4 °C. With four reactors in asynchronous operation, the temperature fluctuation range can be reduced to 3 °C, and 35 °C hot water enters the small temperature difference fan coil units to release heat to the indoor space. Additionally, the heating performance of the chemisorption heat pump is relatively stable in cold regions. When the evaporating temperature decreases from 5 °C to −15 °C, its energy efficiency is almost constant, about 1.47. Eventually, this chemisorption heat pump can shift electricity from peak periods to off-peak periods, and also saves 0.7 to 6.3 yuan per day in space heating cost in comparison to the conventional vapor-compression one.