Rainwater harvesting systems (RWHs) in buildings are benefits in alleviating urban water scarcity. However, climate change and changes in building water demand affect the expected performance of RWHs over their life cycle. Previous studies have only focused on the adaptability of RWHs to climate change without considering the changes in building water demand. To address this issue, this study explores the impact of combined changes in precipitation patterns and domestic water demand on the water-saving efficiency of RWHs over their 25-year life cycle based on historical observations. Three cities in Japan were selected as case studies. The results indicate that during 25-year life cycle of RWHs, precipitation in different cities in Japan shows an increasing trend, while domestic water demand shows a decreasing trend. Such trend resulted in a change of -2.90–4.02% in the water-saving efficiency of the established RWHs and a change of -2.90–3.82% in the theoretical optimal water-saving efficiency, and these effects were greater in RWHs in buildings with high non-potable water demands. Therefore, RWHs in buildings with low non-potable water demand can appropriately reduce the rainwater tanks, while RWHs in buildings with high non-potable water demand must increase the rainwater tank size to eliminate this adverse effect. Furthermore, both reductions in domestic water demand and increases in precipitation have a positive effect on the water-saving efficiency of RWHs. However, when both domestic water demand and precipitation increase or decrease simultaneously, the change in water-saving efficiency of RWHs is related to the non-potable water demand of the building. This study can provide data support and theoretical evidence for the further implementation of RWHs under climate change, and can help stakeholders to optimize and update RWHs to ensure system feasibility.