Billions of people living in developing countries lack access to safe drinking water, not to mention water for handwashing, one of the most effective ways to contain the fast spreading novel coronavirus (COVID -19). The recent global spread of COVID-19 has fostered diverse initiatives such as the ‘Safe Hands’ challenge led by the World Health Organization. Individuals are encouraged to regularly wash their hands for 40–60 s under running water with soap. This call for ‘Safe Hands’ comes at a time when water insecurity and limited access to handwashing facilities in Africa is heightened. In this article, Chitungwiza city in Zimbabwe is used as a case study to assess the implications of the ‘Safe Hands’ challenge for poor municipalities in developing countries and characterize the challenges they face. To do so, interviews were conducted at water points/boreholes used by residents during Zimbabwe’s COVID-19 national lockdown. The calculation of water requirements for proper hand hygiene determined the capacity for water-stressed regions to effectively implement ‘Safe Hands’. On average, it was established that one person consumes an extra 4.5 L per day of water when they practice WHO ‘Safe Hands’ in the context of COVID-19. This increases domestic water demand in Chitungwiza by 9%. Due to water scarcity, people in Chitungwiza were experiencing challenges with practicing ‘Safe Hands’. With their ‘dry taps’ woes, they might not be able to meet the standards of this WHO challenge. Lack of soap also reduced the effectiveness of the ‘Safe Hands’ challenge. This paper proposes short- and long-term measures that would allow effective implementation of the ‘Safe Hands’ by means of sustainable potable water supply. These measures include extensive social awareness and temporary change of household water use behavior. Municipalities are recommended to establish public private partnerships (PPPs) to create immediate and long-term water investments. Structural and transformational reforms would enhance, through flexible planning, investments for both water infrastructure and governance. This narrative has the potential to improve the urban water systems resiliency against future pandemics.