Abstract The percentage of fresh water resource availability in the world is diminishing every year. According to a world economic forum survey, the increase in water demand will result in high scarcity globally in the next two decades. The eradication of the water demand increase and reducing the losses during the transportation of water is challenging. Thus accordingly, an Internet of Things (IoT)-based architecture integrated with Fog for underground water distribution system has been proposed. Towards designing an IoT water distribution architecture for a smart city, we need to first forecast the water demand for consumers. Hence, accordingly, water demand forecasting has been carried out on a daily basis for a period of three months as a case study using autoregressive integrated moving average (ARIMA) and regression analysis. Based on water demand forecasting analysis, a water distribution design for an IoT-based architecture has been carried out using hydraulic engineering design for proper distribution of water with minimal losses which would result in the development of a smart water distribution system (SWDS). This has been carried out using EPANET.