Deep Learning (DL), unlike conventional Artificial Neural network (ANN), is capable of self-learning data features layer by layer in unsupervised manner and creating a data-driven model with the given dataset. DL has been widely applied to big data analytics, graphics object detection, classification, voice recognition and many other problems. This paper presents an integrated data-driven modelling framework that couples DL with the well-developed evolutionary optimization tool in a scalable and heterogeneous high performance computing paradigm. The integrated framework enables modellers to effectively and efficiently construct a model with a given dataset. It is demonstrated that the framework has wide applicability including but not limited to the simulation, optimization and operation decision of water distribution systems. The paper elaborates the development of the deep learning framework with potential applications of facilitating the data fusion, system simulation and predictive analysis, anomaly detection from the time series data (pressures, flows and consumptions etc.), water usage prediction, construction of a meta-model as a surrogate to the physics-based models (hydraulic and water quality) for water distribution management.