Hydrological effects caused by fragmented land cover types in human-dominated agro-ecosystems are poorly understood. In this study, based on visual interpreted high-resolution land cover map, an agro-hydrological model (HYDRUS-dualKc) was used in a distributed manner to investigate the hydrological complexities caused by fragmented land cover types and shallow groundwater. An irrigation system located in the upper Yellow River basin was chosen as a case study area, where field observations were conducted in 2012 and 2013. Combined effects of vegetation, soil, irrigation and groundwater were considered. Simulations of soil moisture and soil salinity fitted well with field observations; evapotranspiration (ETa) was also comparable with remote sensing data. Results showed that the growing season ETa varied from 162 to 567 mm among the different vegetation covers with the values for crop fields usually higher than those for natural lands. The soil evaporation was obviously larger in natural land and the natural vegetation growth was seriously stressed. Through lateral groundwater exchange, the cropland functioned as a stable groundwater recharge zone while the natural land functioned as a discharge zone during the growing season. As a result, 21% of the total water diverted to Jiyuan migrated to natural lands through groundwater due to field percolation and canal seepage, along with 50–75% of the total salt introduced. The fragmented land covers and complex hydrological processes bring many challenges to remote sensing and model simulation. Future land planning and water resources management should consider the hydrological effects caused by the fragmented land cover types and the associated management strategies.