Precision agriculture aims to increase crop yield by employing an efficient resource management scheme, such as estimating irrigation requirements. Reference evapotranspiration (ET0), defined as the process of water loss from the soil and reference plant, is one of the indispensable components on which crop irrigation requirement depends. It is mainly calculated by using empirical models. However, these models require a large climate dataset that is sometimes unavailable in data-scarce regions. The present study focuses on the estimation of ET0 values by using three climate parameters as input variables i.e., minimum temperature (Tmin), maximum temperature (Tmax), and solar radiation (Rs). Moreover, to consider the effect of time-varying characteristics of the ET0 process, deep reinforcement learning (DRL) based ensemble approach, DeepEvap, is introduced to estimate ET0 values. The whole modeling procedure of the proposed ensemble model incorporates three phases. In phase I, the data preprocessing technique is performed on the meteorological data to clean the existing impurities as it affects the performance of any machine learning (ML) based approach. In phase II, four different deep neural network-based models are used to build the estimation model of ET0 and calculate the prediction results. In phase III, the DRL approach is used to ensemble the prediction results of these four models. The meteorological dataset of two stations of India: Ludhiana and Patiala, is selected to validate the proposed approach. The results of the conducted study depict that: (a) The proposed DeepEvap approach is competitive for ET0 prediction by achieving a coefficient of determination (R2) = 0.96. It significantly outperforms four baseline models; (b) The proposed technique also integrates four deep neural network models and works better than existing ensemble approaches.