The ease of access to media resources and computational power has recently generated interest in using vision-based approaches for hydraulic monitoring. A key challenge for non-intrusive, image-based hydrology measurement methods is incorporating different hydraulic variables as prior knowledge with image information. We propose a photogrammetry-based method called L1-Diffusion derived from the convection–diffusion equation commonly used in hydrodynamics with an additional regularization term to estimate the fluid motion field in the image plane, from which the free surface velocity can be further obtained using the photogrammetric projection relationship between the image plane and world coordinates. The inverse problem is used to discuss the relationship between the widely used space–time image velocimetry (STIV) and the proposed L1-Diffusion. To validate the proposed method, unmanned aerial vehicle (UAV) images as well as in-situ acoustic Doppler current profiler (ADCP) experiments were carried out. Based on comparison results with the ADCP measurement and vision-based flow field estimation, the newly proposed L1-Diffusion algorithm can accurately and efficiently estimate the free surface velocity of a river from the image sequences in a variety of scenarios.