Image-based river flow measurement methods have been attracting attention because of their ease of use and safety. Among the image-based methods, the space-time image velocimetry (STIV) technique is regarded as a powerful tool for measuring the streamwise flow because of its high measurement accuracy and robustness. However, depending on the image shooting environment such as stormy weather or nighttime, the conventional automatic analysis methods may generate incorrect values, which has been a problem in building a real-time measurement system. In this study, we tried to solve this problem by incorporating the deep learning method, which has been successful in the field of image analysis in recent years, into the STIV method. The case studies for the three datasets indicated that deep learning can improve the efficiency of the STIV method and can continuously improve performance by learning additional data. The proposed method is suitable for building a real-time measurement system because it has no tuning parameters that need to be adjusted according to the shooting conditions and the calculation speed is fast enough for real-time measurement.