Timely and non-destructive assessment of crop yield is an essential part of agricultural remote sensing (RS). The development of unmanned aerial vehicles (UAVs) has provided a novel approach for RS, and makes it possible to acquire high spatio-temporal resolution imagery on a regional scale. In this study, the rice grain yield was predicted with single stage vegetation indices (VIs) and multi-temporal VIs derived from the multispectral (MS) and digital images. The results showed that the booting stage was identified as the optimal stage for grain yield prediction with VIs at a single stage for both digital image and MS image. And corresponding optimal color index was VARI with R2 value of 0.71 (Log relationship). While the optimal vegetation index NDVI[800,720] based on MS images showed a linear relationship with the grain yield and gained a higher R2 value (0.75) than color index did. The multi-temporal VIs showed a higher correlation with grain yield than the single stage VIs did. And the VIs at two random growth stage with the multiple linear regression function [MLR(VI)] performed best. The highest correlation coefficient were 0.76 with MLR(NDVI[800,720]) at the booting and heading stages (for the MS image) and 0.73 with MLR(VARI) at the jointing and booting stages (for the digital image). In addition, the VIs that showed a high correlation with LAI performed well for yield prediction, and the VIs composed of red edge band (720nm) and near infrared band (800nm) were found to be more effective in predicting yield and LAI at high level. In conclusion, this study has demonstrated that both MS and digital sensors mounted on the UAV are reliable platforms for rice growth and grain yield estimation, and determined the best period and optimal VIs for rice grain yield prediction.