Accurate and timely prediction of oilseed rape yield is crucial in precision agriculture and field remote sensing. We explored the feasibility and potential for predicting oilseed rape yield through the utilization of a UAV-based platform equipped with RGB and multispectral cameras. Genetic algorithm–partial least square was employed and evaluated for effective wavelength (EW) or vegetation index (VI) selection. Additionally, different machine learning algorithms, i.e., multiple linear regression (MLR), partial least squares regression (PLSR), least squares support vector machine (LS-SVM), back propagation neural network (BPNN), extreme learning machine (ELM), and radial basis function neural network (RBFNN), were developed and compared. With multi-source data fusion by combining vegetation indices (color and narrow-band VIs), robust prediction models of yield in oilseed rape were built. The performance of prediction models using the combination of VIs (RBFNN: Rpre = 0.8143, RMSEP = 171.9 kg/hm2) from multiple sensors manifested better results than those using only narrow-band VIs (BPNN: Rpre = 0.7655, RMSEP = 188.3 kg/hm2) from a multispectral camera. The best models for yield prediction were found by applying BPNN (Rpre = 0.8114, RMSEP = 172.6 kg/hm2) built from optimal EWs and ELM (Rpre = 0.8118, RMSEP = 170.9 kg/hm2) using optimal VIs. Taken together, the findings conclusively illustrate the potential of UAV-based RGB and multispectral images for the timely and non-invasive prediction of oilseed rape yield. This study also highlights that a lightweight UAV equipped with dual-image-frame snapshot cameras holds promise as a valuable tool for high-throughput plant phenotyping and advanced breeding programs within the realm of precision agriculture.