RGB and multispectral UAV image fusion for Gramineae weed detection in rice fields

Abstract

In this paper, a new method to fuse low resolution multispectral and high resolution RGB images is introduced, in order to detect Gramineae weed in rice fields with plants at 50 days after emergence (DAE).The images are taken from a fixed-wing unmanned aerial vehicle (UAV) at 60 and 70 m altitude. The proposed method combines the texture information given by a high resolution red–green–blue (RGB) image and the reflectance information given by a low resolution multispectral (MS) image, to obtain a fused RGB-MS image with better weed discrimination features. After analyzing the normalized difference vegetation index (NDVI) and normalized green red difference index (NGRDI) for weed detection, it was found that NGRDI presents better features. The fusion method consists of decomposing the RGB image using the intensity, hue and saturation (IHS) transformation, then, a second order Haar wavelet transformation is applied to the intensity layer (I) and the NGRDI image. From this transformation, the low–low (LL) coefficients of the NGRDI image are replaced by the LL coefficients of the I layer. Finally, the fused image is obtained by transforming the new wavelet coefficients to RGB space. To test the method, a one hectare experimental plot with rice plants at 50 DAE with Gramineae weeds was selected. Additionally, to compare the performance of the method, two indices were used, specifically, the M/MGT index which is the percentage of detected weed area, and the MP index which indicates the precision of weed detection. These indices were evaluated in four validation zones using three Neural Networks (NN) detection systems based on three types of images; namely, RGB, RGB + NGRDI, and fused RGB-NGRDI. The best weed detection performance was obtained by the NN with the fused image, with M/MGT index between 80 and 108% and MP between 70 and 85%.