Segmentation of weeds and crops using multispectral imaging and CRF-enhanced U-Net

Abstract

Weeds are one of the most detrimental challenges to agriculture causing significant losses to beneficial yield potentials as they compete with crops for water, nutrients and sunlight. Early detection of weeds in the field is highly critical for taking appropriate actions, such as applying herbicides, mechanical removal or other remedial treatments, which are less effective or more resource intensive at later stages of crop growth. In this work, a deep learning method has been developed for weed detection. A sunflower dataset comprising multispectral images, the visible band (400–700 nm wavelengths in RGB) and Near Infrared (700–1000 nm wavelengths, NIR), captured at various days and times were used for the study. The deep learning model, ‘U-Net’, was trained with images from the cotyledon emergence through to the subsequent growth stages and tested on images of crops in the last stage of growth, where chemical treatments can be applied. The results of the U-Net were further enhanced by employing conditional random fields to achieve improved segmentation in terms of Intersection over Union (IoU). The proposed method using the Green (530–600 nm wavelengths) + Filtered-NIR + Normalised Difference Vegetation Index (NDVI) (Weier and Herring, Aug. 2000) channels as the input achieved the best mean IoU score of 0.883 on images of 512 × 512 pixels. In the same experiment, soil, crop and weed pixels were correctly predicted with 0.990 IoU, 0.906 IoU, and 0.753 IoU scores, respectively. The results show that the chosen input and the proposed methodology offer a viable approach for early-stage weed detection.