Multisource neural network feature map fusion: An efficient strategy to detect plant diseases

Abstract

The use of RGB cameras or multispectral imaging systems can provide a wide range of applications for crop monitoring, plant phenotyping and disease detection. Although several approaches have been proposed, they increasingly use convolutional neural network-based architectures, which have, however, become increasingly cumbersome for improving classification results and difficult to train with few labeled data. Other increasingly popular approaches consist of using an ensemble of convolutional neural networks, in which each model solves a different problem. Since the inference is time- and resource-consuming due to the execution of multiple models, recent works have focused on transferring knowledge from an ensemble of models to a compact model to obtain better performance. In this paper, we propose an original approach that improves both accuracy and speed by reusing feature maps extracted by heterogeneous models from different data. Linked to each model, a transformation block allows keeping the correct number of feature maps and changing their dimension if necessary. To generate the feature maps, we only need the first layers of the ensemble models, thus taking advantage of ensemble learning methods, while adding only a few layers of a second model dedicated to aggregation of features. This approach allows an ensemble of models to be combined with different architectures that can process different data, such as several representations of the same input image or multispectral images, while being fast enough at the inference stage. This approach is adapted to hierarchical classification tasks by re-exploiting the same feature maps with different transformation blocks, offering accuracy gains in tasks not handled by the ensemble model. The results are provided for the PlantVillage dataset, with RGB images converted to three different color spaces, and for a custom Grapevine Yellow dataset, with multispectral images acquired with two different multispectral cameras.