Towards Automated Analysis of Grain Spikes in Greenhouse Images Using Neural Network Approaches: A Comparative Investigation of Six Methods.

Sajid Ullah,Michael Henke,Narendra Narisetti,Klára Panzarová,Jan Hejatko,Evgeny Gladilin,Martin Trtílek

doi:10.3390/s21227441

Abstract

Automated analysis of small and optically variable plant organs, such as grain spikes, is highly demanded in quantitative plant science and breeding. Previous works primarily focused on the detection of prominently visible spikes emerging on the top of the grain plants growing in field conditions. However, accurate and automated analysis of all fully and partially visible spikes in greenhouse images renders a more challenging task, which was rarely addressed in the past. A particular difficulty for image analysis is represented by leaf-covered, occluded but also matured spikes of bushy crop cultivars that can hardly be differentiated from the remaining plant biomass. To address the challenge of automated analysis of arbitrary spike phenotypes in different grain crops and optical setups, here, we performed a comparative investigation of six neural network methods for pattern detection and segmentation in RGB images, including five deep and one shallow neural network. Our experimental results demonstrate that advanced deep learning methods show superior performance, achieving over 90% accuracy by detection and segmentation of spikes in wheat, barley and rye images. However, spike detection in new crop phenotypes can be performed more accurately than segmentation. Furthermore, the detection and segmentation of matured, partially visible and occluded spikes, for which phenotypes substantially deviate from the training set of regular spikes, still represent a challenge to neural network models trained on a limited set of a few hundreds of manually labeled ground truth images. Limitations and further potential improvements of the presented algorithmic frameworks for spike image analysis are discussed. Besides theoretical and experimental investigations, we provide a GUI-based tool (SpikeApp), which shows the application of pre-trained neural networks to fully automate spike detection, segmentation and phenotyping in images of greenhouse-grown plants.

Highlights

The above tests aimed to evaluate the performance of different spike detection and segmentation models trained on a particular set of images by application to (i) images of the same and (ii) different crop cultivars screened in the same facility as well as to (iii) images of phenotypically more distant wheat cultivars from another greenhouse facility
Our study showed that the performance of deep neural network (DNN) models trained on a relatively modest set of ground truth images depends on the optical spike appearance as well as the spatial spike location within the plant in different crop cultivars
We conclude that DNNs trained on a particular set of plant images can, in general, be expected to show comparable performance by spike detection in other phenotypically similar crop cultivars

Summary

Introduction

The predominant majority of previous works were focused on the analysis of spikes visible on the top of plants grown under field conditions, where researchers were primarily interested in assessing spike counts and density per square area [3,4,5]. In contrast to field images, where spikes are only visible on the top of grain crops, greenhouse images of single plants acquired from different rotational angles in side view potentially enable to assess the amount and phenotype of all spikes, including spikes that emerge on the top, and within the mass of plant leaves, as is often the case for many European wheat cultivars. The high-throughput phenotyping of plants in a controlled greenhouse environment is used for the investigation of effects of environmental conditions, such as drought stress, temperature, light intensity as well as their fluctuations [6,7]

Objectives

Methods

Results

Discussion

Conclusion