Picture-based and conversational decision support to diagnose post-harvest apple diseases

Abstract

This article presents the development of an expert system to support the diagnosis of post-harvest diseases of stored apples. We propose a picture-based and conversational interaction with users, where sampled images depicting symptoms of apples with known diseases are presented to users to elicit their feedback on perceived similarities in order to determine the most likely diagnosis of a diseased target apple. This article makes, besides the description of the industrial application scenario, multiple contributions circled around three rounds of user studies: (i) an usability and effectiveness assessment of the approach, where three user interface configurations are put to a test and the effectiveness of different types of user feedback mechanisms is assessed; (ii) contextual multi-armed bandit approaches for dynamic selection of displayed images with symptoms of diseased apples, that clearly outperform random and greedy sampling baseline strategies; (iii) a comparison of two different strategies for determining the context representation of a contextual multi-armed bandit approach, namely based on PCA of image features and a gamified large-scale user study. We therefore provide design insights for the development of such diagnosis applications on diseases that manifest themselves through visual symptoms in general and, hence, the findings can be also valid for domains other than post-harvest fruit diseases.