On the extraction and relevance ranking of visual morphological traits for taxon identification

Abstract

Deep Neural Networks have proven to be exceptionally successful in applications such as image or speech recognition, yielding prediction results with high accuracy and precision. But due to their black-box nature, Deep Neural Networks are criticized to yield non-transparent and non-explainable results. In the field of taxonomic identification, explainability of the identification results derived by automated image-based identification approaches should rely on common observable morphological traits and their relevance to the derived results. Extending previous work on the deep learning-based Automated Bee Identification System DeepABIS, an approach is presented that equips DeepABIS with a relevance ranking of morphological traits according to their contributions to the prediction results of the automated image-based species identification. The approach shows three steps: (1) Using Visual Backpropagation, a relevance map is generated that maps each pixel of an input image to a relevance score indicating its relevance to the identification result. (2) A deep neural network for semantic segmentation is employed to extract regions of the input image that depict morphological traits (in this work veins, cells and junctions of bee wings). (3) In the fusing step, the morphological traits are assigned aggregated relevance scores using the pixel-wise relevance map obtained in the first step. Experimental results confirm the obtained relevance rankings of the morphological traits by an evaluation using the model-agnostic importance metric Single Feature Importance (SFI).