Heatmap-based Explanation of YOLOv5 Object Detection with Layer-wise Relevance Propagation

Abstract

Deep Neural Networks (DNNs) have been effective in providing solutions to most prevailing problems in the field Computer Vision. Modern Object Detector architecture families such as Single Shot Detector, You Only Look Once (YOLO), Region Based Convolutional Neural Networks (RCNN) and others are powerful models with strong capabilities of distinguishing many common object classes in images with unprecedented speed and accuracy. However, in most of their applications the use of deep networks conceals the inference of object class and location from the input data by introducing many layers of complex black-box functionality. This leads to sometimes bizarre and inexplicable detection outputs. A potential use of these networks in risk-prone applications motivates the development of techniques for visually interpreting their outputs, which could lead to more controlled training and deployment. Layer-wise Relevance Prop-agation (LRP) is a popular “eXplainable AI” (XAI) technique for constructing such explanations, frequently used in investigating the output of DNNs. For object detection, LRP could provide us with a useful heatmap of the input's relevance to the output class, localized within an estimate of a bounding box. In this work, we propose a new and efficient method for conducting relevance propagation in object detector DNNs that leads to visually informative results. In our experiments, we make use of the YOLOv5 model showing meaningful visual explanations of its output. Our implementation philosophy exemplifies how relevance propagation can be regulated for complex modular architectures.