Interpretable Neural Symbol Learning Methods to Fuse Deep Learning Representation and Knowledge Graph: Zhejiang Cuisine Recipe Intangible Cultural Heritage Use Case

Abstract

Deep learning (DL) is difficult to provide explanations verified by non-technical audiences such as end-users or domain experts. This paper uses symbolic knowledge in the form of an expert knowledge graph, and proposes an interpretable neural-symbol learning (RF-YOLOv5) method, designed to learn symbols and deep representations. Finally, the deep learning representation and knowledge map are integrated in the learning process, so as a good basis for interpretability. Among them, the RF-YOLOv5 method involves specific two aspects of interpretation, respectively in reasoning and training time (1) YOLOv5-EXPLANet: experts alignment explained part of the auxiliary network architecture, combined convolutional neural network, using symbol representation, and (2) interpretable artificial intelligence training process, correct and guide the DL process and such symbol representation form of knowledge graph. The camera is placed above the refrigerator to detect the variety of ingredients, and then used in the RF-YOLOv5 method recommended by Zhejiang cuisine recipes, and demonstrates that using our method can improve interpretability while improving interpretability.