A novel intrinsically explainable model with semantic manifolds established via transformed priors

Abstract

Because humans instinctively trust and interact with explainable representations instead of latent features, intrinsically interpretable models (IIMs) aimed at representations with semantic meanings have been extensively studied. Previous IIMs relied heavily on data-driven deep neural networks, which resulted in a tremendous demand for data and complex mappings established between instances and their representations. In this study, inspired by how humans build semantic concepts by using meaningful primitives and recognize semantic instances by matching these instances with transformed primitives, we propose the manifold-based semantic representation model (MSRM), which aims to establish a concise mapping with semantic priors for explainable representations. In the MSRM, to reduce reliance on data, we introduced semantic priors into data-driven learning by building a semantic manifold for each different prior with parameterized transformations and then constructed an explainable representation space with these semantic manifolds for given semantic instances. Specifically, the MSRM represents an input semantic instance in three steps: the model extracts the instance’s low-dimensional features as transformation parameters, transforms semantic priors into semantic variations by using these parameters, and calculates the similarity between the input instance and these semantic variations by using inner products as the model’s explainable representation. Hence, the MSRM provides a succinct, prior-guided, and explainable mapping by establishing semantic manifolds by using transformed priors. For application, we propose a manifold-based semantic convolution (MSConv) for visual representation and a simple classification network that includes only one MSConv. The competitive representation power of the MSRM was theoretically analyzed and experimentally verified.