A survey on uncertainty reasoning and quantification in belief theory and its application to deep learning

Abstract

An in-depth understanding of uncertainty is the first step to making effective decisions under uncertainty. Machine/deep learning (ML/DL) has been hugely leveraged to solve complex problems involved with processing high-dimensional data. However, reasoning and quantifying different uncertainties to achieve effective decision-making have been much less explored in ML/DL than in other Artificial Intelligence (AI) domains. In particular, belief/evidence theories have been studied in Knowledge representation and reasoning (KRR) since the 1960s to reason and measure uncertainties to enhance decision-making effectiveness. Based on our in-depth literature review, only a few studies have leveraged mature uncertainty research in belief/evidence theories in ML/DL to tackle complex problems under different types of uncertainty. Our present survey paper discusses major belief theories and their core ideas dealing with uncertainty causes and types and quantifying them, along with the discussions of their applicability in ML/DL. Particularly, we discuss three main approaches leveraging belief theories in Deep Neural Networks (DNNs), including Evidential DNNs, Fuzzy DNNs, and Rough DNNs, in terms of their uncertainty causes, types, and quantification methods along with their applicability in diverse problem domains. Through an in-depth understanding of the extensive survey on this topic, we discuss insights, lessons learned, limitations of the current state-of-the-art bridging belief theories and ML/DL, and future research directions. This paper conducts an extensive survey by bridging belief theories and deep learning in reasoning and quantifying uncertainty to help researchers initiate uncertainty and decision-making research.