Belief theoretic methods for soft and hard data fusion

Abstract

In many contexts, one is confronted with the problem of extract ing information from large amounts of different types soft data (e.g., text) and hard data (from e.g., physics-based sensing systems). In handling hard data, signal and data processing offers a wealth of methods related to modeling, estimation, tracking, and inference tasks. However, soft data present several challenges that necessitate the development of new data processing methods. For example, with suitable statistical natural language processing (NLP) methods, text can be converted into logic statements that are associated with various forms of associated uncertainty related to the credibility of the statement, the reliability of the text source, and so forth. In combining or fusing soft data with either soft or hard data, one must deploy methods that can suitably preserve and update the uncertainty associated with the data, thereby providing uncertainty bounds related to any inferences regarding semantics. Since standard Bayesian probabilistic approaches have problems with suitably handling uncertain logic statements, there is an emerging need for new methods for processing heterogeneous data. In this paper, we describe a framework for fusing soft and hard data based on the Dempster-Shafer (DS) belief theoretic approach which is well-suited to the task of capturing the types of models and uncertain rules that are more typical of soft data. Since the effectiveness of traditional DS methods has been hampered by high computational requirements, we base the processing framework on our new conditional approach to DS theoretic evidence updating and fusion. We address the issue of laying the foundation for a theoretically justifiable, and computationally efficient framework for fusing soft and hard data taking into account the inherent data uncertainty such as reliability and credibility. Moreover, we present an illustrative ex ample that highlights the potential for the DS conditional approach for fusing heterogeneous data.