Representing and querying uncertain data with complex correlations

Abstract

Many applications such as sensor networks, RFID, scientific experimental measurements, stock market prediction, information extraction, etc., need to manage uncertain data and process complex correlations among uncertain data. In probabilistic database systems, uncertain data are represented through attaching probability value to tuples, maybe attributes. Some probabilistic data models assume that tuples are independent of each other and cannot express data correlations effectively. Although others based on probabilistic graph model can capture the representation of uncertainty and complex correlations, the scalability of query and probabilistic inference cannot satisfy the needs of the applications well. In this paper, a novel probabilistic data model RTx-PDM is proposed. RTx-PDM can not only handle arbitrary uncertain data natively at the attribute or tuple level but also represent the correlations among uncertain data with the intuitive BLOCK structure. Especially, RTx-PDM can effectively express shared and schema-level correlations in a compact way through using BLOCK. Traditional relation operators are extended to support manipulating BLOCKs and representing correlations in the operation results. Experimental results validate our approach and demonstrate the effectiveness of exploiting data correlations during query processing.