Distributed approach for computing rough set approximations of big incomplete information systems

Abstract

The size of information gathered from real world applications today is staggering. To make matters worse, this information may also be incomplete, due to errors in measurement or lack of discipline. The two phenomena give rise to a big incomplete information system (IIS). The processing of a big IIS is difficult because of its two problems, big size and incompleteness, and the present work introduces an approach that addresses both. Specifically, we develop an efficient rough set theoretic (RST) algorithm to compute the approximation space of the IIS, which addresses the incompleteness problem. Then we distribute the computational chores of the algorithm using the MapReduce framework, which addresses the size problem. The approach is explained fully, and a detailed illustrative example is provided. For validation and performance analysis, the approach has been implemented and tested on four publicly-accessible big IISs for many metrics including sizeup, scaleup, and speedup. The experimental results attest to its validity, accuracy and efficiency. A comparison test with similar approaches shows that it has superior performance.