Job-Optimized Map-Side Join Processing Using MapReduce and HBase with Abstract RDF Data

Abstract

The amount of RDF data being published on the Web is increasing at a massive rate. MapReduce-based distributed frameworks have become the general trend in processing SPARQL queries against the RDF data. Currently, query processing systems that use MapReduce have not been able to keep up with increases in semantic annotated data, resulting in non-interactive SPARQL query processing. The principal reason is that intermediate query results from join operations in a MapReduce framework are so massive that network bandwidth and hard disk drive I/O speeds may not keep pace with the processing speed. In this paper, we present an efficient SPARQL processing system that uses MapReduce and HBase. The system runs a job optimized query plan using our proposed abstract RDF data to decrease the amount of intermediate data, thus resulting in faster query processing performance. We also present an efficient algorithm of using Map-side joins while also using the abstract RDF data to filter out unneeded RDF data. Experimental results show that the proposed approach demonstrates better performance when processing queries with a large set of inputs than those found in previous works.