Semantically Query All (Squerall): A Scalable Framework to Analyze Data from Heterogeneous Sources at Different Levels of Granularity

Abstract

In terms of data formats, codecs, and storage capacity, the previous two decades have seen a phenomenal progress. Rather than needing to adjust to one's application demands to the restricted storage solutions available in the past, there is now a wide range of options to pick from in order to best satisfy an application's needs. As a result, there is massive volume of data available in many forms and formats that, when linked and searched together, may yield significant knowledge and insights. We offer data warehouse as a solution and Big data as a solution in this study in order to handle heterogeneous data. However, both of these techniques have drawbacks when it comes to handling diverse data. Afterwards we propose another framework (Squerall), which relies on the ideas of Ontology-Based Data Access (OBDA) to allow querying of diverse heterogeneous sources using a single query language, SPARQL. In Squerall original data is queried on the fly, with no prior data materialization or modification. Squerall, in particular, enables the distributed aggregate and combining of massive data sets. It comes with five data sources out of the box, and it may be programmatically expanded to include other sources and query engines. The framework includes user interfaces for creating required inputs and for assisting non-SPARQL specialists through the process of writing SPARQL queries. It declares data mappings and transformations using RML and FnO, and employs Spark and Presto as query engines. The initiative underscored the importance of developing in this framework, technologies, and processes that enable for decentralized Big Data administration. Furthermore, demonstrating the feasibility and usefulness of OBDA on top of the growing NoSQL movement has a beneficial impact on Semantic Web principles acceptance. This demonstrates Squerall's importance and contribution to the organization.