WSEM QT : a novel approach for quality‐based evaluation of web data sources for a data warehouse

Abstract

The incorporation of suitable external data from the World Wide Web offers an effective solution for enriching the data in the data warehouse (DW). However, the main challenge is the quality-aware selection of web data sources to maintain the quality of the DW. In the previous works, the quality evaluation of web sources is through expert evaluation only, which makes it a very lengthy process. Also, since the quality model consists of mixed quality factors from diverse domains of Web, DW and underlying business, finding an expert possessing an expertise of all these domains is a huge bottleneck in the evaluation process. In order to overcome these existing issues, this study proposes a novel multi-level approach web source evaluation with multi-criteria decision-making and web quality testing tools (WSEMQT) and underlying quality model web quality model for evaluating web sources for the DW. The authors introduce automated web source quality evaluation in the first level of web source based evaluation and multiple dimensions of quality evaluation at the second level of expert-based evaluation. At both the levels, multi-criteria decision-making methods are applied to the evaluation scores obtained to ascertain the ranked list of Web sources. The authors present a real-world academic web data case study which shows that the proposed approach can be executed successfully for real-world problems.