A Novel Rubric and Feature-based Appraisaland Comparison Framework for the Evaluationof Semantic Web Services Composition Approaches

Abstract

Background/Objectives: This article proposes a novel methodology based on rubrics and feature-based schemes for the appraisal and comparison of approaches to semantic Web services composition and discovery. Methods/Statistical analysis: In order to evaluate the Semantic Web services composition and discovery approaches we created a new framework called RFSWS. This framework is the combination of traditional feature-based evaluation schemes and newly developed analytic rubrics tables. Five recently introduced prominent Semantic Web services composition approaches were identified, explained, and then evaluated/compared using RFSWS. Findings: In this work we determined aspects of Semantic Web services composition and discovery processes that can be evaluated as performance criteria in rubric tables. This is a novel application of rubrics, which have traditionally been used for grading student performance by teachers. We created a novel framework called RFSWS consisting of rubric tables and a feature-based evaluation scheme for the evaluation and comparison of Semantic Web service discovery and composition approaches, and applied it in the evaluation of five recently introduced prominent Semantic Web services composition approaches. Considering the shortcomings of existing Semantic Web services composition approaches that were discovered through this evaluation, we proposed an idealized dynamic Semantic Web service discovery and composition method, a yardstick by which all future Semantic Web services composition approaches can be evaluated. Application/ Improvements: Our novel RFSWS framework can be applied in the comprehensive and systematic evaluation/comparison of Semantic Web services discovery and composition approaches.