WSQF: Comprehensive Software Quality Evaluation Framework and Benchmark Based on SQuaRE

Abstract

Conventional quality evaluations of software concentrate on specific quality characteristics. Moreover, the measurement data are limited to specific products and organizations. Consequently, the present state of product quality and quality-in-use characteristics are not fully understood, preventing effective decision-making for software stakeholders. To alleviate this problem, ISO/IEC defined international standards called the SQuaRE (Systems and software Quality Requirements an Evaluation) series for comprehensive quality measurement and evaluation. However, these standards remain rather general and abstract, making them difficult to apply. In this paper, we establish a SQuaRE-based comprehensive software quality evaluation framework, Waseda Software Quality Framework (WSQF), which concretizes many product quality and quality-in-use measurement methods originally defined in the SQuaRE series. By applying the WSQF to 21 commercial ready-to-use software products, we revealed the status of software product quality. A resulted comprehensive benchmark includes trends of the quality measurement values, relationships among quality characteristics, relationship between quality-in-use and product quality, and relationship between the quality characteristics and product contexts within the limits of an application.