An Architecture-Based Multi-Objective Optimization Approach to Testing Resource Allocation

Abstract

Software systems are widely employed in society. With a limited amount of testing resource available, testing resource allocation among components of a software system becomes an important issue. Most existing research on the testing resource allocation problem takes a single-objective optimization approach, which may not adequately address all the concerns in the decision-making process. In this paper, an architecture-based multi-objective optimization approach to testing resource allocation is proposed. An architecture-based model is used for system reliability assessment, which has the advantage of explicitly considering system architecture over the reliability block diagram (RBD)-based models, and has good flexibility to different architectural alternatives and component changes. A system cost modeling approach which is based on well-developed software cost models is proposed, which would be a more flexible, suitable approach to the cost modeling of software than the approach adopted by others which is based on an empirical cost model. A multi-objective optimization model is developed for the testing resource allocation problem, in which the three major concerns in the testing resource allocation problem, i.e., system reliability, system cost, and the total amount of testing resource consumed, are taken into consideration. A multi-objective evolutionary algorithm (MOEA), called multi-objective differential evolution based on weighted normalized sum (WNS-MODE), is developed. Experimental studies are presented, and the experiments show several results. 1) The proposed architecture-based multi-objective optimization approach can identify the testing resource allocation strategy which has a good trade-off among optimization objectives. 2) The developed WNS-MODE is better than the MOEA developed in recent research, called HaD-MOEA, in terms of both solution quality and computational efficiency. 3) The WNS-MODE seems quite robust from the sensitivity analysis results.