Toward a Multi-Criteria Framework for Selecting Software Testing Tools

Abstract

Software testing is a vital part of software engineering process. Automated testing makes this process more accurate and more efficient. Due to the large number and the variety of testing tools, a comprehensive taxonomy of these features is necessary. This research aims at developing a comprehensive taxonomy for testing tools to help software developers choose the appropriate tool for particular software. The framework includes two main parts: (1) a taxonomy of testing tools; (2) multi-criteria selection method. The first part covers different criteria of testing tools. Because these criteria are large in numbers, wide and variant, a taxonomy of these criteria is needed. This taxonomy will help developers distinguish among testing tools based on a wide spectrum of different criteria. The second part of the framework is a multi-criteria selection method; that enables software developers to choose the appropriate testing tool using a systematic and adequate automated manner. It reduces the cost of the selection process and helps beginners in testing to choose testing tools based on scientific and comprehensive criteria. The selection method employs scientific two well-known methods of multi-criteria decision-making techniques; Analytic Hierarchy Process (AHP) and Technique for Order Preference by Similarity to Ideal Solution (TOPSIS). The testing tools taxonomy is well validated by academic professionals in Software engineering and achieved good scores in terms of significance, usefulness and comprehension. Academics reported that the taxonomy is slightly complex and needs to be simplified. The selection method was implemented as a website and validated using different scenarios to prove the quality of selection even in complex cases.