Handling SQL Databases in Automated System Test Generation

Abstract

Automated system test generation for web/enterprise systems requires either a sequence of actions on a GUI (e.g., clicking on HTML links and form buttons) or direct HTTP calls when dealing with web services (e.g., REST and SOAP). When doing white-box testing of such systems, their code can be analyzed, and the same type of heuristics (e.g., the branch distance ) used in search-based unit testing can be employed to improve performance. However, web/enterprise systems do often interact with a database. To obtain higher coverage and find new faults, the state of the databases needs to be taken into account when generating white-box tests. In this work, we present a novel heuristic to enhance search-based software testing of web/enterprise systems, which takes into account the state of the accessed databases. Furthermore, we enable the generation of SQL data directly from the test cases. This is useful when it is too difficult or time consuming to generate the right sequence of events to put the database in the right state. Also, it is useful when dealing with databases that are “read-only” for the system under test, and the actual data are generated by other services. We implemented our technique as an extension of E VO M ASTER , where system tests are generated in the JUnit format. Experiments on six RESTful APIs (five open-source and one industrial) show that our novel techniques improve coverage significantly (up to +16.5%), finding seven new faults in those systems.