Testing of RESTful Web APIs

Abstract

RESTful web APIs nowadays may be considered the de facto standard for web integration, since they enable interoperability between heterogeneous software systems in a standard way, and their usage is widespread in industry. Testing these systems thoroughly is therefore of utmost importance: a single bug in an API could compromise hundreds of services using it, potentially affecting millions of end users. In recent years, there has been an explosion in the number of tools and approaches to test RESTful web APIs, making it difficult for researchers and practitioners to select the right solution for the problem at hand. In this tutorial, we overview some of the main industrial and research tools for testing RESTful APIs, with a primarily practical approach. We analyze different testing tools and frameworks from three different perspectives: a) manual vs automated testing; b) black-box vs white-box testing; and c) online vs offline testing. First, we show the capabilities of industrial tools and libraries for manual testing of web APIs, including REST Assured [3] and Postman [1]. Then, we delve into some of the main research tools for automatically generating test cases for RESTful APIs such as RESTler [6], EvoMaster [5], and RESTest [7]. Finally, we overview existing industrial Testing as a Service (TaaS) platforms such as RapidAPI [2] and Sauce Labs [4], and we show the latest research advances on the provision of continuous online testing of RESTful APIs (including automated test generation and execution) with the RESTest testing ecosystem [8]. We finish the tutorial outlining some of the most pressing research challenges in the domain of web API testing automation, which will hopefully open a range of opportunities for future researchers working on the topic.