Chapter One - Automated Test Oracles: A Survey

Abstract

Abstract Software testing is an essential activity of software development, and oracles are a key pillar of testing. The increasing size of test suites, the growing availability of test case generators that produce enormous amount of test cases, and the repeated execution of large amounts of test cases require automated oracles. Although studied since the late 1970s, in the last decade, test oracles and techniques to automatically generate test oracles have attracted a lot of attention and have witnessed an impressive growth. In this chapter, we survey test oracles with particular attention to their automation. We overview the main evolution of the technology, and we propose some criteria to classify the different approaches. We present the main techniques to automatically generate test oracles classified according to the required information, and we discuss different forms of checkable oracles, defined as oracles expressed in a form directly checkable during the system execution.