Agile Testing of Exceptional Behavior

Abstract

The lack of testing and a priori design of the exceptional behavior are causing many of the problems found in the use of exception handling. As a consequence, exceptions flow in unforeseen ways during the execution of a software system, having a negative impact on reliability. This paper presents an agile approach to test the exceptional behavior of a system. It supports developers in checking whether exceptions, at runtime, travel through the expected paths. It is agile because tests are written without the need for extra documentation and are, themselves, considered live documentation. We have evaluated our approach by applying it to different versions of two production quality Java open source applications (i.e., aTunes and JEdit). Using the proposed approach, we could find twelve bugs --- eight of them previously unknown by the open source projects. In addition, from the viewpoint of automated tests as documentation artifacts, the proposed approach pointed out several differences between versions of the two target systems. We have implemented the proposed approach as an extension of the JUnit framework.