Advances on Improving Automation in Developer Testing

Abstract

Developer testing, a common step in software development, involves generating desirable test inputs and checking the behavior of the program unit under test during the execution of the test inputs. Existing developer testing tools include various techniques to address challenges of generating desirable test inputs and checking the behavior of the program unit under test (referred to as test oracles). In this chapter, we present an overview of techniques implemented in these testing tools to address challenges in improving automation in developer testing. In particular, we focus on a recent state-of-the-art technique, called symbolic execution for test inputs. We briefly describe symbolic execution and discuss various challenges (along with the techniques developed to address those challenges) in generating test inputs automatically. For test inputs, the techniques presented in our chapter are summarized from two main aspects: test efficiency (e.g., with a focus on cost) and test effectiveness (e.g., with a focus on benefit). We conclude this chapter by presenting a new frontier, called cooperative developer testing, that is orthogonal to previous techniques and involves synergistic cooperation between humans and tools for effectively generating desirable test inputs.