Eliminating exception handling errors with dependability cases: a comparative, empirical study

Abstract

Programs fail mainly for two reasons: logic errors in the code and exception failures. Exception failures can account for up to two-thirds of system crashes, hence, are worthy of serious attention. Traditional approaches to reducing exception failures, such as code reviews, walkthroughs, and formal testing, while very useful, are limited in their ability to address a core problem: the programmer's inadequate coverage of exceptional conditions. The problem of coverage might be rooted in cognitive factors that impede the mental generation (or recollection) of exception cases that would pertain in a particular situation, resulting in insufficient software robustness. This paper describes controlled experiments for testing the hypothesis that robustness for exception failures can be improved through the use of various coverage-enhancing techniques: N-version programming, group collaboration, and dependability cases. N-version programming and collaboration are well known. Dependability cases, derived from safety cases, comprise a new methodology based on structured taxonomies and memory aids for helping software designers think about and improve exception handling coverage. All three methods showed improvements over control conditions in increasing robustness to exception failures but dependability cases proved most efficacious in terms of balancing cost and effectiveness.