Performance assertion checking

Abstract

Performance assertion checking is an approach to automating the testing of performance properties of complex systems. System designers write assertions that capture expectations for performance; these assertions are checked automatically against monitoring data to detect potential performance bugs. Automatically checking expectations allows a designer to test a wide range of performance properties as a system evolves: data that meets expectations can be discarded automatically, focusing attention on data indicating potential problems. PSpec is a language for writing performance as sertions together with tools for testing assertions and estimating values for constants in assertions. The language is small and efficiently checkable, yet capable of expressing a wide variety of performance properties. Initial experience indicates that PSpec is a useful tool for performance testing and debugging; it helped uncover several performance bugs in the runtime system of a parallel programming language.