Measuring precision for static and dynamic design pattern recognition as a function of coverage

Abstract

We strive to detect design pattern like patterns in software. This cannot be done efficiently with sufficient precision using only static analysis; we need to combine static and dynamic analysis. In this process, the pattern candidates produced from static analysis are monitored during executions of the software: Candidates detected by static analysis violating the expected dynamic protocol of the pattern are excluded. In this article, we investigate where to put effort when trying to perform high precision pattern detection in code. We do so by investigating which parameters are most important to improve the precision of the detection process: precision of initial static analysis or coverage of the dynamic analysis. Varying the precision of the dynamic analysis is a third important parameter, but this parameter is left as a constant during our experiments. The results show that simple behavioral protocols double the precision when 30% coverage is obtained. We also have indications that simple behavioral protocols give very high precision when high coverage is obtained. In such case, the quality of the static analysis is only interesting for precision if high coverage cannot be reached.