Measuring the quality of design pattern detection results

Abstract

Detecting design patterns in large software systems is a common reverse engineering task that can help the comprehension process of the system's design. While several design pattern detection tools presented in the literature are capable of detecting design patterns automatically, evaluating these detection results is usually done in a manual and subjective fashion. Differences in design pattern definitions, as well as pattern instance counting and presenting, exacerbate the difficulty of evaluating design pattern detection results. In this paper, we present a novel approach to evaluating and comparing design pattern detection results. Our approach, called MoRe, introduces a novel way to present design pattern instances in a uniform fashion. Based on this characterization of design pattern instances, we propose four measures for design pattern detection evaluation that convey a concise assessment of the quality of the results produced by a given detection method. We have implemented these measures, and present case studies that showcase their usefulness.