Experimental evaluation of a lightweight method for augmenting requirements analysis

Abstract

We introduce the concept of augmentation methods, methods that complement other methods by addressing specific non-functional requirements (NFRs). Since most projects do not have dedicated expertise in all relevant NFRs most team members may be comparative novices for that class of NFR. STRAP is a lightweight goal-refinement method for analyzing privacy NFRs. We describe it briefly and then present three experiments to assess its effectiveness and that of several existing privacy frameworks. We analyze the results in terms of method efficiency: the number of analysts needed to find a given proportion of benchmark problems. The alternative methods are generally effective in identifying privacy vulnerabilities but they are inefficient, since the average analyst misses many potential problems. In three distinct application domains, STRAP led to equal or better identification of privacy vulnerabilities and was in all cases more efficient. We conclude that a combination of lightweight structuring and heuristic appropriateness is the reason for these advantages.