A framework for quality assessment of just-in-time requirements: the case of open source feature requests

Abstract

Until now, quality assessment of requirements has focused on traditional up-front requirements. Contrasting these traditional requirements are just-in-time (JIT) requirements, which are by definition incomplete, not specific and might be ambiguous when initially specified, indicating a different notion of correctness. We analyze how the assessment of JIT requirements quality should be performed based on the literature of traditional and JIT requirements. Based on that analysis, we have designed a quality framework for JIT requirements and instantiated it for feature requests in open source projects. We also indicate how the framework can be instantiated for other types of JIT requirements. We have performed an initial evaluation of our framework for feature requests with eight practitioners from the Dutch agile community, receiving overall positive feedback. Subsequently, we have used our framework to assess 550 feature requests originating from three Open Source Software systems (Netbeans, ArgoUML and Mylyn Tasks). In doing so, we obtain a view on the feature request quality for the three open source projects. The value of our framework is threefold: (1) it gives an overview of quality criteria that are applicable to feature requests (at creation time or JIT); (2) it serves as a structured basis for teams that need to assess the quality of their JIT requirements; and (3) it provides a way to get an insight into the quality of JIT requirements in existing projects.