The Design of SREE — A Prototype Potential Ambiguity Finder for Requirements Specifications and Lessons Learned

Abstract

[Context and Motivation] Many a tool for finding ambiguities in natural language (NL) requirements specifications (RSs) is based on a parser and a parts-of-speech identifier, which are inherently imperfect on real NL text. Therefore, any such tool inherently has less than 100% recall. Consequently, running such a tool on a NL RS for a highly critical system does not eliminate the need for a complete manual search for ambiguity in the RS. [Question/Problem] Can an ambiguity-finding tool (AFT) be built that has 100% recall on the types of ambiguities that are in the AFT’s scope such that a manual search in an RS for ambiguities outside the AFT’s scope is significantly easier than a manual search of the RS for all ambiguities? [Principal Ideas/Results] This paper presents the design of a prototype AFT, SREE (Systemized Requirements Engineering Environment), whose goal is achieving a 100% recall rate for the ambiguities in its scope, even at the cost of a precision rate of less than 100%. The ambiguities that SREE searches for by lexical analysis are the ones whose keyword indicators are found in SREE’s ambiguity-indicator corpus that was constructed based on studies of several industrial strength RSs. SREE was run on two of these industrial strength RSs, and the time to do a completely manual search of these RSs is compared to the time to reject the false positives in SREE’s output plus the time to do a manual search of these RSs for only ambiguities not in SREE’s scope. [Contribution] SREE does not achieve its goals. However, the time comparison shows that the approach to divide ambiguity finding between an AFT with 100% recall for some types of ambiguity and a manual search for only the other types of ambiguity is promising enough to justify more work to improve the implementation of the approach. Some specific improvement suggestions are offered.