A methodology for evaluating, comparing, and selecting software safety and reliability standards

Abstract

The regulated industries, which in the FDA's (US Food and Drug Administration) case include the manufacturers of food, cosmetics, pharmaceuticals, and medical devices, are responsible for selecting and justifying the selection and use of appropriate national and/or international consensus standards. This paper documents the results of a scientific inquiry which was undertaken to develop a methodology to evaluate and select individual standards as well as compatible sets of standards. Initially the scope was limited to software safety and reliability standards. Since then it has been expanded to other areas. It was found that six categories of criteria are important when evaluating the potential use of software safety standards: General Factors, Product Characterization, Process Characterization, Personnel Characterization, Risk Management, and Overall Standards Framework. The criteria in each of the six categories must be addressed, How they are addressed is both a technical and a policy decision. The goal should not be to try to find the "ultimate" standard because there isn't any. Rather the goal should be to select a balanced "diet" of standards based on these six categories. The methodology provides a systematic framework from which to make an objective determination about the appropriateness of using a particular standard.