Issues in developing software for safety critical systems

Abstract

This paper discusses the production of software for safety critical systems from two points of view: first, the limits to what we can know about the safety of systems controlled by software; second, principles and techniques for gaining ‘as much confidence as possible’ in the software for critical systems. With respect to the first point the paper discusses the limits in our ability to produce specifications for safe behaviour, and the difficulties of assessing whether or not we have produced a system which is safe. The paper concludes that we can never be certain that we have developed a software controlled critical system that is ‘safe’, but that we can systematically determine which techniques can (if properly applied) reduce uncertainties, or doubts, about system safety. With respect to techniques, the paper introduces two principles for assessing the efficacy of software development methods. It then discusses twelve different facets of software development which are pertinent to the achievement and demonstration of safety, in terms of these principles. These factors range from purely technical issues through quality assurance to project management. The paper concludes with a discussion of the relationship between reliability and confidence that a system is safe.