Runaway Programs: Dr Frankenstein’s Predicament

Abstract

During long-term use every IT system needs to be maintained by making changes that will inexorably undermine good structure and so reduce understanding; in this sense they will always runaway. In general, IT systems defy human comprehension because: 1. Full understanding demands mastery of an excessive span of detail 2. Small misunderstandings can have large repercussions IT systems pose wholly new challenges to human understanding because they are complex discrete systems, quite unlike both natural systems and other engineered complex systems. The discreteness of IT systems is manifest in several ways; it is primarily the single-threaded decision-logic basis of IT systems that is the root problem. Small, “nicely-factored” discrete systems are readily amenable to formal analysis (possibly even proof of correctness); large discrete systems with tight internal couplings (such as most IT systems) are not. Small, “nicely-factored” systems are also readily comprehensible; large ones (made even larger by the avoidance of the succinct power of inter-couplings) are not – they present as a conglomerate of individual components, which, of course, they are if all inter-couplings are eliminated. Complex natural systems achieve stability through redundancy, and predictability through monotonic analogue (rather than discrete) basics. ‘System science’ has failed to encompass systems of discrete complexity. The mathematical basis from which much of programming practice has been developed has severely undervalued redundancy as a system feature. Discrete systems tend to be unpredictable – hence the extreme weakness of IT-system testing as a validation procedure.