Denotational trace: a category-theoretic solution to the practical problems of software execution tracing

Abstract

Here, the use of category theory, long advocated as applicable for manipulating many abstractions found in computer programming, is extended to serve additionally as a meta-level design tool to address a practical, software engineering problem: the design and use of effective software execution tracing systems. Software execution tracing is a widely used technique arising out of basic debugging practices with diverse implementations found in many software engineering processes, but it lacks a useful, theoretical foundation, resulting in engineering problems at several levels: for the designers, implementers and users of tracing systems all of whom have no recourse to rigorously defined and clearly understood engineering abstractions, when it comes to either implementing or using tracing effectively. Category theory has been used widely and successfully in computer science for decades, and more recently for a variety of applications in software engineering, establishing it as a key mathematical basis for this engineering discipline. Practical execution tracing systems have not used the existing theoretical concept of trace already provided by computer science – the trace monoid – because monoidal traces, while nevertheless useful in the theoretical investigation of concurrent and non-deterministic systems, are the consequence of an operational view of semantics, and therefore lack the complex, compositional structure required to encode much of the source-oriented information needed in practical execution traces. As a consequence of the categorical duality between operational and denotational semantics, the novel, dual notion of ‘denotational trace’ is introduced, where a ‘canonical’ denotational trace contains the complete collection of source-oriented and compositional, denotational semantic information required for practical execution tracing activities. To highlight the practical, straightforward nature of both denotational tracing implementation and usage, a canonical, denotational tracer is implemented for a simple language, sufficient to provide some simple examples of nevertheless sophisticated uses for denotational traces, including a specification recovery from execution trace and analyses of space and time complexity using formal reasoning applied to traces (i.e., proof by induction, as implied by categorical reasoning based on the denotational basis, for which induction is the associated proof technique). The novel application of category theory as a meta-level design tool, to the long standing software engineering problem of designing and using effective software execution tracing systems, has resulted in an elegant and systematic solution to those problems, further demonstrating that category theory has much to offer software engineering as a practical toolset based on a solid theoretical grounding. Thus category theory has delivered another useful result in software engineering as further evidence of its general applicability to this field, thereby reinforcing its usefulness for modeling, design and implementation, and suggesting this toolset should become a standard part of the software engineering curriculum.