Formal specification of the x86 instruction set architecture

Abstract

In this thesis we formally specify the x86 instruction set architecture (ISA) by developing an abstract machine that models the behaviour of a modern computer with multiple x86 processors. Our model enables reasoning about low-level system software by providing formal interpretation of thousand pages of the processor vendor documentation written in informal prose. We show how to reduce the problem of ISA formalization to two simpler problems: memory model specification and instruction semantics specification. We solve the former problem by extending the classical Total Store Ordering memory model with caches, translation-lookaside buffers, memory fences, locks, and other features of the x86 processor. In order to make instruction semantics specification readable and compact, we design a new domain-specific language. The language has intuitive syntax for defining registers and instructions, so that any programmer should be able to understand the specification. Although our language is external and not embedded into a formal proof system, the language is based on the same principles as embedded, monadic domain-specific languages. Thus, it is possible to translate specifications from our language to formal proof systems.