Profiling the operational behavior of OS device drivers

Abstract

As the complexity of modern Operating Systems (OS) increases, testing key OS components such as device drivers (DD) becomes increasingly complex given the multitude of possible DD interactions. Currently, DD testing entails a broad spectrum of techniques, where static (requiring source code) and dynamic (requiring the executable image) and static-dynamic testing combinations are employed. Despite the sustained and improving test efforts in the field of driver development, DDs still represent a significant cause of system outages as the coverage is invariably limited by test resources and release time considerations. The basic factor is the inability to exhaustively assess and then cover the operational states, leading to releases of inadequately tested DDs. Consequently, if representative operational activity profiles of DDs within an OS could be obtained, these could significantly improve the understanding of the actual operational DD state space and help focus the test efforts. Focusing on characterizing DD operational activities while assuming no access to source code, this paper proposes a quantitative technique for profiling the runtime behavior of DDs using a set of occurrence and temporal metrics obtained via I/O traffic characterization. Such profiles are used to improve test adequacy against real-world workloads by enabling similarity quantification across them. The profiles also reveal execution hotspots in terms of DD functionalities activated in the field, thus allowing for dedicated test campaigns. A case study on actual Windows XP and Vista drivers using various performance and stability benchmarks as workloads substantiates our proposed approach.