Toward Linux kernel memory safety

Abstract

SummaryThe security of billions of devices worldwide depends on the security and robustness of the mainline Linux kernel. However, the increasing number of kernel‐specific vulnerabilities, especially memory safety vulnerabilities, shows that the kernel is a popular and practically exploitable target. Two major causes of memory safety vulnerabilities are reference counter overflows (temporal memory errors) and lack of pointer bounds checking (spatial memory errors). To succeed in practice, security mechanisms for critical systems like the Linux kernel must also consider performance and deployability as critical design objectives. We present and systematically analyze two such mechanisms for improving memory safety in the Linux kernel, ie, (1) an overflow‐resistant reference counter data structure designed to securely accommodate typical reference counter usage in kernel source code and (2) runtime pointer bounds checking using Intel memory protection extension in the kernel. We have implemented both mechanisms and we analyze their security, performance, and deployability. We also reflect on our experience of engaging with Linux kernel developers and successfully integrating the new reference counter data structure into the mainline Linux kernel.