Enhancing DNN-Based Binary Code Function Search With Low-Cost Equivalence Checking

Abstract

Binary code function search has been used as the core basis of various security and software engineering applications, including malware clustering, code clone detection, and vulnerability audits. Recognizing logically similar assembly functions, however, remains a challenge. Most binary code search tools rely on program structure-level information, such as control flow and data flow graphs, that is extracted using program analysis techniques or deep neural networks (DNNs). However, DNN-based techniques capture lexical-, control structure-, or data flow-level information of binary code for representation learning, which is often too coarse-grained and does not accurately denote program functionality. Additionally, it may exhibit low robustness to a variety of challenging settings, such as compiler optimizations and obfuscations. This paper proposes a general solution for enhancing the top- <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> ranked candidates in DNN-based binary code function search. The key idea is to design a low-cost and comprehensive equivalence check that quickly exposes functionality deviations between the target function and its top- <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> matched functions. Functions that fail this equivalence check can be shaved from the top- <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> list, and functions that pass the check can be revisited to move ahead on the top- <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> ranked candidates, in a deliberate way. We design a practical and efficient equivalence check, named <monospace>BinUSE</monospace> , using <i>under-constrained</i> symbolic execution (USE). USE, a variant of symbolic execution, improves scalability by initiating symbolic execution directly from function entry points and relaxing constraints on function parameters. It eliminates the overhead incurred by path explosion and costly constraints. <monospace>BinUSE</monospace> is specifically designed to deliver an assembly function-level equivalence check, enhancing DNN-based binary code search by reducing its false alarms with low cost. Our evaluation shows that <monospace>BinUSE</monospace> can enable a general and effective enhancement of four state-of-the-art DNN-based binary code search tools when confronted with challenges posed by different compilers, optimizations, obfuscations, and architectures.