Vulnerable Code Research Articles

Systematic Approach to Prevent Code Vulnerabilities using CI/CD Pipelines

This paper discusses a systematic approach to integrating Static Application Security Testing (SAST), Software Composition Analysis (SCA), Code Coverage, and Code Quality Checks into Continuous Integration/Continuous Delivery (CI/CD) pipelines. Modern CI/CD pipelines accelerate software delivery but introduce significant security and quality challenges. By incorporating SAST and SCA for security testing, along with code coverage and quality checks, organizations can prevent code vulnerabilities and ensure the maintainability and reliability of their applications. This approach helps development teams shift security and quality controls left, catching issues early in the development lifecycle. Keywords — CI/CD, DevSecOps, Static Application Security Testing (SAST), Software Composition Analysis (SCA), Code Coverage, Code Quality, Vulnerability

IDENTIFICATION AND LOCALIZATION OF VULNERABILITIES IN SMART CONTRACTS USING ATTENTION VECTORS ANALYSIS IN A BERT-BASED MODEL

Context. With the development of blockchain technology and the increasing use of smart contracts, which are automatically executed in blockchain networks, the significance of securing these contracts has become extremely relevant. Traditional code auditing methods often prove ineffective in identifying complex vulnerabilities, which can lead to significant financial losses. For example, the reentrancy vulnerability that led to the DAO attack in 2016 resulted in the loss of 3.6 million ethers and the split of the Ethereum blockchain network. This underscores the necessity for early detection of vulnerabilities. Objective. The objective of this work is to develop and test an innovative approach for identifying and localizing vulnerabilities in smart contracts based on the analysis of attention vectors in a model using BERT architecture. Method. The methodology described includes data preparation and training a transformer-based model for analyzing smart contract code. The proposed attention vector analysis method allows for the precise identification of vulnerable code segments. The use of the CodeBERT model significantly improves the accuracy of vulnerability identification compared to traditional methods. Specifically, three types of vulnerabilities are considered: reentrancy, timestamp dependence, and tx.origin vulnerability. The data is preprocessed, which includes the standardization of variables and the simplification of functions. Results. The developed model demonstrated a high F-score of 95.51%, which significantly exceeds the results of contemporary approaches, such as the BGRU-ATT model with an F-score of 91.41%. The accuracy of the method in the task of localizing reentrancy vulnerabilities was 82%. Conclusions. The experiments conducted confirmed the effectiveness of the proposed solution. Prospects for further research include the integration of more advanced deep learning models, such as GPT-4 or T5, to improve the accuracy and reliability of vulnerability detection, as well as expanding the dataset to cover other smart contract languages, such as Vyper or LLL, to enhance the applicability and efficiency of the model across various blockchain platforms. Thus, the developed CodeBERT-based model demonstrates high results in detecting and localizing vulnerabilities in smart contracts, which opens new opportunities for research in the field of blockchain platform security.

Vulnerability detection using BERT based LLM model with transparency obligation practice towards trustworthy AI

Vulnerabilities in the source code are one of the main causes of potential threats in software-intensive systems. There are a large number of vulnerabilities published each day, and effective vulnerability detection is critical to identifying and mitigating these vulnerabilities. AI has emerged as a promising solution to enhance vulnerability detection, offering the ability to analyse vast amounts of data and identify patterns indicative of potential threats. However, AI-based methods often face several challenges, specifically when dealing with large datasets and understanding the specific context of the problem. Large Language Model (LLM) is now widely considered to tackle more complex tasks and handle large datasets, which also exhibits limitations in terms of explaining the model outcome and existing works focus on providing overview of explainability and transparency. This research introduces a novel transparency obligation practice for vulnerability detection using BERT based LLMs. We address the black-box nature of LLMs by employing XAI techniques, unique combination of SHAP, LIME, heat map. We propose an architecture that combines the BERT model with transparency obligation practices, which ensures the assurance of transparency throughout the entire LLM life cycle. An experiment is performed with a large source code dataset to demonstrate the applicability of the proposed approach. The result shows higher accuracy of 91.8 % for the vulnerability detection and model explainability outcome is highly influenced by “vulnerable”, “function”, "mysql_tmpdir_list", “strmov” tokens using both SHAP and LIME framework. Heatmap of attention weights, highlights the local token interactions that aid in understanding the model's decision points.

MVD-HG: multigranularity smart contract vulnerability detection method based on heterogeneous graphs

Smart contracts have significant losses due to various types of vulnerabilities. However, traditional vulnerability detection methods rely extensively on expert rules, resulting in low detection accuracy and poor adaptability to novel attacks. To address these problems, in this paper, deep learning methods are combined with smart contract vulnerability code detection approaches. syntax trees (ASTs), which are special isomorphic graph structures, are an important bridge between source code and graph neural networks. By learning the AST, the model can understand the semantics of the source code. Moreover, graph neural networks have an increasing ability to address complex heterogeneous graphs. Therefore, control flow graphs are fused with data flow graphs on the basis of the ASTs to build heterogeneous graphs with richer code semantics. Furthermore, multigranularity analysis of the vulnerability detection results is performed, including coarse-grained contract-level vulnerability detection and fine-grained line-level vulnerability detection. Through this multigranularity detection approach, vulnerabilities in contracts can be identified and analysed more comprehensively, providing a richer perspective and more solutions for vulnerability detection. The experimental results show that the proposed multigranularity vulnerability detection method based on heterogeneous graphs (MVD-HG) improves both the accuracy and range of the detected vulnerability types in contract-level vulnerability detection tasks; moreover, in the line-level vulnerability detection task, the MVD-HG model achieves significant results and addresses the shortcomings of existing methods. In addition, based on code generation methods used in related fields, a data enhancement method based on the source code is developed, which effectively expands the experimental dataset to address the reduced credibility of the results due to insufficient amounts of data.

Vul-LMGNNs: Fusing language models and online-distilled graph neural networks for code vulnerability detection

Code Language Models (codeLMs) and Graph Neural Networks (GNNs) are widely used in code vulnerability detection. However, a critical yet often overlooked issue is that GNNs primarily rely on aggregating information from adjacent nodes, limiting structural information transfer to single-layer updates. In code graphs, nodes and relationships typically require cross-layer information propagation to fully capture complex program logic and potential vulnerability patterns. Furthermore, while some studies utilize codeLMs to supplement GNNs with code semantic information, existing integration methods have not fully explored the potential of their collaborative effects.To address these challenges, we introduce Vul-LMGNNs that integrates pre-trained CodeLMs with GNNs, leveraging knowledge distillation to facilitate cross-layer propagation of both code semantic knowledge and structural information. Specifically, Vul-LMGNNs utilizes Code Property Graphs (CPGs) to incorporate code syntax, control flow, and data dependencies, while employing gated GNNs to extract structural information in the CPG. To achieve cross-layer information transmission, we implement an online knowledge distillation (KD) program that enables a single student GNN to acquire structural information extracted from a simultaneously trained counterpart through an alternating training procedure. Additionally, we leverage pre-trained CodeLMs to extract semantic features from code sequences. Finally, we propose an ”implicit-explicit” joint training framework to better leverage the strengths of both CodeLMs and GNNs. In the implicit phase, we utilize CodeLMs to initialize the node embeddings of each student GNN. Through online knowledge distillation, we facilitate the propagation of both code semantics and structural information across layers. In the explicit phase, we perform linear interpolation between the CodeLM and the distilled GNN to learn a late fusion model. The proposed method, evaluated across four real-world vulnerability datasets, demonstrated superior performance compared to 17 state-of-the-art approaches. Our source code can be accessed via GitHub: https://github.com/Vul-LMGNN/vul-LMGGNN.

Vulnerability detection based on transformer and high‐quality number embedding

SummarySoftware vulnerability detection is an important problem in software security. In recent years, deep learning offers a novel approach for source code vulnerability detection. Due to the similarities between programming languages and natural languages, many natural language processing techniques have been applied to vulnerability detection tasks. However, specific problems within vulnerability detection tasks, such as buffer overflow, involve numerical reasoning. For these problems, the model needs to not only consider long dependencies and multiple relationships between statements of code but also capture the magnitude property of numerical literals in the program through high‐quality number embeddings. Therefore, we propose VDTransformer, a Transformer‐based method that improves source code embedding by integrating word and number embeddings. Furthermore, we employ Transformer encoders to construct a hierarchical neural network that extracts semantic features from the code and enables line‐level vulnerability detection. To evaluate the effectiveness of the method, we construct a dataset named OverflowGen based on templates for buffer overflow. Experimental comparisons on OverflowGen with a well‐known static vulnerability detector and two state‐of‐the‐art deep learning‐based methods confirm the effectiveness of VDTransformer and the importance of high‐quality number embeddings in vulnerability detection tasks involving numerical features.

DetectVul: A statement-level code vulnerability detection for Python

Detecting vulnerabilities in source code using graph neural networks (GNN) has gained significant attention in recent years. However, the detection performance of these approaches relies highly on the graph structure, and constructing meaningful graphs is expensive. Moreover, they often operate at a coarse level of granularity (such as function-level), which limits their applicability to other scripting languages like Python and their effectiveness in identifying vulnerabilities. To address these limitations, we propose DetectVul, a new approach that accurately detects vulnerable patterns in Python source code at the statement level. DetectVul applies self-attention to directly learn patterns and interactions between statements in a raw Python function; thus, it eliminates the complicated graph extraction process without sacrificing model performance. In addition, the information about each type of statement is also leveraged to enhance the model’s detection accuracy. In our experiments, we used two datasets, CVEFixes and Vudenc, with 211,317 Python statements in 21,571 functions from real-world projects on GitHub, covering seven vulnerability types. Our experiments show that DetectVul outperforms GNN-based models using control flow graphs, achieving the best F1 score of 74.47%, which is 25.45% and 18.05% higher than the best GCN and GAT models, respectively.

Using Large Language Models to Classify Some Vulnerabilities in Program Code

The paper studies the effectiveness of using large language models to detect common types of vulnerabilities in Python program code. In particular, using the technique of low-rank adaptation of (LoRA) models, fine-tuning of the CodeBERT-python model is performed. To train the models, we use the author's dataset, which consists of marked-up program code in Python. The trained models are used to detect and classify potential vulnerabilities. To evaluate the effectiveness of models, the number of false positives, false negatives, true positives and true negatives is determined. Also, accuracy, recall and F1-measures are calculated on a test data set for various configurations of model training macro parameters

Survey of source code vulnerability analysis based on deep learning

Amidst the rapid development of the software industry and the burgeoning open-source culture, vulnerability detection within the software security domain has emerged as an ever-expanding area of focus. In recent years, the rapid advancement of artificial intelligence, particularly the notable progress in deep learning for pattern recognition and natural language processing, has catalyzed a surge in research endeavors exploring the integration of deep learning for the enhancement of vulnerability detection techniques. In this paper, we investigate contemporary deep learning-based source code analysis methods, with a concentrated emphasis on those pertaining to static code vulnerability detection. We categorize these methods based on various representations of source code employed during the preprocessing stage, including token-based and graph-based representations of source code, and further subdivided based on the types of deep learning algorithms or graph representations employed. We summarize the basic processes of model training and vulnerability detection under these different representation formats. Furthermore, we explore the limitations inherent in current approaches and provide insights into future trends and challenges for research in this field.

Fine-grained vulnerability detection for medical sensor systems

The Internet of Things (IoT) has revolutionized the healthcare system by connecting medical sensors to the internet, while also posing challenges to the security of medical sensor networks (MSN). Given the extreme sensitivity of medical data, any vulnerability may result in data breaches and misuse, impacting patient safety and privacy. Therefore, safeguarding MSN security is critical. As medical sensor devices rely on smart healthcare software systems for data management and communication, precisely detecting system code vulnerabilities is essential to ensuring network security. Effective software vulnerability detection targets two key objectives: (i) achieving high accuracy and (ii) directly identifying vulnerable code lines for developers to fix. To address these challenges, we introduce Vulcoder, a novel vulnerability-oriented, encoder-driven model based on the Bidirectional Encoder Representations from Transformers (BERT) architecture. We propose a one-to-one mapping function to capture code semantics through abstract syntax trees (AST). Combined with multi-head attention, Vulcoder achieves precise function- and line-level detection of software vulnerabilities in MSN. This accelerates the vulnerability remediation process, thereby strengthening network security. Experimental results on various datasets demonstrate that Vulcoder outperforms previous models in identifying vulnerabilities within MSN. Specifically, it achieves a 1%–419% improvement in function-level prediction F1 scores and a 12.5%–380% increase in line-level localization precision. Therefore, Vulcoder helps enhance security defenses and safeguard patient privacy in MSN, facilitating the development of smart healthcare.

Preventing Security Code Vulnerabilities in the Age of AI (Artificial Intelligence) Code Assistants

Preventing Security Code Vulnerabilities in the Age of AI (Artificial Intelligence) Code Assistants

Hidden code vulnerability detection: A study of the Graph-BiLSTM algorithm

Context:The accelerated growth of the Internet and the advent of artificial intelligence have led to a heightened interdependence of open source products, which has in turn resulted in a rise in the frequency of security incidents. Consequently, the cost-effective, fast and efficient detection of hidden code vulnerabilities in open source software products has become an urgent challenge for both academic and engineering communities. Objectives:In response to this pressing need, a novel and efficient code vulnerability detection model has been proposed: the Graph-Bi-Directional Long Short-Term Memory Network Algorithm (Graph-BiLSTM). The algorithm is designed to enable the detection of vulnerabilities in Github’s code commit records on a large scale, at low cost and in an efficient manner. Methods:In order to extract the most effective code vulnerability features, state-of-the-art vulnerability datasets were compared in order to identify the optimal training dataset. Initially, the Joern tool was employed to transform function-level code blocks into Code Property Graphs (CPGs). Thereafter, structural features (degree centrality, Katz centrality, and closeness centrality) of these CPGs were computed and combined with the embedding features of the node sequences to form a two-dimensional feature vector space for the function-level code blocks. Subsequently, the BiLSTM network algorithm was employed for the automated extraction and iterative model training of a substantial number of vulnerability code samples. Finally, the trained algorithmic model was applied to code commit records of open-source software products on GitHub, achieving effective detection of hidden code vulnerabilities. Conclusion:Experimental results indicate that the PrimeVul dataset represents the most optimal resource for vulnerability detection. Moreover, the Graph-BiLSTM model demonstrated superior performance in terms of accuracy, training cost, and inference time when compared to state-of-the-art algorithms for the detection of vulnerabilities in open-source software code on GitHub. This highlights the significant value of the model for engineering applications.

Towards a Block-Level ML-Based Python Vulnerability Detection Tool

Computer software is driving our everyday life, therefore their security is pivotal. Unfortunately, security flaws are common in software systems, which can result in a variety of serious repercussions, including data loss, secret information disclosure, manipulation, or system failure. Although techniques for detecting vulnerable code exist, the improvement of their accuracy and effectiveness to a practically applicable level remains a challenge. Many existing methods require a substantial amount of human expert labor to develop attributes that indicate vulnerabilities. In previous work, we have shown that machine learning is suitable for solving the issue automatically by learning features from a vast collection of real-world code and predicting vulnerable code locations. Applying a BERT-based code embedding, LSTM models with the best hyperparameters were able to identify seven different security flaws in Python source code with high precision (average of 91%) and recall (average of 83%). Upon the encouraging first empirical results, we go beyond this paper and discuss the challenges of applying these models in practice and outlining a method that solves these issues. Our goal is to develop a hands-on tool for developers that they can use to pinpoint potentially vulnerable spots in their code.

Timing Side-Channel Mitigation via Automated Program Repair

Side-channel vulnerability detection has gained prominence recently due to Spectre and Meltdown attacks. Techniques for side-channel detection range from fuzz testing to program analysis and program composition. Existing side-channel mitigation techniques repair the vulnerability at the IR/binary level or use runtime monitoring solutions. In both cases, the source code itself is not modified, can evolve while keeping the vulnerability, and the developer would get no feedback on how to develop secure applications in the first place. Thus, these solutions do not help the developer understand the side-channel risks in her code and do not provide guidance to avoid code patterns with side-channel risks. In this paper, we present Pendulum , the first approach for automatically locating and repairing side-channel vulnerabilities in the source code, specifically for timing side channels. Our approach uses a quantitative estimation of found vulnerabilities to guide the fix localization, which goes hand-in-hand with a pattern-guided repair. Our evaluation shows that Pendulum can repair a large number of side-channel vulnerabilities in real-world applications. Overall, our approach integrates vulnerability detection, quantization, localization, and repair into one unified process. This also enhances the possibility of our side-channel mitigation approach being adopted into programming environments.

SCL-CVD: Supervised contrastive learning for code vulnerability detection via GraphCodeBERT

Detecting vulnerabilities in source code is crucial for protecting software systems from cyberattacks. Pre-trained language models such as CodeBERT and GraphCodeBERT have been applied in multiple code-related downstream tasks such as code search and code translation and have achieved notable success. Recently, this pre-trained and fine-tuned paradigm has also been applied to detect code vulnerabilities. However, fine-tuning pre-trained language models using cross-entropy loss has several limitations, such as poor generalization performance and lack of robustness to noisy labels. In particular, when the vulnerable code and the benign code are very similar, it is difficult for deep learning methods to differentiate them accurately. In this context, we introduce a novel approach for code vulnerability detection using supervised contrastive learning, namely SCL-CVD, which leverages GraphCodeBERT. This method aims to enhance the effectiveness of existing vulnerable code detection approaches. SCL-CVD represents the source code as data flow graphs. These graphs are then processed by GraphCodeBERT, which has been fine-tuned using a supervised contrastive loss function combined with R-Drop. This fine-tuning process is designed to generate more resilient and representative code embedding. Additionally, we incorporate LoRA (Low-Rank Adaptation) to streamline the fine-tuning process, significantly reducing the time required for model training. Finally, a Multilayer Perceptron (MLP) is employed to detect vulnerable code leveraging the learned representation of code. We designed and conducted experiments on three public benchmark datasets, i.e., Devign, Reveal, Big-Vul, and a combined dataset created by merging these sources. The experimental results demonstrate that SCL-CVD can effectively improve the performance of code vulnerability detection. Compared with the baselines, the proposed approach has a relative improvement of 0.48%∼3.42% for accuracy, 0.93%∼45.99% for precision, 35.68%∼67.48% for recall, and 16.31%∼49.67% for F1-score, respectively. Furthermore, compared to baselines, the model fine-tuning time of the proposed approach is reduced by 16.67%∼93.03%. In conclusion, our approach SCL-CVD offers significantly greater cost-effectiveness over existing approaches.

Безопасный конвейер доставки программного обеспечения

The presence of vulnerabilities in software is a pressing problem. Vulnerabilities can serve as a basis for breach of confidentiality and information leakage. The purpose of this study is to increase the level of software security at all stages of the life cycle from development and implementation to operation. Achieving this goal is possible through automated analysis of program code and increasing the types of vulnerabilities detected. The work proposes a secure software delivery pipeline that allows for static and dynamic analysis of program code, analysis of the search for vulnerabilities in third-party components and Docker images. The article reviewed popular software tools, their distinctive features, and provided justification for the choice of software solutions that form the basis of the developed secure delivery pipeline. The novelty of the work is the ability of the pipeline to automatically detect vulnerabilities at all stages of the software life cycle, from planning and design to testing, deployment and monitoring in a production environment, which allows you to eliminate vulnerabilities at an early stage, thereby increasing the level of software security. Conducted testing and approbation of a secure software delivery pipeline. Based on the assessment results, the developed secure software delivery pipeline showed that on average 98% of vulnerabilities were identified using the Semgrep tool, 90% of vulnerabilities using the OWASP ZAP tool, 96% of vulnerabilities using the Dependency-Track tool, and 88% using the Trivy tool. The results of the study and experimental data showed that on average, as a result of testing, the accuracy of detecting vulnerabilities is 93%. The practical value of the work lies in the fact that the developed secure software delivery pipeline can be used as a tool for detecting program code vulnerabilities by software development specialists, as well as information security specialists of IT companies. The results obtained can be used in the field of secure software development, formalization and interpretation of vulnerabilities in program code, which will make it possible to create new rules for their identification and development of countermeasures to neutralize them.

A vulnerability detection framework by focusing on critical execution paths

Context:Vulnerability detection is critical to ensure software security, and detecting vulnerabilities in smart contract code is currently gaining massive attention. Existing deep learning-based vulnerability detection methods represent the code as a code structure graph and eliminate vulnerability-irrelevant nodes. Then, they learn vulnerability-related code features from the simplified graph for vulnerability detection. However, this simplified graph struggles to represent relatively complete structural information of code, which may affect the performance of existing vulnerability detection methods. Objective:In this paper, we present a novel Vulnerability Detection framework based on Critical Execution Paths (VDCEP), which aims to improve smart contract vulnerability detection. Method:Firstly, given a code structure graph, we deconstruct it into multiple execution paths that reflect rich structural information of code. To reduce irrelevant code information, a path selection strategy is employed to identify critical execution paths that may contain vulnerable code information. Secondly, a feature extraction module is adopted to learn feature representations of critical paths. Finally, we feed all path feature representations into a classifier for vulnerability detection. Also, the feature weights of paths are provided to measure their importance in vulnerability detection. Results:We evaluate VDCEP on a large dataset with four types of smart contract vulnerabilities. Results show that VDCEP outperforms 14 representative vulnerability detection methods by 5.34%–60.88% in F1-score. The ablation studies analyze the effects of our path selection strategy and feature extraction module on VDCEP. Moreover, VDCEP still outperforms ChatGPT by 34.46% in F1-score. Conclusion:Compared to existing vulnerability detection methods, VDCEP is more effective in detecting smart contract vulnerabilities by utilizing critical execution paths. Besides, we can provide interpretable details about vulnerability detection by analyzing the path feature weights.

IGnnVD: A novel software vulnerability detection model based on integrated graph neural networks

Software vulnerability detection is a challenging task in the security field, the boom of deep learning technology promotes the development of automatic vulnerability detection. Compared with sequence-based deep learning models, graph neural network (GNN) can learn the structural features of code, it performs well in the field of vulnerability detection for source code. However, different GNNs have different detection results for the same code, and using a single kind of GNN may lead to high false positive rate and false negative rate. In addition, the complex structure of source code causes single GNN model cannot effectively learn their depth feature, thereby leading to low detection accuracy. To solve these limitations, we propose a software vulnerability detection model called iGnnVD based on the integrated graph neural networks. In the proposed iGnnVD model, the base detectors including GCN, GAT and APPNP are first constructed to capture the bidirectional information in the code graph structure with bidirectional structure; And then, the residual connection is used to aggregate the features while retaining the features each time; Finally, the convolutional layer is used to perform the aggregated classification. In addition, an integration module that analyzes the detection results of three detectors for final classification is designed using a voting strategy to solve the problem of high false positive rate and false negative rate caused by using a single kind of base detector. We perform extensive experiments on three datasets and experimental results show that the proposed iGnnVD model can improve the detection accuracy of vulnerabilities in source code as well as reduce the false positive rate and false negative rate compared with existing deep learning-based vulnerability detection models, it also has good stability.

SVulDetector: Vulnerability detection based on similarity using tree-based attention and weighted graph embedding mechanisms

Vulnerability detection by comparing similarities with known vulnerable code is an important method for improving code security, and is particularly effective in detecting vulnerabilities caused by code reuse. However, vulnerability detection is made difficult by the existence of some different and vulnerability-unrelated statements between codes with the same vulnerability pattern, as well as the small differences between vulnerable and fixed non-vulnerable codes. To address these challenges, we believe that more attention needs to be paid to some core syntactic and semantic information about vulnerabilities, which can help models more accurately identify vulnerable code. Hence, we propose a novel code-similarity-based vulnerability detection approach named SVulDetector. First, it contains a code representation, called Slice Composite Graphs (SCGs), which can represent rich syntactic and semantic information related to vulnerable statements while minimizing the interference from similar vulnerability irrelevant information as much as possible. Next, a tree-based attention mechanism is used to highlight certain key syntactic information in vulnerable code and fixed non-vulnerable code. Finally, SVulDetector highlights key vulnerable node information in the graph-based code representation via a weighted graph embedding mechanism. We extensively evaluated SVulDetector on an improved real-world dataset using both two-class and multi-class vulnerability detection tasks, and the proposed SVulDetector outperforms existing state-of-the-art detection methods.

Survey of techniques to detect common weaknesses in program binaries

Software vulnerabilities resulting from coding weaknesses and poor development practices are common. Attackers can exploit these vulnerabilities and impact the security and privacy of end-users. Most end-user software is distributed as program binaries. Therefore, to increase trust in third-party software, researchers have built techniques and tools to detect and resolve different classes of coding weaknesses in binary software. Our work is motivated by the need to survey the state-of-the-art and understand the capabilities and challenges faced by binary-level techniques that were built to detect the most important coding weaknesses in software binaries. Therefore, in this paper, we first show the most critical coding weaknesses for compiled programming languages. We then survey, explore, and compare the static techniques that were developed to detect each such coding weakness in software binaries. Our other goal in this work is to discover and report the state of published open-source implementations of static binary-level security techniques. For the open-source frameworks that work as documented, we independently evaluate their effectiveness in detecting code vulnerabilities on a suite of program binaries. To our knowledge, this is the first work that surveys and independently evaluates the performance of state-of-the-art binary-level techniques to detect weaknesses in binary software.