Null Dereferences Research Articles

Optimizing demand‐driven null dereference verification via merging branches

AbstractNull dereference is a common type of runtime failure in Java programs, and it is necessary to verify whether a dereference in the program is safe. However, previous works often have redundant path exploration and high false positive rate. In this paper, we propose a merged null dereference verification (MNDV) approach. MNDV employs a backward, path‐sensitive inter‐procedural analysis technique to verify a given dereference as safe or potentially unsafe. It uses a branch merging strategy to remove redundant paths, and a method call's relevance to the null references is checked to determine whether it is necessary to explore the internal codes of the method. We have evaluated the approach in some standard benchmark programs. Compared with some existing approaches, our approach reduces false alarm rate and effectively reduce time and memory consumption.

DeepWukong

Static bug detection has shown its effectiveness in detecting well-defined memory errors, e.g., memory leaks, buffer overflows, and null dereference. However, modern software systems have a wide variety of vulnerabilities. These vulnerabilities are extremely complicated with sophisticated programming logic, and these bugs are often caused by different bad programming practices, challenging existing bug detection solutions. It is hard and labor-intensive to develop precise and efficient static analysis solutions for different types of vulnerabilities, particularly for those that may not have a clear specification as the traditional well-defined vulnerabilities. This article presents D eep W ukong , a new deep-learning-based embedding approach to static detection of software vulnerabilities for C/C++ programs. Our approach makes a new attempt by leveraging advanced recent graph neural networks to embed code fragments in a compact and low-dimensional representation, producing a new code representation that preserves high-level programming logic (in the form of control- and data-flows) together with the natural language information of a program. Our evaluation studies the top 10 most common C/C++ vulnerabilities during the past 3 years. We have conducted our experiments using 105,428 real-world programs by comparing our approach with four well-known traditional static vulnerability detectors and three state-of-the-art deep-learning-based approaches. The experimental results demonstrate the effectiveness of our research and have shed light on the promising direction of combining program analysis with deep learning techniques to address the general static code analysis challenges.

Getafix: learning to fix bugs automatically

Static analyzers help find bugs early by warning about recurring bug categories. While fixing these bugs still remains a mostly manual task in practice, we observe that fixes for a specific bug category often are repetitive. This paper addresses the problem of automatically fixing instances of common bugs by learning from past fixes. We present Getafix, an approach that produces human-like fixes while being fast enough to suggest fixes in time proportional to the amount of time needed to obtain static analysis results in the first place. Getafix is based on a novel hierarchical clustering algorithm that summarizes fix patterns into a hierarchy ranging from general to specific patterns. Instead of an expensive exploration of a potentially large space of candidate fixes, Getafix uses a simple yet effective ranking technique that uses the context of a code change to select the most appropriate fix for a given bug. Our evaluation applies Getafix to 1,268 bug fixes for six bug categories reported by popular static analyzers for Java, including null dereferences, incorrect API calls, and misuses of particular language constructs. The approach predicts exactly the human-written fix as the top-most suggestion between 12% and 91% of the time, depending on the bug category. The top-5 suggestions contain fixes for 526 of the 1,268 bugs. Moreover, we report on deploying the approach within Facebook, where it contributes to the reliability of software used by billions of people. To the best of our knowledge, Getafix is the first industrially-deployed automated bug-fixing tool that learns fix patterns from past, human-written fixes to produce human-like fixes.

Detecting all potential null dereferences based on points-to property sound analysis

ABSTRACTNull dereferences are a bane of programming in languages such as C, and static analysis is an effective method to detect them but often due to false negative. This paper proposes a points-to property sound analysis method to detect all potential null dereferences. First, the feature and detection method of null dereference of C programs are introduced. Second, an abstract memory model Region-based Symbolic Three-Valued Logic (RSTVL) is proposed to describe storage states of memory objects. Third, the property sound analysis theory based on abstract interpretation is proposed, which only analyze a single property of a program applies over-approximation strategy. Then, evaluations’ rules of points-to property based on RSTVL are presented, which guarantees the points-to property of a pointer contains the actual points-to property and detect all potential null dereferences. Experiment results of five real projects show that this method could detect all potential null dereferences with acceptable false-positive rates and efficiency.

Casper: Automatic tracking of null dereferences to inception with causality traces

Fixing a software error requires understanding its root cause. In this paper, we introduce “causality traces”, crafted execution traces augmented with the information needed to reconstruct the causal chain from the root cause of a bug to an execution error. We propose an approach and a tool, called Casper, based on code transformation, which dynamically constructs causality traces for null dereference errors. The core idea of Casper is to replace nulls with special objects, called “ghosts”, that track the propagation of the nulls from inception to their error-triggering dereference. Causality traces are extracted from these ghosts. We evaluate our contribution by providing and assessing the causality traces of 14 real null dereference bugs collected over six large, popular open-source projects.

Designing a verifying compiler: Lessons learned from developing Whiley

An ongoing challenge for computer science is the development of a tool which automatically verifies programs meet their specifications, and are free from runtime errors such as divide-by-zero, array out-of-bounds and null dereferences. Several impressive systems have been developed to this end, such as ESC/Java and Spec#, which build on existing programming languages (e.g., Java, C#). We have been developing a programming language from scratch to simplify verification, called Whiley, and an accompanying verifying compiler. In this paper, we present a technical overview of the verifying compiler and document the numerous design decisions made. Indeed, many of our decisions reflect those of similar tools. However, they have often been ignored in the literature and/or spread thinly throughout. In doing this, we hope to provide a useful resource for those building verifying compilers.

Automated refactoring to the Null Object design pattern

ContextNull-checking conditionals are a straightforward solution against null dereferences. However, their frequent repetition is considered a sign of poor program design, since they introduce source code duplication and complexity that impacts code comprehension and maintenance. The Null Object design pattern enables the replacement of null-checking conditionals with polymorphic method invocations that are bound, at runtime, to either a real object or a Null Object. ObjectiveThis work proposes a novel method for automated refactoring to Null Object that eliminates null-checking conditionals associated with optional class fields, i.e., fields that are not initialized in all class instantiations and, thus, their usage needs to be guarded in order to avoid null dereferences. MethodWe introduce an algorithm for automated discovery of refactoring opportunities to Null Object. Moreover, we specify the source code transformation procedure and an extensive set of refactoring preconditions for safely refactoring an optional field and its associated null-checking conditionals to the Null Object design pattern. The method is implemented as an Eclipse plug-in and is evaluated on a set of open source Java projects. ResultsSeveral refactoring candidates are discovered in the projects used in the evaluation and their refactoring lead to improvement of the cyclomatic complexity of the affected classes. The successful execution of the projects’ test suites, on their refactored versions, provides empirical evidence on the soundness of the proposed source code transformation. Runtime performance results highlight the potential for applying our method to a wide range of project sizes. ConclusionOur method automates the elimination of null-checking conditionals through refactoring to the Null Object design pattern. It contributes to improvement of the cyclomatic complexity of classes with optional fields. The runtime processing overhead of applying our method is limited and allows its integration to the programmer’s routine code analysis activities.

An Approach for Identifying Detecting Objects of Null Dereference

On account of the complexity of programs, it is difficult to identify all detecting objects of null dereference, which is one of the preconditions of null dereference detection. This paper introduces an approach for identifying all detecting objects of null dereference of C programs. First, based on the relationship of dereference expressions with nodes of abstract syntax tree (AST), we identify referenced pointers; then based on the abstract storage described by region-based three value logic (RSTVL) and function summary, we identify detecting objects of null dereference. In order to validate the adequacy of our approach, five real-world projects are utilized for experimental analysis, and the results show that our approach could identify all detecting objects of null dereference.

Detecting Null-dereference Bugs via a Backward Analysis

Null dereference is a common occurring bug in programming languages such as C. In this paper, we propose a path-sensitive and context-sensitive approach that performs a backward dataflow analysis to identify null-dereference bugs. One novel feature of our approach is that with the help of aliasing predicates, it can perform strong updates in presence of aliasing, thus eliminating false positives. The aliasing predicates are introduced on the premise of a canonical representation for program being analyzed. Moreover, a context-sensitive algorithm for inter-procedural null-dereference analysis is also presented in this paper, which also contributes to improve accuracy. We have implemented this approach, and give an evaluation of it on a set of open source benchmarks. The experimental results verify the effectiveness of our approach, and show that it is suitable for exploring large real programs with reasonable accuracy.

Experimenting with parasail

This tutorial provides an opportunity to experiment with a new language designed to support the safe, secure, and productive development of parallel programs. ParaSail is a new language with pervasive parallelism coupled with extensive compile-time checking of annotations in the form of assertions, preconditions, postconditions, etc. ParaSail does all checking at compile time, and eliminates race conditions, null dereferences, uninitialized data access, numeric overflow, out of bounds indexing, etc. as well as statically checking the truth of all user-written assertions. After a short introduction to the language, attendees will receive a prototype ParaSail compiler and an accompanying ParaSail Virtual Machine interpreter for writing and testing ParaSail programs. The tutorial/workshop will finish with a group discussion and feedback on the experience of using this new language.

Null dereference verification via over-approximated weakest pre-conditions analysis

Null dereferences are a bane of programming in languages such as Java. In this paper we propose a sound, demand-driven, inter-procedurally context-sensitive dataflow analysis technique to verify a given dereference as safe or potentially unsafe. Our analysis uses an abstract lattice of formulas to find a pre-condition at the entry of the program such that a null-dereference can occur only if the initial state of the program satisfies this pre-condition. We use a simplified domain of formulas, abstracting out integer arithmetic, as well as unbounded access paths due to recursive data structures. For the sake of precision we model aliasing relationships explicitly in our abstract lattice, enable strong updates, and use a limited notion of path sensitivity. For the sake of scalability we prune formulas continually as they get propagated, reducing to true conjuncts that are less likely to be useful in validating or invalidating the formula. We have implemented our approach, and present an evaluation of it on a set of ten real Java programs. Our results show that the set of design features we have incorporated enable the analysis to (a) explore long, inter-procedural paths to verify each dereference, with (b) reasonable accuracy, and (c) very quick response time per dereference, making it suitable for use in desktop development environments.

Mixing type checking and symbolic execution

Static analysis designers must carefully balance precision and efficiency. In our experience, many static analysis tools are built around an elegant, core algorithm, but that algorithm is then extensively tweaked to add just enough precision for the coding idioms seen in practice, without sacrificing too much efficiency. There are several downsides to adding precision in this way: the tool's implementation becomes much more complicated; it can be hard for an end-user to interpret the tool's results; and as software systems vary tremendously in their coding styles, it may require significant algorithmic engineering to enhance a tool to perform well in a particular software domain. In this paper, we present Mix, a novel system that mixes type checking and symbolic execution. The key aspect of our approach is that these analyses are applied independently on disjoint parts of the program, in an off-the-shelf manner. At the boundaries between nested type checked and symbolically executed code regions, we use special mix rules to communicate information between the off-the-shelf systems. The resulting mixture is a provably sound analysis that is more precise than type checking alone and more efficient than exclusive symbolic execution. In addition, we also describe a prototype implementation, Mixy, for C. Mixy checks for potential null dereferences by mixing a null/non-null type qualifier inference system with a symbolic executor.

Gem #58

Ada and Java are two languages that rely heavily on exceptions. A large part of the Ada data model is based on the fact that data is checked at run time, and will raise various kinds of exceptions such as Constraint_Error when constraints are violated. Similarly, there are many cases where Java performs checks that can raise exceptions, among the most common being checks on casts and null dereferences. It is therefore extremely important to support exceptions that are properly propagated from one language to the other and even potentially caught/handled, without having to worry about the language of origin.

Automated verification of pointer programs in pointer logic

Reasoning about pointer programs is difficult and challenging, while their safety is critical in software engineering. Storeless semantics pioneered by Jonkers provides a method to reason about pointer programs. However, the representation of memory states in Jonkers’ model is costly and redundant. This paper presents a new framework under a more efficient storeless model for automatically verifying properties of pointer programs related to the correct use of dynamically allocated memory such as absence of null dereferences, absence of dangling dereferences, absence of memory leaks, and preservation of structural invariants. The introduced logic-Pointer Logic, is developed to achieve such goals. To demonstrate that Pointer Logic is a useful storeless approach to verification, the Schorr-Waite tree-traversal algorithm which is always considered as a key test for pointer formalizations was verified via our analysis. Moreover, an experimental tool-plcc was implemented to automatically verify a number of non-trivial pointer programs.

Reducing the use of nullable types through non-null by default and monotonic non-null

With Java 5 annotations, the authors note a marked increase in tools that can statically detect potential null dereferences. To be effective, such tools require that developers annotate declarations with nullity modifiers and have annotated API libraries. Unfortunately, in the experience of the authors, specifying moderately large code bases, the use of non-null annotations is more labour intensive than it should be. Motivated by this experience, the authors conducted an empirical study of five open source projects totalling 700K lines-of-code, which confirms that, on average, 75% of reference declarations are meant to be non-null, by design. Guided by these results, the authors propose the adoption of non-null-by-default semantics. This new default has advantages of better matching general practice, lightening developer annotation burden and being safer. The authors also describe the Eclipse Java Modelling Language (JML) Java Development Tooling (JDT), a tool supporting the new semantics, including the ability to read the extensive API library specifications written in the JML. Issues of backwards compatibility are addressed. In a second phase of the empirical study, the authors analysed the uses of null and noted that over half of the nullable field references are only assigned non-null values. For this category of reference, the authors introduce the concept of monotonic non-null type and illustrate the benefits of its use.

Static error detection using semantic inconsistency inference

Inconsistency checking is a method for detecting software errors that relies only on examining multiple uses of a value. We propose that inconsistency inference is best understood as a variant of the older and better understood problem of type inference. Using this insight, we describe a precise and formal framework for discovering inconsistency errors. Unlike previous approaches to the problem, our technique for finding inconsistency errors is purely semantic and can deal with complex aliasing and path-sensitive conditions. We have built a nullde reference analysis of C programs based on semantic inconsistency inference and have used it to find hundreds of previously unknown null dereference errors in widely used C programs.

PSE

In this paper, we describe PSE (Postmortem Symbolic Evaluation), a static analysis algorithm that can be used by programmers to diagnose software failures. The algorithm requires minimal information about a failure, namely its kind (e.g. NULL dereference), and its location in the program's source code. It produces a set of execution traces along which the program can be driven to the given failure. PSE tracks the flow of a single value of interest from the point in the program where the failure occurred back to the points in the program where the value may have originated. The algorithm combines a novel dataflow analysis and memory alias analysis in a manner that allows for precise exploration of the program's behavior in polynomial time. We have applied PSE to the problem of diagnosing potential NULL-dereference errors in a suite of C programs, including several SPEC benchmarks and a large commercial operating system. In most cases, the analysis is able to either validate a pointer dereference, or find precise error traces demonstrating a NULL value for the pointer, in less than a second.

Static verification of dynamically detected program invariants: Integrating Daikon and ESC/Java

This paper shows how to integrate two complementary techniques for manipulating program invariants: dynamic detection and static verification. Dynamic detection proposes likely invariants based on program executions, but the resulting properties are not guaranteed to be true over all possible executions. Static verification checks that properties are always true, but it can be difficult and tedious to select a goal and to annotate programs for input to a static checker. Combining these techniques overcomes the weaknesses of each: dynamically detected invariants can annotate a program or provide goals for static verification, and static verification can confirm properties proposed by a dynamic tool.We have integrated a tool for dynamically detecting likely program invariants, Daikon, with a tool for statically verifying program properties, ESC/Java. Daikon examines run-time values of program variables; it looks for patterns and relationships in those values, and it reports properties that are never falsified during test runs and that satisfy certain other conditions, such as being statistically justified. ESC/Java takes as input a Java program annotated with preconditions, postconditions, and other assertions, and it reports which annotations cannot be statically verified and also warns of potential runtime errors, such as null dereferences and out-of-bounds array indices.Our prototype system runs Daikon, inserts its output into code as ESC/Java annotations, and then runs ESC/Java, which reports unverifiable annotations. The entire process is completely automatic, though users may provide guidance in order to improve results if desired. In preliminary experiments, ESC/Java verified all or most of the invariants proposed by Daikon.