Doubly-linked Lists Research Articles

GhostCell: separating permissions from data in Rust

The Rust language offers a promising approach to safe systems programming based on the principle of aliasing XOR mutability : a value may be either aliased or mutable, but not both at the same time. However, to implement pointer-based data structures with internal sharing, such as graphs or doubly-linked lists, we need to be able to mutate aliased state. To support such data structures, Rust provides a number of APIs that offer so-called interior mutability : the ability to mutate data via method calls on a shared reference. Unfortunately, the existing APIs sacrifice flexibility, concurrent access, and/or performance, in exchange for safety. In this paper, we propose a new Rust API called GhostCell which avoids such sacrifices by separating permissions from data : it enables the user to safely synchronize access to a collection of data via a single permission. GhostCell repurposes an old trick from typed functional programming: branded types (as exemplified by Haskell’s ST monad), which combine phantom types and rank-2 polymorphism to simulate a lightweight form of state-dependent types. We have formally proven the soundness of GhostCell by adapting and extending RustBelt, a semantic soundness proof for a representative subset of Rust, mechanized in Coq.

Graph Inclusion and Matching Algorithms for Programs Manipulating Singly linked Heaps

Programs that manipulate heaps such as singlylinked lists, doublylinked lists, skiplists, and treesare ubiquitous, and hence ensuring their correctness is of utmost importance. Analysing correctness properties for such programs is not trivial since they induce dynamic data structures, leading to unbounded state spaces with intricate patterns. One approach that has been adopted to tackle this problem is the use of symbolic searching techniques. The state space is encoded using graphs where the nodes represent memory cells, and the edges represent pointers between the cells. It is necessary to prune the search to avoid generating massive numbers of graphs, thus making the procedure unpractical. Pruning strategies are defined based on operations such as graph matching and inclusion. In this paper, a set of algorithms for performing these operations are presented. It is demonstrated that the proposed algorithms can handle typical graphs that arise in the verification of heap manipulating programs.

How do programmers use unsafe rust?

Rust’s ownership type system enforces a strict discipline on how memory locations are accessed and shared. This discipline allows the compiler to statically prevent memory errors, data races, inadvertent side effects through aliasing, and other errors that frequently occur in conventional imperative programs. However, the restrictions imposed by Rust’s type system make it difficult or impossible to implement certain designs, such as data structures that require aliasing (e.g. doubly-linked lists and shared caches). To work around this limitation, Rust allows code blocks to be declared as unsafe and thereby exempted from certain restrictions of the type system, for instance, to manipulate C-style raw pointers. Ensuring the safety of unsafe code is the responsibility of the programmer. However, an important assumption of the Rust language, which we dub the Rust hypothesis , is that programmers use Rust by following three main principles: use unsafe code sparingly, make it easy to review, and hide it behind a safe abstraction such that client code can be written in safe Rust. Understanding how Rust programmers use unsafe code and, in particular, whether the Rust hypothesis holds is essential for Rust developers and testers, language and library designers, as well as tool developers. This paper studies empirically how unsafe code is used in practice by analysing a large corpus of Rust projects to assess the validity of the Rust hypothesis and to classify the purpose of unsafe code. We identify queries that can be answered by automatically inspecting the program’s source code, its intermediate representation MIR, as well as type information provided by the Rust compiler; we complement the results by manual code inspection. Our study supports the Rust hypothesis partially: While most unsafe code is simple and well-encapsulated, unsafe features are used extensively, especially for interoperability with other languages.

Adding Decision Procedures to SMT Solvers Using Axioms with Triggers

Satisfiability modulo theories (SMT) solvers are efficient tools to decide the satisfiability of ground formulas, including a number of built-in theories such as congruence, linear arithmetic, arrays, and bit-vectors. Adding a theory to that list requires delving into the implementation details of a given SMT solver, and is done mainly by the developers of the solver itself. For many useful theories, one can alternatively provide a first-order axiomatization. However, in the presence of quantifiers, SMT solvers are incomplete and exhibit unpredictable behavior. Consequently, this approach can not provide us with a complete and terminating treatment of the theory of interest. In this paper, we propose a framework to solve this problem, based on the notion of instantiation patterns, also known as triggers. Triggers are annotations that suggest instances which are more likely to be useful in proof search. They are implemented in all SMT solvers that handle first-order logic and are included in the SMT-LIB format. In our framework, the user provides a theory axiomatization with triggers, along with a proof of completeness and termination properties of this axiomatization, and obtains a sound, complete, and terminating solver for her theory in return. We describe and prove a corresponding extension of the traditional Abstract DPLL Modulo Theory framework. Implementing this mechanism in a given SMT solver requires a one-time development effort. We have implemented the proposed extension in the Alt-Ergo prover and we discuss some implementation details in the paper. To show that our framework can handle complex theories, we prove completeness and termination of a feature-rich axiomatization of doubly-linked lists. Our tests show that our approach results in a better performance of the solver on goals that stem from the verification of programs manipulating doubly-linked lists and sets.

Verification of heap manipulating programs with ordered data by extended forest automata

We present a general framework for verifying programs with complex dynamic linked data structures whose correctness depends on ordering relations between stored data values. The underlying formalism of our framework is that of forest automata (FA), which has previously been developed for verification of heap-manipulating programs. We extend FA with constraints between data elements associated with nodes of the heaps represented by FA, and we present extended versions of all operations needed for using the extended FA in a fully-automated verification approach, based on abstract interpretation. We have implemented our approach as an extension of the Forester tool and successfully applied it to a number of programs dealing with data structures such as various forms of singly- and doubly-linked lists, binary search trees, as well as skip lists.

Scalable data structure detection and classification for C/C++ binaries

Many existing techniques for reversing data structures in C/C ++ binaries are limited to low-level programming constructs, such as individual variables or structs. Unfortunately, without detailed information about a program's pointer structures, forensics and reverse engineering are exceedingly hard. To fill this gap, we propose MemPick, a tool that detects and classifies high-level data structures used in stripped binaries. By analyzing how links between memory objects evolve throughout the program execution, it distinguishes between many commonly used data structures, such as singly- or doubly-linked lists, many types of trees (e.g., AVL, red-black trees, B-trees), and graphs. We evaluate the technique on 10 real world applications, 4 file system implementations and 16 popular libraries. The results show that MemPick can identify the data structures with high accuracy.

Attribute grammars as tree transducers over cyclic representations of infinite trees and their descriptional composition

Evaluation of attributes w.r.t. an attribute grammar can be obtained by inductively computing a function expressing the dependencies of the synthesized attributes on inherited attributes. This higher-order functional approach to attribute evaluation can straightforwardly be implemented in a higher-order lazy functional language like Haskell. The resulting evaluation functions are, however, not easily amenable to optimization when we want to compose two attribute grammars. We present an alternative first-order functional interpretation of attribute grammars where the input tree is replaced by an extended cyclic tree each node of which is aware of its context viewed as an additional child tree. These cyclic representations of zippers (trees with their context) are natural generalizations of doubly-linked lists to trees over an arbitrary signature. Then we show that, up to that representation, descriptional composition of attribute grammars reduces to the composition of tree transducers.

MONOTONIC ABSTRACTION FOR PROGRAMS WITH MULTIPLY-LINKED STRUCTURES

We investigate the use of monotonic abstraction and backward reachability analysis as means of performing shape analysis on programs with multiply pointed structures. By encoding the heap as a vertex- and edge-labeled graph, we can model the low level behaviour exhibited by programs written in the C programming language. Using the notion of signatures, which are predicates that define sets of heaps, we can check properties such as absence of null pointer dereference and shape invariants. We report on the results from running a prototype based on the method on several programs such as insertion into and merging of doubly-linked lists.

Built-in Coloring for Highly-Concurrent Doubly-Linked Lists

This paper presents a novel approach for highly-concurrent nonblocking implementations of doubly-linked lists, based on dynamically maintaining a coloring of the nodes in the list. In these implementations, operations on non-adjacent nodes in the linked-list proceed without interfering with each other. Roughly speaking, the operations are implemented by acquiring nodes in the operation's data set, in the order of their colors, and then making the changes atomically. The length of waiting chains is restricted, thereby increasing concurrency, because the colors are taken from a small set. Operations carefully update the colors of the nodes they modify, so neighboring nodes in the list have different colors. A helping mechanism ensures progress in small neighborhoods of processes that keep taking steps. We use this approach in two new algorithms: CAS-Chromo uses an unary conditional primitive, cas, and allows insertions anywhere in the linked list and removals only at the ends, while DCAS-Chromo allows insertions and removals anywhere but uses a stronger primitive, dcas.

Forest automata for verification of heap manipulation

We consider verification of programs manipulating dynamic linked data structures such as various forms of singly and doubly-linked lists or trees. We consider important properties for this kind of systems like no null-pointer dereferences, absence of garbage, shape properties, etc. We develop a verification method based on a novel use of tree automata to represent heap configurations. A heap is split into several "separated" parts such that each of them can be represented by a tree automaton. The automata can refer to each other allowing the different parts of the heaps to mutually refer to their boundaries. Moreover, we allow for a hierarchical representation of heaps by allowing alphabets of the tree automata to contain other, nested tree automata. Program instructions can be easily encoded as operations on our representation structure. This allows verification of programs based on symbolic state-space exploration together with refinable abstraction within the so-called abstract regular tree model checking. A motivation for the approach is to combine advantages of automata-based approaches (higher generality and flexibility of the abstraction) with some advantages of separation-logic-based approaches (efficiency). We have implemented our approach and tested it successfully on multiple non-trivial case studies.

Attribute Grammars as Recursion Schemes over Cyclic Representations of Zippers

Evaluation of attributes w.r.t. an attribute grammar can be obtained by inductively computing a function expressing the dependencies of the synthesized attributes on inherited attributes. This higher-order functional approach to attribute grammars leads to a straightforward implementation using a higher-order lazy functional language like Haskell. The resulting evaluation functions are, however, not easily amenable to optimization rules. We present an alternative first-order functional interpretation of attribute grammars where the input tree is replaced with an extended cyclic tree each node of which is aware of its context viewed as an additional child tree. By the way, we demonstrate that these cyclic representations of zippers (trees with their context) are natural generalizations of doubly-linked lists to trees over an arbitrary signature.

Automated Theorem Prover for Pointer Logic

This paper presents a technique for designing theorem prover which mainly based on transformation and substitution for Pointer Logic. The technique realized as a tool called APL is implemented. The APL theorem prover is fully automated with which proofs can be recorded and checked efficiently. The tool is tested on pointer programs mainly about singly-linked lists, doubly-linked lists and binary trees.

Verifying properties of well-founded linked lists

We describe a novel method for verifying programs that manipulate linked lists, based on two new predicates that characterize reachability of heap cells. These predicates allow reasoning about both acyclic and cyclic lists uniformly with equal ease. The crucial insight behind our approach is that a circular list invariably contains a distinguished head cell that provides a handle on the list. This observation suggests a programming methodology that requires the heap of the program at each step to be well-founded , i.e., for any field f in the program, every sequence u .f, u .f.f, ... contains at least one head cell. We believe that our methodology captures the most common idiom of programming with linked data structures. We enforce our methodology by automatically instrumenting the program with updates to two auxiliary variables representing these predicates and adding assertions in terms of these auxiliary variables.To prove program properties and the instrumented assertions, we provide a first-order axiomatization of our two predicates. We also introduce a novel induction principle made possible by the well-foundedness of the heap. We use our induction principle to derive from two basic axioms a small set of additional first-order axioms that are useful for proving the correctness of several programs.We have implemented our method in a tool and used it to verify the correctness of a variety of nontrivial programs manipulating both acyclic and cyclic singly-linked lists and doubly-linked lists. We also demonstrate the use of indexed predicate abstraction to automatically synthesize loop invariants for these examples.

Pattern-Based Verification of Programs with Extended Linear Linked Data Structures

The paper deals with the problem of automatic verification of programs with dynamic linked data structures. In particular, the use of pattern-based abstraction of memory configurations is considered. In this approach, one can abstract memory configurations by abstracting away the exact number of adjacent occurrences of certain memory patterns. The paper extends the state-of-the-art in this area by proposing a fully automatic and efficient way of detecting the memory patterns to be used from the memory configurations that the program at hand is generating. The method targets programs manipulating a broad class of extended linear linked data structures having a linear skeleton (possibly bidirectionally-linked or cyclic) with certain additional pointers defined on top of it, which covers many practical dynamic data structures (such as lists, doubly-linked lists, cyclic lists, lists with tail/head pointers, etc.). The experimental results obtained from a prototype implementation of the method show that the method is very competitive and offers a big potential for future extensions.

Doubly-linked opportunities

Does the expression, Opportunity always knocks at the least opportune moment, mean anything to you? Well, it seems to control my life. Recently, while teaching a C++ class, the topic of sorting doublylinked lists came up. I thought this was a subject worth pursuing. so I told my class to consider the problem and that we would cover the matter during the next meeting. Now, it was time for me to do my homework. My usual approach to a situation like this is to review all the textbooks that fill my book shelves and cover my floor. No success. This left me with an opportunity (remember the expression) at a time when I had at least a hundred other things on my To Do list. The result of my efforts are presented below.

Verification of Pointers

<p>Our recent work links type checking in programming languages to verification based on automata. In this survey, we give an overview of our methods and suggest directions for future research.</p><p> </p><p>In our approach, we view data types as invariants and devise a logical and decidable framework for expressing global properties of a store consisting of records and pointers.</p><p>We can express common properties, for example about doubly-linked lists and their algorithms. Such properties seemed to have called for a full Hoare logic beyond the reach of type checking and decidability.</p><p> </p><p>Our work is based on monadic second-order logic. Thus verification boils down to calculations on finite-state automata. This raises specific questions about combinatorial techniques for representing state spaces if these calculations are ever to be carried out on more than simple examples.</p>

Self-Organizing Doubly-Linked Lists

In this paper, we study the problem of maintaining a doubly-linked list (DLL) in approximately optimal order, with respect to the mean search time. We study two types of DLL reorganization strategies. Move-To-End (MTE) [12] and SWAP [14] are two memoryless DLL heuristics obtained from natural extensions of the well-known singly-linked-list (SLL) heuristics, move-to-front and transposition, respectively. We first derive a general sufficient condition which permits comparison of any two DLL heuristics. We use this condition as a guideline to identify families of access distributions for which SWAP yields a lower expected cost than the MTE. We have also presented an absorbing DLL heuristic. The strategy requires one additional memory location and is analogous to the scheme presented in [15]. The reorganization is achieved by moving each element exactly once to its final position in the reorganized list. The scheme is stochastically absorbing and it is shown to be optimal for a restricted family of distributions. Thus, for these distributions, the probability of the scheme converging to the optimal list order can be made as close to unity as desired.

Graph Types

Recursive data structures are abstractions of simple records and pointers. They impose a <em> shape</em> invariant, which is verified at compile-time and exploited to automatically generate code for building, copying, comparing, and traversing values without loss of efficiency. However, such values are always tree shaped, which is a major obstacle to practical use. We propose a notion of <em> graph types </em>, which allow common shapes, such as doubly-linked lists or threaded trees, to be expressed concisely and efficiently. We define regular languages of <em> routing expressions</em> to specify relative addresses of extra pointers in a canonical spanning tree. An efficient algorithm for computing such addresses is developed. We employ a second-order monadic logic to decide well-formedness of graph type specifications. This logic can also be used for automated reasoning about pointer structures.