Rust Programming Language Research Articles

Rustlantis: Randomized Differential Testing of the Rust Compiler

Compilers are at the core of all computer architecture. Their middle-end and back-end are full of subtle code that is easy to get wrong. At the same time, the consequences of compiler bugs can be severe. Therefore, it is important that we develop techniques to increase our confidence in compiler correctness, and to help find the bugs that inevitably happen. One promising such technique that has successfully found many compiler bugs in the past is randomized differential testing , a fuzzing approach whereby the same program is executed with different compilers or different compiler settings to detect any unexpected differences in behavior. We present Rustlantis , the first fuzzer for the Rust programming language that is able to find new correctness bugs in the official Rust compiler. To avoid having to deal with Rust’s strict type and borrow checker, Rustlantis directly generates MIR, the central IR of the Rust compiler for optimizations. The program generation strategy of Rustlantis is a combination of statically tracking the state of the program, obscuring the program state for the compiler, and decoy blocks to lead optimizations astray. This has allowed us to identify 22 previously unknown bugs in the Rust compiler, most of which have been fixed.

Sound Borrow-Checking for Rust via Symbolic Semantics

The Rust programming language continues to rise in popularity, and as such, warrants the close attention of the programming languages community. In this work, we present a new foundational contribution towards the theoretical understanding of Rust’s semantics. We prove that LLBC, a high-level, borrow-centric model previously proposed for Rust’s semantics and execution, is sound with regards to a low-level pointer-based language à la CompCert. Specifically, we prove the following: that LLBC is a correct view over a traditional model of execution; that LLBC’s symbolic semantics are a correct abstraction of LLBC programs; and that LLBC’s symbolic semantics act as a borrow-checker for LLBC, i.e. that symbolically-checked LLBC programs do not get stuck when executed on a heap-and-addresses model of execution. To prove these results, we introduce a new proof style that considerably simplifies our proofs of simulation, which relies on a notion of hybrid states. Equipped with this reasoning framework, we show that a new addition to LLBC’s symbolic semantics, namely a join operation, preserves the abstraction and borrow-checking properties. This in turn allows us to add support for loops to the Aeneas framework; we show, using a series of examples and case studies, that this unlocks new expressive power for Aeneas.

A Contextual Model for Visual Information Processing

Despite significant achievements in the artificial narrow intelligence sphere, the mechanisms of human-like (general) intelligence are still undeveloped. There is a theory stating that the human brain extracts the meaning of information rather than recognizes the features of a phenomenon. Extracting the meaning is finding a set of transformation rules (context) and applying them to the incoming information, producing an interpretation. Then, the interpretation is compared to something already seen and is stored in memory. Information can have different meanings in different contexts. A mathematical model of a context processor and a differential contextual space which can perform the interpretation is discussed and developed in this paper. This study examines whether the basic principles of differential contextual spaces work in practice. The model is developed with Rust programming language and trained on black and white images which are rotated and shifted both horizontally and vertically according to the saccades and torsion movements of a human eye. Then, a picture that has never been seen in the particular transformation, but has been seen in another one, is exposed to the model. The model considers the image in all known contexts and extracts the meaning. The results show that the program can successfully process black and white images which are transformed by shifts and rotations. This research prepares the grounding for further investigations of the contextual model principles with which general intelligence might operate.

Bigtools: a high-performance BigWig and BigBed library in Rust.

The BigWig and BigBed file formats were originally designed for the visualization of next-generation sequencing data through a genome browser. Due to their versatility, these formats have long since become ubiquitous for the storage of processed sequencing data and regularly serve as the basis for downstream data analysis. As the number and size of sequencing experiments continues to accelerate, there is an increasing demand to efficiently generate and query BigWig and BigBed files in a scalable and robust manner, and to efficiently integrate these functionalities into data analysis environments and third-party applications. Here, we present Bigtools, a feature-complete, high-performance, and integrable software library for generating and querying both BigWig and BigBed files. Bigtools is written in the Rust programming language and includes a flexible suite of command line tools as well as bindings to Python. Bigtools is cross-platform and released under the MIT license. It is distributed on Crates.io, Bioconda, and the Python Package Index, and the source code is available at https://github.com/jackh726/bigtools.

Functional Ownership through Fractional Uniqueness

Ownership and borrowing systems, designed to enforce safe memory management without the need for garbage collection, have been brought to the fore by the Rust programming language. Rust also aims to bring some guarantees offered by functional programming into the realm of performant systems code, but the type system is largely separate from the ownership model, with type and borrow checking happening in separate compilation phases. Recent models such as RustBelt and Oxide aim to formalise Rust in depth, but there is less focus on integrating the basic ideas into more traditional type systems. An approach designed to expose an essential core for ownership and borrowing would open the door for functional languages to borrow concepts found in Rust and other ownership frameworks, so that more programmers can enjoy their benefits. One strategy for managing memory in a functional setting is through uniqueness types, but these offer a coarse-grained view: either a value has exactly one reference, and can be mutated safely, or it cannot, since other references may exist. Recent work demonstrates that linear and uniqueness types can be combined in a single system to offer restrictions on program behaviour and guarantees about memory usage. We develop this connection further, showing that just as graded type systems like those of Granule and Idris generalise linearity, a Rust-like ownership model arises as a graded generalisation of uniqueness. We combine fractional permissions with grading to give the first account of ownership and borrowing that smoothly integrates into a standard type system alongside linearity and graded types, and extend Granule accordingly with these ideas.

Profiling Programming Language Learning

This paper documents a year-long experiment to “profile” the process of learning a programming language: gathering data to understand what makes a language hard to learn, and using that data to improve the learning process. We added interactive quizzes to The Rust Programming Language, the official textbook for learning Rust. Over 13 months, 62,526 readers answered questions 1,140,202 times. First, we analyze the trajectories of readers. We find that many readers drop-out of the book early when faced with difficult language concepts like Rust’s ownership types. Second, we use classical test theory and item response theory to analyze the characteristics of quiz questions. We find that better questions are more conceptual in nature, such as asking why a program does not compile vs. whether a program compiles. Third, we performed 12 interventions into the book to help readers with difficult questions. We find that on average, interventions improved quiz scores on the targeted questions by +20

SEGUL: Ultrafast, memory-efficient and mobile-friendly software for manipulating and summarizing phylogenomic datasets.

Phylogenetic studies now routinely require manipulating and summarizing thousands of data files. For most of these tasks, currently available software requires considerable computing resources and substantial knowledge of command-line applications. We develop an ultrafast and memory-efficient software, SEGUL, that performs common phylogenomic dataset manipulations and calculates statistics summarizing essential data features. Our software is available as standalone command-line interface (CLI) and graphical user interface (GUI) applications, and as a library for Rust, R and Python, with possible support of other languages. The CLI and library versions run native on Windows, Linux and macOS, including Apple ARM Macs. The GUI version extends support to include mobile iOS, iPadOS and Android operating systems. SEGUL leverages the high performance of the Rust programming language to offer fast execution times and low memory footprints regardless of dataset size and platform choice. The inclusion of a GUI minimizes bioinformatics barriers to phylogenomics while SEGUL's efficiency reduces economic barriers by allowing analysis on inexpensive hardware. Our support for mobile operating systems further enables teaching phylogenomics where access to computing power is limited.

Profiling with trust: system monitoring from trusted execution environments

AbstractLarge-scale attacks on IoT and edge computing devices pose a significant threat. As a prominent example, Mirai is an IoT botnet with 600,000 infected devices around the globe, capable of conducting effective and targeted DDoS attacks on (critical) infrastructure. Driven by the substantial impacts of attacks, manufacturers and system integrators propose Trusted Execution Environments (TEEs) that have gained significant importance recently. TEEs offer an execution environment to run small portions of code isolated from the rest of the system, even if the operating system is compromised. In this publication, we examine TEEs in the context of system monitoring and introduce the Trusted Monitor (TM), a novel anomaly detection system that runs within a TEE. The TM continuously profiles the system using hardware performance counters and utilizes an application-specific machine-learning model for anomaly detection. In our evaluation, we demonstrate that the TM accurately classifies 86% of 183 tested workloads, with an overhead of less than 2%. Notably, we show that a real-world kernel-level rootkit has observable effects on performance counters, allowing the TM to detect it. Major parts of the TM are implemented in the Rust programming language, eliminating common security-critical programming errors.

Blockchain-Powered Education: A Sustainable Approach for Secured and Connected University Systems

The collection and examination of student data, encompassing academic achievements, awards, and certifications, assume an essential function within the field of education as a means of showing students’ capabilities. Nevertheless, it is crucial to note that regular paper-based records are vulnerable to both physical destruction and the act of fabrication, while standard databases can have security holes. Moreover, the process of manually gathering physical papers from centralized organizations is both laborious and complicated. To address the concerns above and foster sustainability in the field of education, this study first suggests using Scorechain. This innovative solution integrates blockchain technology into a comprehensive data-management system for managing all student-related data. Secondly, by utilizing the inherent security features of blockchain technology, Scorechain develops a stable multi-role hierarchy, increasing the integrity and reliability of data. This also facilitates the efficient transfer of information among various stakeholders, including parents, recruiters, and educational institutions, thus fostering transparency and accountability. Lastly, the Scorechain system facilitates collaboration and data exchange among universities inside a shared network. Scorechain was constructed using the Rust programming language and is based on the Substrate blockchain architecture. It underwent careful development, testing, and analysis to ensure operational efficiency. The feasibility and long-term viability of Scorechain in genuine educational contexts are highlighted as blockchain technology facilitates seamless integration into the education sector.

Verus: Verifying Rust Programs using Linear Ghost Types

The Rust programming language provides a powerful type system that checks linearity and borrowing, allowing code to safely manipulate memory without garbage collection and making Rust ideal for developing low-level, high-assurance systems. For such systems, formal verification can be useful to prove functional correctness properties beyond type safety. This paper presents Verus, an SMT-based tool for formally verifying Rust programs. With Verus, programmers express proofs and specifications using the Rust language, allowing proofs to take advantage of Rust's linear types and borrow checking. We show how this allows proofs to manipulate linearly typed permissions that let Rust code safely manipulate memory, pointers, and concurrent resources. Verus organizes proofs and specifications using a novel mode system that distinguishes specifications, which are not checked for linearity and borrowing, from executable code and proofs, which are checked for linearity and borrowing. We formalize Verus' linearity, borrowing, and modes in a small lambda calculus, for which we prove type safety and termination of specifications and proofs. We demonstrate Verus on a series of examples, including pointer-manipulating code (an xor-based doubly linked list), code with interior mutability, and concurrent code.

Hardware/Software Co-Assurance for the Rust Programming Language Applied to Zero Trust Architecture Development

Zero Trust Architecture requirements are of increasing importance in critical systems development. Zero trust tenets hold that no implicit trust be granted to assets based on their physical or network location. Zero Trust development focuses on authentication, authorization, and shrinking implicit trust zones to the most granular level possible, while maintaining availability and minimizing authentication latency. Performant, high-assurance cryptographic primitives are thus central to successfully realizing a Zero Trust Architecture. The Rust programming language has garnered significant interest and use as a modern, type-safe, memory-safe, and potentially formally analyzable programming language. Our interest in Rust particularly stems from its potential as a hardware/software co-assurance language for developing Zero Trust Architectures. We describe a novel environment enabling Rust to be used as a High-Level Synthesis (HLS) language, suitable for secure and performant Zero Trust application development. Many incumbent HLS languages are a subset of C, and inherit many of the well-known security shortcomings of that language. A Rust-based HLS brings a single modern, type-safe, memory-safe, high-assurance development language for both hardware and software. To study the benefits of this approach, we crafted a Rust HLS subset, and developed a frontend to the hardware/software co-assurance toolchain due to Russinoff and colleagues at Arm, used primarily for floating-point hardware formal verification. This allows us to leverage a number of existing hardware/software co-assurance tools with a minimum investment of time and effort. In this paper, we describe our Rust subset, detail our prototype toolchain, and describe the implementation, performance analysis, formal verification and validation of representative Zero Trust algorithms and data structures written in Rust, emphasizing cryptographic primitives and common data structures.

FeOs: An Open-Source Framework for Equations of State and Classical Density Functional Theory

In this work, we present an open-source software package, referred to as FeOs─Framework for Equations of State and Classical Density Functional Theory. FeOs is a collection of interfaces and data types that can be used (1) to implement thermodynamic equations of state and Helmholtz energy functionals for classical density functional theory, and (2) to compute thermodynamic properties of pure substances and mixtures, phase equilibria, and interfacial properties such as surface tensions and adsorption isotherms. The framework is written in the Rust programming language with a complete Python interface and is designed with a focus on usability and extensibility. It is openly available on GitHub (https://github.com/feos-org/feos). Equations of state can be implemented in Rust, yielding performant code, or as a Python class, which is useful for prototyping and with less emphasis on execution speed. In both cases, the user has to implement a single function: the Helmholtz energy. FeOs then uses generalized (hyper-) dual numbers to evaluate the Helmholtz energy as well as the required exact partial (higher-order) derivatives. Using this type of automatic differentiation delivers performance without the need for implementing any analytical derivatives. The performance is further enhanced by a caching mechanism that avoids duplicate model evaluations. Together with the core interfaces and functionalities for equations of state and classical density functional theory, we provide implementations for multiple models such as the PC-SAFT equation of state (with homo- and heterosegmented group contribution methods) and Helmholtz energy functionals (including segment-based functionals). To showcase a selection of FeOs’ features, an example study of the adsorption of biogas in porous media using the PC-SAFT functional is provided.

Higher-Order Leak and Deadlock Free Locks

Reasoning about concurrent programs is challenging, especially if data is shared among threads. Program correctness can be violated by the presence of data races—whose prevention has been a topic of concern both in research and in practice. The Rust programming language is a prime example, putting the slogan fearless concurrency in practice by not only employing an ownership-based type system for memory management, but also using its type system to enforce mutual exclusion on shared data. Locking, unfortunately, not only comes at the price of deadlocks but shared access to data may also cause memory leaks . This paper develops a theory of deadlock and leak freedom for higher-order locks in a shared memory concurrent setting. Higher-order locks allow sharing not only of basic values but also of other locks and channels, and are themselves first-class citizens. The theory is based on the notion of a sharing topology , administrating who is permitted to access shared data at what point in the program. The paper first develops higher-order locks for acyclic sharing topologies, instantiated in a λ-calculus with higher-order locks and message-passing concurrency. The paper then extends the calculus to support circular dependencies with dynamic lock orders, which we illustrate with a dynamic version of Dijkstra’s dining philosophers problem. Well-typed programs in the resulting calculi are shown to be free of deadlocks and memory leaks, with proofs mechanized in the Coq proof assistant.

Hardware/Software Co-Assurance using the Rust Programming Language and ACL2

The Rust programming language has garnered significant interest and use as a modern, type-safe, memory-safe, and potentially formally analyzable programming language. Our interest in Rust stems from its potential as a hardware/software co-assurance language, with application to critical systems such as autonomous vehicles. We report on the first known use of Rust as a High-Level Synthesis (HLS) language. Most incumbent HLS languages are a subset of C. A Rust-based HLS brings a single modern, type-safe, and memory-safe expression language for both hardware and software realizations with high assurance. As a a study of the suitability of Rust as an HLS, we have crafted a Rust subset, inspired by Russinoff's Restricted Algorithmic C (RAC), which we have imaginatively named Restricted Algorithmic Rust, or RAR. In our first implementation of a RAR toolchain, we simply transpile the RAR source into RAC. By so doing, we leverage a number of existing hardware/software co-assurance tools with a minimum investment of time and effort. In this paper, we describe the RAR Rust subset, detail our prototype RAR toolchain, and describe the implementation and verification of several representative algorithms and data structures written in RAR, with proofs of correctness conducted using the ACL2 theorem prover.

An alignment-free heuristic for fast sequence comparisons with applications to phylogeny reconstruction

BackgroundAlignment-free methods for sequence comparisons have become popular in many bioinformatics applications, specifically in the estimation of sequence similarity measures to construct phylogenetic trees. Recently, the average common substring measure, ACS, and its k-mismatch counterpart, ACSk, have been shown to produce results as effective as multiple-sequence alignment based methods for reconstruction of phylogeny trees. Since computing ACSk takes O(n logkn) time and hence impractical for large datasets, multiple heuristics that can approximate ACSk have been introduced.ResultsIn this paper, we present a novel linear-time heuristic to approximate ACSk, which is faster than computing the exact ACSk while being closer to the exact ACSk values compared to previously published linear-time greedy heuristics. Using four real datasets, containing both DNA and protein sequences, we evaluate our algorithm in terms of accuracy, runtime and demonstrate its applicability for phylogeny reconstruction. Our algorithm provides better accuracy than previously published heuristic methods, while being comparable in its applications to phylogeny reconstruction.ConclusionsOur method produces a better approximation for ACSk and is applicable for the alignment-free comparison of biological sequences at highly competitive speed. The algorithm is implemented in Rust programming language and the source code is available at https://github.com/srirampc/adyar-rs.

RustBelt meets relaxed memory

The Rust programming language supports safe systems programming by means of a strong ownership-tracking type system. In their prior work on RustBelt, Jung et al. began the task of setting Rust’s safety claims on a more rigorous formal foundation. Specifically, they used Iris, a Coq-based separation logic framework, to build a machine-checked proof of semantic soundness for a λ-calculus model of Rust, as well as for a number of widely-used Rust libraries that internally employ unsafe language features. However, they also made the significant simplifying assumption that the language is sequentially consistent. In this paper, we adapt RustBelt to account for the relaxed-memory operations that concurrent Rust libraries actually use, in the process uncovering a data race in the Arc library. We focus on the most interesting technical problem: how to reason about resource reclamation under relaxed memory , using a logical construction we call synchronized ghost state .

АРХІТЕКТУРА СИСТЕМИ ДЕДУБЛІКАЦІЇ ТА РОЗПОДІЛУ ДАНИХ У ХМАРНИХ СХОВИЩАХ ПІД ЧАС РЕЗЕРВНОГО КОПІЮВАННЯ

The conceptual model of the system is developed and described in detail. An intelligent system of deduplication and distribution of data in the cloud storage is developed, the description of the software is described, the stages of the user's work are considered. Testing of the projected system was carried out. Several control samples are described and results are analyzed. The purpose of the system is to deduplicate and distribute data in cloud repositories in such a way that the end result of the backup is to eliminate duplicate pieces of data using distributed computing and cloud repositories. By picking the right approach to distribute tasks and data during deduplication, you can harness the full potential of cloud-based distributed systems to increase backup speed and bandwidth. Analyzes (disadvantages and advantages of using different approaches) are analyzed and effective methods of solution are selected: hybrid block-level deduplication, splitting of data flow on the basis of Rabin's digital imprint, distribution of data based on hash values of blocks of deduplication and use of distributed index. Block-level deduplication involves two types of data flow splitting into blocks, a fixed-length, algorithm-based split. Fixed-length partitioning is rather trivial and fast with respect to the complexity of the algorithm, but the downside is that data is displaced at the beginning of the stream, since the blocks that will follow after the changes will be considered new. However, in the case of partitioning of blocks of variable length, the point of proper partitioning is determined by the algorithm. This algorithm should work with infinite data flows using the ring hash function. The algorithm absorbs each input byte of data from the stream, and as soon as the value of the annular hash function corresponds to the previously specified template, it also serves as a point of splitting the stream into blocks. Thus, if the data is changed or displaced by a couple of bytes, only the data block that covers the data will be considered new. However, in order to track changes and correctly set breakpoints, it is necessary to check the input data for a specific preset digital pattern - a hash value. It is a common practice to calculate a hash value every time an input byte is received in a data stream. The point of partition will be the moment when the resulting hash value matches the specified pattern. To do these calculations effectively, an algorithm has been devised for the ring hash. One of the most common ring hash algorithms is a digital Rabin imprint. During the analysis of the solutions, the Rust programming language for client-side writing, the Scala programming language for the server-side, the Akka distributed computing management tool, and Amazon S3 as the cloud repository were selected.

Secure top most significant genome variants search: iDASH 2017 competition

BackgroundOne of the 3 tracks of iDASH Privacy & Security Workshop 2017 competition was to execute a whole genome variants search on private genomic data. Particularly, the search application was to find the top most significant SNPs (Single-Nucleotide Polymorphisms) in a database of genome records labeled with control or case. In this paper we discuss the solution submitted by our team to this competition.MethodsPrivacy and confidentiality of genome data had to be ensured using Intel SGX enclaves. The typical use-case of this application is the multi-party computation (each party possessing one or several genome records) of the SNPs which statistically differentiate control and case genome datasets.ResultsOur solution consists of two applications: (i) compress and encrypt genome files and (ii) perform genome processing (top most important SNPs search). We have opted for a horizontal treatment of genome records and heavily used parallel processing. Rust programming language was employed to develop both applications.ConclusionsExecution performance of the processing applications scales well and very good performance metrics are obtained. Contest organizers selected it as the best submission amongst other received competition entries and our team was awarded the first prize on this track.

RustBelt: securing the foundations of the Rust programming language

Rust is a new systems programming language that promises to overcome the seemingly fundamental tradeoff between high-level safety guarantees and low-level control over resource management. Unfortunately, none of Rust's safety claims have been formally proven, and there is good reason to question whether they actually hold. Specifically, Rust employs a strong, ownership-based type system, but then extends the expressive power of this core type system through libraries that internally use unsafe features. In this paper, we give the first formal (and machine-checked) safety proof for a language representing a realistic subset of Rust. Our proof is extensible in the sense that, for each new Rust library that uses unsafe features, we can say what verification condition it must satisfy in order for it to be deemed a safe extension to the language. We have carried out this verification for some of the most important libraries that are used throughout the Rust ecosystem.

Rust-Bio: a fast and safe bioinformatics library.

We present Rust-Bio, the first general purpose bioinformatics library for the innovative Rust programming language. Rust-Bio leverages the unique combination of speed, memory safety and high-level syntax offered by Rust to provide a fast and safe set of bioinformatics algorithms and data structures with a focus on sequence analysis. Rust-Bio is available open source under the MIT license at https://rust-bio.github.io. koester@jimmy.harvard.edu Supplementary data are available at Bioinformatics online.