Compositional Reasoning Research Articles

Unifying Static and Dynamic Intermediate Languages for Accelerator Generators

Compilers for accelerator design languages (ADLs) translate high-level languages into application-specific hardware. ADL compilers rely on a hardware control interface to compose hardware units. There are two choices: static control, which relies on cycle-level timing; or dynamic control, which uses explicit signalling to avoid depending on timing details. Static control is efficient but brittle; dynamic control incurs hardware costs to support compositional reasoning. Piezo is an ADL compiler that unifies static and dynamic control in a single intermediate language (IL). Its key insight is that the IL’s static fragment is a refinement of its dynamic fragment: static code admits a subset of the run-time behaviors of the dynamic equivalent. Piezo can optimize code by combining facts from static and dynamic submodules, and it opportunistically converts code from dynamic to static control styles. We implement Piezo as an extension to an existing dynamic ADL compiler, Calyx. We use Piezo to implement a frontend for an existing ADL, a systolic array generator, and a packet-scheduling hardware generator to demonstrate its optimizations and the static–dynamic interactions it enables.

Story of Your Lazy Function’s Life: A Bidirectional Demand Semantics for Mechanized Cost Analysis of Lazy Programs

Lazy evaluation is a powerful tool that enables better compositionality and potentially better performance in functional programming, but it is challenging to analyze its computation cost. Existing works either require manually annotating sharing, or rely on separation logic to reason about heaps of mutable cells. In this paper, we propose a bidirectional demand semantics that allows for extrinsic reasoning about the computation cost of lazy programs without relying on special program logics. To show the effectiveness of our approach, we apply the demand semantics to a variety of case studies including insertion sort, selection sort, Okasaki's banker's queue, and the implicit queue. We formally prove that the banker's queue and the implicit queue are both amortized and persistent using the Rocq Prover (formerly known as Coq). We also propose the reverse physicist's method, a novel variant of the classical physicist's method, which enables mechanized, modular and compositional reasoning about amortization and persistence with the demand semantics.

An Axiomatic Basis for Computer Programming on the Relaxed Arm-A Architecture: The AxSL Logic

Very relaxed concurrency memory models, like those of the Arm-A, RISC-V, and IBM Power hardware architectures, underpin much of computing but break a fundamental intuition about programs, namely that syntactic program order and the reads-from relation always both induce order in the execution. Instead, out-of-order execution is allowed except where prevented by certain pairwise dependencies, barriers, or other synchronisation. This means that there is no notion of the 'current' state of the program, making it challenging to design (and prove sound) syntax-directed, modular reasoning methods like Hoare logics, as usable resources cannot implicitly flow from one program point to the next. We present AxSL, a separation logic for the relaxed memory model of Arm-A, that captures the fine-grained reasoning underpinning the low-overhead synchronisation mechanisms used by high-performance systems code. In particular, AxSL allows transferring arbitrary resources using relaxed reads and writes when they induce inter-thread ordering. We mechanise AxSL in the Iris separation logic framework, illustrate it on key examples, and prove it sound with respect to the axiomatic memory model of Arm-A. Our approach is largely generic in the axiomatic model and in the instruction-set semantics, offering a potential way forward for compositional reasoning for other similar models, and for the combination of production concurrency models and full-scale ISAs.

Event Graph Guided Compositional Spatial-Temporal Reasoning for Video Question Answering.

Video question answering (VideoQA) is challenging since it requires the model to extract and combine multi-level visual concepts from local objects to global actions from complex events for compositional reasoning. Existing works represent the video with fixed-duration clip features that make the model struggle in capturing the crucial concepts in multiple granularities. To overcome this shortcoming, we propose to represent the video with an Event Graph in a hierarchical structure whose nodes correspond to visual concepts of different levels (object, relation, scene and action) and edges indicate their spatial-temporal relationships. We further propose a H ierarchical S patial- T emporal T ransformer (HSTT) which takes nodes from the graph as visual input to realize compositional reasoning guided by the event graph. To fully exploit the spatial-temporal context delivered from the graph structure, on the one hand, we encode the nodes in the order of their semantic hierarchy (depth) and occurrence time (breadth) with our improved graph search algorithm; On the other hand, we introduce edge-guided attention to combine the spatial-temporal context among nodes according to their edge connections. HSTT then performs QA by cross-modal interactions guaranteed by the hierarchical correspondence between the multi-level event graph and the cross-level question. Experiments on the recent challenging AGQA and STAR datasets show that the proposed method clearly outperforms the existing VideoQA models by a large margin, including those pre-trained with large-scale external data. Our code is available at https://github.com/ByZ0e/HSTT.

Contract-Based Design for Hybrid Dynamical Systems and Invariance Properties

This work establishes fundamental principles for verifying contract for interconnected hybrid systems. When system’s hybrid arcs conform to the contract for a certain duration but subsequently violate it, the composition of hybrid dynamical systems becomes challenging. The objective of this work is to analyze the temporal satisfaction of the contract, allowing us to reason about the compositions that do not violate the contract up to a certain point in a hybrid time. Notions of weak and strong satisfaction of an assume-guarantee contract are introduced. These semantics permits the compositional reasoning on hybrid systems of varying complexity depending on the interconnection’s type, feedback or cascade. The results show that both semantics are compatible with cascade composition, while strong semantic is required for feedback composition. Moreover, we have shown how one can go from weak to strong contract satisfaction. Finally, we have studied a particular class of hybrid systems and we have shown that the concept of forward (pre-)invariant relative to a contract makes it possible to deal with feedback compositions. These results are demonstrated throughout the paper with simple numerical examples.

A Closer Look at Video Sampling for Sequential Action Recognition

In recent years, sequential action recognition has attracted increasingly attention as it requires long-term sequential and compositional reasoning of human actions and object interactions. Existing methods perform reasoning either by using snippets that cover very short consecutive frames or key frames sampled from segments, which take a bias process of local and global temporal information. We also find ad-hoc training and ensembling of two separate networks using existing sampling strategies can easily outperform complex state-of-the-art methods, which reveals the complementary nature of current sampling strategies. Motivated by this observation, we propose a simple yet efficient strategy named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Dense Segmental Sampling (DSS)</i> and a novel network architecture named <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Temporal Dense Segment Network (TDSN)</i> to capture the complementary information from DSS. Our TDSN achieves excellent results on benchmark action recognition datasets, which not only validate the proposed strategy but also help highlight the importance along this direction for sequential video reasoning.

Back to the format: A survey on SOS for probabilistic processes

In probabilistic process algebras the classic qualitative description of process behaviour is enriched with quantitative information on it, usually modelled in terms of probabilistic weights and/or distributions over the qualitative behaviour. In this setting, we use behavioural equivalences to check whether two processes show exactly the same behaviour, and, if this is not the case, we can use behavioural metrics to measure the distance between them. Compositional reasoning requires that equivalence, or closeness, of behaviour of two processes are not destroyed when language operators are applied on top of them in order to build larger processes. Formally, the equivalence must be a congruence, and the metric must be uniformly continuous, with respect to language operators. Instead of verifying these compositional properties by hand, operator-by-operator, it is much more convenient to prove them for a class of operators once for all, and to check that the operators one is dealing with are in that class. This is achieved by means of SOS specification formats: they consist in a set of syntactical constraints characterising a class of operators on the patterns of SOS rules, that define the operational semantics of languages. With this survey, we aim to collect and describe the specification formats that have been proposed in the literature to guarantee the compositional properties of (variants of) bisimulation equivalences and bisimulation metrics in the probabilistic setting.

Accounting for Transfer of Learning Using Human Behavior Models

An important characteristic of human learning and decision-making is the flexibility with which we rapidly adapt to novel tasks. To this day, models of human behavior have been unable to emulate the ease and success with which humans transfer knowledge in one context to another. Humans rely on a lifetime of experience and a variety of cognitive mechanisms that are difficult to represent computationally. To address this problem, we propose a novel human behavior model that accounts for human transfer of learning using three mechanisms: compositional reasoning, causal inference, and optimal forgetting. To evaluate this proposed model, we introduce an experiment task designed to elicit human transfer of learning under different conditions. Our proposed model demonstrates a more human-like transfer of learning compared to models that optimize transfer or human behavior models that do not directly account for transfer of learning. The results of the ablation testing of the proposed model and a systematic comparison to human data demonstrate the importance of each component of the cognitive model underlying the transfer of learning.

A contract-based semantics and refinement for hybrid Simulink block diagrams

Simulink is widely used for modelling, simulating, and analysing cyber–physical systems (CPS). However, for safety-critical CPS, Simulink is insufficient to ensure safety since it lacks an official formal semantics to support compositional reasoning and verification. In this paper, we present a contract-based semantic model for hybrid Simulink block diagrams, including both discrete-time and continuous-time blocks. In our semantic formalisation, the semantics of a block is defined as a contract, and we define five operations on contracts, which are sequential composition, parallel composition, feedback composition, variable renaming, and variable hiding. We then develop a refinement calculus for hybrid Simulink block diagrams. Finally, we use a water tank system and a vehicle speed control system as case studies to demonstrate our approach.

Multi-Level Compositional Reasoning for Interactive Instruction Following

Robotic agents performing domestic chores by natural language directives are required to master the complex job of navigating environment and interacting with objects in the environments. The tasks given to the agents are often composite thus are challenging as completing them require to reason about multiple subtasks, e.g., bring a cup of coffee. To address the challenge, we propose to divide and conquer it by breaking the task into multiple subgoals and attend to them individually for better navigation and interaction. We call it Multi-level Compositional Reasoning Agent (MCR-Agent). Specifically, we learn a three-level action policy. At the highest level, we infer a sequence of human-interpretable subgoals to be executed based on language instructions by a high-level policy composition controller. At the middle level, we discriminatively control the agent’s navigation by a master policy by alternating between a navigation policy and various independent interaction policies. Finally, at the lowest level, we infer manipulation actions with the corresponding object masks using the appropriate interaction policy. Our approach not only generates human interpretable subgoals but also achieves 2.03% absolute gain to comparable state of the arts in the efficiency metric (PLWSR in unseen set) without using rule-based planning or a semantic spatial memory. The code is available at https://github.com/yonseivnl/mcr-agent.

Compositional Reasoning for Non-multicopy Atomic Architectures

Rely/guarantee reasoning provides a compositional approach to reasoning about concurrent programs. However, such reasoning traditionally assumes a sequentially consistent memory model and hence is unsound on modern hardware in the presence of data races. In this article, we present a rely/guarantee-based approach for non-multicopy atomic weak memory models, i.e., where a thread’s stores are not simultaneously propagated to all other threads and hence are not observable by other threads at the same time. Such memory models include those of the earlier versions of the ARM processor as well as the POWER processor. This article builds on our approach to compositional reasoning for multicopy atomic architectures, i.e., where a thread’s stores are simultaneously propagated to all other threads. In that context, an operational semantics can be based on thread-local instruction reordering. We exploit this to provide an efficient compositional proof technique in which weak memory behaviour can be shown to preserve rely/guarantee reasoning on a sequentially consistent memory model. To achieve this, we introduce a side-condition, reordering interference freedom on each thread, reducing the complexity of weak memory to checks over pairs of reorderable instructions. In this article, we extend our approach to non-multicopy atomic weak memory models. We utilise the idea of reordering interference freedom between parallel components. This by itself would break compositionality but serves as a vehicle to derive a refined compatibility check between rely and guarantee conditions, which takes into account the effects of propagations of stores that are only partial, i.e., not covering all threads. All aspects of our approach have been encoded and proved sound in Isabelle/HOL.

Towards correctness proof for hybrid Simulink block diagrams

Cyber–physical systems (CPS) are often modelled using Simulink to simulate plant and controller behaviour by block diagrams. However, since the incomplete coverage of simulation, design using Simulink alone is insufficient for ensuring the correctness of the modelled system, especially in safety-critical CPS. The correctness is that the Simulink block diagrams satisfy the formal specification of the system being modelled. In this work, we present a contract-based method, which supports compositional reasoning and refinement, for proving the correctness of hybrid Simulink block diagrams (i.e., the block diagrams mixed continuous–discrete blocks and multi-rate discrete-time block diagrams). To this end, we first define a formal semantics for hybrid Simulink block diagrams. We then use the contract as a specification language. Moreover, we present a satisfaction relation between the block diagram and the contract, and a refinement relation between the contracts. We prove that the refinement relation preserves the correctness: if the hybrid Simulink block diagram satisfies the composition contract and the composition contract refines the system specification, the hybrid block diagram is correct relative to the system specification. Furthermore, the effectiveness of our method is illustrated by a vehicle driving control system.

Merging Inductive Relations

Inductive relations offer a powerful and expressive way of writing program specifications while facilitating compositional reasoning. Their widespread use by proof assistant users has made them a particularly attractive target for proof engineering tools such as QuickChick, a property-based testing tool for Coq which can automatically derive generators for values satisfying an inductive relation. However, while such generators are generally efficient, there is an infrequent yet seemingly inevitable situation where their performance greatly degrades: when multiple inductive relations constrain the same piece of data. In this paper, we introduce an algorithm for merging two such inductively defined properties that share an index. The algorithm finds shared structure between the two relations, and creates a single merged relation that is provably equivalent to the conjunction of the two. We demonstrate, through a series of case studies, that the merged relations can improve the performance of automatic generation by orders of magnitude, as well as simplify mechanized proofs by getting rid of the need for nested induction and tedious low-level book-keeping.

CRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense.

Alternatively inferring on the visual facts and commonsense is fundamental for an advanced visual question answering (VQA) system. This ability requires models to go beyond the literal understanding of commonsense. The system should not just treat objects as the entrance to query background knowledge, but fully ground commonsense to the visual world and imagine the possible relationships between objects, e.g., "fork, can lift, food". To comprehensively evaluate such abilities, we propose a VQA benchmark, Compositional Reasoning on vIsion and Commonsense(CRIC), which introduces new types of questions about CRIC, and an evaluation metric integrating the correctness of answering and commonsense grounding. To collect such questions and rich additional annotations to support the metric, we also propose an automatic algorithm to generate question samples from the scene graph associated with the images and the relevant knowledge graph. We further analyze several representative types of VQA models on the CRIC dataset. Experimental results show that grounding the commonsense to the image region and joint reasoning on vision and commonsense are still challenging for current approaches. The dataset is available at https://cricvqa.github.io.

Goal-Aware RSS for Complex Scenarios via Program Logic

We introduce a <i>goal-aware</i> extension of responsibility-sensitive safety (RSS), a recent methodology for rule-based safety guarantee for automated driving systems (ADS). Making RSS rules guarantee goal achievement&#x2014;in addition to collision avoidance as in the original RSS&#x2014;requires complex planning over long sequences of manoeuvres.To deal with the complexity, we introduce a compositional reasoning framework based on program logic, in which one can systematically develop RSS rules for smaller subscenarios and combine them to obtain RSS rules for bigger scenarios. As the basis of the framework, we introduce a program logic <inline-formula><tex-math notation="LaTeX">$\text{dFHL}$</tex-math></inline-formula> that accommodates continuous dynamics and safety conditions. Our framework presents a <inline-formula><tex-math notation="LaTeX">$\text{dFHL}$</tex-math></inline-formula>-based workflow for deriving goal-aware RSS rules; we discuss its software support, too. We conducted experimental evaluation using RSS rules in a safety architecture. Its results show that goal-aware RSS is indeed effective in realising both collision avoidance and goal achievement.

When privacy fails, a formula describes an attack: A complete and compositional verification method for the applied π-calculus

We prove that three semantics for the applied π-calculus coincide – a testing semantics, a labelled equivalence, and a modal logic – and explain how design decisions in each of these semantics are objectively supported by this convergence. Each of these semantics has a distinct role when verifying security protocols. Testing semantics are suited to security and privacy problems expressed in terms of a process equivalence problem, where tests define the space of attack strategies. A labelled equivalence is suited to proving the given equivalence problem when such security and privacy properties hold; while, dually, the modal logic describes attacks when such security or privacy properties are violated. For this particular investigation into this good-practice design pattern, we select what is perhaps the most powerful (interleaving) testing semantics: open barbed bisimilarity. Selecting open barbed bisimilarity comes with the bonus that it is a congruence hence is suited to compositional reasoning. This reference testing semantics leads to a notion of quasi-open bisimilarity and an intuitionistic modal logic for the applied π-calculus. We argue that, since we characterise the finest (interleaving) testing semantics and tests are attack strategies, our intuitionistic modal logic can describe all attacks whenever a privacy property expressed as an equivalence problem fails.

A concurrent program logic with a future and history

Verifying fine-grained optimistic concurrent programs remains an open problem. Modern program logics provide abstraction mechanisms and compositional reasoning principles to deal with the inherent complexity. However, their use is mostly confined to pencil-and-paper or mechanized proofs. We devise a new separation logic geared towards the lacking automation. While local reasoning is known to be crucial for automation, we are the first to show how to retain this locality for (i) reasoning about inductive properties without the need for ghost code, and (ii) reasoning about computation histories in hindsight. We implemented our new logic in a tool and used it to automatically verify challenging concurrent search structures that require inductive properties and hindsight reasoning, such as the Harris set.

Proving Simulink Block Diagrams Correct via Refinement

Simulink is a well-known block diagram-based tool for modular design and multidomain simulation of Cyber-Physical Systems (CPS). However, the simulation by Simulink cannot completely cover the state space or behavior of a target system, which would not ensure the correctness of the developed block diagrams in Simulink. In this work, we present a contract-based method, which supports compositional reasoning and refinement, for proving the correctness of Simulink block diagrams with discrete-time and continuous-time dynamic behavior. We use the assume-guarantee contract as a specification language. The Simulink block diagrams are correct in the sense that if the block diagrams satisfy the formal specifications of the system being modeled. To prove the correctness of a block diagram, we first define semantics for Simulink block diagrams. We study three composition operators, i.e., serial, parallel, and algebraic loop-free feedback with multistep delays. We present a satisfaction relation between the block diagram and contract and present a refinement relation between the contracts. We prove that if the Simulink block diagram satisfies the composition contract and the composition contract refines the system specifications, the block diagram is correct relative to the system specifications. Furthermore, we demonstrate the effectiveness of our method via a real-world case study originating from the control system of a reservoir. Our method can also provide an idea to verify whether the designed CPS is planted with a logic bomb by attackers.

Feature Interactions on Steroids: On the Composition of ML Models

One of the key differences between traditional software engineering and machine learning (ML) is the lack of specifications for ML models. Traditionally, specifications provide a cornerstone for compositional reasoning and for the divide-and-conquer strategy of how we build large and complex systems from components, but these are hard to come by for machine learned components. While the lack of specification seems like a fundamental new problem at first sight, in fact, software engineers routinely deal with iffy specifications in practice. We face weak specifications, wrong specifications, and unanticipated interactions among specifications. ML may push us further, but the problems are not fundamentally new. Rethinking ML model composition from the perspective of the feature-interaction problem highlights the importance of software design.

A dynamic logic for verification of synchronous models based on theorem proving

Synchronous model is a type of formal models for modelling and specifying reactive systems. It has a great advantage over other real-time models that its modelling paradigm supports a deterministic concurrent behaviour of systems. Various approaches have been utilized for verification of synchronous models based on different techniques, such as model checking, SAT/SMT sovling, term rewriting, type inference and so on. In this paper, we propose a verification approach for synchronous models based on compositional reasoning and term rewriting. Specifically, we initially propose a variation of dynamic logic, called synchronous dynamic logic (SDL). SDL extends the regular program model of first-order dynamic logic (FODL) with necessary primitives to capture the notion of synchrony and synchronous communication between parallel programs, and enriches FODL formulas with temporal dynamic logical formulas to specify safety properties -- a type of properties mainly concerned in reactive systems. To rightly capture the synchronous communications, we define a constructive semantics for the program model of SDL. We build a sound and relatively complete proof system for SDL. Compared to previous verification approaches, SDL provides a divide and conquer way to analyze and verify synchronous models based on compositional reasoning of the syntactic structure of the programs of SDL. To illustrate the usefulness of SDL, we apply SDL to specify and verify a small example in the synchronous model SyncChart, which shows the potential of SDL to be used in practice.