Runtime Verification Research Articles

Incrementally predictive runtime verification

Abstract Runtime verification is a lightweight formal verification technique used to verify the runtime behaviour of software (resp. hardware) systems. Given a formal property, one or more monitors are synthesized to verify the latter against a system execution. A monitor can only conclude the violation of a property when it observes such a violation. Unfortunately, in safety-critical scenarios, this might happen too late for the system to react properly. In such scenarios, it is advised to use predictive runtime verification, where monitors are capable of anticipating (by using a model of the system) future events before actually observing them. In this work, instead of assuming such a model is given, we describe a runtime verification workflow where the model is learnt and incrementally refined by using process mining techniques. We present the approach and the resulting prototype tool.

Special issue for the 36th Italian Conference on Computational Logic (CILC 2021)

This volume contains a selection of the papers presented at the |$36^{\textrm {th}}$|Italian Conference on Computational Logic (CILC 2021), held on September |$7^{\textrm {th}}$|–|$9^{\textrm {th}}$|⁠, |$2021$| in Parma, Italy. The Italian Conference on Computational Logic is the annual conference organized by the Group of researchers and Users of Logic Programming (GULP, www.programmazionelogica.it). Since the first event of the series, which took place in Genoa in |$1986$|⁠, the annual GULP conference represents a major opportunity for researchers and practitioners interested in computational logic to meet and exchange ideas. Furthermore, over the years, the conference broadened its horizons from logic programming to include related topics, such as declarative programming, formal verification of systems, knowledge representation, deductive databases and a wide range of applications in the broader area of Artificial Intelligence. The annual GULP conference is a traditional event of the Italian community of researchers and practitioners interested in logic programming, but its scope has become more and more international over the years. Despite the very peculiar situation caused by the COVID-19 pandemic, the community significantly responded to the call, thus allowing the technical program of CILC |$2021$| to feature |$23$| presentations of high-quality papers, of which |$19$| were original papers. This volume of the Journal of Logic and Computation contains extended and revised versions of a selection of the papers presented at CILC |$2021$|⁠. As a result of an additional review process, during which each submission was reviewed by three referees, |$8$| papers were accepted to be included in this volume. Accepted contributions range from foundations and theoretical results to practical experiences, case studies and applications. They cover a wide range of relevant topics related to computational logic in the broad sense, including agents and multi-agent systems, logic programming, runtime verification and healthcare. In the following, the featured papers are briefly summarized.

RV4JaCa—Towards Runtime Verification of Multi-Agent Systems and Robotic Applications

This paper presents a Runtime Verification (RV) approach for Multi-Agent Systems (MAS) using the JaCaMo framework. Our objective is to bring a layer of security to the MAS. This is achieved keeping in mind possible safety-critical uses of the MAS, such as robotic applications. This layer is capable of controlling events during the execution of the system without needing a specific implementation in the behaviour of each agent to recognise the events. In this paper, we mainly focus on MAS when used in the context of hybrid intelligence. This use requires communication between software agents and human beings. In some cases, communication takes place via natural language dialogues. However, this kind of communication brings us to a concern related to controlling the flow of dialogue so that agents can prevent any change in the topic of discussion that could impair their reasoning. The latter may be a problem and undermine the development of the software agents. In this paper, we tackle this problem by proposing and demonstrating the implementation of a framework that aims to control the dialogue flow in a MAS; especially when the MAS communicates with the user through natural language to aid decision-making in a hospital bed allocation scenario.

On the road with RTLola

This paper is about shipping runtime verification to the masses. It presents the crucial technology enabling everyday car owners to monitor the behaviour of their cars in-the-wild. Concretely, we present an Android app that deploys rtlola runtime monitors for the purpose of diagnosing automotive exhaust emissions. For this, it harvests the availability of cheap Bluetooth adapters to the On-Board-Diagnostics (obd) ports, which are ubiquitous in cars nowadays. The app is a central piece in a set of tools and services we have developed for black-box analysis of automotive vehicles. We detail its use in the context of real driving emission (rde) tests and report on sample runs that helped identify violations of the regulatory framework currently valid in the European Union.

Runtime Verification for Anomaly Detection of Robotic Systems Security

Robotic systems are widely used in industry, agriculture, the inspection of infrastructure, and even in our daily lives. The safety and security of robotic systems have become a primary concern as their interaction with humans increases. In this context, attacks on robotic systems have increased for diversified field applications. It is necessary to accurately detect these abnormal events in these systems as soon as possible. However, these systems also need a runtime verification approach on whether they conform to the established specifications. In this study, runtime verification for anomaly detection methods is proposed for the security of the robot operating system (ROS). Firstly, an anomaly detection method is proposed to detect unexpected situations, such as the number of the received packages being decreased under DoS attacks. Then, a holistic runtime verification architecture is proposed for the anomaly detection method. This architecture consists of three major entities: a verification device, an attacker device, and a robotic platform without losing generality. In the verification device, ROSMonitoring and Oracle are used to implement runtime verification. The proposed architecture is verified through an experimental setup. It is shown that the architecture can be used for runtime verification of different anomaly detection algorithms. A discussion on the security of robotic systems is also presented.

Run-time failure detection via non-intrusive event analysis in a large-scale cloud computing platform

Cloud computing systems fail in complex and unforeseen ways due to unexpected combinations of events and interactions among hardware and software components. These failures are especially problematic when they are silent, i.e., not accompanied by any explicit failure notification, hindering the timely detection and recovery.In this work, we propose an approach to run-time failure detection tailored for monitoring multi-tenant and concurrent cloud computing systems. The approach uses a non-intrusive form of event tracing, without manual changes to the system’s internals to propagate session identifiers (IDs), and builds a set of lightweight monitoring rules from fault-free executions. We evaluated the effectiveness of the approach in detecting failures in the context of the OpenStack cloud computing platform, a complex and “off-the-shelf” distributed system, by executing a campaign of fault injection experiments in a multi-tenant scenario. Our experiments show that the approach detects the failure with an F1 score (0.85) and accuracy (0.77) higher than the ones provided by the OpenStack failure logging mechanisms (0.53 and 0.50) and two non-session-aware run-time verification approaches (both lower than 0.15). Moreover, the approach significantly decreases the average time to detect failures at run-time (∼114 seconds) compared to the OpenStack logging mechanisms.

A Distributed Network-Based Runtime Verification of Full Regular Temporal Properties

As a lightweight method, runtime verification aims to check whether one program execution satisfies a desired property. For online runtime verification, the approach efficiency and property expressiveness are two key points restricting its wide application. In this paper, we propose a distributed network-based parallel runtime verification approach to verifying full regular temporal properties for a suitable subset of C (named by Xd-C) programs in an online manner. With this approach, an Xd-C program is translated into an equivalent Modeling, Simulation and Verification Language (MSVL) program, and a desired property is specified as a Propositional Projection Temporal Logic (PPTL) formula; during the program execution, segments of the generated state sequence are verified in parallel by distributed multi-core machines. Experimental results show that, our approach has a speedup of 2.5X-5.0X over the state-of-art runtime verification approaches and supports full regular temporal properties, meaning that our approach can not only take full advantage of computing and storage resources in a distributed network, but also support more expressive properties.

Decentralized Stream Runtime Verification for Timed Asynchronous Networks

Decentralized Stream Runtime Verification for Timed Asynchronous Networks

Using capabilities for strict runtime invariant checking

In this paper we use pre-existing language support for both reference and object capabilities to enable sound runtime verification of representation invariants. Our invariant protocol is stricter than the other protocols, since it guarantees that invariants hold for all objects involved in execution. Any language already offering appropriate support for reference and object capabilities can support our invariant protocol with minimal added complexity. In our protocol, invariants are simply specified as methods whose execution is statically guaranteed to be deterministic and to not access any externally mutable state. We formalise our approach and prove that our protocol is sound, in the context of a language supporting mutation, dynamic dispatch, exceptions, and non-deterministic I/O. We present case studies showing that our system requires a lighter annotation burden compared to Spec#, and performs orders of magnitude less runtime invariant checks compared to the ‘visible state semantics’ protocols of D and Eiffel.

STAn: analysis of data traces using an event-driven interval temporal logic

The increasing integration of systems into people’s daily routines, especially smartphones, requires ensuring correctness of their functionality and even some performance requirements. Sometimes, we can only observe the interaction of the system (e.g. the smartphone) with its environment at certain time points; that is, we only have access to the data traces produced due to this interaction. This paper presents the tool STAn, which performs runtime verification on data traces that combine timestamped discrete events and sampled real-valued magnitudes. STAn uses the Spin model checker as the underlying execution engine, and analyzes traces against properties described in the so-called event-driven interval temporal logic (eLTL) by transforming each eLTL formula into a network of concurrent automata, written in Promela, that monitors the trace. We present two different transformations for online and offline monitoring, respectively. Then, Spin explores the state space of the automata network and the trace to return a verdict about the corresponding property. We use the proposal to analyze data traces obtained during mobile application testing in different network scenarios.

A formal toolchain for offline and run-time verification of robotic systems

Validation and Verification (V&V) of autonomous robotic system software is becoming a critical issue. Among the V&V techniques at our disposal, formal approaches are among the most rigorous and trustworthy ones. Yet, the level of skills and knowledge required to use and deploy formal methods is usually quite high and rare. In this paper, we describe an approach that starts from a regular, but rigorous, framework to specify and deploy robotic software components, which can also automatically synthesize a formal model of these components.We describe how we can execute the resulting formal model, in place of a traditional implementation, and show how this provides the opportunity to add powerful monitoring and runtime verification capabilities to a system, e.g., to prevent collisions, or trigger an emergency landing. Since the runtime used to execute formal models is specifically designed to be faithful to their semantics, every execution (in the implementation) can be mapped to a trace in the specification. As a result, we can also prove many interesting properties offline, using model-checking techniques. We give several examples, such as properties about schedulability, worst-case traversal time, or mutual exclusion.We believe that having a consistent workflow, from an initial specification of our system, down to a formal, executable specification is a major advance in robotics and opens the way for verification of functional components of autonomous robots and beyond. We illustrate this claim by describing a complete example based on a genuine drone flight controller.

Decentralized Asynchronous Crash-resilient Runtime Verification

Runtime verification is a lightweight method for monitoring the formal specification of a system during its execution. It has recently been shown that a given state predicate can be monitored consistently by a set of crash-prone asynchronous distributed monitors observing the system, only if each monitor can emit verdicts taken from a large enough finite set. We revisit this impossibility result in the concrete context of linear-time logic ( ltl ) semantics for runtime verification, that is, when the correctness of the system is specified by an ltl formula on its execution traces. First, we show that monitors synthesized based on the 4-valued semantics of ltl ( rv-ltl ) may result in inconsistent distributed monitoring, even for some simple ltl formulas. More generally, given any ltl formula φ, we relate the number of different verdicts required by the monitors for consistently monitoring φ, with a specific structural characteristic of φ called its alternation number . Specifically, we show that, for every k ≥ 0 , there is an ltl formula φ with alternation number k that cannot be verified at runtime by distributed monitors emitting verdicts from a set of cardinality smaller than k + 1. On the positive side, we define a family of logics, called distributed ltl (abbreviated as dltl ), parameterized by k ≥ 0, which refines rv-ltl by incorporating 2k + 4 truth values. Our main contribution is to show that, for every k ≥ 0, every ltl formula φ with alternation number k can be consistently monitored by distributed monitors, each running an automaton based on a (2 ⌈ k /2 ⌉ +4)-valued logic taken from the dltl family.

Decentralized runtime verification of message sequences in message-based systems

Decentralized runtime verification of message sequences in message-based systems

PSTMonitor: Monitor synthesis from probabilistic session types

We present PSTMonitor, a tool for the run-time verification of quantitative specifications of message-passing applications, based on probabilistic session types. The key element of PSTMonitor is the detection of executions that deviate from expected probabilistic behaviour. Besides presenting PSTMonitor and its operation, the paper analyses its feasibility in terms of the runtime overheads it induces.

Extending Attack-Fault Trees with Runtime Verification

Autonomous systems are often complex and prone to software failures and cyber-attacks. We introduce RVAFTs, an extension of Attack-Fault Trees (AFTs) with runtime events that can be used to construct runtime monitors. These monitors are able to detect when failures, that can be caused either by an attack or by a fault, occur. The safety and security properties monitored are, in turn, derived from the hierarchical decomposition of RVAFTs. Our approach not only provides further use of AFTs, but also improves the process of instrumentation often required in runtime verification. We explain the principles and provide a simple case study demonstrating how RVAFTs can be used in practice. Through this we are also able to evaluate the detection of faults and attacks as well as assessing the computational overhead of the monitors.

Monitoring ROS2: from Requirements to Autonomous Robots

Runtime verification (RV) has the potential to enable the safe operation of safety-critical systems that are too complex to formally verify, such as Robot Operating System 2 (ROS2) applications. Writing correct monitors can itself be complex, and errors in the monitoring subsystem threaten the mission as a whole. This paper provides an overview of a formal approach to generating runtime monitors for autonomous robots from requirements written in a structured natural language. Our approach integrates the Formal Requirement Elicitation Tool (FRET) with Copilot, a runtime verification framework, through the Ogma integration tool. FRET is used to specify requirements with unambiguous semantics, which are then automatically translated into temporal logic formulae. Ogma generates monitor specifications from the FRET output, which are compiled into hard-real time C99. To facilitate integration of the monitors in ROS2, we have extended Ogma to generate ROS2 packages defining monitoring nodes, which run the monitors when new data becomes available, and publish the results of any violations. The goal of our approach is to treat the generated ROS2 packages as black boxes and integrate them into larger ROS2 systems with minimal effort.

Generating Safe Autonomous Decision-Making in ROS

The Robot Operating System (ROS) is a widely used framework for building robotic systems. It offers a wide variety of reusable packages and a pattern for new developments. It is up to developers how to combine these elements and integrate them with decision-making for autonomous behavior. The feature of such decision-making that is in general valued the most is safety assurance. In this research preview, we present a formal approach for generating safe autonomous decision-making in ROS. We first describe how to improve our existing static verification approach to verify multi-goal multi-agent decision-making. After that, we describe how to transition from the improved static verification approach to the proposed runtime verification approach. An initial implementation of this research proposal yields promising results.

The Discourje project: run-time verification of communication protocols in Clojure

To simplify shared-memory concurrent programming, languages have started to offer core support for high-level communications primitives, in the form of message passing though channels, in addition to lower-level synchronisation primitives. Yet, a growing body of evidence suggests that channel-based programming abstractions also have their issues. The Discourje project aims to help programmers cope with channels and concurrency bugs in Clojure programs, based on dynamic analysis. The idea is that programmers write not only implementations of communication protocols in their Clojure programs, but also specifications. Discourje then offers a run-time verification library to ensure that channel actions in implementations are safe relative to specifications. The aim of this paper is to provide a comprehensive overview of the current state of Discourje, including case studies, theoretical foundations, and practical aspects.

Runtime verification of real-time event streams using the tool HStriver

Runtime verification of real-time event streams using the tool HStriver

RV4JaCa – Runtime Verification for Multi-Agent Systems

This paper presents a Runtime Verification (RV) approach for Multi-Agent Systems (MAS) using the JaCaMo framework. Our objective is to bring a layer of security to the MAS. This layer is capable of controlling events during the execution of the system without needing a specific implementation in the behaviour of each agent to recognise the events. MAS have been used in the context of hybrid intelligence. This use requires communication between software agents and human beings. In some cases, communication takes place via natural language dialogues. However, this kind of communication brings us to a concern related to controlling the flow of dialogue so that agents can prevent any change in the topic of discussion that could impair their reasoning. We demonstrate the implementation of a monitor that aims to control this dialogue flow in a MAS that communicates with the user through natural language to aid decision-making in hospital bed allocation.