Linear Temporal Logic Research Articles

Overcoming Exploration: Deep Reinforcement Learning for Continuous Control in Cluttered Environments From Temporal Logic Specifications

Model-free continuous control for robot navigation tasks using Deep Reinforcement Learning (DRL) that relies on noisy policies for exploration is sensitive to the density of rewards. In practice, robots are usually deployed in cluttered environments, containing many obstacles and narrow passageways. Designing dense effective rewards is challenging, resulting in exploration issues during training. Such a problem becomes even more serious when tasks are described using temporal logic specifications. This work presents a deep policy gradient algorithm for controlling a robot with unknown dynamics operating in a cluttered environment when the task is specified as a Linear Temporal Logic (LTL) formula. To overcome the environmental challenge of exploration during training, we propose a novel path planning-guided reward scheme by integrating sampling-based methods to effectively complete goal-reaching missions. To facilitate LTL satisfaction, our approach decomposes the LTL mission into sub-goal-reaching tasks that are solved in a distributed manner. Our framework is shown to significantly improve performance (effectiveness, efficiency) and exploration of robots tasked with complex missions in large-scale cluttered environments.

A Specification-Guided Framework for Temporal Logic Control of Nonlinear Systems

This article proposes a specification-guided framework for control of nonlinear systems with linear temporal logic (LTL) specifications. In contrast with well-known abstraction-based methods, the proposed framework directly characterizes the winning set, i.e., the set of initial conditions from which a given LTL formula can be realized, over the continuous state space of the system via a monotonic operator. Following this characterization, an algorithm is proposed to practically approximate the operator via an adaptive interval subdivision scheme, which yields a finite-memory control strategy. We show that the proposed algorithm is sound for full LTL specifications, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">robustly complete</i> for specifications recognizable by deterministic Büchi automaton (DBA), the latter in the sense that control strategies can be found whenever the given specification can be satisfied with additional bounded disturbances. Without having to compute and store the abstraction and the resulting product system with the DBA, the proposed method is more memory efficient, which is demonstrated by complexity analysis and performance tests. A preprocessing stage is also devised to reduce computational cost via a decomposition of the specification. We show that the proposed method can effectively solve real-world control problems, such as jet engine compressor control and motion planning for manipulators and mobile robots.

Automatic encoding and repair of reactive high-level tasks with learned abstract representations

We present a framework for the automatic encoding and repair of high-level tasks. Given a set of skills a robot can perform, our approach first abstracts sensor data into symbols and then automatically encodes the robot’s capabilities in Linear Temporal Logic (LTL). Using this encoding, a user can specify reactive high-level tasks, for which we can automatically synthesize a strategy that executes on the robot, if the task is feasible. If a task is not feasible given the robot’s capabilities, we present two methods, one enumeration-based and one synthesis-based, for automatically suggesting additional skills for the robot or modifications to existing skills that would make the task feasible. We demonstrate our framework on a Baxter robot manipulating blocks on a table, a Baxter robot manipulating plates on a table, and a Kinova arm manipulating vials, with multiple sensor modalities, including raw images.

Semantic Analysis of a Linear Temporal Extension of Quantum Logic and Its Dynamic Aspect

Although various dynamic or temporal logics have been proposed to verify quantum protocols and systems, these two viewpoints have not been studied comprehensively enough. We propose Linear Temporal Quantum Logic (LTQL), a linear temporal extension of quantum logic with a quantum implication, and extend it to Dynamic Linear Temporal Quantum Logic (DLTQL). This logic has temporal operators to express transitions by unitary operators (quantum gates) and dynamic ones to express those by projections (projective measurement). We then prove some logical properties of the relationship between these two transitions expressed by LTQL and DLTQL. A drawback in applying LTQL to the verification of quantum protocols is that these logics cannot express the future operator in linear temporal logic. We propose a way to mitigate this drawback by using a translation from (D)LTQL to Linear Temporal Modal Logic (LTML) and a simulation. This translation reduces the satisfiability problem of (D)LTQL formulas to that of LTML with the classical semantics over quantum states.

LDES: detector design for version number attack detection using linear temporal logic based on discrete event system

The Internet Engineering Task Force (IETF) has defined routing protocols for low power and lossy networks (RPL) for constrained devices. RPL constructs DODAGs (destination oriented directed acyclic graphs), to optimize routing. RPL ensures acyclic topology with the DODAG version number. However, the control message’s DODAG version number is not authenticated. So, RPL is vulnerable to network resource attack known as DODAG Version Number (DVN) attack. DVN attack creates a packet delay, packet loss, cyclic topology, etc., in the network. This paper proposes a method for detecting DODAG version number attacks. Several existing schemes to defend against the DVN, such as cryptographic techniques, trust-based, threshold-based and mitigation are computationally intensive or require protocol modification. DVN does not change the packet format or sequence of packets, but can still perform attacks and hence fall under the category of stealthy attacks, which are difficult to detect using traditional intrusion detection system’s (IDS). Discrete-event system (DES) based IDS have been applied in the literature for stealthy attacks that achieve low overhead, low false alarm rate, etc. However, the construction of DES-based IDS for network protocol may lead to errors, as modelling is manual. The resulting IDS, therefore, is unable to guarantee its correctness. This paper proposes linear temporal logic (LTL) based DES paradigm to detect DVN. LTL-based paradigm facilitates formal verification of the DES-based IDS, and hence the correctness of the scheme is ascertained. The proposed technique is simulated using the Contiki cooja simulator. When the percentage of spiteful nodes in the network increases, the true positive rate, and packet delivery rate drops, while the false positive rate and control message overhead increase. The memory requirement for sending the packets and verifying the nodes is minimal. The LTL-based IDS has been formally verified using NuSMV to ensure the correctness of the framework.

Deciding FO-rewritability of Regular Languages and Ontology-Mediated Queries in Linear Temporal Logic

Our concern is the problem of determining the data complexity of answering an ontology-mediated query (OMQ) formulated in linear temporal logic LTL over (Z,&lt;) and deciding whether it is rewritable to an FO(&lt;)-query, possibly with some extra predicates. First, we observe that, in line with the circuit complexity and FO-definability of regular languages, OMQ answering in AC0, ACC0 and NC1 coincides with FO(&lt;,≡)-rewritability using unary predicates x ≡ 0 (mod n), FO(&lt;,MOD)-rewritability, and FO(RPR)-rewritability using relational primitive recursion, respectively. We prove that, similarly to known PSᴘᴀᴄᴇ-completeness of recognising FO(&lt;)-definability of regular languages, deciding FO(&lt;,≡)- and FO(&lt;,MOD)-definability is also PSᴘᴀᴄᴇ-complete (unless ACC0 = NC1). We then use this result to show that deciding FO(&lt;)-, FO(&lt;,≡)- and FO(&lt;,MOD)-rewritability of LTL OMQs is ExᴘSᴘᴀᴄᴇ-complete, and that these problems become PSᴘᴀᴄᴇ-complete for OMQs with a linear Horn ontology and an atomic query, and also a positive query in the cases of FO(&lt;)- and FO(&lt;,≡)-rewritability. Further, we consider FO(&lt;)-rewritability of OMQs with a binary-clause ontology and identify OMQ classes, for which deciding it is PSᴘᴀᴄᴇ-, Π2p- and coNP-complete.

Linear Temporal Public Announcement Logic: A New Perspective for Reasoning About the Knowledge of Multi-classifiers

In this paper, a formal transition system model is presented called Linear Temporal Public Announcement Logic (LTPAL) to extract knowledge in a classification process. The model combines Public Announcement Logic (PAL) and Linear Temporal Logic (LTL). For this purpose, first, an epistemic logic model is created to capture information gathered by classifiers in single-framed data input. Next, using LTL, classifiers are considered for data stream inputs. Then, a verification method is proposed for such data streams. Finally, we formalize natural language properties in LTPAL with a video-stream object detection sample.

Program Synthesis for Cyber-Resilience

Architectural tactics enable stakeholders to achieve cyber-resilience requirements. They permit systems to react, resist, detect, and recover from cyber incidents. This paper presents an approach to generate source code for architectural tactics typically used in safety and mission-critical systems. Our approach extensively relies on the use of the <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Event-B</small> formal method and the <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EventB2Java</small> code generation plugin of the Rodin platform. It leverages the modeling of architectural tactics in the <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Event-B</small> formal language and uses a set of <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EventB2Java</small> transformation rules to generate certified code implementations for the said tactics. Since resilience requirements are statements about a system over time, and because of the fact that the <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Event-B</small> language does not provide (native) support for the writing of temporal specifications, we have implemented a novel Linear Temporal Logic (LTL) extension for <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Event-B</small> . We support several architectural tactics for availability, performance, and security. The generated code is certified in the following sense: discharging proof obligations in Rodin - the platform we use for writing the <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Event-B</small> models - attests to the soundness of the architectural tactics modelled in <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Event-B</small> , and the soundness of the translation encoded by the <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">EventB2Java</small> tool attests to the code correctness. Finally, we demonstrate the usability of our resilience validation approach with the aid of an Autonomous Vehicle System. It further helped us increase our confidence in the soundness of our Event-B LTL extension.

A Subclass of Mu-Calculus with the Freeze Quantifier Equivalent to Register Automata

Register automaton (RA) is an extension of finite automaton by adding registers storing data values. RA has good properties such as the decidability of the membership and emptiness problems. Linear temporal logic with the freeze quantifier (LTL↓) proposed by Demri and Lazić is a counterpart of RA. However, the expressive power of LTL↓ is too high to be applied to automatic verification. In this paper, we propose a subclass of modal µ-calculus with the freeze quantifier, which has the same expressive power as RA. Since a conjunction ψ1 ∧ ψ2 in a general LTL↓ formula cannot be simulated by RA, the proposed subclass prohibits at least one of ψ1 and ψ2 from containing the freeze quantifier or a temporal operator other than X (next). Since the obtained subclass of LTL↓ does not have the ability to represent a cycle in RA, we adopt µ-calculus over the subclass of LTL↓, which allows recursive definition of temporal formulas. We provide equivalent translations from the proposed subclass of µ-calculus to RA and vice versa and prove their correctness.

Safe reinforcement learning under temporal logic with reward design and quantum action selection

This paper proposes an advanced Reinforcement Learning (RL) method, incorporating reward-shaping, safety value functions, and a quantum action selection algorithm. The method is model-free and can synthesize a finite policy that maximizes the probability of satisfying a complex task. Although RL is a promising approach, it suffers from unsafe traps and sparse rewards and becomes impractical when applied to real-world problems. To improve safety during training, we introduce a concept of safety values, which results in a model-based adaptive scenario due to online updates of transition probabilities. On the other hand, a high-level complex task is usually formulated via formal languages, including Linear Temporal Logic (LTL). Another novelty of this work is using an Embedded Limit-Deterministic Generalized Büchi Automaton (E-LDGBA) to represent an LTL formula. The obtained deterministic policy can generalize the tasks over infinite and finite horizons. We design an automaton-based reward, and the theoretical analysis shows that an agent can accomplish task specifications with the maximum probability by following the optimal policy. Furthermore, a reward shaping process is developed to avoid sparse rewards and enforce the RL convergence while keeping the optimal policies invariant. In addition, inspired by quantum computing, we propose a quantum action selection algorithm to replace the existing varepsilon-greedy algorithm for the balance of exploration and exploitation during learning. Simulations demonstrate how the proposed framework can achieve good performance by dramatically reducing the times to visit unsafe states while converging optimal policies.

A proof system for unified temporal logic

Unified Temporal Logic (UTL) combines all characteristics of traditional Linear Temporal Logic (LTL) and Propositional Projection Temporal Logic (PPTL). It can be used to describe full regular and omega-regular properties, which are often encountered in the field of formal verification. To support formal verification with UTL, this paper proposes a proof system for UTL. First, the syntax and semantics of UTL are briefly introduced. Further, axioms and inference rules are formalized. Besides, a number of theorems are derived and proved to refine the system. Moreover, the soundness and completeness of the proof system are proved in detail. To facilitate the completeness proof, some auxiliary lemmas are introduced and proved in advance. Finally, an example is given to illustrate how to use this proof system for verifying properties of a system.

Jpf-logic

We present jpf-logic, an extension of the model checker Java PathFinder (JPF). Our extension jpf-logic provides a framework to check properties expressed in temporal logics such as linear temporal logic (LTL) and computation tree logic (CTL). To support a logic in our framework, we (1) implement a parser for the logic, (2) develop a hierarchy of classes that represent the abstract syntax of the logic and implement a transformation from parse trees of formulas to the corresponding abstract syntax trees, and (3) implement a model checking algorithm that takes as input an abstract syntax tree of a formula and a partial transition system. The latter represents a model of the Java application. All three components have been implemented for CTL. The first two have been implemented for LTL.

Defeasible linear temporal logic

After the seminal work of Kraus, Lehmann and Magidor (formally known as the KLM approach) on conditionals and preferential models, many aspects of defeasibility in more complex formalisms have been studied in recent years. Examples of these aspects are the notion of typicality in description logic and defeasible necessity in modal logic. We discuss a new aspect of defeasibility that can be expressed in the case of temporal logic, which is the normality in an execution. In this contribution, we take Linear Temporal Logic () as case study for this defeasible aspect. has found extensive applications in Computer Science and Artificial Intelligence, notably as a formal framework for representing and verifying computer systems that vary over time. However, some systems may presents exceptions at some innocuous time points where they can be tolerated, or conversely, exceptions at other crucial time points where they need to be addressed. In order to ensure the reliability of such systems, we study a preferential extension of , called defeasible linear temporal logic (). In the first part of this paper, we show how semantics of KLM's preferential models can be integrated with . We also discuss the addition of non-monotonic temporal operators as a way to formalise defeasible properties of these systems. The second part of this paper is a study of the satisfiability problem of sentences. Based on Sistla and Clarke's work on the complexity of the classical language, we show the bounded-model property of two fragments of language. Moreover, we provide a procedure to check the satisfiability of sentences in both of these fragments.

Optimal Probabilistic Motion Planning With Potential Infeasible LTL Constraints

This paper studies optimal motion planning subject to motion and environment uncertainties. By modeling the system as a probabilistic labeled Markov decision process (PL-MDP), the control objective is to synthesize a finite-memory policy, under which the agent satisfies complex high-level tasks expressed as linear temporal logic (LTL) with desired satisfaction probability. In particular, the cost optimization of the trajectory that satisfies infinite horizon tasks is considered, and the trade-off between reducing the expected mean cost and maximizing the probability of task satisfaction is analyzed. The LTL formulas are converted to limit-deterministic Büchi automata (LDBA) with a reachability acceptance condition and a compact graph structure. The novelty of this work lies in considering the cases where LTL specifications can be potentially infeasible and developing a relaxed product MDP between PL- MDP and LDBA. The relaxed product MDP allows the agent to revise its motion plan whenever the task is not fully feasible and quantify the revised plan’s violation measurement. A multi- objective optimization problem is then formulated to jointly consider the probability of task satisfaction, the violation with respect to original task constraints, and the implementation cost of the policy execution. The formulated problem can be solved via coupled linear programs. This work first bridges the gap between probabilistic planning revision of potential infeasible LTL specifications and optimal control synthesis of both plan prefix and plan suffix of the trajectory over the infinite horizons. Experimental results are provided to demonstrate the effectiveness of the proposed framework.

To Explore or Not to Explore: Regret-Based LTL Planning in Partially-Known Environments

In this paper, we investigate the optimal robot path planning problem for high-level specifications described by co-safe linear temporal logic (LTL) formulae. We consider the scenario where the map geometry of the workspace is partially-known. Specifically, we assume that there are some unknown regions, for which the robot does not know their successor regions a priori unless it reaches these regions physically. In contrast to the standard game-based approach that optimizes the worst-case cost, in the paper, we propose to use regret as a new metric for planning in such a partially-known environment. The regret of a plan under a fixed but unknown environment is the difference between the actual cost incurred and the best-response cost the robot could have achieved if it realizes the actual environment with hindsight. We provide an effective algorithm for finding an optimal plan that satisfies the LTL specification while minimizing its regret. A case study on firefighting robots is provided to illustrate the proposed framework. We argue that the new metric is more suitable for the scenario of partially-known environment since it captures the trade-off between the actual cost spent and the potential benefit one may obtain for exploring an unknown region.

A fairness-based refinement strategy to transform liveness properties in Event-B models

Stepwise development supported by the Event-B formalism has been used in the domain of system design and verification. This refinement approach guarantees that safety properties are preserved with Event-B proof obligations, while additional reasoning and fairness assumptions are required to prove the transformation of liveness properties in Event-B models. This paper presents a new proof-based approach that integrates Linear Temporal Logic (LTL) and Event-B for reasoning about the refinement of liveness properties. We first developed an extended version of LTL that could be used in Event-B models with three basic atomic propositions: state-related propositions, event-related propositions, and enabledness of events. Then we employed this extended LTL to express four important temporal properties with Event-B proof obligations and fairness assumptions. Besides the typical Event-B proof obligations, we specified conditions such as relative deadlock freeness, conditional convergence, and fairness assumptions to allow temporal properties to be transformed during refinement steps. The generic fairness-based refinement strategies were developed to replace or strengthen the fairness assumptions in the refinement steps of Event-B models. A reliable retransmission example is used to illustrate the approach. • Formal presentation of temporal properties with proof obligations and fairness assumptions. • Define transformation function to map abstract temporal properties into concrete ones. • Provide fairness-based refinement strategies to transform temporal properties.

Data-Driven Safe Policy Optimization for Black-Box Dynamical Systems With Temporal Logic Specifications.

Learning-based policy optimization methods have shown great potential for building general-purpose control systems. However, existing methods still struggle to achieve complex task objectives while ensuring policy safety during learning and execution phases for black-box systems. To address these challenges, we develop data-driven safe policy optimization (D 2 SPO), a novel reinforcement learning (RL)-based policy improvement method that jointly learns a control barrier function (CBF) for system safety and a linear temporal logic (LTL) guided RL algorithm for complex task objectives. Unlike many existing works that assume known system dynamics, by carefully constructing the data sets and redesigning the loss functions of D 2 SPO, a provably safe CBF is learned for black-box dynamical systems, which continuously evolves for improved system safety as RL interacts with the environment. To deal with complex task objectives, we take advantage of the capability of LTL in representing the task progress and develop LTL-guided RL policy for efficient completion of various tasks with LTL objectives. Extensive numerical and experimental studies demonstrate that D 2 SPO outperforms most state-of-the-art (SOTA) baselines and can achieve over 95% safety rate and nearly 100% task completion rates. The experiment video is available at https://youtu.be/2RgaH-zcmkY.

Failure-Aware Self-Diagnostic Task Planning under Temporal Logic Specifications*

This paper investigates the problem of task planning for high-level specifications described by linear temporal logic (LTL) formulae. Existing works on this topic mainly based on the assumption that the functionalities of the system are always correct during the execution. In this work, we consider the scenario where the system is subject to internal failures that cannot be measured directly but may be inferred by a sequence of actions. The objective is to design a failure-aware task plan such that (i) the system will achieve the LTL task when there is no failure; and (ii) along the designed plan, any potential failure can be detected within a bounded number of steps. We provide a framework for modeling the behavior of the system with potential internal failures. Furthermore, an effective algorithm is designed to synthesize an optimal self-diagnostic plan, in the form of the prefix-suffix structure, such that these two requirements are satisfied. We illustrate the proposed framework by a case study of production task planning.

Software Tool for Distribution of Linear Temporal Logic Specifications

This work presents the implementation of a software tool that ensures the execution of a global Linear Temporal Logic (LTL) specification for a team of mobile agents that are evolving in a static and known environment cluttered with regions of interest. The software is build on formalisms that extend our previous work, with the main contribution of handling strictly more expressive missions that also allow negations (or temporal avoidance) of some regions of interest. An algorithm is provided for decomposing the global LTL mission into independent robots’ tasks, together with a flow diagram of the entire method that solves the path planning problem. Simulations and comparative results are discussed for a case study, in order to support the usefulness of the software tool.

Synthesis of Non-blocking Controllers for Linear Temporal Logic Tasks under Partial Observations⋆

In this paper, we investigate the formal synthesis of discrete controllers for linear temporal logic tasks under partial information. Existing works on this topic mainly focus on find sure winning or almost sure winning control strategies under some assumptions regarding the system's atomic propositions. In this work, we consider non-blockingness as the metric as the achievement of the task. Specifically, we require that at each instant, the controller maintains the possibility to achieve the LTL task under some environment's behavior. We first present an offline algorithm for the computation of the winning region over the belief state space. Then we present an online control algorithm that effective solves the control synthesis problem. The proposed control algorithm is also illustrated by a case study of robot task planning.