Formal Automata Research Articles

Reinforcement Learning for Omega-Regular Specifications on Continuous-Time MDP

Continuous-time Markov decision processes (CTMDPs) are canonical models to express sequential decision-making under dense-time and stochastic environments. When the stochastic evolution of the environment is only available via sampling, model-free reinforcement learning (RL) is the algorithm-of-choice to compute optimal decision sequence. RL, on the other hand, requires the learning objective to be encoded as scalar reward signals. Since doing such translations manually is both tedious and error-prone, a number of techniques have been proposed to translate high-level objectives (expressed in logic or automata formalism) to scalar rewards for discrete-time Markov decision processes. Unfortunately, no automatic translation exists for CTMDPs. We consider CTMDP environments against the learning objectives expressed as omega-regular languages. Omega-regular languages generalize regular languages to infinite-horizon specifications and can express properties given in popular linear-time logic LTL. To accommodate the dense-time nature of CTMDPs, we consider two different semantics of omega-regular objectives: 1) satisfaction semantics where the goal of the learner is to maximize the probability of spending positive time in the good states, and 2) expectation semantics where the goal of the learner is to optimize the long-run expected average time spent in the ''good states'' of the automaton. We present an approach enabling correct translation to scalar reward signals that can be readily used by off-the-shelf RL algorithms for CTMDPs. We demonstrate the effectiveness of the proposed algorithms by evaluating it on some popular CTMDP benchmarks with omega-regular objectives.

Biologically-Inspired Pulse Signal Processing for Intelligence at the Edge.

There is an ever-growing mismatch between the proliferation of data-intensive, power-hungry deep learning solutions in the machine learning (ML) community and the need for agile, portable solutions in resource-constrained devices, particularly for intelligence at the edge. In this paper, we present a fundamentally novel approach that leverages data-driven intelligence with biologically-inspired efficiency. The proposed Sparse Embodiment Neural-Statistical Architecture (SENSA) decomposes the learning task into two distinct phases: a training phase and a hardware embedment phase where prototypes are extracted from the trained network and used to construct fast, sparse embodiment for hardware deployment at the edge. Specifically, we propose the Sparse Pulse Automata via Reproducing Kernel (SPARK) method, which first constructs a learning machine in the form of a dynamical system using energy-efficient spike or pulse trains, commonly used in neuroscience and neuromorphic engineering, then extracts a rule-based solution in the form of automata or lookup tables for rapid deployment in edge computing platforms. We propose to use the theoretically-grounded unifying framework of the Reproducing Kernel Hilbert Space (RKHS) to provide interpretable, nonlinear, and nonparametric solutions, compared to the typical neural network approach. In kernel methods, the explicit representation of the data is of secondary nature, allowing the same algorithm to be used for different data types without altering the learning rules. To showcase SPARK’s capabilities, we carried out the first proof-of-concept demonstration on the task of isolated-word automatic speech recognition (ASR) or keyword spotting, benchmarked on the TI-46 digit corpus. Together, these energy-efficient and resource-conscious techniques will bring advanced machine learning solutions closer to the edge.

A minimization algorithm for automata generated fault trees with priority gates

Fault tree analysis is still widely practiced in high-hazard industries. We propose in this article an algorithm for the reduction of fault tree expressions that are generated from automata representations of failure behaviors. Automata formalisms are increasingly being used to describe systems exhibiting sequence-dependent failures—i.e., the overall outcome like a total failure of the system can depend on the order in which events occur. A set of paths leading to a safety-relevant state is encoded as a standard sum of product canonical form, and without any loss of the significance of the sequencing of events. That is, the corresponding fault tree expression is basically a Boolean formula which is extended with the necessary temporal features (event occurrence priority). Such expressions can then be reduced into minimal canonical forms by using the Boolean methods together with the required temporal logic calculus. Since minimal failure sequences can be determined from the obtained reduced models, the proposed approach can improve the analysis of the dynamic effects of the sequencing of faults and propagated errors in such models. As a consequence, it can have a positive impact on the design of failure prevention measures. A fault tolerant example system exhibiting dynamic behavior is used to highlight the benefits of the approach.

Prebiotic Geochemical Automata at the Intersection of Radiolytic Chemistry, Physical Complexity, and Systems Biology

The tractable history of life records a successive emergence of organisms composed of hierarchically organized cells and greater degrees of individuation. The lowermost object level of this hierarchy is the cell, but it is unclear whether the organizational attributes of living systems extended backward through prebiotic stages of chemical evolution. If the systems biology attributes of the cell were indeed templated upon prebiotic synthetic relationships between subcellular objects, it is not obvious how to categorize object levels below the cell in ways that capture any hierarchies which may have preceded living systems. In this paper, we map out stratified relationships between physical components that drive the production of key prebiotic molecules starting from radiolysis of a small number of abundant molecular species. Connectivity across multiple levels imparts the potential to create and maintain far-from-equilibrium chemical conditions and to manifest nonlinear system behaviors best approximated using automata formalisms. The architectural attribute of “information hiding” of energy exchange processes at each object level is shared with stable, multitiered automata such as digital computers. These attributes may indicate a profound connection between the system complexity afforded by energy dissipation by subatomic level objects and the emergence of complex automata that could have preceded biological systems.

Regular Language Representations in the Constructive Type Theory of Coq

We explore the theory of regular language representations in the constructive type theory of Coq. We cover various forms of automata (deterministic, nondeterministic, one-way, two-way), regular expressions, and the logic WS1S. We give translations between all representations, show decidability results, and provide operations for various closure properties. Our results include a constructive decidability proof for the logic WS1S, a constructive refinement of the Myhill-Nerode characterization of regularity, and translations from two-way automata to one-way automata with verified upper bounds for the increase in size. All results are verified with an accompanying Coq development of about 3000 lines.

Centralized coordination vs. partially-distributed coordination with Reo and constraint automata

High-level concurrency notations and abstractions have several well-known software engineering advantages when it comes to programming concurrency protocols among threads. To also explore their complementary performance advantages, in ongoing work, we are developing compilation technology for a high-level coordination language, Reo, based on this language's formal automaton semantics. By now, as shown in our previous work, our tools are capable of generating code that can compete with carefully hand-crafted code, at least for some protocols. An important prerequisite to further advance this promising technology, now, is to gain a better understanding of how the significantly different compilation approaches that we developed so far, which vary in the amount of parallelism in their generated code, compare against each other. For instance, to better and more reliably tune our compilers, we must learn under which circumstances parallel protocol code, with high throughput but also high latency, outperforms sequential protocol code, with low latency but also low throughput. In this paper, we report on an extensive performance comparison between these approaches for a substantial number of protocols, expressed in Reo. Because we have always formulated our compilation technology in terms of a general kind of communicating automaton (i.e., constraint automata), our findings apply not only to Reo but, in principle, to any language whose semantics can be defined in terms of such automata.

Load estimator-based hybrid controller design for two-interleaved boost converter dedicated to renewable energy and automotive applications

This paper is devoted to the development of a hybrid controller for a two-interleaved boost converter dedicated to renewable energy and automotive applications. The control requirements, resumed in fast transient and low input current ripple, are formulated as a problem of fast stabilization of a predefined optimal limit cycle, and solved using hybrid automaton formalism. In addition, a real time estimation of the load is developed using an algebraic approach for online adjustment of the hybrid controller. Mathematical proofs are provided with simulations to illustrate the effectiveness and the robustness of the proposed controller despite different disturbances. Furthermore, a fuel cell system supplying a resistive load through a two-interleaved boost converter is also highlighted.

Regular Behaviours with Names

Nominal sets provide a framework to study key notions of syntax and semantics such as fresh names, variable binding and α-equivalence on a conveniently abstract categorical level. Coalgebras for endofunctors on nominal sets model, e.g., various forms of automata with names as well as infinite terms with variable binding operators (such as λ-abstraction). Here, we first study the behaviour of orbit-finite coalgebras for functors \(\bar F\) on nominal sets that lift some finitary set functor F. We provide sufficient conditions under which the rational fixpoint of \(\bar F\), i.e. the collection of all behaviours of orbit-finite \(\bar F\)-coalgebras, is the lifting of the rational fixpoint of F. Second, we describe the rational fixpoint of the quotient functors: we introduce the notion of a sub-strength of an endofunctor on nominal sets, and we prove that for a functor G with a sub-strength the rational fixpoint of each quotient of G is a canonical quotient of the rational fixpoint of G. As applications, we obtain a concrete description of the rational fixpoint for functors arising from so-called binding signatures with exponentiation, such as those arising in coalgebraic models of infinitary λ-terms and various flavours of automata.

On the limit cycle stabilization of a DC/DC three-cell converter

This paper investigates a new state feedback controller design for DC/DC three-cell converter. The high dynamic performances of the converter have been translated into a desired limit cycle stabilization problem. The state feedback is established using hybrid automaton formalism, which allows us to design independently a transitory state and steady state control schemes. Finite-time convergence of the trajectory to the steady state and the local asymptotic stability of the desired limit cycle under the proposed controller are proved analytically. Simulation and experimental results confirm the theoretical results and show the effectiveness and robustness of the hybrid controller in spite of load variations.

Execution information rate for some classes of automata

We study the Shannon information rate of accepting runs of various forms of automata. This rate is a complexity indicator for executions of these automata. Accepting runs of finite automata and reversal-bounded nondeterministic counter machines, as well as their restrictions and variations, are investigated and are shown, in many cases, to have computable execution rates. We also study the information rate of behaviors in discrete timed automata. We conduct experiments on C programs showing that estimating the information rates for their executions is feasible in many cases.

Using JFLAP to engage students and improve learning of computer science theory

Many students experience both difficulty and lack of engagement during their introductions to the essential computer science theory concepts of formal languages and automata. JFLAP is a freely-avai...

Finite-time stabilization of the limit cycle of two-cell DC/DC converter: hybrid approach

The last decade has witnessed increasing developments on multilevel converters to rectify the problem of high power processing via switch mode converters. The outcome of these advances is expressed by many innovative topologies such as DC–DC multicellular converter. This paper investigates a state feedback stabilization of two-cell converter limit cycle through hybrid dynamic systems theory. For this purpose, a desired limit cycle is firstly defined by an analysis of PWM control technique. Then, the state feedback is developed using hybrid automaton formalism, which allows us to design independently a transitory-state control scheme and steady-state control scheme. Finite-time stability of the desired limit cycle is proved analytically. Simulation results confirm the theoretical results and show the robustness of the proposed controller in spite of different perturbations. Furthermore, the proposed hybrid controller can be extended to other multilevel converters.

Component interconnect and data access interface for embedded vision applications

IP-based design is used to tackle complexity and reduce time-to-market in systems-on-chip with high-performance requirements. Component integration, the main part in this process, is a complicated and time-consuming task, largely due to interfacing issues. Standard interfaces can help to reduce the integration efforts. However, existing implementations use more resources than necessary and lack of a formalism to capture and manipulate resource requirements and design constraints. In this paper, we propose a novel interface, the Component Interconnect and Data Access (CIDA), and its implementation, based on the interface automata formalism. CIDA can be used to capture system-on-chip architecture, with primarily focus on video processing applications, which are mostly based on data streaming paradigm, with occasional direct memory accesses. We introduce the notion of component-interface clustering for resource reduction and provide a method to automatize this process. With real-life video processing applications implemented in FPGA, we show that our approach can reduce the resource usage (#slices) by an average of 20 % and reduce power consumption by 5 % compared to implementation based on vendor interfaces.

Hybrid Systems Control Based on Smart Space Technology

The paper presents an approach to hybrid systems control based on smart space technology. The approach proposed bears two primary distinctive features: first, hybrid system control is decentralized and various control functions are implemented in separate services, communicating via ontology-based smart space; second, the services are represented by a hybrid automata formalism allowing to employ a model-driven development for hybrid control systems. The approach is illustrated by practical communication scenario of several physical devices in the context of smart home environment.

Complex maintenance programs quantification (CMPQ) to better control production systems

Purpose – The purpose of this paper is to develop a methodology for supporting complex maintenance programs quantification (CMPQ) for industrial systems. The methodology is based on a generic formalization of static and behavioral expert knowledge both on the target system and on the maintenance one. The formalization is carried out first by means of system modelling language (SysML) diagrams to model knowledge concepts and second by the transformation of these concepts into Altarica data flow (ADF) language for developing stochastic simulation. Design/methodology/approach – An industrial case study (ARE system) proposed by the electricite de France (EDF) company is used initially to show a real problem statement on CMPQ. It allows highlighting key scientific issues considered as the basis for methodology development. Main issues are related to static and dynamic knowledge formalization justifying the choice of SysML and ADF languages. The added value of this methodology is finally shown on the same case study serving as benchmark. Findings – This paper demonstrates the suitability of using of SysML language for modelling the CMPQ knowledge and then of ADF language in building executable model implementing simulation as needed for assessing key performance indicators of CMPQ. ADF is based on formal mode automaton. Mapping rules are developed to ensure correspondence between the concepts of these two languages. Research limitations/implications – Additional industrial validations of the methodology should be performed to really evaluate its benefits. Practical implications – This work was made possible thanks to a partnership with the EDF Company (French energy supplier). The results are therefore directly usable at practical industrial levels. Originality/value – The CMPQ methodology proposed is fully generic leading to offering a library of atomic ADF components (COTS) which can be instantiated to develop executable model with regards to each specific application. It allows to favor reusability and makes easier the model development above all for a user who knows nothing about the language.

Simulation framework for the verification of PLC programs in automobile industries

The objective of this study is to propose a framework of virtual plant models for the verification of PLC logic through modeling and simulation. The proposed virtual plant model consists of three types of object: the virtual device model (object model), the intermediary transfer model (functional model), and the PLC program & HMI (dynamic model). A virtual device model consists of a physical part, which is used to represent the properties of a real device and a logical part, which is used to manage the device's operation. For the fidelity of the virtual plant model, an intermediary transfer model controls the virtual device through a PLC program and sends information on the virtual device's state to the supervisory control model. Moreover, the PLC program and HMI are used for constructing communication environments similar to a real manufacturing line. For the implementation of the proposed virtual plant model, this study employs an I/O model based on the formalism of Automata and Discrete Event Systems Specifications (DEVS). As a result of the application to a car assembly line, 18 error codes are detected through the manual mode and have been revised. Finally, we can confirm there is no sequential error in the PLC program by checking the time chart. Moreover, the bottleneck and ramp-up/down times are reduced when a manufacturing system/line is built.

Networks of automata with read arcs: a tool for distributed planning

AbstractA planning problem consists in driving a system from its current state to a set of target states. Problems are generally expressed by a collection of state variables, and actions that change the value of a subset of these variables. Such problems can be modeled as networks of automata, one per state variable, that partially synchronize on some actions. Distributed planning (or factored planning) consists in driving each of these automata to its goal state(s), while preserving the coherence of their interactions. Real planning problems, however, need to model actions that can only be performed in one component when another component is in a specific state. This paper proposes a mechanism to capture this phenomenon, under the form of automata with read arcs, reproducing what already exists for Petri nets. It is shown that a previous approach to distributed planning, based on automata computations, can be extended to this new setting.

A theory of desynchronisable closed loop system

The task of implementing a supervisory controller is non-trivial, even though different theories exist that allow automatic synthesis of these controllers in the form of automata. One of the reasons for this discord is due to the asynchronous interaction between a plant and its controller in implementations, whereas the existing supervisory control theories assume synchronous interaction. As a consequence the implementation suffer from the so-called inexact synchronisation problem. In this paper we address the issue of inexact synchronisation in a process algebraic setting, by solving a more general problem of refinement. We construct an asynchronous closed loop system by introducing a communication medium in a given synchronous closed loop system. Our goal is to find sufficient conditions under which a synchronous closed loop system is branching bisimilar to its corresponding asynchronous closed loop system.

Using finite state automata (FSA) for formal modelling of affordances in human-machine cooperative manufacturing systems

Modelling complex systems poses significant challenges on how one represents the system components and interactions among them. In order to provide a systematic approach to represent human participation as a part of a dynamic system, this paper presents a formal automata model of human-machine cooperative systems that incorporates human capabilities with respect to system conditions. Specifically, we propose a control model for human-involved shop floor systems based on discrete event-based systems (DES) and an environmental concept known as an affordance. When modelling human-involved systems where a human operator is considered a crucial system component, it is necessary to analyse the model complexity that increases significantly due to a human's behavioural patterns. From the perspective of the temporal and physical state domains a human operator's behaviour is usually limited by attention and resource constraints. We investigate these limitations and map them into constrained system affordances, and then construct a formal human-machine cooperative model based on the finite state automaton (FSA) model. The proposed model can provide a framework to combine human activities into systems operations in consideration of human's effectivities and system affordances. A detailed application example is provided to illustrate that the proposed model can effectively be applied to manufacturing control systems.

A Hybrid Automaton Model of the Cement Mill Control

Cement milling circuits are controlled by linear quadratic strategies with the purpose of retaining the circuit stability. Generally, the controlled variables of milling circuits present discontinuities over the range of values they take during the plant operation. The design of linear quadratic control algorithms is based on linear approximations of the nonlinear differential equations that hold over the continuous time ranges of the controlled variables. These approximations make the practical implementation of these algorithms less effective than expected and for this reason there is the need to verify the range of the variable values over which the algorithms can retain the circuit stability before commissioning them. In this paper, an alternative model to that of the nonlinear differential equations is developed for the verification of the milling circuit stability. The model is based on the rectangular hybrid automata formalism. This formalism provides a more systematic and formal way of modeling the plant operation over different ranges of variables presenting discontinuities and does not require to develop informal textual description of the conditions under which different sets of differential equations model the plant operation within the different continuous ranges of the variables. In order to use this model for verifying the circuit stability, the requirements for the circuit stability are expressed in the form of a logical proposition of the ranges of the values within which certain variables of the cement milling circuit must remain. The circuit stability is verified by finding whether the states that are defined by the logical proposition are included in the state space of the automaton. There are, however, cases in which existing algorithms that search into the state space either do not terminate or take very long times in identifying the existence or not of the desired state. After the elapse of a significant portion of time the analyst does not know whether this long time is due to algorithm termination problem or to the very large number of remaining computations. In this paper, a new algorithm was developed which does not search into the state space of the automaton as most of the other algorithms do, but computes the number of automaton iterations required for reaching a state. If this is a finite number one can conclude that the desired state is reachable. This algorithm was used in verifying the circuit stability.