Supervisory Control Theory Research Articles

Offline supervisory control synthesis: taxonomy and recent developments

AbstractThis paper surveys recent advances in supervisory control theory since its 30th anniversary in 2017. We performed a systematic literature search and selected 272 relevant papers, with a focus on automata-based approaches. These were fitted into coherent narratives, based in part on a taxonomy of supervisor synthesis methods. The obtained papers show the fast and vast progress of the field, where recent research covers a wide range of new synthesis optimization techniques, different aspects such as fault tolerance, security, and timing, and new and diverse application domains. This survey refrains from in-depth descriptions of techniques, instead focusing on high-level contributions of recent works, how they relate to each other, and future directions for research.

Optimal control of discrete event systems under uncertain environment based on supervisory control theory and reinforcement learning

Discrete event systems (DESs) are powerful abstract representations for large human-made physical systems in a wide variety of industries. Safety control issues on DESs have been extensively studied based on the logical specifications of the systems in various literature. However, when facing the DESs under uncertain environment which brings into the implicit specifications, the classical supervisory control approach may not be capable of achieving the performance. So in this research, we propose a new approach for optimal control of DESs under uncertain environment based on supervisory control theory (SCT) and reinforcement learning (RL). Firstly, we use SCT to gather deliberative planning algorithms with the aim to safe control. Then we convert the supervised system to Markov Decision Process simulation environments that is suitable for optimal algorithm training. Furthermore, a SCT-based RL algorithm is designed to maximize performance of the system based on the probabilistic attributes of the state transitions. Finally, a case study on the autonomous navigation task of a delivery robot is provided to corroborate the proposed method by multiple simulation experiments. The result shows the proposed approach owning 8.27%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} performance improvement compared with the non-intelligent methods. This research will contribute to further studying the optimal control of human-made physical systems in a wide variety of industries.

Integrating reinforcement learning and supervisory control theory for optimal directed control of discrete-event systems

Directed control is crucial for implementing controllers on programmable logic controllers. A directed controller is one that actively selects at most one controllable event (control command) to execute at any instant. This study investigates the numerical optimization problem of directed control for discrete-event systems, such as traffic systems and robotic systems. By integrating supervisory control theory and reinforcement learning, an algorithmic procedure is designed to synthesize an optimal directed controller, which minimizes the total cost associated with event execution while ensuring the safety and liveness of the controlled system. Two improved Q-learning algorithms incorporating dynamic parameter adaptation strategies are developed to enhance the global search ability of basic Q-learning. To demonstrate applicability and effectiveness, we apply the proposed method to control a guideway system and a multi-train traffic system, respectively. The experimental results indicate the superiority of the proposed method over the comparison methods.

Fuzzy Partitioned Discrete-event System and Its Decentralized Supervisory Control

This work uses the idea of a fuzzy partitioned automaton to examine a fuzzy partitioned discrete-event system and associated decentralized supervisory control theory. We will specifically look at the notion of a fuzzy partitioned automaton and how it may be used to simulate the aforementioned system. In addition, we investigate the decentralized fuzzy supervisory control theory for a fuzzy partitioned discrete-event system and look at the conditions for its existence.

Supervisory Control for Dynamic Feature Configuration in Product Lines

In this paper a framework for engineering supervisory controllers for product lines with dynamic feature configuration is proposed. The variability in valid configurations is described by a feature model. Behavior of system components is achieved using (extended) finite automata and both behavioral and dynamic configuration constraints are expressed by means of requirements as is common in supervisory control theory. Supervisory controller synthesis is applied to compute a behavioral model in which the requirements are adhered to. For the challenges that arise in this setting, multiple solutions are discussed. The solutions are exemplified in the CIF toolset using a model of a coffee machine. A use case of the much larger Body Comfort System product line is performed to showcase feasibility for industrial-sized systems.

Advancing orchestration synthesis for contract automata

Contract automata allow to formally define the behaviour of service contracts in terms of service offers and requests, some of which are moreover optional and some of which are necessary. A composition of contracts is said to be in agreement if all service requests are matched by corresponding offers. Whenever a composition of contracts is not in agreement, it can be refined to reach an agreement using the orchestration synthesis algorithm. This algorithm is a variant of the synthesis algorithm used in supervisory control theory and it is based on the fact that optional transitions are controllable, whereas necessary transitions are at most semi-controllable and cannot always be controlled. In this paper, we present advancements of the orchestration synthesis for contract automata. Notably, we identify the existing limits of the orchestration synthesis and propose a novel orchestration synthesis along with additional constructs to enhance the expressiveness and scalability of contract automata. The proposed advancements have been implemented and experimented on two case studies, one of which originates from the railway domain and the other is a card game.

On the application of the calculus of positively constructed formulas for the study of controlled discrete-event systems

The article is devoted to the development of an approach to solving the main problems of the theory of supervisory control of logical discrete-event systems (DES), based on their representation in the form of positively constructed formulas (PCF). We consider logical DESs in automata form, understood as generators of some regular languages. The PCF language is a complete first-order language, the formulas of which have a regular structure of alternating type quantifiers and do not contain a negation operator in the syntax. It was previously proven that any formula of the classical first-order predicate calculus can be represented as a PCF. PCFs have a visual tree representation and a natural question-and-answer procedure for searching for an inference using a single inference rule. It is shown how the PCF calculus, developed in the 1990s to solve some problems of control of dynamic systems, makes it possible to solve basic problems of the theory of supervisory control, such as checking the criteria for the existence of supervisory control, automatically modifying restrictions on the behavior of the controlled system, and implementing a supervisor. Due to some features of the PCF calculus, it is possible to use a non-monotonic inference. It is demonstrated how the presented PCF-based method allows for additional event processing during inference. The Bootfrost software system, or the so-called prover, designed to refute the obtained PCFs is also presented, and the features of its implementation are briefly described. As an illustrative example, we consider the problem of controlling an autonomous mobile robot.

On Active Learning for Supervisor Synthesis

Supervisory control theory provides an approach to synthesize supervisors for cyber-physical systems using a model of the uncontrolled plant and its specifications. These supervisors can help guarantee the correctness of the closed-loop controlled system. However, access to plant models is a bottleneck for many industries, as manually developing these models is an error-prone and time-consuming process. An approach to obtaining a supervisor in the absence of plant models would help industrial adoption of supervisory control techniques. This paper presents, an algorithm to learn a controllable supervisor in the absence of plant models. It does so by actively interacting with a simulation of the plant by means of queries. If the obtained supervisor is blocking, existing synthesis techniques are employed to prune the blocking supervisor and obtain the controllable and non-blocking supervisor. Additionally, this paper presents an approach to interface the with a PLC to learn supervisors in a virtual commissioning setting. This approach is demonstrated by learning a supervisor of the well-known example simulated in Xcelgo Experior and controlled using a PLC. interacts with the PLC and learns a controllable supervisor for the simulated system. Note to Practitioners—Ensuring the correctness of automated systems is crucial. Supervisory control theory proposes techniques to help build control solutions that have certain correctness guarantees. These techniques rely on a model of the system. However, such models are typically unavailable and hard to create. Active learning is a promising technique to learn models by interacting with the system to be learned. This paper aims to integrate active learning and supervisory control such that the manual step of creating models is no longer needed, thus, allowing the use of supervisory control techniques in the absence of models. The proposed approach is implemented in a tool and demonstrated using a case study.

Safe Performance of an Industrial Autonomous Ground Vehicle in the Supervisory Control Framework

A Cyberphysical system, being an autonomous guided vehicle (AGV) and having diverse applications such as thematic parks and product transfer in manufacturing units, is modeled and controlled. The models of all subsystems of the AGV are provided in discrete event systems (DES) form following the Ramadge–Wonham (R–W) framework. The safe performance of the AGV, being the desired behavior of the system, is presented in the form of desired rules and translated into a set of regular languages. Then, the regular languages are realized as supervisory automata in the framework of Supervisory Control Theory (SCT). To ease implementation and coordination of the control architecture, the supervisors are designed to be in two-state automata forms. The controllability of the regular languages, regarding the AGV, will be proved, using the physical realizability (PR) of the synchronous product of the automata of the system and the supervisors. Also, the nonblocking property of all the controlled automata will be proven to be satisfied. Simulation of the controlled AGV will validate the proposed method.

Assessment of Petri nets Regulation Control methodologies for automation systems

Industry 4.0 and smart manufacturing bring new interesting possibilities and challenges, a particular one is the large-scale automation with minimal set-up time and flexibility. To face this challenge, control approaches based on Discrete Event Systems (DES), such as Supervisory Control Theory (SCT), Generalized Mutual Exclusions Constraints (GMEC) and Petri net-based Regulation Control (RC), may provide convenient solutions; however, few works have been reported in the literature describing the application of these approaches. In this work, the RC approach based on Petri nets is applied to three case studies: (1) an electro-pneumatic cell controlled by a Programming Logic Controller device (PLC); (2) a simulated hydraulic process controlled by a PLC (a virtual commissioning scheme); (3) a simulated robotic manufacturing cell controlled by a software tool. In addition, an assessment of the existing methodologies in RC for modelling, specification, control synthesis and control implementation is conducted. The findings of this work provide three contributions: (1) a practical validation of the applied methodologies and the identification of required extensions; (2) a discussion about the validity of frequently considered assumptions and hypotheses in DES; (3) a reference for practitioners on the implementation of the RC approach, as well as its limitations and advantages.

A multi-objective approach for manufacturing systems with multiple production routes based on supervisory control theory and heuristic algorithms

A multi-objective approach for manufacturing systems with multiple production routes based on supervisory control theory and heuristic algorithms

Leader–follower control of multi-unmanned aerial vehicle based on supervisory control theory for a broad tributary area mapping scenario

This study proposes a discrete event system-based control strategy for autonomous tributary mapping using multi-unmanned aerial vehicle. When considering the unmanned aerial vehicles as discrete event systems, supervisory control theory is used to model and control individual unmanned aerial vehicle behavior in the system. In tributary mapping, the situation changes each time depending on environmental factors (e.g. weather) and the work must be performed in an unstructured environment. Therefore, this article proposes a multi-unmanned aerial vehicle-based supervisory control system to solve real-field problems. Unlike the control systems of recent studies, which mainly deal with continuous-time dynamics, we modeled a multi-unmanned aerial vehicle system based on a discrete event system in which the dynamic states are transitioned by asynchronous events. The proposed multi-unmanned aerial vehicle-based supervisory control system was validated in dynamic simulators and demonstrated that multi-unmanned aerial vehicle satisfies the behavior specifications. The supervisor proposed in this study was validated using a physics-based simulator.

Asynchronous Diagnosability Enforcement in Discrete Event Systems Based on Supervisory Control

An asynchronous fault diagnosis problem consists in determining the occurrence of faults in a plant under the condition that a diagnosis agent, i.e., a diagnoser, is activated asynchronously with the plant. Asynchronous diagnosability is a property implying that any fault in a plant can be detected by observing a finite number of observations in the case of asynchronous activation of the diagnoser and the plant. This article studies the problem of asynchronous diagnosability enforcement in discrete event systems based on supervisory control theory, i.e., to develop a supervisor for an asynchronously undiagnosable plant such that the asynchronous diagnosability of the closed-loop system is guaranteed. Firstly, the classical definition of asynchronous diagnosability is generalized to nonlive systems since a supervisor may introduce deadlocks in a plant even if it is originally live. We then propose a structure called an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">asynchronous-quiescent diagnoser</i> that is used for both online asynchronous diagnosis and asynchronous diagnosability determination. Finally, for a plant that is asynchronously undiagnosable, we develop a supervisor to enforce the asynchronous diagnosability based on its asynchronous-quiescent diagnoser.

A Hybrid Systems-Based Hierarchical Control Architecture for Heterogeneous Field Robot Teams.

Field robot systems have recently been applied in a wide range of research fields. Further automation, development, and activation of such systems require cooperation among heterogeneous robots. Classical control theory is inefficient in managing large-scale complex dynamic systems. Therefore, a discrete-event system based on the supervisory control theory must be introduced to overcome this limitation. In this article, we propose a hybrid system-based hierarchical control architecture using a supervisory control-based high-level controller and a traditional control-based low-level controller. The hybrid system and its dynamics are modeled through a formal method, called hybrid automata, and the behavior specifications are designed to express the control objectives for cooperation. In addition, modular supervisors that are more scalable and maintainable than a centralized supervisory controller were synthesized. The proposed hybrid system and hierarchical control architecture were implemented, validated, and evaluated for performance through a physics-based simulator and field tests. The experimental results confirmed that the robot team satisfied the given specifications and presented systematic results, validating the efficiency of the proposed control architecture.

A model-based deep reinforcement learning approach to the nonblocking coordination of modular supervisors of discrete event systems

Modular supervisory control may lead to conflicts among the modular supervisors for large-scale discrete event systems. The existing methods for ensuring nonblocking control of modular supervisors either exploit favorable structures in the system model to guarantee the nonblocking property of modular supervisors or employ hierarchical model abstraction methods for reducing the computational complexity of designing a nonblocking coordinator. The nonblocking modular control problem is, in general, NP-hard. This study integrates supervisory control theory and a model-based deep reinforcement learning method to synthesize a nonblocking coordinator for the modular supervisors. The deep reinforcement learning method significantly reduces the computational complexity by avoiding the computation of synchronization of multiple modular supervisors and the plant models. The supervisory control function is approximated by the deep neural network instead of a large-sized finite automaton. Furthermore, the proposed model-based deep reinforcement learning method is more efficient than the standard deep Q network algorithm.

Multi-robots Coordination System for Urban Search and Rescue Assistance Based on Supervisory Control Theory

Multi-robots Coordination System for Urban Search and Rescue Assistance Based on Supervisory Control Theory

Synthesis-based engineering of supervisory controllers for ROS-based applications

An ever-increasing number of robotic applications is being created based on the functionalities and the communication facilities offered by the Robotic Operating System (ROS). When applications involve contact with humans, such as in some transportation tasks, a robot supervisory controller shall guarantee that in every possible circumstance safety and task requirements are satisfied. There is no framework provided by ROS to specify supervisory controllers with formal guarantees about the satisfaction of requirements. This is problematic when the complexity of the controller increases, and it is not viable to test every possible scenario during the validation stage of the robotic application. This paper presents novel work that aims at enabling supervisory controller synthesis for ROS-based applications by (a) proposing a novel mapping between concepts from supervisory control theory and ROS, (b) demonstrating that a supervisory control modeling language exists that can be used to describe plant models and to specify requirements and (c) presenting generic finite-state automata models of ROS components, regarded as plant models, which are input for the synthesis procedure. The approach is demonstrated by synthesizing the controller to coordinate state-of-the-art mobile robot navigation modules. Simulations and real-life experiments show that the generated supervisory controller guarantees the correctness of the model with respect to the requirements and that the execution time of a control loop, including the binding code between the supervisory controller and ROS, is on average short enough to be deployed in high-level control loops. The paper is accompanied by a public code repository that contains the implementation of all methods presented in the paper including simulation environments to ease the replication of the results presented.

Supervisor Synthesis for Multi-Agent Systems With Mutually Exclusive Resources

In multi-agent systems (MAS), it is a basic mission to coordinate and control each agent to complete tasks on mutually exclusive resources due to the high cost of some hardware and software in reality. In supervisory control theory (SCT) of discrete-event systems (DES), a supremal supervisor is synthesized with respect to the system and all the specifications to make the system act as desired. However, the size of the plant model increases exponentially with the number of agents in MAS. To alleviate the state explosion problem when applying SCT to MAS, a synchronous product with constraints using bread-first search strategy is proposed to synthesize a supervisor for the system with respect to the mutex resources. And a framework for computing the supremal supervisor for the entire MAS with respect to all the specifications is presented. Experiments are studied to estimate the effectiveness of the proposed approach. Contributions of the research are two-fold. Firstly, the state explosion problem associated with supervisory control of MAS is alleviated in a certain extent. Secondly, compared with decentralized or distributed control, it is not necessary to design a coordinator additionally.

MDESops: An Open-Source Software Tool for Discrete Event Systems Modeled by Automata

The salient features of the new software tool MDESops are presented. MDESops is Python-based and open-source. Its focus is the analysis and control of discrete event systems (DES) modeled by finite-state automata. MDESops has core functions to manipulate deterministic and nondeterministic automata, including parallel composition and determinization. It has functions to analyze diagnosability and opacity properties. MDESops also includes functions that implement both standard and more recent algorithmic procedures from the theory of supervisory control of DES, including synthesis of supervisors for partially-observed systems, synthesis of attackers in systems with compromised sensors, and synthesis of supervisors resilient to deception attacks. It is the hope that MDESops can serve as a useful platform for algorithm development and prototyping in DES research.

Optimal Secret Protection in Discrete Event Systems with Dynamic Clearance Levels

In this paper we propose a general framework to design optimal secret protection policies in discrete event systems. In the system, some secret states are associated with confidentiality levels (possibly different), and our purpose is to design an event-protecting policy such that any user (legal or unauthorized) who visits a secret state must have a security clearance level no less than the required confidentiality level. We consider the criteria of optimality on protecting policies as to protecting policies with a minimum degree of disturbance to legal users’ normal operations. We develop an auxiliary data structure called the generalized secret automaton, based on which we propose a method to design a protecting policy using the classical supervisory control theory. The minimally disruptive protecting policy is then represented by an automaton called the secret enforcer whose state size is polynomial both in the number of the plant states and the number of secret states in the plant.