Control Of Discrete Event Systems Research Articles

Optimal control of discrete event systems under uncertain environment based on supervisory control theory and reinforcement learning

Discrete event systems (DESs) are powerful abstract representations for large human-made physical systems in a wide variety of industries. Safety control issues on DESs have been extensively studied based on the logical specifications of the systems in various literature. However, when facing the DESs under uncertain environment which brings into the implicit specifications, the classical supervisory control approach may not be capable of achieving the performance. So in this research, we propose a new approach for optimal control of DESs under uncertain environment based on supervisory control theory (SCT) and reinforcement learning (RL). Firstly, we use SCT to gather deliberative planning algorithms with the aim to safe control. Then we convert the supervised system to Markov Decision Process simulation environments that is suitable for optimal algorithm training. Furthermore, a SCT-based RL algorithm is designed to maximize performance of the system based on the probabilistic attributes of the state transitions. Finally, a case study on the autonomous navigation task of a delivery robot is provided to corroborate the proposed method by multiple simulation experiments. The result shows the proposed approach owning 8.27%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} performance improvement compared with the non-intelligent methods. This research will contribute to further studying the optimal control of human-made physical systems in a wide variety of industries.

Integrating reinforcement learning and supervisory control theory for optimal directed control of discrete-event systems

Directed control is crucial for implementing controllers on programmable logic controllers. A directed controller is one that actively selects at most one controllable event (control command) to execute at any instant. This study investigates the numerical optimization problem of directed control for discrete-event systems, such as traffic systems and robotic systems. By integrating supervisory control theory and reinforcement learning, an algorithmic procedure is designed to synthesize an optimal directed controller, which minimizes the total cost associated with event execution while ensuring the safety and liveness of the controlled system. Two improved Q-learning algorithms incorporating dynamic parameter adaptation strategies are developed to enhance the global search ability of basic Q-learning. To demonstrate applicability and effectiveness, we apply the proposed method to control a guideway system and a multi-train traffic system, respectively. The experimental results indicate the superiority of the proposed method over the comparison methods.

Quantitatively nonblocking supervisory control of discrete-event systems

In this paper, we propose a new property of quantitative nonblockingness of an automaton with respect to a given cover on its set of marker states. This property quantifies the standard nonblocking property by capturing the practical requirement that every subset (i.e. cell) of marker states can be reached within a prescribed number of steps from any reachable state and following any trajectory of the system. Accordingly, we formulate a new problem of quantitatively nonblocking supervisory control, and characterize its solvability in terms of a new concept of quantitative language completability. It is proven that there exists the unique supremal quantitatively completable sublanguage of a given language, and we develop an effective algorithm to compute the supremal sublanguage. Finally, combining with the algorithm of computing the supremal controllable sublanguage, we design an algorithm to compute the maximally permissive solution to the formulated quantitatively nonblocking supervisory control problem.

Model Sharing and Scalability in the Real-Time Simulation and Intelligent Hierarchical Control of Discrete-Event Systems

Model Sharing and Scalability in the Real-Time Simulation and Intelligent Hierarchical Control of Discrete-Event Systems

Fault Prognosis with Supervisory Control of Discrete-Event Systems under Non-deterministic Observations

Fault Prognosis with Supervisory Control of Discrete-Event Systems under Non-deterministic Observations

Event-Based Modeling of Input Signal Behaviors for Discrete-Event Controllers

Controllers for discrete-event systems are commonly designed using state-based formalisms, like state diagrams and Petri nets. These formalisms are strongly supported by the concept of events, which, from an automation system perspective, can be associated with a simple change in the value of a signal or more complex behavioral evolutions of the signals. In this paper, the characterization of several types of events is proposed, associated with different types of signals, such as Boolean and multivalued signals. The major goal of this characterization is to improve the compactness of the model, benefiting the editing and visual interpretation of the graphical model but keeping precise execution semantics, which in turn allows for the use of computational tools covering the different stages of system development. The behavioral model of the controller is produced using a non-autonomous class of Petri nets, the IOPT nets, and the associated IOPT-Tools, which supports the specification, simulation, property verification, and automatic code generation ready to be deployed into implementation platforms. All the types of proposed events have a behavioral sub-model executed concurrently with the main model of the controller. An application example is provided to illustrate some of the advantages of the adoption of the proposed approach, encapsulating the behavioral dependencies on the evolution of input signals into events.

Modeling and Control of Discrete-Event Systems Under Joint Sensor–Actuator Cyberattacks

Modeling and Control of Discrete-Event Systems Under Joint Sensor–Actuator Cyberattacks

Supervisory Control of Time-Interval Discrete Event Systems

Supervisory Control of Time-Interval Discrete Event Systems

Control of Discrete Event Systems by Using Symbolic Transition Model: An Application to Power Grids

AbstractIn this paper, a new symbolic modeling framework is proposed for the control of discrete event systems (discrete controller synthesis). Reactive infinite-state systems are generally dynamic systems with inputs and outputs. Modeling the outputs in existing symbolic modeling methods for reactive infinite-state systems has not been extensively discussed in the literature. It is crucial to realistically model the outputs and obtain reliable controllers. One of the main contributions and innovations in this study is the ability to symbolically model their output. In this direction, symbolic transition models are offered, where events are defined as guarded symbolic transitions. The framework considers input/output reactive infinite-state systems to satisfy safety and optimization objectives. Effective algorithms and safety control policies are offered to solve the safety control problem (i.e., satisfy the safety objective) for input/output infinite-state systems. Another contribution in this work provides optimization targets within a given cost function based on state transitions, without specifying the target states. The framework is then applied to power grids to demonstrate the effectiveness of the symbolic transition models and experimentally evaluate the new control algorithms. The results showed that the approach is very effective compared to other approaches found in the literature. The computation times required to achieve both safety and optimization objectives are obtained relatively quickly compared to other deterministic methods found in the literature.

Information control in networked discrete event systems

How to control information exchange among different users is an important problem in networked systems with many users/agents. Generally speaking, there are several considerations in control of information exchange in a networked system, including (1) to ensure a friend user has sufficient information to perform its tasks, (2) to deprive an adversary user its information to perform its tasks, (3) to minimize information exchange among friend users so that the risk of information leaking is minimized, and (4) to maximize information broadcasted to all users to achieve maximum transparency. In this paper, we investigate the information control problems in the framework of discrete event systems. Based on the problem at hand, we divide users in a networked system into two or more groups. Users in the same group are consider as friends and users in a different group are consider as adversaries. Several information control problems are investigated and solved using a systematic and rigorous approach. Methods are developed to design controllers that send minimum information to its friends to help them to perform their tasks and broadcast maximum information without helping its adversaries.

Optimal supervisory control of discrete event systems for cyclic tasks

In this paper, we investigate the problem of optimal supervisory control for cyclic tasks in the context of discrete-event systems (DES). We consider the completion of each single task as the visit of a marked state, and overall control objective is to complete tasks cyclically in the sense that marked states are visited infinitely often. Following the standard optimal supervisory control framework, two types of costs, disable cost and occurrence cost, are considered. However, instead of considering the standard accumulated total cost or the average cost per event, we consider the measure for the control performance using the average cost per task. We show that such an optimality measure is more suitable for tasks that need to be completed cyclically. Our goal is to design a live and non-blocking supervisor such that the average cost per task in the worst-case is minimized. To solve the problem, we propose a game-theoretical approach by converting the optimal control problem as a two-player graph game. Structural properties of the converted game are discussed. In particular, we show that this game can be solved by a set of mean payoff decision problems, for which effective algorithms exist. Our problem can be considered as a special instance of the general ratio-game in the literature. However, by exploring new structural property for this problem, we achieve superior computational efficiency when compared to the conventional solution designed for more general problem formulations. Illustrative examples are provided to demonstrate the proposed algorithm.

Optimal Opacity-Enforcing Supervisory Control of Discrete Event Systems on Choosing Cost

To ensure opacity, it is optimal to retain as many as possible occurring event sequences. Contrary to this problem, the other optimal goal is to preserve the minimal occurring event sequences. Based on the choosing cost, an optimal opacity-enforcing problem with minimal discount choosing cost is presented under two constraints in this paper. The first constraint is the opacity of the controlled system. The second is the retention of the secret to the maximum. To solve the model, two scenarios on opacity are considered. For the two scenarios, some algorithms are presented to achieve the optimal solution for the model by using the method of dynamic programming. Then, the solutions produced by the algorithms are proved to be correct by theoretical proof. Finally, some illustrations and an application example on location privacy protection for the algorithms are given.

Supervisory Control of Discrete Event Systems With Dynamic Control Specifications by Petri Nets

Supervisory Control of Discrete Event Systems With Dynamic Control Specifications by Petri Nets

Supervisory control of quantitative Petri nets for fixed‐initial‐credit energy problems using a game structure

AbstractThis work investigates quantitative supervisory control of discrete event systems modeled with Petri nets under the fixed‐initial‐credit energy objective. A weight function referred to as an energy function is defined on a Petri net to characterize the energy level of a transition. The proposed fixed‐initial‐credit energy problem aims to design a supervisor such that the energy level of a transition sequence in a supervised system is higher than 0 under a given initial energy level. The problem is eventually transformed into a two‐player game between a system and a supervisor; supervisor synthesis is reduced to finding a winning strategy in the two‐player game. Instead of enumerating the complete state space of the underlying Petri net, two information structures are utilized, namely the conventional basis reachability graph and the newly proposed essential marking graph, to construct two‐player games based on each of them. It is shown that a winning strategy for a supervisor decoded from the game based on the basis reachability graph of the Petri net is a solution to the problem but is in general restrictive. Further, it is shown that the set of strategies for a supervisor in the game based on the essential marking graph is consistent with that from the game based on the reachability graph of a Petri net. The two developed approaches do not require an exhaustive exploration of the state space of a plant, thus achieving higher efficiency.

A General Architecture for Intersection-Based Decentralized Supervisory Control of Discrete Event Systems

A General Architecture for Intersection-Based Decentralized Supervisory Control of Discrete Event Systems

Hierarchical Supervisory Control Under Partial Observation: Normality

Conditions preserving observability of specifications between the plant and its abstraction are essential for hierarchical supervisory control of discrete-event systems under partial observation. Observation consistency and local observation consistency were identified as such conditions. To preserve normality, only observation consistency is required. Although observation consistency preserves normality between the levels for normal specifications, for specifications that are not normal, observation consistency is insufficient to guarantee that the supremal normal sublanguage computed on the low level and on the high level coincide. We define modified observation consistency, under which the supremal normal sublanguages of different levels coincide. We show that the verification of (modified) observation consistency is <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PSpace</small> -hard for finite automata and undecidable for slightly more expressive models than finite automata. Decidability of (modified) observation consistency is an open problem. Hence we further discuss two stronger conditions that are easy to verify. Finally, we illustrate the conditions on an example of a railroad controller and on a case study of a part of an MRI scanner.

Sensor deception attacks against security in supervisory control systems

This paper investigates the problem of synthesizing sensor deception attacks against security in the context of supervisory control of discrete-event systems (DES). We consider a DES plant controlled by a supervisor, whose security requirement is to maintain the initial-secret of the system undetected, i.e., it does not want to reveal the fact that it was initiated from a secret state. On the other hand, there exists an active attacker that can tamper with the observations received by the supervisor by, e.g., hacking into the communication channel between the sensors and the supervisor. The objective of the attacker is to deceive the supervisor such that the initial-secret is revealed due to incorrect control actions. We investigate the problem from the attacker’s point of view and focus on synthesizing attack strategies that threaten the security of the system. We consider two levels of success of the attacker: one requires that the attacker can detect the initial-secret of the system “almost surely” and the other only requires that the attacker has the possibility to detect the initial-secret of the system. For both cases, we present algorithms for synthesizing successful attack strategies. Our approach is based on the All Attack Structure (AAS) which records state estimates for both the supervisor and the attacker. Structural properties of the security requirements are also leveraged to reduce the synthesis complexity. A running academic example is provided to illustrate the proposed synthesis procedures.

Smart Platform for Monitoring and Control of Discrete Event System in Industry 4.0 Concept

We are now living in a time when the fourth industrial revolution is bringing new technologies with intensive digitalization of all levels of production. This paper presents a complex solution for the monitoring, diagnosis, and control of the production process, in our case, discrete event systems. The proposed solution—a mechatronic platform—further develops and extends our recent results where the overall concept was outlined and some partial tasks were studied. The present paper provides the complete structure of the proposed platform together with implementation details for all functionalities. The developed platform uses the latest technologies, such as OPC UA; industrial communication and visualization protocols and tools; and augmented reality, which enables comfortable interaction with real processes. The laboratory-scaled case study shows the qualities of the proposed solution.

A New Approach for Verification of Delay Co-observability of Discrete-Event Systems

In decentralized networked supervisory control of discrete-event systems (DESs), the local supervisors observe event occurrences subject to observation delays to make correct control decisions. Delay coobservability describes whether these local supervisors can make sufficient observations. In this paper, we provide an efficient way to verify delay coobservability. For each controllable event, we partition the specification language into a finite number of sets such that strings in different sets have different lengths. For each of the sets, we construct a verifier to check if delay coobservability holds for the controllable event. The computational complexity of the proposed approach is polynomial with respect to the number of states, the number of events, and the upper bounds on observation delays and only exponential with respect to the number of local supervisors. It has lower complexity order than the existing approaches. In addition, we investigate the relationship between the decentralized supervisory control of networked DESs and the decentralized fault diagnosis of networked DESs and show that delay <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> -codiagnosability is transformable to delay coobservability. Thus, techniques for the verification of delay coobservability can be leveraged to verify delay <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$K$</tex-math></inline-formula> -codiagnosability.

Supervisory Control of Fuzzy Discrete Event Systems Under Partial Observation

Supervisory control of fuzzy discrete event systems under partial observation is investigated in the paper. Without loss of generality, we consider fuzzy discrete event systems with constraints where a fuzzy discrete event system is modeled by a fuzzy automaton. Sequences of events that can be generated by the system are regarded as constraints and are modeled by a crisp automaton. A supervisor is designed to control the fuzzy discrete event system so that the supervised system is prevented from entering a pre-specified set of illegal/unsafe fuzzy states. A necessary and sufficient condition for the existence of a supervisor is obtained. When the condition is satisfied, an online supervisor can be designed. Fuzzy state estimation problem is first solved, as the supervisor is fuzzy-state-estimate-based. A method is developed to estimate fuzzy state iteratively after observation of each new event. We show that the supervisor so developed ensures the safety of the system and is least restrictive among all possible safe supervisors. Potential of the theoretical results for real-world applications is illustrated through an example of HIV/AIDS treatment decision-making.