Optimal Control Of Systems Research Articles

Optimal control of discrete event systems under uncertain environment based on supervisory control theory and reinforcement learning

Discrete event systems (DESs) are powerful abstract representations for large human-made physical systems in a wide variety of industries. Safety control issues on DESs have been extensively studied based on the logical specifications of the systems in various literature. However, when facing the DESs under uncertain environment which brings into the implicit specifications, the classical supervisory control approach may not be capable of achieving the performance. So in this research, we propose a new approach for optimal control of DESs under uncertain environment based on supervisory control theory (SCT) and reinforcement learning (RL). Firstly, we use SCT to gather deliberative planning algorithms with the aim to safe control. Then we convert the supervised system to Markov Decision Process simulation environments that is suitable for optimal algorithm training. Furthermore, a SCT-based RL algorithm is designed to maximize performance of the system based on the probabilistic attributes of the state transitions. Finally, a case study on the autonomous navigation task of a delivery robot is provided to corroborate the proposed method by multiple simulation experiments. The result shows the proposed approach owning 8.27%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} performance improvement compared with the non-intelligent methods. This research will contribute to further studying the optimal control of human-made physical systems in a wide variety of industries.

Numerical optimal control for nonlinear dynamical systems involving mixed-valued inputs and joint probability path-constraints

Numerical optimal control for nonlinear dynamical systems involving mixed-valued inputs and joint probability path-constraints

Integrating reinforcement learning and supervisory control theory for optimal directed control of discrete-event systems

Directed control is crucial for implementing controllers on programmable logic controllers. A directed controller is one that actively selects at most one controllable event (control command) to execute at any instant. This study investigates the numerical optimization problem of directed control for discrete-event systems, such as traffic systems and robotic systems. By integrating supervisory control theory and reinforcement learning, an algorithmic procedure is designed to synthesize an optimal directed controller, which minimizes the total cost associated with event execution while ensuring the safety and liveness of the controlled system. Two improved Q-learning algorithms incorporating dynamic parameter adaptation strategies are developed to enhance the global search ability of basic Q-learning. To demonstrate applicability and effectiveness, we apply the proposed method to control a guideway system and a multi-train traffic system, respectively. The experimental results indicate the superiority of the proposed method over the comparison methods.

VAN-DAMME: GPU-accelerated and symmetry-assisted quantum optimal control of multi-qubit systems

We present an open-source software package, VAN-DAMME (Versatile Approaches to Numerically Design, Accelerate, and Manipulate Magnetic Excitations), for massively-parallelized quantum optimal control (QOC) calculations of multi-qubit systems. To enable large QOC calculations, the VAN-DAMME software package utilizes symmetry-based techniques with custom GPU-enhanced algorithms. This combined approach allows for the simultaneous computation of hundreds of matrix exponential propagators that efficiently leverage the intra-GPU parallelism found in high-performance GPUs. In addition, to maximize the computational efficiency of the VAN-DAMME code, we carried out several extensive tests on data layout, computational complexity, memory requirements, and performance. These extensive analyses allowed us to develop computationally efficient approaches for evaluating complex-valued matrix exponential propagators based on Padé approximants. To assess the computational performance of our GPU-accelerated VAN-DAMME code, we carried out QOC calculations of systems containing 10 - 15 qubits, which showed that our GPU implementation is 18.4× faster than the corresponding CPU implementation. Our GPU-accelerated enhancements allow efficient calculations of multi-qubit systems, which can be used for the efficient implementation of QOC applications across multiple domains. Program summaryProgram Title: VAN-DAMMECPC Library link to program files::https://doi.org/10.17632/zcgw2n5bjf.1Licensing provisions: GNU General Public License 3Programming language: C++ and CUDANature of problem: The VAN-DAMME software package utilizes GPU-accelerated routines and new algorithmic improvements to compute optimized time-dependent magnetic fields that can drive a system from a known initial qubit configuration to a specified target state with a large (≈1) transition probability.Solution method: Quantum control, GPU acceleration, analytic gradients, matrix exponential, and gradient ascent optimization.

Analysis of a Dry Friction Force Law for the Covariant Optimal Control of Mechanical Systems with Revolute Joints

This contribution shows a geometric optimal control procedure to solve the trajectory generation problem for the navigation (generic motion) of mechanical systems with revolute joints. The mechanical system is analyzed as a nonlinear Lagrangian system affected by dry friction at the joint level. Rayleigh’s dissipation function is used to model this dissipative effect of joint-level friction, and regarded as a potential. Rayleigh’s potential is an invariant scalar quantity from which friction forces derive and are represented by a smooth model that approaches the traditional Coulomb’s law in our proposal. For the optimal control procedure, an invariant cost function is formed with the motion equations and a Riemannian metric. The goal is to minimize the consumed energy per unit time of the system. Covariant control equations are obtained by applying Pontryagin’s principle, and time-integrated using a Finite Elements Method-based solver. The obtained solution is an optimal trajectory that is then applied to a mechanical system using a proportional–derivative plus feedforward controller to guarantee the trajectory tracking control problem. Simulations and experiments confirm that including joint-level friction forces at the modeling stage of the optimal control procedure increases performance, compared with scenarios where the friction is not taken into account, or when friction compensation is performed at the feedback level during motion control.

ADP-based adaptive control of ship course tracking system with prescribed performance

In this paper, an adaptive dynamic programing (ADP)-based adaptive control problem is studied for the ship course tracking system with prescribed performance. Firstly, the complex sea conditions will bring difficulty to ship course tracking when unmanned ship sailing on sea. In order to overcome this difficulty, a prescribed performance feedforward term is designed in the controller to improve the course tracking ability of ship. Secondly, by defining a cost function, an ADP technology is exploited to achieve an optimal control of ship course tracking system. Finally, the stability of the closed loop system is turned out by Lyapunov stability theory and all signals are uniformly ultimately bounded (UUB). Simulation results validate the effectiveness of the proposed method.

Real-time optimal hierarchy control of HVAC system with maximized ancillary service support in smart buildings: An experimental approach

This article proposes a real-time optimum control algorithm for Heating, Ventilation, and Air Conditioning (HVAC) units of a commercial building aimed to maximize the ancillary power available for frequency regulation services. First, the prediction of energy consumption of air conditioning zones using long short-term memory (LSTM) is achieved, which helps forecast the aggregated regulation capacity bid of the building in the energy markets. Furthermore, to minimize energy costs and thermal discomfort of building dwellers, an optimization algorithm is designed considering the day-ahead electricity tariffs. Moreover, using installed sensors in a lab-scaled hardware prototype, the proposed algorithm is also shown to be capable of interacting with a Building Energy Management System (BEMS) and investigating the electrical and thermal properties of the HVAC unit. The BEMS controller uses an ethernet protocol to interface with the sensors installed at various operational points throughout the HVAC system. The optimal operation of the HVAC system has been executed with the real-time thermal feedback system, which counterbalances the uncertain thermal load or renewable power availability in the building confronted in real-time operation. The simulation results show the performance of the Artificial Neural Network (ANN) based compressor model in deep learning integrated with the other components of the HVAC unit and thermal model of the air-conditioning zone. Finally, the experimental results display the real-time implementation of multiple thermal and electrical conditions within the proposed control scheme of the HVAC unit. This shows the potential of ancillary services capacity through commercial buildings can be maximized with flexible loads and thereby help in power grid support.

Machine learning model‐based optimal tracking control of nonlinear affine systems with safety constraints

AbstractThis work focuses on the development of a machine learning (ML) model‐based framework for safe optimal tracking control of a class of nonlinear control‐affine systems to ensure simultaneous closed‐loop stability and safety. Specifically, a novel multilayer feedforward neural network (FNN) with a control‐affine architecture is designed to model nonlinear dynamic systems. Subsequently, a model‐based reinforcement learning (RL) framework is presented, utilizing a novel cost function with Control Lyapunov‐Barrier Function (CLBF) properties, to learn both the control policy and the optimal value function for an infinite‐horizon optimal tracking control problem for nonlinear systems with safety constraints. The efficacy of the proposed methodology is demonstrated through simulations of a one‐link robot manipulator and a chemical process example.

Knowledge-Data Driven Optimal Control for Nonlinear Systems and Its Application to Wastewater Treatment Process.

Optimal control is developed to guarantee nonlinear systems run in an optimum operating state. However, since the operation demands of systems are dynamically changeable, it is difficult for optimal control to obtain reliable optimal solutions to achieve satisfying operation performance. To overcome this problem, a knowledge-data driven optimal control (KDDOC) for nonlinear systems is designed in this article. First, an adaptive initialization strategy, using the knowledge from historical operation information of nonlinear systems, is employed to dynamically preset parameters of KDDOC. Then, the initial performance of KDDOC can be enhanced for nonlinear systems. Second, a knowledge guide-based global best selection mechanism is used to assist KDDOC in searching for the optimal solutions under different operation demands. Then, dynamic optimal solutions of KDDOC can be obtained to adapt to flexible changes in nonlinear systems. Third, a knowledge direct-based exploitation mechanism is presented to accelerate the solving process of KDDOC. Then, the demand response speed of KDDOC can be improved to ensure nonlinear systems with optimal operation performance in different states. Finally, the performance of KDDOC is validated on a simulation and a practical process. Several experimental results illustrate the effectiveness of the proposed optimal control for nonlinear systems.

Optimal Consensus Control for Continuous-Time Linear Multiagent Systems: A Dynamic Event-Triggered Approach.

This article investigates the optimal consensus problem for general linear multiagent systems (MASs) via a dynamic event-triggered approach. First, a modified interaction-related cost function is proposed. Second, a dynamic event-triggered approach is developed by constructing a new distributed dynamic triggering function and a new distributed event-triggered consensus protocol. Consequently, the modified interaction-related cost function can be minimized by applying the distributed control laws, which overcomes the difficulty in the optimal consensus problem that seeking the interaction-related cost function needs all agents' information. Then, some sufficient conditions are obtained to guarantee optimality. It is shown that the developed optimal consensus gain matrices are only related to the designed triggering parameters and the desirable modified interaction-related cost function, relaxing the constraint that the controller design requires the knowledge of system dynamics, initial states, and network scale. Meanwhile, the tradeoff between optimal consensus performance and event-triggered behavior is also considered. Finally, a simulation example is provided to verify the validity of the designed distributed event-triggered optimal controller.

Optimal control of automatic voltage regulator system using hybrid PSO-GWO algorithm-based PID controller

Optimal control of automatic voltage regulator system using hybrid PSO-GWO algorithm-based PID controller

Constrained Reinforcement Learning-Based Closed-Loop Reference Model for Optimal Tracking Control of Unknown Continuous-Time Systems

Constrained Reinforcement Learning-Based Closed-Loop Reference Model for Optimal Tracking Control of Unknown Continuous-Time Systems

Optimal Bipartite Tracking Control for Heterogeneous Systems Under DoS Attacks

Optimal Bipartite Tracking Control for Heterogeneous Systems Under DoS Attacks

Optimal and Robust Load Frequency Control for Hybrid Power System Integrated with Energy Storage Device by Sine Cosine Algorithm

Optimal and Robust Load Frequency Control for Hybrid Power System Integrated with Energy Storage Device by Sine Cosine Algorithm

Development of optimal energy management strategy for proton exchange membrane fuel cell-battery hybrid system for drone propulsion

Drones powered by fuel cells have been developed to ensure long flight distances. Due to the low dynamic responsiveness of proton exchange membrane fuel cell (PEMFC) systems, they must be integrated with batteries to fully meet the power demand profiles of drones. In this study, a dynamic model of a PEMFC-battery hybrid system for drone propulsion is developed in MATLAB-Simulink® to establish the optimal energy management strategy (EMS) for ensuring efficient and safe operation. The proposed PEMFC system comprises two modules of an open-cathode PEMFC stack, which are connected in parallel. To estimate the dynamic behavior and performance of PEMFC, one-dimensional dynamic model of PEMFC considering two-phase transport through the gas diffusion layer is developed and simulated. A zero-dimensional dynamic model of the battery was also developed based on an equivalent circuit model. The resulting PEMFC-battery hybrid system model was simulated to determine the power required for the propulsion of the drone according to a flight scenario comprising five phases: takeoff hover, climb, cruise, descent, and landing hover. Three different EMSs are developed and simulated to define the optimal one for the PEMFC-battery hybrid system by comparing the total hydrogen consumption and the final SOC of the battery As a result, the EMS that equally splits the required power between two PEMFCs exhibits the lowest hydrogen consumption during the flight simulation. This study helps provide the optimal system design and control scheme of drones powered by fuel cells.

Optimal Control of a Roll-to-Roll Dry Transfer Process With Bounded Dynamics Convexification

Abstract For efficient roll-to-roll (R2R) production of flexible electronic components, a precise R2R transfer peeling process is essential, requiring accurate modeling and control. This paper introduces a novel approach to confining the dynamics of a nonlinear R2R mechanical peeling system within a convex set known as a norm-bounded linear differential inclusion (NLDI). This method utilizes constraints on uncertain system variables to create a tighter NLDI representation compared to other convexification techniques. Moreover, it offers drastically reduced computational cost compared to previous methods applied to convexify the R2R peeling system. The NLDI is employed to generate an H∞-optimal controller for the R2R peeling system, and both simulations and experiments demonstrate better dynamic performance compared to other controllers for R2R transfer.

Hybrid model for robust and accurate forecasting building electricity demand combining physical and data-driven methods

The high-precision prediction of building electrical demand is paramount significance for optimal control of building systems and the reduction of operational energy consumption. The pure physical simulation software struggles to depict buildings' energy usage behavior accurately; while purely data-driven methods lack the interpretability in real-world physical scenarios. To address the challenges existing in current models and algorithms, this paper proposes a hybrid model that combines physical mechanism simulation and machine learning to complement each other. Empirical Mode Decomposition (EMD) is utilized to decompose complex data into simple components in pre-processing (RMSE, MAE, and SMAPE all decreased by around 0.2–0.3). And K-means clustering algorithm is used to extract features of building electrical data and determine employee working states (R2 increased from 0.3 to 0.85). Shapley Additive explanations (SHAP) values and Pearson correlation coefficients are used for model interpretability analysis. Combining weather forecasts, the Monte Carlo method is employed to obtain prediction intervals. Furthermore, this study explores the use of transfer learning to save computational costs and reduce the amount of data needed for predicting, especially for high-rise buildings or floors with insufficient historical data. The hybrid model results indicate that the R2 can exceed 0.85, up to 0.9, while pure physical simulation can only achieve 0.3. MAE, MSE, and RMSE remain below 0.07, while SMAPE stays around 0.2. The R2 value of the transfer learning model can surpass 0.8 (MSE, MAE. MAE, MSE, and RMSE remain below 0.2, while SMAPE stays around 0.4). The model's residuals obey normal distribution, indicating the model has strong generalization capabilities.

The maximum principle for lumped-distributed control systems.

This paper concerns the optimal control of lumped-distributed systems, an examplar of which is a mechanical system with rotating components connected by flexible rods. The underlying mathematical model is a controlled semilinear evolution equation, in which nonlinear terms involve a projection of the full state onto a finite dimensional subspace. We derive necessary conditions of optimality in the form of a maximum principle, for a problem formulation which involves pathwise and end-point constraints on the lumped components of the state variable. Perturbational methods are employed in the derivation, which allow for non-smooth data. A key step in the analysis is the reduction of the optimization problem with an infinite dimensional state space, to one with finite dimensional aspects. The computational implications of this reduction will be explored in future work. Necessary conditions for problems with state constraints and end-point constraints can only be achieved under rather strict compatibility hypotheses on the way the dynamics interact with the state constraint and end-point constraint sets, due the infinite dimensionality of the state space. This paper identifies a class of infinite dimensional problems, of engineering significance, for which necessary conditions of optimality can be derived, in the absence of any additional compatibility hypotheses.

Reinforcement learning intermittent optimal formation control for multi-agent systems with disturbances

Abstract This paper investigates disturbance-resistant intermittent event-triggered optimal formation control problems of second-order multi-agent systems by using the reinforcement learning method, which takes into account the influence of network damage including denial-of-service (DoS) and deception attacks, stochastic noises, and unknown external disturbances. Firstly, we propose a novel disturbance observer based on adaptive control to estimate unknown external disturbances under an event-triggered mechanism. Secondly, by use of estimation of disturbances, an innovative intermittent event-triggered optimal formation algorithm is given. By applying theories such as Lyapunov stability and stochastic stability, sufficient conditions are derived to guarantee that all agents achieve the desired formation in mean square sense. Additionally, in the model-free case, the optimal controller is solved using the least squares method, which is computationally less complex than some existing approaches. Finally, the theoretical results are effectively validated through simulation examples.

Chemical-Inspired Material Generation Algorithm (MGA) of Single- and Double-Diode Model Parameter Determination for Multi-Crystalline Silicon Solar Cells

The optimization of solar photovoltaic (PV) cells and modules is crucial for enhancing solar energy conversion efficiency, a significant barrier to the widespread adoption of solar energy. Accurate modeling and estimation of PV parameters are essential for the optimal design, control, and simulation of PV systems. Traditional optimization methods often suffer from limitations such as entrapment in local optima when addressing this complex problem. This study introduces the Material Generation Algorithm (MGA), inspired by the principles of material chemistry, to estimate PV parameters effectively. The MGA simulates the creation and stabilization of chemical compounds to explore and optimize the parameter space. The algorithm mimics the formation of ionic and covalent bonds to generate new candidate solutions and assesses their stability to ensure convergence to optimal parameters. The MGA is applied to estimate parameters for two different PV modules, RTC France and Kyocera KC200GT, considering their manufacturing technologies and solar cell models. The significant nature of the MGA in comparison to other algorithms is further demonstrated by experimental and statistical findings. A comparative analysis of the results indicates that the MGA outperforms the other optimization strategies that previous researchers have examined for parameter estimation of solar PV systems in terms of both effectiveness and robustness. Moreover, simulation results demonstrate that MGA enhances the electrical properties of PV systems by accurately identifying PV parameters under varying operating conditions of temperature and irradiance. In comparison to other reported methods, considering the Kyocera KC200GT module, the MGA consistently performs better in decreasing RMSE across a variety of weather situations; for SD and DD models, the percentage improvements vary from 8.07% to 90.29%.