Optimal Control Architecture Research Articles

Computationally Efficient Inference via Time-Aware Modular Control Systems

Control in multi-agent decision-making systems is an important issue with a wide variety of existing approaches. In this work, we offer a new comprehensive framework for distributed control. The main contributions of this paper are summarized as follows. First, we propose PHIMEC (physics-informed meta control)—an architecture for learning optimal control by employing a physics-informed neural network when the state space is too large for reward-based learning. Second, we offer a way to leverage impulse response as a tool for system modeling and control. We propose IMPULSTM, a novel approach for incorporating time awareness into recurrent neural networks designed to accommodate irregular sampling rates in the signal. Third, we propose DIMAS, a modular approach to increasing computational efficiency in distributed control systems via domain-knowledge integration. We analyze the performance of the first two contributions on a set of corresponding benchmarks and then showcase their combined performance as a domain-informed distributed control system. The proposed approaches show satisfactory performance both individually in their respective applications and as a connected system.

Optimal Control Architecture for Balancing Performance and Cost in Oil and Gas Production Systems

Optimal Control Architecture for Balancing Performance and Cost in Oil and Gas Production Systems

Disturbance observer-based robust optimal control of rate-dependent hysteretic systems: Application to a piezoelectric nano-manipulator

This article investigates the robust and optimal control problem of hysteretic systems with both parameter variations and unmodeled dynamics. By means of a hysteresis compensator based on a modified Prandtl–Ishlinskii model and a feedback controller, a novel robust optimal control architecture is proposed to address the asymmetric and rate-dependent hysteresis nonlinearities and various disturbances, where the compensation errors, parameter variations, and unmodeled dynamics are treated as a bounded disturbance. The proposed controller is composed of two components: a high order sliding mode observer and an extended linear quadratic controller. The high order sliding mode observer is introduced to guarantee that the disturbance observation error converges to the origin in a finite time such that the extended linear quadratic controller enables the predefined optimal performance to be achieved. Moreover, the exponential stability of the closed-loop system has been proven, and some sufficient conditions are given. Finally, the proposed control algorithm is applied to a piezoelectric nano-manipulator system, both the good hysteresis modeling accuracy and excellent tracking performance are demonstrated in the real-time experiments.

Optimal Control Algorithm for Subway Train Operation by Proximal Policy Optimization

With the increasing scale of the urban subway, the total energy consumption of the subway has increased dramatically and poses a great challenge to the comfort of passengers and the punctuality of train operation. In order to ensure on-time train operation and passenger comfort, and at the same time reduce the energy consumption of subway operation, this paper proposes a Proximal Policy Optimization (PPO)-based optimization algorithm for the optimal control of subway train operation. Firstly, a reinforcement learning architecture for optimal control of subway train operation is constructed with the position and speed of train operation as the reinforcement learning state, energy consumption and comfort as the optimization objectives, and train operation time as the constraint. The proposed reinforcement learning model is trained by the PPO algorithm, and the reward scaling is added to the training process to accelerate the training speed and improve the efficiency of the algorithm. The experimental results show that the proposed PPO with reward scaling algorithm can effectively reduce train energy consumption and improve passenger comfort while ensuring on-time train operation.

Approximate Optimal Tracking Control for Partially Unknown Nonlinear Systems via an Adaptive Fixed-Time Observer

This paper investigates a novel adaptive fixed-time disturbance observer (AFXDO)-based approximate optimal tracking control architecture for nonlinear systems with partially unknown dynamic drift and perturbation under an adaptive dynamic programming (ADP) scheme. To attenuate the impact of disturbance, a novel AFXDO was designed based on the principle of a fixed-time stable system without prior information of disturbance, making disturbance observer errors converge to zero in a fixed time independent of initial estimation error. Additionally, approximate optimal control is conducted by incorporating the real-time estimation of AFXDO into a critic-only ADP framework to stabilize the dynamics of tracking errors and strike a balance between consumption and performance. In particular, to address the heavy calculation burden and oscillation phenomenon in the traditional actor–critic structure, an improved adaptive update law with a variable learning rate was developed to update the weight for adjusting the optimal cost function and optimal control policy simultaneously, avoiding the initial chattering phenomenon and achieving a prescribed convergence without resorting to dual networks. With the efforts of AFXDO and a weight law with a variable learning rate, the track errors were achieved with fast transient performance and low control consumptions in a fixed time. By revisiting Lyapunov stability, the tracking error and weight estimation error were proven to be uniformly ultimately bounded, and the designed control tended to optimal control. The simulations were carried out on quadrotor tracking to demonstrate the effectiveness of the developed control scheme, which achieves rapid convergence by lower control consumption in 4 s, where the cost function is reduced by 19.13%.

A Hybrid Deep Reinforcement Learning and Optimal Control Architecture for Autonomous Highway Driving

Autonomous vehicles in highway driving scenarios are expected to become a reality in the next few years. Decision-making and motion planning algorithms, which allow autonomous vehicles to predict and tackle unpredictable road traffic situations, play a crucial role. Indeed, finding the optimal driving decision in all the different driving scenarios is a challenging task due to the large and complex variability of highway traffic scenarios. In this context, the aim of this work is to design an effective hybrid two-layer path planning architecture that, by exploiting the powerful tools offered by the emerging Deep Reinforcement Learning (DRL) in combination with model-based approaches, lets the autonomous vehicles properly behave in different highway traffic conditions and, accordingly, to determine the lateral and longitudinal control commands. Specifically, the DRL-based high-level planner is responsible for training the vehicle to choose tactical behaviors according to the surrounding environment, while the low-level control converts these choices into the lateral and longitudinal vehicle control actions to be imposed through an optimization problem based on Nonlinear Model Predictive Control (NMPC) approach, thus enforcing continuous constraints. The effectiveness of the proposed hierarchical architecture is hence evaluated via an integrated vehicular platform that combines the MATLAB environment with the SUMO (Simulation of Urban MObility) traffic simulator. The exhaustive simulation analysis, carried out on different non-trivial highway traffic scenarios, confirms the capability of the proposed strategy in driving the autonomous vehicles in different traffic scenarios.

Design of an Optimal Scheduling Control System for Smart Manufacturing Processes in Tobacco Industry

The whole process of tobacco production is composed of many components, in which their operation and administration are currently independent. It is required to deploy smart manufacturing workflow for the whole production process, in order to realize centralized effective global scheduling. This requires an advanced administration control platform that has strong abilities of multisource data integration and automatic decision support. To bridge such research gap, this paper designs an optimal scheduling control system for smart manufacturing processes of tobacco industry. First of all, this work discusses major characteristics of future-generation production control patterns in intelligent tobacco factories (ITF). Then, a five-layer architecture for optimal scheduling control of ITF is proposed, which contains Internet-of-Things layer, centralized control layer, model layer, platform layer and operation layer. In addition, a production scheduling optimization strategy is also developed for the proposed system to serve as the software algorithm that drives the running of whole smart manufacturing processes. Finally, this paper presents a comparative analysis of the proposed system’s transformation in a cigarette factory. Naturally, the effectiveness of the proposed production optimization scheduling strategy is verified through simulation.

Distributed Control of Distributed Energy Resources in Active Power Distribution System for Local Power Balance With Optimal Spectral Clustering

This article proposes a distributed control architecture for distributed energy resources (DERs) that include photovoltaics (PVs) and battery energy storage systems (BESSs). The control architecture is based on the virtual clustering of DERs to provide power balance (minimizing the net load (generation load) in each cluster) in each cluster considering the state of charge of the BESS. First, the architecture designs a spectral clustering to develop optimal clusters based on the number of DERs that provide clusters with minimum power transfer between clusters. Then, an error-minimization-based optimal control architecture provides sufficient set points to the BESS intending to provide power balance. The novelty of the proposed clustering is that it provides a distributed dynamic model of the grid with measurements that are stable and scalable. The control approach not only provides an optimal set point for the BESS but also takes into account the state of charge and provides intercluster balance. This framework can also manage the local grid effectively as the distribution is dynamic considering the state of the BESS and the number of DERs. The distributed control objectives are efficient and scalable. The architecture is tested on the IEEE 123-bus system and the IEEE 8500-bus system and observed that the approach provides effective power balance and outperforms the existing methods.

A Hierarchical Architecture for Optimal Unit Commitment and Control of an Ensemble of Steam Generators

A hierarchical architecture for the optimal management of an ensemble of steam generators is presented. The subsystems are coordinated by a multilayer scheme for jointly sustaining a common load. The high level optimizes the load allocation and the generator schedule, considering activation dynamics by a hybrid model. At the medium level, a robust tube-based model predictive control (MPC) tracks a time-varying demand using a centralized—but aggregate—model, whose order does not scale with the number of subsystems. A nonlinear optimization, at medium level, addresses MPC infeasibility due to abrupt changes of ensemble configuration. Low-level decentralized controllers stabilize the generators. This control scheme enables the dynamical modification of the ensemble configuration and plug and play operations. Simulations demonstrate the approach potentialities.

Oscillation Damping of Integrated Transmission and Distribution Power Grid With Renewables Based on Novel Measurement-Based Optimal Controller

Renewable energy resources are gaining fast adoption in the power grid because of their relatively low cost and offered environmental benefits. However, the grid will experience decreased inertia as a result of these added resources. Numerous research studies have focused on improving the grid damping, including the lack of inertia, due to renewable resource integration. In this article, we present an adaptive damping controller to help mitigate oscillations during disturbances on the grid, caused by fault occurrences, sudden load changes, capacitor switching, or even the intermittent nature of the energy sources for most renewable systems, considering both integrated transmission and distribution systems. The test system is developed based on Kundur’s two-area system as the transmission system and the IEEE 123-bus system as the distribution system. Then, the novel proposed adaptive optimal damping control architecture is validated via simulation-based test cases conducted in MATLAB/Simulink using real-life grid data. It is observed that the proposed control architecture not only damps the oscillations more than 10% as compared with the state-of-the-art methods but can also control multiple generators.

On the Design, Modeling and Experimental Verification of a Floating Satellite Platform

In this letter, a floating robotic emulation platform is presented with an autonomous maneuverability for a virtual demonstration of a satellite motion. Such a robotic platform design is characterized by its friction-less, levitating, yet planar motion over a hyper-smooth surface. The design of the robotic platform, integrated with the sensor and actuator units, is briefly described, including the related component specification along with the mathematical model, describing its dynamic motion. Additionally, the article establishes a nonlinear optimal control architecture consisting of a unified model predictive approach for the overall manoeuvre tracking. The efficacy of the proposed modeling and control scheme is demonstrated in multiple experimental studies, where it is depicted that the proposed controller has the potential to address a precise point-to-point manoeuvre with terminal objectives, as well as an excellent path following capability. The proposed design is validated with extensive experimental studies, and it is supported with related results.

A Distributed and Hierarchical Optimal Control Method for Intelligent Connected Vehicles in Multi-Intersection Road Networks

Intelligent connected vehicles (ICVs) technologies will bring significant changes to future transportation, and urban intersections will be an important scenario for the application of ICVs. There exists one significant challenge to address for the control of ICVs in unsignalized, multi-intersection road networks, that is, how to realize the comprehensive optimization of traffic efficiency and energy saving. To solve this problem, the distributed and hierarchical optimal control architecture is first established in this paper, consisting of a cloud decision layer and a vehicle control layer. For the cloud decision layer, the distributed model predictive control (DMPC) method is utilized for distributed optimization control of multi-intersection road network systems, to achieve optimization in terms of traffic efficiency. For the vehicle control layer, based on the reference speed optimized from the cloud decision layer, the DMPC method is further utilized for distributed optimal control of each vehicle platoon, to achieve optimization in terms of energy saving. Finally, the comparative simulation tests are carried out based on MATLAB and SUMO. The feasibility and effectiveness of the proposed method were verified, and the improvement of traffic efficiency and energy saving was achieved.

A new optimal model predictive control scheme for a wind energy conversion system

AbstractThe correct management of power exchange between the doubly‐Fed induction generator (DFIG) and the grid depends on the effective optimal operation of the DFIG based wind energy conversion system (WECS). A modified optimal model predictive controller (MPC) architecture for WECS is proposed in this paper. The proposed scheme is carried out in two stages, as follows: (a) Using the MPC model and the tracking factor, an optimal control law is created, and (b) the aforementioned optimized issue is addressed using the linear matrix inequality (LMI) approach. Some improvements were made to the conventional LMI‐based MPC technique to lower the high computational complexity. To perform active and reactive power control in DFIG, the proposed approach is applied on the rotor‐side converter (RSC) control. The control law is divided into two sections. The first section deals with rotor current, while the second section deals with the RSC switching states. The LMI is used in the second portion of the control law to find the best switching state solution. The proposed method was created using the MATLAB/SIMULINK environment and tested in a 1hP DFIG‐WECS setup. By comparing the obtained results to conventional LMI‐PI, MPC‐based WECS, the effectiveness of the obtained results is verified. Variable wind speed conditions are used to vary all the results. Integral absolute error, integral time absolute error, and Integral square error measurements are used to compare the tracking performance index of all methods.

An event-driven multi-agent based distributed optimal control strategy for HVAC systems in IoT-enabled smart buildings

Smart buildings generally adopt a centralized optimal control for heating, ventilation and air-conditioning (HVAC) systems to improve the building system performance. As a crucial technology adopted in smart buildings, Internet of Things (IoT) based wireless sensor networks (WSNs) are promising platforms for implementing novel distributed optimal control to achieve distributed intelligence effectively. Such a future mist computing paradigm can be hindered by the limited energy resource of WSNs. The event-driven optimization method activates the optimization only when events occur. It can save the energy resource of WSNs, and thus pave the way for implementing the distributed optimal control architecture in IoT-based WSNs. This study therefore proposes an event-driven multi-agent based distributed optimal control strategy for HVAC systems for implementation in IoT-based battery-powered WSNs. The strategy consists of two novel schemes. First, an event determination scheme determines the event threshold by comprehensively considering the system performance and the total energy consumption of individual sensors implementing the distributed optimal control architecture. Second, an event-driven distributed optimization scheme solves the optimization problems with distributed optimization algorithms in IoT sensors of limited data processing capacity when an event occurs. Comparison and case studies are conducted to compare different strategies and validate the proposed strategy. Results show that different strategies require very different sensor energy consumption. The proposed strategy can provide satisfactory system performance while reducing the energy consumption of IoT sensors.

Branch point control at malonyl-CoA node: A computational framework to uncover the design principles of an ideal genetic-metabolic switch

Living organism is an intelligent system coded by hierarchically-organized information to perform precisely-controlled biological functions. Biophysical models are important tools to uncover the design rules underlying complex genetic-metabolic circuit interactions. Based on a previously engineered synthetic malonyl-CoA switch (Xu et al., PNAS, 2014), we have formulated nine differential equations to unravel the design principles underlying an ideal metabolic switch to improve fatty acids production in E. coli. By interrogating the physiologically accessible parameter space, we have determined the optimal controller architecture to configure both the metabolic source pathway and metabolic sink pathway. We determined that low protein degradation rate, medium strength of metabolic inhibitory constant, high metabolic source pathway induction rate, strong binding affinity of the transcriptional activator toward the metabolic source pathway, weak binding affinity of the transcriptional repressor toward the metabolic sink pathway, and a strong cooperative interaction of transcriptional repressor toward metabolic sink pathway benefit the accumulation of the target molecule (fatty acids). The target molecule (fatty acid) production is increased from 50% to 10-folds upon application of the autonomous metabolic switch. With strong metabolic inhibitory constant, the system displays multiple steady states. Stable oscillation of metabolic intermediate is the driving force to allow the system deviate from its equilibrium state and permits bidirectional ON-OFF gene expression control, which autonomously compensates enzyme level for both the metabolic source and metabolic sink pathways. The computational framework may facilitate us to design and engineer predictable genetic-metabolic switches, quest for the optimal controller architecture of the metabolic source/sink pathways, as well as leverage autonomous oscillation as a powerful tool to engineer cell function.

An identifier-actor-optimizer policy learning architecture for optimal control of continuous-time nonlinear systems

An intelligent solution method is proposed to achieve real-time optimal control for continuous-time nonlinear systems using a novel identifier-actor-optimizer (IAO) policy learning architecture. In this IAO-based policy learning approach, a dynamical identifier is developed to approximate the unknown part of system dynamics using deep neural networks (DNNs). Then, an indirect-method-based optimizer is proposed to generate high-quality optimal actions for system control considering both the constraints and performance index. Furthermore, a DNN-based actor is developed to approximate the obtained optimal actions and return good initial guesses to the optimizer. In this way, the traditional optimal control methods and state-of-the-art DNN techniques are combined in the IAO-based optimal policy learning method. Compared to the reinforcement learning algorithms with actor-critic architectures that suffer hard reward design and low computational efficiency, the IAO-based optimal policy learning algorithm enjoys fewer user-defined parameters, higher learning speeds, and steadier convergence properties in solving complex continuous-time optimal control problems (OCPs). Simulation results of three space flight control missions are given to substantiate the effectiveness of this IAO-based policy learning strategy and to illustrate the performance of the developed DNN-based optimal control method for continuous-time OCPs.

Towards a generic optimal co-design of hardware architecture and control configuration for interacting subsystems

In plants consisting of multiple interacting subsystems, the decision on how to optimally select and place actuators and sensors and the accompanying question on how to control the overall plant is a challenging task. Since there is no theoretical framework describing the impact of sensor and actuator placement on performance, an optimization method exploring the possible configurations is introduced in this paper to find a trade-off between implementation cost and achievable performance.Moreover, a novel model-based procedure is presented to simultaneously co-design the optimal number, type and location of actuators and sensors and to determine the corresponding optimal control architecture and accompanying control parameters. This paper adds the optimization of the control architecture to the current state-of-the-art. As an optimization output, a Pareto front is presented, providing insights on the optimal total plant performance related to the hardware and control design implementation cost.The proposed algorithm is not focused on one particular application or a specific optimization problem, but is instead a generally applicable method and can be applied to a wide range of applications (e.g., mechatronic, electrical, thermal). In this paper, the co-design approach is validated on a mechanical setup.

Exergy optimal multi-physics aircraft microgrid control architecture

The more electric aircraft (MEA) concept aims to reduce emissions, fuel costs, and enable incorporation of electric weapon systems and advanced sensor platforms. These systems will further burden the electrical system due to the pulse like loading and require advanced control strategies and distributed energy storage systems to ensure stability. Furthermore, multi-physical coupling of thermal electrical systems are often compartmentalize and analyzed separately, forgoing congruency that could occur if analyzed together. Here, we study how exergy, the amount of useful energy throughout a system, can guide control design and system operation. A multi-physics networked microgrid model was developed of an aircraft with two generation sources, interconnecting power converters, a lumped thermal mass model and pulsed loading. The Hamiltonian Surface Shaping Power Flow Control (HSSPFC) strategy is applied to the electrical grid via idealized and distributed storage elements. The HSSPFC was first developed to solve a general, scalable, form a networked microgrid architecture and then applied to the specific aircraft model. Implementation of the HSSPFC requires an outer loop to balance installed generation and to manage storage. This was accomplished through an exergy optimal set point generation scheme that minimized exergy destruction in the power converters. Bus regulation of within 3% of the desired set point was achieved while servicing a 100 kW pulsed load. A tradeoff between optimization update rate and storage regulation was found to be limited by the algorithm execution speed. Increased optimization update rates were linked to reduced storage use and fewer transients in bus voltage. The thermal model was electrically coupled through pumping loads and by cooling the power electronics. Exergy optimal coolant pump operation was also studied. The minimal exergy and pump energy consumption were obtained by operating the coolant system near the upper temperature limit of the coolant, which minimized cooling electrical loads.

A Distributed Architecture for Robust and Optimal Control of DC Microgrids

This paper presents a distributed, robust, and optimal control architecture for a network of multiple dc–dc converters. The network of converters considered form a dc microgrid in order to regulate a desired dc bus voltage and meet prescribed time-varying power sharing criteria among different energy sources. Such coordinated microgrids provide an important framework for leveraging the benefits of distributed power generation and consumption. The proposed control design seamlessly accommodates communication architectures that range from the centralized to decentralized scenarios with graceful degradation of the performance with lessened communication ability. Moreover, the methods developed are applicable to the case where the desired proportion in which the sources provide the power varies with time. A distinguishing feature of the control design approach is that it regards the net load current as a disturbance signal, lending itself to tractable analysis with tools from the robust and optimal control theory. A quantifiable analysis of the closed-loop stability and the performance of the network of converters is performed; the analysis simplifies to studying the closed-loop performance of an equivalent single-converter system. The control approach is demonstrated through simulations and experiments.

Distributed cooperative optimal control architecture for AC microgrid with renewable generation and storage

AC microgrid is a promising approach to integrate various distributed generators and energy storage into the power system, and provide renewable and reliable energy to the customers. However, highly intermittent distributed energy sources and varying load demands pose increasing challenges for optimal energy management in an AC microgrid. In this paper, a distributed cooperative optimal control method based on the discrete consistency algorithm is proposed to realize the large-scale penetration of renewable energy in an AC microgrid. The proposed approach is implemented through a multi-agent distributed hierarchical control architecture, which only requires a local communication network to exchange the information. The optimal control is achieved through a three-level control architecture. The third control level aims at multi-objective optimal scheduling, considering the factors such as fuel consumption, pollution in the form of emissions and operational maintenance. The second control level is based on the discrete consistency algorithm that optimizes the frequency and voltage references of droop control and also shares the active and reactive power accurately according to the particular demand. The first control level is responsible for reference tracking of the associated components. Compared with the conventional centralized and decentralized controls, the proposed control method offers increased robustness and flexibility in the AC microgrid control, and only requires limited communication between neighboring agents to realize a global optimization. Finally, the effectiveness of the proposed strategy is verified in a typical AC microgrid model by using the PSCAD/EMTDC software.