Optimal Control Actions Research Articles

Triggering Optimal Control of Air Conditioning Systems by Event-Driven Mechanism: Comparing Direct and Indirect Approaches

Real-time optimal control of air conditioning (AC) is important, and should respond to the condition changes for an energy efficient operation. The traditional optimal control triggering mechanism is based on the “time clock” (called time-driven), and has certain drawbacks (e.g., delayed or unnecessary actions). Thus, an event-driven optimal control (EDOC) was proposed. In previous studies, the part-load ratio (PLR) of chiller plants was used as events to trigger optimal control actions. However, PLR is an indirect indicator of operation efficiency, which could misrepresent the system coefficient of performance (SCOP). This study thus proposes to directly monitor the SCOP deviations from the desired SCOP values. Two events are defined based on transient and cumulative SCOP deviations, which are systematically investigated in terms of energy performance and robustness. The PLR-based and SCOP-based EDOC are compared, in which energy saving and optimal control triggering time are analyzed. Results suggest that SCOP-based EDOC has better energy performance compared with PLR-based EDOC, but the frequent event triggering might happen due to the parameter uncertainty. For actual applications, the SCOP-based EDOC can be recommended when the ideal SCOP model is available with the properly-handled uncertainty. Nevertheless, the PLR-based EDOC could still be a more practical option to replace the traditional TDOC considering its acceptable energy performance and better robustness.

Performance based thermal comfort control (PTCC) using deep reinforcement learning for space cooling

With the recent increase in energy consumption in buildings, energy-saving strategies in buildings have become a priority in the energy policies of many countries. Therefore, many recent research studies have emphasized the advanced control methods to attain comfortable thermal conditions while minimizing the energy consumption in buildings. A new approach of thermal comfort control for space cooling/heating system is needed to reflect the changing indoor environment information in real time, and to control various factors (e.g., humidity, air velocity, etc.) that affect not only the temperature but also the thermal comfort.In this study, we propose the Gaussian process regression (GPR) for real-time thermal comfort prediction, a data-driven approach. These data-driven approaches will enable the monitoring of occupants and thermal comfort conditions based on real-time data and situational awareness. Then, based on the thermal comfort performance (PMV) prediction results obtained using the GPR, we investigated control methods involving the integration of systems, i.e., a variable refrigerant flow (VRF) system and a humidifier, instead of using simple set-temperature control for space cooling. For this purpose, deep Q-learning, which is an reinforcement learning method, was employed to derive the VRF and humidification integrated control methods. During zone operation, this algorithm learned an effective control policy based on rewards (thermal comfort and energy consumption) without relying on a thermal dynamics model. Moreover, by comparing the thermal comfort and energy consumption results with those obtained using fixed set-point (rule-based) control and performance-based comfort control for cooling, the efficiency of the proposed performance-based thermal comfort control (PTCC) was evaluated.As a results, it was found that PTCC yielded the optimal control action value that minimized the energy consumption while satisfying the thermal comfort conditions. In addition, applying the proposed PTCC strategy to cooling control could maintain the required performance level of thermal comfort by reflecting changing environmental conditions in real time, unlike the fixed set-point control.

A Predictive Path-Following Controller for Continuous Replanning With Dynamic Roadmaps

This letter proposes a predictive path-following controller for efficient time parameterization during continuous replanning with dynamic roadmaps (DRMs). DRMs are an approach for fast global path planning in changing environments and are based on a precomputed graph in the configuration space and mappings from workspace voxels to vertices and edges in the graph. However, DRMs are a kinematic path planner and require subsequent steps of smoothing and trajectory generation before execution. This makes it hard to replan while the robot is moving, it is so because the time parameterization should account for the current position and velocity of the robot. To cope with this issue, a predictive path-following controller is presented, which computes the optimal control actions for moving the robot along the planned paths while respecting the bounds on maximum velocities and accelerations. Furthermore, it enables continuous replanning, i.e., the computation of new paths at fixed intervals, which is particularly useful in combination with a prediction of moving obstacles. The real-time capable approach is evaluated in simulation scenarios with moving obstacles and is experimentally validated for a dual-arm robot with 12 degrees of freedom.

Forecast-Informed Energy Storage Utilization in Local Area Power Systems

Distributed energy storage is beneficial to energy consumers and system operators, but it is expensive to install and requires careful management. To improve the viability of demand-side energy storage, this study presents a new method of obtaining optimal control actions for a demand-side energy storage system in the presence of distributed generation. The optimal usage policy is determined by framing the selection of charge and discharge actions as a stochastic optimization problem, which is conveniently represented as a Markov decision process. The proposed method uses short-term forecasts of loads and local generation to represent the time-dependence and nonstationarity of the net load profile. In contrast to previous studies, the forecast information is used to restrict the problem state space, reducing the computational complexity of the policy calculation. The method is tested using historic load and generation data with real time-of-use rate schedules and achieves substantial reductions in energy costs over similar existing methods.

Energy-Efficient Base Station Control Framework for 5G Cellular Networks Based on Markov Decision Process

We study the problem of base station (BS) dynamic switching for energy efficient design of fifth-generation (5G) cellular networks and beyond. We formulate this problem as a Markov decision process (MDP) and use an approximation method known as policy rollout to solve it. This method employs Monte Carlo sampling to approximate the Q-value. In this paper, we introduce a novel approach to design an energy-efficient BS control algorithm. We design an MDP-based algorithm to control the on/off switching of BSs in real time; we exploit user mobility and location information in the selection of the optimal control actions. We start our formulation with the simple case of one-user one-ON. We then gradually and systematically extend this formulation to the multi-user multi-ON scenario. Simulation results show the potential of our novel approach of exploiting user mobility information within the MDP framework to achieve significant energy savings while providing quality-of-service guarantees.

Sensor Control in Anti-Submarine Warfare-A Digital Twin and Random Finite Sets Based Approach.

Since the submarine has become the major threat to maritime security, there is an urgent need to find a more efficient method of anti-submarine warfare (ASW). The digital twin theory is one of the most outstanding information technologies, and has been quite popular in recent years. The most influential change produced by digital twin is the ability to enable real-time dynamic interactions between the simulation world and the real world. Digital twin can be regarded as a paradigm by means of which selected online measurements are dynamically assimilated into the simulation world, with the running simulation model guiding the real world adaptively in reverse. By combining digital twin theory and random finite sets (RFSs) closely, a new framework of sensor control in ASW is proposed. Two key algorithms are proposed for supporting the digital twin-based framework. First, the RFS-based data-assimilation algorithm is proposed for online assimilating the sequence of real-time measurements with detection uncertainty, data association uncertainty, noise, and clutters. Second, the computation of the reward function by using the results of the proposed data-assimilation algorithm is introduced to find the optimal control action. The results of three groups of experiments successfully verify the feasibility and effectiveness of the proposed approach.

Improved strategy for optimal control of Civil Engineering structures

Inside the Industrial Revolution 4.0, the concepts as Smart Cities and Smart Society are important parts in protection of humans against natural disasters, as strong earthquakes. Older or newer protection strategies of Civil Engineering structures (bridges, buildings) must be updated with elements of the actual technological revolution as sensors, drones, robots, nanotechnologies, AI, big data, cloud computing and so on. This paper is focused on a optimal structural active control of buildings started by many researchers during the last decades. Now, the target is to improve an optimal strategy developed by authors by proposing a more efficient use of computation. The success of this strategy leads to a faster response of computers that direct the control devices. This provides the ground for a larger range of control devices based on a larger number of sensors and, therefore, a better energy use together with a more precise anti-seismic structural protection

Hybrid physics‐based and data‐driven modeling for bioprocess online simulation and optimization

Model-based online optimization has not been widely applied to bioprocesses due to the challenges of modeling complex biological behaviors, low-quality industrial measurements, and lack of visualization techniques for ongoing processes. This study proposes an innovative hybrid modeling framework which takes advantages of both physics-based and data-driven modeling for bioprocess online monitoring, prediction, and optimization. The framework initially generates high-quality data by correcting raw process measurements via a physics-based noise filter (a generally available simple kinetic model with high fitting but low predictive performance); then constructs a predictive data-driven model to identify optimal control actions and predict discrete future bioprocess behaviors. Continuous future process trajectories are subsequently visualized by re-fitting the simple kinetic model (soft sensor) using the data-driven model predicted discrete future data points, enabling the accurate monitoring of ongoing processes at any operating time. This framework was tested to maximize fed-batch microalgal lutein production by combining with different online optimization schemes and compared against the conventional open-loop optimization technique. The optimal results using the proposed framework were found to be comparable to the theoretically best production, demonstrating its high predictive and flexible capabilities as well as its potential for industrial application.

Event-Triggered Optimal Active Power Control in Islanded Microgrid With Variable Demand and Time-Varying Communication Topology

For islanded microgrids, traditional droop-based control leads to frequency deviation and low economic efficiency. To resolve this issue, a novel event-triggered optimal active power control strategy, which achieves frequency regulation and economic operation in a fully distributed way, is proposed in this paper. Frequency deviations are monitored and used to trigger the load estimation algorithm to track changes in power demand. Based on estimated demand values, a distributed economic dispatch (ED) algorithm, which only requires that the communication topology satisfies the jointly connected condition, is proposed to calculate the optimal generation references for the generators. The ED algorithm has the finite-time convergence property, which is rigorously proved by Lyapunov theory, LaSalle’s invariance principle and homogeneous property. Based on optimal generation references, the parameters of droop controller are reset in real time, which makes microgrid achieve frequency restoration and economic operation simultaneously. Finally, several case studies are presented to illustrate the effectiveness of the proposed control strategy.

Predictive Control for NOx Emission Reductions in Diesel Engine Vehicle Platoon Application

This paper presents a predictive control design for diesel vehicle platoons, such as medium- and heavy-duty truck platoons, to achieve lower NOx emissions. By incorporating vehicle longitudinal dynamics control and powertrain estimation with aftertreatment system control, an enhanced emission performance can be attained for the entire platoon considering different operating conditions for individual vehicles. The proposed method utilizes exhaust gas predictive information within a preview horizon to generate optimal control actions such that the tradeoff between NOx reduction efficiency and NH3 slip in the aftertreatment systems can be well-managed. Performance improvement is demonstrated through simulation studies based on an experimental validated powertrain and aftertreatment system model.

Developed comparative analysis of metaheuristic optimization algorithms for optimal active control of structures

A developed comparative analysis of metaheuristic optimization algorithms has been used for optimal active control of structures. The linear quadratic regulator (LQR) has ignored the external excitation in solving the Riccati equation with no sufficient optimal results. To enhance the efficiency of LQR and overcome the non-optimality problem, six intelligent optimization methods including BAT, BEE, differential evolution, firefly, harmony search and imperialist competitive algorithm have been discretely added to wavelet-based LQR to seek the attained optimum feedback gains. The proposed approach has not required the solution of Riccati equation enabling the excitation effect in controlling process. Employing this advantage by each of six mentioned algorithms to three-story and eight-story structures under different earthquakes led to define (1) the best solution, (2) convergence rate and (3) computational effort of all methods. The purpose of this research is to study the aforementioned methods besides the superiority of ICA in finding the optimal responses for active control problem. Numerical simulations have confirmed that the proposed controller is enabling to significantly reduce the structural responses using less control energy compared to LQR.

Optimal active power control based on MPC for DFIG-based wind farm equipped with distributed energy storage systems

An optimal active power control scheme based on model predictive control (MPC) is proposed for a doubly-fed induction generator (DFIG)-based wind farm equipped with distributed energy storage systems (ESSs). A two-stage optimal control scheme is proposed. In the first stage, the power reference of each WT and the total power command of ESSs are generated, aiming to reduce the fatigue load of WTs by minimizing variations of thrust force and shaft torque. In the second stage, the charge/discharge power of ESSs are optimized to achieve the fair power sharing and maximize the capacity margin. An MPC based optimization problem is formulated for the constrained multiple input and multiple output (MIMO) wind farm system. The dynamics of converters and WTs are taken into account by the MPC. With the proposed control scheme, the active power references are optimized between WTs and ESSs according to their local wind conditions. Fatigue loads of WTs are reduced efficiently by coordinating the DFIG-based WTs and distributed ESSs. A wind farm with 10 DFIG-based WTs was used to validate the control performance of the proposed optimal active power control scheme.

Taking Iron Loss into Account in Control Systems of Synchronous Reluctance Electric Drives

Synchronous reluctance electric drives are being developed due to their advantages of contactlessness, high overload and specific torques, absence of loss in the rotor, and large control range, as well as progress that is being made in the fields of information and power electronics. It is shown that iron loss significantly affects the control algorithms for synchronous reluctance machines. A four-loop vector control system of a synchronous reluctance machine is considered. The main ducts are the current and electromagnetic torque control circuits of the electric machine and the secondary ducts are the stator flux linkage and angular velocity contours. The features of observer synthesis for vector control system taking into account iron losses and magnetic saturation of the system are given. By means of experimental studies, the adequacy of the received observer was shown. The synthesis of algorithms for optimal control of the synchronous reluctance machine, taking into account iron loss, is given. The first algorithm assumes the existence of a table with optimal values of control actions. The second provides for the synthesis of an adaptive torque controller, which will be automatically tuned to the optimal control action. The obtained control algorithms are universal and can be applied in other control systems of synchronous reluctance machines, for example, in direct torque control systems.

Cloth Manipulation Using Random-Forest-Based Imitation Learning

We present a novel approach for robust manipulation of high-DOF deformable objects such as cloth. Our approach uses a random forest-based controller that maps the observed visual features of the cloth to an optimal control action of the manipulator. The topological structure of this random forest-based controller is determined automatically based on the training data consisting visual features and optimal control actions. This enables us to integrate the overall process of training data classification and controller optimization into an imitation learning (IL) approach. Our approach enables learning of robust control policy for cloth manipulation with guarantees on convergence.We have evaluated our approach on different multi-task cloth manipulation benchmarks such as flattening, folding and twisting. In practice, our approach works well with different deformable features learned based on the specific task or deep learning. Moreover, our controller outperforms a simple or piecewise linear controller in terms of robustness to noise. In addition, our approach is easy to implement and does not require much parameter tuning.

“Mean Field Games” as Mathematical Models for Control and Optimization of Business Activity

The article is a review of modern mathematical economic models with the “Mean Field Games” structure. They are currently used for the predictive modelling under given control conditions or for optimizing control actions to achieve the desired result. The mathematical model is a pair of parabolic partial differential equations with a set of initial and boundary conditions for optimizing a given target functional. For them, the discretization is applied to obtain systems of nonlinear algebraic equations which are solved by computer in an iterative way to get the best instant benefit for each agent. This mathematical apparatus is used for the quantitative modelling of the distribution or the use of alternative resources, environmental problems, optimization of wages and insurance, network sales, and other economic activities to predict the aggregate behavior of the great mass of agents looking for instant personal benefit

Centralised model‐predictive decoupled active–reactive power control for three‐level neutral point clamped photovoltaic inverter with preference selective index‐based objective prioritisation

This study presents a single-stage grid-tied three-level neutral point clamped photovoltaic inverter with a centralised model-predictive decoupled active–reactive power control. The proposed centralised model predictive control (CMPC) incorporates the constraints of maximum power extraction, dc-link capacitor voltage balancing and active–reactive power tracking in a single objective function. The dc-link voltage of the inverter is regulated to its reference for extracting the maximum power. In order to eliminate the impact of reactive power exchange on floating dc-link voltage regulation, a decoupled active–reactive power control is used in the CMPC. Furthermore, a preference selective index-based dynamic weighting factor selection approach is introduced to maintain the relative importance between the power tracking and dc-link capacitor voltage balancing. The proposed control approach eliminates the outer dc-link voltage control loop and also, the empirical approach required for the selection of weighting factors. As a result, it ensures an optimal control action in each sampling period to improve the steady-state and dynamic tracking performance of the control objectives. The proposed control approach is experimentally verified by using a 1.2 kW laboratory-scale prototype and the results are presented to demonstrate its effectiveness compared to the classical proportional–integral-based model predictive control.

A new approximate dynamic programming algorithm based on an actor–critic framework for optimal control of alkali–surfactant–polymer flooding

ABSTRACTAn approximate dynamic programming algorithm based on an actor–critic framework is proposed in this article. This algorithm takes the actor–critic framework as the basic framework, in which the actor and the critic are used to approximate the optimal value function and the control strategy, respectively. At first, the linear basis function approximator is used to approximate the value function. Then the method of basis function construction based on system characteristics is introduced. Furthermore, since the injection concentration of ASP flooding has a fixed interval, the action weighting method is adopted to restrict and approximate the optimal control action. The value function parameter and the two strategy parameters are updated by the gradient descent method. Meanwhile the eligibility trace is introduced to accelerate convergence. Finally, ASP flooding with four injection wells and nine production wells is used to test the effect of the proposed method.

Quiet standing: The Single Inverted Pendulum model is not so bad after all.

In the study of balance and postural control the (Single) Inverted Pendulum model (SIP) has been taken for a long time as an acceptable paradigm, with the implicit assumption that only ankle rotations are relevant for describing and explaining sway movements. However, more recent kinematic analysis of quiet standing revealed that hip motion cannot be neglected at all and that ankle-hip oscillatory patterns are characterized by complex in-phase and anti-phase interactions, suggesting that the SIP model should be substituted by a DIP (Double Inverted Pendulum) model. It was also suggested that DIP control could be characterized as a kind of optimal bi-axial active controller whose goal is minimizing the acceleration of the global CoM (Center of Mass). We propose here an alternative where active feedback control is applied in an intermittent manner only to the ankle joint, whereas the hip joint is stabilized by a passive stiffness mechanism. The active control impulses are delivered to the ankle joint as a function of the delayed state vector (tilt rotation angle + tilt rotational speed) of a Virtual Inverted Pendulum (VIP), namely a pendulum that links the ankle to the CoM, embedded in the real DIP. Simulations of such DIP/VIP model, with the hybrid control mechanism, show that it can reproduce the in-phase/anti-phase interaction patterns of the two joints described by several experimental studies. Moreover, the simulations demonstrate that the DIP/VIP model can also reproduce the measured minimization of the CoM acceleration, as an indirect biomechanical consequence of the dynamic interaction between the active control of the ankle joint and the passive control of the hip joint. We suggest that although the SIP model is literally false, because it ignores the ankle-hip coordination, it is functionally correct and practically acceptable for experimental studies that focus on the postural oscillations of the CoM.

Optimal Control of Pretwisted Rotating Thin-Walled Beams via Piezoelectrically Induced Couplings

Problems related to mathematical modeling and optimal active control of pretwisted adaptive blade are considered. The blade is modeled as a rotating thin-walled composite beam embedded with anisotropic piezo-composite layers accounting for nonclassical effects, such as transverse shear and warping inhibitions. The linear-quadratic-regulator feedback control strategy is adopted to study the tailoring of piezo-actuators on vibration suppression. Control authority of piezoelectrically induced transverse shear and bending coupling is highlighted. Tailoring studies using the present model reveal that piezoelectrically induced transverse shear plays an important role on control effectiveness. In addition, the relations between the control authority and the elastic couplings, piezoelectrically induced actuation couplings, pretwist angle, and size and position of piezo-actuators are investigated.

Distributed Secondary and Optimal Active Power Sharing Control for Islanded Microgrids With Communication Delays

As the communication delays in smart grid are considered, the novel secondary voltage and frequency restoration, the optimal active power sharing, and the accurate reactive power sharing protocols are proposed in this paper. The Lyapunov–Krasovskii functions are used to analyze the stability of our proposed distributed secondary control schemes. Sensitivity analysis of time delays is also provided. Whether the communication delays exist or not, the control protocols always guarantee that the output voltage amplitude and frequency of each distributed generator synchronize to the reference values and maintain the optimal active power sharing and the accurate reactive power sharing properties. Moreover, our proposed control protocols are implemented through sparse communication networks and thus meet the plug-and-play feature of the future smart grid. Simulation results are provided to show the effectiveness of the proposed control protocols.