Stable Optimal Control Research Articles

An influent responsive control strategy with machine learning: Q-learning based optimization method for a biological phosphorus removal system

Biological phosphorus removal (BPR) is an economical and sustainable processes for the removal of phosphorus (P) from wastewater, achieved by recirculating activated sludge through anaerobic and aerobic (An/Ae) processes. However, few studies have systematically analyzed the optimal hydraulic retention times (HRTs) in anaerobic and aerobic reactions, or whether these are the most appropriate control strategies. In this study, a novel optimization methodology using an improved Q-learning (QL) algorithm was developed, to optimize An/Ae HRTs in a BPR system. A framework for QL-based BPR control strategies was established and the improved Q function, Qt+1(st,st+1)=Qt(st,st+1)+k·[R(st,st+1)+γ·maxatQt(st,st+1)−Qt(st,st+1)] was derived. Based on the improved Q function and the state transition matrices obtained under different HRT step-lengths, the optimum combinations of HRTs in An/Ae processes in any BPR system could be obtained, in terms of the ordered pair combinations of the <current state-transition state>. Model verification was performed by applying six different influent chemical oxygen demand (COD) concentrations, varying from 150 to 600 mg L−1 and influent P concentrations, varying from 12 to 30 mg L−1. Superior and stable effluent qualities were observed with the optimal control strategies. This indicates that the proposed novel QL-based BPR model performed properly and the derived Q functions successfully realized real-time modelling, with stable optimal control strategies under fluctuant influent loads during wastewater treatment processes.

Intelligent Control/Operational Strategies in WWTPs through an Integrated Q-Learning Algorithm with ASM2d-Guided Reward

The operation of a wastewater treatment plant (WWTP) is a typical complex control problem, with nonlinear dynamics and coupling effects among the variables, which renders the implementation of real-time optimal control an enormous challenge. In this study, a Q-learning algorithm with activated sludge model No. 2d-guided (ASM2d-guided) reward setting (an integrated ASM2d-QL algorithm) is proposed, and the widely applied anaerobic-anoxic-oxic (AAO) system is chosen as the research paradigm. The integrated ASM2d-QL algorithms equipped with a self-learning mechanism are derived for optimizing the control strategies (hydraulic retention time (HRT) and internal recycling ratio (IRR)) of the AAO system. To optimize the control strategies of the AAO system under varying influent loads, Q matrixes were built for both HRTs and IRR optimization through the pair of &lt;max reward-action&gt; based on the integrated ASM2d-QL algorithm. 8 days of actual influent qualities of a certain municipal AAO wastewater treatment plant in June were arbitrarily chosen as the influent concentrations for model verification. Good agreement between the values of the model simulations and experimental results indicated that this proposed integrated ASM2d-QL algorithm performed properly and successfully realized intelligent modeling and stable optimal control strategies under fluctuating influent loads during wastewater treatment.

Stable Optimal Control and Semicontractive Dynamic Programming

We consider discrete-time infinite horizon deterministic optimal control problems with nonnegative cost per stage, and a destination that is cost free and absorbing. The classical linear-quadratic regulator problem is a special case. Our assumptions are very general, and allow the possibility that the optimal policy may not be stabilizing the system, e.g., may not reach the destination either asymptotically or in a finite number of steps. We introduce a new unifying notion of stable feedback policy, based on perturbation of the cost per stage, which in addition to implying convergence of the generated states to the destination, quantifies the speed of convergence. We consider the properties of two distinct cost functions: $J^*$, the overall optimal, and $\hat J$, the restricted optimal over just the stable policies. Different classes of stable policies (with different speeds of convergence) may yield different values of $\hat J$. We show that for any class of stable policies, $\hat J$ is a solution of Bel...

Admittivity imaging from multi-frequency micro-electrical impedance tomography

The aim of this paper is to propose an optimal control optimization algorithm for reconstructing admittivity distributions (i.e., both conductivity and permittivity) from multi-frequency micro-electrical impedance tomography. A convergent and stable optimal control scheme is shown to be obtainable from multi-frequency data. This opens a door for convergence analysis of optimal control type approaches in imaging from internal data. The results of this paper have potential applicability in cancer imaging, cell culturing and differentiation, food sciences, and biotechnology.

A New Approach to Stable Optimal Control of Complex Nonlinear Dynamical Systems

This paper gives a simple approach to designing a controller that minimizes a user-specified control cost for a mechanical system while ensuring that the control is stable. For a user-given Lyapunov function, the method ensures that its time rate of change is negative and equals a user specified negative definite function. Thus a closed-form, optimal, nonlinear controller is obtained that minimizes a desired control cost at each instant of time and is guaranteed to be Lyapunov stable. The complete nonlinear dynamical system is handled with no approximations/linearizations, and no a priori structure is imposed on the nature of the controller. The methodology is developed here for systems modeled by second-order, nonautonomous, nonlinear, differential equations. The approach relies on some recent fundamental results in analytical dynamics and uses ideas from the theory of constrained motion.

Stable optimal control applied to a cylindrical robotic arm

In this paper, an asymptotically stable optimal control is proposed for the trajectory tracking of a cylindrical robotic arm. The proposed controller uses the linear quadratic regulator method and its Riccati equation is considered as an adaptive function. The tracking error of the proposed controller is guaranteed to be asymptotically stable. A simulation shows the effectiveness of the proposed algorithm.