Closed-loop Optimal Strategies Research Articles

Automated self-optimization, intensification, and scale-up of photocatalysis in flow.

The optimization, intensification, and scale-up of photochemical processes constitute a particular challenge in a manufacturing environment geared primarily toward thermal chemistry. In this work, we present a versatile flow-based robotic platform to address these challenges through the integration of readily available hardware and custom software. Our open-source platform combines a liquid handler, syringe pumps, a tunable continuous-flow photoreactor, inexpensive Internet of Things devices, and an in-line benchtop nuclear magnetic resonance spectrometer to enable automated, data-rich optimization with a closed-loop Bayesian optimization strategy. A user-friendly graphical interface allows chemists without programming or machine learning expertise to easily monitor, analyze, and improve photocatalytic reactions with respect to both continuous and discrete variables. The system's effectiveness was demonstrated by increasing overall reaction yields and improving space-time yields compared with those of previously reported processes.

Accelerating Battery Electrolyte Design Process Via Closed-Loop Optimization of Simulations and Experiments

Designing and discovering new material combinations for battery applications has been and remains to be a slow and expensive process. It is a challenging task to develop battery electrolytes that are safe, high-performant, and cost-effective, as there are many design parameters to consider. There are many recent efforts to expedite such a process, ranging from the use of machine learning optimization techniques and lab automation to utilizing computer simulations.We propose an automated, closed-loop optimization strategy to design electrolytes by integrating machine learning-based optimization algorithms, autonomous robotics, and molecular simulations. In particular, we demonstrate the application of our approach for designing room-temperature ionic liquids (RTILs) for Li-ion batteries, which consist of multiple salts, cations, and anions. The composition space is too vast to explore via a conventional approach. Thus, we use intelligent decision-making algorithms for selecting which composition to test next, which is carried out by the in-house robotic platform. The molecular simulations are also used to provide an in-depth understanding of the experimentally observed behavior, and the simulation results are also fed into the optimization algorithm. The presented approach is designed to be general, so it can be applied to other electrochemical applications with minimal modifications to the algorithms or the experimental setup.

Closed-loop scheduling optimization strategy based on particle swarm optimization with niche technology and soft sensor method of attributes-applied to gasoline blending process

Gasoline blending scheduling optimization can bring significant economic and efficient benefits to refineries. However, the optimization model is complex and difficult to build, which is a typical mixed integer nonlinear programming (MINLP) problem. Considering the large scale of the MINLP model, in order to improve the efficiency of the solution, the mixed integer linear programming - nonlinear programming (MILP-NLP) strategy is used to solve the problem. This paper uses the linear blending rules plus the blending effect correction to build the gasoline blending model, and a relaxed MILP model is constructed on this basis. The particle swarm optimization algorithm with niche technology (NPSO) is proposed to optimize the solution, and the high-precision soft-sensor method is used to calculate the deviation of gasoline attributes, the blending effect is dynamically corrected to ensure the accuracy of the blending effect and optimization results, thus forming a prediction-verification-reprediction closed-loop scheduling optimization strategy suitable for engineering applications. The optimization result of the MILP model provides a good initial point. By fixing the integer variables to the MILP-optimal value, the approximate MINLP optimal solution can be obtained through a NLP solution. The above solution strategy has been successfully applied to the actual gasoline production case of a refinery (3.5 million tons per year), and the results show that the strategy is effective and feasible. The optimization results based on the closed-loop scheduling optimization strategy have higher reliability. Compared with the standard particle swarm optimization algorithm, NPSO algorithm improves the optimization ability and efficiency to a certain extent, effectively reduces the blending cost while ensuring the convergence speed.

Automated pH Adjustment Driven by Robotic Workflows and Active Machine Learning

Buffer solutions have tremendous importance in biological systems and in formulated products. Whilst the pH response upon acid/base addition to a mixture containing a single buffer can be described by the Henderson-Hasselbalch equation, modelling the pH response for multi-buffered poly-protic systems after acid/base addition, a common task in all chemical laboratories and many industrial plants, is a challenge. Combining predictive modelling and experimental pH adjustment, we present an active machine learning (ML)-driven closed-loop optimization strategy for automating small scale batch pH adjustment relevant for complex samples (e.g., formulated products in the chemical industry). Several ML models were compared on a generated dataset of binary-buffered poly-protic systems and it was found that Gaussian processes (GP) served as the best performing models. Moreover, the implementation of transfer learning into the optimization protocol proved to be a successful strategy in making the process even more efficient. Finally, practical usability of the developed algorithm was demonstrated experimentally with a liquid handling robot where the pH of different buffered systems was adjusted, offering a versatile and efficient strategy for a pH adjustment processes.

Hierarchical closed-loop optimization strategy for cumulative error of robot machining system

Hierarchical closed-loop optimization strategy for cumulative error of robot machining system

Stochastic optimal control — A concise introduction

<p style='text-indent:20px;'>This is a concise introduction to stochastic optimal control theory. We assume that the readers have basic knowledge of real analysis, functional analysis, elementary probability, ordinary differential equations and partial differential equations. We will present the following topics: (ⅰ) A brief presentation of relevant results on stochastic analysis; (ⅱ) Formulation of stochastic optimal control problems; (ⅲ) Variational method and Pontryagin's maximum principle, together with a brief introduction of backward stochastic differential equations; (ⅳ) Dynamic programming method and viscosity solutions to Hamilton-Jacobi-Bellman equation; (ⅴ) Linear-quadratic optimal control problems, including a careful discussion on open-loop optimal controls and closed-loop optimal strategies, linear forward-backward stochastic differential equations, and Riccati equations.</p>

Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability

<p style='text-indent:20px;'>This paper is concerned with mean-field stochastic linear-quadratic (MF-SLQ, for short) optimal control problems with deterministic coefficients. The notion of weak closed-loop optimal strategy is introduced. It is shown that the open-loop solvability is equivalent to the existence of a weak closed-loop optimal strategy. Moreover, when open-loop optimal controls exist, there is at least one of them admitting a state feedback representation, which is the outcome of a weak closed-loop optimal strategy. Finally, an example is presented to illustrate the procedure for finding weak closed-loop optimal strategies.

AI Fast Track to Battery Fast Charge

In the February 20th issue of Nature, William Chueh and colleagues present a closed-loop optimization strategy for the fast charging of battery cells using early cycle life predictions obtained from machine learning models and Bayesian optimization. 1 The developed strategy uses limited testing to obtain substantial improvements in the cycle life of commercial battery cells and aptly demonstrates how machine learning and AI can fast track the performance optimization of battery materials and cells.

Weak Closed-Loop Solvability of Stochastic Linear Quadratic Optimal Control Problems of Markovian Regime Switching System

In this paper, we investigate the open-loop and weak closed-loop solvabilities of stochastic linear quadratic (LQ, for short) optimal control problem of Markovian regime switching system. Interestingly, these two solvabilities are equivalent. We first provide an alternative characterization of the open-loop solvability of the LQ problem using the perturbation approach. Then, we study the weak closed-loop solvability of the LQ problem of Markovian regime switching system, and establish the equivalent relationship between open-loop and weak closed-loop solvabilities. Finally, we present an example to illustrate the procedure for finding weak closed-loop optimal strategies within the framework of Markovian regime switching system.

Smart energy coordination of autonomous residential home

The smart grid technology permits the revolution of the electrical system from a conventional power grid to an intelligent power network which has led the improvements in electrical system in terms of energy efficiency and sustainable energy integration. This study presents the energy management/coordination scheme for domestic demand using the key strategy of smart grid energy efficiency modelling. The structure consists of combining renewable energy resources, photovoltaic (PV) and wind power generation connected to the utility grid with energy storage system (ESS) in an optimal control manner to coordinate the power flow of a residential home. Based on the demand response schemes in the framework of real-time electricity pricing, this work designs a closed-loop optimal control strategy that is created by the dynamic model of the ESS to compute the system performance index, which is formulated by the cost of the energy flows. A dynamic distributed energy storage strategy (DDESS) is implemented to optimally coordinate the energy system, which reduces the total energy consumption from the main grid of more than 100% of the load demand. The designed model introduces a payback scheme while robustly optimising the energy flows and minimising the utility grid's energy consumption cost.

Weak closed-loop solvability of stochastic linear-quadratic optimal control problems

Recently it has been found that for a stochastic linear-quadratic optimal control problem (LQ problem, for short) in a finite horizon, open-loop solvability is strictly weaker than closed-loop solvability which is equivalent to the regular solvability of the corresponding Riccati equation. Therefore, when an LQ problem is merely open-loop solvable not closed-loop solvable, which is possible, the usual Riccati equation approach will fail to produce a state feedback representation of open-loop optimal controls. The objective of this paper is to introduce and investigate the notion of weak closed-loop optimal strategy for LQ problems so that its existence is equivalent to the open-loop solvability of the LQ problem. Moreover, there is at least one open-loop optimal control admitting a state feedback representation. Finally, we present an example to illustrate the procedure for finding weak closed-loop optimal strategies.

Necessary conditions in stochastic linear quadratic problems and their applications

Stochastic linear quadratic optimal control problems are considered. A unified approach is proposed to treat the necessary optimality conditions of closed-loop optimal strategies and open-loop optimal controls. Notice that the former notion does not rely on initial wealth, while the later one does. Our conclusions of closed-loop optimal strategies are directly derived by suitable variational methods, the approach to which is different from [12], [11]. Moreover, the necessary conditions for closed-loop optimal strategies happen to be sufficient which takes us by surprise. Finally, two applications are given as illustration.

Linear–quadratic stochastic two-person nonzero-sum differential games: Open-loop and closed-loop Nash equilibria

In this paper, we consider a linear–quadratic stochastic two-person nonzero-sum differential game. Open-loop and closed-loop Nash equilibria are introduced. The existence of the former is characterized by the solvability of a system of forward–backward stochastic differential equations, and that of the latter is characterized by the solvability of a system of coupled symmetric Riccati differential equations. Sometimes, open-loop Nash equilibria admit a closed-loop representation, via the solution to a system of non-symmetric Riccati equations, which could be different from the outcome of the closed-loop Nash equilibria in general. However, it is found that for the case of zero-sum differential games, the Riccati equation system for the closed-loop representation of an open-loop saddle point coincides with that for the closed-loop saddle point, which leads to the conclusion that the closed-loop representation of an open-loop saddle point is the outcome of the corresponding closed-loop saddle point as long as both exist. In particular, for linear–quadratic optimal control problem, the closed-loop representation of an open-loop optimal control coincides with the outcome of the corresponding closed-loop optimal strategy, provided both exist.

Indefinite Mean-Field Stochastic Linear-Quadratic Optimal Control: From Finite Horizon to Infinite Horizon

In this paper, the finite-horizon and the infinite-horizon indefinite mean-field stochastic linear-quadratic optimal control problems are studied. Firstly, the open-loop optimal control and the closed-loop optimal strategy for the finite-horizon problem are introduced, and their characterizations, difference and relationship are thoroughly investigated. The open-loop optimal control can be defined for a fixed initial state, whose existence is characterized via the solvability of a linear mean-field forward-backward stochastic difference equation with stationary conditions and a convexity condition. On the other hand, the existence of a closed-loop optimal strategy is shown to be equivalent to any one of the following conditions: the solvability of a couple of generalized difference Riccati equations, the finiteness of the value function for all the initial pairs, and the existence of the open-loop optimal control for all the initial pairs. It is then proved that the solution of the generalized difference Riccati equations converges to a solution of a couple of generalized algebraic Riccati equations. By studying another generalized algebraic Riccati equation, the existence of the maximal solution of the original ones is obtained together with the fact that the stabilizing solution is the maximal solution. Finally, we show that the maximal solution is employed to express the optimal value of the infinite-horizon indefinite mean-field linear-quadratic optimal control. Furthermore, for the question whether the maximal solution is the stabilizing solution, the necessary and the sufficient conditions are presented for several cases.

Rapid and robust control of single quantum dots

The combination of single particle detection and ultrafast laser pulses is an instrumental method to track dynamics at the femtosecond time scale in single molecules, quantum dots and plasmonic nanoparticles. Optimal control of the extremely short-lived coherences of these individual systems has so far remained elusive, yet its successful implementation would enable arbitrary external manipulation of otherwise inaccessible nanoscale dynamics. In ensemble measurements, such control is often achieved by resorting to a closed-loop optimization strategy, where the spectral phase of a broadband laser field is iteratively optimized. This scheme needs long measurement times and strong signals to converge to the optimal solution. This requirement is in conflict with the nature of single emitters whose signals are weak and unstable. Here we demonstrate an effective closed-loop optimization strategy capable of addressing single quantum dots at room temperature, using as feedback observable the two-photon photoluminescence induced by a phase-controlled broadband femtosecond laser. Crucial to the optimization loop is the use of a deterministic and robust-against-noise search algorithm converging to the theoretically predicted solution in a reduced amount of steps, even when operating at the few-photon level. Full optimization of the single dot luminescence is obtained within ~100 trials, with a typical integration time of 100 ms per trial. These times are faster than the typical photobleaching times in single molecules at room temperature. Our results show the suitability of the novel approach to perform closed-loop optimizations on single molecules, thus extending the available experimental toolbox to the active control of nanoscale coherences.

Finite-Horizon Indefinite Mean-Field Stochastic Linear-Quadratic Optimal Control

For the finite-horizon indefinite mean-field stochastic linear-quadratic optimal control problems, the open-loop optimal control and the closed-loop optimal strategy are introduced and investigated together with their characterizations, difference and relationship. The open-loop optimal control can be defined for a fixed initial state, whose existence is characterized via the solvability of a linear mean-field forward-backward stochastic difference equations with stationary conditions. Differently, the closed-loop strategy is a global notion, which involves all the initial pairs. The existence of the closed-loop optimal strategy is shown to be equivalent to the solvability of a couple of generalized difference Riccati equations, the finiteness of the value function for all the initial pairs, and the existence of open-loop optimal strategy for all the initial pairs.

A model predictive control strategy for load shifting in a water pumping scheme with maximum demand charges

This paper defines and simulates a closed-loop optimal control strategy for load shifting in a plant that is charged for electricity on both time-of-use (TOU) and maximum demand (MD). A model predictive control approach is used to implement the closed-loop optimal control model, and the optimization problem is solved with integer programming. The simulated control model yields near optimal switching times which reduce the TOU and MD costs. The results show a saving of 5.8% for the overall plant, and the largest portion of the saving is due to a reduction in MD. The effect of disturbances, model uncertainty and plant failure is also simulated to demonstrate the benefits of a model predictive control model.

Open- and Closed-Loop Multiobjective Optimal Strategies for HIV Therapy Using NSGA-II

In this paper, multiobjective open- and closed-loop optimal treatment strategies for HIV/AIDS are presented. It is assumed that highly active antiretroviral therapy is available for treatment of HIV infection. Amount of drug usage and the quality of treatment are defined as two objectives of a biobjective optimization problem, and Nondominated Sorting Genetic Algorithm II is used to solve this problem. Open- and closed-loop control strategies are used to produce optimal control inputs, and the Pareto frontiers obtained from these two strategies are compared. Pareto frontier, resulted from the optimization process, suggests a set of treatment strategies, which all are optimal from a perspective, and can be used in different medical and economic conditions. Robustness of closed-loop system in the presence of measurement noises is analyzed, assuming various levels of noise.

Closed-Loop, Multiobjective Optimization of Two-Dimensional Gas Chromatography/Mass Spectrometry for Serum Metabolomics

Metabolomics seeks to measure potentially all the metabolites in a biological sample, and consequently, we need to develop and optimize methods to increase significantly the number of metabolites we can detect. We extended the closed-loop (iterative, automated) optimization system that we had previously developed for one-dimensional GC-TOF-MS (O'Hagan, S.; Dunn, W. B.; Brown, M.; Knowles, J. D.; Kell, D. B. Anal. Chem. 2005, 77, 290-303) to comprehensive two-dimensional (GCxGC) chromatography. The heuristic approach used was a multiobjective version of the efficient global optimization algorithm. In just 300 automated runs, we improved the number of metabolites observable relative to those in 1D GC by some 3-fold. The optimized conditions allowed for the detection of over 4000 raw peaks, of which some 1800 were considered to be real metabolite peaks and not impurities or peaks with a signal/noise ratio of less than 5. A variety of computational methods served to explain the basis for the improvement. This closed-loop optimization strategy is a generic and powerful approach for the optimization of any analytical instrumentation.

Cascade closed-loop optimization and control of batch reactors

In this paper, a cascade closed-loop optimization and control strategy for batch reactors is proposed. Based on the reduction of a physical conservation model a cascade system is developed, which can effectively combine optimization and control to achieve good on-line optimization and tracking performance under the common condition where incomplete knowledge of the reaction system exists. A two-tier estimation scheme using a nonlinear observer for heat production rate and reaction rates is also developed. In the reaction rate estimation, calorimetric information is used. The on-line closed-loop optimization strategy uses a descending horizon dynamic optimization algorithm based on nonlinear programming and an additive unknown disturbance for feedback. A simple adaptive nonlinear tracking system is designed based on the generic model control concept. The efficiency of this strategy is demonstrated through simulations on a batch reactor under various operation conditions, such as noisy measurements, varying initial states and model mismatch.