Closed-loop Optimization Research Articles

Co-design of active vibration control and optimal sensor and actuator placement for a flexible wing using reinforcement learning

This paper presents applying reinforcement learning to find the optimal sensor/actuator placement (OSAP) policy and optimal control for the flexible wing. The “co-design” objective is to find the OSAP and its associate controller to render the optimal closed-loop performance. The nonlinear vibration dynamics of the flexible wing are modeled in the linear parameter varying (LPV) approach so that LPV- H ∞ controllers can be designed. The co-design problem is formulated into mixed-integer semi-definite programming (MISDP). As a special form of combinatorial optimization, MIDSP solves integer optimization for sensor/actuator selection and convex optimization for controller design. A modified reinforcement learning algorithm is applied to solve this NP-hard optimization problem and obtain a converged solution. In addition, RL is compared with the greedy algorithm and genetic algorithm to demonstrate its strengths and drawbacks in solving high-dimensional MISDP. The solutions obtained by RL and the greedy algorithm are verified and compared in the high-fidelity simulation with the full-order model.

Laser Simultaneous Localization and Mapping Algorithm Based on Adaptive Features and Closed-Loop Optimization

Laser Simultaneous Localization and Mapping Algorithm Based on Adaptive Features and Closed-Loop Optimization

Towards advanced bioprocess optimization: A multiscale modelling approach

Mammalian cells produce up to 80 % of the commercially available therapeutic proteins, with Chinese Hamster Ovary (CHO) cells being the primary production host. Manufacturing involves a train of reactors, the last of which is typically run in fed-batch mode, where cells grow and produce the required protein. The feeding strategy is decided a priori, from either past operations or the design of experiments and rarely considers the current state of the process. This work proposes a Model Predictive Control (MPC) formulation based on a hybrid kinetic-stoichiometric reactor model to provide optimal feeding policies in real-time, which is agnostic to the culture, hence transferable across CHO cell culture systems. The benefits of the proposed controller formulation are demonstrated through a comparison between an open-loop simulation and closed-loop optimization, using a digital twin as an emulator of the process.

Optimization of strategic closed-loop management mechanism for large-scale power grid enterprises adapting to digital transformation

When facing increasingly complex and fast-changing strategic situation, powerful strategic management system is crucial for large-scale power grid enterprises to ensure scientific decision-making, coordinated action and efficient management. This paper analyzes the new strategic management situation of large-scale power grid enterprises under the background of the integrated development of digital revolution and energy revolution. Based on the new situaiton, this paper carries out the demand analysis of the optimization of strategic management mechanism for power grid enterprises, and puts forward the core content of the strategic closed-loop management mechanism optimization. The mechanism optimization will ensure the forward-looking of power grid enterprises’ strategy formulation under the background of digital transformation. Additionally, the mechanism optimization will promote accurate identification of changes, scientific response and effective prevention of strategic risks.

Automated control and optimization of laser-driven ion acceleration

Abstract The interaction of relativistically intense lasers with opaque targets represents a highly non-linear, multi-dimensional parameter space. This limits the utility of sequential 1D scanning of experimental parameters for the optimization of secondary radiation, although to-date this has been the accepted methodology due to low data acquisition rates. High repetition-rate (HRR) lasers augmented by machine learning present a valuable opportunity for efficient source optimization. Here, an automated, HRR-compatible system produced high-fidelity parameter scans, revealing the influence of laser intensity on target pre-heating and proton generation. A closed-loop Bayesian optimization of maximum proton energy, through control of the laser wavefront and target position, produced proton beams with equivalent maximum energy to manually optimized laser pulses but using only 60% of the laser energy. This demonstration of automated optimization of laser-driven proton beams is a crucial step towards deeper physical insight and the construction of future radiation sources.

Substrate specific closed-loop optimization of carbohydrate protective group chemistry using Bayesian optimization and transfer learning.

A new way of performing reaction optimization within carbohydrate chemistry is presented. This is done by performing closed-loop optimization of regioselective benzoylation of unprotected glycosides using Bayesian optimization. Both 6-O-monobenzoylations and 3,6-O-dibenzoylations of three different monosaccharides are optimized. A novel transfer learning approach, where data from previous optimizations of different substrates is used to speed up the optimizations, has also been developed. The optimal conditions found by the Bayesian optimization algorithm provide new insight into substrate specificity, as the conditions found are significantly different. In most cases, the optimal conditions include Et3N and benzoic anhydride, a new reagent combination for these reactions, discovered by the algorithm, demonstrating the power of this concept to widen the chemical space. Further, the developed procedures include ambient conditions and short reaction times.

Autonomous optimization of an organic solar cell in a 4-dimensional parameter space

Herein, we present an autonomous closed-loop optimization of functional OPV devices by optimizing composition and process parameters. An early prediction model of the efficiency from optical featuers significantly decreases the time of one iteration.

CONFIG: Constrained Efficient Global Optimization for Closed-Loop Control System Optimization with Unmodeled Constraints

In this paper, the CONFIG algorithm, a simple and provably efficient constrained global optimization algorithm, is applied to optimize the closed-loop control performance of an unknown system with unmodeled constraints. Existing Gaussian process based closed-loop optimization methods, either can only guarantee local convergence (e.g., SafeOPT), or have no known optimality guarantee (e.g., constrained expected improvement) at all, whereas the recently introduced CONFIG algorithm has been proven to enjoy a theoretical global optimality guarantee. In this study, we demonstrate the effectiveness of CONFIG algorithm in the applications. The algorithm is first applied to an artificial numerical benchmark problem to corroborate its effectiveness. It is then applied to a classical constrained steady-state optimization problem of a continuous stirred-tank reactor. Simulation results show that our CONFIG algorithm can achieve performance competitive with the popular CEI (Constrained Expected Improvement) algorithm, which has no known optimality guarantee. As such, the CONFIG algorithm offers a new tool, with both a provable global optimality guarantee and competitive empirical performance, to optimize the closed-loop control performance for a system with soft unmodeled constraints. Last, but not least, the open-source code is available as a python package to facilitate future applications.

A Real-Time Gaussian Process-Based Stochastic Controller for Periodic Disturbances

Using a non-parametric Gaussian process regression (GPR) framework, we present in this paper a practical learning technique to model, filter, and predict periodic disturbances. Augmenting a known system model with the GP model of the disturbance results in a structure referred to as latent force model (LFM), which is also a GP with an appropriate covariance function. By representing the LFM as a stochastic state-space model using spectral factorization, we apply a full Bayesian inference through Kalman filter and smoother, and design a linear quadratic regulator (LQR) to achieve an optimal closed-loop performance. On a real-time application, LJ MS15 DC motor under induced periodic disturbances, the proposed approach is implemented successfully. Real-time experiments demonstrate the efficiency and practicality of the proposed LFM-based LQR compared to the classical LQR approach.

Macroscopic hyperpolarization enhanced with quantum optimal control

Hyperpolarization of nuclear spins enhances nuclear magnetic resonance signals, which play a key role for imaging and spectroscopy in the natural and life sciences. This signal amplification unlocks previously inaccessible techniques, such as metabolic imaging of cancer cells. In this work, electron spins from the photoexcited triplet state of pentacene-doped naphthalene crystals are used to polarize surrounding protons. As existing strategies are rendered less effective by experimental constraints, they are replaced with optimal control pulses designed with RedCRAB. In contrast to previous optimal control approaches, which consider an average single nucleus, this closed-loop optimization is macroscopic. A 28% improvement in signal and 15% faster polarization rate is observed. Additionally, a strategy called Autonomously-optimized Repeated Linear Sweep (ARISE) is introduced to efficiently tailor existing hyperpolarization sequences in the presence of experimental uncertainty to enhance their performance. ARISE is expected to be broadly applicable in many experimental settings.

Scaling of the quantum approximate optimization algorithm on superconducting qubit based hardware

Quantum computers may provide good solutions to combinatorial optimization problems by leveraging the Quantum Approximate Optimization Algorithm (QAOA). The QAOA is often presented as an algorithm for noisy hardware. However, hardware constraints limit its applicability to problem instances that closely match the connectivity of the qubits. Furthermore, the QAOA must outpace classical solvers. Here, we investigate swap strategies to map dense problems into linear, grid and heavy-hex coupling maps. A line-based swap strategy works best for linear and two-dimensional grid coupling maps. Heavy-hex coupling maps require an adaptation of the line swap strategy. By contrast, three-dimensional grid coupling maps benefit from a different swap strategy. Using known entropic arguments we find that the required gate fidelity for dense problems lies deep below the fault-tolerant threshold. We also provide a methodology to reason about the execution-time of QAOA. Finally, we present a QAOA Qiskit Runtime program and execute the closed-loop optimization on cloud-based quantum computers with transpiler settings optimized for QAOA. This work highlights some obstacles to improve to make QAOA competitive, such as gate fidelity, gate speed, and the large number of shots needed. The Qiskit Runtime program gives us a tool to investigate such issues at scale on noisy superconducting qubit hardware.

Data-driven model predictive control for closed-loop refracturing design and optimization in naturally fractured shale gas reservoir under geological uncertainty

The reliable and predictive control for closed-loop refracturing design and optimization is of significance to improve the unconventional resource development and recovery in petroleum engineering. To reduce refracturing-decision risk under uncertainty for unconventional shale gas reservoir, we present an efficient and robust two-stage refracturing optimization workflow through hybridizing data-space inversion and non-dominated sorting genetic algorithm-II. Without performing history matching step, we adapt data-space inversion to simultaneously predict post-history pressure field and shale gas production corresponding to specific refracturing parameters given history measurement. In the proposed framework, both the enhanced net present value and enhanced cumulative gas production after refracturing stimulation are regarded as the bi-objective functions while the refracturing parameters including fracture number and fracture half-length are chosen as the decision variables. The proposed two-stage refracturing optimization strategy enables us to effectively identify the potential candidate clusters and/or wells and then search the optimal refracturing parameters efficiently and reasonably.

A closed-loop data-driven optimization framework for the unit commitment problem: A [formula omitted]-learning approach under real-time operation

Real-time operation of electric power systems under high penetration of renewable energy generation has to incorporate strategies for managing the uncertainty associated with this type of power sources. In the case of the unit commitment problem, models based on robust optimization have been widely used for the day-ahead computation of power dispatch and reserves schedule. Typical approaches use uncertainty sets with fixed levels of conservativeness, without considering the real-time performance of the solutions. This paper proposes a solution scheme for the adaptive robust unit commitment problem using online adjusted data-driven uncertainty sets. Conservativeness parameters of the uncertainty sets are dynamically calculated as a function of previous operating results and incoming data via reinforcement learning, resulting in an online learning-optimization framework. The paper also develops an experimental framework to simulate real-time operation under the proposed methodology. Out-of-sample experiments illustrate the effectiveness of the proposed scheme against well-known benchmarks with fixed robustness levels. Results show improvements in power generation costs, voltage violations, and use of reserves. Two systems of different sizes are analyzed, illustrating the scalability of the proposed approach.

IffDetector: Inference-Aware Feature Filtering for Object Detection.

Modern convolutional neural network (CNN)-based object detectors focus on feature configuration during training but often ignore feature optimization during inference. In this article, we propose a new feature optimization approach to enhance features and suppress background noise in both the training and inference stages. We introduce a generic inference-aware feature filtering (IFF) module that can be easily combined with existing detectors, resulting in our iffDetector. Unlike conventional open-loop feature calculation approaches without feedback, the proposed IFF module performs the closed-loop feature optimization by leveraging high-level semantics to enhance the convolutional features. By applying the Fourier transform to analyze our detector, we prove that the IFF module acts as a negative feedback that can theoretically guarantee the stability of the feature learning. IFF can be fused with CNN-based object detectors in a plug-and-play manner with little computational cost overhead. Experiments on the PASCAL VOC and MS COCO datasets demonstrate that our iffDetector consistently outperforms state-of-the-art methods with significant margins.

Closed-Loop Solvability of Stochastic Linear-Quadratic Optimal Control Problems with Poisson Jumps

The stochastic linear–quadratic optimal control problem with Poisson jumps is addressed in this paper. The coefficients in the state equation and the weighting matrices in the cost functional are all deterministic but are allowed to be indefinite. The notion of closed-loop strategies is introduced, and the sufficient and necessary conditions for the closed-loop solvability are given. The optimal closed-loop strategy is characterized by a Riccati integral–differential equation and a backward stochastic differential equation with Poisson jumps. A simple example is given to demonstrate the effectiveness of the main result.

Multiple input algorithm-guided Deep Brain stimulation-programming for Parkinson’s disease patients

Technological advances of Deep Brain Stimulation (DBS) within the subthalamic nucleus (STN) for Parkinson’s disease (PD) provide increased programming options with higher programming burden. Reducing the effort of DBS optimization requires novel programming strategies. The objective of this study was to evaluate the feasibility of a semi-automatic algorithm-guided-programming (AgP) approach to obtain beneficial stimulation settings for PD patients with directional DBS systems. The AgP evaluates iteratively the weighted combination of sensor and clinician assessed responses of multiple PD symptoms to suggested DBS settings until it converges to a final solution. Acute clinical effectiveness of AgP DBS settings and DBS settings that were found following a standard of care (SoC) procedure were compared in a randomized, crossover and double-blind fashion in 10 PD subjects from a single center. Compared to therapy absence, AgP and SoC DBS settings significantly improved (p = 0.002) total Unified Parkinson’s Disease Rating Scale III scores (median 69.8 interquartile range (IQR) 64.6|71.9% and 66.2 IQR 58.1|68.2%, respectively). Despite their similar clinical results, AgP and SoC DBS settings differed substantially. Per subject, AgP tested 37.0 IQR 34.0|37 settings before convergence, resulting in 1.7 IQR 1.6|2.0 h, which is comparable to previous reports. Although AgP long-term clinical results still need to be investigated, this approach constitutes an alternative for DBS programming and represents an important step for future closed-loop DBS optimization systems.

Closed-loop optimization of general reaction conditions for heteroaryl Suzuki-Miyaura coupling

General conditions for organic reactions are important but rare, and efforts to identify them usually consider only narrow regions of chemical space. Discovering more general reaction conditions requires considering vast regions of chemical space derived from a large matrix of substrates crossed with a high-dimensional matrix of reaction conditions, rendering exhaustive experimentation impractical. Here, we report a simple closed-loop workflow that leverages data-guided matrix down-selection, uncertainty-minimizing machine learning, and robotic experimentation to discover general reaction conditions. Application to the challenging and consequential problem of heteroaryl Suzuki-Miyaura cross-coupling identified conditions that double the average yield relative to a widely used benchmark that was previously developed using traditional approaches. This study provides a practical road map for solving multidimensional chemical optimization problems with large search spaces.

Magnetic torque–driven living microrobots for increased tumor infiltration

Biohybrid bacteria-based microrobots are increasingly recognized as promising externally controllable vehicles for targeted cancer therapy. Magnetic fields in particular have been used as a safe means to transfer energy and direct their motion. Thus far, the magnetic control strategies used in this context rely on poorly scalable magnetic field gradients, require active position feedback, or are ill-suited to diffuse distributions within the body. Here, we present a magnetic torque-driven control scheme for enhanced transport through biological barriers that complements the innate taxis toward tumor cores exhibited by a range of bacteria, shown for Magnetospirillum magneticum as a magnetically responsive model organism. This hybrid control strategy is readily scalable, independent of position feedback, and applicable to bacterial microrobots dispersed by the circulatory system. We observed a fourfold increase in translocation of magnetically responsive bacteria across a model of the vascular endothelium and found that the primary mechanism driving increased transport is torque-driven surface exploration at the cell interface. Using spheroids as a three-dimensional tumor model, fluorescently labeled bacteria colonized their core regions with up to 21-fold higher signal in samples exposed to rotating magnetic fields. In addition to enhanced transport, we demonstrated that our control scheme offers further advantages, including the possibility for closed-loop optimization based on inductive detection, as well as spatially selective actuation to reduce off-target effects. Last, after systemic intravenous injection in mice, we showed significantly increased bacterial tumor accumulation, supporting the feasibility of deploying this control scheme clinically for magnetically responsive biohybrid microrobots.

Model predictive control for robust quantum state preparation

A critical engineering challenge in quantum technology is the accurate control of quantum dynamics. Model-based methods for optimal control have been shown to be highly effective when theory and experiment closely match. Consequently, realizing high-fidelity quantum processes with model-based control requires careful device characterization. In quantum processors based on cold atoms, the Hamiltonian can be well-characterized. For superconducting qubits operating at milli-Kelvin temperatures, the Hamiltonian is not as well-characterized. Unaccounted for physics (i.e., mode discrepancy), coherent disturbances, and increased noise compromise traditional model-based control. This work introduces model predictive control (MPC) for quantum control applications. MPC is a closed-loop optimization framework that (i) inherits a natural degree of disturbance rejection by incorporating measurement feedback, (ii) utilizes finite-horizon model-based optimizations to control complex multi-input, multi-output dynamical systems under state and input constraints, and (iii) is flexible enough to develop synergistically alongside other modern control strategies. We show how MPC can be used to generate practical optimized control sequences in representative examples of quantum state preparation. Specifically, we demonstrate for a qubit, a weakly-anharmonic qubit, and a system undergoing crosstalk, that MPC can realize successful model-based control even when the model is inadequate. These examples showcase why MPC is an important addition to the quantum engineering control suite.

A novel framework for online decision-making and feedback optimization of complex products process parameter based on edge-cloud collaboration

Background: Intelligent manufacturing is perceived as a manufacturing mode with powerful learning and cognitive capabilities empowered by information technologies such as the internet of things, edge computing, and cloud computing. The mode can be used to address the problems of low intelligence and poor timeliness of traditional process planning. Methods: The framework includes the multi-objective process planning method based on real-time data, and the process closed-loop optimization mechanism of “cloud-based theoretical process planning plus edge MEC (multi-access edge computing) side online simulation verification and real-time feedback adjustment”, which realizes online process planning and iterative optimization in mass customization. Results: The feasibility of the online analysis method for thin-walled part milling deformation is verified by taking the finishing process of aerospace thin-walled parts as an example. The experimental results show that the simulation time on the single analysis step is reduced from 6s to 1s, and the accuracy rate is 86.9%. Conclusions: A new intelligent process planning theoretical framework integrating with online process planning and autonomous collaborative control, namely, digital twin and multi-access edge computing process planning (DT-MEC-PP) is proposed in this paper.