Closed-loop Optimization Research Articles

A closed-loop maintenance strategy for offshore wind farms: Incorporating dynamic wind farm states and uncertainty-awareness in decision-making

The determination of maintenance strategies is subject to complexity and uncertainty arising from variable offshore wind farm states and inaccuracies in model parameters. The most common method in the existing studies is to adopt an open-loop approach to optimize a maintenance strategy. However, this approach lacks the ability to capture periodic operational state of the wind farm and the awareness of eliminating uncertainty. Consequently, the determined strategy is inadequate to instruct maintenance activities, inducing excessive revenue losses. In this paper, a closed-loop maintenance strategy optimization method is proposed for decision-makers to identify a more profitable manner of wind farm maintenance management. The life-cycle maintenance optimization problem is decomposed into a sequence of sub-optimization problems covering multiple time periods by using a rolling-horizon approach. Each sub-optimization problem is intentionally designed based on the monitored state of the wind farm and the available reliability, availability, and maintainability (RAM) database. Meanwhile, the decision maker consciously mitigates the parameter uncertainty in the maintenance model gradually by updating the current database. Compared to conventional strategies covering the entire lifetime of wind farms, the proposed maintenance strategy is periodically adjusted to provide a series of sub-strategies. The proposed approach was applied in a simulation experiment, a generic small-scale offshore wind farm, to assess its performance. Computational results show that adapting maintenance strategies based on the current state of the wind farm can reduce revenue losses in comparison to conventional open-loop strategies. In addition, the benefits of updating the RAM database in decreasing revenue losses is revealed.

Predictive control of reactor network model using machine learning for hydrogen-rich gas and biochar poly-generation by biomass waste gasification in supercritical water

Supercritical water gasification (SCWG) technology can convert biomass into hydrogen rich gas and biochar. Fluidized bed reactor is promising for the industrialization of this technology, and the reactor dynamic performance study is of great significance for its scaling up. However, current simulation studies mainly focus on steady-state analysis using Computational Fluid Dynamics (CFD) software, it is difficult to conduct dynamic study due to its high computational costs. To this end, a reactor network model (RNM) that accounts for the flow, heat transfer, and kinetic dynamics for fluidized bed reactor is firstly developed based on the partitioning theory, which can reduce the computation time of dynamic simulation from several days to seconds. Additionally, extensive open-loop simulations of the RNM are carried out to generate the dataset for the development of a recurrent neural network (RNN) model. Additionally, current studies mainly focus on open-loop simulation, closed loop optimization is absent. To this end, a framework for building a machine learning (ML) model and a ML-based nonlinear predictive control scheme is developed. Model predictive control (MPC) schemes based on the RNN model is used to optimize the SCWG process to achieve multiple objectives, such as maximizing hydrogen yield and carbon yield. The open loop simulation results demonstrate that H2 yield decreases from 0.00022 to 0.0002 kg s−1 with temperature reducing from 700 to 600 °C. To increase H2 yield to the setpoint, MPC increases temperature and mass flowrate to 708 °C and 417.57 kg h−1. Additionally, MPC increases carbon yield and minimizing CO2 yield by decreasing temperature to 475 °C and increasing mass flowrate to 407.38 kg h−1.

Real-time adaptive sparse-identification-based predictive control of nonlinear processes

This study introduces a sparse identification-based model predictive control (MPC) framework that incorporates on-line updates of the sparse-identified model to account for nonlinear dynamics and model uncertainty in process systems. The methodology involves obtaining a nonlinear first-order ordinary differential equation model using sparse identification for nonlinear dynamics (SINDy), which is integrated into two control schemes: Lyapunov-based MPC (LMPC) for achieving steady-state operation and Lyapunov-based economic MPC (LEMPC) for achieving both closed-loop stability and optimal economic performance. To improve prediction accuracy, an on-line model update scheme is proposed for the SINDy models. Specifically, an error-trigger mechanism that utilizes prediction errors and then uses the most recent process data to update the parameters of the SINDy model in real-time is designed. By incorporating the error-triggered on-line model updates in the SINDy-based LMPC and LEMPC, the dynamic performance of the process is enhanced, ensuring closed-loop stability, optimality, and smooth control actions. Following theoretical results on the boundedness of the closed-loop states and detailed discussions on the selection criteria for parameters of the error-triggered SINDy update scheme, the effectiveness of the proposed methodology is demonstrated through a chemical process example with time-varying disturbances under the LEMPC framework.

A robust model predictive control with constraint modification for gas lift allocation optimization

This paper presents a novel approach for implementing a robust real-time optimization framework under the presence of parametric uncertainty. Conservativeness is an inevitable drawback of a robust control approach. Therefore we aimed to provide a simple and efficient method to mitigate the conservativeness while the robust fulfillment of the constraints is still preserved. The proposed method in this paper is based on the worst-case realization of the uncertainties, however, with constraint modification. The mismatch between measured and predicted output is used directly to modify the active constraint in the optimization problem. The superiority of the method in terms of conservativeness and computational time has been demonstrated in comparison with the other robust optimization counterparts, such as traditional min–max and multi-stage MPC. The promising advantage of the proposed method is that not only it reduces the conservativeness significantly, but also the computational price for this achievement is considerably cheaper than closed-loop optimization methods such as multi-stage MPC.

Feature-Driven Closed-Loop Optimization for Battery Fast Charging Design with Machine Learning

Electric vehicle batteries must possess fast rechargeability. However, fast charging of lithium-ion batteries remains a great challenge. This paper develops a feature-driven closed-loop optimization (CLO) methodology to efficiently design health-conscious fast-charging strategies for batteries. To avoid building an early outcome predictor, the feature highly related to battery end-of-life is used as the optimization objective instead of using the predicted lifetime. This feature is extracted from the battery’s early cycles and the experimental cost is thus reduced. By developing closed-loop multi-channel experiments with Bayesian optimization (BO), the optimal charging protocols with long cycle lives are located quickly and efficiently among 224 four-step, 10 min fast-charging protocols. Experimental results show that BO performs well with different acquisition functions, and a minimum of 12 paralleled channels for each round of experiments are recommended to obtain stable optimization results. Compared with the benchmark, the developed method recommends similar fast-charging protocols with long cycle lives based on much less experimental cost.

Counterfactual safety benefits quantification method for en-route driving behavior interventions

Driving behavior intervention is a dominant traffic safety countermeasure being implemented that has substantially reduced crash occurrence. However, during implementation, the intervention strategy faces the curse of dimensionality as there are multiple candidate intervention locations with various intervention measures and options. Quantifying the interventions’ safety benefits and further implementing the most effective ones could avoid too frequent interventions which may lead to counterproductive safety impacts. Traditional intervention effects quantification approaches rely on observational data, thus failing to control confounding variables and leading to biased results. In this study, a counterfactual safety benefits quantification method for en-route driving behavior interventions was proposed. Empirical data from online ride-hailing services were employed to quantify the safety benefits of en-route safety broadcasting to speed maintenance behavior. Specifically, to effectively control the impacts of confounding variables on the quantification results of interventions, the “if without intervention” case of the intervention case is inferred based on the structural causality model according to the Theory of Planned Behavior (TPB). Then, a safety benefits quantification method based on Extreme Value Theory (EVT) was developed to connect changes of speed maintenance behavior with crash occurrence probabilities. Furthermore, a closed-loop evaluation and optimization framework for the various behavior interventions was established and applied to a subset of Didi’s online ride-hailing service drivers (more than 1.35 million). Analyses results indicated safety broadcasting could effectively reduce driving speed by approximately 6.30 km/h and contribute to an approximate 40% reduction in speeding-related crashes. Besides, empirical application results showed that the whole framework contributed to a remarkable reduction in the fatality rate per 100 million km, from an average of 0.368 to 0.225. Finally, directions for future research in terms of data, counterfactual inference methodology, and research subjects have been discussed.

QuOCS: The quantum optimal control suite

Quantum optimal control includes a family of pulse-shaping algorithms that aim to unlock the full potential of a variety of quantum technologies. The Quantum Optimal Control Suite (QuOCS) unites experimental focus and model-based approaches in a unified framework. Easy usage and installation presented here and the availability of various combinable optimization strategies is designed to improve the performance of many quantum technology platforms, such as color defects in diamond, superconducting qubits, atom- or ion-based quantum computers. It can also be applied to the study of more general phenomena in physics. In this paper, we describe the software and the toolbox of gradient-free and gradient-based algorithms. We then show how the user can connect it to their experiment. In addition, we provide illustrative examples where our optimization suite solves typical quantum optimal control problems, in both open- and closed-loop settings. Integration into existing experimental control software is already provided for the experiment control software Qudi (Binder et al., 2017 [41]), and further extensions are investigated and highly encouraged.QuOCS is available from GitHub, under Apache License 2.0, and can be found on the PyPI repository. Program summaryProgram Title: QuOCS - Quantum Optimal Control SuiteCPC Library link to program files:https://doi.org/10.17632/wjjch757fk.1Developer's repository link:https://github.com/Quantum-OCS/QuOCSLicensing provisions: Apache-2.0Programming language: PythonExternal routines: NumPy [1], SciPy [1], JAX [2]Nature of problem: Quantum systems are typically controlled by time-dependent electromagnetic fields to perform a certain set of quantum operations. Those operations may in turn provide building blocks for various quantum information processing tasks such as quantum computation, communication, simulation, sensing, and metrology. Numerous control strategies exist to design and improve such operations [3]. While some strategies are constructed to target a rather specific problem with high efficiency, others are more general to solve a wide range of applications [4,5]. To access the different algorithms, one has to download different optimization suites with different input and output parameters, making them hard to compare and combine. To benefit from the variety of algorithms, we have devised a customizable and intuitive optimization suite that simultaneously provides access to some of the most popular quantum optimal control algorithms.Solution method: We combine, in a unified framework, some of the frequently used optimal control algorithms which are the dressed Chopped Random Basis method (dCRAB) [6], and Gradient Ascent Pulse Engineering (GRAPE) [7], with an extension to make use of Automatic Differentiation (AD) [8]. The software is able to connect to both models of quantum dynamics in simulations and real-time quantum experiments to perform open- and closed-loop optimization, respectively. With minimal knowledge of optimal control theory, the user can manage to run optimizations of quantum processes using a variety of additional features such as stopping criteria and drift compensation. Logging and data management of the optimization progress and results are also handled by the suite. Its modular structure allows for extensions that accommodate customized algorithms and can be implemented by the user straightforwardly.Additional comments including unusual features: The connection to the experiments is performed by an extension that enables a direct integration to a laboratory control software Qudi [9].

A Periodically Updated Adaptive Sampling Framework for Marine Mobile Observation Platforms

Abstract While numerical models have been developed for several years, some of these have been applied to ocean-state sampling. Adaptive sampling deploys limited assets using prior information; then, observation assets are concentrated in areas of greater sampling value, which is very suitable for an extensive and dynamic marine environment. The improved resolution allows numerical models to be used on mobile platforms. However, the existing adaptive sampling framework for mobile platforms lacks regular interaction with the numerical model. And the observation scheme is easy to deviate from the optimal. This study sets up a closed-loop adaptive sampling framework for mobile platforms that realizes the optimization of model → sampling → model. Linking a coupled model with the sampling points of the mobile platforms, the adaptive method configures key sampling locations to determine when and where the sampling schemes are adjusted. With the aid of a coupled model, we selected an optimization algorithm for the framework and simulated the process under the twin experimental framework. This research provides theoretical technical support for the combination of model and mobile sampling platforms. Significance Statement The ocean is very difficult to observe because it is so vast. How to deploy observing assets is a question worth investigating. We designed an adaptive sampling framework for mobile observing platforms to improve the prediction accuracy of numerical models. This framework uses the prediction to design the observation scheme, then the observation to update the prediction, and then the updated prediction to adjust the sampling scheme, which achieves closed-loop optimization. We first selected an optimization algorithm for this framework by comparing experiments. Based on the optimization algorithm, we simulated the closed-loop optimization process. Simulation results show that this framework can significantly increase the efficiency of long-term ocean observations with mobile devices.

A self-driving laboratory optimizes a scalable process for making functional coatings

Functional coatings are used in a wide range of high surface-area technologies, such as low-E windows and photovoltaics. Solution-based coatings are typically less expensive to produce than vacuum-based coatings; however, it is generally more difficult to produce high-quality coatings using solution-based methods due to lower control over the physical and chemical processes involved. Here, we show how a self-driving laboratory can be used to optimize spray coating parameters. For this demonstration, we optimized the combustion synthesis of spray-cast conductive palladium (Pd) films. The closed-loop optimization yielded films with conductivities of >4 MS/m, which compares favorably with the conductivities of 2–6 MS/m reported for thin Pd films obtained by vacuum-based sputtering processes. The champion coating conditions were scaled up to an 8× larger area using the same spray-coating apparatus while preserving coating quality and conductivity. This work shows how self-driving laboratories can optimize solution-based coatings at scale.

Deep learning based closed-loop well control optimization of geothermal reservoir with uncertain permeability

To maximize the economic benefits of geothermal energy production, it is essential to optimize geothermal reservoir management strategies, in which geologic uncertainty should be considered. In this work, we propose a closed-loop optimization framework, based on deep learning surrogates, for the well control optimization of geothermal reservoirs. In this framework, we construct a hybrid convolution–recurrent neural network surrogate, which combines the convolution neural network (CNN) and long short-term memory (LSTM) recurrent network. The convolution structure can extract spatial information of reservoir property fields and the recurrent structure can approximate sequence-to-sequence mapping. The trained model can predict time-varying production responses (rate, temperature, etc.) for cases with different permeability fields and well control sequences. In this closed-loop optimization framework, production optimization, based on the differential evolution (DE) algorithm, and data assimilation, based on the iterative ensemble smoother (IES), are performed alternately to achieve a real-time well control optimization and to estimate reservoir properties (e.g. permeability) as the production proceeds. In addition, the averaged objective function over the ensemble of geologic parameter estimates is adopted to consider geologic uncertainty in the optimization process. Geothermal reservoir production cases are examined to evaluate the performance of the proposed closed-loop optimization framework. Our results show that the proposed framework can achieve efficient and effective real-time optimization and data assimilation in the geothermal reservoir production process.

Automatic Information Model Generation for Industrial Edge Applications Based on IEC 61499 and OPC UA

The Industry 4.0 and Industrial Internet provides vertical and horizontal integration between edge devices and industrial cloud platforms. A flexible and interoperable information model is crucial for enabling device-level intelligence as well as cloud-edge collaboration. Deep integration between the control and information model is necessary for supporting closed-loop optimization between the cloud and edge devices. This article proposes an automatic information model generation method based on the IEC 61499 and OPC unified architecture standards. Model transformation rules between two models from design time to runtime are presented. A discrete manufacturing case study is used to prove that the proposed method can significantly reduce the development time for distributed information models. The integrated control and information models at runtime can provide strong support for dynamic reconfiguration to address changing requirements.

Closed-loop scheduling optimization strategy based on particle swarm optimization with niche technology and soft sensor method of attributes-applied to gasoline blending process

Gasoline blending scheduling optimization can bring significant economic and efficient benefits to refineries. However, the optimization model is complex and difficult to build, which is a typical mixed integer nonlinear programming (MINLP) problem. Considering the large scale of the MINLP model, in order to improve the efficiency of the solution, the mixed integer linear programming - nonlinear programming (MILP-NLP) strategy is used to solve the problem. This paper uses the linear blending rules plus the blending effect correction to build the gasoline blending model, and a relaxed MILP model is constructed on this basis. The particle swarm optimization algorithm with niche technology (NPSO) is proposed to optimize the solution, and the high-precision soft-sensor method is used to calculate the deviation of gasoline attributes, the blending effect is dynamically corrected to ensure the accuracy of the blending effect and optimization results, thus forming a prediction-verification-reprediction closed-loop scheduling optimization strategy suitable for engineering applications. The optimization result of the MILP model provides a good initial point. By fixing the integer variables to the MILP-optimal value, the approximate MINLP optimal solution can be obtained through a NLP solution. The above solution strategy has been successfully applied to the actual gasoline production case of a refinery (3.5 million tons per year), and the results show that the strategy is effective and feasible. The optimization results based on the closed-loop scheduling optimization strategy have higher reliability. Compared with the standard particle swarm optimization algorithm, NPSO algorithm improves the optimization ability and efficiency to a certain extent, effectively reduces the blending cost while ensuring the convergence speed.

Optimization of the model predictive control meta-parameters through reinforcement learning

Model predictive control (MPC) is increasingly being considered for control of fast systems and embedded applications. However, MPC has some significant challenges for such systems, such as its high computational complexity. Further, the MPC parameters must be tuned, which is largely a trial-and-error process that affects the control performance, the robustness, and the computational complexity of the controller to a high degree. This paper presents a multivariate optimization method based on reinforcement learning (RL) that automatically tunes the control algorithm’s parameters from data to achieve optimal closed-loop performance. The main contribution of our method is the inclusion of state-dependent optimization of the meta-parameters of MPC, i.e. parameters that are non-differentiable wrt. the MPC solution. Our control algorithm is based on an event-triggered MPC, where we learn when the MPC should be re-computed, and a dual-mode MPC and linear state feedback control law applied in between MPC computations. We formulate a novel mixture-distribution RL policy determining the meta-parameters of our control algorithm and show that with joint optimization we achieve improvements that do not present themselves with univariate optimization of the same parameters. We demonstrate our framework on the inverted pendulum control task, reducing the total computation time of the control system by 36% while also improving the control performance by 18.4%.

Bayesian-Based Hybrid Method for Rapid Optimization of NV Center Sensors.

NV centers are among the most promising platforms in the field of quantum sensing. Magnetometry based on NV centers, especially, has achieved concrete development in areas of biomedicine and medical diagnostics. Improving the sensitivity of NV center sensors under wide inhomogeneous broadening and fieldamplitude drift is a crucial issue of continuous concern that relies on the coherent control of NV centers with high average fidelity. Quantum optimal control (QOC) methods provide access to this target; nevertheless, the high time consumption of current methods due to the large number of needful sample points as well as the complexity of the parameter space has hindered their usability. In this paper, we propose the Bayesian estimation phase-modulated (B-PM) method to tackle this problem. In the case of the state transforming of an NV center ensemble, the B-PM method reduced the time consumption by more than 90% compared with the conventional standard Fourier basis (SFB) method while increasing the average fidelity from 0.894 to 0.905. In the AC magnetometry scenario, the optimized control pulse obtained with the B-PM method achieved an eight-fold extension of coherence time T2 compared with the rectangular π pulse. Similar application can be made in other sensing situations. As a general algorithm, the B-PM method can be further extended to the open- and closed-loop optimization of complex systems based on a variety of quantum platforms.

An exploratory architecture using data and knowledge to close the loop between design and Maintenance from a product quality perspective

Products for personalization require frequent dynamic interactions at all stages of the entire lifecycle. Based on this, the researchers have been studying closed-loop product lifecycle management. However, most of them often focus on the strategic and conceptual levels and ignore the technical details of implementation. From a product quality perspective, a three-tier architecture using data and knowledge to close the loop between design, operation, and maintenance (O&M) is proposed to deal with this problem. FMECA automation enabled by industrial knowledge graph and construction of Bayesian network for failure propagation is introduced in detail to enhance efficient integration of design and maintenance phases. A proof-of-concept simulation based on an electromechanical actuator validates the applicability and efficiency of the proposed method, showing a closed-loop optimization of design and O&M services. This article potentially enables manufacturers to implement the linkage of design and O&M service business.

Closed-loop optimization of piezoelectric sensors locations and gains for output feedback active vibration control

Closed-loop optimization of piezoelectric sensors locations and gains for output feedback active vibration control

Open-Source Chromatographic Data Analysis for Reaction Optimization and Screening.

Automation and digitalization solutions in the field of small molecule synthesis face new challenges for chemical reaction analysis, especially in the field of high-performance liquid chromatography (HPLC). Chromatographic data remains locked in vendors' hardware and software components, limiting their potential in automated workflows and data science applications. In this work, we present an open-source Python project called MOCCA for the analysis of HPLC-DAD (photodiode array detector) raw data. MOCCA provides a comprehensive set of data analysis features, including an automated peak deconvolution routine of known signals, even if overlapped with signals of unexpected impurities or side products. We highlight the broad applicability of MOCCA in four studies: (i) a simulation study to validate MOCCA's data analysis features; (ii) a reaction kinetics study on a Knoevenagel condensation reaction demonstrating MOCCA's peak deconvolution feature; (iii) a closed-loop optimization study for the alkylation of 2-pyridone without human control during data analysis; (iv) a well plate screening of categorical reaction parameters for a novel palladium-catalyzed cyanation of aryl halides employing O-protected cyanohydrins. By publishing MOCCA as a Python package with this work, we envision an open-source community project for chromatographic data analysis with the potential of further advancing its scope and capabilities.

Estimation of temperature, local and global radiation of flames, using retrieved hyperspectral imaging

A flame emits energy over a wide spectral region, providing important information about the combustion process. Its associated spectra, classified into continuous and discontinuous features, provide information concerning the temperature of the soot particle and atomic or molecular reactions. However, because of the non-linear and turbulent nature of the flame, added to the high-dimensional nature of the spectral data and the overlap between continuous and discontinuous spectral emissions, complex methods are needed to obtain valuable information for combustion closed-loop control and optimization. In this paper, we propose a method for retrieval of hyperspectral flame images from Liquefied Petroleum Gas (LPG) fuel, using only images captured by a trichromatic camera, but combined with a spectral recovery method. The hyperspectral flame data recovered was used to estimate indexes, such as the flame temperature (in K), the local emitted radiance (at the pixel level) and the global radiance (both in μW/cm2), thus performing a dynamic, spectral and spatial combustion diagnosis in different air/fuel ratios. Our method was validated by performing measurements with a calibrated spectrophotometer, achieving small errors in the retrieval procedure (less than 1%) and in the temperature estimation (less than 4%). The results show that our approach can play an essential role in flame sensing, providing an important tool for monitoring or controlling combustion processes.

Co-design of active vibration control and optimal sensor and actuator placement for a flexible wing using reinforcement learning

This paper presents applying reinforcement learning to find the optimal sensor/actuator placement (OSAP) policy and optimal control for the flexible wing. The “co-design” objective is to find the OSAP and its associate controller to render the optimal closed-loop performance. The nonlinear vibration dynamics of the flexible wing are modeled in the linear parameter varying (LPV) approach so that LPV- H ∞ controllers can be designed. The co-design problem is formulated into mixed-integer semi-definite programming (MISDP). As a special form of combinatorial optimization, MIDSP solves integer optimization for sensor/actuator selection and convex optimization for controller design. A modified reinforcement learning algorithm is applied to solve this NP-hard optimization problem and obtain a converged solution. In addition, RL is compared with the greedy algorithm and genetic algorithm to demonstrate its strengths and drawbacks in solving high-dimensional MISDP. The solutions obtained by RL and the greedy algorithm are verified and compared in the high-fidelity simulation with the full-order model.

Laser Simultaneous Localization and Mapping Algorithm Based on Adaptive Features and Closed-Loop Optimization

Laser Simultaneous Localization and Mapping Algorithm Based on Adaptive Features and Closed-Loop Optimization