Online Optimization Method Research Articles

Bandit Linear Optimization for Sequential Decision Making and Extensive-Form Games

Tree-form sequential decision making (TFSDM) extends classical one-shot decision making by modeling tree-form interactions between an agent and a potentially adversarial environment. It captures the online decision-making problems that each player faces in an extensive-form game, as well as Markov decision processes and partially-observable Markov decision processes where the agent conditions on observed history. Over the past decade, there has been considerable effort into designing online optimization methods for TFSDM. Virtually all of that work has been in the full-feedback setting, where the agent has access to counterfactuals, that is, information on what would have happened had the agent chosen a different action at any decision node. Little is known about the bandit setting, where that assumption is reversed (no counterfactual information is available), despite this latter setting being well understood for almost 20 years in one-shot decision making. In this paper, we give the first algorithm for the bandit linear optimization problem for TFSDM that offers both (i) linear-time iterations (in the size of the decision tree) and (ii) O(sqrt(T)) cumulative regret in expectation compared to any fixed strategy, at all times T. This is made possible by new results that we derive, which may have independent uses as well: 1) geometry of the dilated entropy regularizer, 2) autocorrelation matrix of the natural sampling scheme for sequence-form strategies, 3) construction of an unbiased estimator for linear losses for sequence-form strategies, and 4) a refined regret analysis for mirror descent when using the dilated entropy regularizer.

Online optimization method of cooling water system based on the heat transfer model for cooling tower

The demand for the optimization of cooling water systems to reduce energy consumption is increasing nowadays. Based on the heat transfer performance of a cooling tower, this paper proposed a heat transfer model for cooling towers. To verify the accuracy of the model and to demonstrate the applicability of off-design conditions, an experimental system for evaluating the air/water heat transfer performance of cooling towers was established. The experimental results showed that the proposed model for cooling towers had an ability of high-accuracy prediction and adaptability to off-design conditions. Based on the proposed model for cooling towers, the mathematical model of cooling water system online optimization is established. The hybrid programming particle swarm optimization (HP–PSO) algorithm to optimize the operation of cooling water systems was proposed to solve the online optimization problem of continuous variables and discrete variables in cooling water systems. The advantages of the HP-PSO algorithm were verified by the simulation of a small data center. The simulation results showed that the proposed online optimization method of cooling water systems can reduce energy consumption by 15.3% comparing with other method. This paper can provide an important reference for the operation optimization of cooling water systems in practical engineering.

Bounds for the Tracking Error of First-Order Online Optimization Methods

This paper investigates online algorithms for smooth time-varying optimization problems, focusing first on methods with constant step-size, momentum, and extrapolation-length. Assuming strong convexity, precise results for the tracking iterate error (the limit supremum of the norm of the difference between the optimal solution and the iterates) for online gradient descent are derived. The paper then considers a general first-order framework, where a universal lower bound on the tracking iterate error is established. Furthermore, a method using “long-steps” is proposed and shown to achieve the lower bound up to a fixed constant. This method is then compared with online gradient descent for specific examples. Finally, the paper analyzes the effect of regularization when the cost is not strongly convex. With regularization, it is possible to achieve a non-regret bound. The paper ends by testing the accelerated and regularized methods on synthetic time-varying least-squares and logistic regression problems, respectively.

Emergency Logistics Scheduling Under Uncertain Transportation Time Using Online Optimization Methods

In the immediate aftermath of large-scale disasters, emergency logistics services play important roles in saving lives and reducing losses. Efficient relief logistics scheduling depends on the accurate transport time information for available routes. However, this information cannot be obtained precisely until a vehicle uses the road. Considering the correlation between information acquisition and logistics operations, this paper focuses on a multiperiod online decision-making problem to simulate the information acquiring process. This problem can be referenced for emergency resource scheduling scenarios in which previous decisions impact knowledge for future logistics plans. A multi-trip cumulative capacitated vehicle routing problem with uncertain transportation time is investigated as the basic model. The tradeoff between transportation efficiency and the unknown transport time discovery rate is considered in a multiobjective evolutionary algorithm (MOEA). A memetic algorithm (MA) and a robust optimization (RO) -MA for single-period post-disaster emergency logistics are also proposed to solve the problem for comparison. In these algorithms, evolutionary operators that benefit solution fixing and variation are proposed. In the experiments, a real-world instance is employed. A simulative experimental environment is established. Dynamic information gained within the process of logistics scheduling is highlighted via multi-period online optimization. Different scenarios corresponding to estimates in emergency situations are provided to validate the performance of the algorithms. The experimental results show that the hybrid strategy, MOEA+MA, can obtain the best result in more than half of the considered cases which demonstrates the necessary balance between obtaining information and transportation efficiency.

Integrated Morphing Strategy and Trajectory Optimization of a Morphing Waverider and Its Online Implementation Based on the Neural Network

In previous studies, a variable-sweep-wing morphing waverider was proposed and its aerodynamic performance and dynamic model were provided. This morphing waverider can change sweep angle according to different flight scenario in order to achieve an optimal flight performance. For the traditional fixed-geometry waverider, the main objective of the trajectory planning problem is to find an optimal trajectory consisting of time-dependent variables including the angle of attack, the velocity, the altitude, the flight path angle and so on. As for this morphing waverider, an optimal morphing wing changing strategy is also required to be calculated together with an optimal trajectory. At first, to alleviate computational cost and improve trajectory-planning efficiency, aerodynamic coefficients of the morphing waverider are approximated and modeled as polynomial functions of multiple variables including the Mach number, the angle of attack and the sweep angle. Thereafter, an offline integrated optimization problem of both wing morphing strategy and trajectory based on the Gauss pseudo-spectral method is investigated, which can fulfill actual task requirements and variable constraints. Compared with four specific fixed-wing waverider configurations, this morphing waverider has a larger downrange distance and a lower angle of attack. However, the real flight trajectory of the morphing waverider may deviate from the nominal condition due to multiple uncertainties and this morphing waverider is required to have an intelligent online optimization ability. Therefore, an online optimization method based on the neural network is proposed for both wing morphing strategy and trajectory optimization. Finally, the overall performance of this method is examined by comparisons and Monte Carlo runs.

On the quantitative relationship of the in-cylinder heat to work conversion process of natural gas spark ignited engine under steady state and transient operation conditions

The performance characteristics of an automotive engine under the road operation conditions, i.e. under transient torque and power output conditions, as well as the instant in-cylinder thermal to work conversion efficiency, are very important for the vehicle’s performance and fuel economy, but difficult to measure. In an effort to identify and confirm the quantitative relationships between the engine performance parameters with controlling and operating parameters (such as the engine speed, intake pressure, lambda and so on) for natural gas spark ignited engines, the controlling equations of the in-cylinder indicated thermal efficiency and indicated mean effective pressure are derived. The results show that the derived controlling equations also demonstrate good agreement with experimental data under both steady state and transient operation conditions. This research confirms the feasibility to identify indirectly the natural gas spark ignited engine’s performance under the road conditions through analyzing the measureable or available controlling and operational parameters, and provides a method for on-line analysis and optimization of the engine operation parameters.

A Closed-Form Method for Simultaneous Target Localization and UAV Trajectory Optimization

Unmanned aerial vehicles (UAVs) play a key role in modern surveillance-related missions. A major task for performing these missions is to find the precise location of a moving target in real-time, for which the main challenge is to estimate the target position to high precision using the noisy measurements from the airborne sensors. In this paper, we present a closed-form on-line simultaneous target localization and UAV trajectory optimization method based on the visual platform, which can effectively minimize the localization uncertainty to the target. The proposed method can be elucidated explicitly using two phases, of which, in the target localization phase, the expended information filtering (EIF) is exploited, which can express the predicted Fisher information matrix (FIM) of the target explicitly and iteratively, and in the UAV trajectory optimization phase, the property of the predicted FIM is exploited to establish the UAV waypoint optimization objective by taking account of the UAV motion limit. Compared with existing methods of the same class, the proposed method not only estimates the next target position more correctly, but also takes the error of the target motion into consideration, thus improving the effectiveness of the optimized UAV trajectory. Both simulations and field experiments were conducted, which show that the proposed method outperformed the existing methods.

Near-Optimal Rapid MPC Using Neural Networks: A Primal-Dual Policy Learning Framework

In this article, we propose a novel framework for approximating the MPC policy for linear parameter-varying systems using supervised learning. Our learning scheme guarantees feasibility and near-optimality of the approximated MPC policy with high probability. Furthermore, in contrast to most existing approaches that only learn the MPC policy, we also learn the “dual policy,” which enables us to keep a check on the approximated MPC's optimality online during the control process. If the check deems the control input from the approximated MPC policy safe and near-optimal, then it is applied to the plant; otherwise, a backup controller is invoked, thus filtering out (severely) suboptimal control inputs. The backup controller is only invoked with a bounded (low) probability, where the exact probability level can be chosen by the user. Since our framework does not require solving any optimization problem during the control process, it enables the deployment of MPC on resource-constrained systems. Specifically, we illustrate the utility of the proposed framework on a vehicle dynamics control problem. Compared with online optimization methods, we demonstrate a speedup of up to 62× on a desktop computer and 10× on an automotive-grade electronic control unit, while maintaining high control performance.

An algebraic and suboptimal solution of constrained model predictive control via tangent hyperbolic function

AbstractIn this paper, we propose a novel method to solve the model predictive control (MPC) problem for linear time‐invariant (LTI) systems with input and output constraints. We establish an algebraic control rule to solve the MPC problem to overcome the computational time of online optimization methods. For this purpose, we express system constraints as a continuous function through the tangent‐hyperbolic function, hence the optimization problem is reformulated. There are two steps for the solution of the optimization problem. In the first step, the optimal control signal is determined by the use of the necessary condition for optimality, assuming that there is only input constraint. In the latter, the solution obtained in the first step is revised to keep the system states in a feasible region. It is shown that the solution is suboptimal. The proposed solution method is simulated for three different sample systems, and the results are compared with the classical MPC, which show that the new algebraic method dramatically reduces the computational time of MPC.

On-line optimization of four-zone simulated moving bed chromatography using an Equilibrium-Dispersion Model: II. Experimental validation

In the first theoretical part of this work, an on-line optimization method using the equilibrium-dispersion model of chromatography was developed for the dynamic optimization of periodically operated four-zone simulated moving bed (SMB) processes. In this second part, we present the experimental validation of the proposed concept. To describe the behaviors of real SMB systems, the on-line control unit was modified. Unavoidable system void volumes, which can be described using delay functions corresponding to standard mass balance models for pipes and CSTs, were considered in addition to the chromatographic columns. Furthermore, a new sample collection method that does not interrupt product streams was introduced for process monitoring. The equipment including an external concentration analysis unit was automated via a tailor-made interface program. The on-line control unit estimated the parameters of the process model and optimized the future operating conditions ‘switch-by-switch’. The case study investigated in this work was the separation of racemic mixtures of two bicalutamide enantiomers using a chiral stationary phase applying the conventional and two advanced SMB operating modes.

Total Internal Reflection Transient Absorption Microscopy: An Online Detection Method for Microfluidics.

Microreactors have garnered widespread attention for their tunability and precise control of synthetic parameters to efficiently produce target species. Despite associated advances, a lack of online detection and optimization methods has stalled the progression of microfluidic reactors. Here we employ and characterize a total internal reflection transient absorption microscopy (TIRTAM) instrument to image excited state dynamics on a continuous flow device. The experiments presented demonstrate the capability to discriminate between different chromophores as well as in differentiating the effects of local chemical environments that a chromophore experiences. This work presents the first such online transient absorption measurements and provides a new direction for the advancement and optimization of chemical reactions in microfluidic devices.

Projective Quadratic Regression for Online Learning

This paper considers online convex optimization (OCO) problems - the paramount framework for online learning algorithm design. The loss function of learning task in OCO setting is based on streaming data so that OCO is a powerful tool to model large scale applications such as online recommender systems. Meanwhile, real-world data are usually of extreme high-dimensional due to modern feature engineering techniques so that the quadratic regression is impractical. Factorization Machine as well as its variants are efficient models for capturing feature interactions with low-rank matrix model but they can't fulfill the OCO setting due to their non-convexity. In this paper, We propose a projective quadratic regression (PQR) model. First, it can capture the import second-order feature information. Second, it is a convex model, so the requirements of OCO are fulfilled and the global optimal solution can be achieved. Moreover, existing modern online optimization methods such as Online Gradient Descent (OGD) or Follow-The-Regularized-Leader (FTRL) can be applied directly. In addition, by choosing a proper hyper-parameter, we show that it has the same order of space and time complexity as the linear model and thus can handle high-dimensional data. Experimental results demonstrate the performance of the proposed PQR model in terms of accuracy and efficiency by comparing with the state-of-the-art methods.

Multi-Agent Coordination of Thermostatically Controlled Loads by Smart Power Sockets for Electric Demand Side Management

This article presents a multi-agent control architecture and an online optimization method based on a dynamic average consensus to coordinate the power consumption of a large population of thermostatically controlled loads (TCLs). Our objective is to penalize peaks of power demand, smooth the load profile, and enable demand-side management. The proposed architecture and methods exploit only local measurements of power consumption via smart power sockets (SPSs) with no access to their internal temperature. No centralized aggregator of information is exploited, and agents preserve their privacy by cooperating anonymously only through consensus-based distributed estimation. The interactions among devices occur through an unstructured peer-to-peer (P2P) network over the Internet. Methods for parameter identification, state estimation, and mixed logical modeling of TCLs and SPSs are included. The architecture is designed from a multi-agent and plug-and-play perspective, in which existing household appliances can interact with each other in an urban environment. Finally, a novel low-cost testbed is proposed along with numerical tests and experimental validation.

Process operation optimization using system identification

Process optimization is an important topic in process industry, most process industry optimization works are based on mechanism models or performance test methods. However, it is very difficult to carry out optimization in actual operation because of the difficulty in obtaining the mechanism model, the difficulty in on-line measurement of objective function and the high test cost. In order to solve the problem, an online optimization method based on system identification is proposed. By replacing the unmeasurable variable with the measurable variable, the process model is identified on-line, and the gain of identified model is used as the optimization gradient to find the optimal variable value on-line. The method is verified using both simulation and real plant data.

Stability Analysis of Real-Time Methods for Equality Constrained NMPC

In this paper, a proof of asymptotic stability for the combined system-optimizer dynamics associated with a class of real-time methods for equality constrained nonlinear model predictive control is presented. General Q-linearly convergent online optimization methods are considered and asymptotic stability results are derived for the case where a single iteration of the optimizer is carried out per sampling time. In particular, it is shown that, if the underlying sampling time is sufficiently short, asymptotic stability can be guaranteed. The results constitute an extension to existing attractivity results for the well-known real-time iteration strategy.

Filtering adaptive tracking controller for multivariable nonlinear systems subject to constraints using online optimization method

This paper presents a filtering adaptive tracking control architecture for multivariable nonlinear systems subject to constraints by employing an online optimization method. A piecewise constant adaptive law updates the adaptive parameters which represent the uncertainty estimates by solving the error dynamics between the state predictor and the real system with the neglection of unknowns. A filtering control law is designed to compensate the nonlinear uncertainties and deliver a good tracking performance with guaranteed robustness. In the control scheme, the nonlinear uncertainty compensation component is always active whereas the tracking component is subject to a switching logic. The controller will remain in the tracking mode as long as there exists feasible solution, otherwise this component will be replaced by a new part updated periodically by numerically solving a constrained optimization problem online. The uniform performance bounds are derived for the system states and control inputs as compared to the corresponding signals of a bounded closed-loop reference system, which assumes partial cancellation of uncertainties within the bandwidth of the control signal. Compared to the model predictive control (MPC) method and unconstrained controller, the proposed approach is capable of solving the tracking control problems of multivariable nonlinear systems subject to both constraints and uncertainties.

Online Optimization as a Feedback Controller: Stability and Tracking

This paper develops and analyzes feedback-based online optimization methods to regulate the output of a linear time invariant (LTI) dynamical system to the optimal solution of a time-varying convex optimization problem. The design of the algorithm is based on continuous-time primal-dual dynamics, properly modified to incorporate feedback from the LTI dynamical system, applied to a proximal augmented Lagrangian function. The resultant closed-loop algorithm tracks the solution of the time-varying optimization problem without requiring knowledge of (time varying) disturbances in the dynamical system. The analysis leverages integral quadratic constraints to provide linear matrix inequality (LMI) conditions that guarantee global exponential stability and bounded tracking error. Analytical results show that under a sufficient time-scale separation between the dynamics of the LTI dynamical system and the algorithm, the LMI conditions can be always satisfied. This paper further proposes a modified algorithm that can track an approximate solution trajectory of the constrained optimization problem under less restrictive assumptions. As an illustrative example, the proposed algorithms are showcased for power transmission systems, to compress the time scales between secondary and tertiary control, and allow to simultaneously power rebalancing and tracking of the DC optimal power flow points.

Desired Terminal State Concept in Model Predictive Control: A Case Study

The paper deals with an online optimization control method for dynamical processes called Model Predictive Control (MPC). It is a popular control method in industry and frequently treated in academic areas as well. The standard predictive controllers usually do not guarantee stability especially for the case of short horizons and large control error penalization. Terminal state is one way to ensure stability or at least increase the controller robustness. In the paper, deviation of the predicted terminal state from the desired terminal state is considered as one term of the cost function. Effect of the stability and control quality is demonstrated in the simulated experiments. The application area for online optimization methods is very broad including various logistics and transport problems. If the dynamics of the controlled processes cannot be neglected, the optimization problem must be solved not only for steady state but also for transient behaviour, e.g., by MPC.

Online Monitoring of Wind Turbine Operation Efficiency and Optimization Based on Benchmark Values

Improving the operational efficiency of wind turbines is an important way to enhance the competitiveness of wind power generation among new energy sources. This paper presents a new method for online monitoring and optimization of the operational efficiency of wind turbines. The randomness of the wind speed is analyzed in the form of wind speed variation using historical data, revealing the distribution pattern of wind speed randomness. The fluctuation amplitude of the output power of wind turbines is analyzed in the form of power variation under different wind speed modes. The level of active power is classified by quantiles based on historical data. The sliding window method is adopted to eliminate the jumping fluctuation of the efficiency level caused by the randomness of the wind speed. To identify and explain the reasons for the deviation of the operating efficiency, the benchmark values of operational parameters are obtained from historical operation data by data mining. The low operating efficiency level is adjusted according to the benchmark values of operating parameters. The model has been verified to effectively monitor the operating efficiency of wind turbines and provide operational optimization measures to improve efficiency.

Self-Tuning Method of PID Parameters Based on Belief Rule Base Inference

As a generic inference mechanism, the belief rule-based (BRB) system can effectively integrate quantitative information with qualitative knowledge to model causal relationships of complex application systems. Based on the BRB, this paper develops a novel self-tuning strategy of PID parameters such that the output of closed-loop control system generated by PID controller can accurately follow control input. Firstly, the initial belief rule base is abstracted from expert’s control experiences to depict the highly nonlinear relationship between the variables of control system and each PID parameter. Secondly, the objective function is established to minimize the error between the given control input and the closed loop output, and then the online optimization method via sequential linear programming is presented to optimize the parameters of BRB system so as to adaptively adjust PID parameters by the optimized BRB system in real time. Typical control simulation experiments of DC motor are implemented to illustrate the advantages of the proposed BRB-PID over widely used neural network-based PID.DOI: http://dx.doi.org/10.5755/j01.itc.47.3.19045