Simultaneous Perturbation Stochastic Approximation Research Articles

Enhancement of SPSA algorithm performance using reservoir quality maps: Application to coupled well placement and control optimization problems

Determination of optimal well location is a vital step in the design of an efficient field development plan which aims to maximize the hydrocarbon recovery and profits from the project. This step has become further complicated as the well control parameters are also introduced into the problem which is known as coupled well placement and control optimization problem. As the number of optimization variable increases, computational cost also increases and it becomes a serious challenge, especially in high resolution reservoir models since the optimization process becomes very time demanding. Simultaneous Perturbation Stochastic Approximation (SPSA) has been introduced as an efficient algorithm in finding the optimum solution for problems with a large number of variables. In this work, SPSA algorithm has been modified using reservoir quality maps to enhance and accelerate finding the good solutions with a minimum number of function evaluation which is desirable for high-resolution reservoir models and high dimensional optimization problems. In this modification, reservoir quality maps are employed to navigate production wells toward high potential areas instead of random movement in standard SPSA. The modified algorithm is tested for joint well placement and control optimization problem using both simultaneous and sequential optimization approach in a synthetic and two benchmark models. The results show that modified SPSA outperforms standard SPSA both in terms of final result and convergence trend. Also, in most cases, simultaneous well placement and control optimization approach gives better results compared to sequential approaches for both algorithms.

A stochastic approximation approach to spatio-temporal anchorage planning with multiple objectives

Globalization and subsequent increase in seaborne trade have necessitated efficient planning and management of world’s anchorage areas. These areas serve as a temporary stay area for commercial vessels for various reasons such as waiting for passage or port, fuel services, and bad weather conditions. The research question we consider in this study is how to place these vessels inside a polygon-shaped anchorage area in a dynamic fashion as they arrive and depart, which seems to be the first of its kind in the literature. We specifically take into account the objectives of (1) anchorage area utilization, (2) risk of vessel collisions, and (3) fuel consumption performance. These three objectives define our objective function in a weighted sum scheme. We present a spatio-temporal methodology for this multi-objective anchorage planning problem where we use Monte Carlo simulations to measure the effect of any particular combination of planning metrics (measured in real time for an incoming vessel) on the objective function (measured in steady state). We resort to the Simultaneous Perturbation Stochastic Approximation (SPSA) algorithm for identifying the linear combination of the planning metrics that optimizes the objective function. We present computational experiments on a major Istanbul Straight anchorage, which is one of the busiest in the world, as well as synthetic anchorages. Our results indicate that our methodology significantly outperforms comparable algorithms in the literature for daily anchorage planning. For the Istanbul Straight anchorage, for instance, reduction in risk was 42% whereas reduction in fuel costs was 45% when compared the best of the current state-of-the-art methods. Our methodology can be utilized within a planning expert system that intelligently places incoming vessels inside the anchorage so as to optimize multiple strategic goals. Given the flexibility of our approach in terms of the planning objectives, it can easily be adapted to more general variants of multi-objective spatio-temporal planning problems where certain objects need to be dynamically placed inside two or even-three dimensional spaces in an intelligent manner.

Conquering the Worst Case of Infections in Networks

We develop algorithms to control the scope of an infection spread on a network by allocating a fixed immunization budget to edges of the graph. We assume that the infection propagates according to an independent cascade model and interventions operate by reducing the propensities of edges to transmit the infection. We formulate this problem as a constrained min-max optimization problem with respect to the placements of interventions and the location of the worst-case seed nodes. However, the result is a challenging bilevel mixed integer optimization problem. Furthermore, gradients of the objective function with respect to the continuous variables are unavailable in closed form. We employ tools from derivative-free optimization and stochastic optimization to optimize the objective by iterating between the outer minimization and inner maximization problems. In the inner loop, we use a weighted degree discount (WDD) method to select the seed set of the influence maximization problem. In the outer loop, we utilize two methods: a sample-based simultaneous perturbation Nelder-Mead (SBSP-NM) algorithm and a simultaneous perturbation stochastic approximation (SPSA) algorithm. We perform simulations on synthetic graphs and three larger-scale real-world datasets and illustrate the computational feasibility of our algorithms and their efficacy at controlling epidemic spreads.

Coordination Optimization of VSL Strategy on Urban Expressway and Main Road Intersection Signal

With the rapid urbanization during the past three decades in China, the urban expressway network has arisen with large numbers in many cities. However, the traffic conditions are becoming worse and worse. One of the main reasons is that the connected sections of main road and expressway do not have a coordination by the intersection signal and variable speed limits (VSL) on expressway. In this paper, a new cooperation optimization model of VSL on urban expressway and the intersection signal timing are proposed based on the mesoscopic traffic flow model. Its objective is to minimize the total delay and the number of parking. Its essence is adjusting traffic flow on the network in space-time so that the whole network can be in a state of equilibrium. It is effective to the control of local traffic congestion and can avoid the spread of congestion, thus improving traffic safety. Additionally, the optimal VSL strategy is proposed. And then a solution algorithm based on the simultaneous perturbation stochastic approximation (SPSA) is designed for the proposed model. In the end, a practical example of Yanshan overpass in Ji’nan elevated expressway is used to validate the effectiveness of the proposed method. The simulation results are satisfactory. The conclusions are very useful for traffic management department to implement reasonable traffic control strategy.

An accelerated gradient algorithm for well control optimization

The simulation-assisted optimization of injection and production control settings can be considered as a way for improved management of hydrocarbon reservoirs. Although well control optimization problems are expected to have a relatively smooth landscape, they suffer from having a high number of dimensions. As a result, gradient-based algorithms are typically the preferred choice over derivative-free algorithms. Since the calculation of the exact gradient is not always feasible, recent studies into gradient-based algorithms have focused on ameliorating the accuracy of gradient approximation methods. Less attention has been paid to the framework in which these gradient approximations are utilized. The most common framework is the steepest descent with a backtracking line-search. This study aims to provide an improved framework using an adaptive moment estimation technique. The proposed method is compared to the steepest descent framework in two case studies of increasing complexity. In both frameworks, the gradient is approximated by the simultaneous perturbation stochastic approximation. The first case study is a two-dimensional heterogeneous reservoir model produced under water-flooding. The second case study investigates the injection and production settings of the three-dimensional benchmark case, the Brugge model, whilst incorporating geological uncertainty. The proposed framework showed improvements of up to 91% in convergence speeds and up to 5% in optimal value for the first case study. In addition, the proposed framework showed an improvement of up to 81% in convergence speeds and up to 2% in optimal value for the second case study over the steepest descent framework. For both case studies, the effect of the gradient approximation accuracy (perturbation number) was also investigated. The results indicated that the proposed algorithm is less sensitive to the gradient approximation accuracy than the steepest descent framework. In addition, this study investigated the effect of two popular bound constraint handling techniques in both frameworks. The results indicated that the projection method outperforms the logarithmic transformation method when applied to production optimization problem.

Advising Big Data Transfer Over Dedicated Connections Based on Profiling Optimization

Big data transfer in next-generation scientific applications is now commonly carried out over dedicated channels in high-performance networks (HPNs), where transport protocols play a critical role in maximizing application-level throughput. Optimizing the performance of these protocols is challenging: i) transport protocols perform differently in various network environments, and the protocol choice is not straightforward; ii) even for a given protocol in a given environment, different parameter settings of the protocol may lead to significantly different performance and oftentimes the default setting does not yield the best performance. However, it is prohibitively time-consuming to conduct exhaustive transport profiling due to the large parameter space. In this paper, we propose a PRofiling Optimization Based DAta Transfer Advisor (ProbData) to help end users determine the most effective transport method with the most appropriate parameter settings to achieve satisfactory performance for big data transfer over dedicated connections in HPNs. ProbData employs a fast profiling scheme based on the Simultaneous Perturbation Stochastic Approximation algorithm, namely, FastProf, to accelerate the exploration of the optimal operational zones of various transport methods to improve profiling efficiency. We first present a theoretical background of the optimized profiling approach in ProbData and then detail its design and implementation. The advising procedure and performance benefits of FastProf and ProbData are illustrated and evaluated by both extensive emulations based on real-life performance measurements and experiments over various physical connections in existing production HPNs.

MPS-APO: a rapid and automatic parameter optimizer for multiple-point geostatistics

Multiple-point statistics (MPS) have been widely used in Earth and environmental sciences because of their ability to generate realistic stochastic realizations of complex natural processes. The spatial patterns and statistical information required for MPS modeling are represented by a training image. However, each MPS algorithm has a specific set of parameters that have a direct impact on the quality of pattern reproduction and should be chosen prior to the modeling. While there are some general guidelines for some MPS algorithms, a general parameter interference methodology is currently lacking. To date, the common practice for finding optimal parameters is to carry out a sensitivity analysis, which can be cumbersome especially in complex applications. In this study, we propose the MPS Automatic Parameter Optimizer (MPS-APO), a generic method based on stochastic optimization to rapidly approximate optimal parameters for any MPS method and different types of settings. The MPS-APO formulates an objective function that quantifies spatial pattern reproduction for each set of parameters. The Simultaneous Perturbation Stochastic Approximation (SPSA) optimization method is used because of its computational efficiency, and also its ability to cope with the stochastic nature of the objective function. The optimization proceeds in two steps. The first step aims to optimize the parameters for the best quality regardless of computational cost. When no more improvement can be achieved, the second step minimizes the CPU cost without degrading the spatial structures reproduction attained in the first step. In this study, MPS-APO is performed on different pixel-based and patch-based MPS methods: SNESIM, FILTERSIM, Direct Sampling and Image Quilting. Test cases show that MPS-APO is a useful heuristic to automatically approximate optimal parameters for good patterns reproduction with minimal computational cost. Therefore, it can help non-expert users and increase the usability of MPS methods for practical applications.

A Stochastic Optimization Approach to Magnetometer Calibration With Gradient Estimates Using Simultaneous Perturbations

Triaxial magnetometers are indispensable part of every attitude sensing equipment for providing heading information. However, magnetometer measurements are affected by a number of error sources such as soft iron, hard iron, sensor nonorthogonality, and bias. These erroneous measurements, if not properly calibrated, deteriorate the quality of attitude estimates. In this paper, we present a new approach to calibrate triaxial magnetometer measurements by posing the calibration problem as a stochastic optimization problem. A scalar objective function is derived which involves magnetometer calibration parameters corresponding to all distortions. In order to recursively solve for optimized calibration parameters, a modified version of simultaneous perturbation stochastic approximation for the gradient is proposed for faster convergence. The proposed approach places no restrictions on the number of parameters involved and offers a complete solution by solving for all calibration parameters. The proposed approach is tested and validated using experimental data and the performance is compared against a reference approach.

Adaptive Safe Experimentation Dynamics for Data-Driven Neuroendocrine-PID Control of MIMO Systems

A safe experimentation dynamics (SED) is a game theoretic method that randomly perturbs several elements of its design parameter to search for the optimal design parameter. However, the accuracy of the standard SED can be further improved by proposing an adaptive SED method. This paper aims to develop a method based on adaptive safe experimentation dynamics (ASED) where the updated design parameter is modified to adapt to the change of the objective function. The proposed ASED is then used to tune the parameters of the neuroendocrine-PID controller. The neuroendocrine-PID controller, which is based on the secretion rule of hormone regulation in the human body and well known for its high control accuracy, is chosen to improve the conventional PID controller structure for the MIMO systems. Moreover, it is shown that the proposed neuroendocrine-PID based ASED can solve an unstable convergence problem in the existing neuroendocrine-PID based Simultaneous Perturbation Stochastic Approximation (SPSA). The performance of the proposed neuroendocrine-PID based on the ASED method is evaluated by tracking its performances and computational time. Additionally, the performance of the ASED based method is compared to the standard SED and SPSA based methods. The results of the simulation showed that the ASED method could provide stable convergence by minimizing the function of the given objective. The ASED also obtains a value for the objective function and the total norm error for tracking performance accuracy that is lower compared to other methods.

Efficient Implementation of Second-Order Stochastic Approximation Algorithms in High-Dimensional Problems.

Stochastic approximation (SA) algorithms have been widely applied in minimization problems when the loss functions and/or the gradient information are only accessible through noisy evaluations. Stochastic gradient (SG) descent-a first-order algorithm and a workhorse of much machine learning-is perhaps the most famous form of SA. Among all SA algorithms, the second-order simultaneous perturbation stochastic approximation (2SPSA) and the second-order stochastic gradient (2SG) are particularly efficient in handling high-dimensional problems, covering both gradient-free and gradient-based scenarios. However, due to the necessary matrix operations, the per-iteration floating-point-operations (FLOPs) cost of the standard 2SPSA/2SG is O(p3) , where p is the dimension of the underlying parameter. Note that the O(p3) FLOPs cost is distinct from the classical SPSA-based per-iteration O(1) cost in terms of the number of noisy function evaluations. In this work, we propose a technique to efficiently implement the 2SPSA/2SG algorithms via the symmetric indefinite matrix factorization and show that the FLOPs cost is reduced from O(p3) to O(p2) . The formal almost sure convergence and rate of convergence for the newly proposed approach are directly inherited from the standard 2SPSA/2SG. The improvement in efficiency and numerical stability is demonstrated in two numerical studies.

A reduced random sampling strategy for fast robust well placement optimization

Model-based decision-making in oilfield development often involves hundreds of computationally demanding reservoir simulation runs. In particular, well placement optimization under uncertainty in the geologic representation of the reservoir model is an overly time-consuming procedure as the performance of any proposed well configuration needs to be evaluated over multiple realizations, using computationally expensive flow simulations. To reduce computation, we propose an efficient robust optimization procedure in which at each iteration of the optimization procedure, instead of evaluating the well configuration over all available realizations, we approximate the expected performance using a small subset of randomly selected model realizations. Since the samples are selected randomly, all the realizations are expected to eventually be included in the performance evaluation after a certain number of iterations. However, using only a few random realizations to compute the expected cost function introduces noise in the estimated objective function, necessitating the use of a stochastic optimizer. In this paper, we use the Simultaneous Perturbation Stochastic Approximation (SPSA) algorithm, which is known to be robust against noise in the objective function. We first evaluate the performance of different forms of the SPSA algorithm (including discrete, continuous, and adaptive) using several numerical experiments, followed by a discussion of the properties of the proposed reduced random sampling approach and comparison with global optimization techniques. The method is applied to several numerical experiments, including case studies involving vertical, horizontal, and lateral wells, to evaluate its performance. The results from these experiments indicate that the reduced random sampling approach can provide significant computational gain with minimal impact on the attained optimization performance.

Auto-optimization of production-injection rate for reservoirs with strong natural aquifer at ultra-high water cut stage

Abstract Based on the optimal control theory and taking the production law of reservoirs with strong natural aquifer as the basic constraint, a mathematical model of liquid production for such reservoirs in the later stage of development is established. The model is solved by improved simultaneous perturbation stochastic approximation algorithm (SPSA), and an automatic optimization software for liquid production is developed. This model avoids the disadvantage of traditional optimization methods that only focus on the maximum value of mathematics but ignore the production law of oilfield. It has the advantages of high efficiency of calculation, short period and automatic optimization. It can satisfy the automatic optimization of liquid production in later stage of oilfield development. The software was applied in the oilfield development of D oilfield, Ecuador in South America, and realized the automatic optimization of liquid production in the later stage of oilfield development.

Hybrid fuzzy control for the goethite process in zinc production plant combining type-1 and type-2 fuzzy logics

The goethite process is a complicated reaction system which holds a significant position in zinc hydrometallurgy process. It exhibits nonlinear behavior and time-delay nature because of the chemical reactions. To achieve the stable and real-time control performance, a hybrid fuzzy control strategy integrating a type-1 and a type-2 fuzzy logic controllers is proposed in this paper. According to the on-line measured pH, the type-1 fuzzy controller (T1 FLC) via the Takagi–Sugeno fuzzy control is employed to control the zinc oxide additive rate. A parameter tuning method that does not require the system model is designed for the T1 FLC. Because the ferrous ion concentration cannot be measured on-line, the interval type-2 fuzzy logic controller (IT2 FLC) is utilized to control the oxygen flow rate. According to the feedback information, gradient descent algorithm is used to update the parameters in the IT2 FLC. Because of the unknown system structure, gradient descent information is estimated by the simultaneous perturbation stochastic approximation. Finally, simulations are conducted using the practical production data. The simulation results show the effectiveness of the proposed strategy. The hybrid fuzzy control strategy improves the control performance of the goethite process, compared with PID controller.

Parallel Simultaneous Perturbation Optimization

Stochastic computer simulations enable users to gain new insights into complex physical systems. Optimization is a common problem in this context: users seek to find model inputs that maximize the expected value of an objective function. The objective function, however, is time-intensive to evaluate, and cannot be directly measured. Instead, the stochastic nature of the model means that individual realizations are corrupted by noise. More formally, we consider the problem of optimizing the expected value of an expensive black-box function with continuously-differentiable mean, from which observations are corrupted by Gaussian noise. We present parallel simultaneous perturbation optimization (PSPO), which extends a well-known stochastic optimization algorithm, simultaneous perturbation stochastic approximation, in several important ways. Our modifications allow the algorithm to fully take advantage of parallel computing resources, like high-performance cloud computing. The resulting PSPO algorithm takes fewer time-consuming iterations to converge, automatically chooses the step size, and can vary the error tolerance by step. Theoretical results are supported by a numerical example.

PC–SPSA: Employing Dimensionality Reduction to Limit SPSA Search Noise in DTA Model Calibration

Calibration and validation have long been a significant topic in traffic model development. In fact, when moving to dynamic traffic assignment (DTA) models, the need to dynamically update the demand and supply components creates a considerable burden on the existing calibration algorithms, often rendering them impractical. These calibration approaches are mostly restricted either due to non-linearity or increasing problem dimensionality. Simultaneous perturbation stochastic approximation (SPSA) has been proposed for the DTA model calibration, with encouraging results, for more than a decade. However, it often fails to converge reasonably with the increase in problem size and complexity. In this paper, we combine SPSA with principal components analysis (PCA) to form a new algorithm, we call, PC–SPSA. The PCA limits the search area of SPSA within the structural relationships captured from historical estimates in lower dimensions, reducing the problem size and complexity. We formulate the algorithm, demonstrate its operation, and explore its performance using an urban network of Vitoria, Spain. The practical issues that emerge from the scale of different variables and bounding their values are also analyzed through a sensitivity analysis using a non-linear synthetic function.

Combined active flow and flight control systems design for morphing unmanned aerial vehicles

In this article, combined active flow control system and flight control system design for morphing unmanned aerial vehicles is applied for the first time for autonomous flight performance maximization. For this purpose, longitudinal and lateral dynamics modeling of morphing unmanned aerial vehicle having active flow control manufactured in Erciyes University, Faculty of Aeronautics and Astronautics, Model Aircraft Laboratory is considered in order to obtain simulation environments. Our produced morphing unmanned aerial vehicle is called as ZANKA-II, which has a mass of 6.5 kg, range of 30 km, endurance of 0.5 h, and ceiling altitude of 6000 m. von Karman turbulence modeling is used in order to model atmospheric turbulence during flight in both longitudinal and lateral simulation environments. A stochastic optimization method called as simultaneous perturbation stochastic approximation is also applied for the first time in order to obtain optimum dimensions of morphing parameters (i.e. extension ratios of wingspan and tail span), optimum positions of blowers, and optimum magnitudes of longitudinal and lateral controllers' gains (i.e. P, I, and D gains) while minimizing cost index capturing terms for both longitudinal and lateral autonomous flight performances and there exist lower and upper constraints on all optimization variables in the literature.

Demand Calibration of Multimodal Microscopic Traffic Simulation using Weighted Discrete SPSA

This paper presents a stochastic approximation framework to solve a generalized problem of off-line calibration of demand for a multimodal microscopic (or mesoscopic) network simulation using aggregated sensor data. A key feature of this problem is that demand, although typically treated as a continuous variable is in fact discrete, particularly in the context of agent-based simulation. To address this, we first use a discrete version of the weighted simultaneous perturbation stochastic approximation (W-DSPSA) algorithm for minimizing a generalized least squares (GLS) objective (that measures the distance between simulated and observed measurements), defined over discrete sets. The algorithm computes the gradient at each iteration using a symmetric discrete perturbation of the calibration parameters and a multimodal weight matrix to improve the accuracy of the gradient estimate. The W-DSPSA algorithm is then applied to the large-scale calibration of multimodal origin–destination (OD) flows (including private vehicle (PVT) and public transit (PT) trips) in a microscopic network simulation model of Singapore. The results indicate that an acceptable margin of error on the vehicle loop count (VLC) and bus passenger count (BPC) are achieved at convergence with an improvement of 60%~80% in root mean squared errors. Lastly, we validate the calibration results with observed travel times on the network. Statistical comparison shows good agreements on both point-to-point travel time (PTT) and public buses’ stop-to-stop ride-time (SRT) with the field observations.

Stochastic Source Seeking for Mobile Robots in Obstacle Environments Via the SPSA Method

This paper considers a class of stochastic source-seeking problems to drive a mobile robot to the minimizer of a source signal. Our approach is first analyzed in an obstacle-free scenario, where measurements of the signal at the robot location and information of a contact sensor are required. We extend our results to environments with obstacles under mild assumptions on the step size. Our approach builds on the simultaneous perturbation stochastic approximation idea to obtain information of the signal field. We prove the practical convergence of the algorithms to a ball whose size depends on the step size that contains the location of the source. The novelty relies in that we consider nondifferentiable convex functions, a fixed step size, and the environment may contain obstacles. Our proof methods employ nonsmooth Lyapunov function theory, tools from convex analysis, and stochastic difference inclusions. Finally, we illustrate the applicability of the proposed algorithms in a two-dimensional scenarios.

Improving fuel efficiency of connected and automated transit buses on signallised corridors

This study proposes a trajectory planning strategy for connected and automated transit buses travelling on signallised corridors. The goal is to improve the fuel efficiency of transit buses by taking into consideration of traffic signal timings and bus schedules. The strategy takes advantage of the emerging connected vehicle technologies, allowing for utilising real-time information, including traffic signal timings, bus schedules and dynamic speed limits for trajectory optimisation. The authors construct the proposed strategy as a Min-Sum-Min Model and solve it by integrating Simultaneous Perturbation Stochastic Approximation and Pontryagin's Maximum Principle. The effectiveness of the proposed strategy is demonstrated in a real-world case study. It is discovered that the reduction in bus energy consumption by the proposed strategy is up to 27.8 and 16.4%, respectively, when compared with an aggressive strategy and an alternative strategy.

Inverse Material Characterization through Finite Element Simulation of Material Tests and Numerical Optimization

Abstract This paper presents a new method to characterize the material flow stress and failure models based on scratch test data. A group of parametric studies is first conducted to reveal the sensitivity of typical material responses in micro-scratch tests with respect to various materials constitutive and damage model parameters. The results were then used to guide the development of an inverse material characterization method by matching the finite element predictions with the corresponding test data. The simultaneous perturbation stochastic approximation (SPSA) algorithm was used for an efficient parameter search. The method was implemented to the identification of the Johnson-Cook (J-C) constitutive and damage model parameters for an aluminum alloy based on its micro-scratch tests conducted at room and elevated temperatures. The identification was validated with the morphological measurement of the scratches.