Stochastic Approximation Algorithm Research Articles

Real time power management strategy for hybrid energy storage systems coupled with variable energy sources in power smoothing applications

As the renewable energy sources (RES) production is strongly influenced by multiple geographic factors and highly variable, the need for both energy storage integration and robust real-time power management strategies development is obvious. Wind power represents the largest generating capacity among RES, being at the same time the most fluctuant. The capability to overcome the great disadvantage of wind power variability supports rising its penetration while preserving current operation modes of power systems, so new fashions to achieve this target are of great interest. This paper aims to prove the robustness of a recently introduced power management strategy, able to operate in online conditions, based on simultaneous perturbation stochastic approximation (SPSA) algorithm. To this regard, two different real datasets for wind power profiles with different statistical features are employed. The power management strategy is implemented on a hybrid energy storage system comprising a battery and a flywheel, modeled in Simulink/Matlab. The objectives of the proposed strategy are to reduce the instantaneous power ramp of the profile injected to the grid while smoothening the power profile exchanged by the battery in order to preserve it. Simulations are performed in representative conditions selected on statistical basis. It is demonstrated that the SPSA based power management achieves similar performances in all simulation conditions, proving to be robust. As a performance indicator, the reduction of the power ramp in reference to the 90% CDF threshold is evaluated. It is remarked as an 80% power ramp reduction is obtained towards the grid in both sites. Moreover, the further target is achieved in terms of battery lifetime extension; specifically, the fluctuation of the power profile exchanged by the battery is smoothed by 63% in the first site and 48% in the second, with respect to the flywheel one.

Robust integrated optimization of well placement and control under field production constraints

Field development and control optimization aim to maximize the economic profit of oil and gas production while respecting various constraints such as production limits imposed by the available fluid processing capacity and/or reservoir management strategies. The limitations of the existing optimization workflows are 1) well locations or production/injection controls are optimized independently despite the fact that one affects another, and 2) forthcoming field production limits are ignored during at least one of these optimization stages. This paper presents a robust, multi-level framework for field development and control optimization under fluid processing capacity constraints while considering reservoir description uncertainty. The developed framework is based on sequential iterative optimization of control variables at different levels, where the loop of well placement followed by control optimization continues until no significant improvement is observed in the expected objective value. Simultaneous perturbation stochastic approximation (SPSA) algorithm is employed as the optimizer at all optimization levels. Field production constraints are imposed on the reservoir model using a simplified production network. Smart clustering techniques are applied to systematically select an ensemble of reservoir model realizations as the representative of all available realizations at each optimization level. The developed framework is tested on the Brugge benchmark field case study with a maximum field liquid production limit imposed via the production network. A comparative analysis is performed for each case to investigate the impact of field liquid production constraints on optimal well placement and control strategy. Results demonstrate that ignoring fluid processing capacity constraints, in one or multiple levels of optimization, results in a sub-optimal scenario, highlighting the significance of the proposed optimization framework in robust field development and management. • A robust, integrated framework is presented for field development and control under field production constraints while considering reservoir uncertainty. • Clustering techniques are developed to systematically select representative reservoir model realizations at each optimization level. • Ignoring field production constraints in one or multiple levels of optimization results in sub-optimal scenarios, showing the significance of developed framework in robust field development and control.

The Threshold Optimization of the Canonical Piecewise Linear Function-Based Model With a Modified Quadratic SPSA

The canonical piecewise linear (CPWL) function-based model has been proposed for behavioral modeling of power amplifiers (PAs) because it can provide good modeling accuracy and low complexity. In the CPWL, thresholds are preselected as different constants, which are usually based on experience. The letter focuses on the threshold optimization of the CPWL, and a modified quadratic simultaneous perturbation stochastic approximation (SPSA) algorithm is employed to estimate the coefficients and optimize thresholds of the CPWL-based model. This method is more suitable for hardware implementation than the conventional least square (LS) by coefficient iterations rather than matrix inversion operations. Experimental results show that, compared with the LS, the proposed approach achieves comparable modeling and digital predistortion (DPD) performance of the CPWL-based model with fewer thresholds and coefficients after threshold optimization when a wideband PA is excited by a Long-Term Evolution (LTE)-Advanced signal of 100-MHz bandwidth.

Zeroth-Order Nonconvex Stochastic Optimization: Handling Constraints, High Dimensionality, and Saddle Points

In this paper, we propose and analyze zeroth-order stochastic approximation algorithms for nonconvex and convex optimization, with a focus on addressing constrained optimization, high-dimensional setting, and saddle point avoiding. To handle constrained optimization, we first propose generalizations of the conditional gradient algorithm achieving rates similar to the standard stochastic gradient algorithm using only zeroth-order information. To facilitate zeroth-order optimization in high dimensions, we explore the advantages of structural sparsity assumptions. Specifically, (i) we highlight an implicit regularization phenomenon where the standard stochastic gradient algorithm with zeroth-order information adapts to the sparsity of the problem at hand by just varying the step size and (ii) propose a truncated stochastic gradient algorithm with zeroth-order information, whose rate of convergence depends only poly-logarithmically on the dimensionality. We next focus on avoiding saddle points in nonconvex setting. Toward that, we interpret the Gaussian smoothing technique for estimating gradient based on zeroth-order information as an instantiation of first-order Stein’s identity. Based on this, we provide a novel linear-(in dimension) time estimator of the Hessian matrix of a function using only zeroth-order information, which is based on second-order Stein’s identity. We then provide a zeroth-order variant of cubic regularized Newton method for avoiding saddle points and discuss its rate of convergence to local minima.

Analytical Models for Optimal Design of a Trapezoidal Composite Channel Cross-Section

Abstract In this paper, several analytical models are presented for the optimal design of a trapezoidal composite channel cross-section. The objective function is the cost function per unit length of the channel, which includes the excavation and lining costs. To define the system, design variables including channel depth, channel width, side slopes, freeboard, and roughness coefficients were used. The constraints include Manning’s equation, flow velocity, Froude number, and water surface width. The Simultaneous Perturbation Stochastic Approximation (SPSA) algorithm was used to solve the optimization problem. The results are presented in three parts; in the first part, the optimal values of the design variables and the objective function are presented in different discharges. In the second part, the relationship between cost and design variables in different discharges is presented in the form of conceptual and analytical models and mathematical functions. Finally, in the third part, the changes in the design variables and cost function are presented as a graph based on the discharge variations. Results indicate that the cost increases with increasing water depth, left side slope, equivalent roughness coefficient, and freeboard.

Solving Bayesian risk optimization via nested stochastic gradient estimation

In this article, we aim to solve Bayesian Risk Optimization (BRO), which is a recently proposed framework that formulates simulation optimization under input uncertainty. In order to efficiently solve the BRO problem, we derive nested stochastic gradient estimators and propose corresponding stochastic approximation algorithms. We show that our gradient estimators are asymptotically unbiased and consistent, and that the algorithms converge asymptotically. We demonstrate the empirical performance of the algorithms on a two-sided market model. Our estimators are of independent interest in extending the literature of stochastic gradient estimation to the case of nested risk measures.

Stochastic Approximation With Iterate-Dependent Markov Noise Under Verifiable Conditions in Compact State Space With the Stability of Iterates Not Ensured

This article compiles several aspects of the dynamics of stochastic approximation algorithms with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Markov iterate-dependent</i> noise when the iterates are not known to be stable beforehand. We achieve the same by extending the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">lock-in probability</i> (i.e., the probability of convergence of the iterates to a specific attractor of the limiting ordinary differential equation (o.d.e.) given that the iterates are in its domain of attraction after a sufficiently large number of iterations (say) <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$n_0$</tex-math></inline-formula> ) framework to such recursions. Specifically, with the more restrictive assumption of Markov iterate-dependent noise supported on a bounded subset of the Euclidean space, we give a lower bound for the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">lock-in probability</i> . We use these results to prove almost sure convergence of the iterates to the specified attractor when the iterates satisfy an asymptotic tightness condition. The novelty of our approach is that if the state space of the Markov process is compact, we prove almost sure convergence under much weaker assumptions compared to the work by Andrieu <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> , which solves the general state-space case under much restrictive assumptions by providing sufficient conditions for stability of the iterates. We also extend our single-timescale results to the case where there are two separate recursions over two different timescales. This, in turn, is shown to be useful in analyzing the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">tracking ability</i> of general adaptive algorithms. Additionally, we show that our results can be used to derive a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sample complexity estimate</i> of such recursions, which then can be used for step-size selection.

On stochastic mirror descent with interacting particles: Convergence properties and variance reduction

An open problem in optimization with noisy information is the computation of an exact minimizer that is independent of the amount of noise. A standard practice in stochastic approximation algorithms is to use a decreasing step-size. This however leads to a slower convergence. A second alternative is to use a fixed step-size and run independent replicas of the algorithm and average these. A third option is to run replicas of the algorithm and allow them to interact. It is unclear which of these options works best. To address this question, we reduce the problem of the computation of an exact minimizer with noisy gradient information to the study of stochastic mirror descent with interacting particles. We study the convergence of stochastic mirror descent and make explicit the tradeoffs between communication and variance reduction. We provide theoretical and numerical evidence to suggest that interaction helps to improve convergence and reduce the variance of the estimate.

Moderate deviation principles for nonparametric recursive distribution estimators using Bernstein polynomials

In this paper we prove moderate deviations principles for the recursive estimators of a distribution function defined by the stochastic approximation algorithm based on Bernstein polynomials introduced by Jmaei el al. (J Nonparametr Stat 29:792–805, 2017). We show that the considered estimator gives the same pointwise moderate deviations principle (MDP) as the recursive kernel distribution estimator proposed in Slaoui (Math Methods Stat 23(4):306–325, 2014b) and whose large and moderate deviation principles were established by Slaoui (Stat Interface 12(3):439–455, 2009).

Efficient optimization algorithms for surgical scheduling under uncertainty

In this paper, we develop a stochastic optimization model for a surgical scheduling problem considering a single operating room. We arrange a set of elective surgeries into appropriate time blocks, and determine their planned start time and specific sequence. Due to the complexity of the original formulation, we reformulate our model as a two-stage mixed-integer problem. We consider the planning decision in the first stage and the sequencing decision in the second stage (based on the first one). The goal of this paper is to obtain a nearly optimal schedule in reasonable computational time. The term “optimal” is defined as the lowest surgically related cost while achieving the given threshold with respect to some specific deterministic or stochastic performance measures.The optimization model involves expected and probabilistic formulations that are analytically intractable. This implies that traditional mathematical programming techniques cannot be used directly. Therefore, we propose adapted rapid-screening and stochastic-approximation algorithms to deal with the first-stage and the second-stage problems, respectively. In both algorithms, we can apply either the Laplace transform or simulation methods to either evaluate or estimate the desired performance measures. The experimental results demonstrate that the proposed algorithms are more favorable compared to existing approaches.

Quasi-maximum likelihood estimation of the Component-GARCH model using the stochastic approximation algorithm with application to the S&amp;P 500

The component GARCH (CGARCH) is suitable to better capture the short and long term of the volatility dynamic. Nevertheless, the parameter space constituted by the constraints of the non-negativity of the conditional variance, stationary and existence of moments, is only ex-post defined via the GARCH representation of the CGARCH. This is due to the lack of a general method to determine a priori the relaxed constraints of non-negativity of the CGARCH($N$) conditional variance for any $N\geq 1$. In this paper, a CGARCH parameter space constructed from the GARCH(1,1) component parameter spaces is provided a priori to identifying its GARCH form. Such a space fulfils the relaxed constraints of the CGARCH conditional variance non-negativity to be pre-estimated ensuring the existence of a QML estimation in the sense of the stochastic approximation algorithm. Simulation experiment as well as empirical application to the S\&amp;P500 index are presented and both show the performance of the proposed method.

Joint Calibration for DTA Model Using Islands-GA and PC-SPSA

Dynamic Traffic Assignment (DTA) models are widely used in transportation system management. Calibration is a crucial step to improve the reliability and the accuracy of DTA models. We present a systematic framework to offline calibrate the supply and demand component of a DTA model. The essence of model calibration is an optimization problem, aiming to minimize the discrepancy between field conditions and simulated traffic measurements. To overcome limitations of a single optimization algorithm, a joint approach is developed for the calibration of supply and demand component respectively with different traffic measurements. As the calibration process is a nonlinear and stochastic problem, heuristic algorithms: the Genetic Algorithm (GA) and the Simultaneous Perturbation Stochastic Approximation (SPSA) Algorithm, are implemented as a complement solution. Instead of using the standard GA, to expedite searching efficiency, we introduce the Islands Genetic Algorithm (IGA) and SPSA with Principal Component Analysis (PC-SPSA) to solve the calibration problem. A case study on a network of Munich, Germany, is used to validate the proposed methodology. The promising results indicate that calibration of the supply and demand component of a DTA model with the proposed joint approach improves modelling accuracy. In comparison, IGA outperforms standard GA in terms of convergence speed and solution quality.

Outage-Constrained Robust Beamforming for Intelligent Reflecting Surface Aided Wireless Communication

In intelligent reflecting surface (IRS) aided wireless communication systems, channel state information (CSI) is crucial to achieve its promising passive beamforming gains. However, CSI errors are inevitable in practice and generally correlated over the IRS reflecting elements due to the limited training with discrete phase shifts, which degrade the data transmission rate and reliability. In this paper, we focus on investigating the effect of CSI errors to the outage performance in an IRS-aided multiuser downlink communication system. Specifically, we aim to jointly optimize the active transmit precoding vectors at the access point (AP) and passive discrete phase shifts at the IRS to minimize the AP's transmit power, subject to the constraints on the maximum CSI-error induced outage probability for the users. First, we consider the single-user case and derive the user's outage probability in terms of the mean signal power (MSP) and variance of the received signal at the user. Since there is a trade-off in tuning these two parameters to minimize the outage probability, we propose to maximize their weighted sum with the optimal weight found by one-dimensional search. Then, for the general multiuser case, since the users' outage probabilities are difficult to obtain in closed-form due to the inter-user interference, we propose a novel constrained stochastic successive convex approximation (CSSCA) algorithm, which replaces the non-convex outage probability constraints with properly designed convex surrogate approximations. Simulation results verify the effectiveness of the proposed robust beamfoming algorithms and show their significant performance improvement over various benchmark schemes.

On stochastic accelerated gradient with non-strongly convexity

&lt;abstract&gt;&lt;p&gt;In this paper, we consider stochastic approximation algorithms for least-square and logistic regression with no strong-convexity assumption on the convex loss functions. We develop two algorithms with varied step-size motivated by the accelerated gradient algorithm which is initiated for convex stochastic programming. We analyse the developed algorithms that achieve a rate of $ O(1/n^{2}) $ where $ n $ is the number of samples, which is tighter than the best convergence rate $ O(1/n) $ achieved so far on non-strongly-convex stochastic approximation with constant-step-size, for classic supervised learning problems. Our analysis is based on a non-asymptotic analysis of the empirical risk (in expectation) with less assumptions that existing analysis results. It does not require the finite-dimensionality assumption and the Lipschitz condition. We carry out controlled experiments on synthetic and some standard machine learning data sets. Empirical results justify our theoretical analysis and show a faster convergence rate than existing other methods.&lt;/p&gt;&lt;/abstract&gt;

Stochastic approximation algorithms for superquantiles estimation

This paper is devoted to two different two-time-scale stochastic approximation algorithms for superquantile, also known as conditional value-at-risk, estimation. We shall investigate the asymptotic behavior of a Robbins-Monro estimator and its convexified version. Our main contribution is to establish the almost sure convergence, the quadratic strong law and the law of iterated logarithm for our estimates via a martingale approach. A joint asymptotic normality is also provided. Our theoretical analysis is illustrated by numerical experiments on real datasets.

Stochastic Hybrid Combining Design for Quantized Massive MIMO Systems

Both the power-dissipation and cost of massive multiple-input multiple-output (mMIMO) systems may be substantially reduced by using low-resolution analog-to-digital converters (LADCs) at the receivers. However, both the coarse quantization of LADCs and the inaccurate instantaneous channel state information (ICSI) degrade the performance of quantized mMIMO systems. To overcome these challenges, we propose a novel stochastic hybrid analog-digital combiner (SHC) scheme for adapting the hybrid combiner to the long-term statistics of the channel state information (SCSI). We seek to minimize the transmit power by jointly optimizing the SHC subject to average rate constraints. For the sake of solving the resultant nonconvex stochastic optimization problem, we develop a relaxed stochastic successive convex approximation (RSSCA) algorithm. Simulations are carried out to confirm the benefits of our proposed scheme over the benchmarkers.

Convergence Rates of Distributed Gradient Methods Under Random Quantization: A Stochastic Approximation Approach

We study gradient methods for solving distributed convex optimization problems over a network when the communication bandwidth between the nodes is limited, and so information that is exchanged between the nodes must be quantized. This imperfect communication poses a fundamental challenge, and if not properly accounted for may prevent the convergence of these algorithms. Our first contribution in this article is to propose a modified consensus-based gradient method for solving such problems using random (dithered) quantization. This algorithm can be interpreted as a distributed variant of a two-time-scale stochastic approximation algorithm. We then study the convergence and derive upper bounds on the rates of convergence of the proposed method as a function of the bandwidth available between the nodes and the underlying network topology for both convex and strongly convex objective functions. Our results complement existing literature, where such convergence and explicit formulas of the convergence rates are missing. Finally, we provide numerical simulations to compare the convergence properties of the distributed gradient methods with and without quantization for solving the well-known regression problems over networks, for both quadratic and absolute loss functions.

High-Mobility Multi-Modal Sensing for IoT Network via MIMO AirComp: A Mixed-Timescale Optimization Approach

Multiple-input-multiple-output (MIMO) over-the-air Computation (AirComp) is a promising technique for implementing high-mobility multi-modal sensing (HMS) to compute multiple target functions of distributed data by leveraging the superposition property of wireless multi-access channel, thereby significantly improving the computation efficiency as compared to the traditional orthogonal multi-access scheme. However, two obstacles constraining the potential gain of MIMO AirComp are high hardware cost and power consumption, and huge channel state information (CSI) signaling overhead. To overcome these challenges, we propose a novel mixed-timescale hybrid combining (MHC) scheme to minimize the average mean-square error (MSE) under some practical constraints in the uplink of a IoT network. Specifically, the radio frequency (RF) combiner is adapted to the long-term statistical CSI, while the baseband combiner is adapted to the real-time effective CSI. In order to tackle the resulting stochastic nonconvex optimization problem, we devise a mixed-timescale stochastic successive convex approximation (MSCA) algorithm. Simulation results are presented to verify the effectiveness of our proposed scheme over the state-of-the-art baselines.

Nonparametric relative recursive regression estimators for censored data

In this paper, we propose a relative recursive regression estimator for censored data defined by the stochastic approximation algorithm to deal with the presence of outliers or when the response is usually positive. We give the central limit theorem and the strong pointwise convergence rate for our proposed nonparametric relative recursive estimators under some mild conditions. We finally developed a second generation plug-in bandwidth selection procedure.

Stochastic approximation with nondecaying gain: Error bound and data‐driven gain‐tuning

SummaryOur problem of interest is to minimize a sequence of scalar‐valued loss functionsfk(·)or to locate the root(s) of a sequence of vector‐valued functionsgk(·)corresponding to timewith respect to a parameter. The available information for the minimization/root‐finding is the noise‐corrupted observation(s) of eitherfk(·)orgk(·)evaluated at afewof certain design pointsonly. Taking both the dynamics and randomness into consideration, we use stochastic approximation (SA) algorithms to solve the time‐varying problem, but with a nondecaying gain (stepsize). The tracking performance of the nondecaying gain SA algorithm is guaranteed by a computable bound for the root‐mean‐squared error and mean‐absolute‐deviation. The only assumption imposed on the drifts is that the average distance between two consecutive underlying true parameters is bounded from above—this assumption allows the varying target to change abruptly. The error term arising from using the noisy evaluation offk(·)orgk(·)in constructing the search direction is assumed to have bounded second‐moment—this allows for biased estimator to be used in the SA algorithms. Given the lenient assumptions on the drift and error term, the error bounds apply to a broad class of time‐varying scenarios and are useful for finite‐sample analysis. Since the tracking capability characterizes the average performance across all possible sample paths, it may not provide much guidance on the gain‐tuning strategies useful for one single run. Hence, we propose a data‐dependent gain‐tuning strategy based on estimating the Hessian information and the noise level. The adaptive rule is advantageous as it enables the step size to vary with information gathered during the progress of the estimation procedure.