Finite Difference Stochastic Approximation Research Articles

Optimization of ultraviolet communication links based on finite difference stochastic approximation.

The ultraviolet communication (UV) channel has been shown to have unique features that could be exploited for covert ground-to-ground communications in complex non-line-of-sight (NLOS) scenarios. A key challenge is the determination of optimal configuration of pointing directions of the UV nodes in unknown NLOS environments to maximize the link performance. In this paper, we proposed a novel steering optimization approach based on Finite Difference Stochastic Approximation (FDSA) to simultaneously optimize the transmitter (Tx) and receiver (Rx) pointing directions without any knowledge about the locations and relative orientations of the two nodes. We perform parametric analysis using Monte Carlo channel simulations to investigate and select appropriate key algorithmic parameters and analyze the performance of the proposed algorithm. We also carry out experimentation using our custom designed UV Tx and Rx gimbal systems and demonstrate the utility and efficiency of the proposed steering optimization approach and show that the received photon count can be increased significantly.

Gk: An R Package for the g-and-k and Generalised g-and-h Distributions

The g-and-k and (generalised) g-and-h distributions are flexible univariate distributions which can model highly skewed or heavy tailed data through only four parameters: location and scale, and two shape parameters influencing the skewness and kurtosis. These distributions have the unusual property that they are defined through their quantile function (inverse cumulative distribution function) and their density is unavailable in closed form, which makes parameter inference complicated. This paper presents the gk R package to work with these distributions. It provides the usual distribution functions and several algorithms for inference of independent identically distributed data, including the finite difference stochastic approximation method, which has not been used before for this problem.

Production Optimization of Polymer Flooding Using Improved Monte Carlo Gradient Approximation Algorithm with Constraints

Aiming at optimizing polymer flooding, we establish an optimal control model of polymer flooding, which has an objective function of the net present value (NPV) involving the effect of polymer injection. An improved Monte Carlo gradient approximation (MCGA) algorithm, based on the idea of the ensemble-based optimization (EnOpt) scheme to solve the problem of strongly fluctuating perturbation gradients, is proposed by introducing the covariance matrix of the control vectors to filter and smooth the searching direction. A synthetic heterogeneous reservoir model is built to test the performance of the algorithms including the improved MCGA, standard MCGA and finite difference stochastic approximation (FDSA) algorithm. For the results, the improved MCGA gets closer to the optimal NPV of FDSA than the standard algorithm, and shows the high efficiency of saving calculation time compared with the FDSA. The value of NPV increases more than 20% for the improved algorithm, and the optimal production rates, injection rates, polymer concentrations, polymer slug sizes are obtained simultaneously. This paper subsequently discusses the influence of different time step sizes and oil prices. It can be concluded that moderate step size and relatively low oil price are applicable. Finally, an actual block application of the improved MCGA shows a 11.3% increase of NPV and a 6.5% increase of field oil production total (FOPT), showing the feasibility in optimizing real reservoirs.

System-level post-optimization of Delta–Sigma modulators using finite difference stochastic approximation

Traditional methods for system-level design of Delta---Sigma (źΣ) modulators typically assume linear modeling of the modulator, in which quantization noise is modeled as additive independent white noise. But it is well known that the źΣ modulator is a non-linear system and linear modeling is only an approximation. Also, circuit-level non-idealities, which may greatly change the modulator behavior, are typically neglected at system-level design. As a result, system-level modulator designs obtained from traditional methods may not be realistically optimal. This paper presents a system-level post-optimization method for źΣ modulators so that modulator designs initially obtained from traditional methods are post-optimized considering non-linear and non-ideal characteristics of źΣ modulators including both quantization noise and circuit-level non-idealities. The post-optimization algorithm is based on Finite Difference Stochastic Approximation due to the stochastic nature of modeling of some circuit-level non-idealities in system-level design. During the post-optimization run, each candidate design is simulated for performance measures and stability has always been a must constraint. Results on two źΣ modulators have shown that post-optimized modulator designs outperform the original designs from traditional methods.

Stopping Rules for Optimization Algorithms Based on Stochastic Approximation

Stopping rules are developed for stochastic optimization algorithms, which minimize an unknown objective function using noise corrupted measurements. In particular, the finite-difference stochastic approximation and the simultaneous perturbation stochastic approximation are considered. The candidate solution after an adequate number of iterations is shown to be sufficiently close to the optimal solution in a mean squared sense. These numbers are determined by a priori information only. Furthermore, it is shown that these are polynomial order of the problem size.

A robust and efficient triangulation-based optimization algorithm for stochastic black-box systems

Optimization of process variables is an important, yet difficult, task for systems-level analysis and design of complex stochastic systems. Here, we introduce the Simplex-Triangulation Optimization (STO) algorithm to optimize stochastic black-box systems efficiently in fewer iterations than other comparable algorithms without requiring gradient information or detailed initial guesses. The STO algorithm is shown to converge linearly. Several test functions are utilized to compare the STO algorithm to the Particle Swarm Optimization (PSO) and Finite Difference Stochastic Approximation (FDSA) algorithms, which are often used for parameter optimization in stochastic systems.

Adaptive Lp-norm diversity combining in non-gaussian noise and interference

In this paper, we introduce an adaptive Lp-norm metric for robust coherent, differential, and noncoherent diversity combining in non-Gaussian noise and interference. We consider the general case where all diversity branches may use different combining weights and different Lp-norms. We derive a general closed-form expression for the asymptotic bit error rate (BER) for Lp-norm combining in independent non-identically distributed Ricean fading and non-Gaussian noise and interference with finite moments. The asymptotic BER expression reveals that the diversity gain of Lp-norm combining is independent of the type of noise and the metric parameters. In contrast, the combining gain depends on both the type of noise and the metric parameters. Thus, the asymptotic BER can be minimized by optimizing the Lp-norm metric parameters for the underlying type of noise. For this purpose finite difference stochastic approximation (FDSA) and localized random search (LRS) algorithms are developed. Both adaptive algorithms do not require any a priori knowledge about the underlying noise and are able to track changes in the noise statistics. Simulation results confirm the validity of the derived asymptotic BER expressions, the effectiveness of the proposed adaptive algorithms, and the excellent performance of the proposed adaptive Lp-norm metric compared to other popular metrics.

Parameter Estimation Using a Modified Version of SPSA Algorithm Applied to State Space Models

The objective of this paper is the estimation of unknown static parameters in non-linear non-Gaussian state-space model. The Simultaneous Perturbation Stochastic Approximation (SPSA) algorithm is considered due to its highly efficient gradient approximation. We consider a particle filtering method and employ the SPSA algorithm to maximize recursively the likelihood function. Nevertheless, the SPSA algorithm can become inadequate in models as non-Gaussian state-space model. So that, we have proposed to modify the SPSA algorithm in order to estimate parameters very efficiently in complex models as proposed here reducing its computational cost. An efficient parameter estimator as the Finite Difference Stochastic Approximation (FDSA) algorithm is considered here, in order to compare it with the efficiency of the proposed SPSA algorithm. The proposed algorithm can generate maximum likelihood estimates very efficiently. The performance of proposed SPSA algorithm is shown through simulation using a model with highly multimodal likelihood.

On the use of an SPSA-based model-free feedback controller in active noise control for periodic disturbances in a duct

In this paper, a feedback active noise control (FANC) system using a model-free (MF) controller based on simultaneous perturbation stochastic approximation (SPSA) algorithm is considered. The structure of the FANC system is first described, and the SPSA algorithm is briefly reviewed. Subsequently, the SPSA-based MF control algorithm employed in the FANC system is derived. Computer simulations are carried out to suggest that the SPSA-based MF control algorithm is effective for ANC system and much more efficient than the finite-difference stochastic approximation (FDSA) algorithm. The controller based on the SPSA-based MF control algorithm is implemented on the Texas Instruments digital signal processor (DSP) TMS320VC33. Experimental results show that the proposed scheme is able to significantly reduce periodic disturbances without the need to model the secondary path. At the same time, the simulations and the experimental verification tests also show that the convergence rate of the SPSA-based MF control algorithm is acceptable, and the SPSA-based MF control algorithm has better tracking ability under variable secondary path. This observation implies that the SPSA-based MF controller eliminates the need of modeling of the secondary path for the FANC system.

The magic of SPSA

AbstractThe basics of SPSA (simultaneous perturbation stochastic approximation) initiated and developed in [1] will be desctribed. We point out the advantages of SPSA over the finite difference stochastic approximation (FDSA) or Kiefer‐Wolfowitz (KW) method. Its applicability in noise free optimization and in discrete optimization will be also briefly described. (© 2008 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

On the Choice of Random Directions for Stochastic Approximation Algorithms

Stochastic approximation provides a simple and effective approach for finding roots and minima of functions whose evaluations are contaminated with noise. We investigate variants of the random direction stochastic approximation (RDSA) algorithm for optimizing noisy loss functions in high-dimensional spaces. The most popular variant is random selection from a Bernoulli distribution, which also goes by the name simultaneous perturbation stochastic approximation (SPSA). Viable alternatives include an axis-aligned distribution, normal distribution, and uniform distribution on a spherical shell. Although there are special cases where the Bernoulli distribution is optimal, we identify other cases where it performs worse than the alternatives. We show that performance depends on the orientation of the loss function with respect to its coordinate axes, and consider averages over all orientations. We find that the average asymptotic performance depends only on the radial fourth moment of the distribution of random directions, and is identical for the Bernoulli, axis-aligned, and spherical shell distributions. Of these variants, the spherical shell is optimal in the sense of minimum variance over random orientations of the loss function with respect to the coordinate axes. We also show that for unaligned loss functions, the performance of the Kiefer-Wolfowitz-Blum finite difference stochastic approximation (FDSA) is asymptotically equivalent to the RDSA algorithms, and we observe numerically that the pre-asymptotic performance of FDSA is often superior.

Simulation-Based Optimization with Stochastic Approximation Using Common Random Numbers

The method of Common Random Numbers is a technique used to reduce the variance of difference estimates in simulation optimization problems. These differences are commonly used to estimate gradients of objective functions as part of the process of determining optimal values for parameters of a simulated system. Asymptotic results exist which show that using the Common Random Numbers method in the iterative Finite Difference Stochastic Approximation optimization algorithm (FDSA) can increase the optimal rate of convergence of the algorithm from the typical rate of k−1/3 to the faster k−1/2, where k is the algorithm's iteration number. Simultaneous Perturbation Stochastic Approximation (SPSA) is a newer and often much more efficient optimization algorithm, and we will show that this algorithm, too, converges faster when the Common Random Numbers method is used. We will also provide multivariate asymptotic covariance matrices for both the SPSA and FDSA errors.

Comparative study of stochastic algorithms for system optimization based on gradient approximations

Stochastic approximation (SA) algorithms can be used in system optimization problems for which only noisy measurements of the system are available and the gradient of the loss function is not. This type of problem can be found in adaptive control, neural network training, experimental design, stochastic optimization, and many other areas. This paper studies three types of SA algorithms in a multivariate Kiefer-Wolfowitz setting, which uses only noisy measurements of the loss function (i.e., no loss function gradient measurements). The algorithms considered are: the standard finite-difference SA (FDSA) and two accelerated algorithms, the random directions SA (RDSA) and the simultaneous-perturbation SA (SPSA). RDSA and SPSA use randomized gradient approximations based on (generally) far fewer function measurements than FDSA in each Iteration. This paper describes the asymptotic error distribution for a class of RDSA algorithms, and compares the RDSA, SPSA, and FDSA algorithms theoretically (using mean-square errors computed from asymptotic distributions) and numerically. Based on the theoretical and numerical results, SPSA is the preferable algorithm to use.