Stochastic Approximation Procedure Research Articles

Accelerated distributed expectation-maximization algorithms for the parameter estimation in multivariate Gaussian mixture models

Rapid development for modeling big data requires effective and efficient methods for estimating the parameters involved. Although several accelerated Expectation-Maximization algorithms have been developed, there still exist two major concerns: reducing computational cost and improving model estimation accuracy. We propose three distributed-like algorithms for multivariate Gaussian mixture models, which can accelerate speed and improve estimation accuracy. The first algorithm is distributed algorithm, which is used to speed up the calculation of classic algorithms and improve its estimation accuracy by averaging the one-step estimators obtained from distributed operators. The second algorithm is distributed online algorithm, which is a distributed stochastic approximation procedure that performs online updates when reading online data. The final algorithm is called distributed monotonically over-relaxed algorithm, which uses an over-relaxation factor and a distributing strategy to improve the estimation accuracy of multivariate Gaussian mixture models. We investigate the stability, sensitivity, convergence, and robustness of these algorithms in a numerical study. We also apply these algorithms to three real data sets for validation.

The reversibility of cancelable biometric templates based on iterative perturbation stochastic approximation strategy

Researchers have been proposing different matching frameworks for cancelable template design to improve the matching performance and security of biometric authentication systems. Despite the advantages provided by these systems, they are often vulnerable to different attacks due to their templates' invertible nature. This study utilized an iterative perturbation stochastic approximation procedure to analyze the irreversible order-based encoding approach used in cancelable biometric template transformation. The strategy begins by exploiting the scores corresponding to the encoded words, then a mixed variable-based iterative perturbation is used to generate independent random elements and consequently the corresponding template from the scores. The strategy exploits the vulnerability of three different cancelable systems it has tested, demonstrating that other techniques of similar nature fall short of the security standard for attack mitigation that cancelable biometric systems require. After analyzing the reasons that exploit the system's vulnerability, we use a parameterized thresholding non-uniform quantization as a countermeasure to boost the system's robustness while maintaining a balanced performance-security trade-off. As a result, the system can evade attacks without significantly hindering its matching performance. Finally, generality and computational complexity evaluations on different modalities and strategies validate the attack's efficacy and realism, respectively.

Stochastic Approximation Procedures for Lévy-Driven SDEs

We consider a continuous-time Robbins–Monro-type stochastic approximation procedure for a system described by a (multidimensional) stochastic differential equation driven by a general Lévy process, and we find sufficient conditions for its convergence in terms of Lyapunov functions. While the jump part of the noise may spoil convergence to the root of the drift in some cases, we show that by a suitable choice of noise coefficients we obtain convergence under hypotheses on the drift weaker than those used in the diffusion case or convergence to a selected root in the case of multiple roots of the drift.

Application Stochastic Approximation Procedure in the Presence of Delayed Groups of Clients

Application Stochastic Approximation Procedure in the Presence of Delayed Groups of Clients

Finite-Time Error Bounds of Biased Stochastic Approximation With Application to TD-Learning

Motivated by the recent success of reinforcement learning algorithms, this paper studies a class of biased stochastic approximation (SA) procedures under a mild “ergodicity-like” assumption on the random noise sequence. Building on a multistep Lyapunov function that looks ahead to several future updates to accommodate the stochastic perturbations (thus gaining control over the bias), we prove a general result on the convergence of the SA iterates, and use it to derive non-asymptotic bounds on the mean-square error in the case of constant stepsizes. This novel viewpoint renders finite-time analysis of biased SA algorithms under a family of stochastic perturbations possible. For direct comparison with prior work, we demonstrate these bounds by applying them to TD-learning with linear function approximation, under the Markov chain observation model. The resultant finite-time error bound for TD-learning is the first of its kind, in the sense that it holds i) for the unmodified versions (i.e., without any modification to the updates) using even nonlinear approximators; as well as for Markov chains ii) under sublinear mixing conditions and iii) starting from any initial distribution, at least one of which has to be violated for existing results to be applicable.

Discrete Time Multi-servers Loss Systems and Stochastic Approximation with Delayed Groups of Customers

The stochastic approximation procedure with delayed groups of delayed customers is investigated. The Robbins-Monro stochastic approximation procedure is adjusted to be usable in the presence of delayed groups of delayed customers. Two loss systems are introduced to get an accurate description of the proposed procedure. Each customer comes after fixed time-intervals with the stage of the following customer is accurate according to the outcome of the preceding one, where the serving time of a customer is assumed to be discrete random variable. Some applications of the procedure are given where the analysis of their results is obtained. The analysis shows that efficiencies of the procedure can be increased by minimizing the number of customers of a group irrespective of their service times that may take maximum values. Efficiencies depend on the maximum service time of the customer and on the number of customers of the group. The most important result is that efficiencies of the procedure are increased by increasing the service time distributions as well as service times of customers .This new situation can be applied to increase the number of served customers where the number of served groups will also be increased. The results obtained seem to be acceptable. In general, our proposal can be utilized to other stochastic approximation procedures to increase the production in many fields such as medicine, computer sciences, industry, and applied sciences.

Control problem for the impulse process under stochastic optimization procedure and Levy conditions

A stochastic approximation procedure and a limit generator of the original problem are constructed for a system of stochastic differential equations with Markov switching and impulse perturbation under Levy approximation conditions with control, which is determined by the condition for the extremum of the quality criterion function.The control problem using the stochastic optimization procedure is a generalization of the control problem with the stochastic approximation procedure, which was studied in previous works of the authors. This generalization is not simple and requires non-trivial approaches to solving the problem. In particular we discuss how the behavior of the boundary process depends on the prelimiting stochastic evolutionary system in the ergodic Markov environment. The main assumption is the condition for uniform ergodicity of the Markov switching process, that is, the existence of a stationary distribution for the switching process over large time intervals. This allows one to construct explicit algorithms for the analysis of the asymptotic behavior of a controlled process. An important property of the generator of the Markov switching process is that the space in which it is defined splits into the direct sum of its zero-subspace and a subspace of values, followed by the introduction of a projector that acts on the subspace of zeros.For the first time, a model of the control problem for the diffusion transfer process using the stochastic optimization procedure for control problem is proposed. A singular expansion in the small parameter of the generator of the three-component Markov process is obtained, and the problem of a singular perturbation with the representation of the limiting generator of this process is solved.

Задача керування для імпульсного процесу в умовах процедури стохастичної оптимізації та некласичних схем апроксимації

A stochastic approximation procedure and a limit generator of the original problem are constructed for a system of stochastic differential equations with Markov switching and impulse perturbation under noclassical approximation conditions with control, which is determined by the condition for the extremum of the quality criterion function. The control problem using the stochastic optimization procedure is a generalization of the control problem with the stochastic approximation procedure, which was studied in previous works of the authors. This generalization is not simple and requires non-trivial approaches to solving the problem. In particular we discuss how the behavior of the boundary process depends on the pre-limiting stochastic evolutionary system in the ergodic Markov environment. The main assumption is the condition for uniform ergodicity of the Markov switching process, that is, the existence of a stationary distribution for the switching process over large time intervals. This allows one to construct explicit algorithms for the analysis of the asymptotic behavior of a controlled process. An important property of the generator of the Markov switching process is that the space in which it is defined splits into the direct sum of its zero-subspace and a subspace of values, followed by the introduction of a projector that acts on the subspace of zeros. For the first time, a model of the control problem for the diffusion transfer process using the stochastic optimization procedure for control problem is proposed. A singular expansion in the small parameter of the generator of the three-component Markov process is obtained, and the problem of a singular perturbation with the representation of the limiting generator of this process is solved \KeywordsEng random evolution, nonclassical approximation conditions, stochastic optimization procedure, Markov process.

Stochastic approximation procedure in semi-Markov environment applied to alcohol consumption model

Stochastic approximation procedure in semi-Markov environment applied to alcohol consumption model

Asymptotic Properties of the Impulse Perturbation Process under the Poisson Approximation Conditions with the Equilibrium Point of Quality Criterion

For a system of stochastic differential equations with Markov switchings and impulse perturbation under the Poisson approximation scheme and the conditions of the existence of a single equilibrium point of the quality criterion, the limit generators for the impulse process and the dynamic system are constructed. The complexity of the proposed evolutionary model lies in its three properties. First, the system is under conditions of an external random effect, which is modeled using the Markov switching process. Processes with independent increments, which also depend on the Markov switching process, have certain characteristics between the moments of its restoration, and at the moments of restoration, these characteristics change. Therefore, the so-called "gluing" of trajectories of processes with independent increments occurs. Second, the model contains a Poisson approximation scheme, which is a generalization of the classical averaging scheme and is determined by normalization depending on a small parameter. In the classical approximation scheme in the limit process, we do not see large jumps in the system. The maximum that we get is the shift of the deterministic trajectory. However in the Poisson approximation scheme, which was invented by Korolyuk and Limnios in the monograph of 2005, this problem is eliminated, therefore, both a deterministic shift and large jumps will take place in the limit. Third, the system has a control function, which is determined using the Robins-Monroe stochastic approximation procedure. This procedure solves the problem of determination of the equilibrium point of the regression function and consists in finding the unique solution to the equation relative to control. Assuming the existence of a unique control on every interval, we solve a two-level problem. The question, how the behavior of the limit process depends on the prelimit normalization of the stochastic system in an ergodic Markov environment is studied in the article.

Random Optimization Method to Reduce Cogging Torque of Interior Permanent Magnet Synchronous Motor

The interior permanent magnet synchronous motor (IPMSM) has many advantages like high torque density, high efficiency, and a wide range of operation area since the reluctance torque and magnetic torque can be applied simultaneously. However, cogging torque that is generated by difference in magnetic resistance due to salient rotor topology and its position is one of the most important factors to be considered while designing an IPMSM as it causes vibration and noise. This paper presents a method for reduction of cogging torque in IPMSM by using random optimization. Random optimization is a constrained stochastic approximation procedure that uses random-direction finite difference gradient estimates. Random optimization is advantageous in cases where the objective function is unclear or cannot be differentiated. In this paper, the use of improved random optimization technique than the existing one for optimal design of IPMSM with minimized cogging torque is proposed.

A Discrete-Continuous Hybrid Approach to Periodic Routing of Waste Collection Vehicles With Recycling Operations

Waste management agencies need to plan their waste collection activities in an efficient way such that they not only provide high-quality and timely service to customers but also maximize their net profit from recyclable waste, i.e., the total recycling revenue minus total operating costs. This paper proposes a mixed-integer linear program model for the waste collection problem in urban areas while considering the recycling operation at the sorting facility. A hybrid solution approach with both discrete and continuous optimization components is developed to solve the problem. The continuous component builds upon asymptotic routing cost formulas from the continuous approximation literature, while it is integrated into the discrete optimization component via an iterative stochastic approximation procedure. A series of numerical experiments are conducted, and results show that the proposed model and hybrid solution approach significantly outperform state of the art benchmarks from the literature.

An Application Using Stochastic Approximation Method for Improvement Specific Loss System

In this paper, an application using the modified stochastic approxi-mation procedure which studied to answer the question for Robbins-Monroe procedure. The modified procedure depends on a new form. We use the case of loss system obey Negative Binomial distribution. The efficiency of the proposed procedure is calculated to determine the ways to improve the mentioned loss system, the results which are obtained show that our procedure can serve as a model of stochastic approximation with delayed observations. This new topic can be applied in many fields such as the biological, medical, life time experiments, and some industrial projects, to increase the production, where in our system we depend on, observe a lot items and this items are realized after random time delays. In this paper we referred to the conditions which improve the proposed loss system.

Strong convergence result for solution of stochastic convex minimization problem generated by generalized random iterative scheme

Stochastic approximation procedures have been extensively used in solving stochastic optimization problems. Gradient projection algorithm (GPA) plays an important role in solving constrained convex minimization problems. In this paper, strong convergence result for approximate solution of a constrained stochastic convex minimization problem in Hilbert space is proved. A generalized random iterative scheme based on skewed gradient projection algorithm and a strictly pseudo-contractive mapping is used to generate the solution.

Procedure of Stochastic Approximation for the Diffusion Process with Semi-Markov Switchings

We establish sufficient conditions for the convergence of the procedure of stochastic approximation for the diffusion process in the case of a uniformly ergodic semi-Markov process of switchings of the regression function with the use of a small parameter in the scheme of series.

��I��I��� ��������� ����������ί ����������I� �� ������� ���������-��������

The main asymptotic property of the continuous stochastic approximation procedure, namely the convergence to the Ornstein-Uhlenbeck process, is considered. The regression function of the procedure depends on a uniformly ergonomic Markov process, which describes the external inuence in the form of switches. To obtain sucient conditions for the convergence of the procedure, we use ergodic properties of the generating Markov process generator, the existence of a stationary distribution of this process, and the existence of the potential for a generating Markov process generator. The stochastic approximation procedure, as a random process, is constructed in the form of a dierential equation in the xed state of a Markov process with a corresponding generator of the equation. Asymptoticity over time is achieved by using a small parameter that normalizes time. This made it possible to obtain a normalized stochastic approximation procedure and its dierential representation. The generator of the obtained dierential equation is used to construct a generator of a two-component Markov process, which consists of a procedure and a switching process. The singular representation of the last generator by a small parameter makes it possible to solve the singular perturbation problem and determine the form of the limited generator. Such form denes the representation of a limited process as the Ornstein-Uhlenbeck diusion process. Note that the convergence to the limited process is weak, which follows from the Koroliuk's theorem. The conditions for the existence of the Lyapunov function for the dynamic procedure, which is averaged by the stationary distribution of the Markov process, are important. Additional conditions on the Lyapunov function make it possible to establish the boundedness of the residual terms of the solution of a singular perturbation. The one-dimensional case of the procedure can be expanded to multidimensional with the corresponding complication of calculations of the components of the limited generator. The work summarizes the studies of Nevelson and Khasminsky in the case of a direct inuence of the Markov process on the regression function.

Efficient Adaptive Speech Reception Threshold Measurements Using Stochastic Approximation Algorithms.

This study examines whether speech-in-noise tests that use adaptive procedures to assess a speech reception threshold in noise (SRT50n) can be optimized using stochastic approximation (SA) methods, especially in cochlear-implant (CI) users. A simulation model was developed that simulates intelligibility scores for words from sentences in noise for both CI users and normal-hearing (NH) listeners. The model was used in Monte Carlo simulations. Four different SA algorithms were optimized for use in both groups and compared with clinically used adaptive procedures. The simulation model proved to be valid, as its results agreed very well with existing experimental data. The four optimized SA algorithms all provided an efficient estimation of the SRT50n. They were equally accurate and produced smaller standard deviations (SDs) than the clinical procedures. In CI users, SRT50n estimates had a small bias and larger SDs than in NH listeners. At least 20 sentences per condition and an initial signal-to-noise ratio below the real SRT50n were required to ensure sufficient reliability. In CI users, bias and SD became unacceptably large for a maximum speech intelligibility score in quiet below 70%. In conclusion, SA algorithms with word scoring in adaptive speech-in-noise tests are applicable to various listeners, from CI users to NH listeners. In CI users, they lead to efficient estimation of the SRT50n as long as speech intelligibility in quiet is greater than 70%. SA procedures can be considered as a valid, more efficient, and alternative to clinical adaptive procedures currently used in CI users.

Generalization of a Result of Fabian on the Asymptotic Normality of Stochastic Approximation

Stochastic approximation (SA) is a general framework for analyzing the convergence of a large collection of stochastic root-finding algorithms. The Kiefer–Wolfowitz and stochastic gradient algorithms are two well-known (and widely used) examples of SA. Because of their applicability to a wide range of problems, many results have been obtained regarding the convergence properties of SA procedures. One important reference in the literature, Fabian (1968), derives general conditions for the asymptotic normality of the SA iterates. Since then, many results regarding asymptotic normality of SA procedures have relied heavily on Fabian’s theorem. Unfortunately, some of the assumptions of Fabian’s result are not applicable to some modern implementations of SA in control and learning. In this paper we explain the nature of this incompatibility and show how Fabian’s theorem can be generalized to address the issue.

МЕТОД И АЛГОРИТМЫ ОБНАРУЖЕНИЯ АНОМАЛИИ В ТРАФИКЕ МУЛЬТИСЕРВИСНЫХ СЕТЕЙ СВЯЗИ, ОСНОВАННЫЕ НА НЕЧЕТКОМ ЛОГИЧЕСКОМ ВЫВОДЕ

Introduction: The major requirement imposed to the systems of detection and prevention of malicious invasions into modern telecommunication infrastructures is the ability to find anomalies and invasion threats in real time. The complexity of this problem is in many respects caused by the incompleteness, discrepancy and diversity of the a priori knowledge about the distribution laws peculiar to the traffic of multiservice communication networks, as well as by the changing nature of malicious actions which make computer systems unsafe. Purpose: Increasing the speed and reliability of network traffic anomaly detection when the analyzed information is incomplete and highly heterogeneous. Results: A hybrid method and adaptive algorithms have been proposed for real-time anomaly detection in multiservice communication network traffic. The hybrid method unites the mechanism of non-identificational adaptation to the changing traffic parameters with the fuzzy logical inference used for regulating the algorithm parameters and for analyzing the output data. The adaptive algorithms are focused on combined implementation of modified stochastic approximation and pseudogradient search procedures. An experimental assessment has shown that the functional characteristics of the algorithms are close to the potentially achievable ones. Practical relevance: The developed method and algorithms can be implemented on available hardwaresoftware platforms on the basis of intellectual agent technology. Sharing them with already existing methods and algorithms of invasion detection can considerably increase the efficiency of information security systems in multiservice communication networks.

Stochastic approximation with series of delayed observations

Abstract The stochastic approximation procedure with series of delayed observations is investigated. The procedure is formed by modifying the Robbins–Monro stochastic approximation procedure to be applicable in the presence of series of delayed observations. The modified procedure depends on a new base concerning the relation between service time of the series and service times of its components. Two loss systems are introduced for application to the proposed procedure. This new situation can be applied to increase the production of items in many fields such as biological, medical, life time experiments, and some industrial projects, where items are realized after random time delays. The efficiency of the procedure is computed. Our proposal is general and we expect that it can be applied to any other stochastic approximation procedure.