Discretization Of Stochastic Differential Equations Research Articles

Understanding Short-Range Memory Effects in Deep Neural Networks.

Stochastic gradient descent (SGD) is of fundamental importance in deep learning. Despite its simplicity, elucidating its efficacy remains challenging. Conventionally, the success of SGD is ascribed to the stochastic gradient noise (SGN) incurred in the training process. Based on this consensus, SGD is frequently treated and analyzed as the Euler-Maruyama discretization of stochastic differential equations (SDEs) driven by either Brownian or Lévy stable motion. In this study, we argue that SGN is neither Gaussian nor Lévy stable. Instead, inspired by the short-range correlation emerging in the SGN series, we propose that SGD can be viewed as a discretization of an SDE driven by fractional Brownian motion (FBM). Accordingly, the different convergence behavior of SGD dynamics is well-grounded. Moreover, the first passage time of an SDE driven by FBM is approximately derived. The result suggests a lower escaping rate for a larger Hurst parameter, and thus, SGD stays longer in flat minima. This happens to coincide with the well-known phenomenon that SGD favors flat minima that generalize well. Extensive experiments are conducted to validate our conjecture, and it is demonstrated that short-range memory effects persist across various model architectures, datasets, and training strategies. Our study opens up a new perspective and may contribute to a better understanding of SGD.

Conditional Loss and Deep Euler Scheme for Time Series Generation

We introduce three new generative models for time series that are based on Euler discretization of Stochastic Differential Equations (SDEs) and Wasserstein metrics. Two of these methods rely on the adaptation of generative adversarial networks (GANs) to time series. The third algorithm, called Conditional Euler Generator (CEGEN), minimizes a dedicated distance between the transition probability distributions over all time steps. In the context of Itô processes, we provide theoretical guarantees that minimizing this criterion implies accurate estimations of the drift and volatility parameters. Empirically, CEGEN outperforms state-of-the-art and GANs on both marginal and temporal dynamic metrics. Besides, correlation structures are accurately identified in high dimension. When few real data points are available, we verify the effectiveness of CEGEN when combined with transfer learning methods on model-based simulations. Finally, we illustrate the robustness of our methods on various real-world data sets.

Efficient discretisation of stochastic differential equations

The aim of this study is to find a generic method for generating a path of the solution of a given stochastic differential equation which is more efficient than the standard Euler–Maruyama scheme with Gaussian increments. First we characterize the asymptotic distribution of pathwise error in the Euler–Maruyama scheme with a general partition of time interval and then, show that the error is reduced by a factor when using a partition associated with the hitting times of sphere for the driving d-dimensional Brownian motion. This reduction ratio is the best possible in a symmetric class of partitions. Next we show that a reduction which is close to the best possible is achieved by using the hitting time of a moving sphere that is easier to implement.

Non-asymptotic error bounds for the multilevel Monte Carlo Euler method applied to SDEs with constant diffusion coefficient

In this paper, we are interested in deriving non-asymptotic error bounds for the multilevel Monte Carlo method. As a first step, we deal with the explicit Euler discretization of stochastic differential equations with a constant diffusion coefficient. We prove that, as long as the deviation is below an explicit threshold, a Gaussian-type concentration inequality optimal in terms of the variance holds for the multilevel estimator. To do so, we use the Clark-Ocone representation formula and derive bounds for the moment generating functions of the squared difference between a crude Euler scheme and a finer one and of the squared difference of their Malliavin derivatives.

On two numerical problems in applied probability : discretization of Stochastic Differential Equations and optimization of an expectation depending on a parameter

In the present paper, we first deal with the discretization of stochastic dierential equa- tions. We elaborate on the analysis of the weak error of the Euler scheme by Talay and Tubaro (31) to contruct schemes with quicker weak rate of convergence for SDEs corresponding to an infinitesimal generator with smooth coecients. We also extend this analysis to the case of a discontinuous drift coecient. In a second part, we present two applications of stochastic gradient algorithms in finance.

A note on convergence rate of a linearization method for the discretization of stochastic differential equations

This note discusses convergence rate of a linearization method for the discretization of stochastic differential equations with multiplicative noise. The method is to approximate the drift coefficient by the local linearization method and the diffusion coefficient by the Euler method. The mixed method guarantees the approximated process converges to the original one with the rate of convergence Δ t, where Δ t is the time interval of discretization.

A new numeric-analytical principle for nonlinear deterministic and stochastic dynamical systems

A new implicit local linearization method termed as the locally transversal linearization (LTL) is developed in the present paper for a numeric‐analytical integration of nonlinear dynamical systems under deterministic and/or stochastic excitations. The LTL procedure is developed in such a way that, at a chosen time‐instant, the LTL solution manifold transversally intersects the nonlinear solution manifold at that particular point or cross‐section in the state‐space where the solution vector needs to be obtained. In general, construction of the LTL equation is non‐unique and thus it is generally needed to substitute the conditionally linear LTL‐based solution into the given nonlinear ordinary differential equation (ODE) at a specified time‐instant to ensure a transversal intersection. In the present study, however, it is demonstrated that a transversal intersection at a given time‐instant may as well be ensured by just choosing a few specific forms of the LTL equation, thereby eliminating the need for substitution. This new variant of LTL is particularly suitable for discretization of stochastic differential equations (SDEs) in that one simultaneously avoids dealing with formal derivatives of Wiener processes while satisfying the nonlinear SDE at discrete time‐points. The stochastic LTL procedure is adaptable for obtaining strong pathwise solutions of a general nonlinear dynamical system with continuous (but not necessarily differentiable) vector fields under additive and/or multiplicative excitations. A specific advantage of the LTL method is that for a class of response regime, herein referred to as the phase‐independent regime, it can obtain an accurate description of response irrespective of the choice of the time‐step and without a propagation of local errors. Presently, the method has been numerically illustrated for strong solutions of the hardening Duffing oscillator under deterministic (sinusoidal) or additive stochastic (white noise) excitations or their combination. Limited numerical results on periodic solutions, chaos with or without noise and stochastic resonance are included in this study. The results presented are indicative of the sample path accuracy, consistency, stochastic numerical stability and a partial insensitivity to time‐step sizes of the proposed method.

The Optimal Discretization of Stochastic Differential Equations

We study pathwise approximation of scalar stochastic differential equations. The mean squared L2-error and the expected number n of evaluations of the driving Brownian motion are used for the comparison of arbitrary methods. We introduce an adaptive discretization that reflects the local properties of every single trajectory. The corresponding error tends to zero like c·n−1/2, where c is the average of the diffusion coefficient in space and time. Our method is justified by the matching lower bound for arbitrary methods that are based on n evaluations on the average. Hence the adaptive discretization is asymptotically optimal. The new method is very easy to implement, and about 7 additional arithmetical operations are needed per evaluation of the Brownian motion. Hereby we can determine the complexity of pathwise approximation of stochastic differential equations. We illustrate the power of our method already for moderate accuracies by means of a simulation experiment.

Discretization of stochastic differential equations by the product expansion for the chen series

Following the approach of Sussmann [16] to expand the Chen series as a product of an exponential Lie series based on a Philip Hall basis, this paper constructs the Stratonovich-Taylor-Hall (STH) discretization schemes of both integral and fractional orders for stochastic differential equations. The STH schemes are numerically efficient in the sense that they involve only the minimum numbers of multiple stochastic integrals. General multiple stochastic integrals are regarded as systems of stochastic differential equations that can be solved by lower order schemes. Convergence of the STH schemes is proved in the almost sure sense by means of the discrete stochastic Gronwall inequality

Time discretization of continuous-time filters and smoothers for HMM parameter estimation

In this paper we propose algorithms for parameter estimation of fast-sampled homogeneous Markov chains observed in white Gaussian noise. Our algorithms are obtained by the robust discretization of stochastic differential equations involved in the estimation of continuous-time hidden Markov models (HMM's) via the EM algorithm. We present two algorithms: the first is based on the robust discretization of continuous-time filters that were recently obtained by Elliott to estimate quantities used in the EM algorithm; the second is based on the discretization of continuous-time smoothers, yielding essentially the well-known Baum-Welch re-estimation equations. The smoothing formulas for continuous-time HMM's are new, and their derivation involves two-sided stochastic integrals. The choice of discretization results in equations which are identical to those obtained by deriving the results directly in discrete time. The filter-based EM algorithm has negligible memory requirements; indeed, independent of the number of observations. In comparison the smoother-based discrete-time EM algorithm requires the use of the forward-backward algorithm, which is a fixed-interval smoothing algorithm and has memory requirements proportional to the number of observations. On the other hand, the computational complexity of the filter-based EM algorithm is greater than that of the smoother-based scheme. However, the filters may be suitable for parallel implementation. Using computer simulations we compare the smoother-based and filter-based EM algorithms for HMM estimation. We provide also estimates for the discretization error.

Discretization of stochastic differential equations and econometric forecasting: An application to time‐varying autoregressions

AbstractContinuous time versions of time varying Vector Autoregressions are stochastic differential equations. Optimal discretization of Stochastic Differential Equations cannot be obtained by replacing all differentials by the corresponding first differences. In this paper we obtain the optimal discretization for time varying VARS. The results are applied to predicting the consumer price index.