Underlying Probability Measure Research Articles

Underlying Probability Measure Approximated by Monte Carlo Simulations in Event Prognostics

The prognostic of events, and particularly of failures, is a key step towards allowing preventive decision-making, as in the case of predictive maintenance in Industry 4.0, for example. However, the occurrence time of a future event is subject to uncertainty, so it is natural to think of it as a random variable. In this regard, the default procedure (benchmark) to compute its probability distribution is empirical, through Monte Carlo simulations. Nonetheless, the analytic expression for the probability distribution of the occurrence time of any future event was presented and demonstrated in a recent publication. In this article it is established a direct relationship between these empirical and analytical procedures. It is shown that Monte Carlo simulations numerically approximate the analytically known probability measure when the future event is triggered by the crossing of a threshold.

Three invariants of strange attractors derived through hypergeometric entropy

A new description of strange attractor systems through three geometrical and dynamical invariants is provided. They are the correlation dimension (D) and the correlation entropy (K), both having attracted attention over the past decades, and a new invariant called the correlation concentration (A) introduced in the present study. The correlation concentration is defined as the normalised mean distance between the reconstruction vectors, evaluated by the underlying probability measure on the infinite-dimensional embedding space. These three invariants determine the scaling behaviour of the system’s Rényi-type extended entropy, modelled by Kummer’s confluent hypergeometric function, with respect to the gauge parameter (ρ) coupled to the distance between the reconstruction vectors. The entropy function reproduces the known scaling behaviours of D and K in the ‘microscopic’ limit ρ→∞ while exhibiting a new scaling behaviour of A in the other, ‘macroscopic’ limit ρ→0. The three invariants are estimated simultaneously via nonlinear regression analysis without needing separate estimations for each invariant. The proposed method is verified through simulations in both discrete and continuous systems.

Randomness extraction in computability theory

In this article, we study a notion of the extraction rate of Turing functionals that translate between notions of randomness with respect to different underlying probability measures. We analyze several classes of extraction procedures: (1) a class that generalizes von Neumann’s trick for extracting unbiased randomness from the tosses of a biased coin, (2) a class based on work of by Knuth and Yao (which more properly can be characterized as extracting biased randomness from unbiased randomness), and (3) a class independently developed by Levin and Kautz that generalizes the data compression technique of arithmetic coding. For the first two classes of extraction procedures, we identify a level of algorithmic randomness for an input that guarantees that we attain the extraction rate along that input, while for the third class, we calculate the rate attained along sufficiently random input sequences.

Strichartz and Local Smoothing Estimates for Stochastic Dispersive Equations with Linear Multiplicative Noise

We study a quite general class of stochastic dispersive equations with linear multiplicative noise, including especially the Schrodinger and Airy equations. The pathwise Strichartz and local smoothing estimates are derived here in both the conservative and nonconservative case. In particular, we obtain the P-integrability of constants in these estimates, where P is the underlying probability measure. Several applications are given to nonlinear problems, including local well-posedness of stochastic nonlinear Schrodinger equations with variable coefficients and lower order perturbations, integrability of global solutions to stochastic nonlinear Schrodinger equations with constant coefficients. As another consequence, we prove as well the large deviation principle for the small noise asymptotics.

Affine-equivariant inference for multivariate location under Lp loss functions

We consider the fundamental problem of estimating the location of a d-variate probability measure under an Lp loss function. The naive estimator, that minimizes the usual empirical Lp risk, has a known asymptotic behavior but suffers from several deficiencies for p≠2, the most important one being the lack of equivariance under general affine transformations. In this work, we introduce a collection of Lp location estimators μˆnp,ℓ that minimize the size of suitable ℓ-dimensional data-based simplices. For ℓ=1, these estimators reduce to the naive ones, whereas, for ℓ=d, they are equivariant under affine transformations. Irrespective of ℓ, these estimators reduce to the sample mean for p=2, whereas for p=1, the estimators provide the well-known spatial median and Oja median for ℓ=1 and ℓ=d, respectively. Under very mild assumptions, we derive an explicit Bahadur representation result for μˆnp,ℓ and establish asymptotic normality. We prove that, quite remarkably, the asymptotic behavior of the estimators does not depend on ℓ under spherical symmetry, so that the affine equivariance for ℓ=d is achieved at no cost in terms of efficiency. To allow for large sample size n and/or large dimension d, we introduce a version of our estimators relying on incomplete U-statistics. Under a centro-symmetry assumption, we also define companion tests ϕnp,ℓ for the problem of testing the null hypothesis that the location μ of the underlying probability measure coincides with a given location μ0. For any p, affine invariance is achieved for ℓ=d. For any ℓ and p, we derive explicit expressions for the asymptotic power of these tests under contiguous local alternatives, which reveals that asymptotic relative efficiencies with respect to traditional parametric Gaussian procedures for hypothesis testing coincide with those obtained for point estimation. We illustrate finite-sample relevance of our asymptotic results through Monte Carlo exercises and also treat a real data example.

Multivariate ρ-quantiles: A spatial approach

By substituting an Lp loss function for the L1 loss function in the optimization problem defining quantiles, one obtains Lp-quantiles that, as shown recently, dominate their classical L1-counterparts in financial risk assessment. In this work, we propose a concept of multivariate Lp-quantiles generalizing the spatial (L1-)quantiles introduced by Probal Chaudhuri (J. Amer. Statist. Assoc. 91 (1996) 862–872). Rather than restricting to power loss functions, we actually allow for a large class of convex loss functions ρ. We carefully study existence and uniqueness of the resulting ρ-quantiles, both for a general probability measure over Rd and for a spherically symmetric one. Interestingly, the results crucially depend on ρ and on the nature of the underlying probability measure. Building on an investigation of the differentiability properties of the objective function defining ρ-quantiles, we introduce a companion concept of spatial ρ-depth, that generalizes the classical spatial depth. We study extreme ρ-quantiles and show in particular that extreme Lp-quantiles behave in fundamentally different ways for p≤2 and p>2. Finally, we establish Bahadur representation results for sample ρ-quantiles and derive their asymptotic distributions. Throughout, we impose only very mild assumptions on the underlying probability measure, and in particular we never assume absolute continuity with respect to the Lebesgue measure.

Quantitative statistical robustness for tail-dependent law invariant risk measures

When estimating the risk of a financial position with empirical data or Monte Carlo simulations via a tail-dependent law invariant risk measure such as the Conditional Value-at-Risk (CVaR), it is important to ensure the robustness of the plug-in estimator particularly when the data contain noise. Krätschmer et al. [Comparative and qualitative robustness for law invariant risk measures. Financ. Stoch., 2014, 18, 271–295.] propose a new framework to examine the qualitative robustness of such estimators for the tail-dependent law invariant risk measures on Orlicz spaces, which is a step further from an earlier work by Cont et al. [Robustness and sensitivity analysis of risk measurement procedures. Quant. Finance, 2010, 10, 593–606] for studying the robustness of risk measurement procedures. In this paper, we follow this stream of research to propose a quantitative approach for verifying the statistical robustness of tail-dependent law invariant risk measures. A distinct feature of our approach is that we use the Fortet–Mourier metric to quantify variation of the true underlying probability measure in the analysis of the discrepancy between the law of the plug-in estimator of the risk measure based on the true data and the one based on perturbed data. This approach enables us to derive an explicit error bound for the discrepancy when the risk functional is Lipschitz continuous over a class of admissible sets. Moreover, the newly introduced notion of Lipschitz continuity allows us to examine the degree of robustness for tail-dependent risk measures. Finally, we apply our quantitative approach to some well-known risk measures to illustrate our results and give an example of the tightness of the proposed error bound.

On quantitative stability in infinite-dimensional optimization under uncertainty

The vast majority of stochastic optimization problems require the approximation of the underlying probability measure, e.g., by sampling or using observations. It is therefore crucial to understand the dependence of the optimal value and optimal solutions on these approximations as the sample size increases or more data becomes available. Due to the weak convergence properties of sequences of probability measures, there is no guarantee that these quantities will exhibit favorable asymptotic properties. We consider a class of infinite-dimensional stochastic optimization problems inspired by recent work on PDE-constrained optimization as well as functional data analysis. For this class of problems, we provide both qualitative and quantitative stability results on the optimal value and optimal solutions. In both cases, we make use of the method of probability metrics. The optimal values are shown to be Lipschitz continuous with respect to a minimal information metric and consequently, under further regularity assumptions, with respect to certain Fortet-Mourier and Wasserstein metrics. We prove that even in the most favorable setting, the solutions are at best Hölder continuous with respect to changes in the underlying measure. The theoretical results are tested in the context of Monte Carlo approximation for a numerical example involving PDE-constrained optimization under uncertainty.

A Second-Order Sufficient Optimality Condition for Risk-Neutral Bi-level Stochastic Linear Programs

The expectation functionals, which arise in risk-neutral bi-level stochastic linear models with random lower-level right-hand side, are known to be continuously differentiable, if the underlying probability measure has a Lebesgue density. We show that the gradient may fail to be local Lipschitz continuous under this assumption. Our main result provides sufficient conditions for Lipschitz continuity of the gradient of the expectation functional and paves the way for a second-order optimality condition in terms of generalized Hessians. Moreover, we study geometric properties of regions of strong stability and derive representation results, which may facilitate the computation of gradients.

Symmetries of stochastic differential equations using Girsanov transformations

Aiming at enlarging the class of symmetries of a stochastic differential equation (SDE), we introduce a family of stochastic transformations able to change also the underlying probability measure exploiting Girsanov theorem and we provide new determining equations for the infinitesimal symmetries of the SDE. The well-defined subset of the previous class of measure transformations given by Doob transformations allows us to recover all the Lie point symmetries of the Kolmogorov equation associated with the SDE. This gives the first stochastic interpretation of all the deterministic symmetries of the Kolmogorov equation. The general theory is applied to some relevant stochastic models.

Infinite Swapping using IID Samples

We propose a new method for estimating rare event probabilities when independent samples are available. It is assumed that the underlying probability measures satisfy a large deviation principle with a scaling parameter ε that we call temperature. We show how by combining samples at different temperatures, one can construct an estimator with greatly reduced variance. Although as presented here the method is not as broadly applicable as other rare event simulation methods, such as splitting or importance sampling, it does not require any problem-dependent constructions.

Model uncertainty stochastic mean-field control

We consider the problem of optimal control of a mean-field stochastic differential equation (SDE) under model uncertainty. The model uncertainty is represented by ambiguity about the law of the state X(t) at time t. For example, it could be the law of X(t) with respect to the given, underlying probability measure . This is the classical case when there is no model uncertainty. But it could also be the law with respect to some other probability measure or, more generally, any random measure on with total mass 1. We represent this model uncertainty control problem as a stochastic differential game of a mean-field related type SDE with two players. The control of one of the players, representing the uncertainty of the law of the state, is a measure-valued stochastic process and the control of the other player is a classical real-valued stochastic process u(t). This optimal control problem with respect to random probability processes in a non-Markovian setting is a new type of stochastic control problems that has not been studied before. By constructing a new Hilbert space of measures, we obtain a sufficient and a necessary maximum principles for Nash equilibria for such games in the general nonzero-sum case, and for saddle points in zero-sum games. As an application we find an explicit solution of the problem of optimal consumption under model uncertainty of a cash flow described by a mean-field related type SDE.

Computable approximations for average Markov decision processes in continuous time

Abstract In this paper we study the numerical approximation of the optimal long-run average cost of a continuous-time Markov decision process, with Borel state and action spaces, and with bounded transition and reward rates. Our approach uses a suitable discretization of the state and action spaces to approximate the original control model. The approximation error for the optimal average reward is then bounded by a linear combination of coefficients related to the discretization of the state and action spaces, namely, the Wasserstein distance between an underlying probability measure μ and a measure with finite support, and the Hausdorff distance between the original and the discretized actions sets. When approximating μ with its empirical probability measure we obtain convergence in probability at an exponential rate. An application to a queueing system is presented.

Concentration of the empirical level sets of Tukey’s halfspace depth

Tukey’s halfspace depth has attracted much interest in data analysis, because it is a natural way of measuring the notion of depth relative to a cloud of points or, more generally, to a probability measure. Given an i.i.d. sample, we investigate the concentration of upper level sets of the Tukey depth relative to that sample around their population version. We show that under some mild assumptions on the underlying probability measure, concentration occurs at a parametric rate and we deduce moment inequalities at that same rate. In a computational prospective, we study the concentration of a discretized version of the empirical upper level sets.

Small-time expansions for state-dependent local jump–diffusion models with infinite jump activity

In this article, we consider a Markov process {Xt}t⩾0, which solves a stochastic differential equation driven by a Brownian motion and an independent pure jump component exhibiting both state-dependent jump intensity and infinite jump activity. A second order expansion is derived for the tail probability P[Xt⩾x+y] in small time t, where x is the initial value of the process and y>0. As an application of this expansion and a suitable change of the underlying probability measure, a second order expansion, near expiration, for out-of-the-money European call option prices is obtained when the underlying stock price is modeled as the exponential of the jump–diffusion process {Xt}t⩾0 under the risk-neutral probability measure.

Geometric k-nearest neighbor estimation of entropy and mutual information.

Nonparametric estimation of mutual information is used in a wide range of scientific problems to quantify dependence between variables. The k-nearest neighbor (knn) methods are consistent, and therefore expected to work well for a large sample size. These methods use geometrically regular local volume elements. This practice allows maximum localization of the volume elements, but can also induce a bias due to a poor description of the local geometry of the underlying probability measure. We introduce a new class of knn estimators that we call geometric knn estimators (g-knn), which use more complex local volume elements to better model the local geometry of the probability measures. As an example of this class of estimators, we develop a g-knn estimator of entropy and mutual information based on elliptical volume elements, capturing the local stretching and compression common to a wide range of dynamical system attractors. A series of numerical examples in which the thickness of the underlying distribution and the sample sizes are varied suggest that local geometry is a source of problems for knn methods such as the Kraskov-Stögbauer-Grassberger estimator when local geometric effects cannot be removed by global preprocessing of the data. The g-knn method performs well despite the manipulation of the local geometry. In addition, the examples suggest that the g-knn estimators can be of particular relevance to applications in which the system is large, but the data size is limited.

Optimal Learning Rates for Kernel Partial Least Squares

We study two learning algorithms generated by kernel partial least squares (KPLS) and kernel minimal residual (KMR) methods. In these algorithms, regularization against overfitting is obtained by early stopping, which makes stopping rules crucial to their learning capabilities. We propose a stopping rule for determining the number of iterations based on cross-validation, without assuming a priori knowledge of the underlying probability measure, and show that optimal learning rates can be achieved. Our novel analysis consists of a nice bound for the number of iterations in a priori knowledge-based stopping rule for KMR and a stepping stone from KMR to KPLS. Technical tools include a recently developed integral operator approach based on a second order decomposition of inverse operators and an orthogonal polynomial argument.

Semimartingale properties of the lower Snell envelope in optimal stopping under model uncertainty

Optimal stopping under model uncertainty is a recent topic under research. The classical approach to characterize the solution of optimal stopping is based on the Snell envelope which can be seen as the value process as time runs. The analogous concept under model uncertainty is the so-called lower Snell envelope and in this paper, we investigate its structural properties. We give conditions under which it is a semimartingale with respect to one of the underlying probability measures and show how to identify the finite variation process by a limiting procedure. An example illustrates that without our conditions, the semimartingale property does not hold in general.

Data-driven nonlinear expectations for statistical uncertainty in decisions

In stochastic decision problems, one often wants to estimate the underlying probability measure statistically, and then to use this estimate as a basis for decisions. We shall consider how the uncertainty in this estimation can be explicitly and consistently incorporated in the valuation of decisions, using the theory of nonlinear expectations.

Weak Continuity of Risk Functionals with Applications to Stochastic Programming

Measuring and managing risk has become crucial in modern decision making under stochastic uncertainty. In two-stage stochastic programming, mean risk models are essentially defined by a parametric recourse problem and a quantification of risk. From the perspective of qualitative robustness theory, we discuss sufficient conditions for continuity of the resulting objective functions with respect to perturbation of the underlying probability measure. Our approach covers a fairly comprehensive class of both stochastic programming related risk measures and relevant recourse models. Not only does this unify previous approaches but also extends known stability results for two-stage stochastic programs to models with mixed-integer quadratic recourse and mixed-integer convex recourse, respectively.