Monte Carlo Averages Research Articles

Hierarchical modeling of irregularly spaced financial returns

Volatility modeling is crucial in finance, especially when dealing with intraday transaction‐level asset returns. The irregular and high‐frequency nature of the data presents unique challenges. While stochastic volatility (SV) models are widely used for understanding patterns in volatility of daily stock returns which constitute regularly spaced time series, new classes of models must be introduced for analyzing volatility in irregularly spaced intraday data. Specifically these models must accommodate the random gaps between successive transactional events. By modeling the gaps using autoregressive conditional duration (ACD) models, we describe a hierarchical irregular SV autoregressive conditional duration (IR‐SV‐ACD) model for estimating and forecasting intertransaction gaps and the volatility of log‐returns. We carry out the analysis in the Bayesian framework via the Hamiltonian Monte Carlo (HMC) algorithm with No‐U‐turn sampler (NUTS) in R using the cmdstanr package. The fits and forecasts are obtained using Monte Carlo averages based on the posterior samples. We illustrate this approach using simulation studies and real data analysis for intraday prices available at microseconds level of health stocks traded on the New York Stock Exchange (NYSE). The log‐returns and gaps are calculated for the stocks and are used for modeling.

On the Unraveling of Open Quantum Dynamics

It is well known that the state operator of an open quantum system can be generically represented as the solution of a time-local equation — a quantum master equation. Unraveling in quantum trajectories offers a picture of open system dynamics dual to solving master equations. In the unraveling picture, physical indicators are computed as Monte Carlo averages over a stochastic process valued in the Hilbert space of the system. This approach is particularly adapted to simulate systems in large Hilbert spaces. We show that the dynamics of an open quantum system generically admits an unraveling in the Hilbert space of the system described by a Markov process generated by ordinary stochastic differential equations for which rigorous concentration estimates are available. The unraveling can be equivalently formulated in terms of norm-preserving state vectors or in terms of linear “ostensible” processes trace preserving only on average. We illustrate the results in the case of a two level system in a simple boson environment. Next, we derive the state-of-the-art form of the Diósi-Gisin-Strunz Gaussian random ostensible state equation in the context of a model problem. This equation provides an exact unraveling of open systems in Gaussian environments. We compare and contrast the two unravelings and their potential for applications to quantum error mitigation.

Monte Carlo averaging for uncertainty estimation in neural networks

Although convolutional neural networks (CNNs) are widely used in modern classifiers, they are affected by overfitting and lack robustness leading to overconfident false predictions (FPs). By preventing FPs, certain consequences (such as accidents and financial losses) can be avoided and the use of CNNs in safety- and/or mission-critical applications would be effective. In this work, we aim to improve the separability of true predictions (TPs) and FPs by enforcing the confidence determining uncertainty to be high for TPs and low for FPs. To achieve this, we must devise a suitable method. We proposed the use of Monte Carlo averaging (MCA) and thus compare it with related methods, such as baseline (single CNN), Monte Carlo dropout (MCD), ensemble, and mixture of Monte Carlo dropout (MMCD). This comparison is performed using the results of experiments conducted on four datasets with three different architectures. The results show that MCA performs as well as or even better than MMCD, which in turn performs better than baseline, ensemble, and MCD. Consequently, MCA could be used instead of MMCD for uncertainty estimation, especially because it does not require a predefined distribution and it is less expensive than MMCD.

Computation of Conditional Expectations with Guarantees

Theoretically, the conditional expectation of a square-integrable random variable Y given a d-dimensional random vector X can be obtained by minimizing the mean squared distance between Y and f(X) over all Borel measurable functions f :mathbb {R}^d rightarrow mathbb {R}. However, in many applications this minimization problem cannot be solved exactly, and instead, a numerical method which computes an approximate minimum over a suitable subfamily of Borel functions has to be used. The quality of the result depends on the adequacy of the subfamily and the performance of the numerical method. In this paper, we derive an expected value representation of the minimal mean squared distance which in many applications can efficiently be approximated with a standard Monte Carlo average. This enables us to provide guarantees for the accuracy of any numerical approximation of a given conditional expectation. We illustrate the method by assessing the quality of approximate conditional expectations obtained by linear, polynomial and neural network regression in different concrete examples.

Ligand binding free energy evaluation by Monte Carlo Recursion

The correct evaluation of ligand binding free energies by computational methods is still a very challenging active area of research. The most employed methods for these calculations can be roughly classified into four groups: (i) the fastest and less accurate methods, such as molecular docking, designed to sample a large number of molecules and rapidly rank them according to the potential binding energy; (ii) the second class of methods use a thermodynamic ensemble, typically generated by molecular dynamics, to analyze the endpoints of the thermodynamic cycle for binding and extract differences, in the so-called ‘end-point’ methods; (iii) the third class of methods is based on the Zwanzig relationship and computes the free energy difference after a chemical change of the system (alchemical methods); and (iv) methods based on biased simulations, such as metadynamics, for example. These methods require increased computational power and as expected, result in increased accuracy for the determination of the strength of binding. Here, we describe an intermediate approach, based on the Monte Carlo Recursion (MCR) method first developed by Harold Scheraga. In this method, the system is sampled at increasing effective temperatures, and the free energy of the system is assessed from a series of terms W(b,T), computed from Monte Carlo (MC) averages at each iteration. We show the application of the MCR for ligand binding with datasets of guest-hosts systems (N = 75) and we observed that a good correlation is obtained between experimental data and the binding energies computed with MCR. We also compared the experimental data with an end-point calculation from equilibrium Monte Carlo calculations that allowed us to conclude that the lower-energy (lower-temperature) terms in the calculation are the most relevant to the estimation of the binding energies, resulting in similar correlations between MCR and MC data and the experimental values. On the other hand, the MCR method provides a reasonable view of the binding energy funnel, with possible connections with the ligand binding kinetics, as well. The codes developed for this analysis are publicly available on GitHub as a part of the LiBELa/MCLiBELa project (https://github.com/alessandronascimento/LiBELa).

Probabilistic computations of virial coefficients of polymeric structures described by rigid configurations of spherical particles: A fundamental extension of the ZENO program.

We describe an extension of the ZENO program for polymer and nanoparticle characterization that allows for precise calculation of the virial coefficients, with uncertainty estimates, of polymeric structures described by arbitrary rigid configurations of hard spheres. The probabilistic method of virial computation used for this extension employs a previously developed Mayer-sampling Monte Carlo method with overlap sampling that allows for a reduction of bias in the Monte Carlo averaging. This capability is an extension of ZENO in the sense that the existing program is also based on probabilistic sampling methods and involves the same input file formats describing polymer and nanoparticle structures. We illustrate the extension's capabilities, demonstrate its accuracy, and quantify the efficiency of this extension of ZENO by computing the second, third, and fourth virial coefficients and metrics quantifying the difficulty of their calculation, for model polymeric structures having several different shapes. We obtain good agreement with literature estimates available for some of the model structures considered.

Wave-optics investigation of branch-point density

We use wave-optics simulations to investigate branch-point density (i.e., the number of branch points within the pupil-phase function) in terms of the grid sampling. The goal for these wave-optics simulations is to model plane-wave propagation through homogeneous turbulence, both with and without the effects of a finite inner scale modeled using a Hill spectrum. In practice, the grid sampling provides a gauge for the amount of branch-point resolution within the wave-optics simulations, whereas the Rytov number, Fried coherence diameter, and isoplanatic angle provide parameters to setup and explore the associated deep-turbulence conditions. Via Monte Carlo averaging, the results show that without the effects of a finite inner scale, the branch-point density grows without bound with adequate grid sampling. However, the results also show that as the inner-scale size increases, this unbounded growth (1) significantly decreases as the Rytov number, Fried coherence diameter, and isoplanatic angle increase in strength and (2) saturates with adequate grid sampling. These findings imply that future developments need to include the effects of a finite inner scale to accurately model the multifaceted nature of the branch-point problem in adaptive optics.

Revisiting the Gelman–Rubin Diagnostic

Gelman and Rubin’s (Statist. Sci. 7 (1992) 457–472) convergence diagnostic is one of the most popular methods for terminating a Markov chain Monte Carlo (MCMC) sampler. Since the seminal paper, researchers have developed sophisticated methods for estimating variance of Monte Carlo averages. We show that these estimators find immediate use in the Gelman–Rubin statistic, a connection not previously established in the literature. We incorporate these estimators to upgrade both the univariate and multivariate Gelman–Rubin statistics, leading to improved stability in MCMC termination time. An immediate advantage is that our new Gelman–Rubin statistic can be calculated for a single chain. In addition, we establish a one-to-one relationship between the Gelman–Rubin statistic and effective sample size. Leveraging this relationship, we develop a principled termination criterion for the Gelman–Rubin statistic. Finally, we demonstrate the utility of our improved diagnostic via examples.

Large System Achievable Rate Analysis of RIS-Assisted MIMO Wireless Communication With Statistical CSIT

Reconfigurable intelligent surface (RIS) is an emerging technology to enhance wireless communication in terms of energy cost and system performance by equipping a considerable quantity of nearly passive reflecting elements. This study focuses on a downlink RIS-assisted multiple-input multiple-output (MIMO) wireless communication system that comprises three communication links of Rician channel, including base station (BS) to RIS, RIS to user, and BS to user. The objective is to design an optimal transmit covariance matrix at BS and diagonal phase-shifting matrix at RIS to maximize the achievable ergodic rate by exploiting the statistical channel state information at BS. Therefore, a large-system approximation of the achievable ergodic rate is derived using the replica method in large dimension random matrix theory. This large-system approximation enables the identification of asymptotic-optimal transmit covariance and diagonal phase-shifting matrices using an alternating optimization algorithm. Simulation results show that the large-system results are consistent with the achievable ergodic rate calculated by Monte-Carlo averaging. The results verify that the proposed algorithm can significantly enhance the RIS-assisted MIMO system performance.

Sensitivity estimation for calculated phase equilibria

The development of a consistent framework for Calphad model sensitivity is necessary for the rational reduction of uncertainty via new models and experiments. In the present work, a sensitivity theory for Calphad was developed, and a closed‐form expression for the log‐likelihood gradient and Hessian of a multi‐phase equilibrium measurement was presented. The inherent locality of the defined sensitivity metric was mitigated through the use of Monte Carlo averaging. A case study of the Cr–Ni system was used to demonstrate visualizations and analyses enabled by the developed theory. Criteria based on the classical Cramér–Rao bound were shown to be a useful diagnostic in assessing the accuracy of parameter covariance estimates from Markov Chain Monte Carlo. The developed sensitivity framework was applied to estimate the statistical value of phase equilibria measurements in comparison with thermochemical measurements, with implications for Calphad model uncertainty reduction.

Sensitivity estimation for calculated phase equilibria

Abstract

Wave-optics investigation of turbulence thermal blooming interaction: II. Using time-dependent simulations

Part II of this two-part paper uses wave-optics simulations to look at the Monte Carlo averages associated with turbulence and time-dependent thermal blooming (TDTB). The goal is to investigate turbulence thermal blooming interaction (TTBI). At wavelengths near 1 μm, TTBI increases the amount of constructive and destructive interference (i.e., scintillation) that results from high-power laser beam propagation through distributed-volume atmospheric aberrations. As a result, we use the spherical-wave Rytov number, the number of wind-clearing periods, and the distortion number to gauge the strength of the simulated turbulence and TDTB. These parameters simply greatly given propagation paths with constant atmospheric conditions. In addition, we use the log-amplitude variance and the branch-point density to quantify the effects of TTBI. These metrics result from a point-source beacon being backpropagated from the target plane to the source plane through the simulated turbulence and TDTB. Overall, the results show that the log-amplitude variance and branch-point density increase significantly due to TTBI. This outcome poses a major problem for beam-control systems that perform phase compensation.

Wave-optics investigation of turbulence thermal blooming interaction: I. Using steady-state simulations

Part I of this two-part paper uses wave-optics simulations to look at the Monte Carlo averages associated with turbulence and steady-state thermal blooming (SSTB). The goal is to investigate turbulence thermal blooming interaction (TTBI). At wavelengths near 1 μm, TTBI increases the amount of constructive and destructive interference (i.e., scintillation) that results from high-power laser beam propagation through distributed-volume atmospheric aberrations. As a result, we use the spherical-wave Rytov number and the distortion number to gauge the strength of the simulated turbulence and SSTB. These parameters simplify greatly given propagation paths with constant atmospheric conditions. In addition, we use the log-amplitude variance and the branch-point density to quantify the effects of TTBI. These metrics result from a point-source beacon being backpropagated from the target plane to the source plane through the simulated turbulence and SSTB. Overall, the results show that the log-amplitude variance and branch-point density increase significantly due to TTBI. This outcome poses a major problem for beam-control systems that perform phase compensation.

Adaptive importance sampling and control variates

We construct and investigate an adaptive variance reduction framework in which both importance sampling and control variates are employed. The three lines (Monte Carlo averaging and two variance reduction parameter search lines) run in parallel on a common sequence of uniform random vectors on the unit hypercube. Given that these two variance reduction techniques are effective often in a complementary way, their combined application is well expected to widen the applicability of adaptive variance reduction. We derive convergence rates of the theoretical estimator variance towards its minimum as a fixed computing budget increases, when stochastic approximation runs with optimal constant learning rates. We derive sufficient conditions for the proposed algorithm to attain the minimal estimator variance in the limit, by stochastic approximation with decreasing learning rates or by sample average approximation, when computing budget is unlimitedly available. Numerical results support our theoretical findings and illustrate the effectiveness of the proposed framework.

Deep-turbulence wavefront sensing using digital holography in the on-axis phase shifting recording geometry with comparisons to the self-referencing interferometer.

In this paper, we study the use of digital holography in the on-axis phase-shifting recording geometry for the purposes of deep-turbulence wavefront sensing. In particular, we develop closed-form expressions for the field-estimated Strehl ratio and signal-to-noise ratio for three separate phase-shifting strategies-the four-, three-, and two-step methods. These closed-form expressions compare favorably with our detailed wave-optics simulations, which propagate a point-source beacon through deep-turbulence conditions, model digital holography with noise, and calculate the Monte Carlo averages associated with increasing turbulence strengths and decreasing focal-plane array sampling. Overall, the results show the four-step method is the most efficient phase-shifting strategy and deep-turbulence conditions only degrade performance with respect to insufficient focal-plane array sampling and low signal-to-noise ratios. The results also show the strong reference beam from the local oscillator provided by digital holography greatly improves performance by tens of decibels when compared with the self-referencing interferometer.

Behavior of tiled-aperture arrays fed by vector partially coherent sources.

This paper explores the far-zone behavior of partially coherent arrays. We derive an expression for the far-zone spectral density valid for any array composed of circular elements and fed by fields with Schell-model cross-spectral density functions. This expression is written as the sum of convolution integrals, making it easy to physically interpret. We discuss this expression at length and present examples. Lastly, we validate our analysis by comparing Monte Carlo averages from wave-optics simulations with theory. We conclude this paper with a brief summary of the results and potential uses of our work.

Polychromatic wave-optics models for image-plane speckle. 2. Unresolved objects.

Polychromatic laser light can reduce speckle noise in many wavefront-sensing and imaging applications. To help quantify the achievable reduction in speckle noise, this study investigates the accuracy of three polychromatic wave-optics models under the specific conditions of an unresolved object. Because existing theory assumes a well-resolved object, laboratory experiments are used to evaluate model accuracy. The three models use Monte-Carlo averaging, depth slicing, and spectral slicing, respectively, to simulate the laser-object interaction. The experiments involve spoiling the temporal coherence of laser light via a fiber-based, electro-optic modulator. After the light scatters off of the rough object, speckle statistics are measured. The Monte-Carlo method is found to be highly inaccurate, while depth-slicing error peaks at 7.8% but is generally much lower in comparison. The spectral-slicing method is the most accurate, always producing results within the error bounds of the experiment.

Geometric ergodicity of Gibbs samplers in Bayesian penalized regression models

We consider three Bayesian penalized regression models and show that the respective deterministic scan Gibbs samplers are geometrically ergodic regardless of the dimension of the regression problem. We prove geometric ergodicity of the Gibbs samplers for the Bayesian fused lasso, the Bayesian group lasso, and the Bayesian sparse group lasso. Geometric ergodicity along with a moment condition results in the existence of a Markov chain central limit theorem for Monte Carlo averages and ensures reliable output analysis. Our results of geometric ergodicity allow us to also provide default starting values for the Gibbs samplers.

Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximations

ABSTRACTWe construct a new framework for accelerating Markov chain Monte Carlo in posterior sampling problems where standard methods are limited by the computational cost of the likelihood, or of numerical models embedded therein. Our approach introduces local approximations of these models into the Metropolis–Hastings kernel, borrowing ideas from deterministic approximation theory, optimization, and experimental design. Previous efforts at integrating approximate models into inference typically sacrifice either the sampler’s exactness or efficiency; our work seeks to address these limitations by exploiting useful convergence characteristics of local approximations. We prove the ergodicity of our approximate Markov chain, showing that it samples asymptotically from the exact posterior distribution of interest. We describe variations of the algorithm that employ either local polynomial approximations or local Gaussian process regressors. Our theoretical results reinforce the key observation underlying this article: when the likelihood has some local regularity, the number of model evaluations per Markov chain Monte Carlo (MCMC) step can be greatly reduced without biasing the Monte Carlo average. Numerical experiments demonstrate multiple order-of-magnitude reductions in the number of forward model evaluations used in representative ordinary differential equation (ODE) and partial differential equation (PDE) inference problems, with both synthetic and real data. Supplementary materials for this article are available online.

A Bootstrap Metropolis–Hastings Algorithm for Bayesian Analysis of Big Data

Markov chain Monte Carlo (MCMC) methods have proven to be a very powerful tool for analyzing data of complex structures. However, their computer-intensive nature, which typically require a large number of iterations and a complete scan of the full dataset for each iteration, precludes their use for big data analysis. In this article, we propose the so-called bootstrap Metropolis–Hastings (BMH) algorithm that provides a general framework for how to tame powerful MCMC methods to be used for big data analysis, that is, to replace the full data log-likelihood by a Monte Carlo average of the log-likelihoods that are calculated in parallel from multiple bootstrap samples. The BMH algorithm possesses an embarrassingly parallel structure and avoids repeated scans of the full dataset in iterations, and is thus feasible for big data problems. Compared to the popular divide-and-combine method, BMH can be generally more efficient as it can asymptotically integrate the whole data information into a single simulation run. The BMH algorithm is very flexible. Like the Metropolis–Hastings algorithm, it can serve as a basic building block for developing advanced MCMC algorithms that are feasible for big data problems. This is illustrated in the article by the tempering BMH algorithm, which can be viewed as a combination of parallel tempering and the BMH algorithm. BMH can also be used for model selection and optimization by combining with reversible jump MCMC and simulated annealing, respectively. Supplementary materials for this article are available online.