Variance Reduction Research Articles

Mean-Field-Type Transformers

In this article, we present the mathematical foundations of generative machine intelligence and link them with mean-field-type game theory. The key interaction mechanism is self-attention, which exhibits aggregative properties similar to those found in mean-field-type game theory. It is not necessary to have an infinite number of neural units to handle mean-field-type terms. For instance, the variance reduction in error within generative machine intelligence is a mean-field-type problem and does not involve an infinite number of decision-makers. Based on this insight, we construct mean-field-type transformers that operate on data that are not necessarily identically distributed and evolve over several layers using mean-field-type transition kernels. We demonstrate that the outcomes of these mean-field-type transformers correspond exactly to the mean-field-type equilibria of a hierarchical mean-field-type game. Due to the non-convexity of the operators’ composition, gradient-based methods alone are insufficient. To distinguish a global minimum from other extrema—such as local minima, local maxima, global maxima, and saddle points—alternative methods that exploit hidden convexities of anti-derivatives of activation functions are required. We also discuss the integration of blockchain technologies into machine intelligence, facilitating an incentive design loop for all contributors and enabling blockchain token economics for each system participant. This feature is especially relevant to ensuring the integrity of factual data, legislative information, medical records, and scientifically published references that should remain immutable after the application of generative machine intelligence.

Modal volatility function

We in this article propose a novel non‐parametric estimator for the volatility function within a broad context that encompasses nonlinear time series models as a special case. The new estimator, built on the mode value, is designed to complement existing mean volatility measures to reveal distinct data features. We demonstrate that the suggested modal volatility estimator can be obtained asymptotically as well as if the conditional mean regression function were known, assuming observations are from a strictly stationary and absolutely regular process. Under mild regularity conditions, we establish that the asymptotic distributions of the resulting estimator align with those derived from independent observations, albeit with a slower convergence rate compared to non‐parametric mean regression. The theory and practice of bandwidth selection are discussed. Moreover, we put forward a variance reduction technique for the modal volatility estimator to attain asymptotic relative efficiency while maintaining the asymptotic bias unchanged. We numerically solve the modal regression model with the use of a modified modal‐expectation‐maximization algorithm. Monte Carlo simulations are conducted to assess the finite sample performance of the developed estimation procedure. Two real data analyses are presented to further illustrate the newly proposed model in practical applications. To potentially enhance the accuracy of the bias term, we in the end discuss the extension of the method to local exponential modal estimation. We showcase that the suggested exponential modal volatility estimator shares the same asymptotic variance as the non‐parametric modal volatility estimator but may exhibit a smaller bias.

Variance reduction in Monte Carlo for Integrals

Abstract. Nowadays, the extensive use of Monte Carlo methods in various fields has promoted the scientific decision-making, improved the efficiency of social operation, and enabled better management of many uncertain factors in daily life. However, in some cases, traditional Monte Carlo methods doesn't work so well because of the excessive variance. This paper aims to explore a new numerical integration method called splitting method to minimize intrinsic variance in Monte Carlo simulations. Through this innovative method, we found that the variance was reduced significantly. Despite some oscillating functions which are still difficult to estimate because they have too many turning points, this study provides new insights into variance reduction in Monte Carlo for integrals to optimize calculations in various fields when the modelling functions are not monotonic.

The suppression of ocean waves by biogenic slicks.

Ocean waves are significantly damped by biogenic surfactants, which accumulate at the sea surface in every ocean basin. The growth, development, and breaking of short wind-driven surface waves are key mediators of the air-sea exchange of momentum, heat and trace gases. The mechanisms through which surfactants suppress waves have been studied in great detail through careful laboratory experimentation in quasi-one-dimensional wave tanks. However, the spatial scales over which this damping occurs in structurally complex surfactant slicks on the real ocean have not been resolved. Here, we present the results of field observations of the spatial response of decimetre- to millimetre-scale waves to biogenic surfactant slicks. We found that wave damping in organic material-rich coastal waters resulted in a net (spatio-temporally averaged) reduction of approximately 50% in wave slope variance relative to the open ocean for low to moderate wind speeds. This reduction of wave slope variance is understood to result in a corresponding reduction in momentum input to the wave field. This significant effect had thus far evaded quantification due in large part to the enormous range of scales required for its description-spanning the sea surface microlayer to the ocean submesoscale.

Spread Option Pricing Under Finite Liquidity Framework

This work explores a finite liquidity model to price spread options and assess the liquidity impact. We employ Kirk approximation for computing the spread option price and its delta. The latter is needed since the liquidity impact is caused by the delta hedging of a large investor. Our main contribution is a novel methodology to price spread options in this paradigm. Kirk approximation in conjunction with Monte Carlo simulations yields the spread option prices. Moreover, the antithetic and control variates variance reduction techniques improve the performance of our method. Numerical experiments reveal that the finite liquidity causes a liquidity value adjustment in option prices ranging from 0.53% to 2.81%. The effect of correlation on prices is also explored, and as expected the option price increases due to the diversification effect, but the liquidity impact decreases slightly.

Comprehensive cardiac interval analysis in the WEAR-TECH study cohort by comparing the Apple Watch Series 6 against a simultaneous 12-lead ECG

Abstract Background Wearable devices capable of single-lead electrocardiograms (ECGs) are becoming increasingly prevalent. Exploring their potential for screening conditions beyond atrial fibrillation presents a compelling research opportunity. Objective This study evaluates the agreement between the Apple Watch Series 6 (APW) and simultaneous 12-lead ECG (12L) recordings in measuring PR, QRS, RR, QT, and QTc intervals in a cohort of National Health Service (NHS) cardiology patients. Methods We analysed data from the WEAR-TECH study, which included 400 simultaneous recordings from the APW and 12L ECGs. We included patients with ECGs in sinus rhythm and excluded patients with poor-quality ECGs. No exclusions were made based on underlying cardiac conditions. Measurements of the best three consecutive waveforms in leads I, II, and V5 were performed using EP Calipers (EP Studio) by a single observer. Leads I and II were compared for PR, QRS, and RR intervals, while leads I, II, and V5 were compared for QT and QTc intervals. QTc was calculated using Bazett’s formula. Differences between APW and 12L ECGs were assessed using linear mixed effects regression models with random intercepts and type II Wald chi-square tests. Clinical agreement was measured by averaging the waveforms from 12L leads and the APW and calculating the difference between the means. Results Out of 324 paired ECGs from 162 patients, 8 were excluded due to missing or uninterpretable data. The median age was 64 (IQR: 54 - 74) years, with 117 (75%) being male. PR, QRS, and RR intervals from the APW agreed best with 12L lead II, and QT and QTc intervals from APW agreed best with 12L lead V5, with no significant difference found in any of these comparisons. PR and RR intervals significantly differed between APW and 12L lead I, with the APW tending to overestimate relative to the 12L. Additionally, significant differences were found in 12L leads I and II for QT and QTc, with APW also tending to overestimate these intervals (see Table 1). Clinical agreement between the APW and all 12L leads was a minimum of 21% for a <10ms difference, 45% for a <20ms difference, and 71% for a <40ms difference (see Figure 1). Conclusions The Apple Watch Series 6 demonstrates promising potential for initial outpatient monitoring of PR, QRS, RR, QT, and QTc intervals with reasonable agreement to 12L ECG. This is especially true in lead II for PR, QRS, and RR intervals and lead V5 for QT and QTc intervals. Further reductions in variance between APW and 12L ECG measurements could enhance clinical decision-making accuracy.Table 1:The p-values of APW vs 12L leadFigure 1:Agreement of APW to 12L leads

E-Brachy: New dosimetry package for electronic brachytherapy sources.

Large reported variability in the material composition and geometrical components of the Xoft electronic high-dose-rate brachytherapy Causes inter-source discrepancy in the source output. This variability is due to the manual manufacturing and assembly of thesources. This study aimed to develop a dosimetry software tool called E-Brachy to characterize the Xoft source and quantify the discrepancies in its photon spectrum and dosimetricproperties. E-Brachy is based on the Geant4 Monte Carlo toolkit and consists of two parts. In part one, the geometry and material composition for the source received in the computer-aided design format from the vendor were converted to the geometry description markup language format using the GUIMesh Python tool and integrated into the E-Brachy software. There was a large variation in material composition and thickness for some of the tube components. The simulation started from electrons and resulted in x-ray generations in the anode region. Multithreading, a track length estimation, and the uniform bremsstrahlung splitting variance reduction techniques were used to decrease the simulation time and increase the x-ray production. The photon energy, position, and momentum were saved into a phase space file as the photon exited the source, but before interacting with the external environment. The obtained x-ray energy spectrum was compared with measurements from the National Institute of Standards and Technology (NIST). In part two, by sampling from the generated photons, the dose rates and dosimetric parameters according to the TG-43 protocol were calculated for model S7500 and compared to the ones previously calculated for model S700 source, which were deemed identical by themanufacturer. The material composition that resulted in the most similar spectrum as the measured NIST spectrum with Pearson's correlation coefficient of 0.99 and a calculated Euclidean difference of keV was chosen for further dosimetric analysis of the model S7500 source. Characteristic peaks showed the presence of tungsten, yttrium, and silver in the source components. Differences in dose rates between the two source models surpassed 20% for polar angles , reaching a peak at cm and . The differences in the radial dose function values were within 5%. The relative difference in percentage between the anisotropy function values of the two models was closer to 0 for smaller values, but at higher polar angles, they increased to 300%. A software package called E-Brachy was successfully developed for the characterization and dosimetry of Xoft electronic brachytherapy sources. E-Brachy can be combined with spectral measurements to investigate the inter- and intra-source variability. The software package was tested by comparing the simulated spectra from the S7500 Xoft source model with NIST measurements and its TG-43 parameters with the S700 model. The TG-43 parameters between the two sources significantly exceed the recommendations ofTG-56.

Imaginary latent variables: Empirical testing for detecting deficiency in reflective measures

Imaginary latent variables are variables with negative variances and have been used to implement constraints in measurement models. This article aimed to advance this practice and rationalize the imaginary latent variables as a method to detect possible latent deficiencies in measurement models. This rationale is based on the theory of complex numbers used in the measurement process of common factor model–based structural equation modeling. Modeling an imaginary latent variable produces a potential deficiency within its relative reflective measures through a considerable reduction in common variance indicating the most affected indicator(s).

Strata Design for Variance Reduction in Stochastic Simulation

Stratified sampling is one of the powerful variance reduction methods for analyzing system performance, such as reliability, with stochastic simulation. It divides the input space into disjoint subsets, called strata, to draw samples from each stratum. Partitioning the input space properly and allocating greater computational effort to crucial strata can help accurately estimate system performance with a limited computational budget. How to create strata, however, has yet to be thoroughly examined. Strata design faces the curse of dimensionality and data scarcity as the input dimension increases. We analytically derive the optimal stratification structure that minimizes the estimation variance for univariate problems. Further, reconciling the optimal stratification into decision trees, we devise a robust algorithm for multi-dimensional problems. Numerical experiments and a wind turbine case study demonstrate the superiority of the proposed method in terms of variance reduction, leading to computational efficiency and scalability.

Streaming First‐Order Optimization Methods With Momentum for Dominant Eigenspace Estimation

ABSTRACTIn this article, we consider the dominant eigenspace recovery for PCA in the streaming scenario, and design several new algorithms for this problem by incorporating various momentum schemes into two classical gradient based streaming eigensolvers, which emerges as a frequently‐used trick in improving gradient type methods. Theoretically, we establish the convergence guarantees of the proposed first‐order momentum methods, and prove their decreasing variance under favorable conditions. Extensive numerical experiments are conducted to corroborate the result that momentum does yield variance reduction, and also highlight the advantages of lowering the stepsize sensitivity. In addition, our methods with appropriately chosen momentum yield a dramatically enhanced performance in terms of both the convergence speed and accuracy.

Doubly-robust and heteroscedasticity-aware sample trimming for causal inference

SUMMARY A popular method for variance reduction in causal inference is propensity-based trimming, the practice of removing units with extreme propensities from the sample. This practice has theoretical grounding when the data are homoscedastic and the propensity model is parametric (Yang & Ding, 2018; Crump et al., 2009), but in modern settings where heteroscedastic data are analyzed with non-parametric models, existing theory fails to support current practice. In this work, we address this challenge by developing new methods and theory for sample trimming. Our contributions are three-fold: first, we describe novel procedures for selecting which units to trim. Our procedures differ from previous works in that we trim not only units with small propensities, but also units with extreme conditional variances. Second, we give new theoretical guarantees for inference after trimming. In particular, we show how to perform inference on the trimmed sub-population without requiring that our regressions converge at parametric rates. Instead, we make only fourth-root rate assumptions like those in the double machine learning literature. This result applies to conventional propensity-based trimming as well and thus may be of independent interest. Finally, we propose a bootstrap-based method for constructing simultaneously valid confidence intervals for multiple trimmed sub-populations, which are valuable for navigating the trade-off between sample size and variance reduction inherent in trimming. We validate our methods in simulation, on the 2007-2008 National Health and Nutrition Examination Survey, and on a semi-synthetic Medicare dataset and find promising results in all settings.

New control variates for pricing basket options

Abstract Accepted by: Giorgio Consigli Accurate pricing of basket options, which are financial derivatives on multiple underlying assets, is a challenging and practically important task for financial institutions. We propose several new control variates for accurate, fast and efficient pricing of basket options. The first approach to deriving new control variates is the use of Hermite polynomial approximation of appropriate function of the underlying asset prices, which leads to a Black–Scholes-like analytic solution. This approach is new in the option pricing context and opens up new possibilities in derivative pricing. Further control variates are analytically derived using Jensen’s inequality in one case, and distributional properties of multivariate Wiener processes in other cases. All the newly proposed control variates are shown to lead to excellent variance reduction in numerical experiments based on realistic data. The proposed methods are novel, computationally simple and have a strong potential to replace more conventional methods, such as the geometric lower bound in simulation-based pricing of basket options and similar products used in financial risk management.

A Bayesian network approach to assess the ecosystem integrity of mangroves in Tampamachoco, Veracruz, Mexico

Given the alarming rates of mangrove forest loss, resource managers must count on information regarding the condition of the mangrove forests. We propose a Bayesian network (BN) to assess mangrove forests' ecosystem integrity (EI) to support a mangrove monitoring system in Mexico. This approach allowed us to infer the system's condition based on variables on forest structure and function. We defined the BN structure based on informal interviews with specialists on vegetation and coastal geomorphology. We applied the expectation-maximization learning algorithm to train the model. Data from plots in two mangrove areas of an Avicennia germinans forest, defined based on their undisturbed and disturbed conditions, were used as training datasets. We applied sensitivity analysis to determine the degree of influence of each model variable. We evaluated the prediction capacity of the BN with a k-fold cross-validation (the process is repeated five times, starting from the database in 2 parts). The variables selected for the model were the Holdridge complexity index (HCI, Holdridge et al., 1971), Leaf area index (LAI), litter production (g/month/m2), leaf C, N and P concentration (%), and leaf N:P ratio. The most critical variable to infer mangrove condition was leaf N:P (Variance reduction = 11%), followed by forest structure variables HCI and LAI (Variance reduction > 5%). The cross-validation to test the model resulted in a minimum square error of 0.3, which indicates a reasonable capacity to predict the condition of mangrove integrity. The BN constructed can diagnose the integrity of a monospecific mangrove forest with acceptable precision, considering the environmental factors that define the forest structure and functioning locally. We then asked the experts to review and modify the model to apply to multispecies mangrove ecosystems and environmental contexts.

Transitioning from Self-Monitoring of Blood Glucose to Continuous Glucose Monitoring in Combination with a mHealth App Improves Glycemic Control in People with Type 1 and Type 2 Diabetes.

Introduction: Integrating mobile health (mHealth) apps into daily diabetes management allows users to monitor and track their health data, creating a comprehensive system for managing daily diabetes activities and generating valuable real-world data. This analysis investigates the impact of transitioning from traditional self-monitoring of blood glucose (SMBG) to real-time continuous glucose monitoring (rtCGM), alongside the use of a mHealth app, on users' glycemic control. Methods: Data were collected from 1271 diabetes type 1 and type 2 users of the mySugr® app who made a minimum of 50 SMBG logs 1 month before transitioning to rtCGM and then used rtCGM for at least 6 months. The mean and coefficient of variation of glucose, along with the proportions of glycemic measurements in and out of range, were compared between baseline and 1, 2, 3, and 6 months of rtCGM use. A mixed-effects linear regression model was built to quantify the specific effects of transitioning to a rtCGM sensor in different subsamples. A novel validation analysis ensured that the aggregated metrics from SMBG and rtCGM were comparable. Results: Transitioning to a rtCGM sensor significantly improved glycemic control in the entire cohort, particularly among new users of the mySugr app. Additionally, the sustainability of the change in glucose in the entire cohort was confirmed throughout the observation period. People with type 1 and type 2 diabetes exhibited distinct variations, with type 1 experiencing a greater reduction in glycemic variance, while type 2 displayed a relatively larger decrease in monthly averages.

PGMF-VINS: Perpendicular-Based 3D Gaussian–Uniform Mixture Filter

Visual–Inertial SLAM (VI-SLAM) has a wide range of applications spanning robotics, autonomous driving, AR, and VR due to its low-cost and high-precision characteristics. VI-SLAM is divided into localization and mapping tasks. However, researchers focus more on the localization task while the robustness of the mapping task is often ignored. To address this, we propose a map-point convergence strategy which explicitly estimates the position, the uncertainty, and the stability of the map point (SoM). As a result, the proposed method can effectively improve the quality of the whole map while ensuring state-of-the-art localization accuracy. The convergence strategy mainly consists of a perpendicular-based triangulation and 3D Gaussian–uniform mixture filter, and we name it PGMF, perpendicular-based 3D Gaussian–uniform mixture filter. The algorithm is extensively tested on open-source datasets, which shows the RVM (Ratio of Valid Map points) of our algorithm exhibits an average increase of 0.1471 compared to VINS-mono, with a variance reduction of 68.8%.

Global Convergence of Natural Policy Gradient with Hessian-Aided Momentum Variance Reduction

Global Convergence of Natural Policy Gradient with Hessian-Aided Momentum Variance Reduction

Services Administration In Education. Utilizing Control Charts In Educational Management: An Approach For Assessing The Mathematics Scores In Panhellenic Examinations

Background: The choices offered to students and their parents complicate the process of finding an optimal solution that meets their needs, as competition increases1 . The management of each educational organization is responsible for providing services information and curricula offered by the tutorials, as well as for designing innovative, cost-effective, and efficacious solutions2 . A learning organization with a student-oriented marketing approach is considered by many researchers as an effective mix to mixture that meets the needs of customers, students, and parents3,4. An organization with the above-mentioned features which apprehends its position, predicts the behavior of its customers, meets their needs and preferences, and enables marketing to be much more targeted and successful. As a result, it provides highvalue services, which are its comparative advantage in the market5 . The evaluation and intervention of education service providers and curriculum designers ensures the correction of errors and the adaptation to student and societal needs in all educational system processes. One of the most reliable statistical control tools of quality services and processes is the mean and standard deviation control charts6 . Both respond to the requirements of in educational processes as they are described by the normal distribution. Materials and Methods In this work, the statistical tools have been utilized to analyze the mathematics scores in the economic and scientific studies orientation of the Panhellenic exams from 2003 to 20227 . The objective of this analysis is to investigate the application of control charts as statistical methods and identify the factors responsible for any out-of-control processes. Conclusion: The study findings revealed that the modification of the curricula (all chapters of Algebra topics were removed, and only mathematical analysis topics remained) negatively affected the mean students' scores from 2015 to 2017, with a simultaneous reduction of variance. Distance education which was mandated due to the pandemic crisis did not negatively affect the mean values of the mathematics scores and increased the variance. A decrease in the number of the Panhellenic exams candidates is expected due to the demographic problem of Greece8,9. Moreover, extensive learning gaps for students who were in high school and elementary school during distance education are evident10. Finally, the service administration authorities in educational organizations and those who are responsible for the development of educational strategies should monitor student performance with the use of appropriate tools. This will help them to adapt their policies, educational tools, and curricula to the current conditions if it would be necessary

A physics-informed neural network enhanced importance sampling (PINN-IS) for data-free reliability analysis

Reliability analysis of highly sensitive structures is crucial to prevent catastrophic failures and ensure safety. Therefore, these safety-critical systems are to be designed for extremely rare failure events. Accurate statistical quantification of these events associated with a very low probability of occurrence requires millions of evaluations of the limit state function (LSF) involving computationally expensive numerical simulations. Variance reduction techniques like importance sampling (IS) reduce such repetitions to a few thousand. The use of a data-centric metamodel can further cut it down to a few hundred. In data-centric metamodeling approaches, the actual complex numerical analysis is performed at a few points to train the metamodel for approximating the structural response. On the other hand, a physics-informed neural network (PINN) can predict the structural response based on the governing differential equation describing the physics of the problem, without a single evaluation of the complex numerical solver, i.e., data-free. However, the existing PINN models for reliability analysis have been effective only in estimating a large range of failure probabilities (10-1∼10-3). To address this issue, the present study develops a PINN-based data-free reliability analysis for low failure probabilities (<10-5). In doing so, a two-stage PINN integrated with IS (PINN-IS) is proposed. The first stage is employed to approximate the most probable failure point (MPP) appropriately while the second stage enhances the accuracy of LSF predictions at the IS population centred on the approximated MPP. The effectiveness of the proposed approach is numerically illustrated by three structural reliability analysis examples.

A Modified Wei-Yao-Liu Stochastic Conjugate Gradient Algorithm in Machine Learning

Abstract The Wei-Yao-Liu (WYL) Conjugate Gradient (CG) algorithm exhibits favourable attributes, notably sufficient descent and trust domain characteristics, in the context of solving unconstrained optimization problems. The exploration and enhancement of computational methodologies for stochastic optimization problems have garnered significant attention. In this study, our objective is to improve the Wei-Yao-Liu (WYL) CG algorithm and introduce an effective optimization method tailored for solving stochastic optimization problems. We will present a comprehensive framework for analysing the convergence of this algorithm and, under suitable conditions, establish its sufficient descent properties, trust region features, and global convergence of stationary points. We will also give an improved CG algorithm to calculate a framework for variance reduction. Additionally, we will conduct experimental comparisons to showcase the competitiveness of our proposed algorithm.

Meat tenderness in Australian lamb: Data editing, environmental variation and their effects in genetic parameter estimation

Breeding for meat quality increases the value of lambs and requires reliable genetic parameters to achieve balanced genetic progress. Meat tenderness, accomplished by selecting for lower shear force, is an important eating quality trait because of its relationship with consumer satisfaction. Factors influencing shear force, include the pH and temperature decline post-mortem which can contribute towards higher shear force values and increased variation across contemporary groups. This study explored if genetic parameters for shear force change when post slaughter covariates and heterogeneous variance are corrected for, using data from 32,223 animals from different sheep breeds. Results showed that removing extreme individuals and contemporary groups with high mean shear force values reduced residual variance, followed by a smaller reduction in additive genetic variance and little effect on heritability. Results show that edited data performed better at predicting progeny performance and reduced potential bias introduced in the genetic evaluation due to data quality. The effect of including post-slaughter covariates in the genetic analysis was tested by estimating different model predictability through regression of estimated breeding values against offspring performance, showing that the model including hot carcass weight performed better followed by the one including both carcass weight and C-site fat depth. Our results highlight that historic and current in-plant recording practices do not provide the capacity to account for non-genetic factors associated with abattoir environment that might be impacting the ability to accurately calculate shear force breeding values. In that sense, genetic evaluation can be improved by applying more rigorous data editing.