Stochastic Approximation Research Articles

A stochastic approximation for the finite-size Kuramoto–Sakaguchi model

We perform a stochastic model reduction of the Kuramoto–Sakaguchi model for finitely many coupled phase oscillators with phase frustration. Whereas in the thermodynamic limit coupled oscillators exhibit stationary states and a constant order parameter, finite-size networks exhibit persistent temporal fluctuations of the order parameter. These fluctuations are caused by the interaction of the synchronised oscillators with the non-entrained oscillators. We present numerical results suggesting that the collective effect of the non-entrained oscillators on the synchronised cluster can be approximated by a Gaussian process. This allows for an effective closed evolution equation for the synchronised oscillators driven by a Gaussian process which we approximate by a two-dimensional Ornstein–Uhlenbeck process. Our reduction reproduces the stochastic fluctuations of the order parameter and leads to a simple stochastic differential equation for the order parameter.

Optimal Estimators of Cross-Partial Derivatives and Surrogates of Functions

Computing cross-partial derivatives using fewer model runs is relevant in modeling, such as stochastic approximation, derivative-based ANOVA, exploring complex models, and active subspaces. This paper introduces surrogates of all the cross-partial derivatives of functions by evaluating such functions at N randomized points and using a set of L constraints. Randomized points rely on independent, central, and symmetric variables. The associated estimators, based on NL model runs, reach the optimal rates of convergence (i.e., O(N−1)), and the biases of our approximations do not suffer from the curse of dimensionality for a wide class of functions. Such results are used for (i) computing the main and upper bounds of sensitivity indices, and (ii) deriving emulators of simulators or surrogates of functions thanks to the derivative-based ANOVA. Simulations are presented to show the accuracy of our emulators and estimators of sensitivity indices. The plug-in estimates of indices using the U-statistics of one sample are numerically much stable.

Modeling soybean growth: A mixed model approach.

The evaluation of plant and animal growth, separately for genetic and environmental effects, is necessary for genetic understanding and genetic improvement of environmental responses of plants and animals. We propose to extend an existing approach that combines nonlinear mixed-effects model (NLMEM) and the stochastic approximation of the Expectation-Maximization algorithm (SAEM) to analyze genetic and environmental effects on plant growth. These tools are widely used in many fields but very rarely in plant biology. During model formulation, a nonlinear function describes the shape of growth, and random effects describe genetic and environmental effects and their variability. Genetic relationships among the varieties were also integrated into the model using a genetic relationship matrix. The SAEM algorithm was chosen as an efficient alternative to MCMC methods, which are more commonly used in the domain. It was implemented to infer the expected growth patterns in the analyzed population and the expected curves for each variety through a maximum-likelihood and a maximum-a-posteriori approaches, respectively. The obtained estimates can be used to predict the growth curves for each variety. We illustrate the strengths of the proposed approach using simulated data and soybean plant growth data obtained from a soybean cultivation experiment conducted at the Arid Land Research Center, Tottori University. In this experiment, plant height was measured daily using drones, and the growth was monitored for approximately 200 soybean cultivars for which whole-genome sequence data were available. The NLMEM approach improved our understanding of the determinants of soybean growth and can be successfully used for the genomic prediction of growth pattern characteristics.

Anharmonic and quantum effects in Pm3̄ AlM(M = Hf, Zr)H6 under high pressure: A first-principles study.

First-principles calculations were employed to investigate the impact of quantum ionic fluctuations and lattice anharmonicity on the crystal structure and superconductivity of Pm3̄ AlM(M = Hf, Zr)H6 at pressures of 0.3-21.2GPa (AlHfH6) and 4.7-39.5GPa (AlZrH6) within the stochastic self-consistent harmonic approximation. A correction is predicted for the crystal lattice parameters, phonon spectra, and superconducting critical temperatures, previously estimated without considering ionic fluctuations on the crystal structure and assuming the harmonic approximation for lattice dynamics. The findings suggest that quantum ionic fluctuations have a significant impact on the crystal lattice parameters, phonon spectra, and superconducting critical temperatures. Based on our anharmonic phonon spectra, the structures will be dynamically stable at 0.3GPa for AlHfH6 and 6.2GPa for AlZrH6, ∼6 and 7GPa lower than pressures given by the harmonic approximation, respectively. Due to the anharmonic correction of their frequencies, the electron-phonon coupling constants (λ) are suppressed by 28% at 11GPa for AlHfH6 and 22% at 30GPa for AlZrH6, respectively. The decrease in λ causes Tc to be overestimated by ∼12K at 11GPa for AlHfH6 and 30GPa for AlZrH6. Even if the anharmonic and quantum effects are not as strong as those of Pm3̄n-AlH3, our results also indicate that metal hydrides with hydrogen atoms in interstitial sites are subject to anharmonic effects. Our results will inevitably stimulate future high-pressure experiments on synthesis, structural, and conductivity measurements.

Pharmacokinetic-pharmacodynamic modelling and simulation of methotrexate dosing in patients with rheumatoid arthritis.

To develop a non-linear mixed-effects population pharmacokinetic and pharmacodynamic (PK-PD) model describing the change in the concentration of methotrexate polyglutamates in erythrocytes (ery-MTX-PGn with "n" number of glutamate, representing PK component) and how this relates to modified 28-joint Disease Activity Score incorporating erythrocyte sedimentation rate (DAS-28-3) for rheumatoid arthritis (RA), representing PD component. An existing PK model was fitted to data from a study consisting of 117 RA patients. The estimation of population PK-PD parameters was performed using stochastic approximation expectation maximisation algorithm in Monolix 2021R2. The model was used to perform Monte Carlo simulations of a loading dose regimen (50mg subcutaneous methotrexate as loading doses, then 20mg weekly oral methotrexate) compared to a standard dosing regimen (10mg weekly oral methotrexate for 2 weeks, then 20mg weekly oral methotrexate). Every 40 nmol/L increase in ery-MTX-PG3-5 total concentration correlated with 1-unit reduction in DAS-28-3. Significant covariate effects on the therapeutic response of methotrexate included the use of prednisolone in the first 4 weeks (positive use correlated with 25% reduction in DAS-28-3 when other variables were constant) and patient age (every 10-year increase in age correlated with 3.4% increase in DAS-28-3 when other variables were constant). 4 methotrexate loading doses led to a higher percentage of patients achieving a good/moderate response compared to the standard regimen (Week 4: 87.6% vs. 39.8%; Week 10: 64.7% vs. 57.0%). A loading dose regimen was more likely to achieve higher ery-MTX-PG concentration and better therapeutic response after 4 weeks of methotrexate treatment.

Fighting Noise with Noise: A Stochastic Projective Quantum Eigensolver.

In the current noisy intermediate scale quantum era of quantum computation, available hardware is severely limited by both qubit count and noise levels, precluding the application of many current hybrid quantum-classical algorithms to nontrivial quantum chemistry problems. In this paper we propose applying some of the fundamental ideas of conventional Quantum Monte Carlo algorithms─stochastic sampling of both the wave function and the Hamiltonian─to quantum algorithms in order to significantly decrease quantum resource costs. In the context of an imaginary-time propagation based projective quantum eigensolver, we present a novel approach to estimating physical observables which can lead to an order of magnitude reduction in the required sampling of the quantum state to converge the ground state energy of a system relative to current state-of-the-art eigensolvers. The method can be equally applied to excited-state calculations and, combined with stochastic approximations of the system Hamiltonian, provides a promising near-term approach to Hamiltonian simulation for general chemistry on quantum devices.

Creation of an identifier database for the ammonia synthesis computer control system

The object of research is the control system of the synthesis department of the large-tonnage ammonia production unit of the AM-1360 series. An analysis of the functioning conditions of the synthesis department was carried out. The significant influence of the content of inerts in the synthesis cycle on the energy efficiency of ammonia production is shown, the optimal value of which depends both on the accepted level of prices for energy carriers and on the method of using purge gases. The need to create a computer-integrated control technology to optimize the use of purge gases is established. The function of the identifier of such a control technology is defined, namely, continuous refinement based on current information about the state of the main sections of the synthesis department, such as the synthesis column, primary and secondary condensation. The operation of these stations takes place in the conditions of seasonal and daily changes in the heat load, which causes parametric uncertainty of such parameters of the connection of the mathematical model as the concentration of ammonia in the circulating gas at the outlet of the stations listed above. Numerical assessment of these uncertainties according to the results of analytical studies for such technological objects is most often performed using stochastic approximation methods. Let’s note that the significant inertia of the objects of the synthesis department under certain conditions makes it impossible to adapt the parameter to its actual value. An algorithmic base has been created for the formation of an information array of the identifier of the computer-integrated control technology of the ammonia synthesis department, which ensures the separation of transient modes under conditions of uncertainty and allows to perform the task of identifying processes in the synthesis column, in the primary and secondary condensation units. The proposed algorithm allows to perform convergence in the department both on the general and on the component material balances. The algorithm is implemented in the MatLab application program package and tested by means of simulation based on experimental data of industrial operation of AM-1360 series ammonia synthesis units.

Flight control system design of UAV with wing incidence angle simultaneously and stochastically varied

PurposeThis study aims to simultaneously and stochastically maximize autonomous flight performance of a variable wing incidence angle having an unmanned aerial vehicle (UAV) and its flight control system (FCS) design.Design/methodology/approachA small UAV is produced in Iskenderun Technical University Drone Laboratory. Its wing incidence angle is able to change before UAV flight. FCS parameters and wing incidence angle are simultaneously and stochastically designed to maximize autonomous flight performance using an optimization method named simultaneous perturbation stochastic approximation. Obtained results are also benefitted during UAV flight simulations.FindingsApplying simultaneous and stochastic design approach for a UAV having passively morphing wing incidence angle and its flight control system, autonomous flight performance is maximized.Research limitations/implicationsPermission of the Directorate General of Civil Aviation in Turkish Republic is necessary for real-time flights.Practical implicationsSimultaneous stochastic variable wing incidence angle having UAV and its flight control system design approach is so useful for maximizing UAV autonomous flight performance.Social implicationsSimultaneous stochastic variable wing incidence angle having UAV and its flight control system design methodology succeeds confidence, excellent autonomous performance index and practical service interests of UAV users.Originality/valueCreating an innovative method to recover autonomous flight performance of a UAV and generating an innovative procedure carrying out simultaneous stochastic variable wing incidence angle having UAV and its flight control system design idea.

Implementation of unknown parameter estimation procedure for hybrid and discrete non‐linear systems

AbstractThe application of the hybrid extended Kalman filter (HEKF), hybrid unscented Kalman filter (HUKF), hybrid particle filter (HPF), and hybrid extended Kalman particle filter (HEKPF) is discussed for hybrid non‐linear filter problems, when prediction equations are continuous‐time and the update equations are discrete‐time, and also the discrete extended Kalman filter (DEKF), discrete unscented Kalman filter (DUKF), discrete particle filter (DPF), and discrete extended Kalman particle filter (DEKPF) for discrete‐time non‐linear filter problems, when prediction equations and update equations are discrete‐time. In order to assess the performance of the filters, the authors consider the non‐linear dynamics for a re‐entry vehicle. The filters are used in two hybrid and discrete states to estimate the position, velocity, and drag parameter associated with the re‐entry vehicle. Theoretical topics concerning estimating the drag parameter of a vehicle in re‐entry phase have been dealt with. Drag parameter estimation is carried out using a combination of hybrid filters and discrete filters as an effective estimator and fixed value, forgetting factor, and Robbins‐Monro stochastic approximation methods as the noise covariance matrix adjuster of the parameter.

Impact of quantum fluctuations thermally renormalize lattice vibrations on superconducting state of transition metal functionalized two-dimensional hydrides monolayer

Exploring superconducting materials stands as a pivotal pursuit in condensed matter physics, particularly, the investigation of superconductivity in two-dimensional metal hydrides is of paramount importance due to its intriguing nature. Our study focuses on elucidating the metallic state of van der Waals layered TM-H (TM = Ti, Zr, Hf) compounds, a key factor in predicting their superconducting (SC) characteristics. Leveraging an evolutionary algorithm rooted in density functional theory, we predicted the structures of hydrides, including Ti2H2, Ti2H4, Zr2H2, Zr2H4, Zr2H5, Hf2H2, Hf2H4, Hf2H5, and determined their energetically stable structures. Along with exploring the potential for SC, we conducted a comprehensive examination of relevant electronic properties. A significant aspect addressed was the influence of anharmonic phonon properties in determining the stochastic self-consistent harmonic approximation. These findings underscore the pivotal role of anharmonicity in determining the SC of two-dimensional metal hydrides.

Review of Two-timescale Stochastic Approximation Algorithms

Artificial Intelligence has gradually become an important force to drive human beings into the intelligent era, and machine learning has made great contributions to the rise and development of Artificial Intelligence. Stochastic approximation (SA) is a com-monly used optimization algorithm in machine learning, and with the complexity of practical problem scenarios, two-timescale SA have received extensive attention and research. In this paper, the basic idea and development process of SA are introduced firstly, followed by the description of several algorithmic frameworks for linear and nonlinear SA, and the specific applications of two-timescale SA in the fields of opti-mization and reinforcement learning are also introduced. Finally, the two-timescale SA is summarized and outlooked.

Prelimit Coupling and Steady-State Convergence of Constant-stepsize Nonsmooth Contractive SA

Motivated by Q-learning, we study nonsmooth contractive stochastic approximation (SA) with constant stepsize. We focus on two important classes of dynamics: 1) nonsmooth contractive SA with additive noise, and 2) synchronous and asynchronous Q-learning, which features both additive and multiplicative noise. For both dynamics, we establish weak convergence of the iterates to a stationary limit distribution in Wasserstein distance. Furthermore, we propose a prelimit coupling technique for establishing steady-state convergence and characterize the limit of the stationary distribution as the stepsize goes to zero. Using this result, we derive that the asymptotic bias of nonsmooth SA is proportional to the square root of the stepsize, which stands in sharp contrast to smooth SA. This bias characterization allows for the use of Richardson-Romberg extrapolation for bias reduction in nonsmooth SA.

First-Order Rhombohedral-to-Cubic Phase Transition in Photoexcited GeTe.

Photoexcited GeTe undergoes a nonthermal phase transition from a rhombohedral to a rocksalt crystalline phase. The microscopic mechanism and the nature of the transition are unclear. By using constrained density functional perturbation theory and by accounting for quantum anharmonicity within the stochastic self-consistent harmonic approximation, we show that the nonthermal phase transition is strongly first order and does not involve phonon softening, at odds with the thermal one. The transition is driven by the closure of the single particle gap in the photoexcited rhombohedral phase. Finally, we show that ultrafast x-ray diffraction data are consistent with a coexistence of the two phases, as expected in a first order transition. Our results are relevant for the understanding of phase transitions and bonding in phase change materials.

Stochastic optimal tuning for flight control system of morphing arm octorotor

PurposeThis study aims to discuss the simultaneous longitudinal and lateral flight control of the octorotor, a rotary wing unmanned aerial vehicle (UAV), for the first time under the effect of morphing and to improve autonomous flight performance.Design/methodology/approachThis study aims to design and control the octorotor flight control system with stochastic optimal tuning under morphnig effect. For this purpose, models of different arm lengths of the octorotor were drawn in the Solidworks program. The morphing was carried out by simultaneously lengthening or shortening the arm lengths of the octorotor. The morphing rate was estimated by using simultaneous perturbation stochastic approximation (SPSA). The stochastic gradient descent algorithm, which is frequently used in machine learning, was used to estimate the changing moments of inertia with the change of arm lengths. The proportional integral derivative (PID) controller has been preferred as an octorotor control algorithm because of its simplicity of structure. The PID gains required to control both longitudinal and lateral flight were also estimated with SPSA.FindingsWith SPSA, three longitudinal flight PID gains, three lateral flight PID gains and one morphing ratio were estimated. PID gains remained within the limits set for SPSA, giving satisfactory results. In addition, the cost index created was 93% successful. The gradient descent algorithm used for the moment of inertia estimation achieved the optimum result in 1,570 iterations. However, in the simulations made with the obtained data, longitudinal and lateral flight was successfully carried out.Originality/valueOctorotor longitudinal and lateral flight control was performed quickly and effectively with the proposed method. In addition, the desired parameters were obtained with the optimization methods used, and the longitudinal and lateral flight of the octorotor was successfully carried out in the desired trajectory.

Phase 1 Open-Label Study of Omigapil in Patients With LAMA2- or COL6-Related Dystrophy.

Omigapil is a small molecule which inhibits the GAPDH-Siah1-mediated apoptosis pathway. Apoptosis is a pathomechanism underlying the congenital muscular dystrophy subtypes LAMA2-related dystrophy (LAMA2-RD) and COL6-related dystrophy (COL6-RD). Studies of omigapil in the (dyw/dyw) LAMA2-RD mouse model demonstrated improved survival, and studies in the (dy2J/dy2J) LAMA2-RD mouse model and the (Col6a1-/-) COL6-RD mouse model demonstrated decreased apoptosis. A phase 1 open-label, sequential group, ascending oral dose, cohort study of omigapil in patients with LAMA2-RD or COL6-RD ages 5-16 years was performed (1) to establish the pharmacokinetic (PK) profile of omigapil at a range of doses, (2) to evaluate the safety and tolerability of omigapil at a range of doses, and (3) to establish the feasibility of conducting disease-relevant clinical assessments. Patients were enrolled in cohorts of size 4, with each patient receiving 4 weeks of vehicle run-in and 12 weeks of study drug (at daily doses ranging from 0.02 to 0.08 mg/kg). PK data from each cohort were analyzed before each subsequent dosing cohort was enrolled. A novel, adaptive dose-finding method (stochastic approximation with virtual observation recursion) was used to allow for dose escalation/reduction between cohorts based on PK data. Twenty patients were enrolled at the NIH (LAMA2-RD: N = 10; COL6-RD: N = 10). Slightly greater than dose-proportional increases in systemic exposure to omigapil were seen at doses 0.02-0.08 mg/kg/d. The dose which achieved patient exposure within the pre-established target area under the plasma concentration-vs-time curve (AUC0-24h) range was 0.06 mg/kg/d. In general, omigapil was safe and well tolerated. No consistent changes were seen in the disease-relevant clinical assessments during the duration of the study. This study represents the thus far only clinical trial of a therapeutic small molecule for LAMA2-RD and COL6-RD, completed with an adaptive trial design to arrive at dose adjustments. The trial met its primary end point and established that the PK profile of omigapil is suitable for further development in pediatric patients with LAMA2-RD or COL6-RD, the most common forms of congenital muscular dystrophy. While within the short duration of the study disease-relevant clinical assessments did not demonstrate significant changes, this study establishes the feasibility of performing interventional clinical trials in these rare disease patient populations. This study provides Class IV evidence of omigapil in a dose-finding phase 1 study. Clinical Trials NCT01805024.

Asynchronous Parallel Large-Scale Gaussian Process Regression.

Gaussian process regression (GPR) is an important nonparametric learning method in machine learning research with many real-world applications. It is well known that training large-scale GPR is a challenging task due to the required heavy computational cost and large volume memory. To address this challenging problem, in this article, we propose an asynchronous doubly stochastic gradient algorithm to handle the large-scale training of GPR. We formulate the GPR to a convex optimization problem, i.e., kernel ridge regression. After that, in order to efficiently solve this convex kernel problem, we first use the random feature mapping method to approximate the kernel model and then utilize two unbiased stochastic approximations, i.e., stochastic variance reduced gradient and stochastic coordinate descent, to update the solution asynchronously and in parallel. In this way, our algorithm scales well in both sample size and dimensionality, and speeds up the training computation. More importantly, we prove that our algorithm has a global linear convergence rate. Our experimental results on eight large-scale benchmark datasets with both regression and classification tasks show that the proposed algorithm outperforms the existing state-of-the-art GPR methods.

A closed-measure approach to stochastic approximation

This paper introduces a new method to tackle the issue of the almost sure convergence of stochastic approximation algorithms defined from a differential inclusion. Under the assumption of slowly decaying step-sizes, we establish that the set of essential accumulation points of the iterates belongs to the Birkhoff centre associated with the differential inclusion. Unlike previous works, our results do not rely on the notion of asymptotic pseudotrajectories, predominant technique to address the convergence problem. They follow as a consequence of Young's superposition principle for closed measures. This perspective bridges the gap between Young's principle and the notion of invariant measure of set-valued dynamical systems introduced by Faure and Roth. Also, the proposed method allows for obtaining sufficient conditions under which the velocities locally compensate around any essential accumulation point.

A Stochastic Inexact Sequential Quadratic Optimization Algorithm for Nonlinear Equality-Constrained Optimization

A stochastic algorithm is proposed, analyzed, and tested experimentally for solving continuous optimization problems with nonlinear equality constraints. It is assumed that constraint function and derivative values can be computed but that only stochastic approximations are available for the objective function and its derivatives. The algorithm is of the sequential quadratic optimization variety. Distinguishing features of the algorithm are that it only employs stochastic objective gradient estimates that satisfy a relatively weak set of assumptions (while using neither objective function values nor estimates of them) and that it allows inexact subproblem solutions to be employed, the latter of which is particularly useful in large-scale settings when the matrices defining the subproblems are too large to form and/or factorize. Conditions are imposed on the inexact subproblem solutions that account for the fact that only stochastic objective gradient estimates are employed. Convergence results are established for the method. Numerical experiments show that the proposed method vastly outperforms a stochastic subgradient method and can outperform an alternative sequential quadratic programming algorithm that employs highly accurate subproblem solutions in every iteration. Funding: This material is based upon work supported by the National Science Foundation [Awards CCF-1740796 and CCF-2139735] and the Office of Naval Research [Award N00014-21-1-2532].

Population Pharmacokinetics and Limited Sampling Strategy of Mycophenolate Mofetil for Indian Patients With Lupus Nephritis.

Mycophenolic acid is widely used to treat lupus nephritis (LN). However, it exhibits complex pharmacokinetics with large interindividual variability. This study aimed to develop a population pharmacokinetic (popPK) model and a 3-sample limited sampling strategy (LSS) to optimize therapeutic drug monitoring in Indian patients with LN. Five blood samples from each LN patient treated with mycophenolic acid were collected at steady-state predose and 1, 2, 4, and 6 hours postdose. Demographic parameters were tested as covariates to explain interindividual variability. PopPK analysis was performed using Monolix and the stochastic approximation expectation-maximization algorithm. An LSS was derived from 500 simulated pharmacokinetic (PK) profiles using maximum a posteriori Bayesian estimation to estimate individual PK parameters and area under the curve (AUC). The LSS-calculated AUC was compared with the AUC calculated using the trapezoidal rule and all the simulated samples. A total of 51 patients were included in this study. Based on the 245 mycophenolic acid concentrations, a 1-compartmental model with double absorption using gamma distributions best fitted the data. None of the covariates improved the model significantly. The model was internally validated using diagnostic plots, prediction-corrected visual predictive checks, and bootstrapping. The best LSS included samples at 1, 2, and 4 hours postdose and exhibited good performances in an external dataset (root mean squared error, 21.9%; mean bias, -4.20%). The popPK model developed in this study adequately estimated the PK of mycophenolic acid in adult Indian patients with LN. This simple LSS can optimize TDM based on the AUC in routine practice.

Enhancing buildings' energy efficiency prediction through advanced data fusion and fuzzy classification

This study proposes a method to predict buildings' energy efficiency based on available descriptive information and without a physical visit, by merging diverse datasets and employing advanced classification techniques. By integrating geographical, structural, legal, and socio-economic data with Energy Performance Certificate (EPC) observations, our approach yields a rich learning set. Through variable selection methods like forward selection with KNN and simultaneous perturbation stochastic approximation for fuzzy KNN, we refine model variables. Comparing fuzzy and hard classification using KNN, Kriging or Random Forest approaches, we find fuzzy classification more adept at capturing nuanced energy inefficiency indicators. Our study highlights the importance of mass energy efficiency prediction for sustainable renovation efforts.