Stochastic Approximation Research Articles

Flight control system design of UAV with wing incidence angle simultaneously and stochastically varied

PurposeThis study aims to simultaneously and stochastically maximize autonomous flight performance of a variable wing incidence angle having an unmanned aerial vehicle (UAV) and its flight control system (FCS) design.Design/methodology/approachA small UAV is produced in Iskenderun Technical University Drone Laboratory. Its wing incidence angle is able to change before UAV flight. FCS parameters and wing incidence angle are simultaneously and stochastically designed to maximize autonomous flight performance using an optimization method named simultaneous perturbation stochastic approximation. Obtained results are also benefitted during UAV flight simulations.FindingsApplying simultaneous and stochastic design approach for a UAV having passively morphing wing incidence angle and its flight control system, autonomous flight performance is maximized.Research limitations/implicationsPermission of the Directorate General of Civil Aviation in Turkish Republic is necessary for real-time flights.Practical implicationsSimultaneous stochastic variable wing incidence angle having UAV and its flight control system design approach is so useful for maximizing UAV autonomous flight performance.Social implicationsSimultaneous stochastic variable wing incidence angle having UAV and its flight control system design methodology succeeds confidence, excellent autonomous performance index and practical service interests of UAV users.Originality/valueCreating an innovative method to recover autonomous flight performance of a UAV and generating an innovative procedure carrying out simultaneous stochastic variable wing incidence angle having UAV and its flight control system design idea.

Implementation of unknown parameter estimation procedure for hybrid and discrete non‐linear systems

AbstractThe application of the hybrid extended Kalman filter (HEKF), hybrid unscented Kalman filter (HUKF), hybrid particle filter (HPF), and hybrid extended Kalman particle filter (HEKPF) is discussed for hybrid non‐linear filter problems, when prediction equations are continuous‐time and the update equations are discrete‐time, and also the discrete extended Kalman filter (DEKF), discrete unscented Kalman filter (DUKF), discrete particle filter (DPF), and discrete extended Kalman particle filter (DEKPF) for discrete‐time non‐linear filter problems, when prediction equations and update equations are discrete‐time. In order to assess the performance of the filters, the authors consider the non‐linear dynamics for a re‐entry vehicle. The filters are used in two hybrid and discrete states to estimate the position, velocity, and drag parameter associated with the re‐entry vehicle. Theoretical topics concerning estimating the drag parameter of a vehicle in re‐entry phase have been dealt with. Drag parameter estimation is carried out using a combination of hybrid filters and discrete filters as an effective estimator and fixed value, forgetting factor, and Robbins‐Monro stochastic approximation methods as the noise covariance matrix adjuster of the parameter.

Impact of quantum fluctuations thermally renormalize lattice vibrations on superconducting state of transition metal functionalized two-dimensional hydrides monolayer

Exploring superconducting materials stands as a pivotal pursuit in condensed matter physics, particularly, the investigation of superconductivity in two-dimensional metal hydrides is of paramount importance due to its intriguing nature. Our study focuses on elucidating the metallic state of van der Waals layered TM-H (TM = Ti, Zr, Hf) compounds, a key factor in predicting their superconducting (SC) characteristics. Leveraging an evolutionary algorithm rooted in density functional theory, we predicted the structures of hydrides, including Ti2H2, Ti2H4, Zr2H2, Zr2H4, Zr2H5, Hf2H2, Hf2H4, Hf2H5, and determined their energetically stable structures. Along with exploring the potential for SC, we conducted a comprehensive examination of relevant electronic properties. A significant aspect addressed was the influence of anharmonic phonon properties in determining the stochastic self-consistent harmonic approximation. These findings underscore the pivotal role of anharmonicity in determining the SC of two-dimensional metal hydrides.

Review of Two-timescale Stochastic Approximation Algorithms

Artificial Intelligence has gradually become an important force to drive human beings into the intelligent era, and machine learning has made great contributions to the rise and development of Artificial Intelligence. Stochastic approximation (SA) is a com-monly used optimization algorithm in machine learning, and with the complexity of practical problem scenarios, two-timescale SA have received extensive attention and research. In this paper, the basic idea and development process of SA are introduced firstly, followed by the description of several algorithmic frameworks for linear and nonlinear SA, and the specific applications of two-timescale SA in the fields of opti-mization and reinforcement learning are also introduced. Finally, the two-timescale SA is summarized and outlooked.

Prelimit Coupling and Steady-State Convergence of Constant-stepsize Nonsmooth Contractive SA

Motivated by Q-learning, we study nonsmooth contractive stochastic approximation (SA) with constant stepsize. We focus on two important classes of dynamics: 1) nonsmooth contractive SA with additive noise, and 2) synchronous and asynchronous Q-learning, which features both additive and multiplicative noise. For both dynamics, we establish weak convergence of the iterates to a stationary limit distribution in Wasserstein distance. Furthermore, we propose a prelimit coupling technique for establishing steady-state convergence and characterize the limit of the stationary distribution as the stepsize goes to zero. Using this result, we derive that the asymptotic bias of nonsmooth SA is proportional to the square root of the stepsize, which stands in sharp contrast to smooth SA. This bias characterization allows for the use of Richardson-Romberg extrapolation for bias reduction in nonsmooth SA.

First-Order Rhombohedral-to-Cubic Phase Transition in Photoexcited GeTe.

Photoexcited GeTe undergoes a nonthermal phase transition from a rhombohedral to a rocksalt crystalline phase. The microscopic mechanism and the nature of the transition are unclear. By using constrained density functional perturbation theory and by accounting for quantum anharmonicity within the stochastic self-consistent harmonic approximation, we show that the nonthermal phase transition is strongly first order and does not involve phonon softening, at odds with the thermal one. The transition is driven by the closure of the single particle gap in the photoexcited rhombohedral phase. Finally, we show that ultrafast x-ray diffraction data are consistent with a coexistence of the two phases, as expected in a first order transition. Our results are relevant for the understanding of phase transitions and bonding in phase change materials.

Stochastic optimal tuning for flight control system of morphing arm octorotor

PurposeThis study aims to discuss the simultaneous longitudinal and lateral flight control of the octorotor, a rotary wing unmanned aerial vehicle (UAV), for the first time under the effect of morphing and to improve autonomous flight performance.Design/methodology/approachThis study aims to design and control the octorotor flight control system with stochastic optimal tuning under morphnig effect. For this purpose, models of different arm lengths of the octorotor were drawn in the Solidworks program. The morphing was carried out by simultaneously lengthening or shortening the arm lengths of the octorotor. The morphing rate was estimated by using simultaneous perturbation stochastic approximation (SPSA). The stochastic gradient descent algorithm, which is frequently used in machine learning, was used to estimate the changing moments of inertia with the change of arm lengths. The proportional integral derivative (PID) controller has been preferred as an octorotor control algorithm because of its simplicity of structure. The PID gains required to control both longitudinal and lateral flight were also estimated with SPSA.FindingsWith SPSA, three longitudinal flight PID gains, three lateral flight PID gains and one morphing ratio were estimated. PID gains remained within the limits set for SPSA, giving satisfactory results. In addition, the cost index created was 93% successful. The gradient descent algorithm used for the moment of inertia estimation achieved the optimum result in 1,570 iterations. However, in the simulations made with the obtained data, longitudinal and lateral flight was successfully carried out.Originality/valueOctorotor longitudinal and lateral flight control was performed quickly and effectively with the proposed method. In addition, the desired parameters were obtained with the optimization methods used, and the longitudinal and lateral flight of the octorotor was successfully carried out in the desired trajectory.

Phase 1 Open-Label Study of Omigapil in Patients With LAMA2- or COL6-Related Dystrophy.

Omigapil is a small molecule which inhibits the GAPDH-Siah1-mediated apoptosis pathway. Apoptosis is a pathomechanism underlying the congenital muscular dystrophy subtypes LAMA2-related dystrophy (LAMA2-RD) and COL6-related dystrophy (COL6-RD). Studies of omigapil in the (dyw/dyw) LAMA2-RD mouse model demonstrated improved survival, and studies in the (dy2J/dy2J) LAMA2-RD mouse model and the (Col6a1-/-) COL6-RD mouse model demonstrated decreased apoptosis. A phase 1 open-label, sequential group, ascending oral dose, cohort study of omigapil in patients with LAMA2-RD or COL6-RD ages 5-16 years was performed (1) to establish the pharmacokinetic (PK) profile of omigapil at a range of doses, (2) to evaluate the safety and tolerability of omigapil at a range of doses, and (3) to establish the feasibility of conducting disease-relevant clinical assessments. Patients were enrolled in cohorts of size 4, with each patient receiving 4 weeks of vehicle run-in and 12 weeks of study drug (at daily doses ranging from 0.02 to 0.08 mg/kg). PK data from each cohort were analyzed before each subsequent dosing cohort was enrolled. A novel, adaptive dose-finding method (stochastic approximation with virtual observation recursion) was used to allow for dose escalation/reduction between cohorts based on PK data. Twenty patients were enrolled at the NIH (LAMA2-RD: N = 10; COL6-RD: N = 10). Slightly greater than dose-proportional increases in systemic exposure to omigapil were seen at doses 0.02-0.08 mg/kg/d. The dose which achieved patient exposure within the pre-established target area under the plasma concentration-vs-time curve (AUC0-24h) range was 0.06 mg/kg/d. In general, omigapil was safe and well tolerated. No consistent changes were seen in the disease-relevant clinical assessments during the duration of the study. This study represents the thus far only clinical trial of a therapeutic small molecule for LAMA2-RD and COL6-RD, completed with an adaptive trial design to arrive at dose adjustments. The trial met its primary end point and established that the PK profile of omigapil is suitable for further development in pediatric patients with LAMA2-RD or COL6-RD, the most common forms of congenital muscular dystrophy. While within the short duration of the study disease-relevant clinical assessments did not demonstrate significant changes, this study establishes the feasibility of performing interventional clinical trials in these rare disease patient populations. This study provides Class IV evidence of omigapil in a dose-finding phase 1 study. Clinical Trials NCT01805024.

Asynchronous Parallel Large-Scale Gaussian Process Regression.

Gaussian process regression (GPR) is an important nonparametric learning method in machine learning research with many real-world applications. It is well known that training large-scale GPR is a challenging task due to the required heavy computational cost and large volume memory. To address this challenging problem, in this article, we propose an asynchronous doubly stochastic gradient algorithm to handle the large-scale training of GPR. We formulate the GPR to a convex optimization problem, i.e., kernel ridge regression. After that, in order to efficiently solve this convex kernel problem, we first use the random feature mapping method to approximate the kernel model and then utilize two unbiased stochastic approximations, i.e., stochastic variance reduced gradient and stochastic coordinate descent, to update the solution asynchronously and in parallel. In this way, our algorithm scales well in both sample size and dimensionality, and speeds up the training computation. More importantly, we prove that our algorithm has a global linear convergence rate. Our experimental results on eight large-scale benchmark datasets with both regression and classification tasks show that the proposed algorithm outperforms the existing state-of-the-art GPR methods.

A closed-measure approach to stochastic approximation

This paper introduces a new method to tackle the issue of the almost sure convergence of stochastic approximation algorithms defined from a differential inclusion. Under the assumption of slowly decaying step-sizes, we establish that the set of essential accumulation points of the iterates belongs to the Birkhoff centre associated with the differential inclusion. Unlike previous works, our results do not rely on the notion of asymptotic pseudotrajectories, predominant technique to address the convergence problem. They follow as a consequence of Young's superposition principle for closed measures. This perspective bridges the gap between Young's principle and the notion of invariant measure of set-valued dynamical systems introduced by Faure and Roth. Also, the proposed method allows for obtaining sufficient conditions under which the velocities locally compensate around any essential accumulation point.

A Stochastic Inexact Sequential Quadratic Optimization Algorithm for Nonlinear Equality-Constrained Optimization

A stochastic algorithm is proposed, analyzed, and tested experimentally for solving continuous optimization problems with nonlinear equality constraints. It is assumed that constraint function and derivative values can be computed but that only stochastic approximations are available for the objective function and its derivatives. The algorithm is of the sequential quadratic optimization variety. Distinguishing features of the algorithm are that it only employs stochastic objective gradient estimates that satisfy a relatively weak set of assumptions (while using neither objective function values nor estimates of them) and that it allows inexact subproblem solutions to be employed, the latter of which is particularly useful in large-scale settings when the matrices defining the subproblems are too large to form and/or factorize. Conditions are imposed on the inexact subproblem solutions that account for the fact that only stochastic objective gradient estimates are employed. Convergence results are established for the method. Numerical experiments show that the proposed method vastly outperforms a stochastic subgradient method and can outperform an alternative sequential quadratic programming algorithm that employs highly accurate subproblem solutions in every iteration. Funding: This material is based upon work supported by the National Science Foundation [Awards CCF-1740796 and CCF-2139735] and the Office of Naval Research [Award N00014-21-1-2532].

Population Pharmacokinetics and Limited Sampling Strategy of Mycophenolate Mofetil for Indian Patients With Lupus Nephritis.

Mycophenolic acid is widely used to treat lupus nephritis (LN). However, it exhibits complex pharmacokinetics with large interindividual variability. This study aimed to develop a population pharmacokinetic (popPK) model and a 3-sample limited sampling strategy (LSS) to optimize therapeutic drug monitoring in Indian patients with LN. Five blood samples from each LN patient treated with mycophenolic acid were collected at steady-state predose and 1, 2, 4, and 6 hours postdose. Demographic parameters were tested as covariates to explain interindividual variability. PopPK analysis was performed using Monolix and the stochastic approximation expectation-maximization algorithm. An LSS was derived from 500 simulated pharmacokinetic (PK) profiles using maximum a posteriori Bayesian estimation to estimate individual PK parameters and area under the curve (AUC). The LSS-calculated AUC was compared with the AUC calculated using the trapezoidal rule and all the simulated samples. A total of 51 patients were included in this study. Based on the 245 mycophenolic acid concentrations, a 1-compartmental model with double absorption using gamma distributions best fitted the data. None of the covariates improved the model significantly. The model was internally validated using diagnostic plots, prediction-corrected visual predictive checks, and bootstrapping. The best LSS included samples at 1, 2, and 4 hours postdose and exhibited good performances in an external dataset (root mean squared error, 21.9%; mean bias, -4.20%). The popPK model developed in this study adequately estimated the PK of mycophenolic acid in adult Indian patients with LN. This simple LSS can optimize TDM based on the AUC in routine practice.

Enhancing buildings' energy efficiency prediction through advanced data fusion and fuzzy classification

This study proposes a method to predict buildings' energy efficiency based on available descriptive information and without a physical visit, by merging diverse datasets and employing advanced classification techniques. By integrating geographical, structural, legal, and socio-economic data with Energy Performance Certificate (EPC) observations, our approach yields a rich learning set. Through variable selection methods like forward selection with KNN and simultaneous perturbation stochastic approximation for fuzzy KNN, we refine model variables. Comparing fuzzy and hard classification using KNN, Kriging or Random Forest approaches, we find fuzzy classification more adept at capturing nuanced energy inefficiency indicators. Our study highlights the importance of mass energy efficiency prediction for sustainable renovation efforts.

Online Stochastic DCA With Applications to Principal Component Analysis.

Stochastic algorithms are well-known for their performance in the era of big data. In this article, we study nonsmooth stochastic Difference-of-Convex functions (DC) programs-the major class of nonconvex stochastic optimization, which have a variety of applications in divers domains, in particular, machine learning. We propose new online stochastic algorithms based on the state-of-the-art DC Algorithm (DCA)-a powerful approach in nonconvex programming framework, in the online context of streaming data continuously generated by some (unknown) source distribution. The new schemes use the stochastic approximations (SAs) principle: deterministic quantities of the standard DCA are replaced by their noisy estimators constructed using newly arriving samples. The convergence analysis of the proposed algorithms is studied intensively with the help of tools from modern convex analysis and martingale theory. Finally, we study several aspects of the proposed algorithms on an important problem in machine learning: the expected problem in principal component analysis (PCA).

Kapitza Stabilization of Quantum Critical Order

Dynamical perturbations modify the states of classical systems in surprising ways and give rise to important applications in science and technology. For example, Floquet engineering exploits the possibility of band formation in the frequency domain when a strong, periodic variation is imposed on parameters such as spring constants. We describe here Kapitza engineering, where a drive field oscillating at a frequency much higher than the characteristic frequencies for the linear response of a system changes the potential energy surface so much that maxima found at equilibrium become local minima, in precise analogy to the celebrated Kapitza pendulum where the unstable inverted configuration, with the mass above rather than below the fulcrum, actually becomes stable. Our starting point is a quantum field theory of the Ginzburg-Devonshire type, suitable for many condensed matter systems, including particularly ferroelectrics and quantum paralectrics. We show that an off-resonance oscillatory electric field generated by a laser-driven terahertz source can induce ferroelectric order in the quantum-critical limit. Heating effects are estimated to be manageable using pulsed radiation; “hidden” radiation-induced order can persist to low temperatures without further pumping due to stabilization by strain. We estimate the Ginzburg-Devonshire free-energy coefficients in SrTiO3 using density-functional theory and the stochastic self-consistent harmonic approximation accelerated by a machine-learned force field. Although we find that SrTiO3 is not an optimal choice for Kapitza stabilization, we show that scanning for further candidate materials can be performed at the computationally convenient density-functional theory level. We suggest second harmonic generation, soft-mode spectroscopy, and x-ray diffraction experiments to characterize the induced order. Published by the American Physical Society 2024

Uncertainty Computation at Finite Distance in Nonlinear Mixed Effects Models-a New Method Based on Metropolis-Hastings Algorithm.

The standard errors (SE) of the maximum likelihood estimates (MLE) of the population parameter vector in nonlinear mixed effect models (NLMEM) are usually estimated using the inverse of the Fisher information matrix (FIM). However, at a finite distance, i.e. far from the asymptotic, the FIM can underestimate the SE of NLMEM parameters. Alternatively, the standard deviation of the posterior distribution, obtained in Stan via the Hamiltonian Monte Carlo algorithm, has been shown to be a proxy for the SE, since, under some regularity conditions on the prior, the limiting distributions of the MLE and of the maximum a posterior estimator in a Bayesian framework are equivalent. In this work, we develop a similar method using the Metropolis-Hastings (MH) algorithm in parallel to the stochastic approximation expectation maximisation (SAEM) algorithm, implemented in the saemix R package. We assess this method on different simulation scenarios and data from a real case study, comparing it to other SE computation methods. The simulation study shows that our method improves the results obtained with frequentist methods at finite distance. However, it performed poorly in a scenario with the high variability and correlations observed in the real case study, stressing the need for calibration.

Quantum optimization algorithms: Energetic implications

SummarySince the dawn of quantum computing (QC), theoretical developments like Shor's algorithm proved the conceptual superiority of QC over traditional computing. However, such quantum supremacy claims are difficult to achieve in practice because of the technical challenges of realizing noiseless qubits. In the near future, QC applications will need to rely on noisy quantum devices that offload part of their work to classical devices. One way to achieve this is by using parameterized quantum circuits in optimization or even in machine learning tasks. The energy requirements of quantum algorithms have not yet been studied extensively. In this article, we explore several optimization algorithms using both theoretical insights and numerical experiments to understand their impact on energy consumption. Specifically, we highlight why and how algorithms like quantum natural gradient descent, simultaneous perturbation stochastic approximations or circuit learning methods, are at least to more energy efficient than their classical counterparts; why feedback‐based quantum optimization is energy‐inefficient; and how techniques like Rosalin can improve the energy efficiency of other algorithms by a factor of 20. Finally, we use the NchooseK high‐level programming model to run optimization problems on both gate‐based quantum computers and quantum annealers. Empirical data indicate that these optimization problems run faster, have better success rates, and consume less energy on quantum annealers than on their gate‐based counterparts.

Finite-Time High-Probability Bounds for Polyak–Ruppert Averaged Iterates of Linear Stochastic Approximation

This paper provides a finite-time analysis of linear stochastic approximation (LSA) algorithms with fixed step size, a core method in statistics and machine learning. LSA is used to compute approximate solutions of a d-dimensional linear system [Formula: see text] for which [Formula: see text] can only be estimated by (asymptotically) unbiased observations [Formula: see text]. We consider here the case where [Formula: see text] is an a sequence of independent and identically distributed random variables sequence or a uniformly geometrically ergodic Markov chain. We derive pth moment and high-probability deviation bounds for the iterates defined by LSA and its Polyak–Ruppert-averaged version. Our finite-time instance-dependent bounds for the averaged LSA iterates are sharp in the sense that the leading term we obtain coincides with the local asymptotic minimax limit. Moreover, the remainder terms of our bounds admit a tight dependence on the mixing time [Formula: see text] of the underlying chain and the norm of the noise variables. We emphasize that our result requires the LSA step size to scale only with logarithm of the problem dimension d. Funding: The work of A. Durmus and E. Moulines was partly supported by [Grant ANR-19-CHIA-0002]. This project received funding from the European Research Council [ERC-SyG OCEAN Grant 101071601]. The research of A. Naumov and S. Samsonov was prepared within the framework of the HSE University Basic Research Program.

RPEM: Randomized Monte Carlo parametric expectation maximization algorithm.

Inspired from quantum Monte Carlo, by sampling discrete and continuous variables at the same time using the Metropolis-Hastings algorithm, we present a novel, fast, and accurate high performance Monte Carlo Parametric Expectation Maximization (MCPEM) algorithm. We named it Randomized Parametric Expectation Maximization (RPEM). We compared RPEM with NONMEM's Importance Sampling Method (IMP), Monolix's Stochastic Approximation Expectation Maximization (SAEM), and Certara's Quasi-Random Parametric Expectation Maximization (QRPEM) for a realistic two-compartment voriconazole model with ordinary differential equations using simulated data. We show that RPEM is as fast and as accurate as the algorithms IMP, QRPEM, and SAEM for the voriconazole model in reconstructing the population parameters, for the normal and log-normal cases.

The generalized Robbins–Monro process and its application to psychophysical experiments for threshold estimation

In classical psychophysics, the study of threshold and underlying representations is of theoretical interest, and the relevant issue of finding the stimulus intensity corresponding to a certain threshold level is an important topic. In the literature, researchers have developed various adaptive (also known as ‘up-down’) methods, including the fixed step-size and variable step-size methods, for the estimation of threshold. A common feature of this family of methods is that the stimulus to be assigned to the current trial depends upon the participant’s response in the previous trial(s), and very often a binary response format is adopted. A well-known earlier work of the variable step-size adaptive methods is the Robbins–Monro process (and its accelerated version). However, previous studies have paid little attention to other facets of response variables (in addition to the binary response variable) that could be jointly embedded into the process. This article concerns a generalization of the Robbins–Monro process by incorporating an additional response variable, such as the response time or the response confidence, into the process. We first prove the consistency of the estimator from the generalized method. We then conduct a Monte Carlo simulation study to explore some finite-sample properties of the estimator from the generalized method with either the response time or the response confidence as the variable of interest, and compare its performance with the original method. The results show that the two methods (and their accelerated version) are comparable. The issue of relative efficiency is also discussed.