General Convergence Research Articles

Hyperparameter estimation using a resolution matrix for Bayesian sensing

Bayesian image processing has progressively increased in importance in various fields of the natural sciences. It utilizes prior knowledge and forward models of the observational processes through Bayes’ theorem, enabling the accurate estimation of model parameters that represent the physical quantities of the target. Moreover, using hyperparameter estimation, we can determine the hidden physical parameters that govern the processes in and the structure of the target and sensing systems, such as the spatial continuity of the model parameters and the magnitude of the observational noise. Such a general framework, which uses Bayesian estimation to understand the essential physics of a target system, can be called ‘Bayesian sensing’. This paper discusses the physical meaning of and the mechanism underlying Bayesian sensing using the concept of resolution in the spatial-inversion problem. The spatial resolution of the model parameters can be mapped using a resolution matrix, more rigorously, a model resolution matrix defined as a linear mapping from the true model parameters to the recovered model parameters. We formulate the resolution matrix for Bayesian image processing and also show that in terms of resolution, the optimal hyperparameters are obtained from internally consistent equations that connect the estimated optimal hyperparameters with the actual ones calculated from the estimated model parameters. In addition, we show the equivalence of the internally consistent equations to the expectation-maximization (EM) algorithm and formulate the confidence intervals for the estimated hyperparameters, which indicate the general convergence of the hyperparameter estimates. We also show the effectiveness of the proposed method by performing synthetic numerical tests for two inversion-problem settings: linear travel-time seismic tomography and image deblurring. The resulting equations can contribute to understanding the hidden physical processes in and the structure of the target and observation systems for various problems.

Nonparametric inference for reversed mean models with panel count data

Panel count data typically refer to data arising from studies with recurrent events, in which subjects are observed only at discrete time points rather than under continuous observations. We investigate a general situation where a recurrent event process is eventually truncated by an informative terminal event and we are particularly interested in behaviors of the recurrent event process near the terminal event. We propose a reversed mean model for estimating the mean function of the recurrent event process. We develop a two-stage sieve likelihood-based method to estimate the mean function, which overcomes the computational difficulties arising from a nuisance functional parameter involved in the likelihood. The consistency and the convergence rate of the two-stage estimator are established. Allowing for the convergence rate slower than the standard rate, we develop the general weak convergence theory of M-estimators with a nuisance functional parameter, and then apply it to the proposed estimator for deriving the asymptotic normality. Furthermore, a class of two-sample tests is developed. The proposed methods are evaluated with extensive simulation studies and illustrated with panel count data from the Chinese Longitudinal Healthy Longevity Study.

Generic Convergence to Periodic Orbits for Monotone Flows with Respect to 2-Cones and Applications to SEIRS Models

Generic Convergence to Periodic Orbits for Monotone Flows with Respect to 2-Cones and Applications to SEIRS Models

Asymptotically Optimal Inventory Control for Assemble-to-Order Systems

We consider assemble-to-order (ATO) inventory systems with a general bill of materials and general deterministic lead times. Unsatisfied demands are always backlogged. We apply a four-step asymptotic framework to develop inventory policies for minimizing the long-run average expected total inventory cost. Our approach features a multistage stochastic program (SP) to establish a lower bound on the inventory cost and determine parameter values for inventory control. Our replenishment policy deviates from the conventional constant base stock policies to accommodate nonidentical lead times. Our component allocation policy differentiates demands based on backlog costs, bill of materials, and component availabilities. We prove that our policy is asymptotically optimal on the diffusion scale, that is, as the longest lead time grows, the percentage difference between the average cost under our policy and its lower bound converges to zero. In developing these results, we formulate a broad stochastic tracking model and prove general convergence results from which the asymptotic optimality of our policy follows as specialized corollaries. Funding: This study is based on work supported by the National Science Foundation [Grant CMMI-1363314].

The infinite product of contraction semigroups on l^{1}({mathbb {N}}) and l^{infty }({mathbb {N}})

I provide an example of a family of commuting contraction semigroups (e^{t B_{n}})_{nin {mathbb {N}}} defined on l^{1}({mathbb {N}}) such that the product semigroup prod _{n=1}^{infty }e^{tB_{n}} exists and has bounded generator. The infinite product of the corresponding family of adjoint semigroups (e^{t B_{n}^{*}})_{nin {mathbb {N}}} defined on l^{infty }({mathbb {N}}) also exists and its generator is bounded. I give explicit formulae for these generators. The results follow from a general convergence theorem for such semigroups proved in Arendt et al. (J Funct Anal 160: 524–542, 1998).

Pathwise convergence under Knightian uncertainty

In this note, we will obtain the first quasi-surely convergence rate of approximation of stochastic differential equations driven by G-Brownian motion. We carry it out by making a relation between Lp and quasi-surely convergences by considering this point that in general Lp convergence does not imply quasi-surely one and neither does quasi-surely convergence. This result will show that the rate of quasi-surely convergence can not exceed that of p-th mean.

An Approximation Theorem and Generic Convergence of Solutions of Inverse Quasivariational Inequality Problems

In this paper, we mainly obtain an approximation theorem and generic convergence of solutions for inverse quasivariational inequality problems. First, we define the concept of the approximate solution to inverse quasivariational inequality problems under bounded rationality theory. Afterward, an approximation theorem that satisfies fairly mild assumptions is proved. Moreover, we establish a function space and discuss the convergence properties of solutions for inverse quasivariational inequality problems by the method of set-valued analysis. Finally, we prove that most of inverse quasivariational inequality problems are stable in the case of perturbation of the objective function. These results are new, which improve the corresponding outcomes of the recent literatures.

An Optimal Iterative Technique for Multiple Root Finder of Nonlinear Problems

In this paper, an optimal higher-order iterative technique to approximate the multiple roots of a nonlinear equation has been presented. The proposed technique has special properties: a two-point method that does not involve any derivatives, has an optimal convergence of fourth-order, is cost-effective, is more stable, and has better numerical results. In addition to this, we adopt the weight function approach at both substeps (which provide us with a more general form of two-point methods). Firstly, the convergence order is studied for multiplicity m=2,3 by Taylor’s series expansion and then general convergence for m≥4 is proved. We have demonstrated the applicability of our methods to six numerical problems. Out of them: the first one is the well-known Van der Waals ideal gas problem, the second one is used to study the blood rheology model, the third one is chosen from the linear algebra (namely, eigenvalue), and the remaining three are academic problems. We concluded on the basis of obtained CPU timing, computational order of convergence, and absolute errors between two consecutive iterations for which our methods illustrate better results as compared to earlier studies.

Variance reduction on general adaptive stochastic mirror descent

In this work, we investigate the idea of variance reduction by studying its properties with general adaptive mirror descent algorithms in nonsmooth nonconvex finite-sum optimization problems. We propose a simple yet generalized framework for variance reduced adaptive mirror descent algorithms named SVRAMD and provide its convergence analysis in both the nonsmooth nonconvex problem and the P-L conditioned problem. We prove that variance reduction reduces the SFO complexity of adaptive mirror descent algorithms and thus accelerates their convergence. In particular, our general theory implies that variance reduction can be applied to algorithms using time-varying step sizes and self-adaptive algorithms such as AdaGrad and RMSProp. Moreover, the convergence rates of SVRAMD recover the best existing rates of non-adaptive variance reduced mirror descent algorithms without complicated algorithmic components. Extensive experiments in deep learning validate our theoretical findings.

On convergence of numerical solutions for the compressible MHD system with weakly divergence-free magnetic field

Abstract We study a general convergence theory for the analysis of numerical solutions to a magnetohydrodynamic system describing the time evolution of compressible, viscous, electrically conducting fluids in space dimension $d$$(=2,3)$. First, we introduce the concept of dissipative weak (DW) solutions and prove the weak–strong uniqueness property for DW solutions, meaning a DW solution coincides with a classical solution emanating from the same initial data on the lifespan of the latter. Next, we introduce the concept of consistent approximations and prove the convergence of consistent approximations towards the DW solution, as well as the classical solution. Interpreting the consistent approximation as the energy stability and consistency of numerical solutions, we have built a nonlinear variant of the celebrated Lax equivalence theorem. Finally, as an application of this theory, we show the convergence analysis of two numerical methods.

A new stochastic gradient descent possibilistic clustering algorithm

Several well known clustering algorithms have their own online counterparts, in order to deal effectively with the big data issue, as well as with the case where the data become available in a streaming fashion. However, very few of them follow the stochastic gradient descent philosophy, despite the fact that the latter enjoys certain practical advantages (such as the possibility of (a) running faster than their batch processing counterparts and (b) escaping from local minima of the associated cost function), while, in addition, strong theoretical convergence results have been established for it. In this paper a novel stochastic gradient descent possibilistic clustering algorithm, called O- PCM 2 is introduced. The algorithm is presented in detail and it is rigorously proved that the gradient of the associated cost function tends to zero in the L 2 sense, based on general convergence results established for the family of the stochastic gradient descent algorithms. Furthermore, an additional discussion is provided on the nature of the points where the algorithm may converge. Finally, the performance of the proposed algorithm is tested against other related algorithms, on the basis of both synthetic and real data sets.

New iterative conjugate gradient method for nonlinear unconstrained optimization

Conjugate gradient methods (CG) are an important class of methods for solving unconstrained optimization problems, especially for large-scale problems. Recently, they have been much studied. In this paper, we propose a new conjugate gradient method for unconstrained optimization. This method is a convex combination of Fletcher and Reeves (abbreviated FR), Polak–Ribiere–Polyak (abbreviated PRP) and Dai and Yuan (abbreviated DY) methods. The new conjugate gradient methods with the Wolfe line search is shown to ensure the descent property of each search direction. Some general convergence results are also established for this method. The numerical experiments are done to test the efficiency of the proposed method, which confirms its promising potentials.

Limit sets in global multiobjective optimization

ABSTRACT Inspired by the recently introduced branch-and-bound method for continuous multiobjective optimization problems from G. Eichfelder, P. Kirst, L. Meng, O. Stein [A general branch-and-bound framework for continuous global multiobjective optimization. J Glob Optim. 2021;80:195–227], we study for a general class of branch-and-bound methods in which sense the generated terminal enclosure and the terminal provisional nondominated set approximate the nondominated set when the termination accuracy is driven to zero. Our convergence analysis of the enclosures relies on constructions from the above paper, but is self-contained and also covers the mixed-integer case. The analysis for the provisional nondominated set is based on general convergence properties of the epsilon-nondominated set, and hence it is also applicable to other algorithms which generate such points. Furthermore, we discuss post-processing steps for the terminal enclosure and provide numerical illustrations for the cases of two and three objective functions.

The gradient discretisation method for the chemical reactions of biochemical systems

PurposeThe main purpose of this paper is to introduce the gradient discretisation method (GDM) to a system of reaction diffusion equations subject to non-homogeneous Dirichlet boundary conditions. Then, the authors show that the GDM provides a comprehensive convergence analysis of several numerical methods for the considered model. The convergence is established without non-physical regularity assumptions on the solutions.Design/methodology/approachIn this paper, the authors use the GDM to discretise a system of reaction diffusion equations with non-homogeneous Dirichlet boundary conditions.FindingsThe authors provide a generic convergence analysis of a system of reaction diffusion equations. The authors introduce a specific example of numerical scheme that fits in the gradient discretisation method. The authors conduct a numerical test to measure the efficiency of the proposed method.Originality/valueThis work provides a unified convergence analysis of several numerical methods for a system of reaction diffusion equations. The generic convergence is proved under the classical assumptions on the solutions.

Finite-sum smooth optimization with SARAH

We introduce NC-SARAH for non-convex optimization as a practical modified version of the original SARAH algorithm that was developed for convex optimization. NC-SARAH is the first to achieve two crucial performance properties at the same time—allowing flexible minibatch sizes and large step sizes to achieve fast convergence in practice as verified by experiments. NC-SARAH has a close to optimal asymptotic convergence rate equal to existing prior variants of SARAH called SPIDER and SpiderBoost that either use an order of magnitude smaller step size or a fixed minibatch size. For convex optimization, we propose SARAH++ with sublinear convergence for general convex and linear convergence for strongly convex problems; and we provide a practical version for which numerical experiments on various datasets show an improved performance.

Sharpened dynamics alternative and its C1-robustness for strongly monotone discrete dynamical systems

For strongly monotone dynamical systems, the dynamics alternative for smooth discrete-time systems turns out to be a perfect analogy of the celebrated Hirsch's limit-set dichotomy for continuous-time semiflows. In this paper, we first present a sharpened dynamics alternative for C1-smooth strongly monotone discrete-time dissipative system {F0n}n∈N (with an attractor A), which concludes that there is a positive integer m such that any orbit is either manifestly unstable; or asymptotic to a linearly stable cycle whose minimal period is bounded by m. Furthermore, we show the C1-robustness of the sharpened dynamics alternative, that is, for any C1-perturbed system {Fϵn}n∈N (Fϵ not necessarily monotone), any orbit initiated nearby A will admit the sharpened dynamics alternative with the same m. The improved generic convergence to cycles for the C1-system {F0n}n∈N, as well as for the perturbed system {Fϵn}n∈N, is thus obtained as by-products of the sharpened dynamics alternative and its C1-robustness. The results are applied to nonlocal C1-perturbations of a time-periodic parabolic equations and give typical convergence to periodic solutions whose minimal periods are uniformly bounded.

Worth the wait: How South Africa's renewable energy auctions perform compared to Europe's leading countries

Auctions for renewable energy support allocation are on a triumphant global advance. We compare performance of renewable energy auctions in terms of effectiveness (realisation rate and period) and efficiency (price outcomes) from two world regions: South Africa and European countries (Germany, France, the Netherlands). We develop an assessment framework for compliance incentivisation in auction design, covering qualification criteria, penalties, and compliance monitoring. We find that a 100% realisation rate in South Africa is connected to a strong compliance package, while European countries have moderate realisation rates (23%–87%) with relatively lenient compliance packages. We observe that realisation periods are correlated with project size rather than granted realisation period. Although South African projects generally have longer realisation periods (34 months for PV and 40 months for onshore wind) than European projects (16.5–30 months for PV and 19–25 months for onshore wind), they perform comparably considering that they are 10–13 times larger. Comparing average auction prices with equivalent technology-specific LCOE estimates, we find a general convergence towards the global average, with South Africa having the sharpest price decline (75% PV and 54% onshore wind), albeit starting from the highest level. Our findings, especially on importance of compliance incentivisation and weak impact of granted realisation periods, are valid across world regions and can support policymakers everywhere in designing effective and efficient renewable energy auctions.

Coupling Parareal with Optimized Schwarz Waveform Relaxation for Parabolic Problems

We propose and analyse a parallel method, both in the time and space directions, that couples the Parareal algorithm with the Optimized Schwarz waveform relaxation (OSWR) method, with only few OSWR iterations in the fine propagator and with a simple coarse propagator deduced from the Backward Euler method. The analysis of this coupled method is presented for a one-dimensional advection-reaction-diffusion equation. We prove a general convergence result for this method via energy estimates. Numerical results for two-dimensional advection-diffusion problems and for a diffusion equation with strong heterogeneities are presented, to illustrate the performance of the coupled Parareal-OSWR algorithm.

A general approach to the study of the convergence of Picard iteration with an application to Halley's method for multiple zeros of analytic functions

In this paper, we define a new wide class of iteration functions and then we use it to prove a general convergence theorem that provides exact domain of initial approximations to guarantee the high Q-order of convergence of Picard iteration generated by this class of functions. As an application of this theorem, we prove some local convergence theorems about the famous Halley's method for simple and multiple zeros of analytic functions. All obtained results are supplied with a priori and a posteriori error estimates as well as with assessments of the asymptotic error constants.

Convergence of Output per Worker in G7 Countries: Evidence from the Club Convergence Test

In this study, it is aimed to present the club convergence hypothesis within the framework of per capita output indicator by using annual data for G7 countries and 1950-2018 period. For this purpose, using convergence test proposed by Phillips and Sul (2007), it was analyzed whether the trade openness shows convergence clubs or not According to Phillips and Sul (2007) analysis, there is no general convergence club for all countries and different convergence clubs. Phillips and Sul (2007) analysis revealed that 2 per capita out convergence clubs and 1 divergence club.