General Convex Research Articles

On q-Hermite–Hadamard–Mercer and midpoint-Mercer inequalities for general convex functions with their computational analysis

This paper establishes some new inequalities of Hermite–Hadamard–Mercer type for [Formula: see text]-convex functions in the framework of [Formula: see text]-calculus and classical calculus. Some new [Formula: see text]-midpoint-Mercer type inequalities for the [Formula: see text]-differentiable [Formula: see text]-convex functions are also proved. Moreover, the computational analysis of newly established inequalities for convex and [Formula: see text]-convex functions are given to prove that the bounds of this paper are better than those of the existing ones. It is also shown with mathematical examples that the newly established inequalities are valid for [Formula: see text]-convex functions in [Formula: see text]-calculus.

A Minibatch Stochastic Gradient Descent-Based Learning Metapolicy for Inventory Systems with Myopic Optimal Policy

Stochastic gradient descent (SGD) has proven effective in solving many inventory control problems with demand learning. However, it often faces the pitfall of an infeasible target inventory level that is lower than the current inventory level. Several recent works have been successful in resolving this issue in various inventory systems. However, their techniques are rather sophisticated and difficult to apply to more complicated scenarios, such as multiproduct and multiconstraint inventory systems. In this paper, we address the infeasible target inventory-level issue from a new technical perspective; we propose a novel minibatch SGD-based metapolicy. Our metapolicy is flexible enough to be applied to a general inventory systems framework covering a wide range of inventory management problems with myopic clairvoyant optimal policy. By devising the optimal minibatch scheme, our metapolicy achieves a regret bound of [Formula: see text] for the general convex case and [Formula: see text] for the strongly convex case. To demonstrate the power and flexibility of our metapolicy, we apply it to three important inventory control problems, multiproduct and multiconstraint systems, multiechelon serial systems, and one-warehouse and multistore systems, by carefully designing application-specific subroutines. We also conduct extensive numerical experiments to demonstrate that our metapolicy enjoys competitive regret performance, high computational efficiency, and low variances among a wide range of applications. This paper was accepted by J. George Shanthikumar, data science. Funding: J. Xie and S. Yuan are supported by the National Natural Science Foundation of China (NSFC) [Grant 72331011]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/mnsc.2023.00920 .

Birkhoff attractors of dissipative billiards

Abstract We study the dynamics of dissipative billiard maps within planar convex domains. Such maps have a global attractor. We are interested in the topological and dynamical complexity of the attractor, in terms both of the geometry of the billiard table and of the strength of the dissipation. We focus on the study of an invariant subset of the attractor, the so-called Birkhoff attractor. On the one hand, we show that for a generic convex table with ‘pinched’ curvature, the Birkhoff attractor is a normally contracted manifold when the dissipation is strong. On the other hand, for a mild dissipation, we prove that, generically, the Birkhoff attractor is complicated, both from the topological and the dynamical points of view.

Stabilizing and Accelerating Federated Learning on Heterogeneous Data With Partial Client Participation.

Federated learning (FL) commonly encourages the clients to perform multiple local updates before the global aggregation, thus avoiding frequent model exchanges and relieving the communication bottleneck between the server and clients. Though empirically effective, the negative impact of multiple local updates on the stability of FL is not thoroughly studied, which may result in a globally unstable and slow convergence. Based on sensitivity analysis, we define in this paper a local-update stability index for the general FL, as measured by the maximum inter-client model discrepancy after the multiple local updates that mainly stems from the data heterogeneity. It enables to determine how much the variation of client's models with multiple local updates may influence the global model, and can also be linked with the convergence and generalization. We theoretically derive the proposed local-update stability for current state-of-the-art FL methods, providing possible insight to understanding their motivation and limitation from a new perspective of stability. For example, naively executing the parallel acceleration locally at clients would harm the local-update stability. Motivated by this, we then propose a novel accelerated yet stabilized FL algorithm (named FedANAG) based on the server- and client-level Nesterov accelerated gradient (NAG). In FedANAG, the global and local momenta are elaborately designed and alternatively updated, while the stability of local update is enhanced with help of the global momentum. We prove the convergence of FedANAG for strongly convex, general convex and non-convex settings. We then conduct evaluations on both the synthetic and real-world datasets to first validate our proposed local-update stability. The results further show that across various data heterogeneity and client participation ratios, FedANAG not only accelerates the global convergence by reducing the required number of communication rounds to a target accuracy, but converges to an eventually higher accuracy.

Localization in Boundary-Driven Lattice Models

Several systems display an equilibrium condensation transition, where a finite fraction of a conserved quantity is spatially localized. The presence of two conservation laws may induce the emergence of such transition in an out-of-equilibrium setup, where boundaries are attached to different and subcritical heat baths. We study this phenomenon in a class of stochastic lattice models, where the local energy is a general convex function of the local mass, mass and energy being both globally conserved in the isolated system. We obtain exact results for the nonequilibrium steady state (spatial profiles, mass and energy currents, Onsager coefficients) and we highlight important differences between equilibrium and out-of-equilibrium condensation.

Three Ellipsoids for External Approximation of the Half-Ball

Introduction. Ellipsoids that approximate the half-ball in Rn (n≥2) and their volume is smaller than the volume of the ball can be used to construct algorithms for solving the problem of minimization of a convex (smooth or non-smooth) function, the problem of minimization of a convex function on a sphere, a general problem convex programming, saddle point problems of convex-concave functions and others. The speed of convergence of such algorithms will be determined by the ratio of the volume of the approximating ellipsoid to the volume of the ball. The purpose of the work is to describe properties of three ellipsoids for approximation of a n-dimensional half-ball, the volume reduction factor of which depends only on n - dimension of space. The first is the well-known ellipsoid of minimum volume, which is used in the classical Yudin-Nemirovsky-Shor ellipsoid method. It provides a minimum volume reduction factor and can be used if n≥2. If n=1, then the classical ellipsoid method replaces the dichotomy method. The other two ellipsoids are close to the minimum volume ellipsoid and at large n guarantee volume reduction coefficients close to the minimum. The results. It is shown that when n=1 the first approximate ellipsoid provides a coefficient of reduction of the segment length equal to 2–√2 ≈ 0.5858. The second approximate ellipsoid is new and the volume reduction factor for it is slightly larger than the factor for the first approximate ellipsoid. If n=1, then it provides a coefficient of reduction of the segment length equal to (√5–1)/2 ≈ 0.6180. For three ellipsoids, comparative results are given in terms of volume reduction coefficients (n=1/20) and the number of iterations for solving problems with relative accuracy 1e-10 (n=1/30). Conclusions. A new ellipsoid has been constructed to approximate the n-dimensional half-ball, which is close in volume to the minimal ellipsoid. The volume reduction factor of the proposed ellipsoid is approximated by an asymptotic formula 1–1/(2n)+1/n², which differs little from the formula 1–1/(2n) for an ellipsoid of minimum volume.

On Newton–Cotes Formula-Type Inequalities for Multiplicative Generalized Convex Functions via Riemann–Liouville Fractional Integrals with Applications to Quadrature Formulas and Computational Analysis

In this article, we develop multiplicative fractional versions of Simpson’s and Newton’s formula-type inequalities for differentiable generalized convex functions with the help of established identities. The main motivation for using generalized convex functions lies in their ability to extend results beyond traditional convex functions, encompassing a broader class of functions, and providing optimal approximations for both lower and upper bounds. These inequalities are very useful in finding the error bounds for the numerical integration formulas in multiplicative calculus. Applying these results to the Quadrature formulas demonstrates their practical utility in numerical integration. Furthermore, numerical analysis provides empirical evidence of the effectiveness of the derived findings. It is also demonstrated that the newly proven inequalities extend certain existing results in the literature.

Convergence analysis of iteratively regularized Landweber iteration with uniformly convex constraints in Banach spaces

In Banach spaces, the convergence analysis of iteratively regularized Landweber iteration (IRLI) is recently studied via conditional stability estimates. But the formulation of IRLI does not include general non-smooth convex penalty functionals, which is essential to capture special characteristics of the sought solution. In this paper, we formulate a generalized form of IRLI so that its formulation includes general non-smooth uniformly convex penalty functionals. We study the convergence analysis and derive the convergence rates of the generalized method solely via conditional stability estimates in Banach spaces for both the perturbed and unperturbed data. We also discuss few examples of inverse problems on which our method is applicable.

Weight-adaptive augmented Lagrange multiplier sequential convex programming for nonlinear trajectory optimization

Sequential convex programming has become a prominent research focus in aerospace trajectory optimization due to its high computational efficiency. One critical challenge in sequential convex programming is the infeasibility of subproblems, which significantly impacts its stability. To address this issue, certain constraints must be relaxed using relaxation variables, which are then incorporated into the objective function as a penalty term. However, the choice of penalty weight greatly influences the optimality and convergence of sequential convex programming. Therefore, this paper proposes a weight-adaptive augmented Lagrange multiplier sequential convex programming algorithm to tackle the challenge. First, this paper analyzes the difficulty of determining an appropriate penalty weight from the perspective of its penalty effect. Based on this analysis, an augmented penalty term is introduced, and the update rule for the augmented Lagrange multiplier is formulated using the Karush-Kuhn-Tucker conditions. Subsequently, a method with adaptive weight adjustment is designed based on the change in the penalty term to further improve the algorithm's solving speed. Finally, the proposed method is validated through numerical simulations in a Mars-powered landing problem and a small glide vehicle reentry problem. The results show that the proposed method is 20.59 % and 25.16 % faster compared to the general sequential convex programming algorithm, respectively. Furthermore, the proposed method is demonstrated to be robust to the initial choice of penalty weight.

Distributed event-triggered algorithm for convex optimization with coupled constraints

This paper develops a distributed primal–dual algorithm via an event-triggered mechanism to solve a class of convex optimization problems subject to local set constraints, coupled equality and inequality constraints. Different from some existing distributed algorithms with the diminishing step-sizes, our algorithm uses the constant step-sizes, and is shown to achieve an exact convergence to an optimal solution with an ergodic convergence rate of O(1/k) for general convex objective functions, where k>0 is the iteration number. Based on the event-triggered communication mechanism, the proposed algorithm can effectively reduce the communication cost without sacrificing the convergence rate. Finally, a numerical example is presented to verify the effectiveness of the proposed algorithm.

A novel family of Q1-finite volume element schemes on quadrilateral meshes

A novel family of isoparametric bilinear finite volume element schemes are constructed and analyzed to solve the anisotropic diffusion problems on general convex quadrilateral meshes. These new schemes are obtained by employing a special quadrature rule to approximate the line integrals in classical Q1-finite volume element method. The new quadrature rule is a linear combination of trapezoidal and midpoint rules, and the weights depend on a parameter ωK. The novelty of this work is that, for any fully anisotropic diffusion tensor, we provide some specific ωK to ensure the coercivity result of the proposed schemes on arbitrary parallelogram, quasi-parallelogram, trapezoidal and some general convex quadrilateral meshes. More interesting is that, the parameter ωK can only involves the anisotropic diffusion tensor and the geometry of quadrilateral cell. An optimal H1 error estimate is also proved on quasi-parallelogram meshes. Finally, the theoretical findings are validated by several numerical examples.

Some Classical Inequalities Associated with Generic Identity and Applications

In this paper, we derive a new generic equality for the first-order differentiable functions. Through the utilization of the general identity and convex functions, we produce a family of upper bounds for numerous integral inequalities like Ostrowski’s inequality, trapezoidal inequality, midpoint inequality, Simpson’s inequality, Newton-type inequalities, and several two-point open trapezoidal inequalities. Also, we provide the numerical and visual explanation of our principal findings. Later, we provide some novel applications to the theory of means, special functions, error bounds of composite quadrature schemes, and parametric iterative schemes to find the roots of linear functions. Also, we attain several already known and new bounds for different values of γ and parameter ξ.

Bi-affine scaling iterative method for convex quadratic programming with bound constraints

To solve general convex quadratic programming problems with bound constraints, this paper proposes a new interior point iterative method that is easy to be implemented. The method exhibits a simple and sufficiently smooth search direction, and possesses the characteristics of affine scaling. Under the limited optimal stepsize rule, starting from an arbitrary interior point, any accumulation point of the generated sequence is an optimal solution of the corresponding problem. Furthermore, due to the absence of introducing dual variables and solving equations, the proposed method is more suitable for solving large-scale problems. Preliminary numerical results indicate that the new method has advantages in terms of both efficiency and accuracy.

Accelerated optimization through time-scale analysis of inertial dynamics with asymptotic vanishing and Hessian-driven dampings

ABSTRACT Gradient optimization algorithms can be effectively studied from the perspective of ordinary differential equations (ODEs), where these algorithms are obtained through the temporal discretization of the continuous dynamics. This approach provides a powerful tool to comprehend acceleration phenomena in optimization, thanks to time-scale techniques and Lyapunov analysis. In the context of a Hilbert setting, our study focuses on the rapid optimization properties of inertial dynamics, which combine asymptotically vanishing viscous damping with Hessian-driven damping. These dynamics are derived as high-resolution ODEs from both the Nesterov accelerated gradient method and the Ravine method. For a general differentiable convex function f, choosing the viscous damping coefficient in the form of α / t with 3 $ ]]> α > 3 guarantees an inverse quadratic convergence rate of the values, i.e. o ( 1 / t 2 ) , the weak convergence of trajectories towards optimal solutions, and the fast convergence of gradients towards zero. By carefully scaling the dynamics temporally and based on the tuning of the damping coefficient in front of the Hessian term, we identify the limit dynamic as α becomes large. In particular, we examine the case where the limit dynamic corresponds to the Levenberg-Marquardt regularization of Newton's continuous method. This explanation accounts for the aforementioned fast convergence properties and provides fresh insights into the complexity of these methods, which play a central role in optimization for high-dimensional problems. The interplay between numerical algorithms and continuous-time ODEs plays a fundamental role in our analysis for the design and understanding of accelerated optimization algorithms. Numerical experiments are presented to illustrate and support the theoretical results.

Group inference of high-dimensional single-index models

For the supervised and semi-supervised settings, a group inference method is proposed for regression parameters in high-dimensional semi-parametric single-index models with an unknown random link function. The inference procedure is based on least squares, which can be extended to other general convex loss functions. The proposed test statistics are weighted quadratic forms of the regression parameter estimates, in which the weight could be a non-random matrix or the sample covariance matrix of the covariates. The proposed method could detect dense but weak signals and deal with high correlation of covariates inside the group. A ‘contaminated test statistic’ is established in the semi-supervised regime to decrease the variance. The asymptotic properties of the resulting estimators are established. The finite-sample behaviour of the proposed method is evaluated through extensive simulation studies. Applications to two genomic datasets are provided.

A geometric integration approach to smooth optimization: foundations of the discrete gradient method

Abstract Discrete gradient methods are geometric integration techniques that can preserve the dissipative structure of gradient flows. Due to the monotonic decay of the function values, they are well suited for general convex and nonconvex optimization problems. Both zero- and first-order algorithms can be derived from the discrete gradient method by selecting different discrete gradients. In this paper, we present a thorough analysis of the discrete gradient method for optimization that provides a solid theoretical foundation. We show that the discrete gradient method is well-posed by proving the existence of iterates for any positive time step, as well as uniqueness in some cases, and propose an efficient method for solving the associated discrete gradient equation. Moreover, we establish an $\text{O}(1/k)$ convergence rate for convex objectives and prove linear convergence if instead the Polyak–Łojasiewicz inequality is satisfied. The analysis is carried out for three discrete gradients—the Gonzalez discrete gradient, the mean value discrete gradient, and the Itoh–Abe discrete gradient—as well as for a randomised Itoh–Abe method. Our theoretical results are illustrated with a variety of numerical experiments, and we furthermore demonstrate that the methods are robust with respect to stiffness.

A Feasible Method for General Convex Low-Rank SDP Problems

A Feasible Method for General Convex Low-Rank SDP Problems

An accelerated lyapunov function for Polyak’s Heavy-ball on convex quadratics

AbstractIn 1964, Polyak showed that the Heavy-ball method, the simplest momentum technique, accelerates convergence of strongly-convex problems in the vicinity of the solution. While Nesterov later developed a globally accelerated version, Polyak’s original algorithm remains simpler and more widely used in applications such as deep learning. Despite this popularity, the question of whether Heavy-ball is also globally accelerated or not has not been fully answered yet, and no convincing counterexample has been provided. This is largely due to the difficulty in finding an effective Lyapunov function: indeed, most proofs of Heavy-ball acceleration in the strongly-convex quadratic setting rely on eigenvalue arguments. Our work adopts a different approach: studying momentum through the lens of quadratic invariants of simple harmonic oscillators. By utilizing the modified Hamiltonian of Störmer-Verlet integrators, we are able to construct a Lyapunov function that demonstrates an $$O(1/k^2)$$ O ( 1 / k 2 ) rate for Heavy-ball in the case of convex quadratic problems. Our novel proof technique, though restricted to linear regression, is found to work well empirically also on non-quadratic convex problems, and thus provides insights on the structure of Lyapunov functions to be used in the general convex case. As such, our paper makes a promising first step towards potentially proving the acceleration of Polyak’s momentum method and we hope it inspires further research around this question.

Convex optimization via inertial algorithms with vanishing Tikhonov regularization: fast convergence to the minimum norm solution

In a Hilbertian framework, for the minimization of a general convex differentiable function f, we introduce new inertial dynamics and algorithms that generate trajectories and iterates that converge fastly towards the minimizer of f with minimum norm. Our study is based on the non-autonomous version of the Polyak heavy ball method, which, at time t, is associated with the strongly convex function obtained by adding to f a Tikhonov regularization term with vanishing coefficient ε(t)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varepsilon (t)$$\\end{document}. In this dynamic, the damping coefficient is proportional to the square root of the Tikhonov regularization parameter ε(t)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varepsilon (t)$$\\end{document}. By adjusting the speed of convergence of ε(t)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varepsilon (t)$$\\end{document} towards zero, we will obtain both rapid convergence towards the infimal value of f, and the strong convergence of the trajectories towards the element of minimum norm of the set of minimizers of f. In particular, we obtain an improved version of the dynamic of Su-Boyd-Candès for the accelerated gradient method of Nesterov. This study naturally leads to corresponding first-order algorithms obtained by temporal discretization. In the case of a proper lower semicontinuous and convex function f, we study the proximal algorithms in detail, and show that they benefit from similar properties.

A relaxation viewpoint to Unbalanced Optimal Transport: Duality, optimality and Monge formulation

We present a general convex relaxation approach to study a wide class of Unbalanced Optimal Transport problems for finite non-negative measures with possibly different masses. These are obtained as the lower semicontinuous and convex envelope of a cost for non-negative Dirac masses. New general primal-dual formulations, optimality conditions, and metric-topological properties are carefully studied and discussed.