Composite Convex Functions Research Articles

Novel Mean-Type Inequalities via Generalized Riemann-Type Fractional Integral for Composite Convex Functions: Some Special Examples

This study deals with a novel class of mean-type inequalities by employing fractional calculus and convexity theory. The high correlation between symmetry and convexity increases its significance. In this paper, we first establish an identity that is crucial in investigating fractional mean inequalities. Then, we establish the main results involving the error estimation of the Hermite–Hadamard inequality for composite convex functions via a generalized Riemann-type fractional integral. Such results are verified by choosing certain composite functions. These results give well-known examples in special cases. The main consequences can generalize many known inequalities that exist in other studies.

On the novel Hermite-Hadamard inequalities for composite inverse functions

The goal of this research is to discover some identities in the general form of the sum of left and right-sided weighted fractional integrals of a function concerning to another function. Using composite convex functions, several fractional Hermite-Hadamard inequalities are derived. The veracity of the inequalities established is demonstrated by drawing graphs of such relationships. Furthermore, our findings generalize a number of previously published outcomes. These findings will aid in the study of fractional differential equations and fractional boundary value problems with unique solutions.

Hermite-Hadamard Type Inequalities for Composite Log-Convex Functions

© 2020 by authors, all rights reserved. Hermite-Hadamard type inequalities related to convex functions are widely being studied in functional analysis. Researchers have refined the convex functions as quasi-convex, h-convex, log-convex, m-convex, (α,m)-convex and many more. Subsequently, the Hermite-Hadamard type inequalities have been obtained for these refined convex functions. In this paper, we firstly review the Hermite-Hadamard type inequality for both convex functions and log-convex functions. Then, the definition of composite convex function and the Hermite-Hadamard type inequalities for composite convex functions are also reviewed. Motivated by these works, we then make some refinement to obtain the definition of composite log-convex functions, namely composite-ϕ−1 log-convex function. Some examples related to this definition such as GG-convexity and HG-convexity are given. We also define k-composite log-convexity and k-composite-ϕ−1 log-convexity. We then prove a lemma and obtain some Hermite-Hadamard type inequalities for composite log-convex functions. Two corollaries are also proved using the theorem obtained; the first one by applying the exponential function and the second one by applying the properties of k-composite log-convexity. Also, an application for GG-convex functions is given. In this application, we compare the inequalities obtained from this paper with the inequalities obtained in the previous studies. The inequalities can be applied in calculating geometric means in statistics and other fields.

Accelerated Regularized Newton Methods for Minimizing Composite Convex Functions

In this paper, we study accelerated regularized Newton methods for minimizing objectives formed as a sum of two functions: one is convex and twice differentiable with Hölder-continuous Hessian, and the other is a simple closed convex function. For the case in which the Hölder parameter $\nu\in [0,1]$ is known, we propose methods that take at most $\mathcal{O}\big({1 \over \epsilon^{1/(2+\nu)}}\big)$ iterations to reduce the functional residual below a given precision $\epsilon > 0$. For the general case, in which the $\nu$ is not known, we propose a universal method that ensures the same precision in at most $\mathcal{O}\big({1 \over \epsilon^{2/[3(1+\nu)]}}\big)$ iterations without using $\nu$ explicitly in the scheme.

Adaptive Restart of the Optimized Gradient Method for Convex Optimization.

First-order methods with momentum such as Nesterov's fast gradient method are very useful for convex optimization problems, but can exhibit undesirable oscillations yielding slow convergence for some applications. An adaptive restarting scheme can improve the convergence rate of the fast gradient methd, when the parameter of a strongly convex cost function is unknown or when the iterates of the algorithm enter a locally well-conditioned region. Recently, we introduced an optimized gradient method, a first-order algorithm that has an inexpensive per-iteration computational cost similar to that of the fast gradient method, yet has a worst-case cost function convergence bound that is twice smaller than that of the fast gradient method and that is optimal for large-dimensional smooth convex problems. Building upon the success of accelerating the fast gradient method using adaptive restart, this paper investigates similar heuristic acceleration of the optimized gradient method. We first derive new step coefficients of the optimized gradient method for a strongly convex quadratic problem with known function parameters, yielding a convergence rate that is faster than that of the analogous version of the fast gradient method. We then provide a heuristic analysis and numerical experiments that illustrate that adaptive restart can accelerate the convergence of the optimized gradient method. Numerical results also illustrate that adaptive restart is helpful for a proximal version of the optimized gradient method for nonsmooth composite convex functions.

ANOTHER LOOK AT THE FAST ITERATIVE SHRINKAGE/THRESHOLDING ALGORITHM (FISTA).

This paper provides a new way of developing the "Fast Iterative Shrinkage/Thresholding Algorithm (FISTA)" [3] that is widely used for minimizing composite convex functions with a nonsmooth term such as the ℓ1 regularizer. In particular, this paper shows that FISTA corresponds to an optimized approach to accelerating the proximal gradient method with respect to a worst-case bound of the cost function. This paper then proposes a new algorithm that is derived by instead optimizing the step coefficients of the proximal gradient method with respect to a worst-case bound of the composite gradient mapping. The proof is based on the worst-case analysis called Performance Estimation Problem in [11].

Stochastic Gradient Made Stable: A Manifold Propagation Approach for Large-Scale Optimization

Stochastic gradient descent (SGD) holds as a classical method to build large scale machine learning models over big data. A stochastic gradient is typically calculated from a limited number of samples (known as mini-batch), which potentially incurs a high variance and causes the estimated parameters to bounce around the optimal solution. To improve the stability of stochastic gradient, recent years have witnessed the proposal of several semi-stochastic gradient descent algorithms, which distinguish themselves from standard SGD by incorporating global information into gradient computation. In this paper, we contribute a novel stratified semi-stochastic gradient descent (S3GD) algorithm to this nascent research area, accelerating the optimization of a large family of composite convex functions. Though theoretically converging faster, prior semi-stochastic algorithms are found to suffer from high iteration complexity, which makes them even slower than SGD in practice on many datasets. In our proposed S3GD, the semi-stochastic gradient is calculated based on efficient manifold propagation, which can be numerically accomplished by sparse matrix multiplications. This way S3GD is able to generate a highly-accurate estimate of the exact gradient from each mini-batch with largely-reduced computational complexity. Theoretic analysis reveals that the proposed S3GD elegantly balances the geometric algorithmic convergence rate against the space and time complexities during the optimization. The efficacy of S3GD is also experimentally corroborated on several large-scale benchmark datasets.

A splitting primal-dual proximity algorithm for solving composite optimization problems

Our work considers the optimization of the sum of a non-smooth convex function and a finite family of composite convex functions, each one of which is composed of a convex function and a bounded linear operator. This type of problem is associated with many interesting challenges encountered in the image restoration and image reconstruction fields. We developed a splitting primal-dual proximity algorithm to solve this problem. Further, we propose a preconditioned method, of which the iterative parameters are obtained without the need to know some particular operator norm in advance. Theoretical convergence theorems are presented. We then apply the proposed methods to solve a total variation regularization model, in which the L2 data error function is added to the L1 data error function. The main advantageous feature of this model is its capability to combine different loss functions. The numerical results obtained for computed tomography (CT) image reconstruction demonstrated the ability of the proposed algorithm to reconstruct an image with few and sparse projection views while maintaining the image quality.

The stable duality of DC programs for composite convex functions

In this paper, we consider a composite DC optimization problem with a cone-convex system in locally convex Hausdorff topological vector spaces. By using the properties of the epigraph of the conjugate functions, some necessary and sufficient conditions which characterize the strong Fenchel-Lagrange duality and the stable strong Fenchel-Lagrange duality are given. We apply the results obtained to study the minmax optimization problem and $l_1$ penalty problem.

The Stable Farkas Lemma for composite convex functions in infinite dimensional spaces

In this paper, we consider a general composite convex optimization problem with a cone-convex system in locally convex Hausdorff topological vector spaces. Some Fenchel conjugate transforms for the composite convex functions are derived to obtain the equivalent condition of the Stable Farkas Lemma, which is formulated by using the epigraph of the conjugates for the convex functions involved and turns out to be weaker than the classic Slater condition. Moreover, we get some necessary and sufficient conditions for stable duality results of the composite convex functions and present an example to illustrate that the monotonic increasing property of the outer convex function in the objective function is essential. Our main results in this paper develop some recently results.

The Toland-Fenchel-Lagrange duality of DC programs for composite convex functions

In this paper, by virtue of the epigraph technique, we construct a new kind of closedness-type constraintqualification, which is the sufficient and necessary condition to guarantee thestrong duality between a cone constraint composite optimization problem and its dual problem holds. Under this closedness-type constraint qualification condition, we obtain a formula of subdifferential for composite functions and study a cone constraint composite DC optimization problem in locally convex Hausdorff topological vector spaces.

Proximity for sums of composite functions

We propose an algorithm for computing the proximity operator of a sum of composite convex functions in Hilbert spaces and investigate its asymptotic behavior. Applications to best approximation and image recovery are described.

A trust region algorithm for minimization of locally Lipschitzian functions

The classical trust region algorithm for smooth nonlinear programs is extended to the nonsmooth case where the objective function is only locally Lipschitzian. At each iteration, an objective function that carries both first and second order information is minimized over a trust region. The term that carries the first order information is an iteration function that may not explicitly depend on subgradients or directional derivatives. We prove that the algorithm is globally convergent. This convergence result extends the result of Powell for minimization of smooth functions, the result of Yuan for minimization of composite convex functions, and the result of Dennis, Li and Tapia for minimization of regular functions. In addition, compared with the recent model of Pang, Han and Rangaraj for minimization of locally Lipschitzian functions using a line search, this algorithm has the same convergence property without assuming positive definiteness and uniform boundedness of the second order term. Applications of the algorithm to various nonsmooth optimization problems are discussed.