Step-size Rule Research Articles

Modified subgradient extragradient method to solve variational inequalities

In this paper, we introduce a new method to solve pseudomonotone variational inequalities with the Lipschitz condition in a real Hilbert space. This problem is a general mathematical problem in the sense that it unifies a number of the mathematical problems as a particular case, such as the optimization problems, the equilibrium problems, the fixed point problems, the saddle point problems and Nash equilibrium point problems. The new method is constructed around two methods: the extragradient method and the inertial method. The proposed method uses a new stepsize rule based on local operator information rather than its Lipschitz constant or any other line search method. The proposed method does not require any knowledge of the Lipschitz constant of an operator. The strong convergence of the proposed method is well-established. Finally, we conduct a number of numerical experiments to determine the performance and superiority of the proposed method.

Strong convergence inertial projection algorithm with self-adaptive step size rule for pseudomonotone variational inequalities in Hilbert spaces

Abstract In this paper, we introduce a new algorithm for solving pseudomonotone variational inequalities with a Lipschitz-type condition in a real Hilbert space. The algorithm is constructed around two algorithms: the subgradient extragradient algorithm and the inertial algorithm. The proposed algorithm uses a new step size rule based on local operator information rather than its Lipschitz constant or any other line search scheme and functions without any knowledge of the Lipschitz constant of an operator. The strong convergence of the algorithm is provided. To determine the computational performance of our algorithm, some numerical results are presented.

Two generalized non-monotone explicit strongly convergent extragradient methods for solving pseudomonotone equilibrium problems and applications

The main objective of this paper is to introduce two new proximal-like methods to solve the equilibrium problem in a real Hilbert space. The equilibrium problem is a general mathematical problem that unites several useful mathematical problems, including optimization problems, variational inequalities, fixed-point problems, saddle point problems, complementary problems, and Nash equilibrium problems. Both new methods are analogous to the well-known extragradient method, which has been used in the literature to solve variational inequality problems. The proposed methods make use of a non-monotone variable step size rule that is revised for each iteration and is determined mainly by previous iterations. The advantage of these methods is that they can be used without prior knowledge of Lipschitz-type constants or any line-search method. By allowing for some mild condition, the strong convergence of both methods is established. Numerical studies are presented to demonstrate the computational behavior of new methods and to compare them to other existing methods.

An explicit subgradient extragradient algorithm with self-adaptive stepsize for pseudomonotone equilibrium problems in Banach spaces

In this paper, we introduce an explicit subgradient extragradient algorithm for solving equilibrium problem with a bifunction satisfying pseudomonotone and Lipschitz-like condition in a 2-uniformly convex and uniformly smooth Banach space. We also defined a new self-adaptive stepsize rule and prove a convergence result for solving the equilibrium problem without any prior estimate of the Lipschitz-like constants of the bifunction. Furthermore, we provide some numerical examples to illustrate the efficiency and accuracy of the proposed algorithm. This result improves and extends many recent results in this direction in the literature.

Iterative regularization methods for solving equilibrium problems

The paper concerns with two new schemes for approximating solutions of equilibrium problems involving monotone and Lipschitz-type bifunctions in Hilbert spaces. We describe how to incorporate the two-step proximal-like method (modified extragradient method) with the regularization technique and then we propose two iterative regularization algorithms for solving equilibrium problems. Unlike the viscosity-like methods, we establish the strong convergence of the new algorithms based on the regularization. The first algorithm is designed to work with the prior knowledge of Lipschitz-type constants of bifunction while the second one, with a simple stepsize rule, is done without this requirement. In order to illustrate the effectiveness and the convergence of the algorithms, we provide several numerical experiments in comparisons with other well known algorithms, which show that our new algorithms are effective for solving equilibrium problems.wi

Iterative regularization methods with new stepsize rules for solving variational inclusions

The paper concerns with three iterative regularization methods for solving a variational inclusion problem of the sum of two operators, the one is maximally monotone and the another is monotone and Lipschitz continuous, in a Hilbert space. We first describe how to incorporate regularization terms in the methods of forward-backward types, and then establish the strong convergence of the resulting methods. With several new stepsize rules considered, the methods can work with or without knowing previously the Lipschitz constant of cost operator. Unlike known hybrid methods, the strong convergence of the proposed methods comes from the regularization technique. Several applications to signal recovery problems and optimal control problems together with numerical experiments are also presented in this paper. Our numerical results have illustrated the fast convergence and computational effectiveness of the new methods over known hybrid methods.

Modified proximal-like extragradient methods for two classes of equilibrium problems in Hilbert spaces with applications

The objective of this study is to introduce two new extragradient methods to solve equilibrium problems involving two distinct classes of bifunction. Based on the pseudomonotonicity hypothesis and the specific Lipschitz-type cost bifunction condition, we have shown a weak convergence theorem for the first proposed method. The bifunction is strongly pseudomonotone in the second method, but the step-size rule does not depend on the strongly pseudomonotone constant and Lipschitz-type constants. In contrast, the first convergence result, we prove a strong convergence theorem with the use of a particular class of variable step-size rule sequences. To confirm the validity of the proposed convergence results, we examine two well-known Nash–Cournot equilibrium models for the numerical experiment and show that the competitive advantage of our proposed methods is based on time of execution and number of iterations.

Accelerating mini-batch SARAH by step size rules

StochAstic Recursive grAdient algoritHm (SARAH), originally proposed for convex optimization and also proven to be effective for general nonconvex optimization, has received great attention because of its simple recursive framework for updating stochastic gradient estimates. The performance of SARAH significantly depends on the choice of the step size sequence. However, SARAH and its variants often manually select a best-tuned step size, which is time consuming in practice. Motivated by this gap, we propose a variant of the Barzilai-Borwein (BB) method, referred to as the Random Barzilai-Borwein (RBB) method, to determine the step size for SARAH in the mini-batch setting, leading to a new SARAH method: MB-SARAH-RBB. We prove that MB-SARAH-RBB converges linearly in expectation for strongly convex objective functions. Moreover, we analyze the gradient complexity of MB-SARAH-RBB and show that it is better than the original method. To further confirm the efficacy of the RBB method, we propose the MB-SARAH+-RBB method, by incorporating it into the MB-SARAH + method. Numerical experiments on standard data sets indicate that our proposed methods outperform or match state-of-the-art algorithms.

Approximation Results for Variational Inequalities Involving Pseudomonotone Bifunction in Real Hilbert Spaces

In this paper, we introduce two novel extragradient-like methods to solve variational inequalities in a real Hilbert space. The variational inequality problem is a general mathematical problem in the sense that it unifies several mathematical models, such as optimization problems, Nash equilibrium models, fixed point problems, and saddle point problems. The designed methods are analogous to the two-step extragradient method that is used to solve variational inequality problems in real Hilbert spaces that have been previously established. The proposed iterative methods use a specific type of step size rule based on local operator information rather than its Lipschitz constant or any other line search procedure. Under mild conditions, such as the Lipschitz continuity and monotonicity of a bi-function (including pseudo-monotonicity), strong convergence results of the described methods are established. Finally, we provide many numerical experiments to demonstrate the performance and superiority of the designed methods.

Regularization extragradient methods for equilibrium programming in Hilbert spaces

ABSTRACT The paper introduces two new numerical methods for solving a variational inequality problem whose constraint set is expressed as the solution set of a monotone and Lipschitz-type equilibrium problem in a Hilbert space. We present how to combine regularization terms in an extragradient method and prove that the iterative sequences generated by the resulting methods converge strongly to a solution of equilibrium problem which solves the associated variational inequality problem. Theorems of strong convergence are analysed which are based on the incorporated Tikhonov regularization method for equilibrium problems. The first method is designed in the case where the Lipschitz-type constants of bifunction are known. While the second method can be implemented more easily without the prior knownledge of Lipschitz-type constants. The reason is that the second method have used a new stepsize rule whose computation is simple and easy to check at each step. Several numerical experiments are performed and they have demonstrated the effectiveness and the fast convergence of the new methods over existing methods.

A new weak convergence non-monotonic self-adaptive iterative scheme for solving equilibrium problems

<abstract> A number of methods have been proposed to solve the equilibrium problems, one of which is an extragradient method that is particularly interesting and effective. In this paper, we introduce a modified subgradient extragradient method to solve the equilibrium problems in a real Hilbert space. The proposed method uses a non-monotonic step size rule based on local bi-function information instead of its Lipschitz-type constant or other line search method and is capable of solving pseudo-monotone equilibrium problems. Our method only needs to solve a strongly convex programming problem per iteration. Applications of the designed algorithm are presented in order to solve fixed-point problems and variational inequalities. Finally, several computational experiments are studied to confirm the effectiveness of the proposed method. The results of our study include many similar literature studies and detailed numerical studies also show their potential usefulness. </abstract>

Accelerated non-monotonic explicit proximal-type method for solving equilibrium programming with convex constraints and its applications

&lt;abstract&gt;&lt;p&gt;The main objective of this study is to introduce a new two-step proximal-type method to solve equilibrium problems in a real Hilbert space. This problem is a general mathematical model and includes a number of mathematical problems as a special case, such as optimization problems, variational inequalities, fixed point problems, saddle time problems and Nash equilibrium point problems. A new method is analogous to the famous two-step extragradient method that was used to solve variational inequality problems in a real Hilbert space established previously. The proposed iterative method uses an inertial scheme and a new non-monotone stepsize rule based on local bifunctional values rather than any line search method. A strong convergence theorem for the constructed method is proven by letting mild conditions on a bifunction. These results are being used to solve fixed point problems as well as variational inequalities. Finally, we considered two test problems, and the computational performance was presented to show the performance and efficiency of the proposed method.&lt;/p&gt;&lt;/abstract&gt;

Two Bregman projection methods for solving variational inequalities

ABSTRACT Using Bregman distances, we propose two extragradient-like methods for solving variational inequality problems with Lipschitz cost operators in a Hilbert space. Weak and strong convergence theorems for our algorithms are established when the cost operator is either monotone or pseudomonotone. The variable stepsizes are generated by the algorithms at each iterative stage without any line search procedure. Our stepsize rule allows the algorithms to be easily implemented without prior knowledge of the Lipschitz constant of the cost operator. We also provide several numerical findings in order to illustrate our theoretical results.

Fast and Simple Bregman Projection Methods for Solving Variational Inequalities and Related Problems in Banach Spaces

In this paper, we study the problem of finding a common solution to variational inequality and fixed point problems for a countable family of Bregman weak relatively nonexpansive mappings in real reflexive Banach spaces. Two inertial-type algorithms with adaptive step size rules for solving the problem are presented and their strong convergence theorems are established. The usage of the Bregman distances and the Armijo line search technique (which avoids the need to know a priori the Lipschitz constant of the involved operators), enable great flexibility of the proposed scheme, and besides their theoretical extensions, it might also have a practical potential.

A modified self‐adaptive extragradient method for pseudomonotone equilibrium problem in a real Hilbert space with applications

In this paper, we consider an improvement of the extragradient method to figure out the numerical solution for pseudomonotone equilibrium problems in arbitrary real Hilbert space. A new method is proposed with an inertial scheme and a self adaptive step size rule that is revised on each iteration based on the previous three iterations. The weak convergence of the method is proved by assuming standard cost bifunction assumptions. We also consider the application of our results to solve different kinds of variational inequality problems and a particular class of fixed point problems. For a numerical part, we study the well‐known Nash‐Cournot equilibrium model and other test problems to support our well‐established convergence results and to ensure that our proposed method has a competitive edge over CPU time and a number of iterations.

An Edge Storage Acceleration Service for Collaborative Mobile Devices

Fueled by the advances in the Internet of Things, and the growing capacity of smart mobile devices at the edge of the Internet, we have witnessed a growing trend in research and development for edge computing and edge storage, which extends the abilities of single mobile device on the edge through on-demand collaboration among multiple geographically distributed mobile devices. In this article, we address several technical challenges that are unique for collaborative storage at the edge due to the unique characteristics of mobile devices. First, we formalize the collaborative storage problem as an optimization problem. Second, we design an Acceleration Algorithm for Collaborative Storage, called A2CS, based on the architecture of Alternating Direction Method of Multipliers (ADMM). Specifically, we use the Nesterov’s Acceleration strategy and the step size rules in the process of updating variables and determining the optimal speed of convergence. We develop a novel collaborative storage policy in order to guide the whole lifecycle of collaborative storage. Finally, we conduct a series of experiments for acceleration performance analysis and validation. We show that A2CS delivers a better convergence performance with different step size rules, compared with two existing approaches: the ADMM baseline and the ADMM-OR (ADMM with Over-Relaxation), achieving the acceleration percentage by at least 25.33 percent and at most 64.01 percent. In addition, by conducting the utility performance comparison analysis with the existing Average Distribution Strategy (ADS) and the existing Distance Preferred Distribution Strategy (DPDS), we show the advantage of A2CS over both ADS and DPDS with respect to the total utility and energy consumption.

On Halpern-type sequences with applications in variational inequality problems

We introduce a Halpern-type sequence and give a necessary and sufficient condition for a strong convergence of this sequence. In particular, we obtain two strong convergence theorems for approximation of a fixed point of nonexpansive mappings or of quasi-nonexpansive ones. We also apply our result for various iterative methods in variational inequality problem. For the Lipschitz continuous mappings, we deal with the extragradient method of Korpelevič, the subgradient extragradient method of Censor et al. and the extragradient of Tseng where the step size rule is priorly or posteriorly chosen. For the non-Lipschitz continuous mappings, we use our results to deduce the convergence results of Shehu and Iyiola and of Thong and Gibali. Our approach allows us to conclude many new results with some new assumptions.

A Gradient Projection Algorithm with a New Stepsize for Nonnegative Sparsity-Constrained Optimization

Nonnegative sparsity-constrained optimization problem arises in many fields, such as the linear compressing sensing problem and the regularized logistic regression cost function. In this paper, we introduce a new stepsize rule and establish a gradient projection algorithm. We also obtain some convergence results under milder conditions.

Strong convergence of new iterative projection methods with regularization for solving monotone variational inequalities in Hilbert spaces

In this paper, we introduce two new numerical methods for solving a variational inequality problem involving a monotone and Lipschitz continuous operator in a Hilbert space. We describe how to incorporate a regularization term depending on a parameter in the projection method and then establish the strong convergence of the resulting iterative regularization projection methods. Unlike known hybrid methods, the strong convergence of the new methods comes from the regularization technique. The first method is designed to work in the case where the Lipschitz constant of cost operator is known, whereas the second one is more easily implemented without this requirement. The reason is because the second method has used a simple computable stepsize rule. The variable stepsizes are generated by the second method at each iteration and based on the previous iterates. These stepsizes are found with only one cheap computation without line‐search procedure. Several numerical experiments are implemented to show the computational effectiveness of the new methods over existing methods.

Weak and strong convergence of inertial Tseng's extragradient algorithms for solving variational inequality problems

In this paper, we introduce two inertial Tseng's extragradient algorithms with the Armijo-like step size rule for solving variational inequality problems involving monotone and Lipschitz continuous operators. Strong and weak convergence theorems are established in Hilbert spaces. Some numerical experiments are provided to illustrate the efficiency and advantage of the proposed algorithms.