Subgradient Method Research Articles

Scheduling of steelmaking-continuous casting process with different processing routes using effective surrogate Lagrangian relaxation approach and improved concave–convex procedure

This paper studies a steelmaking-continuous casting scheduling problem with different processing routes. We model this problem as a mixed-integer nonlinear programming problem. Next, Lagrangian relaxation approach is introduced to solve this problem by relaxing the coupling constraints. Due to the nonseparability in Lagrangian functions, we design an improved concave–convex procedure to decompose the Lagrangian relaxation problem into three tractable subproblems and analyse the convergence of the improved concave–convex procedure under some assumptions. Furthermore, we present an effective surrogate subgradient algorithm with global convergence to solve the Lagrangian dual problem. Lastly, computational experiments on the practical production data show the effectiveness of the proposed surrogate subgradient method for solving this steelmaking-continuous casting scheduling problem.

A subgradient proximal method for solving a class of monotone multivalued variational inequality problems

It is well known that the algorithms with using a proximal operator can be not convergent for monotone variational inequality problems in the general case. Malitsky (Optim. Methods Softw. 33 (1) 140–164, ??) proposed a proximal extrapolated gradient algorithm ensuring convergence for the problems, where the constraints are a finite-dimensional vector space. Based on this proximal extrapolated gradient techniques, we propose a new subgradient proximal iteration method for solving monotone multivalued variational inequality problems with the closed convex constraint. At each iteration, two strongly convex subprograms are required to solve separately by using proximal operators. Then, the algorithm is convergent for monotone and Lipschitz continuous cost mapping. We also use the proposed algorithm to solve a jointly constrained Cournot-Nash equilibirum model. Some numerical experiment and comparison results for convex nonlinear programming confirm efficiency of the proposed modification.

Decentralized online convex optimization based on signs of relative states

In this paper, we study a class of decentralized online convex optimization problems with time-varying loss functions over multi-agent networks. We propose a decentralized online subgradient method by using only the signs of the relative states of neighbors, which considerably reduces the sensing and communication requirements for the agents. We show that, despite the loss of information, the proposed algorithm can still achieve O(T) regret bound (T is the time horizon), which matches that of the standard distributed online subgradient method with exact relative state information. We further investigate the scenario of using noisy signs, where the measurements of the directions of the relative states are perturbed by noise. We show that the regret bound is not affected as long as the noise is not too large, which manifests certain noise-tolerance property of the proposed algorithm. Additionally, we extend the algorithm to the case of bandit feedback, where only the values of the local loss functions at two points are revealed to agents at each time. We demonstrate that the regret bound is not influenced by bandit feedback in order sense. Finally, numerical experiments on decentralized online least squares and logistic regression are conducted to corroborate the efficacy of the proposed algorithms.

Backscatter-Enabled NOMA for Future 6G Systems: A New Optimization Framework Under Imperfect SIC

This letter proposes a new optimization framework for power-domain non-orthogonal multiple access (PD-NOMA) ambient backscatter communication (AmBC) system with imperfect successive interference cancellation (SIC) decoding. In particular, we jointly optimize the transmit power of the source and the reflection coefficient of the backscatter tag to enhance the sum-rate of the system. Closed-form solutions are derived for the considered joint optimization problem via iteratively computing the Lagrange multipliers by the sub-gradient method. Simulation results validate the superiority of the proposed joint optimization framework over other benchmark frameworks, namely suboptimal PD-NOMA AmBC, optimal pure PD-NOMA, and orthogonal multiple access.

Coordinated management of aggregated electric vehicles and thermostatically controlled loads in hierarchical energy systems

This paper presents an energy management model for electric vehicles (EVs) and thermostatically controlled loads (TCLs) in intelligent energy systems based on the transactive control of aggregators. The management strategy will penetrate through three physical layers of electricity networks: transmission layer, distribution layer, and behind-meter layer. In the proposed framework, the aggregated EVs are modeled as a battery energy storage system (BESS), and the aggregated TCLs are modeled as a virtual energy storage system (VESS) at the behind-meter layer. A deep learning method, namely a hybrid of convolutional neural networks and long short-term memory (CNN-LSTM), is used to forecast the local loads of EVs and TCLs. The aggregators can dispatch these controllable loads directly as demand management to fit the predicted load curve. Peer-to-peer (P2P) trading is realized at the distribution level, and distributed optimization is utilized since the information between each aggregator is opaque. The primal problem is decoupled into subproblems of aggregators. The sub-gradient method is employed to update the multipliers of each decomposed Lagrange function. After the local energy transaction is cleared at the distribution level, wind generators and thermal generators are centrally dispatched at the transmission level based on the conventional optimal power flow model. The proposed hierarchy framework is verified in the IEEE 30-bus system. Simulation results reveal that the scalability issue of single-layer centralized dispatch can be well addressed, and end-users’ information privacy can be protected. The coordinated management of EVs and TCLs also brings in economic and environmental benefits.

Numerical solution of some systems of nonlinear algebraic equations

The problem of numerical solution of systems of nonlinear algebraic equations with even powers is studied in this paper. Nondifferentiable (nonsmooth) unconstrained minimization problems are associated with the system under consideration, with objective functions, based on discrete l 1 - and l ∞ - norm, respectively, by using these two norms as proximity criteria, and the considered system is solved by minimizing the sum of absolute values of residuals, in the case of l 1 - norm, or by minimizing the maximal residual, in the case of l ∞ - norm, respectively. Also, a “differentiable” unconstrained minimization problem (least squares problem) is associated with the considered system of nonlinear equations, with objective function, based on the discrete l 2 -norm, and the considered system is solved by minimizing the sum of squares of the residuals. Subgradients of the objective functions of unconstrained minimization problems are calculated, and a subgradient method for solving the associated problems is presented. The iterative gradient method for solving the corresponding “differentiable” discrete least squares problem, based on l 2 - norm, is also outlined, and gradients of the objective function of the corresponding “differentiable” unconstrained minimization problems are calculated.

A Study on Distributed Optimization over Large-Scale Networked Systems

Distributed optimization is a very important concept with applications in control theory and many related fields, as it is high fault-tolerant and extremely scalable compared with centralized optimization. Centralized solution methods are not suitable for many application domains that consist of large number of networked systems. In general, these large-scale networked systems cooperatively find an optimal solution to a common global objective during the optimization process. Thus, it gives us an opportunity to analyze distributed optimization techniques that is demanded in most distributed optimization settings. This paper presents an analysis that provides an overview of decomposition methods as well as currently existing distributed methods and techniques that are employed in large-scale networked systems. A detailed analysis on gradient like methods, subgradient methods, and methods of multipliers including the alternating direction method of multipliers is presented. These methods are analyzed empirically by using numerical examples. Moreover, an example highlighting the fact that the gradient method fails to solve distributed problems in some circumstances is discussed under numerical results. A numerical implementation is used to demonstrate that the alternating direction method of multipliers can solve this particular problem, by revealing its robustness compared with the gradient method. Finally, we conclude the paper with possible future research directions.

Strong convergence of inertial subgradient extragradient algorithm for solving pseudomonotone equilibrium problems

In this paper, we propose a new modified subgradient extragradient method for solving equilibrium problems involving pseudomonotone and Lipchitz-type bifunctions in Hilbert spaces. We establish the strong convergence of the proposed method under several suitable conditions. In addition, the linear convergence is obained under strong pseudomonotonicity assumption. Our results generalize and extend some related results in the literature. Finally, we provide numerical experiments to illustrate the performance of the proposed algorithm.

On the Analysis of Variance-reduced and Randomized Projection Variants of Single Projection Schemes for Monotone Stochastic Variational Inequality Problems

Classical extragradient schemes and their stochastic counterpart represent a cornerstone for resolving monotone variational inequality problems. Yet, such schemes have a per-iteration complexity of two projections onto a convex set and require two evaluations of the map, the former of which could be relatively expensive. We consider two related avenues where the per-iteration complexity is significantly reduced: (i) A stochastic projected reflected gradient method requiring a single evaluation of the map and a single projection; and (ii) A stochastic subgradient extragradient method that requires two evaluations of the map, a single projection onto the associated feasibility set, and a significantly cheaper projection (onto a halfspace) computable in closed form. Under a variance-reduced framework reliant on a sample-average of the map based on an increasing batch-size, we prove almost sure convergence of the iterates to a random point in the solution set for both schemes. Additionally, non-asymptotic rate guarantees are derived for both schemes in terms of the gap function; notably, both rates match the best-known rates obtained in deterministic regimes. To address feasibility sets given by the intersection of a large number of convex constraints, we adapt both of the aforementioned schemes to a random projection framework. We then show that the random projection analogs of both schemes also display almost sure convergence under a weak-sharpness requirement; furthermore, without imposing the weak-sharpness requirement, both schemes are characterized by the optimal rate in terms of the gap function of the projection of the averaged sequence onto the set as well as the infeasibility of this sequence. Preliminary numerics support theoretical findings and the schemes outperform standard extragradient schemes in terms of the per-iteration complexity.

Beam Allocation and Power Optimization for Energy-Efficiency in Multiuser mmWave Massive MIMO System.

This paper studies beam allocation and power optimization scheme to decrease the hardware cost and downlink power consumption of a multiuser millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) system. Our target is to improve energy efficiency (EE) and decrease power consumption without obvious system performance loss. To this end, we propose a beam allocation and power optimization scheme. First, the problem of beam allocation and power optimization is formulated as a multivariate mixed-integer non-linear programming problem. Second, due to the non-convexity of this problem, we decompose it into two sub-problems which are beam allocation and power optimization. Finally, the beam allocation problem is solved by using a convex optimization technique. We solve the power optimization problem in two steps. First, the non-convex problem is converted into a convex problem by using a quadratic transformation scheme. The second step implements Lagrange dual and sub-gradient methods to solve the optimization problem. Performance analysis and simulation results show that the proposed algorithm performs almost identical to the exhaustive search (ES) method, while the greedy beam allocation and suboptimal beam allocation methods are far from the ES. Furthermore, experiment results demonstrated that our proposed algorithm outperforms the compared the greedy beam allocation method and the suboptimal beam allocation scheme in terms of average service ratio.

Primal-dual subgradient method for constrained convex optimization problems

This paper considers a general convex constrained problem setting where functions are not assumed to be differentiable nor Lipschitz continuous. Our motivation is in finding a simple first-order method for solving a wide range of convex optimization problems with minimal requirements. We study the method of weighted dual averages (Nesterov in Math Programm 120(1): 221–259, 2009) in this setting and prove that it is an optimal method.

A new self-adaptive algorithm for solving pseudomonotone variational inequality problems in Hilbert spaces

ABSTRACT In this paper, we revisit the subgradient extragradient method for solving a pseudomonotone variational inequality problem with the Lipschitz condition in real Hilbert spaces. A new algorithm based on the subgradient extragradient method with the technique of choosing a new step size is proposed. The weak convergence of the proposed algorithm is established under the pseudomonotonicity and the Lipschitz continuity as well as without using the sequentially weakly continuity of the variational inequality mapping and the nonasymptotic convergence rate of the proposed algorithm is presented, while the strong convergence theorem of the proposed algorithm is also proved under the strong pseudomonotonicity and the Lipschitz continuity hypotheses. In order to show the computational effectiveness of our algorithm, some numerical results are provided.

Modified projected subgradient method for solving pseudomonotone equilibrium and fixed point problems in Banach spaces

Motivated by the work of D.V. Hieu and J.-J. Strodiot [Strong convergence theorems for equilibrium problems and fixed point problems in Banach spaces, J. Fixed Point Theory Appl., (2018), 20:131], we introduce a new projected subgradient method for solving pseudomonotone equilibrium and fixed point problem in Banach spaces. The main iterative steps in the proposed method use a projection method and do not require any Lipschitz-like condition on the equilibrium bifunction. A strong convergence result is proved under mild conditions and we applied our algorithm to solving pseudomonotone variational inequalities in Banach spaces. Also, we provide some numerical examples to illustrate the performance of the proposed method and compare it with other methods in the literature.

Halpern-Subgradient Extragradient Method for Solving Equilibrium and Common Fixed Point Problems in Reflexive Banach Spaces

In this paper, using the concept of Bregman distance, we introduce a new Bregman subgradient extragradient method for solving equilibrium and common fixed point problems in a real reflexive Banach space. The algorithm is designed, such that the stepsize is chosen without prior knowledge of the Lipschitz constants. We also prove a strong convergence result for the sequence that is generated by our algorithm under mild conditions. We apply our result to solving variational inequality problems, and finally, we give some numerical examples to illustrate the efficiency and accuracy of the algorithm.

Augmented Lagrangian based hybrid subgradient method for solving aircraft maintenance routing problem

In this paper, a new version of the aircraft maintenance routing problem, which is an important component of the airline planning process, is studied. We define the big-cycle aircraft maintenance routing problem on a connection network and formulate it as an asymmetric travelling salesman problem with fleet size and maintenance violation constraints, which takes maintenance resource availability into account. In the connection network, flight legs are represented by nodes and connections between flight legs are denoted by arcs. In our approach, decision variables denote the connections between flight legs. We develop a hybrid solution method for this problem, which combines the Gasimov’s modified subgradient algorithm and the ant colony optimization metaheuristic. The performance of the proposed method, is demonstrated and analyzed through detailed computational experiments. Performances of the proposed method, conventional subgradient algorithm and ant colony optimization algorithm are compared on different test problems. The analysis of the quality of the results demonstrates that the proposed method outperforms the mentioned methods.

Subgradient method with feasible inexact projections for constrained convex optimization problems

In this paper, we propose a new inexact version of the projected subgradient method to solve nondifferentiable constrained convex optimization problems. The method combine ϵ-subgradient method with a procedure to obtain a feasible inexact projection onto the constraint set. Asymptotic convergence results and iteration-complexity bounds for the sequence generated by the method employing the well-known exogenous stepsizes, Polyak's stepsizes, and dynamic stepsizes are established.

Modified subgradient extragradient method for system of variational inclusion problem and finite family of variational inequalities problem in real Hilbert space

For the purpose of this article, we introduce a modified form of a generalized system of variational inclusions, called the generalized system of modified variational inclusion problems (GSMVIP). This problem reduces to the classical variational inclusion and variational inequalities problems. Motivated by several recent results related to the subgradient extragradient method, we propose a new subgradient extragradient method for finding a common element of the set of solutions of GSMVIP and the set of a finite family of variational inequalities problems. Under suitable assumptions, strong convergence theorems have been proved in the framework of a Hilbert space. In addition, some numerical results indicate that the proposed method is effective.

Weakly-convex–concave min–max optimization: provable algorithms and applications in machine learning

Min–max problems have broad applications in machine learning, including learning with non-decomposable loss and learning with robustness to data distribution. Convex–concave min–max problem is an active topic of research with efficient algorithms and sound theoretical foundations developed. However, it remains a challenge to design provably efficient algorithms for non-convex min–max problems with or without smoothness. In this paper, we study a family of non-convex min–max problems, whose objective function is weakly convex in the variables of minimization and is concave in the variables of maximization. We propose a proximally guided stochastic subgradient method and a proximally guided stochastic variance-reduced method for the non-smooth and smooth instances, respectively, in this family of problems. We analyse the time complexities of the proposed methods for finding a nearly stationary point of the outer minimization problem corresponding to the min–max problem.

Asynchronous Distributed Optimization via Dual Decomposition and Block Coordinate Subgradient Methods

In this article, we study the problem of minimizing the sum of potentially nondifferentiable convex cost functions with partially overlapping dependences in an asynchronous manner, where communication in the network is not coordinated. We study the behavior of an asynchronous algorithm based on dual decomposition and block coordinate subgradient methods under assumptions weaker than those used in the literature. At the same time, we allow different agents to use local stepsizes with no global coordination. Sufficient conditions are provided for almost sure convergence to the solution of the optimization problem. Under additional assumptions, we establish a sublinear convergence rate that, in turn, can be strengthened to the linear convergence rate if the problem is strongly convex and has Lipschitz gradients. We also extend available results in the literature by allowing multiple and potentially overlapping blocks to be updated at the same time with nonuniform and potentially time-varying probabilities assigned to different blocks. A numerical example is provided to illustrate the effectiveness of the algorithm.

Mathematical Optimization Techniques in Computational of Biological Models

The aim of this research is focused on the mathematical optimization in computational systems biological models based on find the good method that gives converges faster. Our strategy is to use Bundle Method instead of using Subgradient Methods. Also, we improve the convergence of theoretical properties of the methods. The basic idea of approximation is the subdifferential of the objective function by using subgradients from previous iterations of a bundle method.