Subgradient Method Research Articles

Improving two-dimensional linear discriminant analysis with L1 norm for optimizing EEG signal

Dimensionality reduction is a critical factor in processing high-dimensional datasets. The L1 norm-based Two-Dimensional Linear Discriminant Analysis (L1-2DLDA) is widely used for this purpose, but it remains sensitive to outliers and classes with large deviations, which deteriorates its performance. To address this limitation, the present study proposed Pairwise Sample Distance Two-Dimensional Linear Discriminant Analysis (PSD2DLDA), a novel method that modeled L1-2DLDA using pair-wise sample distances. To improve computational effectiveness, this study also introduced a streamlined variant, Pairwise Class Mean Distance Two-Dimensional Linear Discriminant Analysis (PCD2DLDA), which was based on distances between class mean pairs. Different from previous studies, this study utilized the projected sub-gradient method to optimize these two improved methods. Meanwhile, this study explored the interrelationship, limitations, and applicability of these two improved methods. The comparative experimental results on three datasets validated the outstanding performance of PSD2DLDA and PCD2DLDA methods. In particular, PSD2DLDA exhibited superior robustness compared to PCD2DLDA. Furthermore, applying these two methods to optimize electroencephalogram (EEG) signals effectively enhanced the decoding accuracy of motor imagery neural patterns, which offered a promising strategy for optimizing EEG signals processing in brain-computer interface (BCI) applications.

Solving discrete network design problem using disjunctive constraints

AbstractThis paper introduces a deterministic algorithm to solve the discrete network design problem (DNDP) efficiently. This non‐convex bilevel optimization problem is well‐known as an non deterministic polynomial (NP)‐hard problem in strategic transportation planning. The proposed algorithm optimizes budget allocation for large‐scale network improvements deterministically and with computational efficiency. It integrates disjunctive programming with an improved partial linearized subgradient method to enhance performance without significantly affecting solution quality. We evaluated our algorithm on the mid‐scale Sioux Falls and large‐scale Chicago networks. We assess the proposed algorithm's accuracy by examining the objective function's value, specifically the total travel time within the network. When tested on the mid‐scale Sioux Falls network, the algorithm achieved an average 46% improvement in computational efficiency, compared to the best‐performing method discussed in this paper, albeit with a 4.17% higher total travel time than the most accurate one, as the value of the objective function. In the application to the large‐scale Chicago network, the efficiency improved by an average of 99.48% while the total travel time experienced a 4.34% increase. These findings indicate that the deterministic algorithm proposed in this research improves the computational speed while presenting a limited trade‐off with solution precision. This deterministic approach offers a structured, predictable, and repeatable method for solving DNDP, which can advance transportation planning, particularly for large‐scale network applications where computational efficiency is paramount.

Robust Bilinear Form Identification: A Subgradient Method with Geometrically Decaying Stepsize in the Presence of Heavy-Tailed Noise

Robust Bilinear Form Identification: A Subgradient Method with Geometrically Decaying Stepsize in the Presence of Heavy-Tailed Noise

Three novel inertial subgradient extragradient methods for quasi-monotone variational inequalities in Banach spaces

Three novel inertial subgradient extragradient methods for quasi-monotone variational inequalities in Banach spaces

An integrated blood supply chain network design during a pandemic

The emergency supply of blood in the wake of pandemic has proved challenging. We develop in this paper a bi-objective blood supply chain (BSC) model that uses two-stage stochastic optimisation to optimise the expected cost of the supply chain and the average blood delivery time when a pandemic occurs. The model considers not only vertically and horizontally integrated mechanisms of the BSC network, but also several specific assumptions about the pandemic, such as the risk of disruption in donor groups, and the restriction of ‘no cross blood donation’. To alleviate blood shortages in the context of a pandemic, we further extended our model, which allows for blood type substitution according to ABO/Rh(d)-compatibility. To efficiently solve the model and generate the Pareto front, we develop a hybrid solution algorithm to solve the proposed model, including the ε-constraint method to transform the bi-objective model into a single-objective model, the subgradient method for finding a lower bound, and two heuristic methods for finding an upper bound. Moreover, we perform extensive numerical studies to assess the performance of the proposed approach. We also discuss the impacts of the key parameters, integration mechanisms, and planning blood type substitution on the BSC. Finally, we also apply the proposed model to a real case study of the BSC in Chongqing, China.

A feasible smoothing accelerated projected gradient method for nonsmooth convex optimization

Smoothing accelerated gradient methods achieve faster convergence rates than that of the subgradient method for some nonsmooth convex optimization problems. However, Nesterov's extrapolation may require gradients at infeasible points, and thus they cannot be applied to some structural optimization problems. We introduce a variant of smoothing accelerated projected gradient methods where every variable is feasible. The O(k−1log⁡k) convergence rate is obtained using the Lyapunov function. We conduct a numerical experiment on the robust compliance optimization of a truss structure.

Real-time pricing considering carbon constraints for smart grid with multiple energy sources

With the escalation of fossil fuel pollution and the challenges of climate change, the smart grid is forced to address the carbon emissions associated with fossil fuel use. Real-time pricing of smart grid is an effective means to regulate the balance of supply and demand in the grid as well as to achieve energy conservation and environmental protection. This paper takes traditional fossil energy and clean energy power supply as the background, takes social welfare maximization as the goal, combines carbon emission cost with traditional power generation cost, and constructs a real-time pricing model that takes into account carbon emission cost, so that thermal power enterprises participate in the power market and carbon market at the same time. By using duality methods, we transformed the original optimization problem into an equivalent one. Furthermore, a distributed iterative algorithm based on the subgradient projection method is employed to solve the dual problem. The simulation results verify the feasibility and effectiveness of the proposed real-time pricing strategy.

Inertial subgradient projection techniques for solving a class of bilevel equilibrium problems

ABSTRACT In this paper, we introduce a new algorithm for solving bilevel equilibrium problems with nonexpansive and pseudocontractive mappings in a real Hilbert space. The proposed algorithm is based on subgradient method with Armijo-type linesearch process, subgradient method, and inertial projection techniques. By choosing the suitable regular parameters, we prove strong convergence of the algorithm under classical conditions of the cost bifunctions.

An Investigation on LP Decoding of Short Binary Linear Codes With the Subgradient Method

An Investigation on LP Decoding of Short Binary Linear Codes With the Subgradient Method

A new spectral conjugate subgradient method with application in computed tomography image reconstruction

A new spectral conjugate subgradient method is presented to solve nonsmooth unconstrained optimization problems. The method combines the spectral conjugate gradient method for smooth problems with the spectral subgradient method for nonsmooth problems. We study the effect of two different choices of line search, as well as three formulas for determining the conjugate directions. In addition to numerical experiments with standard nonsmooth test problems, we also apply the method to several image reconstruction problems in computed tomography, using total variation regularization. Performance profiles are used to compare the performance of the algorithm using different line search strategies and conjugate directions to that of the original spectral subgradient method. Our results show that the spectral conjugate subgradient algorithm outperforms the original spectral subgradient method, and that the use of the Polak–Ribière formula for conjugate directions provides the best and most robust performance.

An effective subgradient algorithm via Mifflin’s line search for nonsmooth nonconvex multiobjective optimization

We propose a descent subgradient algorithm for unconstrained nonsmooth nonconvex multiobjective optimization problems. To find a descent direction, we present an iterative process that efficiently approximates the ɛ-subdifferential of each objective function. To this end, we develop a new variant of Mifflin’s line search in which the subgradients are arbitrary and its finite convergence is proved under a semismooth assumption. To reduce the number of subgradient evaluations, we employ a backtracking line search that identifies the objectives requiring an improvement in the current approximation of the ɛ-subdifferential. Meanwhile, for the remaining objectives, new subgradients are not computed. Unlike bundle-type methods, the proposed approach can handle nonconvexity without the need for algorithmic adjustments. Moreover, the quadratic subproblems have a simple structure, and hence the method is easy to implement. We analyze the global convergence of the proposed method and prove that any accumulation point of the generated sequence satisfies a necessary Pareto optimality condition. Furthermore, our convergence analysis addresses a theoretical challenge in a recently developed subgradient method. Through numerical experiments, we observe the practical capability of the proposed method and evaluate its efficiency when applied to a diverse range of nonsmooth test problems.

Dispersed-sparsity-aware LMS algorithm for scattering-sparse system identification

We develop a least-mean-squares (LMS) algorithm to identify scattering-sparse systems, where the few significantly large coefficients of the unknown impulse response are dispersed along the full length, instead of grouped into clusters. Many wireless communication channels such as the underwater acoustic channels, cellular communication channels and aviation channels are typical scattering-sparse systems. In the proposed strategy, the difference between the ℓ1-norm and ℓ∞,1-norm of the uniformly-divided tap-weight vector of the adaptive filter is utilized as a penalty and is introduced into the mean-square-error cost function, where the ℓ∞,1-norm of the tap-weight vector is used to locate the unknown large coefficients in view of the characteristics of dispersed sparsity. A dispersed-sparsity-aware LMS (DS-LMS) algorithm is then proposed by following the stochastic subgradient method. We study the mean and mean-square behaviors of the proposed algorithm. We also provide theoretical guidelines for the parameter settings that ensure that the proposed DS-LMS algorithm converges to a lower mean-square-deviation (MSD) level than the traditional LMS algorithm. Simulation results verify the effectiveness of the proposed DS-LMS algorithm, and correctness of the theoretical findings.

Joint Optimization Algorithm for Latency-Energy and Privacy Combining Discrete Differential Evolution and Convex Optimization

A privacy protection task offloading algorithm combining discrete differential evolution and convex optimization is proposed to resolve the new position privacy leakage problem for mobile terminal due to transferring tasks to multiple edge servers for computation via wireless networks. First, the privacy cost of offloading decision is defined for the characteristics of the position privacy problem combined with the standard deviation theory. Second, the objective problem is modeled as a joint optimization of position privacy protection, offloading decision and transmission power, and the total cost of task offloading is an optimization objective. Finally, to handle the objective problem efficiently, offloading decision variable is decoupled from the transmission power variable to decrease the computational complexity of the problem; the discrete differential evolution algorithm is designed to accumulate the approximate optimal offloading decision, and the optimal transmission power is efficiently solved by variable substitution method and subgradient method. Simulations reveal that the algorithm has excellent convergence; it improves the position privacy protection level of mobile terminal and reduces the total cost of task offloading while maintaining lower energy consumption and latency.

Convergence of projected subgradient method with sparse or low-rank constraints

Convergence of projected subgradient method with sparse or low-rank constraints

Transport capacity optimization for high-speed rail network considering flexible train composition and additional capacity pool

To balance the limited capacity and constantly changing demand in the high-speed rail network, a transport capacity optimization problem is investigated by adjusting train composition through uncoupling/coupling operations at the origin station, scheduling additional trains from the capacity pool, and optimizing the seat allocation among ODs. To make joint decisions, a two-stage stochastic programming model is formulated to minimize both operating cost and penalty cost for unsatisfied passengers, while incorporating various constraints such as flow conservation, loading capacity, flexible composition, etc. The two-stage model is linearized into a mixed-integer linear programming model, which can be effectively solved for small networks but is still intractable to solve for realistic networks. Therefore, a Lagrangian relaxation algorithm is proposed integrating parallel computing and sub-gradient method. Finally, two sets of case tests, including a toy network and a real-world case of a high-speed rail network in China, are implemented to show the performance of the proposed approach. The results illustrate that the parallel heuristic algorithm could generate a high-quality solution with a slight discrepancy from the optimal solution, while achieving a commendable computation time. Furthermore, the findings emphasize the advantages of applying flexible train composition and additional capacity pool in the high-speed railway system for cost-saving and supply–demand matching.

Halpern inertial subgradient extragradient algorithm for solving equilibrium problems in Banach spaces

ABSTRACT In this work, we study an equilibrium problem involving a pseudomonotone bifunction in the context of uniformly smooth and 2-uniformly convex real Banach spaces. For the purpose of solving the fixed point problem of a finite family of multi-valued relatively nonexpansive mappings and the pseudomonotone equilibrium problem, we introduce an inertial subgradient extragradient method and establish its strong convergence. To increase the step size and enhance the performance of our iterative method, we use a new parameter that is independent of the inertial extrapolation method. We point out that our step size is chosen in a self-adaptive manner, making it simple to calculate our iterative algorithm without having to know the Lipschitz constants beforehand. Finally, we give some numerical results for the proposed algorithm and comparison with some other known algorithms.

Primal Subgradient Methods with Predefined Step Sizes

Primal Subgradient Methods with Predefined Step Sizes

Modified subgradient extragradient method for solution of split variational inequality problem with application to generalized Nash equilibrium problem

In this research, we propose modified subgradient extragradient method to find the common solution of split variational inequality problem and common fixed point problem of a finite family of demicontractive mappings without knowing norm of bounded linear operator and Lipschitz constants of the associated pseudomonotone operators. We obtain the strong convergence result for the proposed method under certain mild conditions. This result extends and generalizes many recent results in this direction. Furthermore, we demonstrate how our findings may be used to solve a generalized Nash equilibrium problem and LASSO problem. Finally, we give numerical experiments to validate the performance of our algorithm and compare it with other existing methods.

A Stochastic Inexact Sequential Quadratic Optimization Algorithm for Nonlinear Equality-Constrained Optimization

A stochastic algorithm is proposed, analyzed, and tested experimentally for solving continuous optimization problems with nonlinear equality constraints. It is assumed that constraint function and derivative values can be computed but that only stochastic approximations are available for the objective function and its derivatives. The algorithm is of the sequential quadratic optimization variety. Distinguishing features of the algorithm are that it only employs stochastic objective gradient estimates that satisfy a relatively weak set of assumptions (while using neither objective function values nor estimates of them) and that it allows inexact subproblem solutions to be employed, the latter of which is particularly useful in large-scale settings when the matrices defining the subproblems are too large to form and/or factorize. Conditions are imposed on the inexact subproblem solutions that account for the fact that only stochastic objective gradient estimates are employed. Convergence results are established for the method. Numerical experiments show that the proposed method vastly outperforms a stochastic subgradient method and can outperform an alternative sequential quadratic programming algorithm that employs highly accurate subproblem solutions in every iteration. Funding: This material is based upon work supported by the National Science Foundation [Awards CCF-1740796 and CCF-2139735] and the Office of Naval Research [Award N00014-21-1-2532].

Newtonian Property of Subgradient Method with Optimization of Metric Matrix Parameter Correction

Newtonian Property of Subgradient Method with Optimization of Metric Matrix Parameter Correction