Methods For Unconstrained Optimization Research Articles

Convergence of a new nonmonotone memory gradient method for unconstrained multiobjective optimization via robust approach

Convergence of a new nonmonotone memory gradient method for unconstrained multiobjective optimization via robust approach

A two-point heuristic to calculate the stepsize in subgradient method with application to a network design problem

We introduce a heuristic rule for calculating the stepsize in the subgradient method for unconstrained convex nonsmooth optimization which, unlike the classic approach, is based on retaining some information from previous iteration. The rule is inspired by the well known two-point stepsize by Barzilai and Borwein (BB) [6] for smooth optimization and it coincides with (BB) in case the function to be minimised is convex quadratic.Under the use of appropriate safeguards we demonstrate that the method terminates at a point that satisfies an approximate optimality condition.The proposed approach is tested in the framework of Lagrangian relaxation for integer linear programming where the Lagrangian dual requires maximization of a concave and nonsmooth (piecewise affine) function. In particular we focus on the relaxation of the Minimum Spanning Tree problem with Conflicting Edge Pairs (MSTC). Comparison with classic subgradient method is presented. The results on some widely used academic test problems are provided too.

An accelerated hybrid Riemannian conjugate gradient method for unconstrained optimization

With the goal to deal with a series of optimization problems on general matrix manifolds with differentiable objective functions, we propose an accelerated hybrid Riemannian conjugate gradient technique. Specifically, the acceleration scheme of the proposed method using a modified stepsize which is multiplicatively determined by the Wolfe line search. The search direction of the proposed algorithm is determined by the hybrid conjugate parameter with computationally promising. We showed that the suggested approach converges globally to a stationary point. Our approach performs better than the state of art Riemannian conjugate gradient algorithms, as illustrated by computations on problems such as the orthogonal Procrustes problem and the Brockett-cost-function minimization problem.

A family of accelerated hybrid conjugate gradient method for unconstrained optimization and image restoration

A family of accelerated hybrid conjugate gradient method for unconstrained optimization and image restoration

Integration of Modified Classical Conjugate Gradient Methods for Unconstrained Optimization

The integration of modified classical conjugate gradient methods (CGMs) for unconstrained optimization represents a crucial and evolving area of research within the field of optimization algorithms. Over time, numerous studies have put forth diverse modifications and novel approaches to enhance the effectiveness of classical CGMs. These modifications aim to address specific challenges and improve the overall performance of optimization algorithms in unconstrained scenarios. In order to tackle unconstrained optimization challenges and improve our understanding of their synergies, this ongoing study aims to unify different modified classical CGMs. Conventional CGMs have proven effective for optimization tasks, and a range of different approaches have been produced by carefully modifying these techniques. The main goal of this paper is to combine these modified versions, with particular attention to those that have similar numerators. The integration process involves systematically merging the advantageous aspects of these modified methods to develop not only innovative but also more resilient approaches to unconstrained optimization problems. The ultimate goal of this unification effort is to capitalize on the strengths inherent in different approaches to create a cohesive framework that significantly improves overall optimization performance. To thoroughly assess the efficacy of the integrated methods, a series of comprehensive performance tests are conducted. These tests include a meticulous comparison of outcomes with those of classical CGMs, providing valuable insights into the relative strengths and weaknesses of the modified approaches across diverse optimization scenarios. The evaluation criteria encompass convergence rates, solution accuracy, and computational efficiency. The conclusive outcome demonstrates that the unified approaches consistently outperform individual methods across all three crucial evaluation criteria.

A New Dai-Liao Conjugate Gradient Method based on Approximately Optimal Stepsize for Unconstrained Optimization

Conjugate gradient methods are a class of very effective iterative methods for large-scale unconstrained optimization. In this paper, a new Dai-Liao conjugate gradient method for solving large-scale unconstrained optimization problem is proposed. Based on the approximately optimal stepsize for the gradient method, we derive three new choices for the important parameters tk in Dai-Liao conjugate gradient method. The search direction satisfies the sufficient descent condition, and the global convergences of the proposed method for uniformly convex and general functions are proved under some mild conditions. Numerical experiments on a set of test problems from the CUTEst library show that the proposed method is superior to some well-known conjugate gradient methods.

Shape-changing trust-region methods using multipoint symmetric secant matrices

In this work, we consider methods for large-scale and nonconvex unconstrained optimization. We propose a new trust-region method whose subproblem is defined using a so-called ‘shape-changing’ norm together with densely-initialized multipoint symmetric secant (MSS) matrices to approximate the Hessian. Shape-changing norms and dense initializations have been successfully used in the context of traditional quasi-Newton methods, but have yet to be explored in the case of MSS methods. Numerical results suggest that trust-region methods that use densely-initialized MSS matrices together with shape-changing norms outperform MSS with other trust-region methods.

Two diagonal conjugate gradient like methods for unconstrained optimization

Two diagonal conjugate gradient like methods for unconstrained optimization

A new family of hybrid conjugate gradient method for unconstrained optimization and its application to regression analysis

We know many conjugate gradient algorithms (CG) for solving unconstrained optimization problems. In this paper, based on the three famous Liu–Storey (LS), Fletcher–Reeves (FR) and Polak–Ribiére–Polyak (PRP) conjugate gradient methods, a new hybrid CG method is proposed. Furthermore, the search direction satisfies the sufficient descent condition independent of the line search. Likewise, we prove, under the strong Wolfe line search, the global convergence of the new method. In this respect, numerical experiments are performed and reported, which show that the proposed method is efficient and promising. In virtue of this, the application of the proposed method for solving regression models of COVID-19 is provided.

New Subspace Method for Unconstrained Derivative-Free Optimization

This article defines an efficient subspace method, called SSDFO , for unconstrained derivative-free optimization problems where the gradients of the objective function are Lipschitz continuous but only exact function values are available. SSDFO employs line searches along directions constructed on the basis of quadratic models. These approximate the objective function in a subspace spanned by some previous search directions. A worst-case complexity bound on the number of iterations and function evaluations is derived for a basic algorithm using this technique. Numerical results for a practical variant with additional heuristic features show that, on the unconstrained CUTEst test problems, SSDFO has superior performance compared to the best solvers from the literature.

A Modified Wei-Yao-Liu Conjugate Gradient Method for Unconstrained Optimization and Motion Control of Robotic Motion Manipulators

A Modified Wei-Yao-Liu Conjugate Gradient Method for Unconstrained Optimization and Motion Control of Robotic Motion Manipulators

Optimization on the symplectic Stiefel manifold: SR decomposition-based retraction and applications

Numerous problems in optics, quantum physics, stability analysis, and control of dynamical systems can be brought to an optimization problem with matrix variable subjected to the symplecticity constraint. As this constraint nicely forms a so-called symplectic Stiefel manifold, Riemannian optimization is preferred, because one can borrow ideas from unconstrained optimization methods after preparing necessary geometric tools. Retraction is arguably the most important one which decides the way iterates are updated given a search direction. Two retractions have been constructed so far: one relies on the Cayley transform and the other is designed using quasi-geodesic curves. In this paper, we propose a new retraction which is based on an SR matrix decomposition. We prove that its domain contains the open unit ball which is essential in proving the global convergence of the associated gradient-based optimization algorithm. Moreover, we consider three applications—symplectic target matrix problem, symplectic eigenvalue computation, and symplectic model reduction of Hamiltonian systems—with various examples. The extensive numerical comparisons reveal the strengths of the proposed optimization algorithm.

A Structured Fletcher-Revees Spectral Conjugate Gradient Method for Unconstrained Optimization with Application in Robotic Model

A Structured Fletcher-Revees Spectral Conjugate Gradient Method for Unconstrained Optimization with Application in Robotic Model

A homogeneous Rayleigh quotient with applications in gradient methods

Given an approximate eigenvector, its (standard) Rayleigh quotient and harmonic Rayleigh quotient are two well-known approximations of the corresponding eigenvalue. We propose a new type of Rayleigh quotient, the homogeneous Rayleigh quotient, and analyze its sensitivity with respect to perturbations in the eigenvector. Furthermore, we study the inverse of this homogeneous Rayleigh quotient as stepsize for the gradient method for unconstrained optimization. The notion and basic properties are also extended to the generalized eigenvalue problem.

A regularized limited memory subspace minimization conjugate gradient method for unconstrained optimization

A regularized limited memory subspace minimization conjugate gradient method for unconstrained optimization

Low-rank Representation with Adaptive Dimensionality Reduction via Manifold Optimization for Clustering

The dimensionality reduction techniques are often used to reduce data dimensionality for computational efficiency or other purposes in existing low-rank representation (LRR)-based methods. However, the two steps of dimensionality reduction and learning low-rank representation coefficients are implemented in an independent way; thus, the adaptability of representation coefficients to the original data space may not be guaranteed. This article proposes a novel model, i.e., low-rank representation with adaptive dimensionality reduction (LRRARD) via manifold optimization for clustering, where dimensionality reduction and learning low-rank representation coefficients are integrated into a unified framework. This model introduces a low-dimensional projection matrix to find the projection that best fits the original data space. And the low-dimensional projection matrix and the low-rank representation coefficients interact with each other to simultaneously obtain the best projection matrix and representation coefficients. In addition, a manifold optimization method is employed to obtain the optimal projection matrix, which is an unconstrained optimization method in a constrained search space. The experimental results on several real datasets demonstrate the superiority of our proposed method.

Joint Design of a Simultaneous Reflection and Transmission RIS in Mode-Switching Mode to Assist NOMA Systems.

Simultaneous transmitting and reflecting reconfigurable intelligent surfaces (STAR-RISs) can reflect signals and transmissive signals simultaneously and can extend the coverage of signals. A conventional RIS mainly focuses on the case where the signal source and the target are on the same side. In this paper, a STAR-RIS-assisted non-orthogonal multiple access (NOMA) downlink communication system is considered to maximize the achievable rate for users by jointly optimizing the power-allocation coefficients, active beamforming, and STAR-RIS beamforming under the mode-switching (MS) protocol. The critical information of the channel is first extracted using the Uniform Manifold Approximation and Projection (UMAP) method. Based on the key extracted channel features, STAR-RIS elements and users are clustered individually using the fuzzy C-mean clustering (FCM) method. The alternating optimization method decomposes the original optimization problem into three sub-optimization problems. Finally, the sub-problems are converted to unconstrained optimization methods using penalty functions for the solution. Simulation results show that when the number of elements of RIS is 60, the achievable rate of the STAR-RIS-NOMA system is about 18% higher than that of the RIS-NOMA system.

An Accelerated Regularized Chebyshev–Halley Method for Unconstrained Optimization

In machine learning, most models can be transformed into unconstrained optimization problems, so how to solve the unconstrained optimization problem for different objective functions is always a hot issue. In this paper, a class of unconstrained optimization where objection function has [Formula: see text]th-order derivative and Lipschitz continuous simultaneously is studied. To handle such problems, we propose an accelerated regularized Chebyshev–Halley method based on the Accelerated Hybrid Proximal Extragradient (A-HPE) framework. It proves that convergence complexity of the proposed method is [Formula: see text], which is consistent with the lower iteration complexity bound for third-order tensor methods. Numerical experiments on functions in machine learning demonstrate the promising performance of the proposed method.

Efficient implementation of numerical methods for solving bearing cavitation problems using symmetric system solvers

Since the successful application of the Elrod method to the Jakobsson–Floberg–Olsson (JFO) theory, many new alternative methods have been presented. In particular, over the past decade, the efficiency of cavitation algorithms has significantly improved, as demonstrated by the Fischer-Burmeister based unconstrained optimization methods and the modulus-based methods based on the generalized absolute value equation. This paper briefly reviewed the principles of these algorithms and compared their computational efficiency. Pseudocodes were included to illustrate their implementation. Taking advantage of linear direct solvers, a novel equivalent symmetric matrix equation was presented to promote the efficiency of these algorithms. A case of a surface-textured thrust bearing with multiple dimples was studied. The results showed that the method can save more than 2–5 times the computing time.

Computing the Action Ground State for the Rotating Nonlinear Schrödinger Equation

We consider the computations of the action ground state for a rotating nonlinear Schr\"odinger equation. It reads as a minimization of the action functional under the Nehari constraint. In the focusing case, we identify an equivalent formulation of the problem which simplifies the constraint. Based on it, we propose a normalized gradient flow method with asymptotic Lagrange multiplier and establish the energy-decaying property. Popular optimization methods are also applied to gain more efficiency. In the defocusing case, we prove that the ground state can be obtained by the unconstrained minimization. Then the direct gradient flow method and unconstrained optimization methods are applied. Numerical experiments show the convergence and accuracy of the proposed methods in both cases, and comparisons on the efficiency are discussed. Finally, the relation between the action and the energy ground states are numerically investigated.