Gradient Optimization Method Research Articles

OS-LALM-OGM Algorithm-Based Computed Tomography Image for Characteristics and Comorbidities of Patients before Transcatheter Aortic Valve Implantation.

Based on the ordered subsets (OS), a linear augmentation Lagrangian method (OS-LALM) was constructed, which was then combined with the optimized gradient method (OGM) to construct the OS-LALM-OGM, so as to discuss application of the computed tomography (CT) images based on OS-LALM-OGM in evaluation of clinical manifestations and complications of patients before transcatheter aortic valve implantation (TAVI). The OS-LALM-OGM was compared with the filtered back projection (FBP) and OS-LALM. In addition, it was applied to evaluate the conditions of 128 patients before TAVI. It was found that the peak signal-to-noise ratio (PSNR) of OS-LALM-OGM was greater than that of the FBP and OS-LALM when the number of iterations was 5, 20, and 40, while the root mean square error (RMSD) was the opposite (P < 0.05). The proportion of dyspnea was the highest, 38.28%, followed by angina (19.53%) and fainting (21.09%). The long diameter of the annulus and the average inner diameter of the annulus measured by the CT image based on the OS-LALM-OGM algorithm were greatly larger than the inner diameter of the aortic annulus measured by the CT based on the FBP algorithm (P < 0.05); the evaluation sensitivity (95.24%) and specificity (85.85%) of CT based on the OS-LALM-OGM algorithm were obviously greater than those of X-ray, which were 84.43% and 76.77%, respectively (P < 0.05). In short, the OS-LALM-OGM proposed had a relatively excellent effect on CT image reconstruction. The CT image based on the OS-LALM-OGM algorithm showed a better evaluation performance for patients before TAVI than the traditional FBP algorithm, showing higher sensitivity and specificity.

Proximal gradient method for nonconvex and nonsmooth optimization on Hadamard manifolds

Proximal gradient method for nonconvex and nonsmooth optimization on Hadamard manifolds

AN EFFICIENT HYBRID DERIVATIVE-FREE PROJECTION ALGORITHM FOR CONSTRAINT NONLINEAR EQUATIONS

In this paper, by combining the Solodov and Svaiter projection technique with the conjugate gradient method for unconstrained optimization proposed by Mohamed et al. (2020), we develop a derivative-free conjugate gradient method to solve nonlinear equations with convex constraints. The proposed method involves a spectral parameter which satisfies the sufficient descent condition. The global convergence is proved under the assumption that the underlying mapping is Lipschitz continuous and satisfies a weaker monotonicity condition. Numerical experiment shows that the proposed method is efficient.

An Inexact Optimal Hybrid Conjugate Gradient Method for Solving Symmetric Nonlinear Equations

This article presents an inexact optimal hybrid conjugate gradient (CG) method for solving symmetric nonlinear systems. The method is a convex combination of the optimal Dai–Liao (DL) and the extended three-term Polak–Ribiére–Polyak (PRP) CG methods. However, two different formulas for selecting the convex parameter are derived by using the conjugacy condition and also by combining the proposed direction with the default Newton direction. The proposed method is again derivative-free, therefore the Jacobian information is not required throughout the iteration process. Furthermore, the global convergence of the proposed method is shown using some appropriate assumptions. Finally, the numerical performance of the method is demonstrated by solving some examples of symmetric nonlinear problems and comparing them with some existing symmetric nonlinear equations CG solvers.

Linear convergence of a nonmonotone projected gradient method for multiobjective optimization

Linear convergence of a nonmonotone projected gradient method for multiobjective optimization

A dai-liao hybrid conjugate gradient method for unconstrained optimization

One of todays’ best-performing CG methods is Dai-Liao (DL) method which depends on non-negative parameter and conjugacy conditions for its computation. Although numerous optimal selections for the parameter were suggested, the best choice of remains a subject of consideration. The pure conjugacy condition adopts an exact line search for numerical experiments and convergence analysis. Though, a practical mathematical experiment implies using an inexact line search to find the step size. To avoid such drawbacks, Dai and Liao substituted the earlier conjugacy condition with an extended conjugacy condition. Therefore, this paper suggests a new hybrid CG that combines the strength of Liu and Storey and Conjugate Descent CG methods by retaining a choice of Dai-Liao parameterthat is optimal. The theoretical analysis indicated that the search direction of the new CG scheme is descent and satisfies sufficient descent condition when the iterates jam under strong Wolfe line search. The algorithm is shown to converge globally using standard assumptions. The numerical experimentation of the scheme demonstrated that the proposed method is robust and promising than some known methods applying the performance profile Dolan and Mor´e on 250 unrestricted problems. Numerical assessment of the tested CG algorithms with sparse signal reconstruction and image restoration in compressive sensing problems, file restoration, image video coding and other applications. The result shows that these CG schemes are comparable and can be applied in different fields such as temperature, fire, seismic sensors, and humidity detectors in forests, using wireless sensor network techniques.

Good Characteristics of The New Spectral Conjugate Gradient Method for Unconstrained Optimization

The spectral conjugate gradient (SCG) method is an effective method to solve large-scale nonlinear unconstrained optimization problems. In this work, we propose a new SCG method in which performance is numerically analyzed. We established the descent property and global convergence conditions based on assumptions through the strongWolfe-Powell line search. Numerical results were performed using benchmark functions widely used in many conventional functions to evaluate the efficiency of the proposed method.Subject Classification: 90C30, 90C06, 65K05, 65K10.

A New Spectral Conjugate Gradient Method for Nonlinear Unconstrained Optimization

Theconjugate gradient method is widely used to solve large scale unconstrainedoptimization problems. However, the rate of convergence conjugate gradient method is linearunless it restarted. In this study, we present a new spectral conjugategradient modification formula with restart property obtains the globalconvergence and descent properties.In addition, we proposed a new restart condition for Fletcher-Reeves conjugate gradient formula. The numerical resultsdemonstrated that the modified Fletcher-Reeves parameter and the new CG formulawith their restart conditions are more efficient and robustness than otherconventional methods.

Two-versions of descent conjugate gradient methods for large-scale unconstrained optimization

&lt;p&gt;The conjugate gradient methods are noted to be exceedingly valuable for solving large-scale unconstrained optimization problems since it needn't the storage of matrices. Mostly the parameter conjugate is the focus for conjugate gradient methods. The current paper proposes new methods of parameter of conjugate gradient type to solve problems of large-scale unconstrained optimization. A Hessian approximation in a diagonal matrix form on the basis of second and third-order Taylor series expansion was employed in this study. The sufficient descent property for the proposed algorithm are proved. The new method was converged globally. This new algorithm is found to be competitive to the algorithm of fletcher-reeves (FR) in a number of numerical experiments.&lt;/p&gt;

A Linear Hybridization of Dai-Yuan and Hestenes-Stiefel Conjugate Gradient Method for Unconstrained Optimization

A Linear Hybridization of Dai-Yuan and Hestenes-Stiefel Conjugate Gradient Method for Unconstrained Optimization

A variant of Dai-Yuan conjugate gradient method for unconstrained optimization and its application in portfolio selection

The quasi-Newton (QN) method are among the efficient variants of conjugate gradient (CG) method for solving unconstrained optimization problems. The QN method utilizes the gradients of the function while ignoring the available value information at every iteration. In this paper, we extended the Dai-Yuan [39] coefficient in designing a new CG method for large-scale unconstrained optimization problems. An interesting feature of our method is that its algorithm not only uses the available gradient value, but also consider the function value information. The global convergence of the proposed method was established under some suitable Wolfe conditions. Extensive numerical computation have been carried out which show that the average performance of this new algorithm is efficient and promising. In addition, the proposed method was extended to solve practical application problem of portfolio selection.

Automatic Calibration Module for an Urban Drainage System Model

The purpose of the study was to present an automated module for the calibration of urban drainage system models. A prepared tool based on the Open Water Analytics toolkit included 12 additional calibration parameters as compared to the existing similar solutions. The module included a gradient optimization method that allowed adjustment of up to five parameters simultaneously, and a trial-and-error method that provided the possibility of testing one or two parameters. The user interface was built in MS Excel to simplify use of the developed tool. The user can select preferable parameters for calibration, choose the optimization method, and determine the limits for the calculated values. The performance and functionality of the automatic calibration module was tested in two scenarios using the drainage model of a 10 ha heavily developed area in Tallinn, Estonia. The calibration results revealed that the maximum deviation between the modelled and measured flow rates was less than 5% for both cases. This is a reasonably good fit for drainage models, which typically encounter numerous uncertainties. Therefore, it was concluded that the module can be successfully used for calibrating hydraulic models created in SWMM5.

Adaptive Gradient Methods for Constrained Convex Optimization and Variational Inequalities

We provide new adaptive first-order methods for constrained convex optimization. Our main algorithms AdaACSA and AdaAGD+ are accelerated methods, which are universal in the sense that they achieve nearly-optimal convergence rates for both smooth and non-smooth functions, even when they only have access to stochastic gradients. In addition, they do not require any prior knowledge on how the objective function is parametrized, since they automatically adjust their per-coordinate learning rate. These can be seen as truly accelerated Adagrad methods for constrained optimization. We complement them with a simpler algorithm AdaGrad+ which enjoys the same features, and achieves the standard non-accelerated convergence rate. We also present a set of new results involving adaptive methods for unconstrained optimization and variational inequalities arising from monotone operators.

ACMo: Angle-Calibrated Moment Methods for Stochastic Optimization

Stochastic gradient descent (SGD) is a widely used method for its outstanding generalization ability and simplicity. Adaptive gradient methods have been proposed to further accelerate the optimization process. In this paper, we revisit existing adaptive gradient optimization methods with a new interpretation. Such new perspective leads to a refreshed understanding of the roles of second moments in stochastic optimization. Based on this, we propose Angle-Calibration Moment method (ACMo), a novel stochastic optimization method. It enjoys the benefits of second moments with only first moment updates. Theoretical analysis shows that ACMo is able to achieve the same convergence rate as mainstream adaptive methods. Experiments on a variety of CV and NLP tasks demonstrate that ACMo has a comparable convergence to state-of-the-art Adam-type optimizers, and even a better generalization performance in most cases. The code is available at https://github.com/Xunpeng746/ACMo.

Investigation of Optimization Algorithms for Neural Network Solutions of Optimal Control Problems with Mixed Constraints

In this paper, we consider the problem of selecting the most efficient optimization algorithm for neural network approximation—solving optimal control problems with mixed constraints. The original optimal control problem is reduced to a finite-dimensional optimization problem by applying the necessary optimality conditions, the Lagrange multiplier method and the least squares method. Neural network approximation models are presented for the desired control functions, trajectory and conjugate factors. The selection of the optimal weight coefficients of the neural network approximation was carried out using the gravitational search algorithm and the basic particle swarm algorithm and the genetic algorithm. Computational experiments showed that evolutionary optimization algorithms required the smallest number of iterations for a given accuracy in comparison with the classical gradient optimization method; however, the multi-agent optimization methods were performed later for each operation. As a result, the genetic algorithm showed a faster convergence rate relative to the total execution time.

Stabilization of the fluidic pinball with gradient-enriched machine learning control

Abstract

The stochastic multi-gradient algorithm for multi-objective optimization and its application to supervised machine learning

Optimization of conflicting functions is of paramount importance in decision making, and real world applications frequently involve data that is uncertain or unknown, resulting in multi-objective optimization (MOO) problems of stochastic type. We study the stochastic multi-gradient (SMG) method, seen as an extension of the classical stochastic gradient method for single-objective optimization. At each iteration of the SMG method, a stochastic multi-gradient direction is calculated by solving a quadratic subproblem, and it is shown that this direction is biased even when all individual gradient estimators are unbiased. We establish rates to compute a point in the Pareto front, of order similar to what is known for stochastic gradient in both convex and strongly convex cases. The analysis handles the bias in the multi-gradient and the unknown a priori weights of the limiting Pareto point. The SMG method is framed into a Pareto-front type algorithm for calculating an approximation of the entire Pareto front. The Pareto-front SMG algorithm is capable of robustly determining Pareto fronts for a number of synthetic test problems. One can apply it to any stochastic MOO problem arising from supervised machine learning, and we report results for logistic binary classification where multiple objectives correspond to distinct-sources data groups.

Dynamic mode decomposition analysis of the common research model with adjoint-based gradient optimization

Aerodynamic shape refinement optimization for passenger aircraft is difficult and requires a significant workload. The adjoint-based gradient optimization method can quickly find local optimal solutions based on the initial shape in these types of problems. The optimization model of the common research model for the drag coefficient minimization and wing thickness constraints with a large-scale grid is established, and the drag coefficient is reduced by 10.2 counts while maintaining the lift coefficient. The stress-blended eddy simulation is used for unsteady simulations, and the optimized configuration can effectively eliminate oscillations in the middle of the upper wing surface. The spanwise flow is reduced and the pressure response on the wing surface is due primarily to shock chordal motion. For aerodynamic analyses with similar shapes, the dynamic mode decomposition (DMD) analysis shows that the upper wing surface mode amplitudes and spanwise instability modes of the optimized design are weaker, and the fluctuations of the pressure are more stable. Therefore, DMD is suitable for refined shape optimization analyses.

A Modified Hybrid Conjugate Gradient Method for Unconstrained Optimization

The nonlinear conjugate gradient algorithms are a very effective way in solving large-scale unconstrained optimization problems. Based on some famous previous conjugate gradient methods, a modified hybrid conjugate gradient method was proposed. The proposed method can generate decent directions at every iteration independent of any line search. Under the Wolfe line search, the proposed method possesses global convergence. Numerical results show that the modified method is efficient and robust.

Sparse signal reconstruction in compressive sensing via derivative-free iterative method

Finding sparse solutions to under-determined or ill-conditioned equations is a problem that usually arise in compressive sensing. In this article, a derivative-free iterative method is presented for recovering sparse signal in compressive sensing by approximating the solution to a convex constrained nonlinear equation. The proposed method is derived from the modified Polak-Ribiere-Polyak conjugate gradient method for unconstrained optimization. The global convergence is established under mild assumptions. Preliminary numerical results in recovering sparse signal are given to show that the proposed method is efficient.