Conjugate Gradient Descent Method Research Articles

Predicting compressive strength of blended cement concrete with ANNs

Predicting the compressive strength of concrete is important to assess the load-carrying capacity of a structure. However, the use of blended cements to accrue the technical, economic and environmental benefits has increased the complexity of prediction models. Artificial Neural Networks (ANNs) have been used for predicting the compressive strength of ordinary Portland cement concrete, i.e., concrete produced without the addition of supplementary cementing materials. In this study, models to predict the compressive strength of blended cement concrete prepared with a natural pozzolan were developed using regression models and single- and 2-phase learning ANNs. Back-propagation (BP), Levenberg-Marquardt (LM) and Conjugate Gradient Descent (CGD) methods were used for training the ANNs. A 2-phase learning algorithm is proposed for the first time in this study for predictive modeling of the compressive strength of blended cement concrete. The output of these predictive models indicates that the use of a 2-phase learning algorithm will provide better results than the linear regression model or the traditional single-phase ANN models.

Optimal response to non-equilibrium disturbances under truncated Burgers–Hopf dynamics

We model and compute the average response of truncated Burgers–Hopf dynamics to finite perturbations away from the Gibbs equipartition energy spectrum using a dynamical optimization framework recently conceptualized in a series of papers. Non-equilibrium averages are there approximated in terms of geodesic paths in probability space that ‘best-fit’ the Liouvillean dynamics over a family of quasi-equilibrium trial densities. By recasting the geodesic principle as an optimal control problem, we solve numerically for the non-equilibrium responses using an augmented Lagrangian, non-linear conjugate gradient descent method. For moderate perturbations, we find an excellent agreement between the optimal predictions and the direct numerical simulations of the truncated Burgers–Hopf dynamics. In this near-equilibrium regime, we argue that the optimal response theory provides an approximate yet predictive counterpart to fluctuation-dissipation identities.

A sufficient descent conjugate gradient method and its global convergence

In this paper, a DL-type conjugate gradient method is presented. The given method is a modification of the Dai–Liao conjugate gradient method. It can also be considered as a modified LS conjugate gradient method. For general objective functions, the proposed method possesses the sufficient descent condition under the Wolfe line search and is globally convergent. Numerical comparisons show that the proposed algorithm slightly outperforms the PRP+ and CG-descent gradient algorithms as well as the Barzilai–Borwein gradient algorithm.

Linear Convergence of Descent Methods for the Unconstrained Minimization of Restricted Strongly Convex Functions

Linear convergence rates of descent methods for unconstrained minimization are usually proved under the assumption that the objective function is strongly convex. Recently it was shown that the weaker assumption of restricted strong convexity suffices for linear convergence of the ordinary gradient descent method. A decisive difference to strong convexity is that the set of minimizers of a restricted strongly convex function need be neither a singleton nor bounded. In this paper we extend the linear convergence results under this weaker assumption to a larger class of descent methods including restarted nonlinear CG, BFGS, and its damped limited memory variants, L-D-BFGS. For twice continuously differentiable objective functions we even obtain r-step superlinear convergence for the CG_DESCENT conjugate gradient method of Hager and Zhang, where r is greater than or equal to the rank of the Hessian at a minimizer. This is remarkable since the Hessian of a restricted strongly convex function need not have full rank. Furthermore we show that convex quadratic splines and objective functions of the unconstrained duals to some linearly constrained optimization problems are restricted strongly convex. In particular this holds for the regularized basis pursuit problem and its analogues for nuclear norm minimization and principal component pursuit.

Real time dynamic MRI by exploiting spatial and temporal sparsity

Online imaging requires that the reconstruction of current frame only depends on the previous frames, and real time imaging is the desired case. In this work, we propose a novel scheme for real time dynamic magnetic resonance imaging (dMRI) reconstruction. Different from previous methods, the reconstructions of the second frame to the last frame are independent in our scheme, which only require the first frame as the reference image. Therefore, this scheme can be naturally implemented in parallel. After the first frame is reconstructed, all the later frames can be processed as soon as the k-space data are acquired. As an extension of the conventional spatial total variation, a new online model called dynamic total variation is used to exploit the sparsity on both spatial and temporal domains in dMRI. In real time dMRI, each frame is required to be reconstructed very fast. We then design a novel reweighted least squares algorithm to solve the challenging problem. Motivated by the special structure of partial Fourier transform in sparse MRI, this algorithm is accelerated by the preconditioned conjugate gradient descent method. The proposed method is compared with 4 state-of-the-art online and offline methods on two in-vivo cardiac dMRI datasets. The experimental results show that our method significantly outperforms previous online methods, and is comparable to the offline methods in terms of reconstruction accuracy.

Some nonlinear conjugate gradient methods based on spectral scaling secant equations

Three conjugate gradient methods based on the spectral equations are proposed. One is a conjugate gradient method based on the spectral scaling secant equation proposed by Cheng and Li (J Optim Thoery Appl 146:305–319, 2010), which gives the most efficient Dai–Kou conjugate gradient method with sufficient descent in Dai and Kou (SIAM J Optim 23:296–320, 2013) family of conjugate gradient methods. The other is a special case in Dai and Liao (Appl Math Optim 43:87–101, 2001) family of conjugate gradient methods, which adopts a special parameter, an approximation to the spectral of the Hessian of the objective function. Another is a sufficient descent conjugate gradient method, which is based on the second one and can be viewed as a three-term conjugate gradient method based on a spectral scaling secant equation. Convergence properties of the three methods are discussed, and numerical results imply that conjugate gradient methods based on the spectral secant equations perform well, especially when the sufficient descent condition is satisfied.

Application of artificial neural network model in predicting physicochemical characteristics of pharmaceutically developed wafers of loratadine

This study aimed to apply the simultaneous optimization method incorporating artificial neural network (ANN) using multi‐layer perceptron (MLP) model to develop buccoadhesive pharmaceutical wafers containing loratadine with an optimized physicochemical property and drug release. The amount of sodium carboxymethyl cellulose and lactose monohydrate at three levels (−1, 0, +1) for each was selected as casual factors. Bioadhesive strength, disintegration time, percent swelling index and t70% as wafer properties were selected as output variables. Nine buccoadhesive wafers were prepared according to a 32 factorial design and their physicochemical property and dissolution tests were performed. Commercially available Statistica Neural Network Software (Stat Soft, Inc., Tulsa, OK, USA) was used throughout the study. The training process of MLP was completed until a satisfactory value of root mean square for the test data was obtained using back propagation, conjugate gradient descent method. This work exemplifies the probability for an ANN with MLP, to support in development of buccoadhesive wafers with enviable characteristics. Key words: Artificial neural network, buccoadhesive, loratadine, multilayer perceptron, pharmaceutical wafers

A descent extension of the Polak–Ribière–Polyak conjugate gradient method

Based on an eigenvalue analysis, a descent class of two-parameter extension of the conjugate gradient method proposed by Polak and Ribière (1969), and Polyak (1969) is suggested. It is interesting that the one-parameter class of descent conjugate gradient methods proposed by Yu et al. (2008) is a member of the suggested class. Global convergence analysis for the methods of the suggested class is made briefly. Preliminary numerical results are reported; they demonstrate proper choices for the parameters of the suggested class of the conjugate gradient methods that may lead to a promising computational performance.

Norm descent conjugate gradient methods for solving symmetric nonlinear equations

Nonlinear conjugate gradient method is very popular in solving large-scale unconstrained minimization problems due to its simple iterative form and lower storage requirement. In the recent years, it was successfully extended to solve higher-dimension monotone nonlinear equations. Nevertheless, the research activities on conjugate gradient method in symmetric equations are just beginning. This study aims to developing, analyzing, and validating a family of nonlinear conjugate gradient methods for symmetric equations. The proposed algorithms are based on the latest, and state-of-the-art descent conjugate gradient methods for unconstrained minimization. The series of proposed methods are derivative-free, where the Jacobian information is needless at the full iteration process. We prove that the proposed methods converge globally under some appropriate conditions. Numerical results with differentiable parameter's values and performance comparisons with another solver CGD to demonstrate the superiority and effectiveness of the proposed algorithms are reported.

A survey of sufficient descent conjugate gradient methods for unconstrained optimization

In this decade, nonlinear conjugate gradient methods have been focused on as effective numerical methods for solving large-scale unconstrained optimization problems. Especially, nonlinear conjugate gradient methods with the sufficient descent property have been studied by many researchers. In this paper, we review sufficient descent nonlinear conjugate gradient methods.

Ptychographic overlap constraint errors and the limits of their numerical recovery using conjugate gradient descent methods

Ptychographic coherent x-ray diffractive imaging is a form of scanning microscopy that does not require optics to image a sample. A series of scanned coherent diffraction patterns recorded from multiple overlapping illuminated regions on the sample are inverted numerically to retrieve its image. The technique recovers the phase lost by detecting the diffraction patterns by using experimentally known constraints, in this case the measured diffraction intensities and the assumed scan positions on the sample. The spatial resolution of the recovered image of the sample is limited by the angular extent over which the diffraction patterns are recorded and how well these constraints are known. Here, we explore how reconstruction quality degrades with uncertainties in the scan positions. We show experimentally that large errors in the assumed scan positions on the sample can be numerically determined and corrected using conjugate gradient descent methods. We also explore in simulations the limits, based on the signal to noise of the diffraction patterns and amount of overlap between adjacent scan positions, of just how large these errors can be and still be rendered tractable by this method.

An Efficient Approach for Solving Mesh Optimization Problems Using Newton’s Method

We present an efficient approach for solving various mesh optimization problems. Our approach is based on Newton’s method, which uses both first-order (gradient) and second-order (Hessian) derivatives of the nonlinear objective function. The volume and surface mesh optimization algorithms are developed such that mesh validity and surface constraints are satisfied. We also propose several Hessian modification methods when the Hessian matrix is not positive definite. We demonstrate our approach by comparing our method with nonlinear conjugate gradient and steepest descent methods in terms of both efficiency and mesh quality.

Several Guaranteed Descent Conjugate Gradient Methods for Unconstrained Optimization

This paper investigates a general form of guaranteed descent conjugate gradient methods which satisfies the descent conditiongkTdk≤-1-1/4θkgk2 θk>1/4and which is strongly convergent whenever the weak Wolfe line search is fulfilled. Moreover, we present several specific guaranteed descent conjugate gradient methods and give their numerical results for large-scale unconstrained optimization.

Sufficient Descent Conjugate Gradient Methods for Solving Convex Constrained Nonlinear Monotone Equations

Two unified frameworks of some sufficient descent conjugate gradient methods are considered. Combined with the hyperplane projection method of Solodov and Svaiter, they are extended to solve convex constrained nonlinear monotone equations. Their global convergence is proven under some mild conditions. Numerical results illustrate that these methods are efficient and can be applied to solve large-scale nonsmooth equations.

A Modified Conjugate Gradient Method for Unconstrained Optimization

Conjugate gradient methods are an important class of methods for solving unconstrained optimization problems, especially for large-scale problems. Recently, they have been much studied. In this paper, we further study the conjugate gradient method for unconstrained optimization. We focus our attention to the descent conjugate gradient method. This paper presents a modified conjugate gradient method. An interesting feature of the presented method is that the direction is always a descent direction for the objective function. Moreover, the property is independent of the line search used. Under mild conditions, we prove that the modified conjugate gradient method with Armijo-type line search is globally convergent. We also present some numerical results to show the efficiency of the proposed method. DOI: http://dx.doi.org/10.11591/telkomnika.v11i11.2894

Some sufficient descent conjugate gradient methods and their global convergence

Modified Hestense–Stiefel, Polak–Ribiere–Polyak and Liu–Storey conjugate gradient methods are developed using some new techniques. The proposed methods can generate sufficient descent directions without any line search. Under some conditions, global convergence results of the methods are established when the Wolfe or Armijo line search is used. Moreover, the $$r$$ -linear convergence rate of the methods are analyzed. Numerical comparisons are given with some existing conjugate gradient methods using the unconstrained optimization problems in the CUTEr library.

SU‐E‐T‐661: Nonlinear Continuous Method for Non‐Negatively Constrained Inverse Problem of IMRT Planning

Purpose: The intensity‐modulated radiation therapy (IMRT) is an effective therapeutic technique for the treatment of tumors in the human body without inflicting serious damage to normal tissues. IMRT plans include an optimization strategy for minimizing an objective function of radiation beam weights. We present a novel approach for solving non‐negatively constrained inverse problems in IMRT treatment planning, based on the idea of continuous‐time dynamical methods using nonlinear differential equations. Our method can compute a feasible solution without the second derivative of an objective function, which is required for gradient‐based optimization algorithms such as conjugate gradient and steepest descent methods. Methods: We first consider the stability of an equilibrium, which corresponds to the desired planning in the IMRT system, by the use of Lyapunov's stability theorem. Then we perform some phantom experiments using phantom data simulating a clinical setup. Results: We prove theoretically that a Kullback‐Leibler (KL) divergence can be a Lyapunov function for the IMRT planning system, in consistent case. It means that the KL‐divergence measure decreases along the solution to the differential equation. Additionally, we show that the intensity of any radiation beam is not negative. Because the planning system can be created as an analog electronic circuit, its implementation in actual hardware yields dramatically faster planning as a physical phenomenon with real parallel computing, than software‐based iterative methods. Conclusion: The proposed method provides not only the reduction of a computational cost but also no production of a solution with an unphysical negative radiation beam coefficient in solving IMRT planning inverse problems. Moreover the theoretical results on convergence properties of solutions are confirmed by numerical experiments. Based on the results, we found that we can obtain a feasible solution violating the physical dose constraints minimally, even if the system is inconsistent.

Sufficient descent conjugate gradient methods for large-scale optimization problems

Sufficient descent condition is very crucial in establishing the global convergence of nonlinear conjugate gradient method. In this paper, we modified two conjugate gradient methods such that both methods satisfy this property. Under suitable conditions, we prove the global convergence of the proposed methods. Numerical results show that the proposed methods are efficient for the given test problems.

A Hybrid Nonlinear Conjugate Gradient Method with Sufficient Descent Property

In this paper, on the basis of the recently developed HZ (Hager-Zhang) method [SIAM J. Optim., 16 (2005), pp. 170-192], we propose a hybrid descent conjugate gradient method which reserves the sufficient descent property of the HZ method. Under suitable conditions, we prove the global convergence of the proposed method. Extensive numerical experiments show that the method is promising for the test problems from the CUTE library.

Two minimal positive bases based direct search conjugate gradient methods for computationally expensive functions

Positive basis is an important concept in direct search methods. Although any positive basis can ensure the convergence in theory, the maximum positive bases are often used to construct direct search algorithms. In this paper, two direct search methods for computational expensive functions are proposed based on the minimal positive bases. The Coope---Price's frame-based direct search framework is employed to insure convergence. PRP+ method and a recently developed descent conjugate gradient method are employed respectively to accelerate convergence. The data profiles and the performance profiles of the numerical experiments show that the proposed methods are effective for computational expensive functions.