Conjugate Gradient Approach Research Articles

A novel projected Fletcher‐Reeves conjugate gradient approach for finite‐time optimal robust controller of linear constraints optimization problem: Application to bipedal walking robots

SummaryFor finite‐time optimal robust control problem of bipedal walking robot, a class of global and feasible projected Fletcher‐Reeves conjugate gradient approach is proposed based on an online convex optimization algorithm. The optimal robust controllers are solved by projected Fletcher‐Reeves conjugate gradient approach. The approach can rapidly converge to a stable gait cycle by selecting an initial gait. Under some suitable conditions, we provide a rigorous proof of global convergence and well‐defined properties for projected Fletcher‐Reeves conjugate gradient approach. To demonstrate the effectiveness of the bipedal walking robot, we will conduct numerical simulations on the model of 3‐link robot with nonlinear, impulsive, and underactuated dynamics. Furthermore, to indicate the availability of high‐dimensional robotic system, the main result is illustrated on a nonlinear impulsive model of a bipedal walking robot through simulations via finite‐time optimal robust controller. Numerical results show that the projected Fletcher‐Reeves conjugate gradient approach is feasible and effective for bipedal walking robots. Therefore, it is reasonable to infer that the optimal robust control approach can be used in practical systems.

The application of fuzzy logistic equations in population growth with parameter estimation via minimization

This paper presents a numerical solution for the first order fuzzy logistic equations by extended Runge-Kutta fourth order method with estimated parameters. The parameters are estimated by minimization technique using conjugate gradient approach. Then, the fuzzy logistic model with the estimated parameters is used to fit the population growth in Malaysia. Numerical example is given to show the efficiency of the proposed model. Keywords: Fuzzy logistic equations, Population growth, Parameter estimation, Minimization technique

Wiener filter reloaded: fast signal reconstruction without preconditioning

We present a high performance solution to the Wiener filtering problem via a formulation that is dual to the recently developed messenger technique. This new dual messenger algorithm, like its predecessor, efficiently calculates the Wiener filter solution of large and complex data sets without preconditioning and can account for inhomogeneous noise distributions and arbitrary mask geometries. We demonstrate the capabilities of this scheme in signal reconstruction by applying it on a simulated cosmic microwave background (CMB) temperature data set. The performance of this new method is compared to that of the standard messenger algorithm and the preconditioned conjugate gradient (PCG) approach, using a series of well-known convergence diagnostics and their processing times, for the particular problem under consideration. This variant of the messenger algorithm matches the performance of the PCG method in terms of the effectiveness of reconstruction of the input angular power spectrum and converges smoothly to the final solution. The dual messenger algorithm outperforms the standard messenger and PCG methods in terms of execution time, as it runs to completion around 2 and 3-4 times faster than the respective methods, for the specific problem considered.

Impulse noise removal based on new hybrid conjugate gradient approach

Impulse noise removal based on new hybrid conjugate gradient approach

Enhanced Derivative-Free Conjugate Gradient Method for Solving Symmetric Nonlinear Equations

In this article, an enhanced conjugate gradient approach for solving symmetric nonlinear equations is propose without computing the Jacobian matrix. This approach is completely derivative and matrix free. Using classical assumptions the proposed method has global convergence with nonmonotone line search. Some reported numerical results shows the approach is promising.<p style="margin: 0px; text-indent: 0px; -qt-block-indent: 0; -qt-paragraph-type: empty;"> </p>

Simultaneous elastic parameter inversion in 2-D/3-D TTI medium combined later arrival times

Traditional traveltime inversion for anisotropic medium is, in general, based on a “weak” assumption in the anisotropic property, which simplifies both the forward part (ray tracing is performed once only) and the inversion part (a linear inversion solver is possible). But for some real applications, a general (both “weak” and “strong”) anisotropic medium should be considered. In such cases, one has to develop a ray tracing algorithm to handle with the general (including “strong”) anisotropic medium and also to design a non-linear inversion solver for later tomography. Meanwhile, it is constructive to investigate how much the tomographic resolution can be improved by introducing the later arrivals. For this motivation, we incorporated our newly developed ray tracing algorithm (multistage irregular shortest-path method) for general anisotropic media with a non-linear inversion solver (a damped minimum norm, constrained least squares problem with a conjugate gradient approach) to formulate a non-linear inversion solver for anisotropic medium. This anisotropic traveltime inversion procedure is able to combine the later (reflected) arrival times. Both 2-D/3-D synthetic inversion experiments and comparison tests show that (1) the proposed anisotropic traveltime inversion scheme is able to recover the high contrast anomalies and (2) it is possible to improve the tomographic resolution by introducing the later (reflected) arrivals, but not as expected in the isotropic medium, because the different velocity (qP, qSV and qSH) sensitivities (or derivatives) respective to the different elastic parameters are not the same but are also dependent on the inclination angle.

FR type methods for systems of large-scale nonlinear monotone equations

A large class of iterative methods for solving nonlinear monotone systems is developed in recent years. In this paper we propose some new FR type directions in the frame of algorithm which is a combination of conjugate gradient approach and hyperplane projection technique. Derivative-free, function-value-based line search combined with projection procedure is used for globalization strategy. Numerical performances of methods with different search directions are compared.

Application of a GPU-accelerated hybrid preconditioned conjugate gradient approach for large 3D problems in computational geomechanics

This paper presents a hybrid preconditioning technique for Conjugate Gradient method and discusses its parallel implementation on Graphic Processing Unit (GPU) for solving large sparse linear systems arising from application of interior point methods to conic optimization problems in the context of nonlinear Finite Element Limit Analysis (FELA) for computational Geomechanics. For large 3D problems, the use of direct solvers in general becomes prohibitively expensive due to exponentially growing memory requirements and computational time. Besides, the so-called saddle-point systems resulting from use of optimization framework is not an exemption. On the other hand, although preconditioned iterative methods have moderate storage requirements and therefore can be applied to much larger problems than direct methods, they usually exhibit high number of iterations to reach convergence. In present paper, we show that this problem can be effectively tackled using the proposed hybrid preconditioner along with an elaborate implementation on GPU. Furthermore, numerical results verify the robustness and efficiency of the proposed technique.

A new class of efficient and globally convergent conjugate gradient methods in the Dai–Liao family

In this paper, we propose a new conjugate gradient (CG) method which belongs to the CG methods of Dai–Liao family [New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim. 43 (2001), pp. 87–101]. Babaie-Kafaki et al. [Two new conjugate gradient methods based on modified secant equations, J. Comput. Appl. Math. 234 (2010), pp. 1374–1386] made some modifications on the Yabe and Takano's CG approach [Global convergence properties of nonlinear conjugate gradient methods with modified secant condition, Comput. Optim. Appl. 28 (2004), pp. 203–225] and received some appealing results in theory and practice. Here, we introduce an efficient updating rule for the parameters of the Yabe and Takano's CG algorithm. Under some standard assumptions, we establish the global convergence property of the new suggested algorithm on uniformly convex and general functions. Numerical results on some testing problems from CUTEr collection show the priority of the proposed method to some existing CG methods in practice.

On the prediction of graphene’s elastic properties with reactive empirical bond order potentials

The elastic properties of graphene as described by the reactive empirical bond order potential are studied through uniaxial tensile tests calculations at both zero temperature, with a conjugate gradient approach, and room temperature, with molecular dynamics simulations. A perfect linear elastic behavior is observed at 0K up to ≈0.1% strain. The Young’s modulus and Poisson’s ratio obtained with this potential are of ≈730GPa and 0.39, respectively, with little chirality effects. These values differ significantly from former estimations, much closer to experimental values. We show that these former values have certainly been obtained by neglecting the effect of atomic relaxation, leading to a severe inaccuracy. At larger strains, an extended apparent linear domain is observed in the stress–strain curves, which is relevant to Young’s modulus calculations at finite temperature. Our molecular dynamics simulations at 300K have allowed obtaining the following, chirality dependent, apparent Young’s moduli, 860 and 761GPa, and Poisson’s ratios, 0.12 and 0.23, for armchair and zigzag loadings, respectively.

An adaptive preconditioned conjugate gradient approach for extremely large optimisation problems arising in computational geomechanics

The feasibility of preconditioned conjugate gradient (PCG) for solving extremely large sparse linear systems arising in three-dimensional finite element limit analysis of geotechnical stability problems is studied. With problem size increasing beyond few millions of variables, the use of direct solvers becomes prohibitive (both in storage and computational times) even on modern computers. This motivates the study on possible employment of iterative methods to solve systems of equations resulting from interior point method-based limit analysis formulations. In this paper, an adaptive preconditioning technique based on incomplete Cholesky (IC) preconditioner, as well as a novel implementation scheme for preconditioned conjugate gradient method, is proposed to effectively deal with highly ill-conditioned systems, such as those which occur at last iterations of IPM methods. Numerical tests on some ultra-large scale sample geotechnical problems, along with comparison with other potential solution methods, verify the robustness and efficiency of the proposed approach.

Application of Reinforcement Learning Algorithms for the Adaptive Computation of the Smoothing Parameter for Probabilistic Neural Network.

In this paper, we propose new methods for the choice and adaptation of the smoothing parameter of the probabilistic neural network (PNN). These methods are based on three reinforcement learning algorithms: Q(0)-learning, Q(λ)-learning, and stateless Q-learning. We regard three types of PNN classifiers: the model that uses single smoothing parameter for the whole network, the model that utilizes single smoothing parameter for each data attribute, and the model that possesses the matrix of smoothing parameters different for each data variable and data class. Reinforcement learning is applied as the method of finding such a value of the smoothing parameter, which ensures the maximization of the prediction ability. PNN models with smoothing parameters computed according to the proposed algorithms are tested on eight databases by calculating the test error with the use of the cross validation procedure. The results are compared with state-of-the-art methods for PNN training published in the literature up to date and, additionally, with PNN whose sigma is determined by means of the conjugate gradient approach. The results demonstrate that the proposed approaches can be used as alternative PNN training procedures.

Accelerating the cosmic microwave background map-making procedure through preconditioning

Estimation of the sky signal from sequences of time ordered data is one of the key steps in Cosmic Microwave Background (CMB) data analysis, commonly referred to as the map-making problem. Some of the most popular and general methods proposed for this problem involve solving generalised least squares (GLS) equations with non-diagonal noise weights given by a block-diagonal matrix with Toeplitz blocks. In this work we study new map-making solvers potentially suitable for applications to the largest anticipated data sets. They are based on iterative conjugate gradient (CG) approaches enhanced with novel, parallel, two-level preconditioners. We apply the proposed solvers to examples of simulated non-polarised and polarised CMB observations, and a set of idealised scanning strategies with sky coverage ranging from nearly a full sky down to small sky patches. We discuss in detail their implementation for massively parallel computational platforms and their performance for a broad range of parameters characterising the simulated data sets. We find that our best new solver can outperform carefully-optimised standard solvers used today by a factor of as much as 5 in terms of the convergence rate and a factor of up to $4$ in terms of the time to solution, and to do so without significantly increasing the memory consumption and the volume of inter-processor communication. The performance of the new algorithms is also found to be more stable and robust, and less dependent on specific characteristics of the analysed data set. We therefore conclude that the proposed approaches are well suited to address successfully challenges posed by new and forthcoming CMB data sets.

Fast iterative algorithm for the reconstruction of multishot non-cartesian diffusion data.

To accelerate the motion-compensated iterative reconstruction of multishot non-Cartesian diffusion data. The motion-compensated recovery of multishot non-Cartesian diffusion data is often performed using a modified iterative sensitivity-encoded algorithm. Specifically, the encoding matrix is replaced with a combination of nonuniform Fourier transforms and composite sensitivity functions, which account for the motion-induced phase errors. The main challenge with this scheme is the significantly increased computational complexity, which is directly related to the total number of composite sensitivity functions (number of shots × number of coils). The dimensionality of the composite sensitivity functions and hence the number of Fourier transforms within each iteration is reduced using a principal component analysis-based scheme. Using a Toeplitz-based conjugate gradient approach in combination with an augmented Lagrangian optimization scheme, a fast algorithm is proposed for the sparse recovery of diffusion data. The proposed simplifications considerably reduce the computation time, especially in the recovery of diffusion data from under-sampled reconstructions using sparse optimization. By choosing appropriate number of basis functions to approximate the composite sensitivities, faster reconstruction (close to 9 times) with effective motion compensation is achieved. The proposed enhancements can offer fast motion-compensated reconstruction of multishot diffusion data for arbitrary k-space trajectories.

A Hybrid Proposed Imperialist Competitive Algorithm with Conjugate Gradient Approach for Large Scale Global Optimization

This paper presents a novel hybrid imperialist competitive algorithm called ICA-CG algorithm. Such an algorithm combines the evolution ideas of the imperialist competitive algorithm and the classic optimization ideas of the conjugate gradient, based on the compensation for solving the large scale optimization. In the ICA-CG algorithm, the process of every iteration is divided into two stages. In the first stage, the randomly, rapidity and wholeness of the imperialist competitive Algorithm are used. In the second stage, one of the common optimization classical techniques, that called conjugate gradient to move imperialist countries, is used. Experimental results for five well known test problems have shown the superiority of the new ICA-CG algorithm, in large scale optimization, compared with the classical GA, ICA, PSO and ABC algorithms, with regard to the convergence of speed and quality of obtained solutions.

Introducing conjugate gradient optimization for modified HL-RF method

Purpose– Generally, iterative methods which have some instability solutions in complex structural and non-linear mechanical problems are used to compute reliability index. The purpose of this paper is to establish a non-linear conjugate gradient (NCG) optimization algorithm to overcome instability solution of the Hasofer-Lind and Rackwitz-Fiessler (HL-RF) method in first-order reliability analysis. The NCG algorithms such as the Conjugate-Descent (CD) and the Liu-Storey (LS) are used for determining the safety index. An algorithm is found based on the new line search in the reliability analysis.Design/methodology/approach– In the proposed line search for calculating the safety index, search direction is computed by using the conjugate gradient approach and the HL-RF method based on the new and pervious gradient vector of the reliability function. A simple step size is presented for the line search in the proposed algorithm, which is formulated by the Wolfe conditions based on the new and previous safety index results in the reliability analysis.Findings– From the current work, it is concluded that the proposed NCG algorithm has more efficient, robust and appropriate convergence in comparison with the HL-RF method. The proposed methods can eliminate numerical instabilities of the HL-RF iterative algorithm in highly non-linear performance function and complicated structural limit state function. The NGC optimization is applicable to reliability analysis and it is correctly converged on the reliability index. In the NCG method, the CD algorithm is slightly more efficient than the LS algorithm.Originality/value– This paper usefully shows how the HL-RF algorithm and the NCG scheme are formulated in first-order reliability analysis. The proposed algorithm is validated from six numerical and structural examples taken from the literature. The HL-RF method is not converged on several non-linear mathematic and complex structural examples, while the two proposed conjugate gradient methods are appropriately converged for all examples. The CD algorithm is converged about twice faster than the LS algorithm in most of the problems. Therefore, application of the NCG method is possible in reliability analysis.

Fast quantitative susceptibility mapping with L1-regularization and automatic parameter selection.

To enable fast reconstruction of quantitative susceptibility maps with total variation penalty and automatic regularization parameter selection. ℓ(1) -Regularized susceptibility mapping is accelerated by variable splitting, which allows closed-form evaluation of each iteration of the algorithm by soft thresholding and fast Fourier transforms. This fast algorithm also renders automatic regularization parameter estimation practical. A weighting mask derived from the magnitude signal can be incorporated to allow edge-aware regularization. Compared with the nonlinear conjugate gradient (CG) solver, the proposed method is 20 times faster. A complete pipeline including Laplacian phase unwrapping, background phase removal with SHARP filtering, and ℓ(1) -regularized dipole inversion at 0.6 mm isotropic resolution is completed in 1.2 min using MATLAB on a standard workstation compared with 22 min using the CG solver. This fast reconstruction allows estimation of regularization parameters with the L-curve method in 13 min, which would have taken 4 h with the CG algorithm. The proposed method also permits magnitude-weighted regularization, which prevents smoothing across edges identified on the magnitude signal. This more complicated optimization problem is solved 5 times faster than the nonlinear CG approach. Utility of the proposed method is also demonstrated in functional blood oxygen level-dependent susceptibility mapping, where processing of the massive time series dataset would otherwise be prohibitive with the CG solver. Online reconstruction of regularized susceptibility maps may become feasible with the proposed dipole inversion.

Online tracking of deformable objects under occlusion using dominant points

This paper deals with tracking of deformable objects in the presence of occlusion using dominant point representation of the boundary contour. A novel nonintegral time propagation model for propagating the dominant points is proposed. It uses an initial guess generated from a linear operation and an analytical conjugate gradient approach for online robust learning of the shape deformation and motion model. A scheme is presented to automatically detect and correct the region of large local deformation. In order to deal with occlusion, admissible restrictions on deformation and motion of the object are automatically determined. The proposed method overcomes the need of offline learning and learns the deformation and motion model of the object using very few initial frames of the input video. The performance of the method is demonstrated using varieties of videos of different objects.

Analysis Based Blind Compressive Sensing

In this letter, we address the problem of blindly reconstructing compressively sensed signals by exploiting the co-sparse analysis model. In the analysis model it is assumed that a signal multiplied by an analysis operator results in a sparse vector. We propose an algorithm that learns the operator adaptively during the reconstruction process. The arising optimization problem is tackled via a geometric conjugate gradient approach. Different types of sampling noise are handled by simply exchanging the data fidelity term. Numerical experiments are performed for measurements corrupted with Gaussian as well as impulsive noise to show the effectiveness of our method.

Two derivative-free projection approaches for systems of large-scale nonlinear monotone equations

This study proposes two derivative-free approaches for solving systems of large-scale nonlinear equations, where the underlying functions of the systems are continuous and satisfy a monotonicity condition. First, the framework generates a specific direction then employs a backtracking line search along this direction to construct a new point. If the new point solves the problem, the process will be stopped. Under other circumstances, the projection technique constructs an appropriate hyperplane strictly separating the current iterate from the solutions of the problem. Then the projection of the new point onto the hyperplane will determine the next iterate. Thanks to the low memory requirement of derivative-free conjugate gradient approaches, this work takes advantages of two new derivative-free conjugate gradient directions. Under appropriate conditions, the global convergence result of the recommended procedures is established. Preliminary numerical results indicate that the proposed approaches are interesting and remarkably promising.