Conjugate Gradient Research Articles

A NEW CONJUGATE GRADIENT METHOD BASED ON LOGISTIC MAPPING FOR UNCONSTRAINED OPTIMIZATION AND ITS APPLICATION IN REGRESSION ANALYSIS

The study tackles the critical need for efficient optimization techniques in unconstrained optimization problems, where conventional techniques often suffer from slow and inefficient convergence. There is still a need for algorithms that strike a balance between computational efficiency and robustness, despite advancements in gradient-based techniques. This work introduces a novel conjugate gradient algorithm based on the logistic mapping formula. As part of the methodology, descent conditions are established, and the suggested algorithm's global convergence properties are thoroughly examined. Comprehensive numerical experiments are used for empirical validation, and the new algorithm is compared to the Polak-Ribière-Polyak (PRP) algorithm. The suggested approach performs better than the PR algorithm, according to the results, and is more efficient since it needs fewer function evaluations and iterations to reach convergence. Furthermore, the usefulness of the suggested approach is demonstrated by its actual use in regression analysis, notably in the modelling of population estimates for the Kurdistan Region of Iraq. In contrast to conventional least squares techniques, the method maintains low relative error rates while producing accurate predictions. All things considered, this study presents the novel conjugate gradient algorithm as an effective tool for handling challenging optimisation problems in both theoretical and real-world contexts.

Short Paper - Quadratic minimization: from conjugate gradient to an adaptive Polyak’s momentum method with Polyak step-sizes

Short Paper - Quadratic minimization: from conjugate gradient to an adaptive Polyak’s momentum method with Polyak step-sizes

A modified PRP conjugate gradient method with inertial extrapolation for sparse signal reconstruction

It is widely known that the inertial technique of the heavy-ball method can accelerate its convergence speed. In this paper, by embedding the inertial technique in the famous PRP conjugate gradient method, we propose a modified PRP conjugate gradient method with inertial extrapolation (PRPCG-IE) for sparse signal reconstruction. Its direction satisfies the sufficient descent property, which is independent of any line search. Global convergence of PRPCG-IE is established under some standard conditions. PRPCG-IE is applied to two sparse signal reconstruction problems with noise, and preliminary experimental results demonstrate the effectiveness of PRPCG-IE.

A new approach for spatio-temporal interface treatment in fluid–solid interaction using artificial neural networks employing coupled partitioned fluid–solid solvers

Partitioned fluid–solid interaction (FSI) problems involving non-conforming grids pose formidable challenge in interface treatment, especially for information exchange, interface tracking, and field variable interpolation between solvers in both space and time. These demand special considerations for accurate and efficient simulations. This paper presents an application of artificial neural networks (ANN) for the interface treatment in a coupled FSI problem employing partitioned solvers. A shallow time-series ANN (nonlinear auto-regressive model with exogenous inputs, NARX) scheme is proposed to handle the exchange of Neumann/Dirichlet information at the coupling interface. This scheme involves two interface treatment models that were developed and analysed. The proposed models interpolate and transfer loads from the fluid to the solid domains, and conversely, displacements from the solid to the fluid domains between non-collocated grids. To validate this approach, we tested it on a 3D FSI problem, which involved damped oscillations of a flexible flap submerged in a fluid cavity. Adequately trained NARX interface models demonstrate reliable input–output mapping and accurate prediction of transient behaviour at the interface. Additionally, we explored the concept of reduced-order modelling (ROM) in the time domain. This allowed us to reduce the model’s complexity by half. Different training algorithms were evaluated to enhance the efficiency and performance of the proposed scheme. The study demonstrates that NARX networks trained with Bayesian Regularization (BR) and Levenberg–Marquardt (LM) algorithms exhibit the best accuracy, while the scaled conjugate gradient (SCG)-based training method provides better computational efficiency with acceptable accuracy. Overall, the NARX interface models provide precise performance and offer a viable potential for applications in FSI problems requiring accurate and faster computations.

New error estimates for the conjugate gradient method

The conjugate gradient method is the default iterative method for the solution of linear systems of equations with a large symmetric positive definite matrix A. The development of techniques for estimating the norm of the error in iterates computed by this method has received considerable attention. Available methods for bracketing the A-norm of the error evaluate pairs of Gauss and Gauss-Radau quadrature rules to determine lower and upper bounds. The latter methods require a user to allocate a node (the Radau node) between the origin and the smallest eigenvalue of the system matrix. The determination of such a node generally demands further computations to estimate the location of the smallest eigenvalue. An approach that avoids the need for a lower bound of the smallest eigenvalue is to replace the Gauss-Radau quadrature rule by an anti-Gauss rule; see, e.g., Golub and Meurant (1997), Golub and Meurant (2010), Golub and Strakoš (1994), Meurant (1997), Meurant (1999). However, this approach may sometimes yield inaccurate error norm estimates. This paper proposes the use of pairs of Gauss and associated optimal averaged Gauss quadrature rules to estimate the A-norm of the error in iterates determined by the conjugate gradient method.

Overlapping domain decomposition methods for finite volume discretizations

Two-level additive overlapping domain decomposition methods are applied to solve the linear system arising from the cell-centered finite volume discretization methods (FVMs) for the elliptic problems. The conjugate gradient (CG) methods are used to accelerate the convergence. To analyze the preconditioned CG algorithm, a discrete L2 norm, an H1 norm, and an H1 semi-norm are introduced to connect the matrices resulting from the FVMs and related bilinear forms. It has been proved that, with a small overlap, the condition number of the preconditioned systems does not depend on the number of the subdomains. The result is similar to that for the conforming finite element. Numerical experiments confirm the theory.

The inverse problem of identifying complex hyperbolic equation source terms in electromagnetic propagation

Abstract This article investigates the inverse problem of determining the source term of the hyperbolic equation for electromagnetic propagation using terminal data. This study is an important method for identifying propagation sources in electromagnetics. Unlike wave equations, the complexity of the underlying equations can make theoretical analysis quite difficult. Firstly, the uniqueness of the inverse problem was proved using the energy method. Then, based on the optimal control framework, the inverse problem was transformed into an optimal control problem, and the existence of the optimal solution and its necessary conditions were established. Secondly, the global uniqueness and stability of the optimal solution have been proven, which is a completely new conclusion. This has laid a solid theoretical foundation for numerical algorithms. Finally, it is proposed to apply the Landweber iteration method and conjugate gradient method to this problem, and some numerical examples are provided to demonstrate the effectiveness and convergence speed of these two algorithms.

Some combined techniques of spectral conjugate gradient methods with applications to robotic and image restoration models

Some combined techniques of spectral conjugate gradient methods with applications to robotic and image restoration models

Research on wheel–rail impact dynamics due to combined rail weld irregularities and polygonal wheel effects considering 3D contact geometry

The geometric irregularities of rail welds and polygonal wheels are common defects observed in high-speed railways, leading to the generation of high-frequency impact forces and vibrations in the wheel–rail contact system. This research establishes a novel vehicle-track coupled dynamic model that incorporates a 3D wheel–rail contact model using meshing grid and conjugate gradient methods to study the high-magnitude wheel–rail impact forces caused by rail weld irregularities and polygonal wheel combinations. The primary objective of this study is to develop a precise 3D contact model that incorporates the actual geometry of wheel and rail surface irregularities into the calculation of vehicle-track dynamic interactions. The study evaluates the effects of 3D rail weld irregularities and polygonal wheels on wheel–rail dynamic interaction by comparing results with those obtained from a conventional vehicle-track coupled model that considers a 2D contact model. The results show that the lengths and depths of polygonal wheels and rail weld irregularities, as well as vehicle speeds, significantly increase the wheel/rail dynamic impact forces. The results also indicate that the widening of irregularities not only significantly affects the wheel–rail contact force but also has an important influence on the contact stiffness.

A Fourth-Order Tensorial Wiener Filter Using the Conjugate Gradient Method

The recently developed iterative Wiener filter using a fourth-order tensorial (FOT) decomposition owns appealing performance in the identification of long length impulse responses. It relies on the nearest Kronecker product representation (with particular intrinsic symmetry features), together with low-rank approximations. Nevertheless, this new iterative filter requires matrix inversion operations when solving the Wiener–Hopf equations associated with the component filters. In this communication, we propose a computationally efficient version that relies on the conjugate gradient (CG) method for solving these sets of equations. The proposed solution involves a specific initialization of the component filters and sequential connections between the CG cycles. Different FOT-based decomposition setups are also analyzed from the point of view of the resulting parameter space. Experimental results obtained in the context of echo cancellation confirm the good behavior of the proposed approach and its superiority in comparison to the conventional Wiener filter and other decomposition-based versions.

Two-step Fourier single-pixel imaging for secure and efficient hidden information transmission

In the rapidly evolving field of optical information security, single-pixel imaging (SPI) has emerged as a promising technique for hidden information transmission. However, traditional SPI methods face significant challenges, including the need for excessive modulation patterns and the vulnerability of encrypted information during transmission. Furthermore, the field lacks efficient methods to reconstruct both plaintext and ciphertext images from the same set of single-pixel measurements. Here, we propose a novel and efficient encryption strategy for Fourier single-pixel imaging (FSPI) that addresses these critical challenges. Our approach integrates two key innovations: a two-step Fourier-total variation conjugate gradient descent (F-TVCGD) method and a dual-key decryption mechanism. The F-TVCGD method significantly reduces the number of modulation patterns required for image reconstruction, enhancing efficiency and minimizing data redundancy. Our dual-key mechanism enables the reconstruction of both plaintext and ciphertext images from a single set of single-pixel measurements using different decryption keys, significantly enhancing security without compromising efficiency. The incorporation of Fourier symmetric patterns improves the convergence robustness of the symmetric gradient descent (SGD) algorithm, leading to superior performance under challenging conditions such as sparse sampling and noise attacks. Numerical simulations and optical experiments validate our method's improvements in both accuracy and security compared to traditional approaches. Our findings demonstrate that the proposed F-TVCGD and SGD strategies effectively address the challenges of excessive modulation patterns and information vulnerability in FSPI.

Low-Complexity SAOR and Conjugate Gradient Accelerated SAOR Based Signal Detectors for Massive MIMO Systems

Low-Complexity SAOR and Conjugate Gradient Accelerated SAOR Based Signal Detectors for Massive MIMO Systems

A short-term load demand forecasting: Levenberg–Marquardt (LM), Bayesian regularization (BR), and scaled conjugate gradient (SCG) optimization algorithm analysis

A short-term load demand forecasting: Levenberg–Marquardt (LM), Bayesian regularization (BR), and scaled conjugate gradient (SCG) optimization algorithm analysis

Nonlinear finite element formulation for thin-walled conical shells

This study presents a novel finite element formulation to predict the geometrically nonlinear response of conical shells under a wide range of practical loading conditions. The formulation expresses the discretized equilibrium equations in terms of the first Piola-Kirchhoff stress tensor and its conjugate gradient of the virtual displacements, is based on the kinematics of Love-Kirchhoff thin shell theory and the Saint-Venant-Kirchhoff constitutive model, and captures the follower effect of pressure loading. The formulation takes advantage of the axisymmetric nature of the shell geometries by adopting a Fourier series to characterize the displacement distributions along the circumferential direction while using Hermitian interpolation along the meridional direction. Comparisons with general shell models show the accuracy of the formulation under various loading conditions with a minimal number of degrees of freedom, resulting in a significant computational efficiency compared to conventional general-purpose shell solutions.

Compressed sensing with smooth L0 constraints for moving force identification from bridge response measurements

Abstract Compressed sensing (CS), as an emerging information sampling technique, has been successfully applied in the field of moving force identification (MFI). However, existing MFI CS models often fail to obtain the optimal sparse solutions and frequently underestimate the amplitude of local impact forces. To effectively address this issue, a new CS method is proposed for MFI based on smooth L0 norm constraints and bridge response measurements. Firstly, a smooth function is used to approximate the L0 norm, establishing a noise CS reconstruction model for MFI. The introduction of the smoothing function can locally convexify the original MFI problem and enhance the smoothness and differentiability of the objective function, making the optimization problem easier to solve. Subsequently, the Polak–Ribiere–Polyak formula is adopted to point the descent direction of the new objective function, and the sparse solution is iteratively advanced through the conjugate gradient algorithm. Finally, the applicability and feasibility of the proposed method is confirmed by numerical simulations and vehicle–bridge interaction tests, respectively. The results show that the proposed method can accurately identify moving forces from limited measurements of bridge responses. Compared with existing methods, it can provide more precise sparse solutions with higher robustness to measurement noises, and address the issue of underestimating on the amplitude of local impact forces, which is expected to enhance the performance and in-situ applicability of MFI.

Numerical simulation of shale gas reservoir based on ILU-GMRES method

ABSTRACT The numerical simulation of shale gas reservoirs requires transforming mathematical models into large linear equation systems, and solving large linear equation systems requires a lot of time. A novel ILU-GMRES method for solving large linear equations is presented, which treats the sub matrix discretized at nodes as the basic elements of the coefficient matrix. By combining the incomplete LU decomposition method (ILU) with the generalized minimum residue method (GMRES), the discretized large linear equations are iteratively solved. A comparison was made between the new method and other methods for calculating the actual shale gas reservoir model. The results reveal that the new method has a faster calculation speed in solving linear equation systems. The calculation time of the novel ILU-GMRES method is 1029s, the calculation time of the generalized minimum residual method (GMRES) method is 2709s, the calculation time of the Gauss–Seidel method (GS) is 3786s, and the calculation time of the successive over relaxation method (SOR) method is 3386s, the calculation time of the preprocessing conjugate gradient method (PCG) is 3115s. The calculation speed of the novel method is almost 2.6 times faster than the GMRES method, the novel method is more effective in the numerical simulation of shale gas.

Fast convergence of trust-regions for non-isolated minima via analysis of CG on indefinite matrices

AbstractTrust-region methods (TR) can converge quadratically to minima where the Hessian is positive definite. However, if the minima are not isolated, then the Hessian there cannot be positive definite. The weaker Polyak–Łojasiewicz (PŁ) condition is compatible with non-isolated minima, and it is enough for many algorithms to preserve good local behavior. Yet, TR with an exact subproblem solver lacks even basic features such as a capture theorem under PŁ. In practice, a popular inexact subproblem solver is the truncated conjugate gradient method (tCG). Empirically, TR-tCG exhibits superlinear convergence under PŁ. We confirm this theoretically. The main mathematical obstacle is that, under PŁ, at points arbitrarily close to minima, the Hessian has vanishingly small, possibly negative eigenvalues. Thus, tCG is applied to ill-conditioned, indefinite systems. Yet, the core theory underlying tCG is that of CG, which assumes a positive definite operator. Accordingly, we develop new tools to analyze the dynamics of CG in the presence of small eigenvalues of any sign, for the regime of interest to TR-tCG.

New hybrid conjugate gradient method for nonlinear optimization with application to image restoration problems

New hybrid conjugate gradient method for nonlinear optimization with application to image restoration problems

Exploring artificial neural networks for the forward kinematics of a SCARA robotic manipulator using varied datasets and training optimizers

Abstract Although analytical methods are traditionally employed, the solution to the Forward Kinematics (FK) problem for Selective Compliance Assembly Robot Arm (SCARA) manipulator robots can prove intricate and computationally demanding. Recognizing this challenge, this study endeavors to introduce an intelligent approach by leveraging Artificial Neural Networks (ANNs) to address the FK problem specifically tailored for a four-degree-of-freedom (4-DoF) SCARA robot. To train the ANNs, we employ three distinct datasets, one with a fixed step size, one with a random step size, and one based on a sinusoidal signal. Moreover, the objective is to scrutinize the ANNs performance under the influence of three distinct training algorithms: Levenberg-Marquardt (LM), Bayesian Regularization (BR), and Scaled Conjugate Gradient (SCG). Through a systematic comparison of various ANN models, diverse training algorithms, and the three chosen datasets, the investigation reveals that optimal Mean Squared Error (MSE) results are achieved with random step size datasets for models with two hidden layers using the LM algorithm (MSE = 8.6099e-05). For the BR algorithm, the best MSE (8.0535e-05) was obtained with sinusoidal datasets and three hidden layers. For the SCG algorithm, the optimal MSE (1.1144e-04) was achieved with fixed step size datasets and one hidden layer. The accuracy of the ANN model is significantly influenced by the dataset, the choice of training optimizer, and the configuration of hidden layers. Consequently, further research could explore hybrid approaches that integrate evolutionary algorithms to leverage their respective strengths and improve overall ANN model performance.

Deep-Unfolded Tikhonov-Regularized Conjugate Gradient Algorithm for MIMO Detection

In addressing the multifaceted problem of multiple-input multiple-output (MIMO) detection in wireless communication systems, this work highlights the pressing need for enhanced detection reliability under variable channel conditions and MIMO antenna configurations. We propose a novel method that sets a new standard for deep unfolding in MIMO detection by integrating the iterative conjugate gradient method with Tikhonov regularization, combining the adaptability of modern deep learning techniques with the robustness of classical regularization. Unlike conventional techniques, our strategy treats the Tikhonov regularization parameter, as well as the step size and search direction coefficients of the conjugate gradient (CG) method, as trainable parameters within the deep learning framework. This enables dynamic adjustments based on varying channel conditions and MIMO antenna configurations. Detection performance is significantly improved by the proposed approach across a range of MIMO configurations and channel conditions, consistently achieving lower bit error rate (BER) and normalized minimum mean square error (NMSE) compared to well-known techniques like DetNet and CG. The proposed method has superior performance over CG and other model-based methods, especially with a small number of iterations. Consequently, the simulation results demonstrate the flexibility of the proposed approach, making it a viable choice for MIMO systems with a range of antenna configurations, modulation orders, and different channel conditions.