Conjugate Gradient Research Articles

Overlapping domain decomposition methods for finite volume discretizations

Two-level additive overlapping domain decomposition methods are applied to solve the linear system arising from the cell-centered finite volume discretization methods (FVMs) for the elliptic problems. The conjugate gradient (CG) methods are used to accelerate the convergence. To analyze the preconditioned CG algorithm, a discrete L2 norm, an H1 norm, and an H1 semi-norm are introduced to connect the matrices resulting from the FVMs and related bilinear forms. It has been proved that, with a small overlap, the condition number of the preconditioned systems does not depend on the number of the subdomains. The result is similar to that for the conforming finite element. Numerical experiments confirm the theory.

The inverse problem of identifying complex hyperbolic equation source terms in electromagnetic propagation

Abstract This article investigates the inverse problem of determining the source term of the hyperbolic equation for electromagnetic propagation using terminal data. This study is an important method for identifying propagation sources in electromagnetics. Unlike wave equations, the complexity of the underlying equations can make theoretical analysis quite difficult. Firstly, the uniqueness of the inverse problem was proved using the energy method. Then, based on the optimal control framework, the inverse problem was transformed into an optimal control problem, and the existence of the optimal solution and its necessary conditions were established. Secondly, the global uniqueness and stability of the optimal solution have been proven, which is a completely new conclusion. This has laid a solid theoretical foundation for numerical algorithms. Finally, it is proposed to apply the Landweber iteration method and conjugate gradient method to this problem, and some numerical examples are provided to demonstrate the effectiveness and convergence speed of these two algorithms.

Some combined techniques of spectral conjugate gradient methods with applications to robotic and image restoration models

Some combined techniques of spectral conjugate gradient methods with applications to robotic and image restoration models

Research on wheel–rail impact dynamics due to combined rail weld irregularities and polygonal wheel effects considering 3D contact geometry

The geometric irregularities of rail welds and polygonal wheels are common defects observed in high-speed railways, leading to the generation of high-frequency impact forces and vibrations in the wheel–rail contact system. This research establishes a novel vehicle-track coupled dynamic model that incorporates a 3D wheel–rail contact model using meshing grid and conjugate gradient methods to study the high-magnitude wheel–rail impact forces caused by rail weld irregularities and polygonal wheel combinations. The primary objective of this study is to develop a precise 3D contact model that incorporates the actual geometry of wheel and rail surface irregularities into the calculation of vehicle-track dynamic interactions. The study evaluates the effects of 3D rail weld irregularities and polygonal wheels on wheel–rail dynamic interaction by comparing results with those obtained from a conventional vehicle-track coupled model that considers a 2D contact model. The results show that the lengths and depths of polygonal wheels and rail weld irregularities, as well as vehicle speeds, significantly increase the wheel/rail dynamic impact forces. The results also indicate that the widening of irregularities not only significantly affects the wheel–rail contact force but also has an important influence on the contact stiffness.

A Fourth-Order Tensorial Wiener Filter Using the Conjugate Gradient Method

The recently developed iterative Wiener filter using a fourth-order tensorial (FOT) decomposition owns appealing performance in the identification of long length impulse responses. It relies on the nearest Kronecker product representation (with particular intrinsic symmetry features), together with low-rank approximations. Nevertheless, this new iterative filter requires matrix inversion operations when solving the Wiener–Hopf equations associated with the component filters. In this communication, we propose a computationally efficient version that relies on the conjugate gradient (CG) method for solving these sets of equations. The proposed solution involves a specific initialization of the component filters and sequential connections between the CG cycles. Different FOT-based decomposition setups are also analyzed from the point of view of the resulting parameter space. Experimental results obtained in the context of echo cancellation confirm the good behavior of the proposed approach and its superiority in comparison to the conventional Wiener filter and other decomposition-based versions.

Two-step Fourier single-pixel imaging for secure and efficient hidden information transmission

In the rapidly evolving field of optical information security, single-pixel imaging (SPI) has emerged as a promising technique for hidden information transmission. However, traditional SPI methods face significant challenges, including the need for excessive modulation patterns and the vulnerability of encrypted information during transmission. Furthermore, the field lacks efficient methods to reconstruct both plaintext and ciphertext images from the same set of single-pixel measurements. Here, we propose a novel and efficient encryption strategy for Fourier single-pixel imaging (FSPI) that addresses these critical challenges. Our approach integrates two key innovations: a two-step Fourier-total variation conjugate gradient descent (F-TVCGD) method and a dual-key decryption mechanism. The F-TVCGD method significantly reduces the number of modulation patterns required for image reconstruction, enhancing efficiency and minimizing data redundancy. Our dual-key mechanism enables the reconstruction of both plaintext and ciphertext images from a single set of single-pixel measurements using different decryption keys, significantly enhancing security without compromising efficiency. The incorporation of Fourier symmetric patterns improves the convergence robustness of the symmetric gradient descent (SGD) algorithm, leading to superior performance under challenging conditions such as sparse sampling and noise attacks. Numerical simulations and optical experiments validate our method's improvements in both accuracy and security compared to traditional approaches. Our findings demonstrate that the proposed F-TVCGD and SGD strategies effectively address the challenges of excessive modulation patterns and information vulnerability in FSPI.

Low-Complexity SAOR and Conjugate Gradient Accelerated SAOR Based Signal Detectors for Massive MIMO Systems

Low-Complexity SAOR and Conjugate Gradient Accelerated SAOR Based Signal Detectors for Massive MIMO Systems

A short-term load demand forecasting: Levenberg–Marquardt (LM), Bayesian regularization (BR), and scaled conjugate gradient (SCG) optimization algorithm analysis

A short-term load demand forecasting: Levenberg–Marquardt (LM), Bayesian regularization (BR), and scaled conjugate gradient (SCG) optimization algorithm analysis

Compressed sensing with smooth L0 constraints for moving force identification from bridge response measurements

Abstract Compressed sensing (CS), as an emerging information sampling technique, has been successfully applied in the field of moving force identification (MFI). However, existing MFI CS models often fail to obtain the optimal sparse solutions and frequently underestimate the amplitude of local impact forces. To effectively address this issue, a new CS method is proposed for MFI based on smooth L0 norm constraints and bridge response measurements. Firstly, a smooth function is used to approximate the L0 norm, establishing a noise CS reconstruction model for MFI. The introduction of the smoothing function can locally convexify the original MFI problem and enhance the smoothness and differentiability of the objective function, making the optimization problem easier to solve. Subsequently, the Polak–Ribiere–Polyak formula is adopted to point the descent direction of the new objective function, and the sparse solution is iteratively advanced through the conjugate gradient algorithm. Finally, the applicability and feasibility of the proposed method is confirmed by numerical simulations and vehicle–bridge interaction tests, respectively. The results show that the proposed method can accurately identify moving forces from limited measurements of bridge responses. Compared with existing methods, it can provide more precise sparse solutions with higher robustness to measurement noises, and address the issue of underestimating on the amplitude of local impact forces, which is expected to enhance the performance and in-situ applicability of MFI.

Numerical simulation of shale gas reservoir based on ILU-GMRES method

ABSTRACT The numerical simulation of shale gas reservoirs requires transforming mathematical models into large linear equation systems, and solving large linear equation systems requires a lot of time. A novel ILU-GMRES method for solving large linear equations is presented, which treats the sub matrix discretized at nodes as the basic elements of the coefficient matrix. By combining the incomplete LU decomposition method (ILU) with the generalized minimum residue method (GMRES), the discretized large linear equations are iteratively solved. A comparison was made between the new method and other methods for calculating the actual shale gas reservoir model. The results reveal that the new method has a faster calculation speed in solving linear equation systems. The calculation time of the novel ILU-GMRES method is 1029s, the calculation time of the generalized minimum residual method (GMRES) method is 2709s, the calculation time of the Gauss–Seidel method (GS) is 3786s, and the calculation time of the successive over relaxation method (SOR) method is 3386s, the calculation time of the preprocessing conjugate gradient method (PCG) is 3115s. The calculation speed of the novel method is almost 2.6 times faster than the GMRES method, the novel method is more effective in the numerical simulation of shale gas.

Fast convergence of trust-regions for non-isolated minima via analysis of CG on indefinite matrices

AbstractTrust-region methods (TR) can converge quadratically to minima where the Hessian is positive definite. However, if the minima are not isolated, then the Hessian there cannot be positive definite. The weaker Polyak–Łojasiewicz (PŁ) condition is compatible with non-isolated minima, and it is enough for many algorithms to preserve good local behavior. Yet, TR with an exact subproblem solver lacks even basic features such as a capture theorem under PŁ. In practice, a popular inexact subproblem solver is the truncated conjugate gradient method (tCG). Empirically, TR-tCG exhibits superlinear convergence under PŁ. We confirm this theoretically. The main mathematical obstacle is that, under PŁ, at points arbitrarily close to minima, the Hessian has vanishingly small, possibly negative eigenvalues. Thus, tCG is applied to ill-conditioned, indefinite systems. Yet, the core theory underlying tCG is that of CG, which assumes a positive definite operator. Accordingly, we develop new tools to analyze the dynamics of CG in the presence of small eigenvalues of any sign, for the regime of interest to TR-tCG.

New hybrid conjugate gradient method for nonlinear optimization with application to image restoration problems

New hybrid conjugate gradient method for nonlinear optimization with application to image restoration problems

Exploring artificial neural networks for the forward kinematics of a SCARA robotic manipulator using varied datasets and training optimizers

Abstract Although analytical methods are traditionally employed, the solution to the Forward Kinematics (FK) problem for Selective Compliance Assembly Robot Arm (SCARA) manipulator robots can prove intricate and computationally demanding. Recognizing this challenge, this study endeavors to introduce an intelligent approach by leveraging Artificial Neural Networks (ANNs) to address the FK problem specifically tailored for a four-degree-of-freedom (4-DoF) SCARA robot. To train the ANNs, we employ three distinct datasets, one with a fixed step size, one with a random step size, and one based on a sinusoidal signal. Moreover, the objective is to scrutinize the ANNs performance under the influence of three distinct training algorithms: Levenberg-Marquardt (LM), Bayesian Regularization (BR), and Scaled Conjugate Gradient (SCG). Through a systematic comparison of various ANN models, diverse training algorithms, and the three chosen datasets, the investigation reveals that optimal Mean Squared Error (MSE) results are achieved with random step size datasets for models with two hidden layers using the LM algorithm (MSE = 8.6099e-05). For the BR algorithm, the best MSE (8.0535e-05) was obtained with sinusoidal datasets and three hidden layers. For the SCG algorithm, the optimal MSE (1.1144e-04) was achieved with fixed step size datasets and one hidden layer. The accuracy of the ANN model is significantly influenced by the dataset, the choice of training optimizer, and the configuration of hidden layers. Consequently, further research could explore hybrid approaches that integrate evolutionary algorithms to leverage their respective strengths and improve overall ANN model performance.

Deep-Unfolded Tikhonov-Regularized Conjugate Gradient Algorithm for MIMO Detection

In addressing the multifaceted problem of multiple-input multiple-output (MIMO) detection in wireless communication systems, this work highlights the pressing need for enhanced detection reliability under variable channel conditions and MIMO antenna configurations. We propose a novel method that sets a new standard for deep unfolding in MIMO detection by integrating the iterative conjugate gradient method with Tikhonov regularization, combining the adaptability of modern deep learning techniques with the robustness of classical regularization. Unlike conventional techniques, our strategy treats the Tikhonov regularization parameter, as well as the step size and search direction coefficients of the conjugate gradient (CG) method, as trainable parameters within the deep learning framework. This enables dynamic adjustments based on varying channel conditions and MIMO antenna configurations. Detection performance is significantly improved by the proposed approach across a range of MIMO configurations and channel conditions, consistently achieving lower bit error rate (BER) and normalized minimum mean square error (NMSE) compared to well-known techniques like DetNet and CG. The proposed method has superior performance over CG and other model-based methods, especially with a small number of iterations. Consequently, the simulation results demonstrate the flexibility of the proposed approach, making it a viable choice for MIMO systems with a range of antenna configurations, modulation orders, and different channel conditions.

Performance Modeling of the Fenton Process Used as a Single Unit for Treating Raw Textile Effluent

This study investigates the direct application of the Fenton Oxidation Process (FOP) to untreated textile wastewater, specifically from a woven fabric production facility. Under optimized conditions (pH 3, 0.7 g/L Fe+2, 2 mM H2O2), the process achieved significant removal efficiencies: 81% Chemical Oxygen Demand (COD), 80% Suspended Solids (SS), and 93% color removal. Artificial Neural Networks (ANN) and NARX-ANN were utilized in Matlab R2020a to model FOP performance, employing Levenberg-Marquardt (trainlm) and Scaled Conjugate Gradient (trainscg) algorithms. With a 9-20-3 network topology, the ANN model demonstrated superior predictive capability, achieving an R2 of 0.9843.

A projected hybridization of the Hestenes–Stiefel and Dai–Yuan conjugate gradient methods with application to nonnegative matrix factorization

A projected hybridization of the Hestenes–Stiefel and Dai–Yuan conjugate gradient methods with application to nonnegative matrix factorization

A New Hybrid Descent Algorithm for Large-Scale Nonconvex Optimization and Application to Some Image Restoration Problems

Conjugate gradient methods are widely used and attractive for large-scale unconstrained smooth optimization problems, with simple computation, low memory requirements, and interesting theoretical information on the features of curvature. Based on the strongly convergent property of the Dai–Yuan method and attractive numerical performance of the Hestenes–Stiefel method, a new hybrid descent conjugate gradient method is proposed in this paper. The proposed method satisfies the sufficient descent property independent of the accuracy of the line search strategies. Under the standard conditions, the trust region property and the global convergence are established, respectively. Numerical results of 61 problems with 9 large-scale dimensions and 46 ill-conditioned matrix problems reveal that the proposed method is more effective, robust, and reliable than the other methods. Additionally, the hybrid method also demonstrates reliable results for some image restoration problems.

A Hybrid Sobolev Gradient Method for Learning NODEs

The inverse problem of supervised reconstruction of depth-variable (time-dependent) parameters in ordinary differential equations is considered, with the typical application of finding weights of a neural ordinary differential equation (NODE) for a residual network with time continuous layers. The differential equation is treated as an abstract and isolated entity, termed a standalone NODE (sNODE), to facilitate for a wide range of applications. The proposed parameter reconstruction is performed by minimizing a cost functional covering a variety of loss functions and penalty terms. Regularization via penalty terms is incorporated to enhance ethical and trustworthy AI formulations. A nonlinear conjugate gradient mini-batch optimization scheme (NCG) is derived for the training having the benefit of including a sensitivity problem. The model (differential equation)-based approach is thus combined with a data-driven learning procedure. Mathematical properties are stated for the differential equation and the cost functional. The adjoint problem needed is derived together with the sensitivity problem. The sensitivity problem itself can estimate changes in the output under perturbation of the trained parameters. To preserve smoothness during the iterations, the Sobolev gradient is calculated and incorporated. Numerical results are included to validate the procedure for a NODE and synthetic datasets and compared with standard gradient approaches. For stability, using the sensitivity problem, a strategy for adversarial attacks is constructed, and it is shown that the given method with Sobolev gradients is more robust than standard approaches for parameter identification.

A Modified Wei-Yao-Liu Stochastic Conjugate Gradient Algorithm in Machine Learning

Abstract The Wei-Yao-Liu (WYL) Conjugate Gradient (CG) algorithm exhibits favourable attributes, notably sufficient descent and trust domain characteristics, in the context of solving unconstrained optimization problems. The exploration and enhancement of computational methodologies for stochastic optimization problems have garnered significant attention. In this study, our objective is to improve the Wei-Yao-Liu (WYL) CG algorithm and introduce an effective optimization method tailored for solving stochastic optimization problems. We will present a comprehensive framework for analysing the convergence of this algorithm and, under suitable conditions, establish its sufficient descent properties, trust region features, and global convergence of stationary points. We will also give an improved CG algorithm to calculate a framework for variance reduction. Additionally, we will conduct experimental comparisons to showcase the competitiveness of our proposed algorithm.

A Modified Three-terms CG Method with the MWWP Line Search and its Application to Image Restoration

Abstract In this article, an innovative method, which integrates the optimized three-term Polak–Ribière–Polyak (PRP) conjugate gradient (CG) method with the improved WWP line search rule, is presented and applied in the field of image restoration. The proposed search direction exhibits sufficient descent and trust region characteristics, maintaining its effectiveness regardless of the specific line search rule employed. We established the global convergence of the presented algorithm and eliminated the dependence on the Lipschitz continuity assumption. The numerical comparisons with the PRP CG method revealed the superior performance of the designed algorithm in the field of image restoration. This research outcome provides strong support for further advancements in the field of image recovery.