Gradient Method Research Articles

Algorithms for optimizing systems with multiple extremum functionals

The problem of minimizing (maximizing) multiple extremum functionals (infinite-dimensional optimization) is considered. This problem cannot be solved by conventional gradient methods. New gradient methods with adaptive relaxation of steps in the vicinity of local extrema are proposed. The efficiency of the proposed methods is demonstrated by the example of optimizing the shape of a hydraulic gun nozzle with respect to the objective functional, which is the average force of the hydraulic pulse jet momentum acting on an obstacle. Two local maxima are found, the second of which is global; in the second maximum, the average force of the jet momentum is three times higher than in the first maximum. The corresponding nozzle shape is optimal. Conventional gradient methods have not found any maximum; i.e., they were unable to solve the problem.

Overlapping domain decomposition methods for finite volume discretizations

Two-level additive overlapping domain decomposition methods are applied to solve the linear system arising from the cell-centered finite volume discretization methods (FVMs) for the elliptic problems. The conjugate gradient (CG) methods are used to accelerate the convergence. To analyze the preconditioned CG algorithm, a discrete L2 norm, an H1 norm, and an H1 semi-norm are introduced to connect the matrices resulting from the FVMs and related bilinear forms. It has been proved that, with a small overlap, the condition number of the preconditioned systems does not depend on the number of the subdomains. The result is similar to that for the conforming finite element. Numerical experiments confirm the theory.

Integrative hybrid deep learning for enhanced breast cancer diagnosis: leveraging the Wisconsin Breast Cancer Database and the CBIS-DDSM dataset

The objective of this investigation was to improve the diagnosis of breast cancer by combining two significant datasets: the Wisconsin Breast Cancer Database and the DDSM Curated Breast Imaging Subset (CBIS-DDSM). The Wisconsin Breast Cancer Database provides a detailed examination of the characteristics of cell nuclei, including radius, texture, and concavity, for 569 patients, of which 212 had malignant tumors. In addition, the CBIS-DDSM dataset—a revised variant of the Digital Database for Screening Mammography (DDSM)—offers a standardized collection of 2,620 scanned film mammography studies, including cases that are normal, benign, or malignant and that include verified pathology data. To identify complex patterns and trait diagnoses of breast cancer, this investigation used a hybrid deep learning methodology that combines Convolutional Neural Networks (CNNs) with the stochastic gradients method. The Wisconsin Breast Cancer Database is used for CNN training, while the CBIS-DDSM dataset is used for fine-tuning to maximize adaptability across a variety of mammography investigations. Data integration, feature extraction, model development, and thorough performance evaluation are the main objectives. The diagnostic effectiveness of the algorithm was evaluated by the area under the Receiver Operating Characteristic Curve (AUC-ROC), sensitivity, specificity, and accuracy. The generalizability of the model will be validated by independent validation on additional datasets. This research provides an accurate, comprehensible, and therapeutically applicable breast cancer detection method that will advance the field. These predicted results might greatly increase early diagnosis, which could promote improvements in breast cancer research and eventually lead to improved patient outcomes.

The inverse problem of identifying complex hyperbolic equation source terms in electromagnetic propagation

Abstract This article investigates the inverse problem of determining the source term of the hyperbolic equation for electromagnetic propagation using terminal data. This study is an important method for identifying propagation sources in electromagnetics. Unlike wave equations, the complexity of the underlying equations can make theoretical analysis quite difficult. Firstly, the uniqueness of the inverse problem was proved using the energy method. Then, based on the optimal control framework, the inverse problem was transformed into an optimal control problem, and the existence of the optimal solution and its necessary conditions were established. Secondly, the global uniqueness and stability of the optimal solution have been proven, which is a completely new conclusion. This has laid a solid theoretical foundation for numerical algorithms. Finally, it is proposed to apply the Landweber iteration method and conjugate gradient method to this problem, and some numerical examples are provided to demonstrate the effectiveness and convergence speed of these two algorithms.

Some combined techniques of spectral conjugate gradient methods with applications to robotic and image restoration models

Some combined techniques of spectral conjugate gradient methods with applications to robotic and image restoration models

A Comprehensive Method for Example-Based Color Transfer with Holistic–Local Balancing and Unit-Wise Riemannian Information Gradient Acceleration

Color transfer, an essential technique in image editing, has recently received significant attention. However, achieving a balance between holistic color style transfer and local detail refinement remains a challenging task. This paper proposes an innovative color transfer method, named BHL, which stands for Balanced consideration of both Holistic transformation and Local refinement. The BHL method employs a statistical framework to address the challenge of achieving a balance between holistic color transfer and the preservation of fine details during the color transfer process. Holistic color transformation is achieved using optimal transport theory within the generalized Gaussian modeling framework. The local refinement module adjusts color and texture details on a per-pixel basis using a Gaussian Mixture Model (GMM). To address the high computational complexity inherent in complex statistical modeling, a parameter estimation method called the unit-wise Riemannian information gradient (uRIG) method is introduced. The uRIG method significantly reduces the computational burden through the second-order acceleration effect of the Fisher information metric. Comprehensive experiments demonstrate that the BHL method outperforms state-of-the-art techniques in both visual quality and objective evaluation criteria, even under stringent time constraints. Remarkably, the BHL method processes high-resolution images in an average of 4.874 s, achieving the fastest processing time compared to the baselines. The BHL method represents a significant advancement in the field of color transfer, offering a balanced approach that combines holistic transformation and local refinement while maintaining efficiency and high visual quality.

Research on wheel–rail impact dynamics due to combined rail weld irregularities and polygonal wheel effects considering 3D contact geometry

The geometric irregularities of rail welds and polygonal wheels are common defects observed in high-speed railways, leading to the generation of high-frequency impact forces and vibrations in the wheel–rail contact system. This research establishes a novel vehicle-track coupled dynamic model that incorporates a 3D wheel–rail contact model using meshing grid and conjugate gradient methods to study the high-magnitude wheel–rail impact forces caused by rail weld irregularities and polygonal wheel combinations. The primary objective of this study is to develop a precise 3D contact model that incorporates the actual geometry of wheel and rail surface irregularities into the calculation of vehicle-track dynamic interactions. The study evaluates the effects of 3D rail weld irregularities and polygonal wheels on wheel–rail dynamic interaction by comparing results with those obtained from a conventional vehicle-track coupled model that considers a 2D contact model. The results show that the lengths and depths of polygonal wheels and rail weld irregularities, as well as vehicle speeds, significantly increase the wheel/rail dynamic impact forces. The results also indicate that the widening of irregularities not only significantly affects the wheel–rail contact force but also has an important influence on the contact stiffness.

On a minimization problem of the maximum generalized eigenvalue: properties and algorithms

AbstractWe study properties and algorithms of a minimization problem of the maximum generalized eigenvalue of symmetric-matrix-valued affine functions, which is nonsmooth and quasiconvex, and has application to eigenfrequency optimization of truss structures. We derive an explicit formula of the Clarke subdifferential of the maximum generalized eigenvalue and prove the maximum generalized eigenvalue is a pseudoconvex function, which is a subclass of a quasiconvex function, under suitable assumptions. Then, we consider smoothing methods to solve the problem. We introduce a smooth approximation of the maximum generalized eigenvalue and prove the convergence rate of the smoothing projected gradient method to a global optimal solution in the considered problem. Also, some heuristic techniques to reduce the computational costs, acceleration and inexact smoothing, are proposed and evaluated by numerical experiments.

A Fourth-Order Tensorial Wiener Filter Using the Conjugate Gradient Method

The recently developed iterative Wiener filter using a fourth-order tensorial (FOT) decomposition owns appealing performance in the identification of long length impulse responses. It relies on the nearest Kronecker product representation (with particular intrinsic symmetry features), together with low-rank approximations. Nevertheless, this new iterative filter requires matrix inversion operations when solving the Wiener–Hopf equations associated with the component filters. In this communication, we propose a computationally efficient version that relies on the conjugate gradient (CG) method for solving these sets of equations. The proposed solution involves a specific initialization of the component filters and sequential connections between the CG cycles. Different FOT-based decomposition setups are also analyzed from the point of view of the resulting parameter space. Experimental results obtained in the context of echo cancellation confirm the good behavior of the proposed approach and its superiority in comparison to the conventional Wiener filter and other decomposition-based versions.

Development of a gradient method for sulfamethoxazole, trimethoprim, isoniazid, and pyridoxine hydrochloride in rabbit plasma through QbD-driven investigation

The current study developed a method for quantifying four drugs—Sulfamethoxazole, Trimethoprim, Isoniazid, and Pyridoxine—in rabbit plasma. The method uses gradient liquid chromatography based on analytical quality by design. To achieve separation, a Eclip Plus C18 (250 mm × 5 mm, 4.6 µm) column with L1 packing was used, and analytes were detected at 254 nm at ambient temperature. The optimized mobile phase consisted of 50 mM potassium dihydrogen phosphate buffer (pH 6.5) and Methanol. The concentration of Methanol was 3% (0–5 min), 15% (5–15 min), 55% (15–27 min), and 3% Methanol until the end of the 30-min runtime, and the flow rate was set at 0.95 mL/min. Control Noise Experimentation was used to screen studies, revealing that flow rate, pH, and Methanol concentration significantly affected the analytical attributes. The study identified critical attributes (resolution and asymmetric factor) and developed a quality target method profile. A central composition design was used to optimize the essential parameters. The method developed for the drugs showed peaks at retention times of 6.990 min for Isoniazid, 7.880 min for Pyridoxine, 15.530 min for Sulfamethoxazole, and 26.890 min for Trimethoprim, respectively. The method was validated with linearity in the range of 10–640 ng ml−1, with R2 of 0.9993, 0.9987, 0.9993, and 0.9992 for Sulfamethoxazole, Trimethoprim, Isoniazid, and Pyridoxine, respectively.

3D Controlled‐source electromagnetic modelling in anisotropic media using secondary potentials and a cascadic multigrid solver

AbstractQuantitative interpretation of the data from controlled‐source electromagnetic methods, whether via forward modelling or inversion, requires solving a considerable number of forward problems, and multigrid methods are often employed to accelerate the solving process. In this study, a new extrapolation cascadic multigrid method is employed to solve the large sparse complex linear system arising from the finite element approximation of Maxwell's equations using secondary potentials. The equations using secondary potentials are discretized by the classic nodal finite element method on nonuniform rectilinear grids. The resulting linear systems are solved by the extrapolation cascadic multigrid method with a new prolongation operator and preconditioned Stabilized bi‐conjugate gradient method smoother. High‐order interpolation and global extrapolation formulas are utilized to construct the multigrid prolongation operator. The extrapolation cascadic multigrid method with the new prolongation operator is easier to implement and more flexible in application than the original one. Finally, several synthetic examples including layered models, models with anisotropic anomalous bodies or layers, are used to validate the accuracy and efficiency of the proposed method. Numerical results show that the extrapolation cascadic multigrid method improves the efficiency of 3D controlled‐source electromagnetic forward modelling a lot, compared with traditional iterative solvers and some state‐of‐the‐art methods or software (e.g., preconditioned flexible generalized minimal residual method, emg3d) in the considered models and grid settings. The efficiency benefit is more evident as the number of unknowns increases, and the proposed method is more efficient at low frequencies. The extrapolation cascadic multigrid method can also be used to solve systems of equations arising from related applications, such as induction logging, airborne electromagnetic, etc.

Random Natural Gradient

Hybrid quantum-classical algorithms appear to be the most promising approach for near-term quantum applications. An important bottleneck is the classical optimization loop, where the multiple local minima and the emergence of barren plateaux make these approaches less appealing. To improve the optimization the Quantum Natural Gradient (QNG) method \cite{stokes2020quantum} was introduced – a method that uses information about the local geometry of the quantum state-space. While the QNG-based optimization is promising, in each step it requires more quantum resources, since to compute the QNG one requires O(m2) quantum state preparations, where m is the number of parameters in the parameterized circuit. In this work we propose two methods that reduce the resources/state preparations required for QNG, while keeping the advantages and performance of the QNG-based optimization. Specifically, we first introduce the Random Natural Gradient (RNG) that uses random measurements and the classical Fisher information matrix (as opposed to the quantum Fisher information used in QNG). The essential quantum resources reduce to linear O(m) and thus offer a quadratic "speed-up", while in our numerical simulations it matches QNG in terms of accuracy. We give some theoretical arguments for RNG and then benchmark the method with the QNG on both classical and quantum problems. Secondly, inspired by stochastic-coordinate methods, we propose a novel approximation to the QNG which we call Stochastic-Coordinate Quantum Natural Gradient that optimizes only a small (randomly sampled) fraction of the total parameters at each iteration. This method also performs equally well in our benchmarks, while it uses fewer resources than the QNG.

Fabrication of a flexible inner prosthetic socket via the FGM technique

Purpose The purpose of this study was to fabricate a flexible inner socket with enhanced stiffness and hardness distribution by using the functional gradient method (FGM). The FGM technique can improve the comfort and flexibility of amputees through the use of a socket that is built via the direct method. Design/methodology/approach Six flexible inner socket samples were fabricated with varying weight fractions of rice husk ash-to-silicone rubber. The tensile strength and hardness of each sample were assessed. Then, numerical analyses were conducted using SOLIDWORKS software to evaluate the pressure distribution on the inner and outer layers of the flexible socket. Findings The hardness and stiffness of the fabricated flexible inner socket gradually increased with the weight ratio of rice husk ash-to-silicone rubber, so when it was in contact with the skin, it approximated the stiffness and hardness of the skin to ensure comfort, and when reaching a higher value in the socket contact layer, it prevented penetration through the flexible inner socket. In addition, the pressure distribution at the external layer of the flexible inner socket has improved. Research limitations/implications A budget of US$500 limited the research to create a flexible inner socket that keeps the socket from penetrating the skin. Practical implications The FGM technique created a flexible inner socket that balances hardness and stiffness to ensure comfort and prevent wounds for its users, lower limb amputees. The commercial value resides in the accessibility of a secure and comfortable flexible inner socket for amputees worldwide, enabling them to overcome the issue of excessive stiffness typically associated with sockets made using the direct method. Originality/value This study introduces the use of FGM to fabricate a flexible inner prosthetic socket with enhanced stiffness and hardness distribution. The approach of using varying weight fractions of rice husk ash-to-silicone rubber to improve the comfort and flexibility of prosthetic sockets is a novel contribution to the field. Given the high stiffness of flexible internal sockets and their ability to maintain flexibility in the part in contact with the skin, such sockets manufactured using this method prevent pain and skin ulcers that previously occurred when sockets are manufactured via the direct method.

Multi-Agent Reinforcement Learning for Smart Community Energy Management

This paper investigates a Local Strategy-Driven Multi-Agent Deep Deterministic Policy Gradient (LSD-MADDPG) method for demand-side energy management systems (EMS) in smart communities. LSD-MADDPG modifies the conventional MADDPG framework by limiting data sharing during centralized training to only discretized strategic information. During execution, it relies solely on local information, eliminating post-training data exchange. This approach addresses critical challenges commonly faced by EMS solutions serving dynamic, increasing-scale communities, such as communication delays, single-point failures, scalability, and nonstationary environments. By leveraging and sharing only strategic information among agents, LSD-MADDPG optimizes decision-making while enhancing training efficiency and safeguarding data privacy—a critical concern in the community EMS. The proposed LSD-MADDPG has proven to be capable of reducing energy costs and flattening the community demand curve by coordinating indoor temperature control and electric vehicle charging schedules across multiple buildings. Comparative case studies reveal that LSD-MADDPG excels in both cooperative and competitive settings by ensuring fair alignment between individual buildings’ energy management actions and community-wide goals, highlighting its potential for advancing future smart community energy management.

Numerical simulation of shale gas reservoir based on ILU-GMRES method

ABSTRACT The numerical simulation of shale gas reservoirs requires transforming mathematical models into large linear equation systems, and solving large linear equation systems requires a lot of time. A novel ILU-GMRES method for solving large linear equations is presented, which treats the sub matrix discretized at nodes as the basic elements of the coefficient matrix. By combining the incomplete LU decomposition method (ILU) with the generalized minimum residue method (GMRES), the discretized large linear equations are iteratively solved. A comparison was made between the new method and other methods for calculating the actual shale gas reservoir model. The results reveal that the new method has a faster calculation speed in solving linear equation systems. The calculation time of the novel ILU-GMRES method is 1029s, the calculation time of the generalized minimum residual method (GMRES) method is 2709s, the calculation time of the Gauss–Seidel method (GS) is 3786s, and the calculation time of the successive over relaxation method (SOR) method is 3386s, the calculation time of the preprocessing conjugate gradient method (PCG) is 3115s. The calculation speed of the novel method is almost 2.6 times faster than the GMRES method, the novel method is more effective in the numerical simulation of shale gas.

Fast convergence of trust-regions for non-isolated minima via analysis of CG on indefinite matrices

AbstractTrust-region methods (TR) can converge quadratically to minima where the Hessian is positive definite. However, if the minima are not isolated, then the Hessian there cannot be positive definite. The weaker Polyak–Łojasiewicz (PŁ) condition is compatible with non-isolated minima, and it is enough for many algorithms to preserve good local behavior. Yet, TR with an exact subproblem solver lacks even basic features such as a capture theorem under PŁ. In practice, a popular inexact subproblem solver is the truncated conjugate gradient method (tCG). Empirically, TR-tCG exhibits superlinear convergence under PŁ. We confirm this theoretically. The main mathematical obstacle is that, under PŁ, at points arbitrarily close to minima, the Hessian has vanishingly small, possibly negative eigenvalues. Thus, tCG is applied to ill-conditioned, indefinite systems. Yet, the core theory underlying tCG is that of CG, which assumes a positive definite operator. Accordingly, we develop new tools to analyze the dynamics of CG in the presence of small eigenvalues of any sign, for the regime of interest to TR-tCG.

DPNN-ac4C: A Dual-Path Neural Network with self-attention mechanism for identification of N4‑acetylcytidine (ac4C) in mRNA.

The modification of N4-acetylcytidine (ac4C) in RNA is a conserved epigenetic mark that plays a crucial role in post-transcriptional regulation, mRNA stability, and translation efficiency. Traditional methods for detecting ac4C modifications are laborious and costly, necessitating the development of efficient computational approaches for accurate identification of ac4C sites in mRNA. We present DPNN-ac4C, a dual-path neural network with a self-attention mechanism for the identification of ac4C sites in mRNA. Our model integrates embedding modules, bidirectional GRU networks, convolutional neural networks, and self-attention to capture both local and global features of RNA sequences. Extensive evaluations demonstrate that DPNN-ac4C outperforms existing models, achieving an AUROC of 91.03%, accuracy of 82.78%, MCC of 65.78%, and specificity of 84.78% on an independent test set. Moreover, DPNN-ac4C exhibits robustness under the Fast Gradient Method (FGM) attack, maintaining a high level of accuracy in practical applications. The model code and dataset are publicly available on GitHub (https://github.com/shock1ng/DPNN-ac4C).

New hybrid conjugate gradient method for nonlinear optimization with application to image restoration problems

New hybrid conjugate gradient method for nonlinear optimization with application to image restoration problems

Anatomic Interpretability in Neuroimage Deep Learning: Saliency Approaches for Typical Aging and Traumatic Brain Injury.

The black box nature of deep neural networks (DNNs) makes researchers and clinicians hesitant to rely on their findings. Saliency maps can enhance DNN explainability by suggesting the anatomic localization of relevant brain features. This study compares seven popular attribution-based saliency approaches to assign neuroanatomic interpretability to DNNs that estimate biological brain age (BA) from magnetic resonance imaging (MRI). Cognitively normal (CN) adults ( N = 13,394, 5,900 males; mean age: 65.82 ± 8.89 years) are included for DNN training, testing, validation, and saliency map generation to estimate BA. To study saliency robustness to the presence of anatomic deviations from normality, saliency maps are also generated for adults with mild traumatic brain injury (mTBI, \(\:N\) = 214, 135 males; mean age: 55.3 ± 9.9 years). We assess saliency methods' capacities to capture known anatomic features of brain aging and compare them to a surrogate ground truth whose anatomic saliency is known a priori . Anatomic aging features are identified most reliably by the integrated gradients method, which outperforms all others through its ability to localize relevant anatomic features. Gradient Shapley additive explanations, input × gradient, and masked gradient perform less consistently but still highlight ubiquitous neuroanatomic features of aging (ventricle dilation, hippocampal atrophy, sulcal widening). Saliency methods involving gradient saliency, guided backpropagation, and guided gradient-weight class attribution mapping localize saliency outside the brain, which is undesirable. Our research suggests the relative tradeoffs of saliency methods to interpret DNN findings during BA estimation in typical aging and after mTBI.

Analysis of Fractional Order-Adaptive Systems Represented by Error Model 1 Using a Fractional-Order Gradient Approach

In adaptive control, error models use system output error and adaptive laws to update controller parameters for control or identification tasks. Fractional-order calculus, involving non-integer-order derivatives and integrals, is increasingly important for modeling, estimation, and control due to its ability to generalize classical methods and offer improved robustness to disturbances. This paper addresses the gap in the literature where fractional-order gradient methods have not yet been extensively applied in identification and adaptive control schemes. We introduce a fractional-order error model with fractional-order gradient (FOEM1-FG), which integrates fractional gradient operators based on the Caputo fractional derivative. By using theoretical analysis and simulations, we confirm that FOEM1-FG maintains stability and ensures bounded output errors across a variety of input signals. Notably, the fractional gradient’s performance improves as the order, β, increases with β>1, leading to faster convergence. Compared to existing integer-order methods, the proposed approach provides a more flexible and efficient solution in adaptive identification and control schemes. Our results show that FOEM1-FG offers superior stability and convergence characteristics, contributing new insights to the field of fractional calculus in adaptive systems.