Gradient Method Research Articles

Numerical simulation of shale gas reservoir based on ILU-GMRES method

ABSTRACT The numerical simulation of shale gas reservoirs requires transforming mathematical models into large linear equation systems, and solving large linear equation systems requires a lot of time. A novel ILU-GMRES method for solving large linear equations is presented, which treats the sub matrix discretized at nodes as the basic elements of the coefficient matrix. By combining the incomplete LU decomposition method (ILU) with the generalized minimum residue method (GMRES), the discretized large linear equations are iteratively solved. A comparison was made between the new method and other methods for calculating the actual shale gas reservoir model. The results reveal that the new method has a faster calculation speed in solving linear equation systems. The calculation time of the novel ILU-GMRES method is 1029s, the calculation time of the generalized minimum residual method (GMRES) method is 2709s, the calculation time of the Gauss–Seidel method (GS) is 3786s, and the calculation time of the successive over relaxation method (SOR) method is 3386s, the calculation time of the preprocessing conjugate gradient method (PCG) is 3115s. The calculation speed of the novel method is almost 2.6 times faster than the GMRES method, the novel method is more effective in the numerical simulation of shale gas.

Fast convergence of trust-regions for non-isolated minima via analysis of CG on indefinite matrices

AbstractTrust-region methods (TR) can converge quadratically to minima where the Hessian is positive definite. However, if the minima are not isolated, then the Hessian there cannot be positive definite. The weaker Polyak–Łojasiewicz (PŁ) condition is compatible with non-isolated minima, and it is enough for many algorithms to preserve good local behavior. Yet, TR with an exact subproblem solver lacks even basic features such as a capture theorem under PŁ. In practice, a popular inexact subproblem solver is the truncated conjugate gradient method (tCG). Empirically, TR-tCG exhibits superlinear convergence under PŁ. We confirm this theoretically. The main mathematical obstacle is that, under PŁ, at points arbitrarily close to minima, the Hessian has vanishingly small, possibly negative eigenvalues. Thus, tCG is applied to ill-conditioned, indefinite systems. Yet, the core theory underlying tCG is that of CG, which assumes a positive definite operator. Accordingly, we develop new tools to analyze the dynamics of CG in the presence of small eigenvalues of any sign, for the regime of interest to TR-tCG.

DPNN-ac4C: A Dual-Path Neural Network with self-attention mechanism for identification of N4‑acetylcytidine (ac4C) in mRNA.

The modification of N4-acetylcytidine (ac4C) in RNA is a conserved epigenetic mark that plays a crucial role in post-transcriptional regulation, mRNA stability, and translation efficiency. Traditional methods for detecting ac4C modifications are laborious and costly, necessitating the development of efficient computational approaches for accurate identification of ac4C sites in mRNA. We present DPNN-ac4C, a dual-path neural network with a self-attention mechanism for the identification of ac4C sites in mRNA. Our model integrates embedding modules, bidirectional GRU networks, convolutional neural networks, and self-attention to capture both local and global features of RNA sequences. Extensive evaluations demonstrate that DPNN-ac4C outperforms existing models, achieving an AUROC of 91.03%, accuracy of 82.78%, MCC of 65.78%, and specificity of 84.78% on an independent test set. Moreover, DPNN-ac4C exhibits robustness under the Fast Gradient Method (FGM) attack, maintaining a high level of accuracy in practical applications. The model code and dataset are publicly available on GitHub (https://github.com/shock1ng/DPNN-ac4C).

New hybrid conjugate gradient method for nonlinear optimization with application to image restoration problems

New hybrid conjugate gradient method for nonlinear optimization with application to image restoration problems

Anatomic Interpretability in Neuroimage Deep Learning: Saliency Approaches for Typical Aging and Traumatic Brain Injury.

The black box nature of deep neural networks (DNNs) makes researchers and clinicians hesitant to rely on their findings. Saliency maps can enhance DNN explainability by suggesting the anatomic localization of relevant brain features. This study compares seven popular attribution-based saliency approaches to assign neuroanatomic interpretability to DNNs that estimate biological brain age (BA) from magnetic resonance imaging (MRI). Cognitively normal (CN) adults ( N = 13,394, 5,900 males; mean age: 65.82 ± 8.89 years) are included for DNN training, testing, validation, and saliency map generation to estimate BA. To study saliency robustness to the presence of anatomic deviations from normality, saliency maps are also generated for adults with mild traumatic brain injury (mTBI, \(\:N\) = 214, 135 males; mean age: 55.3 ± 9.9 years). We assess saliency methods' capacities to capture known anatomic features of brain aging and compare them to a surrogate ground truth whose anatomic saliency is known a priori . Anatomic aging features are identified most reliably by the integrated gradients method, which outperforms all others through its ability to localize relevant anatomic features. Gradient Shapley additive explanations, input × gradient, and masked gradient perform less consistently but still highlight ubiquitous neuroanatomic features of aging (ventricle dilation, hippocampal atrophy, sulcal widening). Saliency methods involving gradient saliency, guided backpropagation, and guided gradient-weight class attribution mapping localize saliency outside the brain, which is undesirable. Our research suggests the relative tradeoffs of saliency methods to interpret DNN findings during BA estimation in typical aging and after mTBI.

Analysis of Fractional Order-Adaptive Systems Represented by Error Model 1 Using a Fractional-Order Gradient Approach

In adaptive control, error models use system output error and adaptive laws to update controller parameters for control or identification tasks. Fractional-order calculus, involving non-integer-order derivatives and integrals, is increasingly important for modeling, estimation, and control due to its ability to generalize classical methods and offer improved robustness to disturbances. This paper addresses the gap in the literature where fractional-order gradient methods have not yet been extensively applied in identification and adaptive control schemes. We introduce a fractional-order error model with fractional-order gradient (FOEM1-FG), which integrates fractional gradient operators based on the Caputo fractional derivative. By using theoretical analysis and simulations, we confirm that FOEM1-FG maintains stability and ensures bounded output errors across a variety of input signals. Notably, the fractional gradient’s performance improves as the order, β, increases with β>1, leading to faster convergence. Compared to existing integer-order methods, the proposed approach provides a more flexible and efficient solution in adaptive identification and control schemes. Our results show that FOEM1-FG offers superior stability and convergence characteristics, contributing new insights to the field of fractional calculus in adaptive systems.

High‐temperature thermal conductivity measurement of porous ceramic coatings with a high heat flux CO2 laser rig

AbstractYttrium‐stabilized zirconia (YSZ) plays a crucial role in thermal barrier coatings widely used to protect metallic components in gas turbine engines against high‐temperature combustion product gases and harsh environments. The thermal conductivity of these coatings is a critical parameter influencing the gas turbine design, performance, and service life. In this study, a laser temperature gradient method is utilized to measure the thermal conductivity of porous YSZ coatings. The method employs specialized laser energy delivery to create a one‐dimensional temperature gradient through the sample thickness, closely simulating the operational condition. This approach provides an effective, direct means to determine the thermal conductivity of high‐temperature heat‐resistant porous ceramic materials.

Complex signals parameters optimization on the base of linear approximations using the gradient method and Newton’s method

The article examines the effectiveness of the gradient descent and Newton methods for optimizing the parameters of ensembles of complex signals. Algorithms have been developed and implemented that increase the accuracy of setting parameters and ensure reasonable optimization of spectral, temporal and statistical characteristics of signals. The effectiveness of the application of the methods was confirmed experimentally on the example of reducing the error and increasing the level of immunity. The obtained results substantiate the improvement of the parameters of complex signals, which proves the efficiency of use for wireless telecommunication systems in order to ensure stable and reliable operation in conditions of dynamic changes in the environment and a high level of interference.The article compares the mathematical methods of optimization, namely the gradient method and Newton's method, proposes mathematical models and constructs algorithms that empirically prove the effectiveness of the application of the studied mathematical methods in the specified scientific area - for optimizing the parameters of ensembles of complex signals. Scientific works [1-6, 9, 12] present algorithms based on the gradient method and Newton's method, but they do not consider in detail the comparative analysis of the effectiveness of these methods for optimizing the parameters of ensembles of complex signals for implementation in various scientific and practical tasks. The effectiveness of the algorithms proposed in the article was confirmed experimentally, which made it possible to reduce the error and improve the characteristics of ensembles of complex signals.As a result of the experiments using the methods of gradient descent and Newton, a significant reduction of the error and an improvement of the stability of the signals were achieved. Newton's method reduced the error from 0.1 to 0.0027, justifying the high accuracy of setting the signal parameters. The gradient descent method provided a stable reduction of the gradient norm from 12.75 to less than 1.23, effectively reducing the interference level, i.e. increasing the interference immunity.

The Frank-Wulf method in modeling information systems

Subject of research: the process of choosing the best option for the structure of an information system under given conditions at the design stage. Purpose of research: to increase the efficiency and quality of the validity of decision-making, to reduce time and financial costs when choosing the structure of an information system at the design stage. Methods and objects of research: the object of research is an information system. The problem of nonlinear programming is formulated. A mathematical model of choosing the optimal variant of an information system is presented. By the method of expert assessments, information system projects with point values are determined. Information system projects with maximum values of scores are selected to determine the optimal option. The method of the conditional Frank-Wolf gradient determines the optimal solution of a nonlinear mathematical model. The essence of gradient methods is a sequential change in the value of the objective function when moving from the starting point of the domain of acceptable solutions to the optimal value of the function. The algorithm of the Frank-Wulff method includes the following steps: determining the initial value of the point of the domain of acceptable solutions, calculating the gradient, determining the optimal solution to a linear programming problem, moving to a new point, checking the condition for the end of the iterative process. The iterative process of determining the optimal solution ends when the calculated value of the calculation accuracy is less than the given value. Main results of research: preliminary projects of the information system were determined using the method of expert assessments, the optimal solution of the mathematical model of nonlinear programming by the Frank-Wolf method was found. A mathematical model for determining the optimal design of an information system will improve the efficiency and quality of the validity of decisions made, reduce financial costs and design time of information systems. The results obtained can be used in further research on this topic.

Laplacian edge detection method based on a new approximation of the second-order fractional derivative in the Caputo-Fabrizio sense

Edge detection is an important issue in image processing and computer vision problems. It includes all mathematical methods that aim to identify the discontinuous points in the image. This process leads to construct curves that indicate the boundaries of an object and therefore extract feature information. In recent years, this task has received great attention from researchers and scientists, as we find in the literature many methods with their different algorithms. Up to now, there are two methods, the gradient and Laplacian methods. The first depends on the first derivative, where pixels with a high gradient are considered edges, whereas the second depends on the second derivative, such as the pixel whose second derivative is zero, it is marked as an edge point. Although these methods have achieved remarkable success, they are sensitive to noise and sometimes produce false edges. For this reason, in this paper, we propose an efficient edge detection method based on the fractional Laplacian operator in the Caputo-Fabrizio sense. In this method we develop a new approximation formula to the second-order with the Caputo-Fabrizio fractional derivative. Then, we construct a new fractional mask to compute the Laplacian convolution. To check the effectiveness of our proposed method, we have provided some test images that are affected by a Gaussian noise. The numerical results and visual observation show that our proposed edge detection method gives better performance compared to the classical Laplacian method in terms of the peak signal to noise ratio (psnr) and fine details extraction.

Oracle-Net for Nonlinear Compressed Sensing in Electrical Impedance Tomography Reconstruction Problems

Sparse recovery principles play an important role in solving many nonlinear ill-posed inverse problems. We investigate a variational framework with learned support estimation for compressed sensing sparse reconstructions, where the available measurements are nonlinear and possibly corrupted by noise. A graph neural network, named Oracle-Net, is proposed to predict the support from the nonlinear measurements and is integrated into a regularized recovery model to enforce sparsity. The derived nonsmooth optimization problem is then efficiently solved through a constrained proximal gradient method. Error bounds on the approximate solution of the proposed Oracle-based optimization are provided in the context of the ill-posed Electrical Impedance Tomography problem (EIT). Numerical solutions of the EIT nonlinear inverse reconstruction problem confirm the potential of the proposed method which improves the reconstruction quality from undersampled measurements, under sparsity assumptions.

Deep-Unfolded Tikhonov-Regularized Conjugate Gradient Algorithm for MIMO Detection

In addressing the multifaceted problem of multiple-input multiple-output (MIMO) detection in wireless communication systems, this work highlights the pressing need for enhanced detection reliability under variable channel conditions and MIMO antenna configurations. We propose a novel method that sets a new standard for deep unfolding in MIMO detection by integrating the iterative conjugate gradient method with Tikhonov regularization, combining the adaptability of modern deep learning techniques with the robustness of classical regularization. Unlike conventional techniques, our strategy treats the Tikhonov regularization parameter, as well as the step size and search direction coefficients of the conjugate gradient (CG) method, as trainable parameters within the deep learning framework. This enables dynamic adjustments based on varying channel conditions and MIMO antenna configurations. Detection performance is significantly improved by the proposed approach across a range of MIMO configurations and channel conditions, consistently achieving lower bit error rate (BER) and normalized minimum mean square error (NMSE) compared to well-known techniques like DetNet and CG. The proposed method has superior performance over CG and other model-based methods, especially with a small number of iterations. Consequently, the simulation results demonstrate the flexibility of the proposed approach, making it a viable choice for MIMO systems with a range of antenna configurations, modulation orders, and different channel conditions.

Synthesis and Performance Evaluation of Zwitterionic C-N Bonded Triazole-Tetrazole-Based Primary Explosives.

Developing advanced metal-free nitrogen-enriched primary explosives is challenging due to the inherent risks associated with their synthesis and handling. However, there is an urgent need to develop novel lead-free, nitrogen-rich primary explosives that offer balanced energetic properties. C-N bonded bicyclic compound 3-azido-1-(1H-tetrazol-5-yl)-1H-1,2,4-triazol-5-amine (4), its salts, and 3,5-diazido-1H-1,2,4-triazole (8) were synthesized from inexpensive starting materials resulting in a fine blend of sensitivity and stability. These compounds exhibit high nitrogen content (79.78 to 83.43%), good thermal stability (129-210 °C), excellent detonation performance (VOD: 8592-9361 ms-1, DP: 27.1-33.8 GPa), and acceptable sensitivity (IS: 2.5-30 J, FS: 72-288 N). The hot needle tests of compounds 4 and 8 exhibit excellent ignition performance. All of the newly synthesized compounds were fully characterized using infrared spectroscopy (IR), high-resolution mass spectroscopy (HRMS), multinuclear magnetic spectroscopy (NMR), elemental analysis (EA), thermogravimetric analysis-differential scanning calorimetry (TGA-DSC), and 2, 4, and 8 were confirmed by single-crystal X-ray crystallographic studies. The molecular electrostatic potential (ESP), noncovalent interactions reduced density gradient (NCI-RDG) method, and QTAIM analysis were performed to investigate the intermolecular interactions. Together with promising performance properties, ease of synthesis, and ignitability, they are highly suitable candidates to pave new avenues for future applications.

Towards topology optimisation of poroelastic materials using field-based heuristics

In passive noise control, it is well known that the shapes of acoustic porous material absorbers can be modified to achieve favourable resonances resulting in high sound absorption. However, finding the optimised shapes for a given sound field environment is challenging as it involves solving an inverse problem. Recently, the use of topology optimisation for designing the shapes of acoustic porous material absorbers has gained interest. Previous studies suggest that conventional gradient methods can often result in local optimal results that may be poorer in overall absorption than what can be achieved by using heuristic optimisation techniques. In this study, the use of field variables such as acoustic pressures and velocities in developing better heuristic optimisation methods are explored with the aim of avoiding cumbersome gradient computations. The acoustic shapes produced by the field-based heuristics are compared with traditional approaches to develop insights as to what patterns in the shape designs contribute to improved sound absorption in various target frequencies.

An embedded differential evolution approach to adaptive covariance inflation with the ensemble Kalman filter

ABSTRACT The limited ensemble sizes and the neglected model errors in ensemble-based Data Assimilation (DA) usually lead to underestimation of the error variance, and covariance inflation ‘inflates’ the covariance of a forecast or analysis ensemble by some positive factor in each assimilation cycle. In practice, the value of the inflation factor is often set by trial and error. Thus, a variety of adaptive covariance inflation (ACI) algorithms have been proposed to improve assimilation performance by adjusting the optimal parameters. Most of these used traditional optimization algorithms, such as gradient methods and relaxation methods. However, Evolutionary Algorithms(EAs), which encompass a range of population-based stochastic search techniques widely used in various engineering optimization problems, are rarely seen in DA. In this study, we developed a new framework for adaptive inflation factors, introducing differential evolution (DE) algorithm into local ensemble Transform Kalman filter (LETKF) that can explore the most suitable/optimal inflation factors online. A ‘micro-DE’ based data assimilation method can obtain better assimilation results step by step without taking a much longer time. The feasibility and effectiveness of the algorithm is demonstrated in an idealized Lorenz-96 model with simulated observations. Under both perfect and imperfect-model scenarios, it is found that the ‘micro-DE’LETKF method is capable of outperforming the original LETKF with a considerable lower computational cost. The adaptive inflation factor is optimally updated at each assimilation cycle either around the upper bound or around the lower bound, which means that only an adaptive factor helps mitigate the ensemble collapse due to a lack of spread. However, the further study will consider the application of the methodology to more complex atmospheric models.

A projected hybridization of the Hestenes–Stiefel and Dai–Yuan conjugate gradient methods with application to nonnegative matrix factorization

A projected hybridization of the Hestenes–Stiefel and Dai–Yuan conjugate gradient methods with application to nonnegative matrix factorization

Deep bidirectional hierarchical matrix factorization model for hyperspectral unmixing

Existing deep nonnegative matrix factorization-based approaches treat shallow and deep layers equally or with similar strategies, neglecting the heterogeneous physical structures between the shallow and progressively deeper layers, thus failing to explore the latent different sub-manifold structures in the hyperspectral image. In this paper, we propose a deep nonnegative matrix factorization model with bidirectional constraints to achieve hyperspectral unmixing. The sub-manifold structures in hyperspectral image are fully exploited by filtering and penalizing the shallow abundance layer with a denoised regularizer and a manifold regularizer. In contrast to the shallow abundance layer, the remaining layers are constrained by an extremely common regularizer to avoid over-denoising and maintain fidelity. In this way, the fine cues between different substances are exploited to a large extent. Additionally, the overall reconstruction error can be well controlled because the performance of the designed feedback mechanism can be fine-tuned by the inverse hierarchical constraints. Finally, we employ Nesterov's optimal gradient method to solve the optimization problem effectively. Experiment results are conducted on both synthetic datasets and real datasets, and all results show that the proposed method is superior to recent canonical unmixing methods.

On the Gradient Method in One Portfolio Management Problem

This study refines the methodology for solving stochastic optimal control problems with quality criteria that include the sum of the quality functional of the classical formulation and an extremal measure. A two-level optimization solution of these kinds of problems is presented already for the case where the quality functional consists only of the extremal measure. Our study shows the possibility of solving the original time inconsistency problem through solving a two-level optimization problem, where the outer problem is solved by gradient methods since the value function is convex and the inner problem is solved by classical methods. Some experiments were carried out and confirmed the validity of the theory. The results of the study can be applied to the case of portfolio management with quality criteria containing the Conditional Value-at-Risk (CVaR) metric.

A New Hybrid Descent Algorithm for Large-Scale Nonconvex Optimization and Application to Some Image Restoration Problems

Conjugate gradient methods are widely used and attractive for large-scale unconstrained smooth optimization problems, with simple computation, low memory requirements, and interesting theoretical information on the features of curvature. Based on the strongly convergent property of the Dai–Yuan method and attractive numerical performance of the Hestenes–Stiefel method, a new hybrid descent conjugate gradient method is proposed in this paper. The proposed method satisfies the sufficient descent property independent of the accuracy of the line search strategies. Under the standard conditions, the trust region property and the global convergence are established, respectively. Numerical results of 61 problems with 9 large-scale dimensions and 46 ill-conditioned matrix problems reveal that the proposed method is more effective, robust, and reliable than the other methods. Additionally, the hybrid method also demonstrates reliable results for some image restoration problems.

Improved decision-making through life event prediction: A case study in the financial services industry

Life event prediction is an important tool for customer relationship management (CRM), because life events shift customers’ preferences towards different products and services. Existing life event research mainly uses cross-sectional data, whereas in the CRM field, incorporating longitudinal data is increasingly common. Because longitudinal data can capture the dynamics of customer behavior, opportunities arise to benchmark the power of longitudinal customer data for predictions of cross-sectional versus longitudinal life events. Therefore, this study compares statistical and machine learning (SaML) classifiers, such as logistic regression, random forest, and XGBoost, with long- and short-term memory networks (LSTM), using data represented in both cross-sectional and longitudinal setups for life event prediction. Through a real-life longitudinal customer data set from a European bank, the authors represent the longitudinal data in a cross-sectional data format, using featurization in the form of aggregation. The available data cover 42 end-of-month snapshots for 760,438 unique customers. For marketing decision-making literature, this article (1) introduces three novel life events (i.e., primary, secondary, and rental residence purchases) to life event predictions; (2) offers guidance for how to leverage longitudinal customer data, according to the comparison of various featurization approaches and benchmarking SaML classifiers against LSTM; and (3) clarifies the importance of features and timing for improving marketing decision-making dynamically. The results show that aggregating features over time is preferable as a featurization approach for cross-sectional modeling using SaML classifiers. Furthermore, LSTM can capture behavioral changes over time, unlike SaML classifiers. It also performs significantly better than SaML classifiers on the area under curve and F1 metrics. Insights into the uses of integrated gradients reveal that feature importance changes over time. An integrated gradients method can assist decision-makers in their efforts to plan effective communication with customers in advance, such as by allocating more resources to customers who exhibit high probabilities of a particular life event occurrence.