Iterative Gradient Method Research Articles

Tatian polynomial-based annular pupil wavefront optimization method for high-NA extreme ultraviolet lithography

The advanced technology node has an exact error tolerance. The effect of projection objective aberration on imaging cannot be neglected. In order to reduce the effect of aberration, considering the central obscuration of the high-NA extreme ultraviolet projection objective, we propose a Tatian polynomial-based annular pupil wavefront optimization (TBAPWO) method. An active pupil consisting of 64-term Tatian polynomials is incorporated into the imaging model. The suitable coefficients of the Tatian polynomials are solved using the gradient iteration method. The optimized active pupil can improve lithographic imaging quality. Simulation results show that the TBAPWO method can effectively reduce the pattern error of imaging by more than 30%, which indicates that the effect of aberration on imaging is controlled.

Multi-step-length gradient iterative method for separable nonlinear least squares problems

Multi-step-length gradient iterative method for separable nonlinear least squares problems

Robustness and Security in Deep Learning: Adversarial Attacks and Countermeasures

Deep learning models have demonstrated remarkable performance across various domains, yet their susceptibility to adversarial attacks remains a significant concern. In this study, we investigate the effectiveness of three defense mechanisms—Baseline (No Defense), Adversarial Training, and Input Preprocessing—in enhancing the robustness of deep learning models against adversarial attacks. The baseline model serves as a reference point, highlighting the vulnerability of deep learning systems to adversarial perturbations. Adversarial Training, involving the augmentation of training data with adversarial examples, significantly improves model resilience, demonstrating higher accuracy under both Fast Gradient Sign Method (FGSM) and Iterative Gradient Sign Method (IGSM) attacks. Similarly, Input Preprocessing techniques mitigate the impact of adversarial perturbations on model predictions by modifying input data before inference. However, each defense mechanism presents trade-offs in terms of computational complexity and performance. Adversarial Training requires additional computational resources and longer training times, while Input Preprocessing techniques may introduce distortions affecting model generalization. Future research directions may focus on developing more sophisticated defense mechanisms, including ensemble methods, gradient masking, and certified defense strategies, to provide robust and reliable deep learning systems in real-world scenarios. This study contributes to a deeper understanding of defense mechanisms against adversarial attacks in deep learning, highlighting the importance of implementing robust strategies to enhance model resilience.

Friendly Attacks to Improve Channel Coding Reliability

This paper introduces a novel approach called "friendly attack" aimed at enhancing the performance of error correction channel codes. Inspired by the concept of adversarial attacks, our method leverages the idea of introducing slight perturbations to the neural network input, resulting in a substantial impact on the network's performance. By introducing small perturbations to fixed-point modulated codewords before transmission, we effectively improve the decoder's performance without violating the input power constraint. The perturbation design is accomplished by a modified iterative fast gradient method. This study investigates various decoder architectures suitable for computing gradients to obtain the desired perturbations. Specifically, we consider belief propagation (BP) for LDPC codes; the error correcting code transformer, BP and neural BP (NBP) for polar codes, and neural BCJR for convolutional codes. We demonstrate that the proposed friendly attack method can improve the reliability across different channels, modulations, codes, and decoders. This method allows us to increase the reliability of communication with a legacy receiver by simply modifying the transmitted codeword appropriately.

A Minimum Residual Based Gradient Iterative Method for a Class of Matrix Equations

A Minimum Residual Based Gradient Iterative Method for a Class of Matrix Equations

Logarithmic total variation regularization via preconditioned conjugate gradient method for sparse reconstruction of bioluminescence tomography

Background and objectiveBioluminescence Tomography (BLT) is a powerful optical molecular imaging technique that enables the noninvasive investigation of dynamic biological phenomena. It aims to reconstruct the three-dimensional spatial distribution of bioluminescent sources from optical measurements collected on the surface of the imaged object. However, BLT reconstruction is a challenging ill-posed problem due to the scattering effect of light and the limitations in detecting surface photons, which makes it difficult for existing methods to achieve satisfactory reconstruction results. In this study, we propose a novel method for sparse reconstruction of BLT based on a preconditioned conjugate gradient with logarithmic total variation regularization (PCG-logTV). MethodThis PCG-logTV method incorporates the sparsity of overlapping groups and enhances the sparse structure of these groups using logarithmic functions, which can preserve edge features and achieve more stable reconstruction results in BLT. To accelerate the convergence of the algorithm solution, we use the preconditioned conjugate gradient iteration method on the objective function and obtain the reconstruction results. We demonstrate the performance of our proposed method through numerical simulations and in vivo experiment. Results and conclusionsThe results show that the PCG-logTV method obtains the most accurate reconstruction results, and the minimum position error (LE) is 0.254mm, which is 26%, 31% and 34% of the FISTA (0.961), IVTCG (0.81) and L1-TV (0.739) methods, and the root mean square error (RMSE) and relative intensity error (RIE) are the smallest, indicating that it is closest to the real light source. In addition, compared with the other three methods, the PCG-logTV method also has the highest DICE similarity coefficient, which is 0.928, which means that this method can effectively reconstruct the three-dimensional spatial distribution of bioluminescent light sources, has higher resolution and robustness, and is beneficial to the preclinical and clinical studies of BLT.

Fuzzy Adaptive Parameter in the Dai–Liao Optimization Method Based on Neutrosophy

The impact of neutrosophy has increased rapidly in many areas of science and technology in recent years. Furthermore, numerous applications of the neutrosophic theory have become more usual. We aim to use neutrosophy to enhance Dai–Liao conjugate gradient (CG) iterative method. In particular, we suggest and explore a new neutrosophic logic system intended to compute the essential parameter t required in Dai–Liao CG iterations. Theoretical examination and numerical experiments signify the effectiveness of the introduced method for controlling t. By incorporation of the neutrosophy in the Dai–Liao conjugate gradient principle, we established novel Dai–Liao CG iterations for solving large-scale unconstrained optimization problems. Global convergence is proved under standard assumptions and with the use of the inexact line search. Finally, computational evidence shows the computational effectiveness of the proposed fuzzy neutrosophic Dai–Liao CG method.

Boosting Adversarial Attacks with Nadam Optimizer

Deep neural networks are extremely vulnerable to attacks and threats from adversarial examples. These adversarial examples deliberately crafted by attackers can easily fool classification models by adding imperceptibly tiny perturbations on clean images. This brings a great challenge to image security for deep learning. Therefore, studying and designing attack algorithms for generating adversarial examples is essential for building robust models. Moreover, adversarial examples are transferable in that they can mislead multiple different classifiers across models. This makes black-box attacks feasible for practical applications. However, most attack methods have low success rates and weak transferability against black-box models. This is because they often overfit the model during the production of adversarial examples. To address this issue, we propose a Nadam iterative fast gradient method (NAI-FGM), which combines an improved Nadam optimizer with gradient-based iterative attacks. Specifically, we introduce the look-ahead momentum vector and the adaptive learning rate component based on the Momentum Iterative Fast Gradient Sign Method (MI-FGSM). The look-ahead momentum vector is dedicated to making the loss function converge faster and get rid of the poor local maximum. Additionally, the adaptive learning rate component is used to help the adversarial example to converge to a better extreme point by obtaining adaptive update directions according to the current parameters. Furthermore, we also carry out different input transformations to further enhance the attack performance before using NAI-FGM for attack. Finally, we consider attacking the ensemble model. Extensive experiments show that the NAI-FGM has stronger transferability and black-box attack capability than advanced momentum-based iterative attacks. In particular, when using the adversarial examples produced by way of ensemble attack to test the adversarially trained models, the NAI-FGM improves the success rate by 8% to 11% over the other attack methods. Last but not least, the NAI-DI-TI-SI-FGM combined with the input transformation achieves a success rate of 91.3% on average.

Joint Dual-Structural Constrained and Non-negative Analysis Representation Learning for Pattern Classification

ABSTRACT In recent years, analysis dictionary learning (ADL) model has attracted much attention from researchers, owing to its scalability and efficiency in representation-based classification. Despite the supervised label information embedding, the classification performance of analysis representation suffers from the redundant and noisy samples in real-world datasets. In this paper, we propose a joint Dual-Structural constrained and Non-negative Analysis Representation (DSNAR) learning model. First, the supervised latent structural transformation term is considered implicitly to generate a roughly block diagonal representation for intra-class samples. However, this discriminative structure is fragile and weak in the presence of noisy and redundant samples. To highlight both intra-class similarity and inter-class separation for class-oriented representation, we then explicitly incorporate an off-block suppressing term on the ADL model, together with a non-negative representation constraint, to achieve a well-structured and meaningful interpretation of the contributions from all class-oriented atoms. Moreover, a robust classification scheme in latent space is proposed to avoid accidental incorrect predictions with noisy information. Finally, the DSNAR model is alternatively solved by the K-SVD method, iterative re-weighted method and gradient method efficiently. Extensive classification results on five benchmark datasets validate the performance superiority of our DSNAR model compared to other state-of-the-art DL models.

Spline adaptive filtering algorithm based on different iterative gradients: Performance analysis and comparison

Two novel spline adaptive filtering (SAF) algorithms are proposed by combining different iterative gradient methods, i.e., Adagrad and RMSProp, named SAF-Adagrad and SAF-RMSProp, in this paper. Detailed convergence performance and computational complexity analyses are carried out also. Furthermore, compared with existing SAF algorithms, the influence of step-size and noise types on SAF algorithms are explored for nonlinear system identification under artificial datasets. Numerical results show that the SAF-Adagrad and SAF-RMSProp algorithms have better convergence performance than some existing SAF algorithms (i.e., SAF-SGD, SAF-ARC-MMSGD, and SAF-LHC-MNAG). The analysis results of various measured real datasets also verify this conclusion. Overall, the effectiveness of SAF-Adagrad and SAF-RMSProp are confirmed for the accurate identification of nonlinear systems.

Empirijska jednačina mahovog broja kao funkcija odnosa stagnacionog pritiska za kvazijednodimenzionalni stišljivi protok

In the present work for a quasi-one-dimensional isentropic compressible flow model, an empirical equation of the Mach number is constructed as a function of the stagnation pressure ratio for an analytical equation that algebraic procedures cannot invert. The Excel 2019 Solver tool was applied to calibrate the coefficients and exponents of the empirical equation during its construction for the Mach number range from 1 to 10 and 1 to 5. A specific heat ratio from 1.1 to 1.67 and the generalized reduced gradient iterative method were used to minimize the sum of squared error, which was set as the objective function. The results show that for Mach 1 to 10, an error of less than 0.063% is obtained, and for Mach 1 to 5, an error of less than 0.00988% is obtained. It is concluded that the empirical equation obtained is a mathematical model that reproduces the trajectories of the inverted curves of the analytical equation studied.

Spectral Methods in Nonlinear Optics Equations for Non-Uniform Grids Using an Accelerated NFFT Scheme

In this work, we propose the use of non-homogeneous grids in 1D and 2D for the study of various nonlinear physical equations using spectral methods. As is well known, the use of spectral methods allow a faster resolution of the problem via the application of the ubiquitous Fast Fourier Transform (FFT) algorithm. We will center our investigation on the search of fast and accurate schemes to solve the spectral operators in the Fourier space. In particular, we will use the Conjugate Gradient (CG) iterative method, with a preconditioning matrix to accelerate the inversion process of the non-uniform Fast Fourier Transform (NFFT). As it will be shown, the results obtained are in good agreement with the expected values.

Parameter identification of dual-rate Hammerstein-Volterra nonlinear systems by the hybrid particle swarm-gradient algorithm based on the auxiliary model

This paper aims at the parameter estimation of dual-rate Hammerstein-Volterra (DR-HV) systems. The auxiliary model (AM) method is applied to solve the incomplete identification data caused by the dual-rate sampling. Besides, combining the particle swarm optimization (PSO) and the gradient search principle, this paper proposes an improved algorithm — the hybrid particle swarm-gradient based on the auxiliary model (AM-HPSG) algorithm, which is employed to the DR-HV identification. The AM-HPSG method converts the nonlinear identification issue into the optimization problem in the parameter space. Moreover, the application of the gradient mutation strategy in the AM-HPSG algorithm can not only improve the optimization speed and the identification accuracy, but also solve the premature problem of the basic PSO method. To verify the feasibility of the AM-HPSG algorithm, the second-order DR-HV model and the third-order DR-HV models are considered in simulation. And it can be easy to find that the AM-HPSG performs much better than the PSO and the gradient iterative (GI) methods through algorithm comparison.

Comparison of Accelerated Versions of the Iterative Gradient Method to Ameliorate the Spatial Resolution of Microwave Radiometer Products

In this study, the enhancement of the spatial resolution of microwave radiometer measurements is addressed by contrasting the accuracy of a gradient-like antenna pattern deconvolution method with its accelerated versions. The latter are methods that allow reaching a given accuracy with a reduced number of iterations. The analysis points out that accelerated methods result in improved performance when dealing with spot-like discontinuities; while they perform in a similar way to the canonical gradient method in case of large discontinuities. A key application of such techniques is the research on global warming and climate change, which has recently gained critical importance in many scientific fields, mainly due to the huge societal and economic impact of such topics over the entire planet. In this context, the availability of reliable long time series of remotely sensed Earth data is of paramount importance to identify and study climate trends. Such data can be obtained by large-scale sensors, with the obvious drawback of a poor spatial resolution that strongly limits their applicability in regional studies. Iterative gradient techniques allow obtaining super-resolution gridded passive microwave products that can be used in long time series of consistently calibrated brightness temperature maps in support of climate studies.

Factor gradient iterative algorithm for solving a class of discrete periodic Sylvester matrix equations

At present, gradient iteration methods have been used to solve various Sylvester matrix equations and proved effective. Based on this method, we generalize the factor gradient iterative method (FGI) for solving forward periodic Sylvester matrix equations (FPSME) and backward periodic Sylvester matrix equations (BPSME). To accelerate the convergence of the iterative method, we refer to Gauss-Seidel and Jacobi iterative construction ideas and use the latest matrix information in the FGI iterative method to obtain the modified factor gradient iterative (MFGI) method. Then, the convergence of the proposed methods and the selection of optimal factors are proved. The last numerical examples illustrate the effectiveness and applicability of the iterative methods.

Demystifying the Transferability of Adversarial Attacks in Computer Networks

Convolutional Neural Networks (CNNs) models are one of the most frequently used deep learning networks, and extensively used in both academia and industry. Recent studies demonstrated that adversarial attacks against such models can maintain their effectiveness even when used on models other than the one targeted by the attacker. This major property is known as transferability, and makes CNNs ill-suited for security applications. In this paper, we provide the first comprehensive study which assesses the robustness of CNN-based models for computer networks against adversarial transferability. Furthermore, we investigate whether the transferability property issue holds in computer networks applications. In our experiments, we first consider five different attacks: the Iterative Fast Gradient Method (I-FGSM), the Jacobian-based Saliency Map (JSMA), the Limited-memory Broyden Fletcher Goldfarb Shanno BFGS (L-BFGS), the Projected Gradient Descent (PGD), and the DeepFool attack. Then, we perform these attacks against three well-known datasets: the Network-based Detection of IoT (N-BaIoT) dataset, the Domain Generating Algorithms (DGA) dataset, and the RIPE Atlas dataset. Our experimental results show clearly that the transferability happens in specific use cases for the I-FGSM, the JSMA, and the LBFGS attack. In such scenarios, the attack success rate on the target network range from 63.00% to 100%. Finally, we suggest two shielding strategies to hinder the attack transferability, by considering the Most Powerful Attacks (MPAs), and the mismatch LSTM architecture.

Adaptive momentum variance for attention-guided sparse adversarial attacks

The phenomenon that deep neural networks are vulnerable to adversarial examples has been found for several years. Under the black-box setting, transfer-based methods usually produce the adversarial examples on a white-box model, which serves as the surrogate model in the black-box attack, and hope that the same adversarial examples can also fool the black-box model. However, these methods have high success rates for the surrogate model and exhibit weak transferability for the black-box model. In addition, some studies have shown that deep neural networks are also vulnerable to sparse alterations of the input, but existing sparse attacks mainly focus on the number of attacked pixels without restricting the size of the perturbations, which is perceptible to human eyes. To address the above problems, we propose a transfer-based sparse attack method, called adaptive momentum variance based iterative gradient method with a class activation map, where the method considers a simple adaptive momentum variance and a refining perturbation mechanism to improve the transferability of adversarial examples. Also, a class activation map, which is also known as attention mechanism, is employed to explore the relationship between the number of the perturbed pixels and the attack performance in the case of limiting the intensity of perturbation. The proposed method is compared with a number of the state-of-the-art transfer-based adversarial attack methods on the ImageNet dataset, and the empirical results demonstrate that our method achieves a significant increase in transferability with only attacking about 50% of the pixels.

Using incomplete Cholesky factorization to increase the time step in molecular dynamics simulations

Incomplete Cholesky (IC) factorization is widely used as a preconditioner for accelerating the convergence of the conjugate gradient iterative method. The IC factorization has been used in continuum mechanics and other applications that require solutions of elliptic partial differential equations. In this study, we propose an efficient use of the IC factorization to increase the time step and accelerate molecular dynamics simulations. Previously, we proposed the semi-implicit Hessian correction (SimHec) scheme (Washio et al., 2021) for overdamped Langevin dynamics of polymer simulations. SimHec constructs an approximation of the Hessian matrix by superposing the corrected elemental Hessian matrices associated with the interactions within a limited bandwidth along the polymer chain. The resulting narrow-bandwidth system is efficiently solved by an overlapped skyline solver in parallel. In this study, we integrate the non-local interactions in the polymer chain into the corrected Hessian matrix, and approximate the components of the Hessian matrix outside the narrow bandwidth using the IC factorization. This strategy allows us to use time steps that are 150–1000 times larger than those used in the explicit scheme without a severe increase in the computational load.

Power-Optimal Control of a Stirling Engine's Frictional Piston Motion.

The power output of Stirling engines can be optimized by several means. In this study, the focus is on potential performance improvements that can be achieved by optimizing the piston motion of an alpha-Stirling engine in the presence of dissipative processes, in particular mechanical friction. We use a low-effort endoreversible Stirling engine model, which allows for the incorporation of finite heat and mass transfer as well as the friction caused by the piston motion. Instead of performing a parameterization of the piston motion and optimizing these parameters, we here use an indirect iterative gradient method that is based on Pontryagin’s maximum principle. For the varying friction coefficient, the optimization results are compared to both, a harmonic piston motion and optimization results found in a previous study, where a parameterized piston motion had been used. Thus we show how much performance can be improved by using the more sophisticated and numerically more expensive iterative gradient method.

Optimizing the Piston Paths of Stirling Cycle Cryocoolers

Abstract The ideal Stirling cycle provides a clear control strategy for the piston paths of ideal representations of Stirling cycle machines. For non-equilibrium Stirling cycle machines however, piston paths aiming to emulate the ideal cycle’s four strokes will not necessarily yield best performance. In this contribution, we ask the question: What are the COP-optimal piston paths for specific non-equilibrium Stirling cryocoolers? To this end, we consider a low-effort Stirling cryocooler model that consists of a set of coupled ordinary differential equations and takes several loss phenomena into account. For this model and an exemplary parameter set, piston path optimizations are done with an indirect iterative gradient method based on optimal control theory. The optimizations are repeated for two different kinds of volume constraints for the working spaces: one representing an alpha-Stirling configuration, the other a beta-Stirling configuration. Compared to harmonic piston paths, the optimal piston paths lead to significant improvements in COP of ca. 88 % for the alpha-Stirling and ca. 117 % for the beta-Stirling at the maximum-COP operational frequency. Additionally—and even though the optimizations were performed for maximum COP—cooling power was increased with even lager ratios.