Optimal Step Size Research Articles

An enhanced neighborhood differential method for potential landslide identification from stacking-InSAR results

Stacking-InSAR is widely used for identifying potential landslide. However, in Stacking-InSAR results, there are often challenges to clearly identify the potential landslide due to wide-scale system errors, especially the elevation-related atmospheric disturbances, leading to low precision of potential landslide identification. This paper introduces an enhanced neighborhood differential method that employs a concept of step size to identify potential landslide from Stacking-InSAR. Applied to the mountainous regions in the northwestern part of Sichuan Province, China, this method successfully identified 117 potential landslides, verified through field investigations. Compared to traditional Stacking-InSAR, this method additionally identified 34 potential landslides. Further analysis indicates that the optimal step size in the neighborhood differential method should ensure that one differential pixel is located inside the potential landslide area while the other is outside, thereby making the step size closely related to the spatial extent of the potential landslide. The results indicate that this method could effectively reduce wide-scale system error in Stacking-InSAR results, providing an important technical reference for the rapid and efficient identification of potential landslide over a wide mountainous area in the future.

Inverse Estimation of Swirl Cooling Heat Transfer Coefficient

Swirl cooling is one of the most widely studied techniques in the field of heat transfer enhancement. This paper presents a novel, non-intrusive, and rather inexpensive inverse method to estimate the local heat transfer coefficient along the swirl chamber. First, the direct problem of two-dimensional heat conduction inside the chamber’s wall is solved numerically. The optimal number of sensors, their axial and radial positions, and other parameters concerning the inverse algorithm are determined using simulated experiments. The experimental setup is built based on these parameters. Transient temperature readings from several thermocouples embedded inside the chamber wall are fed to an inverse algorithm based on the conjugate gradient method. This algorithm takes advantage of an adjoint differential equation and the sensitivity differential equation to calculate the gradients and optimal step sizes, respectively. The local transient heat transfer coefficient is estimated for three Reynolds numbers along four straight lines parallel to the axis of the chamber.

Use estimated signal and noise to adjust step size for image restoration

Image deblurring is a challenging inverse problem, especially when there is additive noise to the observation. To solve such an inverse problem in an iterative manner, it is important to control the step size for achieving a stable and robust performance. We designed a method that controls the progress of iterative process in solving the inverse problem without the need for a user-specified step size. The method searches for an optimal step size under the assumption that the signal and noise are two independent stochastic processes. Experiments show that the method can achieve good performance in the presence of noise and imperfect knowledge about the blurring kernel. Tests also show that, for different blurring kernels and noise levels, the difference between two consecutive estimates given by the new method tends to remain more stable and stay in a smaller range, as compared to those given by some existing techniques. This stable feature makes the new method more robust in the sense that it is easier to select a stopping threshold for the new method to use in different scenarios.

Random Learning Leads to Faster Convergence in ‘Model‐Free’ ILC: With Application to MIMO Feedforward in Industrial Printing

ABSTRACTModel‐free iterative learning control (ILC) can lead to high performance by attenuating repeating disturbances completely, using dedicated experiments on the real system to replace the traditional model. The aim of this paper is to develop a fast data‐driven method for MIMO ILC that uses random learning in the form of efficient unbiased gradient estimates. This is achieved by developing a stochastic conjugate gradient algorithm, in which the search direction and optimal step size are generated using dedicated experiments. The approach is applied to MIMO automated feedforward tuning. Simulation and experimental results show that the method is superior to earlier stochastic and deterministic methods.

A New Method Based on Locally Optimal Step Length in Accelerated Gradient Descent for Quantum State Tomography.

Quantum state tomography (QST) is one of the key steps in determining the state of the quantum system, which is essential for understanding and controlling it. With statistical data from measurements and Positive Operator-Valued Measures (POVMs), the goal of QST is to find a density operator that best fits the measurement data. Several optimization-based methods have been proposed for QST, and one of the most successful approaches is based on Accelerated Gradient Descent (AGD) with fixed step length. While AGD with fixed step size is easy to implement, it is computationally inefficient when the computational time required to calculate the gradient is high. In this paper, we propose a new optimal method for step-length adaptation, which results in a much faster version of AGD for QST. Numerical results confirm that the proposed method is much more time-efficient than other similar methods due to the optimized step size.

Bi-affine scaling iterative method for convex quadratic programming with bound constraints

To solve general convex quadratic programming problems with bound constraints, this paper proposes a new interior point iterative method that is easy to be implemented. The method exhibits a simple and sufficiently smooth search direction, and possesses the characteristics of affine scaling. Under the limited optimal stepsize rule, starting from an arbitrary interior point, any accumulation point of the generated sequence is an optimal solution of the corresponding problem. Furthermore, due to the absence of introducing dual variables and solving equations, the proposed method is more suitable for solving large-scale problems. Preliminary numerical results indicate that the new method has advantages in terms of both efficiency and accuracy.

The spectral conjugate gradient method in variational adjoint assimilation for model terrain correction III: convergence analysis

Abstract In the former parts of this paper, the spectral conjugate gradient (SCG) algorithm was proposed to identify the optimal bottom terrain used for predicting the meteorological fields. In this part, the convergence analysis of the SCG method is presented to verify its validity when solving a typical ill-posed inverse problem. The scaled cost function, the scaled gradient norm, the linear convergence rate, and the Hessian approximation condition number are evaluated for convergence analysis. The SCG method reduces the cost functional value and the gradient norm sufficiently. It always performs linear rates of convergence with a quickly reduced condition number of the Hessian matrix. Sometimes it provides very fast linear convergence rates. The SCG code with optimal step size and Andrei’s equation is effective on ill-posed test problems.

Pre-conditioning CQ algorithm for solving the split feasibility problem and its application to image restoration problem

In this study, we define two pre-conditioning CQ algorithms for the split feasibility problem by using self-adaptive and line-search techniques in the real Hilbert space. One of the self-adaptive techniques is used to obtain optimal step size. These techniques do not require prior knowledge of the operator norm or estimation of the matrix norm. We also establish weak convergence theorems with respect to particular norms. As an application, numerical experiments show that the proposed algorithm is effective in image restoration problem.

Designing optimal prototype filters for maximally decimated Cosine Modulated filter banks with rapid convergence

An analytic design of a prototype filter for M-channel maximally decimated cosine-modulated Near Perfect Reconstruction (NPR) filter banks is proposed in this work. The prototype filter is created using the least-square (CLS) method with weighted constraints, which is one-dimensional and requires single-parameter optimization. Compared to existing approaches, this suggested method achieves rapid convergence by analytically determining the optimal step size, ensuring the 3 dB cutoff frequency at π/2 M. The simulation results for design examples outperform the techniques in the available literature in terms of amplitude and aliasing distortion, reaching distortion around 2.4489 × 10−4 and 3.4907 × 10−9, respectively. This optimization algorithm's usefulness is further demonstrated with the sub-band coding of ECG signals. Implementing optimal prototype filters has tangible real-world effects, especially in critical sectors like healthcare and communications, improving diagnostics accuracy, data transmission efficiency, and overall performance.

Construction and simulation of a joint scale model for power electronic converters based on wavelet decomposition and reconstruction algorithms.

In power electronics systems, system design and operation often involve multiple time and space scales, ranging from nanosecond switching dynamics to hour-level system operation behavior. Due to the complexity of these systems and the rise of wide-gap semiconductor technology, a series of multi-scale phenomena have emerged that are difficult to ignore. The high frequency of switching operations makes multi-scale effects particularly significant, including the fast dynamic response of the power loop, EMI, and heat conduction problems. They are key factors that must be considered in the design to ensure the efficient and reliable operation of power electronic devices. This study proposes the construction and simulation of a joint scale model for power electronic converters based on wavelet decomposition and reconstruction algorithms to address the multi-scale phenomenon and limitations of single-scale power electronic converters. Firstly, a joint scale model for power electronic converters at both macro and micro-scales was established, targeting both single-scale models and simple combinations of multiple scale models for power electronic converters. The traditional single-scale model is sufficient to describe the average behavior of the converter, but it has serious limitations in capturing fast transient processes and high-frequency switching behavior in power electronic systems. These limitations often manifest themselves when there is a need to capture fine timescales of detail. By transforming between the time domain and the frequency domain, wavelet decomposition enables the model to capture both macroscopic average characteristics and microscopic transient dynamics. The wavelet reconstruction algorithm can simulate all kinds of fast changes in the actual working process more accurately and compress irrelevant information while retaining key signal features, so as to optimize the simulation performance of the model. Secondly, this algorithm is used to analyze BC in short time scale. Finally, the short time scale characteristics of power electronic converters are analyzed. Experimental results show that the fusion of wavelet decomposition and reconstruction algorithm enhances the accuracy of the power electronic converter model and improves the performance of the system. The model achieves an error reduction of nearly 3% in the calculation step size of 10-7s, which has a significant impact on the high precision requirements of high-frequency operations. In addition, the optimal calculation step size of 8×10-8s achieves an error reduction of more than 14%, making an important contribution to the transient analysis and fine structure simulation. The wavelet algorithm can improve the accuracy of multi-scale modeling in power electronic system and reduce the simulation time. The reduction of error not only shows the improvement of the accuracy of the model, but also shows its practical significance in the design and test of the actual power electronic system. The reduction in error reveals the ability to more accurately predict and mitigate potential performance problems in matching tests with actual hardware, as well as its ability to adapt to emerging wide bandgap semiconductor materials and structures.

A New Dai-Liao Conjugate Gradient Method based on Approximately Optimal Stepsize for Unconstrained Optimization

Conjugate gradient methods are a class of very effective iterative methods for large-scale unconstrained optimization. In this paper, a new Dai-Liao conjugate gradient method for solving large-scale unconstrained optimization problem is proposed. Based on the approximately optimal stepsize for the gradient method, we derive three new choices for the important parameters tk in Dai-Liao conjugate gradient method. The search direction satisfies the sufficient descent condition, and the global convergences of the proposed method for uniformly convex and general functions are proved under some mild conditions. Numerical experiments on a set of test problems from the CUTEst library show that the proposed method is superior to some well-known conjugate gradient methods.

A variable diagonal-matrix-step-size APA robust to impulsive noises

This study introduces a variable diagonal-matrix-step-size affine projection algorithm (APA), which shows robustness against to impulsive noises. Unlike the normal scalar step-size method, the independent step size for each input vector is used as the entry of the diagonal matrix, and the optimal step size at each time step is ascertained by conducting mean-square-deviation (MSD) analysis. In addition, a recursive equation of weight update has been derived without approximating posteriori error as priori error. Using the Lagrange multiplier method, we formulate a scalar step size affine projection algorithm that shows resilience to impulsive noises. This algorithm emerges as a solution to a constrained optimization problem. To further enhance convergence performance, a diagonal-matrix-step size is introduced and its MSD is analyzed. The step size at each time step is optimized by interpreting the MSD mathematically, resulting in the variable diagonal-matrix-step size APA (VDMSS-APA). Various simulations exhibit low steady-state error and fast convergence rate of the proposed algorithm.

Kernel Adaptive Filtering Over Complex Networks.

This brief is concerned with the problem of kernel adaptive filtering for a complex network. First, a coupled kernel least mean square (KLMS) algorithm is developed for each node to uncover its nonlinear measurement function by using a series of input-output data. Subsequently, an upper bound is derived for the step-size of the coupled KLMS algorithm to guarantee the mean square convergence. It is shown that the upper bound is dependent on the coupling weights of the complex network. Especially, an optimal step size is obtained to achieve the fastest convergence speed and a suboptimal step size is presented for the purpose of practical implementations. Besides, a coupled kernel recursive least square (KRLS) algorithm is further proposed to improve the filtering performance. Finally, simulations are provided to verify the validity of the theoretical results.

Guidance Through Surrogate: Toward a Generic Diagnostic Attack.

Adversarial training (AT) is an effective approach to making deep neural networks robust against adversarial attacks. Recently, different AT defenses are proposed that not only maintain a high clean accuracy but also show significant robustness against popular and well-studied adversarial attacks, such as projected gradient descent (PGD). High adversarial robustness can also arise if an attack fails to find adversarial gradient directions, a phenomenon known as "gradient masking." In this work, we analyze the effect of label smoothing on AT as one of the potential causes of gradient masking. We then develop a guided mechanism to avoid local minima during attack optimization, leading to a novel attack dubbed guided projected gradient attack (G-PGA). Our attack approach is based on a "match and deceive" loss that finds optimal adversarial directions through guidance from a surrogate model. Our modified attack does not require random restarts a large number of attack iterations or a search for optimal step size. Furthermore, our proposed G-PGA is generic, thus it can be combined with an ensemble attack strategy as we demonstrate in the case of auto-attack, leading to efficiency and convergence speed improvements. More than an effective attack, G-PGA can be used as a diagnostic tool to reveal elusive robustness due to gradient masking in adversarial defenses.

Accuracy improvement of the load forecasting in the district heating system by the informer-based framework with the optimal step size selection

Accurate load forecasting is crucial for effectively regulating regional heat network systems. However, existing forecasting methods often rely on subjective experience to determine the forecasting step, which is limited by the presence of thermal inertia, leading to suboptimal accuracy. To address this limitation, an optimal step size selection method based on the Informer-based framework is proposed to enhance load forecasting accuracy in heat exchange stations. This method leverages the Attention mechanism within the Informer model, enabling the capture of global information in a single step. To verify the effectiveness of the proposed method, real operational data from a typical thermal power plant in North China is utilized to analyze and test the impact of data distribution and prediction step size on the model's prediction capability. The performance is evaluated using Mean Square Error (MSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE). Comparative analysis against Support Vector Regression (SVR) and Long Short-Term Memory (LSTM) models demonstrates that the Informer algorithm with optimal prediction step size achieves the highest prediction accuracy. Notably, the proposed method achieved a minimum reduction of 62.7 %, 46.5 %, and 42.9 % in MSE, MAE, and MAPE, respectively, significantly surpassing the performance of alternative prediction methods.

Mathematical Modelling of Tuberculosis and Diabetes Co-Infection Using the Non- Standard Finite Difference Scheme

One of the major health challenge facing Africa and in particular, Kenya is the risk of Tuberculosis and Diabetes. To understand the dynamics of this, a nine compartmental model for tuberculosis-diabetes co-infection is formulated. The Non-standard finite difference Scheme (NSFD) of the model is formulated from the first-order ordinary differential equations (ode) to avoid full implicit schemes that are computationally expensive. The overly small step sizes in NSFD give the user autonomy in controlling the accuracy of the results, making it suitable for disease control applications. Numerical simulations with different step sizes of the NSFD for the TB-Diabetes model are carried out to find the optimal step size, h. A comparison of the best resultant numerical simulation based on optimal h in NSDF indicates NSFD gives better results when compared with the corresponding first-order ode. The phase-plane analysis revealed that the NSFD formulated for tuberculosis and diabetes co-infection is generally asymptotically stable. Future studies should consider formulating the proposed model with varied control parameters such as medication to compare the results with those from first-order ode.

Accelerated Projection Algorithm Based on Smoothing Approximation for Distributed Nonsmooth Optimization

In this paper, a distributed smoothing accelerated projection algorithm (DSAPA) is proposed to address constrained non-smooth convex optimization problems over undirected multi-agent networks in a distributed manner, where the objective function is free of the assumption of Lipschitz gradient or strong convexity. First, based on a distributed exact penalty method, the original optimization problem is translated to a problem of standard assignment without consensus constraints. Then, a novel DSAPA by combining the smoothing approximation with Nesterov's accelerated schemes is proposed. In addition, we provide a systematic analysis to derive an upper bound on the convergence rate in terms of the objective function based on penalty function and to choose the optimal step size accordingly. Our results demonstrate that the proposed DSAPA can reach <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$O(\frac{\log (k)}{k})$</tex-math></inline-formula> when the optimal step size is chosen. Finally, the effectiveness and correctness of the proposed algorithm are verified by numerical and practical application examples.

The Weighted, Relaxed Gradient-Based Iterative Algorithm for the Generalized Coupled Conjugate and Transpose Sylvester Matrix Equations

By applying the weighted relaxation technique to the gradient-based iterative (GI) algorithm and taking proper weighted combinations of the solutions, this paper proposes the weighted, relaxed gradient-based iterative (WRGI) algorithm to solve the generalized coupled conjugate and transpose Sylvester matrix equations. With the real representation of a complex matrix as a tool, the necessary and sufficient conditions for the convergence of the WRGI algorithm are determined. Also, some sufficient convergence conditions of the WRGI algorithm are presented. Moreover, the optimal step size and the corresponding optimal convergence factor of the WRGI algorithm are given. Lastly, some numerical examples are provided to demonstrate the effectiveness, feasibility and superiority of the proposed algorithm.

Optimizing signal smoothing using HERS algorithm and time fractional diffusion equation

Signal processing is often affected by various sources of noise that can distort or modify the signals. Removing these noises from the original signal is a crucial step in signal processing, and researchers have proposed several approaches to address this issue. However, achieving an optimized solution remains a challenge. In this study, we introduce a novel approach called the Hybrid Ebola-based Reptile Search (HERS) model based on Time Fractional Diffusion Equation (TFDE). The TFDE is a conventional diffusion equation used for preserving the peak smoothness of spectra signals. In our proposed technique, we consider the processing spectrum of the signal as the reference signal, which serves as the design for the diffusion equation. By applying the diffusion function, we achieve signal peak preservation and smoothing, referred to as the filtering of diffusion. One potential challenge with the time fractional order diffusion equation is its susceptibility to variations in the time step size. To address this, we employ the HERS algorithm to select an optimal time step size that enables efficient signal smoothing. To validate the effectiveness of the proposed technique, we conduct simulations and compare the results with conventional techniques such as the wavelet model, Savitzky-Golay, and regularization techniques. The performance evaluation confirms the superiority of our proposed HERS-TFDE approach in noise removal and signal smoothing. This research aims to contribute to the development of an optimized solution for noise removal in signal processing, leveraging the Hybrid Ebola Reptile Search algorithm and TFDE. The findings have the potential to enhance various signal-processing applications where noise reduction is critical.

Development of an Efficient Variable Step-Size Gradient Method Utilizing Variable Fractional Derivatives

The fractional gradient method has garnered significant attention from researchers. The common view regarding fractional-order gradient methods is that they have a faster convergence rate compared to classical gradient methods. However, through conducting theoretical convergence analysis, we have revealed that the maximum convergence rate of the fractional-order gradient method is the same as that of the classical gradient method. This discovery implies that the superiority of fractional gradients may not reside in achieving fast convergence rates compared to the classical gradient method. Building upon this discovery, a novel variable fractional-type gradient method is proposed with an emphasis on automatically adjusting the step size. Theoretical analysis confirms the convergence of the proposed method. Numerical experiments demonstrate that the proposed method can converge to the extremum point both rapidly and accurately. Additionally, the Armijo criterion is introduced to ensure that the proposed gradient methods, along with various existing gradient methods, can select the optimal step size at each iteration. The results indicate that, despite the proposed method and existing gradient methods having the same theoretical maximum convergence speed, the introduced variable step size mechanism in the proposed method consistently demonstrates superior convergence stability and performance when applied to practical problems.