Optimal Step Size Research Articles

Principal Component Tracking for Massive MIMO Channels in High Mobility Scenarios With Diagonal Step Size Matrix

The present article proposes a novel adaptive algorithm with an optimal diagonal step size matrix for multi-dimensional channel principal component tracking in two dimensional massive multiple-input and multiple-output (M-MIMO) systems. First, we prove that the weighted subspace algorithm globally converges to the stationary stochastic process’ major eigenvectors. Then, using the maximum likelihood criterion, we optimize the weight coefficient matrix and derive the convergent condition for the step size range in order to maintain the algorithm stability. To accelerate the convergence, we initially suggest the diagonal step size matrix for multi-dimensional eigenvector tracking. Simultaneously, an optimal diagonal step size matrix is derived, which not only accelerates the convergence speed distinctively but also improves the tracking of multi-dimensional eigenvectors. Moreover, the transient behavior during the adaptation process is investigated in a straight-forward way and the relationship between the convergence time constant and the eigenvalues of the received signals is uncovered. Finally, simulations reveal that the proposed approach outperforms established algorithms such as Oja <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$^{\prime}\text{s}$ </tex-math></inline-formula> , Delmas and gradient descent algorithms. This approach establishes a sound foundation for tracking channel state information in M-MIMO systems with great mobility.

Wirtinger Flow for Nonconvex Blind Demixing With Optimal Step Size

The prospect of massive deployment of devices for Internet-of-Things (IoT) motivates grant-free access for simultaneously uplink transmission by multiple nodes. Blind demixing represents a promising technique for recovering multiple such source signals over unknown channels. Recent studies show Wirtinger Flow (WF) algorithm can be effective in blind demixing. However, existing theoretical results on WF step size selection tend to be conservative and slow down convergence rates. To overcome this limitation, we propose an improved WF (WF-OPT) by optimizing its step size in each iteration and expediting the convergence. We provide a theoretical guarantee on the strict contraction of WF-OPT and present the upper bounds of the contraction ratio. Simulation results demonstrate the expected convergence gains.

Accelerated Conjugate Gradient for Second-Order Blind Signal Separation

This paper proposes a new adaptive algorithm for the second-order blind signal separation (BSS) problem with convolutive mixtures by utilising a combination of an accelerated gradient and a conjugate gradient method. For each iteration of the adaptive algorithm, the search point and the search direction are obtained based on the current and the previous iterations. The algorithm efficiently calculates the step size for the accelerated conjugate gradient algorithm in each iteration. Simulation results show that the proposed accelerated conjugate gradient algorithm with optimal step size converges faster than the accelerated descent algorithm and the steepest descent algorithm with optimal step size while having lower computational complexity. In particular, the number of iterations required for convergence of the accelerated conjugate gradient algorithm is significantly lower than the accelerated descent algorithm and the steepest descent algorithm. In addition, the proposed system achieves improvement in terms of the signal to interference ratio and signal to noise ratio for the dominant speech outputs.

Optimal Step Size Technique for Frequency Domain and Partition Block Adaptive Filters for PEM based Acoustic Feedback Cancellation

The adaptive filtering approach has been commonly used to perform acoustic feedback cancellation (AFC) in digital hearing-aids due to its reliable performance and feasibility. Because the loudspeaker and microphone are close together in hearing aids, the corresponding signals are highly correlated, resulting in biased estimation if adaptive filters are used. This problem can be addressed with the help of the decorrelation prefilter by incorporating the Prediction Error Method (PEM) technique into AFC. Frequency-Domain Adaptive Filters (FDAF) are preferable over the time-domain implementation to achieve better performance in terms of convergence and computational complexity. In addition, Partition-Block Frequency-Domain Adaptive Filters (PBFDAF) offers low processing delay. However, because of their fixed step-size, there is a trade-off between initial convergence and steady-state misalignment in the widely used frequency-domain algorithms. While Variable Step-Size (VSS) algorithms can help with this issue, VSS techniques for frequency-domain algorithms have not been extensively studied in the context of PEM-AFC. Hence, in this paper, we presented an Optimal Step-Size (OSS) technique for both the FDAF-PEM_AFC and PBFDAF-PEM_AFC algorithms to simultaneously accomplish fast convergence and minimal steady-state error. A Feedback Path Change Detector (FPCD) was also incorporated into the proposed algorithms to address the problem of convergence in non-stationary feedback paths. The results of simulations show that the proposed algorithms are clearly superior, and they are encouraging.

Distributed Newton Optimization With Maximized Convergence Rate

The distributed optimization problem is set up in a collection of nodes interconnected via a communication network. The goal is to find the minimizer of a global objective function formed by the sum of local functions known at individual nodes. A number of methods, having different advantages, are available for addressing this problem. The goal of this article is to achieve the maximum possible convergence rate. As the first step toward this end, we propose a new method, which we show converges faster than other available options. As the second step toward our goal, we complement the proposed method with a fully distributed method for estimating the optimal step size that maximizes the convergence rate. We provide theoretical guarantees for the convergence of the resulting method in a neighborhood of the solution. We present numerical experiments showing that, when using the same step size, our method converges significantly faster than its rivals. Experiments also show that the distributed step-size estimation method achieves an asymptotic convergence rate very close to the theoretical maximum.

Variable Matrix-Type Step-Size Affine Projection Sign Algorithm for System Identification in the Presence of Impulsive Noise

This paper presents a novel variable matrix-type step-size affine projection sign algorithm (VMSS-APSA) characterized by robustness against impulsive noise. To mathematically derive a matrix-type step size, VMSS-APSA utilizes mean-square deviation (MSD) for the modified version of the original APSA. Accurately establishing the MSD of APSA is impossible. Therefore, the proposed VMSS-APSA derives the upper bound of the MSD using the upper bound of the L1-norm of the measurement noise. The optimal matrix-type step size is calculated at each iteration by minimizing the upper bound of the MSD, thereby improving the filter performance in terms of convergence rate and steady-state estimation error. Because a novel cost function of the proposed VMSS-APSA was designed to maintain a form similar to the original APSA, they have symmetric characteristics. Simulation results demonstrate that the proposed VMSS-APSA improves filter performance in a system-identification scenario in the presence of impulsive noise.

A New Spectral Conjugate Gradient method for solving unconstrained Optimization problems

The spectral conjugate gradient methods are fascinating, and it has been shown that they are useful for strictly convex quadratic reduction when used properly. To handle large-scale unconstrained optimization issues, a novel spectral conjugate gradient approach is suggested in this study. We devise a new methodology for determining the spectral and conjugate parameters, motivated by the benefits of the approximate optimum step size strategy utilized in the gradient method. Additionally, the new search direction meets the spectral property as well as the sufficient descent criterion. The presented method's global convergence is established under a set of appropriate assumptions. Consider the unconstrained optimization problem with the following n variables : min⁡f(x) ,x∈R^n (1) The conjugate gradient methods are among the most effective optimization strategies for achieving the solution of problem (1), where f:R^n→R is a continuous differentiable function. The conjugate gradient technique has the following form : x_(k+1)=x_k+α_k d_k ,k=0,1,2,3,… (2) Where x_0 is the starting point, α_k is a step size , g_k=∇f(x) and d_k can be taken as : d_k={█(-g_k ∶ k=0@-g_k+β_k d_(k-1) ∶ k≥1)┤ (3)

Design of delayless multi-sampled subband functional link neural network with application to active noise control

To accelerate the convergence speed of the functional link neural network (FLNN) particularly for colored input signals, this paper proposes a delayless multi-sampled multiband-structured subband FLNN (DMSFLNN) structure. Then, to update the weights of the DMSFLNN, a normalized subband adaptive algorithm is devised. Next, the stability conditions, optimal step size and computational complexity are investigated. Moreover, the proposed method is applied to the nonlinear active noise control, obtaining the delayless multi-sampled multiband-structured filtered-s normalized least mean square (DMSFsNLMS) algorithm. Finally, simulation results demonstrate that the proposed method improves the convergence speed of the FLNN.

On the relaxed gradient-based iterative methods for the generalized coupled Sylvester-transpose matrix equations

In this paper, we propose the full-rank and reduced-rank relaxed gradient-based iterative algorithms for solving the generalized coupled Sylvester-transpose matrix equations. We provide analytically the necessary and sufficient condition for the convergence of the proposed iterative algorithm and give explicitly the optimal step size such that the convergence rate of the algorithm is maximized. Some numerical examples are examined to confirm the feasibility and efficiency of the proposed algorithms.

Stochastic gradient line Bayesian optimization for efficient noise-robust optimization of parameterized quantum circuits

Optimizing parameterized quantum circuits is a key routine in using near-term quantum devices. However, the existing algorithms for such optimization require an excessive number of quantum-measurement shots for estimating expectation values of observables and repeating many iterations, whose cost has been a critical obstacle for practical use. We develop an efficient alternative optimization algorithm, stochastic gradient line Bayesian optimization (SGLBO), to address this problem. SGLBO reduces the measurement-shot cost by estimating an appropriate direction of updating circuit parameters based on stochastic gradient descent (SGD) and further utilizing Bayesian optimization (BO) to estimate the optimal step size for each iteration in SGD. In addition, we formulate an adaptive measurement-shot strategy and introduce a technique of suffix averaging to reduce the effect of statistical and hardware noise. Our numerical simulation demonstrates that the SGLBO augmented with these techniques can drastically reduce the measurement-shot cost, improve the accuracy, and make the optimization noise-robust.

Convergence Analysis and Approximate Optimal Temporal Step Sizes for Some Finite Difference Methods Discretising Fisher's Equation

In this study, we obtain a numerical solution for Fisher's equation using a numerical experiment with three different cases. The three cases correspond to different coefficients for the reaction term. We use three numerical methods namely; Forward-Time Central Space (FTCS) scheme, a Nonstandard Finite Difference (NSFD) scheme, and the Explicit Exponential Finite Difference (EEFD) scheme. We first study the properties of the schemes such as positivity, boundedness, and stability and obtain convergence estimates. We then obtain values of L1 and L∞ errors in order to obtain an estimate of the optimal time step size at a given value of spatial step size. We determine if the optimal time step size is influenced by the choice of the numerical methods or the coefficient of reaction term used. Finally, we compute the rate of convergence in time using L1 and L∞ errors for all three methods for the three cases.

An efficient gradient method with approximately optimal stepsizes based on regularization models for unconstrained optimization

It is widely accepted that the stepsize is of great significance to gradient method. An efficient gradient method with approximately optimal stepsizes mainly based on regularization models is proposed for unconstrained optimization. More specifically, if the objective function is not close to a quadratic function on the line segment between the current and latest iterates, regularization model is exploited carefully to generate approximately optimal stepsize. Otherwise, quadratic approximation model is used. In addition, when the curvature is non-positive, special regularization model is developed. The convergence of the proposed method is established under some weak conditions. Extensive numerical experiments indicated the proposed method is very promising. Due to the surprising efficiency, we believe that gradient methods with approximately optimal stepsizes can become strong candidates for large-scale unconstrained optimization.

On the new spectral conjugate gradient‐type method for monotone nonlinear equations and signal recovery

We present a new approach for constructing a spectral conjugate gradient‐type method for solving nonlinear equations. The proposed method uses an approximate optimal step size together with the memoryless Broyden–Fletcher–Goldfarb–Shanno (BFGS) formula to generate a new choice of the spectral conjugate gradient‐type direction that satisfies the sufficient descent condition without line search requirement. The global convergence of the method is achieved under some mild assumptions. Numerical experiments on both nonlinear monotone equations and signal reconstruction problems reveal the efficiency of the new approach.

Optimal error bounds for non-expansive fixed-point iterations in normed spaces

This paper investigates optimal error bounds and convergence rates for general Mann iterations for computing fixed-points of non-expansive maps. We look for iterations that achieve the smallest fixed-point residual after n steps, by minimizing a worst-case bound $$\Vert x^n-Tx^n\Vert \le R_n$$ derived from a nested family of optimal transport problems. We prove that this bound is tight so that minimizing $$R_n$$ yields optimal iterations. Inspired from numerical results we identify iterations that attain the rate $$R_n=O(1/n)$$ , which we also show to be the best possible. In particular, we prove that the classical Halpern iteration achieves this optimal rate for several alternative stepsizes, and we determine analytically the optimal stepsizes that attain the smallest worst-case residuals at every step n, with a tight bound $$R_n\approx \frac{4}{n+4}$$ . We also determine the optimal Halpern stepsizes for affine non-expansive maps, for which we get exactly $$R_n=\frac{1}{n+1}$$ . Finally, we show that the best rate for the classical Krasnosel’skiĭ–Mann iteration is $$\varOmega (1/\sqrt{n})$$ , and present numerical evidence suggesting that even extended variants cannot reach a faster rate.

Multi-Objective Spotted Hyena Optimization-Based Control of Grid-Tied PV System

This article describes the multi-objective spotted hyena optimization (MOSHO) based control of a three-phase three-wire grid-tied dual-stage PV system. The MOSHO delivers an optimized step size ( δ ) for the incremental conductance (InC) based maximum power point tracking (MPPT) algorithm, as well as optimizes the PI controller gains for DC bus voltage ( V d c ) regulation. The δ optimized MPPT facilitates the faster and more accurate tracking of maximum power point with fewer fluctuations at steady-state. The MOSHO optimized PI controller regulates the V dc and delivers better stability with reduced V dc variations in DC bus voltage during steady-state and dynamic states, i.e. irradiation variation, abnormal and unbalanced grid voltage. The optimized DC bus generates an accurate loss component of current, which further enhances the voltage source converter (VSC) performance in terms of precise reference currents generation and power quality improvement. The proposed system performs satisfactorily in a MATLAB simulation environment as per IEEE519 standards.

Bilinear Optimal Control of an Advection-Reaction-Diffusion System

We consider the bilinear optimal control of an advection-reaction-diffusion system, where the control arises as the velocity field in the advection term. Such a problem is generally challenging from both theoretical analysis and algorithmic design perspectives, mainly because the state variable depends nonlinearly on the control variable and, an additional divergence-free constraint on the control is coupled together with the state equation. Mathematically, the proof of the existence of optimal solutions is delicate, and, up to now, only some results have been known for a few special cases where additional restrictions are imposed on the space dimension and the regularity of the control. We prove the existence of optimal controls and derive the first-order optimality conditions in general settings without any extra assumptions. Computationally, the well-known conjugate gradient (CG) method can be applied conceptually. However, due to the additional divergence-free constraint on the control variable and the nonlinear relation between the state and control variables, it is challenging to compute the gradient and the optimal stepsize at each CG iteration, and thus nontrivial to implement the CG method. To address these issues, we advocate a fast inner preconditioned CG method to ensure the divergence-free constraint and an efficient inexactness strategy to determine an appropriate stepsize. An easily implementable nested CG method is thus proposed for solving such a complicated problem. For the numerical discretization, we combine finite difference methods for the time discretization and finite element methods for the space discretization. Efficiency of the proposed nested CG method is promisingly validated by the results of some preliminary numerical experiments.

An Iterative Least Squares Method for Proton CT Image Reconstruction.

Clinically useful proton Computed Tomography images will rely on algorithms to find the three-dimensional proton stopping power distribution that optimally fits the measured proton data. We present a least squares iterative method with many features to put proton imaging into a more quantitative framework. These include the definition of a unique solution that optimally fits the protons, the definition of an iteration vector that takes into account proton measurement uncertainties, the definition of an optimal step size for each iteration individually, the ability to simultaneously optimize the step sizes of many iterations, the ability to divide the proton data into arbitrary numbers of blocks for parallel processing and use of graphical processing units, and the definition of stopping criteria to determine when to stop iterating. We find that it is possible, for any object being imaged, to provide assurance that the image is quantifiably close to an optimal solution, and the optimization of step sizes reduces the total number of iterations required for convergence. We demonstrate the use of these algorithms on real data.

Accelerated Gradient Descent Estimation for Rational Models by Using Volterra Series: Structure Identification and Parameter Estimation

This brief studies a rational model identification problem using a novel gradient descent algorithm. The rational model is approximated by a second-order Volterra system, whose parameters and structure are estimated through an accelerated gradient descent algorithm. This algorithm improves the convergence rates by computing an optimal step-size and introducing a preconditioning method. In addition, it can avoid the eigenvalue calculation if the preconditioning matrix is equal to the inverse of the information matrix. A numerical experiment is given to illustrate the performances of the proposed algorithm.

Modified narrowband active noise control system with frequency mismatch tolerance

A common system used for canceling sinusoidal noise caused by revolving equipment such as machines, propellers and motors is the narrowband active noise control (NANC). In NANC, the frequency mismatch (FM) between a reference signal and the primary noise influences the noise reduction performance. The impact of the FM is eliminated by estimating frequency and changing the step size. The conventional method changes the step size by hand, which includes a significant expense and is not unreliabl. In this paper, a step size optimization algorithm with golden section search (GSS) and virtual system was proposed, which can automatically find the most suitable step size. The optimal step size was obtained by comparing the mean square errors at the golden section points. The virtual system was utilized to isolate the noise reduction system and the step size optimization system. Along these lines, the proposed method enhanced the noise decrement of the existing NANC system. Simulations were undertaken to evaluate the performance of the proposed algorithm.

Analysis of the Frequency Interference in the Narrowband Active Noise Control System

The narrowband active noise control (ANC) algorithm works well in the multi-tone noise reduction system. However, the performance of this algorithm degenerates significantly when the noisy frequencies become very close (e. g. frequency gap is 3Hz where the sampling frequency is 5000Hz), which can be seen as frequency interference effects. Up to now, however, this effect has not been researched in detail. In this work, the frequency interference effects are analyzed and discussed. The convergence process, the noise reduction value, the stability condition and the optimum step size are obtained by extensive math formulation derivation, provided two frequencies are close and fixed. Theoretical results show that, tones with large frequency difference can be approximated to independent convergence while the system with close frequencies suffers from poor convergence rate. The value of step size plays an important role in convergence, but the interference effects could never be eliminated by adjusting step size. Simulations are conducted to verify the analysis and the results match well with the analysis.