Optimal Step Length Research Articles

From theoretical guarantee to practical performance: selectable and optimal step-lengths for IHT and HTP algorithms in compressed sensing

From theoretical guarantee to practical performance: selectable and optimal step-lengths for IHT and HTP algorithms in compressed sensing

A New Method Based on Locally Optimal Step Length in Accelerated Gradient Descent for Quantum State Tomography.

Quantum state tomography (QST) is one of the key steps in determining the state of the quantum system, which is essential for understanding and controlling it. With statistical data from measurements and Positive Operator-Valued Measures (POVMs), the goal of QST is to find a density operator that best fits the measurement data. Several optimization-based methods have been proposed for QST, and one of the most successful approaches is based on Accelerated Gradient Descent (AGD) with fixed step length. While AGD with fixed step size is easy to implement, it is computationally inefficient when the computational time required to calculate the gradient is high. In this paper, we propose a new optimal method for step-length adaptation, which results in a much faster version of AGD for QST. Numerical results confirm that the proposed method is much more time-efficient than other similar methods due to the optimized step size.

Sufficient condition based on nearly optimal order RIC for IHT algorithm

Iterative hard thresholding (IHT) is one class of sparse signal recovery algorithms widely used in compressed sensing. The well-known restricted isometry constant (RIC) is one of the most popular frameworks used to ensure the convergence of recovery algorithms. In previous literature, sufficient conditions to guarantee the recovery performance of the IHT algorithm have usually been established based on RICs with orders 3 s and 2 s , while no such condition in terms of RIC with order less than 2 s has been reported yet. In this study, based on the theoretical optimal step-length, we develop the RIC condition γ < ( 5 − 1 ) / 8 ≈ 0.1545 for successful recovery via the IHT algorithm. Here, γ = δ s if s is an even number, otherwise γ = δ s + 1 . To the best of our knowledge, this is the first result based on RIC with nearly optimal order for the IHT algorithm.

Effects of step lengths on biomechanical characteristics of lower extremity during split squat movement.

Objective: To quantify the effects of increasing the step length of the split squat on changes in kinematics, kinetics, and muscle activation of the lower extremity. Methods: Twenty male college students participated in the test (age: 23.9 ± 3.7, height: 175.1 ± 4.9). Data on kinematics, kinetics, and EMG were collected during split squat exercise at four different step lengths in a non-systematic manner. One-way repeated measurements ANOVA were used to compare characteristic variables of peak angle, moment, and RMS among the four step length conditions. Results: The step length significantly changes the peak angles of the hip (p = 0.011), knee (p = 0.001), ankle (p < 0.001) joint, and the peak extension moment of the hip (p < 0.001), knee (p = 0.002) joint, but does not affect the ankle peak extension moment (p = 0.357) during a split squat. Moreover, a significant difference was observed in the EMG of gluteus maximus (p < 0.001), vastus medialis (p = 0.013), vastus lateralis (p = 0.020), biceps femoris (p = 0.003), Semitendinosus (p < 0.001), medialis gastrocnemius (p = 0.035) and lateralis gastrocnemius (p = 0.005) during four step lengths, but no difference in rectus femoris (p = 0.16). Conclusion: Increases in step length of split squat had a greater activation on the hip extensor muscles while having a limited impact on the knee extensor muscles. The ROM, joint moment, and muscle activation of the lead limb in the split squat all should be considered in cases of individual preventative or rehabilitative prescription of the exercise. Moreover, the optimal step length for strength training in healthy adults appears to be more suitable when it is equal to the length of the individual lower extremity.

Optimal step length for the maximal decrease of a self-concordant function by the Newton method

Optimal step length for the maximal decrease of a self-concordant function by the Newton method

Extending the inverse sequential quasi-Newton method for on-line monitoring and controlling of process conditions in the solidification of alloys

This work introduces the sequential quasi-Newton method (SQNM) to solve the inverse design problem of estimating the transient boundary heat fluxes that produce desired evolutions of the solidus and liquidus isotherms in alloys’ solidification processes. This is an unprecedented application for sequential gradient-based methods. The final goal is monitoring and controlling the evolution of the mushy zone in a near real-time fashion, which is of high technological interest to the foundry industry. We assess the performance of the SQNM in solving several inverse design problems in one- and two-dimensional domains. The method satisfactorily estimates the boundary heat fluxes using relatively few future measurements. However, some instabilities have been detected in the SQNM-predicted heat fluxes. To alleviate this issue, we proposed a modified version of SQNM, say mSQNM, that employs an inexact line search strategy to find the optimal step length during the iterative estimation procedure. We finally show how mSQNM improves the estimation of the boundary heat fluxes.

Fault Diagnosis of Rolling Bearings Based on Optimal Noise and Optimal Step Length Superimposed Segmental Bistable Stochastic Resonance

Fault Diagnosis of Rolling Bearings Based on Optimal Noise and Optimal Step Length Superimposed Segmental Bistable Stochastic Resonance

Adaptive step-length selection in gradient boosting for Gaussian location and scale models

Tuning of model-based boosting algorithms relies mainly on the number of iterations, while the step-length is fixed at a predefined value. For complex models with several predictors such as Generalized additive models for location, scale and shape (GAMLSS), imbalanced updates of predictors, where some distribution parameters are updated more frequently than others, can be a problem that prevents some submodels to be appropriately fitted within a limited number of boosting iterations. We propose an approach using adaptive step-length (ASL) determination within a non-cyclical boosting algorithm for Gaussian location and scale models, as an important special case of the wider class of GAMLSS, to prevent such imbalance. Moreover, we discuss properties of the ASL and derive a semi-analytical form of the ASL that avoids manual selection of the search interval and numerical optimization to find the optimal step-length, and consequently improves computational efficiency. We show competitive behavior of the proposed approaches compared to penalized maximum likelihood and boosting with a fixed step-length for Gaussian location and scale models in two simulations and two applications, in particular for cases of large variance and/or more variables than observations. In addition, the underlying concept of the ASL is also applicable to the whole GAMLSS framework and to other models with more than one predictor like zero-inflated count models, and brings up insights into the choice of the reasonable defaults for the step-length in the simpler special case of (Gaussian) additive models.

Optimal Time Step Length for Lagrangian Interacting-Particle Simulations of Diffusive Mixing

Optimal Time Step Length for Lagrangian Interacting-Particle Simulations of Diffusive Mixing

The exact worst-case convergence rate of the gradient method with fixed step lengths for L-smooth functions

In this paper, we study the convergence rate of the gradient (or steepest descent) method with fixed step lengths for finding a stationary point of an L-smooth function. We establish a new convergence rate, and show that the bound may be exact in some cases, in particular when all step lengths lie in the interval (0, 1/L]. In addition, we derive an optimal step length with respect to the new bound.

Optimal step length for the Newton method: case of self-concordant functions

The theoretical foundation of path-following methods is the performance analysis of the (damped) Newton step on the class of self-concordant functions. However, the bounds available in the literature and used in the design of path-following methods are not optimal. In this contribution we use methods of optimal control theory to compute the optimal step length of the Newton method on the class of self-concordant functions, as a function of the initial Newton decrement, and the resulting worst-case decrease of the decrement. The exact bounds are expressed in terms of solutions of ordinary differential equations which cannot be integrated explicitly. We provide approximate numerical and analytic expressions which are accurate enough for use in optimization methods. Consequently, the neighbourhood of the central path in which the iterates of path-following methods are required to stay can be enlarged, enabling faster progress along the central path during each iteration and hence fewer iterations to achieve a given accuracy.

Effect of random walk methods on searching efficiency in swarm robots for area exploration

The objective of area exploration is to traverse the area effectively and random walk methods are the most commonly used searching strategy for swarm robots. Existing research mainly compares the effectiveness of various random walk methods through experimental verification, which has relatively large limitations. In order to make the application of the random walk methods more convenient, this paper quantitatively analyzes the searching efficiency (SE) of random walk methods. Firstly, the formula of the mean square displacement (MSD) of the robot position is given, and it is shown that the mean and the variance of the random step length are the factors that affect the SE. In addition, in order to produce the suitable step length, a truncated random walk method is constructed to make the generated step lengths follow a given distribution and the step lengths are within a specified range, thereby improving the SE. Thirdly, the correlations between the step length threshold (SLT), the area of the region, and the number of robots are provided based on MSD and truncated random walk method. When the area of region and the number of robots are fixed, there exists a SLT. When the expectation of the step length is greater than SLT, the swarm robots can achieve the highest SE. The area exploration task of swarm robots are carried out in simulation experiments and the coverage ratio is used to evaluate the SE of each random walk method. The experimental results show that when the area and the number of robots are given, there exist an optimal step length, which can enable the robots to achieve the optimal search.

A Joint TDOA/FDOA Localization Algorithm Using Bi‐iterative Method with Optimal Step Length

A Joint TDOA/FDOA Localization Algorithm Using Bi‐iterative Method with Optimal Step Length

A hybrid gradient method for strictly convex quadratic programming

AbstractIn this article, we present a reliable hybrid algorithm for solving convex quadratic minimization problems. At thekth iteration, two points are computed: first, an auxiliary point is generated by performing a gradient step using an optimal steplength, and second, the next iteratexk + 1is obtained by means of weighted sum of with the penultimate iteratexk − 1. The coefficient of the linear combination is computed by minimizing the residual norm along the line determined by the previous points. In particular, we adopt an optimal, nondelayed steplength in the first step and then use a smoothing technique to impose a delay on the scheme. Under a modest assumption, we show that our algorithm is Q‐linearly convergent to the unique solution of the problem. Finally, we report numerical experiments on strictly convex quadratic problems, showing that the proposed method is competitive in terms of CPU time and iterations with the conjugate gradient method.

Efficient 3-D velocity model building using joint inline and crossline plane-wave wave-equation migration velocity analyses

SUMMARY Wave-equation migration velocity analysis (WEMVA) is an image-domain inversion method for velocity model building. Automatic plane-wave WEMVA (PWEMVA) calculates the moveouts of plane-wave common-image gathers (CIGs) by searching a best-fitting parabola with semblance analysis and backprojects residual CIG moveouts into wavefield wave paths with a reflection tomographic kernel. However, 3-D PWEMVA is very computationally expensive because 3-D reflection tomographic inversion requires at least five 3-D reverse-time migrations per iteration and stores two types of source wavefields at model boundaries. We develop a joint inline and crossline PWEMVA method for efficient 3-D velocity model building. We alternatively implement the inline and crossline PWEMVAs with a constraint for each other, in which we iteratively construct the 3-D velocity model update through 1-D spline interpolation of 2-D gradients. The inline and crossline joint inversion is practical since PWEMVA only inverts for low-wavenumber velocity perturbations along wave paths, and the method can take less than 1 per cent of the computational cost of full 3-D PWEMVA. To construct unaliased plane waves for our joint inline and crossline PWEMVA, we develop a 3-D data interpolation method in the frequency–wavenumber (FK) domain to recover regularly and randomly missing traces. The method minimizes the misfit on sufficiently localized data subsets with iterative optimal step lengths and a gradient preconditioner that iteratively selects dominant dips along different azimuths. In numerical experiments, we use a 3-D synthetic seismic data set and a land 3-D field seismic data set acquired at the Farnsworth CO2-EOR (enhanced oil recovery) field to demonstrate the efficacy of our velocity model building and data interpolation methods.

Adaptive Powell’s Identification of Elastic Constants of Composite Glass Girder with Layered Shell Element Theory

For the composite glass box girder, the generalized Bayesian objective function of elastic constants of the structure was derived based on layered shell element theory. Mechanical performances of the composite glass box girder were solved by layered shell element method. Combined with quadratic parabolic interpolation search scheme of optimized step length, the adaptive Powell’s optimization theory was taken to complete the stochastic identification of elastic constants of composite glass box girder. Then the adaptive Powell’s identification steps of elastic constants of the structure were presented in detail and the adaptive Powell’s identification procedure was accomplished. From some classic examples, it is finally achieved that the adaptive Powell’s identification of elastic constants of composite glass box girder has perfect convergence and numerical stability, which testifies that the adaptive Powell’s identification theory of elastic constants of composite glass box girder is correct and reliable. The stochastic characteristics of systematic responses and elastic constants are well deliberated in generalized Bayesian objective function. And in iterative processes, the adaptive Powell’s identification is irrelevant with the complicated partial differentiation of the systematic responses from the layered shell element model to the elastic constants, which proves high computation efficiency.

Adaptive iterative Hessian sketch via A-optimal subsampling

Iterative Hessian sketch (IHS) is an effective sketching method for modeling large-scale data. It was originally proposed by Pilanci and Wainwright (J Mach Learn Res 17(1):1842–1879, 2016) based on randomized sketching matrices. However, it is computationally intensive due to the iterative sketch process. In this paper, we analyze the IHS algorithm under the unconstrained least squares problem setting and then propose a deterministic approach for improving IHS via A-optimal subsampling. Our contributions are threefold: (1) a good initial estimator based on the A-optimal design is suggested; (2) a novel ridged preconditioner is developed for repeated sketching; and (3) an exact line search method is proposed for determining the optimal step length adaptively. Extensive experimental results demonstrate that our proposed A-optimal IHS algorithm outperforms the existing accelerated IHS methods.

The spectral conjugate gradient method in variational adjoint assimilation for model terrain correction I: Theoretical frame

A spectral conjugate gradient (SCG) method is proposed within the mathematical framework of a variational adjoint assimilation system to correct the bottom terrain of a shallow-water equations model. The formulation of this method is described from the mathematical point of view with determination of the descent direction by using Andrei’s limited-memory form and the step length by solving the tangent linear model. It is proved to benefit from (i) the iterative regularization strategy, (ii) the inverse Hessian approximation involving the second-order information from Broyden-Fletcher-Goldfarb-Shanno class (BFGS), and (iii) the optimal step length. The regularization term introduced to the cost function will allow the incorporation of the known information about the desired bottom terrain and guarantee the uniqueness of the optimal solution.

Correction for Microstructure Effect on Residual Stress Measurement of SR-FSW Joint with Lcr Wave

Correction method for microstructure effect on residual stress of self-reacting friction stir weld joint measured with longitudinal critically refracted wave was discussed in this study. Two longitudinal critically refracted wave transducers, the spacing between which was fixed, were used, then the acoustoelastic formula was optimized, and difference in time of flight between longitudinal critically refracted wave, which replaced propagation velocity as characteristic parameter for residual stress measurement, was determined by the cross correlation theory, at last the influence of microstructure on residual stress of self-reacting friction stir weld joint was solved. Results show that as stress increases, the deviation in difference in time of flight becomes obvious, and when stress reaches stress turning point, experimental result is not agreed with the acoustoelastic theory as stress increases further. Step length is an important factor for difference in time of flight, and one cycle is seen as optimal step length. For acoustoelastic coefficient, the value of nugget zone is biggest, and the value of (heat affected zone + thermal mechanical affected zone) is smallest, in addition, the value of advancing side is slightly bigger than that of retreating side. Finally, residual stress of self-reacting friction stir weld joint was corrected with longitudinal critically refracted wave, and the result is verified by hole-drilling.

An alternating projection algorithm for the “approximate” GCD calculation

In the paper an approach is proposed for calculating the “best” approximate GCD of a set of coprime polynomials. The algorithm is motivated by the factorisation of the Sylvester resultant matrix of polynomial sets with nontrivial GCD. In the (generic) case of coprime polynomial sets considered here the aim is to minimise the norm of the residual error matrix of the inexact factorisation in order to compute the “optimal” approximate GCD. A least-squares alternating projection algorithm is proposed as an alternative to the solution of the corresponding optimisation problem via nonlinear programming techniques. The special structure of the problem in this case, however, means that the algorithm can be reduced to a sequence of standard subspace projections and hence no need arises to compute gradient vectors, Hessian matrices or optimal step-lengths. An estimate of the asymptotic convergence rate of the algorithm is finally established via the inclination of two subspaces.