Suitable Step Size Research Articles

Extragradient method with feasible inexact projection to variational inequality problem

The variational inequality problem in finite-dimensional Euclidean space is addressed in this paper, and two inexact variants of the extragradient method are proposed to solve it. Instead of computing exact projections on the constraint set, as in previous versions extragradient method, the proposed methods compute feasible inexact projections on the constraint set using a relative error criterion. The first version of the proposed method provided is a counterpart to the classic form of the extragradient method with constant steps. In order to establish its convergence we need to assume that the operator is pseudo-monotone and Lipschitz continuous, as in the standard approach. For the second version, instead of a fixed step size, the method presented finds a suitable step size in each iteration by performing a line search. Like the classical extragradient method, the proposed method does just two projections into the feasible set in each iteration. A full convergence analysis is provided, with no Lipschitz continuity assumption of the operator defining the variational inequality problem.

Adaptive stepsize estimation based accelerated gradient descent algorithm for fully complex-valued neural networks

Nesterov accelerated gradient (NAG) method is an efficient first-order algorithm for optimization problems. To ensure the convergence, it usually takes a relatively conservative constant as the stepsize. However, the choice of stepsize has a great impact on the optimization process. In this paper, two adaptive stepsize estimation methods are proposed for complex-valued NAG algorithm for efficient training of fully complex-valued neural networks. The basic idea of the first one is to adaptively determine suitable stepsize by estimating the local smoothness constant of the loss function with the norm of approximate complex Hessian matrix. Its validity is theoretically analyzed by means of the decomposition of complex matrix. Furthermore, by introducing a new parameter design method for multi-step quasi-Newton condition, an improved stepsize estimation is presented. Experimental results on pattern recognition, channel equalization, wind forecasting and synthetic aperture radar (SAR) target classification demonstrate the effectiveness of the proposed methods.

Structured BFGS Method for Optimal Doubly Stochastic Matrix Approximation

Doubly stochastic matrix plays an essential role in several areas such as statistics and machine learning. In this paper we consider the optimal approximation of a square matrix in the set of doubly stochastic matrices. A structured BFGS method is proposed to solve the dual of the primal problem. The resulting algorithm builds curvature information into the diagonal components of the true Hessian, so that it takes only additional linear cost to obtain the descent direction based on the gradient information without having to explicitly store the inverse Hessian approximation. The cost is substantially fewer than quadratic complexity of the classical BFGS algorithm. Meanwhile, a Newton-based line search method is presented for finding a suitable step size, which in practice uses the existing knowledge and takes only one iteration. The global convergence of our algorithm is established. We verify the advantages of our approach on both synthetic data and real data sets. The experimental results demonstrate that our algorithm outperforms the state-of-the-art solvers and enjoys outstanding scalability.

Maximum-principle-preserving high-order discontinuous Galerkin methods for incompressible Euler equations on overlapping meshes

In this paper, we construct a new local discontinuous Galerkin (LDG) algorithm to solve the incompressible Euler equation in two space dimensions on overlapping meshes. This method solves the vorticity, velocity field and potential function on different meshes. Different from the traditional LDG method, the overlapping meshes used in this paper make the velocity to be continuous along the interfaces of the primitive meshes. Therefore, the upwind fluxes can be applied. We derive two sufficient conditions to obtain the maximum principle of vorticity. The first one is the divergence-free numerical approximation of the velocity field. This condition further grants that the scheme of the vorticity equation keeps constant solutions. The second one is to preserve the positivity of the numerical vorticity. We select suitable time step sizes to construct positive numerical cell averages of the vorticity provided the vorticity in the previous time step is positive. Then a slope limiter can be applied to enforce the positivity of the numerical approximation of the vorticity. Thanks to the above two conditions, we can arbitrarily add constants to the vorticity function and construct high-order MPP LDG methods on overlapping meshes for the two-dimensional incompressible Euler equation in the vorticity stream function formulation. Numerical tests will be given to demonstrate the good performance of the proposed method.

A time adaptive multirate Dirichlet–Neumann waveform relaxation method for heterogeneous coupled heat equations

AbstractWe consider partitioned time integration for heterogeneous coupled heat equations. First and second order multirate, as well as time‐adaptive Dirichlet‐Neumann Waveform relaxation (DNWR) methods are derived. In 1D and for implicit Euler time integration, we analytically determine optimal relaxation parameters for the fully discrete scheme.We test the robustness of the relaxation parameters on the second order multirate method in 2D. DNWR is shown to be very robust and consistently yielding fast convergence rates, whereas the closely related Neumann‐Neumann Waveform relaxtion (NNWR) method is slower or even diverges.The waveform approach naturally allows for different timesteps in the subproblems. In a performance comparison for DNWR, the time‐adaptive method dominates the multirate method due to automatically finding suitable stepsize ratios. Overall, we obtain a fast, robust, multirate and time adaptive partitioned solver for unsteady conjugate heat transfer.

On minimal focal distance of a focused ultrasound probe for neuromodulation.

Focal distance is a key parameter for a focused ultrasound probe, especially in mouse brain stimulation where targets are right below the skull. A closed-form solution for the minimal focal distance with a given transducer size was derived in this study to facilitate precise focal spot alignment with targets in the mouse brain. The spherical profile corresponding to the minimal focal distance does not produce accurate focusing. An iterative algorithm based on Snell's law was introduced for lens profile calculation. With a suitable step size, an accurate lens profile can be obtained for the minimal focal distance.

A fully adaptive explicit stabilized integrator for advection–diffusion–reaction problems

A novel second order family of explicit stabilized Runge–Kutta–Chebyshev methods for advection–diffusion–reaction equations is introduced. The new methods outperform existing schemes for relatively high Peclet number due to their favorable stability properties and explicitly available coefficients. The construction of the new schemes is based on stabilization using second kind Chebyshev polynomials first used in the construction of the stochastic integrator SK-ROCK. An adaptive algorithm to implement the new scheme is proposed. This algorithm is able to automatically select the suitable step size, number of stages, and damping parameter at each integration step. Numerical experiments that illustrate the efficiency of the new algorithm are presented.

A non-convex optimization framework for large-scale low-rank matrix factorization

Low-rank matrix factorization problems such as non negative matrix factorization (NMF) can be categorized as a clustering or dimension reduction technique. The latter denotes techniques designed to find representations of some high dimensional dataset in a lower dimensional manifold without a significant loss of information. If such a representation exists, the features ought to contain the most relevant features of the dataset. Many linear dimensionality reduction techniques can be formulated as a matrix factorization. In this paper, we combine the conjugate gradient (CG) method with the Barzilai and Borwein (BB) gradient method, and propose a BB scaling CG method for NMF problems. The new method does not require to compute and store matrices associated with Hessian of the objective functions. Moreover, adopting a suitable BB step size along with a proper nonmonotone strategy which comes by the size of convex parameter ηk, results in a new algorithm that can significantly improve the CPU time, efficiency, the number of function evaluation. Convergence result is established and numerical comparisons of methods on both synthetic and real-world datasets show that the proposed method is efficient in comparison with existing methods and demonstrate the superiority of our algorithms.

Parallel Solver for Oscillatory Stiff Systems of ODEs

The aim of this study will be to design Parallel solver (PS) for oscillatory stiff systems of ordinary differential equations (ODEs). PS will be constructed via a type of specially transformed exponentially fitted multinomial approximant in accordance with the behaviour of the solution. The method of interpolation and collocation will be utilized. The principal local truncation errors of PS will be used to derive a suitable step size and decide the error tolerance criteria for establishing the convergence of PS. Some examples of stiff ODEs will be examined and compared with existing methods to show case the efficiency and accuracy of the scheme. Parallel solver will be seen as a unique model for solving stiff ODEs without dependent on absolute stability as required by backward differentiation formula.

CMUA-Watermark: A Cross-Model Universal Adversarial Watermark for Combating Deepfakes

Malicious applications of deepfakes (i.e., technologies generating target facial attributes or entire faces from facial images) have posed a huge threat to individuals' reputation and security. To mitigate these threats, recent studies have proposed adversarial watermarks to combat deepfake models, leading them to generate distorted outputs. Despite achieving impressive results, these adversarial watermarks have low image-level and model-level transferability, meaning that they can protect only one facial image from one specific deepfake model. To address these issues, we propose a novel solution that can generate a Cross-Model Universal Adversarial Watermark (CMUA-Watermark), protecting a large number of facial images from multiple deepfake models. Specifically, we begin by proposing a cross-model universal attack pipeline that attacks multiple deepfake models iteratively. Then, we design a two-level perturbation fusion strategy to alleviate the conflict between the adversarial watermarks generated by different facial images and models. Moreover, we address the key problem in cross-model optimization with a heuristic approach to automatically find the suitable attack step sizes for different models, further weakening the model-level conflict. Finally, we introduce a more reasonable and comprehensive evaluation method to fully test the proposed method and compare it with existing ones. Extensive experimental results demonstrate that the proposed CMUA-Watermark can effectively distort the fake facial images generated by multiple deepfake models while achieving a better performance than existing methods. Our code is available at https://github.com/VDIGPKU/CMUA-Watermark.

Block Solver for Multidimensional Systems of Ordinary Differential Equations

This research study aimed at developing block solver for multidimensional systems (BSMS) of ordinary differential equations. This method will be formulated via interpolation and collocation techniques with multinomial as the basis function approximate. The block solver has the capacity to utilize each principal local truncation errors to generate the convergence criteria that will ensure convergence. Some theoretical properties will be stated. The process for executing the block solver will be done via the idea of the convergence criteria introduced. Step by step implementation algorithm will be specified. Some selected model applications will be worked out and a suitable step size will be determined to satisfy the convergence criteria in order to enhance the accuracy and efficiency of the method. The implementation of BSMS is coded in Mathematica and executed under the platform of Mathematica Kernel 9.

Accelerating the gradient projection algorithm for solving the non-additive traffic equilibrium problem with the Barzilai-Borwein step size

The non-additive traffic equilibrium problem (NaTEP) overcomes the inadequacies of the additivity assumption in traditional traffic equilibrium models by relaxing the cost incurred on each path that is not a simple sum of the link costs on that path. The computation of the NaTEP heavily depends on the efficiency of the step size determination. This paper aims to accelerate the path-based gradient projection (GP) algorithm for solving the NaTEP using the Barzilai-Borwein (BB) step size scheme. The GP algorithm with the BB step size scheme uses the solution information of the last two iterations to determine a suitable step size and avoids extra evaluations of the mapping value. The proposed algorithm only needs to perform one-time projection onto the nonnegative orthant at each iteration. Two approaches with and without column generation are considered in the GP algorithm implementation. A non-additive shortest path algorithm is adopted for the column generation approach. Numerical results on four transportation networks demonstrate the superior efficiency and robustness of the GP algorithm with the BB step size scheme over the self-adaptive GP method.

Modified narrowband active noise control system with frequency mismatch tolerance

A common system used for canceling sinusoidal noise caused by revolving equipment such as machines, propellers and motors is the narrowband active noise control (NANC). In NANC, the frequency mismatch (FM) between a reference signal and the primary noise influences the noise reduction performance. The impact of the FM is eliminated by estimating frequency and changing the step size. The conventional method changes the step size by hand, which includes a significant expense and is not unreliabl. In this paper, a step size optimization algorithm with golden section search (GSS) and virtual system was proposed, which can automatically find the most suitable step size. The optimal step size was obtained by comparing the mean square errors at the golden section points. The virtual system was utilized to isolate the noise reduction system and the step size optimization system. Along these lines, the proposed method enhanced the noise decrement of the existing NANC system. Simulations were undertaken to evaluate the performance of the proposed algorithm.

An analysis of stochastic variance reduced gradient for linear inverse problems * *The work of BJ is supported by UK EPSRC Grant EP/T000864/1. The work of JZ was substantially supported by Hong Kong RGC General Research Fund (Projects 14306718 and 14306719).

Stochastic variance reduced gradient (SVRG) is a popular variance reduction technique for accelerating stochastic gradient descent (SGD). We provide a first analysis of the method for solving a class of linear inverse problems in the lens of the classical regularization theory. We prove that for a suitable constant step size schedule, the method can achieve an optimal convergence rate in terms of the noise level (under suitable regularity condition) and the variance of the SVRG iterate error is smaller than that by SGD. These theoretical findings are corroborated by a set of numerical experiments.

DESIGN OF POROUS STEP BEARING BY CONSIDERING DIFFERENT FERRO FLUID LUBRICATION FLOW MODELS

The aim of this article is to carry out analysis to enhance the performance of the ferro fluid lubricated porous step bearing. The porous coating is assorted to the lower flat impermeable surface. A step is there in upper surface which approaching to lower surface. This study considered that magnetic field is flexible and oblique to the lower surface. By considering Jenkins Model, expressions for pressure and load capacity are obtained. The non-dimensional load capacity is also calculated for various parameters. Based on results, it is observed that load capacity increases if suitable step size is considered. Also, comparison between R. E. Rosensweig Model and Jenkins Model is carried out. Finally, it is advised to design a porous step bearing, one should consider Jenkins Model over R. E. Rosensweig Model.

Bregman primal\u2013dual first-order method and application to sparse semidefinite programming

We present a new variant of the Chambolle–Pock primal–dual algorithm with Bregman distances, analyze its convergence, and apply it to the centering problem in sparse semidefinite programming. The novelty in the method is a line search procedure for selecting suitable step sizes. The line search obviates the need for estimating the norm of the constraint matrix and the strong convexity constant of the Bregman kernel. As an application, we discuss the centering problem in large-scale semidefinite programming with sparse coefficient matrices. The logarithmic barrier function for the cone of positive semidefinite completable sparse matrices is used as the distance-generating kernel. For this distance, the complexity of evaluating the Bregman proximal operator is shown to be roughly proportional to the cost of a sparse Cholesky factorization. This is much cheaper than the standard proximal operator with Euclidean distances, which requires an eigenvalue decomposition.

Application of XRF Scanning to Different Geological Archives

AbstractHigh‐resolution XRF scanning is widely used on marine and lake sediment cores as a means of rapidly acquiring elemental concentrations from closely spaced intervals with no damage to the samples. Therefore, guidance on how to obtain reliable datasets and select suitable step sizes for different geological media are of great importance. Here we apply this efficient analytical method to loess, stalagmite, and tridacna samples. The results show that 11 elements (Al, Si, K, Ca, Ti, Mn, Fe, Rb, Sr, Zr, and Ba), eight elements (Ca, Si, K, Fe, Cu, Ni, Sc, and Sr) and seven elements (Ca, Sr, Cu, Fe, Sc, Ni, and Si) can be robustly detected by this method for loess, stalagmite and tridacna archives respectively, demonstrating the capacity to reconstruct high‐resolution paleoclimate changes. For loess cores, efficiency emission decay of the X‐ray tube, water content, matrix, and grain size effects are the main factors influencing elemental intensities. The efficiency emission decay of the X‐ray tube, scanning path, scanning interval, and radiation area have a significant effect on element intensities of stalagmite and tridacna samples. Based on our investigations, we suggest optimal resolutions for scanning these three archives for millennial to seasonal‐scale paleoclimatic reconstructions. We compare our analyses with existing results from traditional (discrete) analyses, demonstrating similar scales of variability. Our results suggest that geochemical proxies measured by XRF scanning, such as Rb/Sr, Zr/Rb, and Ca/Ti ratios of loess and the Sr/Ca ratio of speleothem and tridacna, can be effectively used to reconstruct high‐resolution paleoclimate changes.

Linearized symmetric multi-block ADMM with indefinite proximal regularization and optimal proximal parameter

The proximal term plays a significant role in the literature of proximal Alternating Direction Method of Multipliers (ADMM), since (positive-definite or indefinite) proximal terms can promote convergence of ADMM and further simplify the involved subproblems. However, an overlarge proximal parameter decelerates the convergence numerically, though the convergence can be established with it. In this paper, we thus focus on a Linearized Symmetric ADMM (LSADMM) with proper proximal terms for solving a family of multi-block separable convex minimization models, and we determine an optimal (smallest) value of the proximal parameter while convergence of this LSADMM can be still ensured. Our LSADMM partitions the data into two group variables and updates the Lagrange multiplier twice in different forms with suitable step sizes. The region of the proximal parameter, involved in the second group subproblems, is partitioned into the union of three different sets. We show the global convergence and sublinear ergodic convergence rate of LSADMM for the two cases, while a counter-example is given to illustrate that convergence of LSADMM can not be guaranteed for the remaining case. Theoretically, we obtain the optimal lower bound of the proximal parameter. Numerical experiments on the so-called latent variable Gaussian graphical model selection problems are presented to demonstrate performance of the proposed algorithm and the significant advantage of the optimal lower bound of the proximal parameter.

Some Runge-Kutta Algorithms with Error Control and Variable Stepsizes for Solving a Class of Differential-Algebraic Equations

In this paper, we propose and discuss numerical algorithms for solving a class of nonlinear differential-algebraic equations (DAEs). These algorithms are based on half-explicit Runge-Kutta methods (HERK) that have been studied recently for solving strangeness-free DAEs. The main idea of this work is to use the half-explicit variants of some well-known embedded Runge-Kutta methods such as Runge-Kutta-Fehlberg and Dormand-Prince pairs. Thus, we can estimate local errors and choose suitable stepsizes accordingly to a given tolerance. The cases of unstructured and structured DAEs are investigated and compared. Finally, some numerical experiences are given for illustrating the efficiency of the algorithms.

High-capacity PVO-based reversible data hiding scheme using changeable step size

Reversible data hiding (RDH) algorithm based on pixel-value-ordering (PVO) has received widespread attention because of its excellent performance. PVO algorithm divides the host image into non-overlapped equal-sized blocks, then achieves data embedding by modifying the maximum and minimum values of each block. Every pixel block can be a host of watermark data, so the smaller number of pixel blocks limit the embedding capacity (EC). In our work, a novel PVO with changeable step size (CPVO) is presented which can choose suitable step size based on the number of watermark data bits, even that allow one block to overlap other ones. Take the block size 2 × 2 as an example, we can set step size 2 × 1 or another one in CPVO. Consequently, with a block selection skill based on the noise level (NL) of a pixel block, CPVO can embed more hidden data bits into a host image. Compared to the original PVO-based schemes, experimental results show that our proposed scheme increases the EC by 2 ∼ 3 times, and the marked image quality keeps much higher, even outperforms some other state-of-the-art works in some test images.