Overrelaxation Parameter Research Articles

Finitely convergent iterative methods with overrelaxations revisited

We study the finite convergence of iterative methods for solving convex feasibility problems. Our key assumptions are that the interior of the solution set is nonempty and that certain overrelaxation parameters converge to zero, but with a rate slower than any geometric sequence. Unlike other works in this area, which require divergent series of overrelaxations, our approach allows us to consider some summable series. By employing quasi-Fejérian analysis in the latter case, we obtain additional asymptotic convergence guarantees, even when the interior of the solution set is empty.

Overrelaxed Sinkhorn–Knopp Algorithm for Regularized Optimal Transport

This article describes a set of methods for quickly computing the solution to the regularized optimal transport problem. It generalizes and improves upon the widely used iterative Bregman projections algorithm (or Sinkhorn–Knopp algorithm). We first proposed to rely on regularized nonlinear acceleration schemes. In practice, such approaches lead to fast algorithms, but their global convergence is not ensured. Hence, we next proposed a new algorithm with convergence guarantees. The idea is to overrelax the Bregman projection operators, allowing for faster convergence. We proposed a simple method for establishing global convergence by ensuring the decrease of a Lyapunov function at each step. An adaptive choice of the overrelaxation parameter based on the Lyapunov function was constructed. We also suggested a heuristic to choose a suitable asymptotic overrelaxation parameter, based on a local convergence analysis. Our numerical experiments showed a gain in convergence speed by an order of magnitude in certain regimes.

Finitely convergent deterministic and stochastic iterative methods for solving convex feasibility problems

We propose finitely convergent methods for solving convex feasibility problems defined over a possibly infinite pool of constraints. Following other works in this area, we assume that the interior of the solution set is nonempty and that certain overrelaxation parameters form a divergent series. We combine our methods with a very general class of deterministic control sequences where, roughly speaking, we require that sooner or later we encounter a violated constraint if one exists. This requirement is satisfied, in particular, by the cyclic, repetitive and remotest set controls. Moreover, it is almost surely satisfied for random controls.

Low Complexity Iterative Rake Decision Feedback Equalizer for Zero-Padded OTFS Systems

This paper presents a linear complexity iterative rake detector for the recently proposed orthogonal time frequency space (OTFS) modulation scheme. The basic idea is to extract and coherently combine the received multipath components of the transmitted symbols in the delay-Doppler grid using maximal ratio combining (MRC) to improve the SNR of the combined signal. We reformulate the OTFS input-output relation in simple vector form by placing guard null symbols or zero padding (ZP) in the delay-Doppler grid and exploiting the resulting circulant property of the blocks of the channel matrix. Using this vector input-output relation we propose a low complexity iterative decision feedback equalizer (DFE) based on MRC. The performance and complexity of the proposed detector favorably compares with the state of the art message passing detector. An alternative time domain MRC based detector is also proposed for even faster detection. We further propose a Gauss-Seidel based over-relaxation parameter in the rake detector to improve the performance and the convergence speed of the iterative detection. We also show how the MRC detector can be combined with outer error-correcting codes to operate as a turbo DFE scheme to further improve the error performance. All results are compared with a baseline orthogonal frequency division multiplexing (OFDM) scheme employing a single tap minimum mean square error (MMSE) equalizer.

Enhancing lpCMFD Acceleration with Successive Overrelaxation for Neutron Transport Source Iteration

We present the new iterative method lpCMFD-SOR, which combines the linear prolongation coarse-mesh finite difference (lpCMFD) scheme with the method of successive overrelaxation (SOR) for neutron transport source iteration (SI). The lpCMFD method is the latest coarse-mesh finite difference (CMFD)–type acceleration scheme and is unconditionally stable and more effective than the standard CMFD method. The SOR method is a variant of the Gauss-Seidel method for solving a linear system of equations, resulting in faster convergence. The idea is to update the scattering source with overrelaxation to speed up the coupled transport-diffusion SI. Fourier analysis shows that the lpCMFD-SOR method converges for a relaxation parameter in the range of . It becomes less effective when underrelaxed (i.e., ) and increasingly more effective as increases above 1 until reaching the optimal overrelaxation value, which is, however, problem dependent. The optimal overrelaxation parameter increases with both the scattering ratio and the optical thickness of the problem. Numerical experiments have confirmed the Fourier analysis results. In general, the SOR method can further enhance the convergence rate of the lpCMFD method by more than 40% for neutron transport problems.

Douglas--Rachford Splitting and ADMM for Nonconvex Optimization: Tight Convergence Results

Although originally designed and analyzed for convex problems, the alternating direction method of multipliers (ADMM) and its close relatives, Douglas-Rachford splitting (DRS) and Peaceman-Rachford splitting (PRS), have been observed to perform remarkably well when applied to certain classes of structured nonconvex optimization problems. However, partial global convergence results in the nonconvex setting have only recently emerged. In this paper we show how the Douglas-Rachford envelope (DRE), introduced in 2014, can be employed to unify and considerably simplify the theory for devising global convergence guarantees for ADMM, DRS and PRS applied to nonconvex problems under less restrictive conditions, larger prox-stepsizes and over-relaxation parameters than previously known. In fact, our bounds are tight whenever the over-relaxation parameter ranges in $(0,2]$. The analysis of ADMM uses a universal primal equivalence with DRS that generalizes the known duality of the algorithms.

Some convergence results of OS-EM algorithm for ill-posed integral equations

ABSTRACT In this paper, we study the properties of iteration OS-EM reconstruction method. We propose a speed-enhanced tomographic reconstruction algorithm that is accelerated to ordered-subset expectation-maximization by overrelaxation parameters. In the case of exact data, we concentrate on showing the monotonicity properties of the methods and convergence to the solution of the equation. Furthermore, in the case of perturbed data, we obtain some similar qualitative conclusions of the EM reconstruction method.

Convergence analysis of an accelerated expectation‐maximization algorithm for ill‐posed integral equations

The maximum‐likelihood expectation‐maximization (EM) algorithm has attracted considerable interest in single‐photon emission computed tomography, because it produces superior images in addition to be being flexible, simple, and allowing a physical interpretation. However, it often needs a large number of calculations because of the algorithm's slow rate of convergence. Therefore, there is a large body of literature concerning the EM algorithm's acceleration. One of the accelerated means is increasing an overrelaxation parameter, whereas we have not found any analysis in this method that would provide an immediate answer to the questions of the convergence. In this paper, our main focus is on the continuous version of an accelerated EM algorithm based on Lewitt and Muehllenner. We extend their conclusions to the infinite‐dimensional space and interpret and analyze the convergence of the accelerated EM algorithm. We also obtain some new properties of the modified algorithm. Copyright © 2015 John Wiley & Sons, Ltd.

The convergence of the block cyclic projection with an overrelaxation parameter for compressed sensing based tomography

The convergence of the block cyclic projection for compressed sensing based tomography (BCPCS) algorithm had been proven recently in the case of underrelaxation parameter λ∈(0,1]. In this paper, we prove its convergence with overrelaxation parameter λ∈(1,2). As a result, the convergence of the other two algorithms (BCAVCS and BDROPCS) with overrelaxation parameter λ∈(1,2) in a special case is derived. Experiments are given to demonstrate the convergence behavior of the BCPCS algorithm with different values of λ.

Total antioxidant capacity in acute pancreatitis

After introducing the parallel Schwarz overrelaxation method for linear systems, we analyse the convergence factor of the method in detail. The optimal overrelaxation parameter ω of the method is discussed in this paper. Some examples are also shown.

Preface

Preface

Second‐order treatment of the interface of domain decomposition method for parabolic problems

AbstractEstimations for the values on interface lines are necessary in a domain decomposition method. However, the accuracy of the estimations is of the first order for most of unconditionally stable domain decomposition schemes. In this paper, a second order of accuracy for the estimations on interface lines is presented. With the new scheme, the optimal number of decomposed subdomains has been observed and proposed. Moreover when the SOR iterative method is used, the optimal over‐relaxation parameter is also studied. Copyright © 2010 John Wiley & Sons, Ltd.

A Modified Nonlinear Inexact Uzawa Algorithm with a Variable Relaxation Parameter for the Stabilized Saddle Point Problem

This paper proposes a modified nonlinear inexact Uzawa (MNIU) algorithm for solving the stabilized saddle point problem, by introducing a variable overrelaxation parameter to speed up convergence. MNIU is an inner-outer iteration method with variable inner accuracy. We give a detailed error analysis for the convergence of MNIU, based upon a newly defined error norm which helps to handle variable inner accuracy for the Uzawa method. We also simply formulate the optimal overrelaxation parameter. Sufficient conditions are given for the convergence of MNIU. We show that MNIU converges in a relatively large range for the variable inner accuracy setting. For constant inner accuracy $\delta$, MNIU is convergent when $\delta<1$, too. Compared with the original nonlinear inexact Uzawa algorithm (NIU) that converges only for constant accuracy and $\delta<1/3$, this is a significant improvement. We also show a practical approach for estimating the optimal overrelaxation parameter for numerical computation. Numerical experiments of MNIU are given and compared with other methods. These results confirm the significant improvements of MNIU.

Convergence of SOR in MoM Analysis of Array Antenna

SUMMARY Convergence of the iterative method based on the successive overrelaxation (SOR) method is investigated to solve the matrix equation in the moment analysis of array antennas. It is found this method can be applied to the sub domain method of moments with fast convergence if the grouping technique is applied and the over-relaxation parameter is properly selected, and the computation time for solving the matrix equation can be reduced to be almost proportional to the second power of the

Overrelaxation Applied to Lower-Upper Symmetric Gauss-Seidel Method for Hypersonic Flow

An implicit time-marching method for acceleration of the convergence of solution for the compressible Navier-Stokes equations is presented. This is based on the lower-upper symmetric-Gauss-Seidel (LU-SGS) method proposed by Yoon and Jameson. A very simple overrelaxation is additionally executed in the LU-SGS method coupled with diagonal flux splitting. Then, an overrelaxation parameter w, a parameter similar to that in the well-known successive-overrelaxation method is introduced. The calculated results for a hypersonic viscous flows around a cylinder and through a compression corner indicate that the convergence is accelerated by a proper value of ω(1 < ω < 2). This method is also applied to a hypersonic thermochemical nonequilibrium flow around a sphere

Subsets and overrelaxation in iterative imagereconstruction

A number of iterative image reconstruction algorithms were integrated into one formula characterizing each algorithm by only two parameters: overrelaxation and number of subsets. From the formula it follows that the ordered-subsets iteration (OS-EM) is equivalent to iteration with overrelaxation, where the OS level corresponds to the overrelaxation parameter. Algorithms represented by the formula were studied with respect to speed of convergence and image characteristics. In particular, OS-EM was compared with a single-projection iteration procedure using an optimized sequence of overrelaxation parameters (HOSP) which combines rapid convergence with reduced storage requirements. As a result, OS-EM with a constant number of subsets either needed more iteration steps than HOSP or provoked additional noise, depending on the number of subsets used during iteration. OS-EM can be improved by using decreasing OS levels, imitating the decreasing overrelaxation parameters used for HOSP. The resulting OS-EM may be slightly more rapid than HOSP, due to the increasing number of projections used simultaneously.

Computation of high overrelaxation parameters in iterative image reconstruction

Single-projection iterative reconstruction allows the use of high overrelaxation parameters. The parameters can be optimized in a pragmatic way by studying their influence on figures of merit extracted from the image. The resulting parameters are high in the beginning and drop down during further iteration. The value of the overrelaxation parameter which gives the best image after one iteration step is easily found. However, this value will change if an optimum image is postulated after two or more iteration steps. A procedure has been developed which rapidly finds the parameters to be used during a sequence of iteration steps to obtain optimum results for a later approximation. Comparing optimization during one and two steps, the initial parameters obtained with two-step optimization are about twice as high as those with one-step optimization, accelerating convergence by nearly a factor of two. Optimization within three or more steps again leads to different parameter sequences which, however, provide only minor convergence improvement. This indicates that different sequences of parameters may result in similar images, and parameters derived from a typical data set may be widely used for other data. Computation of new parameter sequences by the optimizing procedure is recommended if data from new equipment or with new characteristics have to be processed.

Iterative reconstruction of PET images using a high-overrelaxation single-projection algorithm

An iterative image reconstruction procedure is described which is able to calculate high-precision images within eight iterative steps. High initial overrelaxation parameters are used which drop down towards about one during continuing iteration. The parameters are determined pragmatically, postulating maximum gain of image quality during a sequence of iterative steps. Results with simulated and measured data show that parameters derived from one data set may be widely used for other data sets. The acceleration of iterative reconstruction achieved by the estimation of optimum overrelaxation parameters is important for large data sets, especially for fully 3D reconstruction.

Remarks on the optimal convolution kernel for CSOR waveform relaxation

The convolution SOR waveform relaxation method is a numerical method for solving large-scale systems of ordinary differential equations on parallel computers. It is similar in spirit to the SOR acceleration method for solving linear systems of algebraic equations, but replaces the multiplication with an overrelaxation parameter by a convolution with a time-dependent overrelaxation function. Its convergence depends strongly on the particular choice of this function. In this paper, an analytic expression is presented for the optimal continuous-time convolution kernel and its relation to the optimal kernel for the discrete-time iteration is derived. We investigate whether this analytic expression can be used in actual computations. Also, the validity of the formulae that are currently used to determine the optimal continuous-time and discrete-time kernels is extended towards a larger class of ODE systems.

On iterative methods for solving a semi-linear eigenvalue problem

For the semi-linear eigenvalue problem of the form Ax = lF{x), where F is a nonlinear map-ping, we present some methods for numerical solution. For this problem, we first describe a practical SOR method: in this method, the overrelaxation parameter automatically estimated is used instead of the optimum value, since the eigenvalue is not known a priori. We discuss the convergence of this Newton-like method. We then present a conjugate gradient (CG) method for the eigenvalue problem. We also discuss some preconditioning techniques for the present CG method. Finally, a comparison of the convergence rates for these methods is made with a numerical example.