Randomized Kaczmarz method for solving total least squares solution of multilinear equations

On spectral properties and fast initial convergence of the Kaczmarz method

ABSTRACT By rigorously analyzing the asymptotic convergence property of the relaxed greedy randomized Kaczmarz method, we obtain an upper bound on its asymptotic convergence rate that is sharper than the existing one. Also, this upper bound naturally leads to a strengthened convergence guarantee for the greedy randomized Kaczmarz method. Consequently, this result improves and enriches the convergence theory of the greedy randomized Kaczmarz method with or without a relaxation parameter, when these iteration methods are applied to solve large‐scale and consistent system of linear equations.

Extended sparse Kaczmarz method with surrogate hyperplane for sparse solutions to inconsistent linear systems

A novel multi-step randomized Kaczmarz method for solving large linear systems

Accelerated Kaczmarz methods via randomized sketch techniques for solving consistent linear systems

In this paper, we propose a novel adaptive stochastic extended iterative method, which can be viewed as an improved extension of the randomized extended Kaczmarz method, for finding the unique minimum Euclidean norm least-squares solution of a given linear system. In particular, we introduce three equivalent stochastic reformulations of the linear least-squares problem: stochastic unconstrained and constrained optimization problems, and the stochastic multiobjective optimization problem. We then alternately employ the adaptive variants of the stochastic heavy ball momentum (SHBM) method, which utilize iterative information to update the parameters, to solve the stochastic reformulations. We prove that our method converges R R -linearly in expectation, addressing an open problem in the literature related to designing theoretically supported adaptive SHBM methods. Numerical experiments show that our adaptive stochastic extended iterative method has strong advantages over the nonadaptive one.

This paper presents a novel framework for the analysis and design of randomized algorithms for solving linear systems, including consistent or inconsistent, full rank or rank-deficient. The framework is formulated with four randomized sampling parameters, which allows for the unification of existing randomization algorithms, such as the doubly stochastic Gauss-Seidel (DSGS) method, randomized Kaczmarz (RK) method, and randomized coordinate descent (RCD) method. Compared with the projection-based block algorithms where a pseudoinverse for solving a least-squares problem is utilized at each iteration, our design is pseudoinverse-free. Furthermore, the flexibility of the new approach also enables the design of a number of new methods as special cases. Polyak's heavy ball momentum technique is also incorporated into the framework to improve the convergence behaviour of the method. An alternative convergence analysis of momentum variants of randomized iterative methods is proposed, where smaller convergence factors for RK and RCD with momentum are obtained. Additionally, an accelerated linear rate for the case of the norm of expected iterates is proven. Finally, numerical experiments are provided to confirm our results.

Greedy Kaczmarz methods for nonlinear equation

Accelerating convergence of randomized extended Kaczmarz methods with double-space residuals

The exponential growth in data generation has driven the search for more efficient solutions to classical computational problems, especially in numerical linear algebra. In this context, randomization techniques have stood out for enabling fast and accurate approximations of operations such as matrix decomposition, solutions of linear systems, and dimensionality reduction. This article investigates the foundations, algorithms, and applications of the main randomized approaches, including Random Projection, Randomized SVD, and the randomized Kaczmarz method. Computational experiments were conducted using simulated matrices, comparing performance and relative error against deterministic methods. The results demonstrate significant gains in execution time with minimal loss of precision, highlighting the potential of these techniques in contexts such as data science, logistics, engineering, and cloud computing. It is concluded that randomization represents a powerful and promising tool in modern scientific computing, especially in environments with large volumes of data.

A surrogate hyperplane Kaczmarz method with oblique projection for solving linear systems

Inverse Laplace transforms (ILTs) are fundamental to a wide range of scientific and engineering applications—from diffusion NMR spectroscopy to medical imaging—yet their numerical inversion remains severely ill-posed, particularly in the presence of noise or sparse data. The primary objective of this study is to develop robust and efficient numerical methods that improve the stability and accuracy of ILT reconstructions under challenging conditions. In this work, we introduce a novel family of Kaczmarz-based ILT solvers that embed advanced regularization directly into the iterative projection framework. We propose three algorithmic variants—Tikhonov–Kaczmarz, total variation (TV)–Kaczmarz, and Wasserstein–Kaczmarz—each incorporating a distinct penalty to stabilize solutions and mitigate noise amplification. The Wasserstein–Kaczmarz method, in particular, leverages optimal transport theory to impose geometric priors, yielding enhanced robustness for multi-modal or highly overlapping distributions. We benchmark these methods against established ILT solvers—including CONTIN, maximum entropy (MaxEnt), TRAIn, ITAMeD, and PALMA—using synthetic single- and multi-modal diffusion distributions contaminated with 1% controlled noise. Quantitative evaluation via mean squared error (MSE), Wasserstein distance, total variation, peak signal-to-noise ratio (PSNR), and runtime demonstrates that Wasserstein–Kaczmarz attains an optimal balance of speed (0.53 s per inversion) and accuracy (MSE = 4.7×10−8), while TRAIn achieves the highest fidelity (MSE = 1.5×10−8) at a modest computational cost. These results elucidate the inherent trade-offs between computational efficiency and reconstruction precision and establish regularized Kaczmarz solvers as versatile, high-performance tools for ill-posed inverse problems.

K-means clustering based maximal residual (block) Kaczmarz methods for solving large scale system of linear equations

ABSTRACTWhen solving linear systems , , and are given, but the measurements often contain corruptions. Inspired by recent work on the quantile‐randomized Kaczmarz method, we propose an acceleration of the randomized Kaczmarz method in the uncorrupted setting using quantile information. We show that the proposed acceleration converges faster than the randomized Kaczmarz algorithm. In addition, we show that our proposed approach can be used in conjunction with the quantile‐randomized Kaczmarz algorithm, without adding additional computational complexity, to produce both a fast and robust iterative method for solving large, sparsely corrupted linear systems that are sufficiently well‐conditioned. Our extensive experimental results support the use of the revised algorithm.

Modified partially randomized extended Kaczmarz method with residual for solving large sparse linear systems

Abstract The randomized Kaczmarz methods are a popular and effective family of iterative methods for solving large-scale linear systems of equations, which have also been applied to linear feasibility problems. In this work, we propose a new block variant of the randomized Kaczmarz method, B-MRK, for solving linear feasibility problems defined by matrices. We show that B-MRK converges linearly in expectation to the feasible region. Furthermore, we extend the method to solve tensor linear feasibility problems defined under the tensor t-product. A tensor randomized Kaczmarz (TRK) method, TRK-L, is proposed for solving linear feasibility problems that involve mixed equality and inequality constraints. Additionally, we introduce another TRK method, TRK-LB, specifically tailored for cases where the feasible region is defined by linear equality constraints coupled with bound constraints on the variables. We show that both of the TRK methods converge linearly in expectation to the feasible region. Moreover, the effectiveness of our methods is demonstrated through numerical experiments on various Gaussian random data and applications in image deblurring.

The global block Kaczmarz method using double greedy strategy

The quaternion relaxed greedy randomized Kaczmarz method with adaptive parameters for solving quaternion matrix equation

The calculation of grasping force and displacement is important for multi-fingered stable grasping and research on slipping damage. By linearizing the friction cone, the robot multi-fingered grasping problem can be represented as a linear complementarity problem (LCP) with a saddle-point coefficient matrix. Because the solution methods for LCP proposed in the field of numerical computation cannot be applied to this problem and the Pivot method can only be used for solving specific grasping problems, the LCP is converted into a non-smooth system of equations for solving it. By combining the Newton method with the subgradient and Kaczmarz methods, a Newton-subgradient non-smooth greedy randomized Kaczmarz (NSNGRK) method is proposed to solve this non-smooth system of equations. The convergence of the proposed method is established. Our numerical experiments indicate its feasibility and effectiveness in solving the grasping force and displacement problems of multi-fingered grasping.