Minimum-norm Least-squares Solution Research Articles

Randomized Kaczmarz method with adaptive stepsizes for inconsistent linear systems

We investigate the randomized Kaczmarz method that adaptively updates the stepsize using readily available information for solving inconsistent linear systems. A novel geometric interpretation is provided which shows that the proposed method can be viewed as an orthogonal projection method in some sense. We prove that this method converges linearly in expectation to the unique minimum Euclidean norm least-squares solution of the linear system, and provide a tight upper bound for the convergence of the proposed method. Numerical experiments are also given to illustrate the theoretical results.

A novel quaternion linear matrix equation solver through zeroing neural networks with applications to acoustic source tracking

<abstract><p>Due to its significance in science and engineering, time-varying linear matrix equation (LME) problems have received a lot of attention from scholars. It is for this reason that the issue of finding the minimum-norm least-squares solution of the time-varying quaternion LME (ML-TQ-LME) is addressed in this study. This is accomplished using the zeroing neural network (ZNN) technique, which has achieved considerable success in tackling time-varying issues. In light of that, two new ZNN models are introduced to solve the ML-TQ-LME problem for time-varying quaternion matrices of arbitrary dimension. Two simulation experiments and two practical acoustic source tracking applications show that the models function superbly.</p></abstract>

Some solutions to a third-order quaternion tensor equation

<abstract><p>The paper deals with the third-order quaternion tensor equation. Based on the Qt multiplication operation, we derive solvability conditions and also get the general solution, the least-squares solution, the minimum-norm solution and the minimum-norm least-squares solution of the tensor equation $ \mathcal{A} \ast_{\mathbb{Q}} \mathcal{X} = \mathcal{B} $. Finally, two numerical examples are presented.</p></abstract>

Efficient recursive least squares solver for rank-deficient matrices

Updating a linear least-squares solution can be critical for near real-time signal-processing applications. The Greville algorithm proposes a simple formula for updating the pseudoinverse of a matrix A∈Rn×m with rank r. In this paper, we explicitly derive a similar formula by maintaining a general rank factorization, which we call rank-Greville. Based on this formula, we implemented a recursive least-squares algorithm exploiting the rank-deficiency of A, achieving the update of the minimum-norm least-squares solution in O(mr) operations and, therefore, solving the linear least-squares problem from scratch in O(nmr) operations. We empirically confirmed that this algorithm displays a better asymptotic time complexity than LAPACK solvers for rank-deficient matrices. The numerical stability of rank-Greville was found to be comparable to Cholesky-based solvers. Nonetheless, our implementation supports exact numerical representations of rationals, due to its remarkable algebraic simplicity.

A Method for Radiation Pattern Reconstruction of Phased-Array Antenna

In this letter, we propose an approach for reconstructing the radiation pattern of a phased-array antenna by application of the minimum norm least-square solution. Compared with conventional methods, our construction process only requires fewer beam patterns and distortion is also largely mitigated by compensating the mutual coupling between the elements in the antenna. Our approach has low reconstruction error for the copolar and cross-polar patterns while being valid for a large scanning range. Moreover, we design a dual-polarization microstrip patch array antenna to test the effectiveness of the proposed method.

Inverse partitioned matrix-based semi-random incremental ELM for regression

Incremental extreme learning machine has been verified that it has the universal approximation capability. However, there are two major issues lowering its efficiency: one is that some “random” hidden nodes are inefficient which decrease the convergence rate and increase the structural complexity, the other is that the final output weight vector is not the minimum norm least-squares solution which decreases the generalization capability. To settle these issues, this paper proposes a simple and efficient algorithm in which the parameters of even hidden nodes are calculated by fitting the residual error vector in the previous phase, and then, all existing output weights are recursively updated based on inverse partitioned matrix. The algorithm can reduce the inefficient hidden nodes and obtain a preferable output weight vector which is always the minimum norm least-squares solution. Theoretical analyses and experimental results show that the proposed algorithm has better performance on convergence rate, generalization capability and structural complexity than other incremental extreme learning machine algorithms.

Quantum Algorithms and Circuits for Linear Equations with Infinite or No Solutions

Linear equations with infinite or no solutions play key roles in machine learning and optimization. However, the existed quantum algorithms cannot be applied directly for these classes of equations. In this paper, based on the modification of algorithm (Wossnig et al., Phys. Rev. Lett. 120, 050502 2017), we describe a quantum algorithm to compute the minimal norm solution or minimum norm least-squares solution for equations with infinite or no solutions, respectively. It can be shown that the presented algorithm can achieve an exponential speedup over the best classical algorithm. Furthermore, the corresponding quantum circuit is designed on a quantum computer.

Bias Correction for Polarization Measurement in Phased Array Antenna Via Pattern Reconstruction Method

A reconstruction technique of the antenna pattern for correcting the polarization measurement bias is proposed in this paper. In the case of the prior knowledge of patterns at some scanning angles, the pattern at other beam position can be estimated by application of the minimum norm least-square solution. Once the pattern is reconstructed, the corresponding projection matrix could be obtained in the presence of the mutual coupling and edge effects in the polarimetric phased array antenna. Thereby, the correction matrix for the polarization scattering matrix is repaired. A microstrip patch array antenna is designed to demonstrate the validity of the proposed method, and the performance of bias correction is manifested using several polarimetric variables. Specifically, the bias of differential reflectivity ( $Z_{DR}^{b}$ ), the bias of linear depolarization ratio ( $L_{DR}^{b}$ ) and the phase difference are less than 0.14 dB, 46.85 dB, 3.31° in a wide scanning angle range, respectively.

Weighted Moore-Penrose inverses and fundamental theorem of even-order tensors with Einstein product

We treat even-order tensors with Einstein product as linear operators from tensor space to tensor space, define the null spaces and the ranges of tensors, and study their relationship. We extend the fundamental theorem of linear algebra for matrix spaces to tensor spaces. Using the new relationship, we characterize the least-squares (ℳ) solutions to a multilinear system and establish the relationship between the minimum-norm (N) least-squares (ℳ) solution of a multilinear system and the weighted Moore-Penrose inverse of its coefficient tensor. We also investigate a class of even-order tensors induced by matrices and obtain some interesting properties.

The Minimum Norm Least-Squares Solution in Reduction by Krylov Subspace Methods

In recent years, model order reduction (MOR) of interconnect system has become an important technique to reduce the computation complexity and improve the verification efficiency in the nanometer VLSI design. The Krylov subspaces techniques in existing MOR methods are efficient, and have become the methods of choice for generating small-scale macro-models of the large-scale multi-port RCL networks that arise in VLSI interconnect analysis. Although the Krylov subspace projection-based MOR methods have been widely studied over the past decade in the electrical computer-aided design community, all of them do not provide a best optimal solution in a given order. In this paper, a minimum norm least-squares solution for MOR by Krylov subspace methods is proposed. The method is based on generalized inverse (or pseudo-inverse) theory. This enables a new criterion for MOR-based Krylov subspace projection methods. Two numerical examples are used to test the PRIMA method based on the method proposed in this paper as a standard model.

Fault Estimation for Fuzzy Delay Systems: A Minimum Norm Least Squares Solution Approach.

This paper mainly focuses on the problem of fault estimation for a class of Takagi-Sugeno fuzzy systems with state delays. A minimum norm least squares solution (MNLSS) approach is first introduced to establish a fault estimation compensator, which is able to optimize the fault estimator. Compared with most of the existing fault estimation methods, the MNLSS-based fault estimation method can effectively decrease the effect of state errors on the accuracy of fault estimation. Finally, three examples are given to illustrate the effectiveness and merits of the proposed method.

Projection Methods for Ill-Posed Problems Revisited

Abstract We consider the discretization of least-squares problems for linear ill-posed operator equations in Hilbert spaces. The main subject of this article concerns conditions for convergence of the associated discretized minimum-norm least-squares solution to the exact solution using exact attainable data. The two cases of global convergence (convergence for all exact solutions) or local convergence (convergence for a specific exact solution) are investigated. We review the existing results and prove new equivalent conditions when the discretized solution always converges to the exact solution. An important tool is to recognize the discrete solution operator as an oblique projection. Hence, global convergence can be characterized by certain subspaces having uniformly bounded angles. We furthermore derive practically useful conditions when this holds and put them into the context of known results. For local convergence, we generalize results on the characterization of weak or strong convergence and state some new sufficient conditions. We furthermore provide an example of a bounded sequence of discretized solutions which does not converge at all, not even weakly.

Moore–Penrose inverse of tensors via Einstein product

In this paper, we define the Moore–Penrose inverse of tensors with the Einstein product, and the explicit formulas of the Moore–Penrose inverse of some block tensors are obtained. The general solutions of some multilinear systems are given and we also give the minimum-norm least-square solution of some multilinear systems using the Moore–Penrose inverse of tensors.

Minimum norm least-squares solution to general complex coupled linear matrix equations via iteration

This paper deals with the problem of finding the minimum norm least-squares solution of a quite general class of coupled linear matrix equations defined over field of complex numbers. To this end, we examine a gradient-based approach and present the convergence properties of the algorithm. The highlight of the elaborated results in the current work is using a new sight of view for construction of the gradient-based algorithm which turns out that we can ignore some of the limitations assumed by the authors in the recently published works for the application of the algorithm to obtain the solution of the referred problems. To the best of our knowledge, so far, computing the optimal convergence factor of the algorithm to determine the (least-squares) solution of general complex linear matrix equations has left as a project to be investigated. In the current work, we determine the optimal convergence factor of the algorithm. Some numerical experiments are reported to illustrate the validity of the presented results.

LSMR Iterative Method for General Coupled Matrix Equations

By extending the idea of LSMR method, we present an iterative method to solve the general coupled matrix equations∑k=1qAikXkBik=Ci,i=1,2,…,p, (including the generalized (coupled) Lyapunov and Sylvester matrix equations as special cases) over some constrained matrix groups(X1,X2,…,Xq), such as symmetric, generalized bisymmetric, and(R,S)-symmetric matrix groups. By this iterative method, for any initial matrix group(X1(0),X2(0),…,Xq(0)), a solution group(X1*,X2*,…,Xq*)can be obtained within finite iteration steps in absence of round-off errors, and the minimum Frobenius norm solution or the minimum Frobenius norm least-squares solution group can be derived when an appropriate initial iterative matrix group is chosen. In addition, the optimal approximation solution group to a given matrix group(X¯1,X¯2,…,X¯q)in the Frobenius norm can be obtained by finding the least Frobenius norm solution group of new general coupled matrix equations. Finally, numerical examples are given to illustrate the effectiveness of the presented method.

A condensed Cramer’s rule for the minimum-norm least-squares solution of linear equations

In this paper, we will derive the condensed Cramer’s rule of Werner for minimal-norm least-squares solution of linear equations Ax=b from the Cramer’s rule of Ben-Israel and Verghese. In addition, a new condensed Cramer’s rule will be obtained in this paper.

Preprocessing in matlab inconsistent linear system for a meaningful least squares solution

Mathematical models of many physical/statistical problems are systems of linear equations. Due to measurement and possible human errors/mistakes in modeling/data, as well as due to certain assumptions to reduce complexity, inconsistency (contradiction) is injected into the model, viz. the linear system. While any inconsistent system irrespective of the degree of inconsistency has always a least-squares solution, one needs to check whether an equation i.e. an information is too much inconsistent or, equivalently, too much contradictory. Such an equation will affect/distort the least-squares solution to such an extent that it becomes unacceptable/unfit to be used in a real-world application. We propose an algorithm, in Matlab, which (i) can detect and prune numerically redundant linear equations from the system, if necessary, as these do not add any new information to a non-least-squares model, although they do have significant impact in a least-squares model, (ii) detects contradictory linear equations along with a degree of contradiction (inconsistency index) and then (iii) obtain the minimum norm least-squares solution of the acceptably inconsistent reduced (pruned) linear system as well as that of non-reduced linear system without too contradictory equations. The resulting two solution vectors will be different in general and have important implication in a real-world environment. The algorithms presented in Matlab may reduce the computational and storage complexities and also may improve the accuracy of the solution. These also detect and provide the necessary warning if there exists a highly contradictory equation in the model. In addition, we suggest a thorough relook into the mathematical modeling to determine the reason why unacceptable contradiction has occurred thus prompting one to make necessary corrections/modifications to the models - both mathematical and, if necessary, physical. We will focus here mainly on the non-over-determined linear systems rather than over-determined systems which are often usually the case in a least-squares problem.

The minimum-norm least-squares solution of a linear system and symmetric rank-one updates

In this paper, we study the Moore-Penrose inverse of a symmetric rank-one perturbed matrix from which a finite method is proposed for the minimum-norm least-squares solution to the system of linear equations Ax = b. This method is guaranteed to produce the required result.

Should Pruning be a Pre-Processor of any Linear System?

There are many real-world problems whose mathematical models turn out to be linear systems Ax = b, where A is an m x n matrix. Each equation of the linear system is an information. An information, in a physical problem, such as 4 mangoes, 6 bananas, and 5 oranges cost $10, is mathematically modeled as an equation 4x(sub 1) + 6x(sub 2) + 5x(sub 3) = 10 , where x(sub 1), x(sub 2), x(sub 3) are each cost of one mango, that of one banana, and that of one orange, respectively. All the information put together in a specified context, constitutes the physical problem and need not be all distinct. Some of these could be redundant, which cannot be readily identified by inspection. The resulting mathematical model will thus have equations corresponding to this redundant information and hence are linearly dependent and thus superfluous. Consequently, these equations once identified should be better pruned in the process of solving the system. The benefits are (i) less computation and hence less error and consequently a better quality of solution and (ii) reduced storage requirements. In literature, the pruning concept is not in vogue so far although it is most desirable. It is assumed that at least one information, i.e. one equation is known to be correct and which will be our first equation. In a numerical linear system, the system could be slightly inconsistent or inconsistent of varying degree. If the system is too inconsistent, then we should fall back on to the physical problem (PP), check the correctness of the PP derived from the material universe, modify it, if necessary, and then check the corresponding mathematical model (MM) and correct it. In nature/material universe, inconsistency is completely nonexistent. If the MM becomes inconsistent, it could be due to error introduced by the concerned measuring device and/or due to assumptions made on the PP to obtain an MM which is relatively easily solvable or simply due to human error. No measuring device can usually measure a quantity with an accuracy greater that 0.005% or, equivalently with a relative error less than 0.005%. Hence measurement error is unavoidable in a numerical linear system when the quantities are continuous (or even discrete with extremely large number). Assumptions, though not desirable, are usually made when we find the problem sufficiently difficult to be solved within the available means/tools/resources and hence distort the PP and the corresponding MM. The . error thus introduced in the system could (not always necessarily though) make the system somewhat inconsistent. If the inconsistency (contradiction) is too much then one should definitely not proceed to solve the system in terms of getting a least-squares solution or the minimum-norm least-squares solution. All these solutions will be invariably of no real-world use. If, on the other hand, inconsistency is reasonably low, i.e. the system is near-consistent or, equivalently, has near-linearly-dependent rows, then the foregoing solutions are useful. Pruning in such a near-consistent system should be performed based on the desired accuracy and on the definition of near-linear dependence. In this article, we discuss pruning over various kinds of linear systems and strongly suggest its use as a pre-processor or as a part of an algorithm. Ideally pruning should (i) be a part of the solution process (algorithm) of the system, (ii) reduce both computational error and complexity of the process, and (iii) take into account the numerical zero defined in the context. These are precisely what we achieve through our proposed O(mn2) algorithm presented in Matlab, that uses a subprogram of solving a single linear equation and that has embedded in it the pruning.

An iterative algorithm for a least squares solution of a matrix equation

In this article, we propose an iterative algorithm to compute the minimum norm least-squares solution of AXB+CYD=E, based on a matrix form of the algorithm LSQR for solving the least squares problem. We then apply this algorithm to compute the minimum norm least-squares centrosymmetric solution of min X ‖AXB−E‖ F . Numerical results are provided to verify the efficiency of the proposed method.