Least-squares Solution Research Articles

On computing sparse generalized inverses

The M-P (Moore-Penrose) pseudoinverse is used in several linear-algebra applications. It is convenient to construct sparse block-structured matrices satisfying some relevant properties of the M-P pseudoinverse for specific applications. Aiming at row-sparse generalized inverses, we consider 2,1-norm minimization (and generalizations). We show that a 2,1-norm minimizing generalized inverse satisfies two additional M-P properties, including one needed for computing least-squares solutions. We present formulations related to finding row-sparse generalized inverses that can be solved very efficiently, which we verify numerically.

Localization of brain signals by alternating projection

A popular approach for modeling brain activity in MEG and EEG is based on a small set of current dipoles, where each dipole represents the combined activation of a local area of the brain. Here, we address the problem of multiple dipole localization with a novel solution called Alternating Projection (AP). The AP solution is based on minimizing the least-squares (LS) criterion by transforming the multi-dimensional optimization required for direct LS solution, to a sequential and iterative solution in which one source at a time is localized, while keeping the other sources fixed. Results from simulated, phantom, and human MEG data demonstrated the high accuracy of the AP method, with superior localization results than popular scanning methods from the multiple-signal classification (MUSIC) and beamformer families. In addition, the AP method was more robust to forward model errors resulting from head rotations and translations, as well as different cortex tessellation grids for the forward and inverse solutions, with consistently higher localization accuracy in low SNR and highly correlated sources.

Weighted radial basis collocation method for large deformation analysis of rubber-like materials

A nonlinear formulation, based on the total Lagrange description of the weighted radial basis collocation method (WRBCM), is proposed for the large deformation analysis of rubber-like materials where the materials are hyperelastic and nearly incompressible. The WRBCM based on the strong form collocation is a genuinely meshfree method that eliminates the need for meshing. As a result, it effectively circumvents challenges associated with mesh distortion during large deformation analysis. The proper weights that should be imposed on the boundary collocation equations are first derived to achieve the optimal convergence in hyperelasticity. In the WRBCM, the support of the shape function remains unchanged throughout material deformation, thereby guaranteeing the absence of tension instability during large deformation analysis. Additionally, by combining WRBCM with a least-squares solution, volumetric locking in nearly incompressible hyperelastic problems can be suppressed. This is due to the infinite continuity possessed by the radial basis approximation, which ensures the divergence-free condition. Several numerical examples are examined, demonstrating the high accuracy and exponential convergence of WRBCM in hyperelastic large deformation analysis. Moreover, no volumetric locking can be observed which further substantiates the effectiveness of applying nonlinear WRBCM to nearly incompressible hyperelastic problems.

On the Convergence of the Randomized Block Kaczmarz Algorithm for Solving a Matrix Equation

A randomized block Kaczmarz method and a randomized extended block Kaczmarz method are proposed for solving the matrix equation AXB=C, where the matrices A and B may be full-rank or rank-deficient. These methods are iterative methods without matrix multiplication, and are especially suitable for solving large-scale matrix equations. It is theoretically proved that these methods converge to the solution or least-square solution of the matrix equation. The numerical results show that these methods are more efficient than the existing algorithms for high-dimensional matrix equations.

The Block Landweber Iterative Method for Light Field Reconstruction from a Focal Stack

Light field imaging involves reconstructing a 4D light field from a 3D focal stack, which makes it challenging to reconstruct the light field from incomplete projection data. To address this problem, a linear projection system is established to model the focal stack imaging process using discrete refocusing equations. Based on this system, we propose the block Landweber iterative method to find the least-squares solution. This method computes the sparse matrix while iterating, which overcomes the problem of data storage. The 2-norm of the block matrix is utilized as the weighted matrix to normalize every block matrix on an identical scale, delivering an effective relaxation strategy under the convergence condition in the inconsistent case, which yields better reconstruction results and accelerates the convergence speed. The experimental results based on the image quality assessments of reference and non-reference images show that our method achieved better reconstruction results compared to other relevant common methods, even with fewer focal stacks and higher angle resolution.

Application of regularization method with sparsity prior in riser vortex-induced vibration reconstruction

In the experimental investigation or in-situ monitoring of the vortex-induced vibration (VIV) of offshore risers, it is often necessary to reconstruct the full-field structural response based on discrete observations. Conventionally, this can be done using the modal decomposition method. However, the method can be invalid with fewer measurements than unknown modes and is unstable when the data contains measurement noises. This study presents a novel sparse modal decomposition method based on the compressed sensing theory and the Compressive Sampling Matching Pursuit (CoSaMP) algorithm. It formulates the reconstruction problem in a regularization framework with a prior related to the sparsity of the modal coordinates, which differs from the classic method that seeks a least-square solution. Verification tests demonstrate that our method outperforms the classic method regarding accuracy and stability, especially when the measurements are noisy and insufficient, highlighting its potential applications in engineering.

A generalized closed-form solution for 3D registration of two-point sets under isotropic and anisotropic scaling

Fitting of two-point sets under non-uniform scaling involves several applications, including ultrasound-guided surgery, where ultrasound images generally show anisotropy between its lateral and depth resolution. However, the current state-of-the-art only accomplishes the closed-form solution under the assumption of isotropic scaling, and only the iterative form of the solution is available for registration regardless of scaling type. A generalized closed-form solution, including anisotropic scaling, could robust the computation in the applications of multi-modality image fusion and surgical navigation based on ultrasound images. In this paper, a generalized closed-form solution (GCFS) that accounts for isotropic scaling and anisotropic scaling is proposed. GCFS finds the least-squares solution of rotation, translation, and scaling based on the singular value decomposition (SVD). The method was demonstrated to 1) register two 3D point sets under the effect of anisotropic scaling and 2) register two coplanar 3D point sets under the effect of anisotropic scaling. This study evaluates 3D registration between the preoperative model and intraoperative ultrasonic scanning of a femur and 2D registration of ultrasound images to the phantom for ultrasound calibration. In addition, the conditions that affect the transformation estimations like estimating reflection and noise degeneracy were also considered in the evaluation.

A novel estimation procedure for robust CANDECOMP/PARAFAC model fitting

The parameter estimation in CANDECOMP/PARAFAC (CP) is carried out by alternating least squares (ALS) that yields least-squares solutions and provides consistent outcomes. At the same time it has several drawbacks, like sensitivity to the presence of outliers in the data, issues with the computational efficiency in terms of processing time and memory requirements, as well as susceptibility to degeneracy conditions. These weaknesses have been addressed, but there is no outlier-robust procedure that at the same time is highly computationally efficient, especially for large data sets. A novel procedure based on an integrated estimation algorithm is proposed. This is an alternative to ALS, which guards against outliers and is computationally efficient at the same time. The performance of the new method is demonstrated on an extensive simulation study and an empirical example.

Real-Time Tomographic Inversion of Truncated Ionospheric GNSS Radio Occultations

This paper presents a new way of combining Abel inversion and the Chapman model with a linearly increasing scale height to retrieve ionospheric electron density vertical profiles from truncated-sounding radio-occultation data. A linear Vary–Chap model is used to cover the blind region due to data truncation, with parameters estimated by enumeration of the possible values in a grid centered around a set of parameters compatible with ionospheric physics. The resulting electron density is estimated with its corresponding error from the linear least-squares solution presenting the smaller post-fit residual on the input GNSS carrier-phase measurements. The results, tested on a set of representative GNSS RO measurements obtained by COSMIC/FORMOSAT-3, show that this method can retrieve EDVPs with a predominant absolute and relative error of 1010e−m−3 and 5%, respectively, and in less than 10 s per profile, which makes this method suitable for near real-time applications in upcoming missions such as EUMETSAT Polar System-Second Generation.

The QLY least-squares and the QLY least-squares minimal-norm of linear dual least squares problems

ABSTRACT In this paper, we define a QLY total order ≤ Q over D m to compare the magnitude of dual vectors. Then we consider the QLY least-squares problem and give its compact formula. Meanwhile, by comparing with a least-squares and the least-squares minimal-norm solutions, we can investigate a QLY least-squares and the QLY least-squares minimal-norm of linear dual least-squares problems. In particular, in the presence of a least-squares solution, we can get a QLY least-squares solution to be more accurate than a least-squares solution under the QLY total order.

Randomized Kaczmarz method with adaptive stepsizes for inconsistent linear systems

We investigate the randomized Kaczmarz method that adaptively updates the stepsize using readily available information for solving inconsistent linear systems. A novel geometric interpretation is provided which shows that the proposed method can be viewed as an orthogonal projection method in some sense. We prove that this method converges linearly in expectation to the unique minimum Euclidean norm least-squares solution of the linear system, and provide a tight upper bound for the convergence of the proposed method. Numerical experiments are also given to illustrate the theoretical results.

Distributed Linear Equations Over Random Networks

Distributed linear algebraic equation over networks, where nodes hold a part of problem data and cooperatively solve the equation via node-to-node communications, is a basic distributed computation task receiving an increasing research attention. Communications over a network have a stochastic nature, with both temporal and spatial dependence due to link failures, packet dropouts, or node recreation, etc. In this article, we study the convergence and convergence rate of distributed linear equation protocols over a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ast$</tex-math></inline-formula> -mixing random network, where the temporal and spatial dependencies between the node-to-node communications are allowed. When the network linear equation admits exact solutions, we prove the exponential convergence rate of the distributed projection consensus algorithm in the mean-squared sense. Motivated by the randomized Kaczmarz algorithm, we also propose a distributed randomized projection consensus algorithm, where each node randomly selects one row of local linear equations for projection per iteration, and establish an exponential rate of convergence. When the network linear equation admits no exact solution, we prove that a distributed gradient-descent-like algorithm with diminishing step-sizes can drive all nodes’ states to a least-squares solution at a sublinear rate. These results collectively illustrate that distributed computations may overcome communication correlations if the prototype algorithms enjoy certain contractive properties or are designed with suitable parameters.

Efficient Vertical Federated Learning Method for Ridge Regression of Large-Scale Samples

Integrating data from multiple parties to achieve cross-institutional machine learning is an important trend in Industry 4.0 era. However, the privacy risks from sharing data pose a significant challenge to data integration. To integrate data without sharing data and meet large-scale samples' modeling needs, we propose two vertical federation learning algorithms for ridge regression via least-squares solution for two-party and multi-party scenarios, respectively. Compared with the state-of-the-art algorithms, our algorithms only need one round of calculation for the optimization instead of iteration. Furthermore, our algorithms can effectively handle large-scale samples due to the number of cryptographic operations in our algorithms being independent of the number of samples. Through our proposed the matrix secure agent computing theory and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\delta$</tex-math></inline-formula> -data indistinguishability theory, we provide quantitative theoretical guarantees for the security of our algorithms. Our algorithms satisfy complete data indistinguishability under the “semi-honest” assumption and the quantitative security under the “malicious” assumption. The experiments show that our proposed algorithm takes only about 400 seconds to handle up to 9.6 million large-scale samples, while the state-of-the-art algorithms take close to 1000 seconds to handle every 1000 samples, which embodies the advantage of our algorithms in handling large-scale samples.

과도 결정 선형 시스템에서의 보통과 수직 거리 최소 제곱 해에 대한 대수 및 기하적 분석

In this paper, we make an in-depth analysis on the ordinary least-squares solution and the orthogonal distance least-squares solution to an overdetermined linear system, having the number n of constraints greater than the number m of unknowns. Using the barycentric coordinate, algebraic and geometric analysis is made for the case of n=2 and m=3, yielding a constraint triangle, and for the case of n=3 and m=4, yielding a constraint tetrahedron. First, for an n-dimensional overdetermined system, the ordinary least squares solution that minimizes the sum of the squares of m residuals and the orthogonal distance least squares solution that minimizes the sum of the squares of m orthogonal distances are formulated. Next, for the cases of n=2 and m=3, and n=3 and m=4, the following analysis is made: The relationship between the ordinary least-squares solution and the centroid of the constraint triangle/tetrahedron is identified, the changing pattern of the ordinary least-squares solution due to the scaling of an overdetermined system is examined, and the relationship between the orthogonal distance least squares solution and the symmedian point of the constraint triangle/tetrahedron is identified. Finally, to demonstrate the applicability of the ordinary least-squares and the orthogonal distance least-squares solutions, the velocity estimation of a mobile robot using a polygonal array of optical mice is provided.

Least squares decoding in binomial frequency division multiplexing

AbstractThis paper proposes a method that can reduce the complexity of a system matrix by analyzing the characteristics of a pseudoinverse matrix to receive a binomial frequency division multiplexing (BFDM) signal and decode it using the least squares (LS) method. The system matrix of BFDM can be expressed as a band matrix, and as this matrix contains many zeros, its amount of calculation when generating a transmission signal is quite small. The LS solution can be obtained by multiplying the received signal by the pseudoinverse matrix of the system matrix. The singular value decomposition of the system matrix indicates that the pseudoinverse matrix is a band matrix. The signal‐to‐interference ratio is obtained from their eigenvalues. Meanwhile, entries that do not contribute to signal generation are erased to enhance calculation efficiency. We decode the received signal using the pseudoinverse matrix and the removed pseudoinverse matrix to obtain the bit error rate performance and to analyze the difference.

Zeroing Neural Network Based on Neutrosophic Logic for Calculating Minimal-Norm Least-Squares Solutions to Time-Varying Linear Systems

Zeroing Neural Network Based on Neutrosophic Logic for Calculating Minimal-Norm Least-Squares Solutions to Time-Varying Linear Systems

Robust Adaptive Beamforming Based on Covariance Matrix Reconstruction via Steering Vector Estimation

The performance of the sample matrix inverse (SMI) beamformer degrades greatly when the signal-to-noise ratio (SNR) increases because the signal of interest (SOI) is mistaken as interferences and suppressed. To avoid this situation, the interference-plus-noise covariance matrix (INCM) is introduced via steering vector (SV) estimation for robust adaptive beamforming (RAB). To avoid the convex optimization for the SV estimation, a vertical error vector is constructed based on the property of subspace in the proposed method, and the SV error neighborhood table is built in advance to lower the computational complexity. Through the Capon spectrum search, the initial directions of the SOI and interference signals are estimated, and more accurate SVs are confirmed through neighborhood optimization in the table. Next, the interference covariance matrix (ICM) is generated by the estimated SVs and the noise covariance matrix (NCM) is obtained by the least-square (LS) solution based on the corrected SVs. Finally, INCM is reconstructed and the weight vector is computed for RAB. The main advantage of the proposed method is robust against unknown arbitrary-type mismatches. Simulation results demonstrate the effectiveness and robustness of the proposed method.

Generalized n-Dimensional Rigid Registration: Theory and Applications.

The generalized rigid registration problem in high-dimensional Euclidean spaces is studied. The loss function is minimized with an equivalent error formulation by the Cayley formula. The closed-form linear least-square solution to such a problem is derived which generates the registration covariances, i.e., uncertainty information of rotation and translation, providing quite accurate probabilistic descriptions. Simulation results indicate the correctness of the proposed method and also present its efficiency on computation-time consumption, compared with previous algorithms using singular value decomposition (SVD) and linear matrix inequality (LMI). The proposed scheme is then applied to an interpolation problem on the special Euclidean group SE(n) with covariance-preserving functionality. Finally, experiments on covariance-aided Lidar mapping show practical superiority in robotic navigation.

Weighted Radial Basis Collocation Method for the Nonlinear Inverse Helmholtz Problems

In this paper, a meshfree weighted radial basis collocation method associated with the Newton’s iteration method is introduced to solve the nonlinear inverse Helmholtz problems for identifying the parameter. All the measurement data can be included in the least-squares solution, which can avoid the iteration calculations for comparing the solutions with part of the measurement data in the Galerkin-based methods. Appropriate weights are imposed on the boundary conditions and measurement conditions to balance the errors, which leads to the high accuracy and optimal convergence for solving the inverse problems. Moreover, it is quite easy to extend the solution process of the one-dimensional inverse problem to high-dimensional inverse problem. Nonlinear numerical examples include one-, two- and three-dimensional inverse Helmholtz problems of constant and varying parameter identification in regular and irregular domains and show the high accuracy and exponential convergence of the presented method.

A new image restoration model associated with special elliptic quaternionic least-squares solutions based on LabVIEW

In this paper, we take advantage of the elliptic complex matrix representation of elliptic quaternion matrices. Then we obtain the methods of the elliptic quaternionic least-squares solution, the pure elliptic quaternionic least-squares solution, and the real least-squares solution with the least norm of the elliptic quaternion matrix equation AX=B. We also apply the newly obtained method of the pure elliptic quaternionic least-squares solution with the least norm to the color image restoration based on the LabVIEW program. In this context, we propose a new image restoration model called “ELSI image restoration model” associated with elliptic quaternionic least-squares solutions.