Least-squares Solution Research Articles

A new extension of the core inverse

There are a number of extensions of the core inverse represented by the products of known generalized inverses, like the core-EP inverse, DMP inverse, GC inverse and so on. However, none of them is an inner inverse of the matrix, and especially for an arbitrary nilpotent matrix, such inverses are always null. Our goal is to use a new technic to introduce an extension of the core inverse that preserves the interesting property of being an inner inverse of the matrix which need not necessarily be the null matrix of a nilpotent matrix. We define the extended core inverse for square complex matrices combining the sum and the difference of three known generalized inverses. Various properties and representations of the extended core inverse are developed. The extended dual core inverse is investigated too. We apply the extended core inverse to solve some systems of linear equations and one minimization problem. A significant normal equation related to least-squares solutions can be solved using the extended core inverse.

Supervised Machine Learning Techniques for Breeding Value Prediction in Horses: An Example Using Gait Visual Scores.

Gait scores are widely used in the genetic evaluation of horses. However, the nature of such measurement may limit genetic progress since there is subjectivity in phenotypic information. This study aimed to assess the application of machine learning techniques in the prediction of breeding values for five visual gait scores in Campolina horses: dissociation, comfort, style, regularity, and development. The dataset contained over 5000 phenotypic records with 107,951 horses (14 generations) in the pedigree. A fixed model was used to estimate least-square solutions for fixed effects and adjusted phenotypes. Variance components and breeding values (EBV) were obtained via a multiple-trait model (MTM). Adjusted phenotypes and fixed effects solutions were used to train machine learning models (using the EBV from MTM as target variable): artificial neural network (ANN), random forest regression (RFR) and support vector regression (SVR). To validate the models, the linear regression method was used. Accuracy was comparable across all models (but it was slightly higher for ANN). The highest bias was observed for ANN, followed by MTM. Dispersion varied according to the trait; it was higher for ANN and the lowest for MTM. Machine learning is a feasible alternative to EBV prediction; however, this method will be slightly biased and over-dispersed for young animals.

Synthesis of maximally sparse conformal circular arc array with a required beam pattern by unitary matrix pencil method

This paper extends the unitary matrix pencil (UMP) method to synthesize maximally sparse conformal circular-arc array with a required beam pattern. Due to the nonlinearity between the circular-arc array pattern and its element pattern, Fourier transform preprocessing for the required beam pattern is introduced to achieve a mathematical expression, i.e., sum of a series of undamped complex exponentials, which is related to array element positions and their excitations. Then, the UMP method is used to determine the reduced number of elements and their position distributions. Moreover, the complex excitations of array elements are reconstructed by obtaining the least-square solution of an over-determined equation. A set of examples for synthesizing sparse conformal circular-arc arrays with different desired patterns and E-type patch element including the mutual coupling are conducted. Results show that the proposed UMP method can achieve a considerably lower pattern reconstruction error with a reduced number of elements than results in the literature, which demonstrates its effectiveness and robustness.

Underwater maneuvering target motion analysis using delayed azimuth angles

Accurate position information of an underwater target is the premise of successfully executing underwater missions, and this paper focuses on underwater target motion analysis (TMA) using delayed azimuth angles. Considering that the underwater sound speed (around 1500 m/s) is much smaller than the speed of light (3 × 108 m/s), the underwater TMA needs to adopt the non-stop model considering the effect of the target movement during signal propagation on state estimation, instead of the traditional go-stop model ignoring the target movement during signal propagation. The existing non-stop model is based on a constant-velocity target, and the model mismatch will make the estimated results biased when it is applied to a maneuvering target. In this paper, we focus on the constant-acceleration (CA) target and propose two methods to solve this problem. One is to extend the non-stop model to the case of a constant-acceleration target. We build the corresponding least squares (LS) estimator and derive the expression of the root mean square error (RMSE) of the LS solution. The other is to directly offset the estimation bias of the CA go-stop model and a bias-compensated algorithm is proposed to improve the performance. Simulations prove that the proposed methods can result in a smaller RMSE than the traditional CA go-stop model. Their RMSEs decrease with the number of measurements and observers and increase with random angle errors. When the number of measurements is small, the RMSE of the established CA non-stop model is smaller than that of the bias-compensated algorithm. As the number of measurements increases, their estimation precisions become gradually equal. The running time of the proposed methods increases with the number of measurements and observers. When the number of observers is small, the bias-compensated algorithm has the advantage of needing less running time. In addition, the running time of the bias-compensated algorithm does not approximately change with the random angle errors, while that of the CA non-stop model increases with the random angle errors.

Distributed Algorithms for Solving a Least-Squares Solution of Linear Algebraic Equations

Distributed Algorithms for Solving a Least-Squares Solution of Linear Algebraic Equations

Simple MATLAB and Python scripts for multi-exponential analysis.

Multi-exponential decay is prevalent in magnetic resonance spectroscopy, relaxation, and imaging. This paper describes simple MATLAB and Python functions and scripts for regularized multi-exponential analysis methods for 1D and 2D data and example test problems and experiments. Regularized least-squares solutions provide production-quality outputs with robust stopping rules in ~5 and ~20 lines of code for 1D and 2D inversions, respectively. The software provides an open-architecture simple solution for transforming exponential decay data to the distribution of their decay lifetimes. Examples from magnetic resonance relaxation of a complex fluid, a Danish North Sea crude oil, and fluid mixtures in porous materials-brine/crude oil mixture in North Sea reservoir chalk-are presented. Developed codes may be incorporated in other software or directly used by other researchers, in magnetic resonance relaxation, diffusion, and imaging or other physical phenomena that require multi-exponential analysis.

Tensor-Based Least-Squares Solutions for Multirelational Signals and Applications.

The approach of least squares (LSs) has been quite popular and widely adopted for the common linear regression analysis, which can give rise to the solution to an arbitrary critically-, over-, or under-determined system. Such a linear regression analysis can be easily applied for linear estimation and equalization in signal processing for cybernetics. Nonetheless, the current LS approach for linear regression is unfortunately limited to the dimensionality of data, that is, the exact LS solution can involve only a data matrix. As the dimension of data increases and such data need to be represented by a tensor, the corresponding exact tensor-based LS (TLS) solution does not exist due to the lack of a pertinent mathematical framework. Lately, some alternatives such as tensor decomposition and tensor unfolding were proposed to approximate the TLS solutions to the linear regression problems involving tensor data, but these techniques cannot provide the exact or true TLS solution. In this work, we would like to make the first-ever attempt to present a new mathematical framework for facilitating the exact TLS solutions involving tensor data. To demonstrate the applicability of our proposed new scheme, numerical experiments regarding machine learning and robust speech recognition are illustrated and the associated memory and computational complexities are also studied.

Weighted 1MP and MP1 inverses for operators

Abstract The main aim of this paper is to extend the concepts of the 1MP and MP1 inverses defined for rectangular complex matrices. We present the weighted 1MP and MP1 inverses for a bounded linear operator between two Hilbert spaces as two new kinds of generalized inverses. The notions of the weighted 1MP and MP1 inverses are new in the context of rectangular complex matrices too. We establish a number of characterizations and some representations of the weighted 1MP and MP1 inverses. Several operator equations are solved applying the weighted 1MP and MP1 inverses. A special case of one of these equations is the normal equation which is related with the least-squares solution. As consequences of our results, we obtain new properties of the 1MP and MP1 inverses.

Distributed algorithms of solving linear matrix equations via double-layered networks

This paper focuses on designing distributed algorithms for solving the linear matrix equation in the form of AXB=F via a double-layered multi-agent network. Firstly, we equivalently transform the least squares (LS) problem of AXB=F into two subproblems. We introduce a double-layered multi-agent network, in which each agent only has access to partial information of matrices A, B and F, to solve these two subproblems in a distributed structure. Subsequently, we develop continuous-time and discrete-time distributed algorithms to obtain an LS solution of AXB=F. In particular, the sufficient and necessary conditions about the step-sizes are established, under which the discrete-time algorithm can linearly converge to an LS solution. Compared with the previous results in which each agent needs to communicate multiple matrix variables with the neighbors, the proposed algorithms effectively reduce communication burden for each agent since only one matrix variable is communicated in the upper-layer network and the communication variables in the lower-layered network are vectors with lower dimensions than matrices. Finally, an application example is provided to demonstrate the effectiveness of the proposed algorithms.

A Reduced Complexity Acoustic-Based 3D DoA Estimation with Zero Cyclic Sum.

Accurate direction of arrival (DoA) estimation is paramount in various fields, from surveillance and security to spatial audio processing. This work introduces an innovative approach that refines the DoA estimation process and demonstrates its applicability in diverse and critical domains. We propose a two-stage method that capitalizes on the often-overlooked secondary peaks of the cross-correlation function by introducing a reduced complexity DoA estimation method. In the first stage, a low complexity cost function based on the zero cyclic sum (ZCS) condition is used to allow for an exhaustive search of all combinations of time delays between pairs of microphones, including primary peak and secondary peaks of each cross-correlation. For the second stage, only a subset of the time delay combinations with the lowest ZCS cost function need to be tested using a least-squares (LS) solution, which requires more computational effort. To showcase the versatility and effectiveness of our method, we apply it to the challenging acoustic-based drone DoA estimation scenario using an array of four microphones. Through rigorous experimentation with simulated and actual data, our research underscores the potential of our proposed DoA estimation method as an alternative for handling complex acoustic scenarios. The ZCS method demonstrates an accuracy of 89.4%±2.7%, whereas the ZCS with the LS method exhibits a notably higher accuracy of 94.0%±3.1%, showcasing the superior performance of the latter.

Solving least-squares problems in directed networks: A distributed approach

In this paper, we introduce a distributed algorithm that is specifically designed to tackle the least-squares problem within a network system of linear algebraic equations. Our focus is on static directed multi-agent networks, where each agent possesses knowledge of a distinct subset of the linear equations. Furthermore, we examine a scenario where agents lack information about their “out-degrees” at any given time. By imposing the strong connectivity condition on the communication network, we establish that the local estimated solution of each agent exhibits exponential convergence towards the least-squares solution of the corresponding network system of linear algebraic equations.

Unsupervised deep learning with convolutional neural networks for static parallel transmit design: A retrospective study.

To mitigate inhomogeneity at 7T for multi-channel transmit arrays using unsupervised deep learning with convolutional neural networks (CNNs). Deep learning parallel transmit (pTx) pulse design has received attention, but such methods have relied on supervised training and did not use CNNs for multi-channel maps. In this work, we introduce an alternative approach that facilitates the use of CNNs with multi-channel maps while performing unsupervised training. The multi-channel maps are concatenated along the spatial dimension to enable shift-equivariant processing amenable to CNNs. Training is performed in an unsupervised manner using a physics-driven loss function that minimizes the discrepancy of the Bloch simulation with the target magnetization, which eliminates the calculation of reference transmit RF weights. The training database comprises 3824 2D sagittal, multi-channel maps of the healthy human brain from 143 subjects. data were acquired at 7T using an 8Tx/32Rx head coil. The proposed method is compared to the unregularized magnitude least-squares (MLS) solution for the target magnetization in static pTx design. The proposed method outperformed the unregularized MLS solution for RMS error and coefficient-of-variation and had comparable energy consumption. Additionally, the proposed method did not show local phase singularities leading to distinct holes in the resulting magnetization unlike the unregularized MLS solution. Proposed unsupervised deep learning with CNNs performs better than unregularized MLS in static pTx for speed and robustness.

Time-domain extended-source full-waveform inversion: Algorithm and practical workflow

Extended-source full-waveform inversion (ES-FWI) first computes wavefields with data-driven source extensions such that the simulated data in inaccurate velocity models match the observed counterpart sufficiently well to prevent cycle skipping. Then, the source extensions are minimized to update the model parameters toward the true medium. This two-step workflow is iterated until data and sources are matched. It has been recently indicated that the source extensions are the least-squares solutions of the scattered data-fitting problem. As a result, the source extensions are computed by propagating backward in time the deconvolved data residuals by the damped data-domain Hessian of the scattered data-fitting problem. Estimating these weighted data residuals is the main computational bottleneck of time-domain ES-FWI. To mitigate this burden, we approximate the inverse data-domain Hessian by mono- and multidimensional matching filters with two simulations per source. We implement time-domain ES-FWI with the alternating-direction method of multipliers and total-variation regularization. Moreover, we apply ES-FWI with a multiscale approach involving frequency continuation and layer stripping, with the latter being implemented with an offset-time-dependent weighting operator. In this framework, we further regularize the inversions while mitigating their computational burden by matching the grid interval to the frequency bandwidth. Finally, the overall workflow combines ES-FWI and classical FWI during the early and late stages of the multiscale approach, respectively. We illustrate that the sensitivity of ES-FWI to the accuracy of the approximated inverse data-domain Hessian depends on the complexity of the targeted model, the data anatomy, and the accuracy of the starting model. In the case of the 2004 BP salt model, we determine that the layer stripping is necessary when the inverse data-domain Hessian is approximated by a 2D Gabor matching filter and the starting model is crude, whereas this feature is not necessary with the Marmousi II model.

Star Image Registration Modeling and Parameter Calibration for a 3CMOS Star Sensor.

This paper presents an image registration method specifically designed for a star sensor equipped with three complementary metal oxide semiconductor (CMOS) detectors. Its purpose is to register the red-, green-, and blue-channel star images acquired from three CMOS detectors, assuring the precision of star image fusion and centroid extraction in subsequent stages. This study starts with a theoretical analysis aimed at investigating the effect of inconsistent three-channel imaging parameters on the position of feature points. Based on this analysis, this paper establishes a registration model for transforming the red- and blue-channel star images into the green channel's coordinate system. Subsequently, the method estimates model parameters by finding a nonlinear least-squares solution. The experimental results demonstrate the correctness of the theoretical analysis and the proposed registration method. This method can achieve subpixel alignment accuracy in both the x and y directions, thus effectively ensuring the performance of subsequent operation steps in the 3CMOS star sensor.

Least squares solutions of matrix equation $ AXB = C $ under semi-tensor product

<abstract><p>This paper mainly studies the least-squares solutions of matrix equation $ AXB = C $ under a semi-tensor product. According to the definition of the semi-tensor product, the equation is transformed into an ordinary matrix equation. Then, the least-squares solutions of matrix-vector and matrix equations respectively investigated by applying the derivation of matrix operations. Finally, the specific form of the least-squares solutions is given.</p></abstract>

Distributed Algorithms for Linear Equations Over General Directed Networks.

This article deals with linear equations of the form Ax = b . By reformulating the original problem as an unconstrained optimization problem, we first provide a gradient-based distributed continuous-time algorithm over weight-balanced directed graphs, in which each agent only knows partial rows of the augmented matrix (A b) . The algorithm is also applicable to time-varying networks. By estimating a right-eigenvector corresponding to 0 eigenvalue of the out-Laplacian matrix in finite time, we further propose a distributed algorithm over weight-unbalanced communication networks. It is proved that each solution of the designed algorithms converges exponentially to an equilibrium point. Moreover, the convergence rate is given out clearly. For linear equations without solution, these algorithms are used to obtain a least-squares solution in approximate sense. These theoretical results are illustrated by four numerical examples.

An Adaptive Graduated Nonconvexity Loss Function for Robust Nonlinear Least-Squares Solutions

An Adaptive Graduated Nonconvexity Loss Function for Robust Nonlinear Least-Squares Solutions

Exact and least-squares solutions of a generalized Sylvester-transpose matrix equation over generalized quaternions

<abstract><p>We have considered a generalized Sylvester-transpose matrix equation $ AXB + CX^TD = E, $ where $ A, B, C, D, $ and $ E $ are given rectangular matrices over a generalized quaternion skew-field, and $ X $ is an unknown matrix. We have applied certain vectorizations and real representations to transform the matrix equation into a matrix equation over the real numbers. Thus, we have investigated a solvability condition, general exact/least-squares solutions, minimal-norm solutions, and the exact/least-squares solution closest to a given matrix. The main equation included the equation $ AXB = E $ and the Sylvester-transpose equation. Our results also covered such matrix equations over the quaternions, and quaternionic linear systems.</p></abstract>

The best approximation problems between the least-squares solution manifolds of two matrix equations

In this paper, we will deal with the following two classes of best approximation problems about the linear manifolds: <b>Problem 1.</b> Given matrices $ A_1, B_1, C_1, $ and $ D_1 \in {\mathbb R}^{ m \times n} $, find $ d(L_1, L_2) = \min_{X\in L_1, Y\in L_2}\|X-Y\|, $ and find $ \hat{X}\in L_1, \hat{Y}\in L_2 $ such that $ \|\hat{X}-\hat{Y}\| = d(L_1, L_2) $, where $ L_1 = \left\{{X \in {\mathbb {SR}} ^{n \times n} \left|{\ \|A_1X-B_1\| = \min}\right.} \right\} $ and $ L_2 = \left\{{Y \in {\mathbb {SR}} ^{n \times n} \left|{\ \|C_1Y-D_1\| = \min}\right.} \right\} $. <b>Problem 2.</b> Given matrices $ A_2, B_2, E_2, F_2 \in {\mathbb R}^{ m \times n} $ and $ C_2, D_2, G_2, H_2 \in {\mathbb R}^{ n \times p} $, find $ d(L_3, L_4) = \min_{X\in L_3, Y\in L_4}\|X-Y\|, $ and find $ \tilde{X}\in L_3, \tilde{Y}\in L_4 $ such that $ \|\tilde{X}-\tilde{Y}\| = d(L_3, L_4) $, where $ L_3 = \left\{{X \in {\mathbb {R}}^{n \times n} \left|{\ \|A_2X-B_2\|^2+||XC_2-D_2\|^2 = \min}\right.} \right\} $ and $ L_4 = \left\{{Y \in {\mathbb {R}} ^{n \times n} \left|{\ \|E_2Y-F_2\|^2+||YG_2-H_2\|^2 = \min}\right.} \right\} $. We obtain explicit formulas for $ d(L_1, L_2) $ and $ d(L_3, L_4), $ and all the matrices in question by using the singular value decompositions and the canonical correlation decompositions of matrices.

Passive TDOA Emitter Localization Using Fast Hyperbolic Hough Transform

A fast Hough transform (HT)-based hyperbolic emitter localization system is proposed to process time difference of arrival (TDOA) measurements. The position-fixing problem is provided for cases where the source is known to be on a given plane (i.e., the elevation of the source is known), while the sensors can be deployed anywhere in the three-dimensional space. The proposed solution provides fast evaluation and guarantees the determination of the global optimum. Another favorable property of the proposed solution is that it is robust against faulty sensor measurements (outliers). A fast evaluation method involving the hyperbolic Hough transform is proposed, and the global convergence property of the algorithm is proven. The performance of the algorithm is compared to that of the least-squares solution, other HT-based solutions, and the theoretical limit (the Cramér–Rao lower bound), using simulations and real measurement examples.