Normal Equation Method Research Articles

Sparse Approximate Pseudoinverse Preconditioning for Sparse Supervised Learning Problems with More Features than Samples

In this paper, we propose a new technique for handling a class of sparse supervised learning problems, specifically focusing on regression, binary, and multi-class classification problems. Sparse regression problems are associated with datasets including both arithmetic and categorical features and by encoding the dataset leads to an underdetermined sparse linear system. Furthermore, logistic regression can be used for both sparse binary and multi-class classification problems, solving an underdetermined sparse linear system. A new technique called Sparse Approximate Pseudoinverse Preconditioning (SAPP), namely the Explicit Preconditioned Conjugate Gradient for Normal Equations (EPCGNE) method based on Generic Approximate Sparse Pseudoinverse matrices is introduced for solving underdetermined sparse least square problems. Numerical experiments were carried out demonstrating a significant improvement of the performance metrics for the proposed SAPP scheme compared to other learners.

A predictive TDM-PON resource allocation using the ANN method based on equilibrium points of discrete dynamical systems

Resource allocation is one of the challenges in the time division multiple access (TDM) passive optical network (PON) area. Much research has been done to find an efficient resource allocation method. Using machine learning (ML) and finding proper ML algorithms in predictive dynamic bandwidth allocation remains an open research question. This paper aims to predict resource allocation accurately and efficiently based on historical requests. Artificial neural networks (ANN) with historically allocated bandwidth from different time series are used to build the prediction model. This model is constructed by the normal equation method. The proposed model needs a small dataset for its training phase, while its accuracy is higher than other models. The discrete dynamical system builds to find an equilibrium point, which is a stable point in the system. The proposed model can be implemented as a pre-training, or it can be dynamically refined when the model is running by using some of its output as feedback. The system evaluation verifies that the proposed DBA (EQ-ANN) elevates the quality of service (QoS) metrics, such as system throughput, packet drop probability, and packet means delay. The results show that the system throughput improves by up to 5%, and packet loss improves by up to 15%. Moreover, the prediction accuracy of the system reached 99.48%.

A Normal Equation-Based Extreme Learning Machine for Solving Linear Partial Differential Equations

Abstract This paper develops an extreme learning machine for solving linear partial differential equations (PDEs) by extending the normal equations approach for linear regression. The normal equations method is typically used when the amount of available data is small. In PDEs, the only available ground truths are the boundary and initial conditions (BC and IC). We use the physics-based cost function use in state-of-the-art deep neural network-based PDE solvers called physics-informed neural network (PINN) to compensate for the small data. However, unlike PINN, we derive the normal equations for PDEs and directly solve them to compute the network parameters. We demonstrate our method’s feasibility and efficiency by solving several problems like function approximation, solving ordinary differential equations (ODEs), and steady and unsteady PDEs on regular and complicated geometries. We also highlight our method’s limitation in capturing sharp gradients and propose its domain distributed version to overcome this issue. We show that this approach is much faster than traditional gradient descent-based approaches and offers an alternative to conventional numerical methods in solving PDEs in complicated geometries.

Structure Optimization of Electromagnetic Railgun Armature Based on Machine Learning

The armature is an important part of the electromagnetic railgun. It may break, wear, and transition during the working process, which increases the difficulty of maintaining the railgun and impairs the service life of the railgun. Therefore, studying the armature structure of the electromagnetic railgun is of great significance to the design of the electromagnetic railgun. This paper selects the five armature structure parameters of armature shoulder thickness, tail length, wing tip thickness, front end thickness, and interference, and focuses on "using armature structure parameters to predict prestress" and "using prestress and other armature to predict the interference of structural parameters". This paper uses the linear regression method in machine learning to construct the model, and use the normal equation method and the regularized normal equation method to optimize the loss function. The results show that the normal equation method is generally effective for the electromagnetic railgun structure optimization problem discussed in this paper.

Efficient architecture for improving differential equations based on normal equation method in deep learning

Deep learning has been employed to build applications and greatly promoted the development of the industries from many areas. Among the deep learning algorithms, normal equation method is widely used and is very time-consuming. Thus, it is very urgent to improve normal equation method. First, we propose a systolic Gaussian elimination. Second, we propose a systolic Gauss-Jordan elimination. By integrating other designs, we build an efficient architecture for improving differential equations in normal equation method. We implement our design in the development environment of artificial intelligence, which shows that it is very efficient for deep learning and its applications.

Research on Armature Structure Optimization of Rail Gun Based on Multiple Linear Regression

The armature structure is the key to the electromagnetic orbit emission. In the initial stage of launch, a good initial contact between the armature and the orbit can effectively avoid transition. First, an L16(54) orthogonal table was established in this article to obtain 16 groups of numerical simulation schemes, and the range and variance analyses of numerical results were carried out. The results show that the main factors affecting the peak contact pressure and stress are shoulder thickness and interference. The secondary factors are the tail length, tail tip thickness, and leading foot thickness; second, a machine learning method based on a multiple linear regression normal equation method is adopted to predict its contact state. The prediction error can be controlled within 10%; finally, parametric scanning is carried out on the basis of machine learning and a new method of armature structure optimization is proposed to improve the initial contact state. The optimized armature can greatly reduce its stress peak. This research provides a new method for the armature structure optimization design.

Error bounds for computed least squares estimators

This paper is concerned with normwise errors in the LS estimation for linear regression. It provides probabilistic tail bounds for the normwise error between the computed least squares estimator and the parameter vector, when the least squares problem is solved in floating point arithmetic using either the normal equations method or a backward stable method (for example, using the Householder QR factorization or the singular value decomposition). These bounds are used to provide a condition under which the computationally more efficient normal equations method can safely be used instead of a backward stable method, without any loss of accuracy in the computed estimator.

Noise estimation for hyperspectral subspace identification on FPGAs

We present a reliable and efficient FPGA implementation of a procedure for the computation of the noise estimation matrix, a key stage for subspace identification of hyperspectral images. Our hardware realization is based on numerically stable orthogonal transformations, avoids the numerical difficulties of the normal equations method for the solution of linear least squares problems (LLS), and exploits the special relations between coupled LLS problems arising in the hyperspectral image. Our modular implementation decomposes the QR factorization that comprises a significant part of the cost into a sequence of suboperations, which can be efficiently computed on an FPGA.

Research on Fluid Viscous Damper Parameters of Cable-Stayed Bridge in Northwest China

To optimize the aseismic performance of nonlinear fluid viscous dampers (FVD) of cable-stayed bridge in the highly seismic zone, Xigu Yellow River Bridge in northwest China is taken as an example. Nonlinear time-history analysis method is used to research on the relation among the internal forces, displacements, and damping parameters of the 650 tonnage FVD. The method of getting the minimum of binary functions is used to obtain the optimal parameters of FVD. Also, the 1 : 1 full-scale FVD model is made and used in the constitutive relation test. Then the test result of the damping parameters can be got by normal equation method. The optimized method to obtain the damping parameters is further verified. The results indicate that seismic response in key positions of the cable-stayed bridge can be reduced by installing longitudinal nonlinear FVD between the towers and girders if choosing reasonable damping parameters C and ξ. The optimal damping parameters can be calculated accurately by the proposed method of optimizing damping parameters of nonlinear FVD, and the constitutive relation test verifies the correctness of the optimization analysis method. Conclusions concerned can be applied to the design of nonlinear FVD for cable-stayed bridges.

Parallel iterative refinement linear least squares solvers based on all-reduce operations

We present the novel parallel linear least squares solvers ARPLS-MPIR and ARPLS-IR.We provide a careful analysis of their communication cost.We analyze their parallel runtime behavior experimentally.Mixed precision iterative refinement reduces computation and communication cost for well-conditioned problems.ARPLS-IR is also applicable to ill-conditioned problems and especially for narrow problems outperforms DPLASMA and ScaLapack. We present the novel parallel linear least squares solvers ARPLS-IR and ARPLS-MPIR for dense overdetermined linear systems. All internode communication of our ARPLS solvers arises in the context of all-reduce operations across the parallel system and therefore they benefit directly from efficient implementations of such operations. Our approach is based on the semi-normal equations, which are in general not backward stable. However, the method is stabilised by using iterative refinement. We show that performing iterative refinement in mixed precision also increases the parallel performance of the algorithm. We consider different variants of the ARPLS algorithm depending on the conditioning of the problem and in this context also evaluate the method of normal equations with iterative refinement. For ill-conditioned systems, we demonstrate that the semi-normal equations with standard iterative refinement achieve the best accuracy compared to other parallel solvers.We discuss the conceptual advantages of ARPLS-IR and ARPLS-MPIR over alternative parallel approaches based on QR factorisation or the normal equations. Moreover, we analytically compare the communication cost to an approach based on communication-avoiding QR factorisation. Numerical experiments on a high performance cluster illustrate speed-ups up to 3820 on 2048 cores for ill-conditioned tall and skinny matrices over state-of-the-art solvers from DPLASMA or ScaLAPACK.

Low-Complexity SIC Detection Algorithms for Multiple-Input Multiple-Output Systems

A new design for successive interference cancellation (SIC) detection for multiple-input multiple-output systems is introduced, and it is developed on the basis of the method of normal equations commonly used to solve the linear least squares problem. On the basis of this design, optimal-ordered and suboptimal-ordered SIC detection algorithms are derived. It is shown that the proposed optimal-ordered SIC detection algorithm offers a complexity reduction ratio of 1.11-1.25 compared to the fastest known optimal-ordered SIC detection algorithm for intermediate and large numbers of antennas and in terms of the average complexity. On the other hand, the proposed suboptimal-ordered SIC detection algorithm requires a lower complexity than the proposed optimal-ordered one and provides a bit-error-rate performance close to that of the optimal-ordered one and better than those of the other suboptimal-ordered SIC detection algorithms.

A New Method for Aggregate Proportion Design of Mixture

When we make design of mixture ratio, we always use normal equation method (NEM), but the solution value of that method may doesn’t meet our demand. When we use the new method that proposed in this paper, we can solve the problem. The new method makes an improvement for NEM. By iterative algorithm, the new method uses the gradation data calculated by NEM as initial target value, canceling the relevant aggregate if the result of proportion is negative, and setting the gradation to the boundary value if the target value goes beyond limits. According to the adjusted object value, using NEM again, the new aggregate proportion can be solved. Then the new gradation of mixture will come into being the next target value. Finally, the accurate result will meet our need. The new improved method has good engineering applicability and high accuracy

A multi‐level adaptive solution strategy for 3D inverse problems in pool boiling

Purpose – The purpose of this paper is to present an efficient algorithm based on a multi‐level adaptive mesh refinement strategy for the solution of ill‐posed inverse heat conduction problems arising in pool boiling using few temperature observations.Design/methodology/approach – The stable solution of the inverse problem is obtained by applying the conjugate gradient method for the normal equation method together with a discrepancy stopping rule. The resulting three‐dimensional direct, adjoin and sensitivity problems are solved numerically by a space‐time finite element method. A multi‐level computational approach, which uses an a posteriori error estimator to adaptively refine the meshes on different levels, is proposed to speed up the entire inverse solution procedure.Findings – This systematic approach can efficiently solve the large‐scale inverse problem considered without losing necessary detail in the estimated quantities. It is shown that the choice of different termination parameters in the disc...

A Fast Randomized Algorithm for Orthogonal Projection

We describe an algorithm that, given any full-rank matrix $A$ having fewer rows than columns, can rapidly compute the orthogonal projection of any vector onto the null space of $A$, as well as the orthogonal projection onto the row space of $A$, provided that both $A$ and its adjoint $A^*$ can be applied rapidly to arbitrary vectors. As an intermediate step, the algorithm solves the overdetermined linear least-squares regression involving $A^*$ and may therefore be used for this purpose as well. In many circumstances, the technique can accelerate interior-point methods for convex optimization, including linear programming (see, for example, Chapter 11 of [S. J. Wright, Primal-Dual Interior-Point Methods, SIAM, Philadelphia, 1997]). The basis of the algorithm is an obvious but numerically unstable scheme (typically known as the method of normal equations); suitable use of a preconditioner yields numerical stability. We generate the preconditioner rapidly via a randomized procedure that succeeds with extremely high probability. We provide numerical examples demonstrating the superior accuracy of the randomized method over direct use of the normal equations.

Measurements of 3D propagation in the shelf environment

In the Fall of 2007 measurements were made to calibrate the acoustic environment on the shallow water shelf off the coast of Florida. The continental shelf is quite flat (less than 1 degree slope) for a region approximately 10 km wide. Transmissions to a cross-shelf 900 m horizontal line array were made from a source transiting along the shelf. The signal transmissions include broadband LFM (20-420 Hz) and a comb of narrow band frequencies spanning the same range. Narrowband beamforming results show the clearly identified source. The arrival angle for the source is as expected until a distance of approximately 30-40 km when there is an apparent bearing shift in-shore of up to 25 degrees. This behavior is expected in propagation on a shelf (of greater slope) but its behavior is surprising. The phenomenon was observed for several runs at various source and water depths. In order to explain the phenomenon, a hybrid adiabatic normal mode-Parabolic Equation method will be applied to the environment. This model computes the vertical modes and phase speeds at each location and then uses the PE to propagate each mode individually using it's spatially varying phase speed and attenuation. Comparisons of theory and data will be made.

On inexact hermitian and skew-Hermitian splitting methods for non-Hermitian positive definite linear systems

We study theoretical properties of two inexact Hermitian/skew-Hermitian splitting (IHSS) iteration methods for the large sparse non-Hermitian positive definite system of linear equations. In the inner iteration processes, we employ the conjugate gradient (CG) method to solve the linear systems associated with the Hermitian part, and the Lanczos or conjugate gradient for normal equations (CGNE) method to solve the linear systems associated with the skew-Hermitian part, respectively, resulting in IHSS(CG, Lanczos) and IHSS(CG, CGNE) iteration methods, correspondingly. Theoretical analyses show that both IHSS(CG, Lanczos) and IHSS(CG, CGNE) converge unconditionally to the exact solution of the non-Hermitian positive definite linear system. Moreover, their contraction factors and asymptotic convergence rates are dominantly dependent on the spectrum of the Hermitian part, but are less dependent on the spectrum of the skew-Hermitian part, and are independent of the eigenvectors of the matrices involved. Optimal choices of the inner iteration steps in the IHSS(CG, Lanczos) and IHSS(CG, CGNE) iterations are discussed in detail by considering both global convergence speed and overall computation workload, and computational efficiencies of both inexact iterations are analyzed and compared deliberately.

Models of gas-grain chemistry in interstellar cloud cores with a stochastic approach to surface chemistry

We present a gas-grain model of homogeneous cold cloud cores with time-independent physical conditions. In the model, the gas-phase chemistry is treated via rate equations while the diffusive granular chemistry is treated stochastically. The two phases are coupled through accretion and evaporation. A small network of surface reactions accounts for the surface production of the stable molecules water, formaldehyde, methanol, carbon dioxide, ammonia, and methane. The calculations are run for a time of 10 7 years at three different temperatures: 10 K, 15 K, and 20 K. The results are compared with those produced in a totally deterministic gas-grain model that utilizes the rate equation method for both the gas-phase and surface chemistry. The results of the different models are in agreement for the abundances of the gaseous species except for later times when the surface chemistry begins to affect the gas. The agreement for the surface species, however, is somewhat mixed. The average abundances of highly reactive surface species can be orders of magnitude larger in the stochastic-deterministic model than in the purely deterministic one. For non-reactive species, the results of the models can disagree strongly at early times, but agree to well within an order of magnitude at later times for most molecules. Strong exceptions occur for CO and H2CO at 10 K, and for CO2 at 20 K. The agreement seems to be best at a temperature of 15 K. As opposed to the use of the normal rate equation method of surface chemistry, the modified rate method is in significantly better agreement with the stochastic-deterministic approach. Comparison with observations of molecular ices in dense clouds shows mixed agreement.

Conversion of the hot torsion test results into flow curve with multiple regimes of hardening

The method of Fields and Backofen has been commonly used to reduce the data obtained by hot torsion test into flow curves. The method, however, is most suitable for materials with monotonic strain hardening behaviour. Other methods such as Stüwe’s method, tubular specimens, differential testing and the inverse method, each suffer from similar drawbacks.It is shown in the current work that for materials with multiple regimes of hardening any method based on an assumption of constant hardening indices introduces some errors into the flow curve obtained from the hot torsion test. Therefore such methods do not enable accurate prediction of onset of recrystallisation where slow softening occurs.A new method to convert results from the hot torsion test into flow curves by taking into account the variation of constitutive parameters during deformation is presented. The method represents the torque twist data by a parametric linear least square model in which Euler and hyperbolic coefficients are used as the parameters. A closed form relationship obtained from the mathematical representation of the data is employed next for flow stress determination. Two different solution strategies, the method of normal equations and singular value decomposition, were used for parametric modelling of the data with hyperbolic basis functions. The performance of both methods is compared.Experimental data obtained by FHTTM, a flexible hot torsion test machine developed at IROST, for a C–Mn austenitic steel was used to demonstrate the method. The results were compared with those obtained using constant strain and strain rate hardening characteristics.

Distribution system state estimation

A three-phase distribution system state estimation algorithm is proposed in this paper. The normal equation method is used to compute the real-time states of distribution systems modeled by their actual a-b-c phases. A current based formulation is introduced and compared with other formulations. Observability analysis for the proposed distribution system state estimation is discussed. Test results indicate that the normal equation method is applicable to the distribution system state estimation and the current based rectangular form formulation is suitable for this application. >

A Comparison of Block Pivoting and Interior-Point Algorithms for Linear Least Squares Problems with Nonnegative Variables

In this paper we discuss the use of block principal pivoting and predictor-corrector methods for the solution of large-scale linear least squares problems with nonnegative variables (NVLSQ). We also describe two implementations of these algorithms that are based on the normal equations and corrected seminormal equations (CSNE) approaches. We show that the method of normal equations should be employed in the implementation of the predictor-corrector algorithm. This type of approach should also be used in the implementation of the block principal pivoting method, but a switch to the CSNE method may be useful in the last iterations of the algorithm. Computational experience is also included in this paper and shows that both the predictor-corrector and the block principal pivoting algorithms are quite efficient to deal with large-scale NVLSQ problems.