Singular Value Decomposition Solution Research Articles

Privacy-Preserving Federated Singular Value Decomposition

Singular value decomposition (SVD) is a fundamental technique widely used in various applications, such as recommendation systems and principal component analyses. In recent years, the need for privacy-preserving computations has been increasing constantly, which concerns SVD as well. Federated SVD has emerged as a promising approach that enables collaborative SVD computation without sharing raw data. However, existing federated approaches still need improvements regarding privacy guarantees and utility preservation. This paper moves a step further towards these directions: we propose two enhanced federated SVD schemes focusing on utility and privacy, respectively. Using a recommendation system use-case with real-world data, we demonstrate that our schemes outperform the state-of-the-art federated SVD solution. Our utility-enhanced scheme (utilizing secure aggregation) improves the final utility and the convergence speed by more than 2.5 times compared with the existing state-of-the-art approach. In contrast, our privacy-enhancing scheme (utilizing differential privacy) provides more robust privacy protection while improving the same aspect by more than 25%.

Complementary waveforms for range sidelobe suppression based on a singular value decomposition approach

AbstractWhile Doppler resilient complementary waveforms (DRCWs) have previously been considered to suppress range sidelobes within a Doppler interval of interest in radar systems, their ability to provide Doppler resilience can be further improved. A new singular value decomposition (SVD)‐based DRCW construction is proposed, in which both transmit pulse trains (made up of complementary pairs) and receive pulse weights are jointly considered. Besides, using the proposed SVD‐based method, a theoretical bound is derived for the range sidelobes within the Doppler interval of interest. Moreover, based on the SVD solutions, a challenging non‐convex optimization problem is formulated and solved to maximise the signal‐to‐noise ratio (SNR) with the constraint of low range sidelobes. It is shown that, compared with existing DRCWs, the proposed SVD‐based DRCW has better Doppler resilience. Further, the new optimised SVD‐based DRCW has a higher SNR while maintaining the same Doppler resilience.

Estimation Method for Mixed Near-Field and Far-Field Sources Based on Four-Order Cumulant and Polarization Sensitive Array

A new multidimensional parameters joint estimation method of mixed near-field and far-field sources based on polarization sensitive array, fourth-order cumulant, joint diagonalization technology, and propagator method is presented, which can realize the joint estimation of DOA (Direction of Arrival), range, frequency, polarization auxiliary angle, and polarization phase difference without multidimensional spectral peak searching and parameter pairing, and it is suitable for any additive Gaussian noise environment and effective for reducing the loss of array aperture. The algorithm skillfully constructs the fourth-order cumulant matrix by using the output on the label of specific dipole pairs of the received array, which effectively avoids the matrix rank reduction caused by the far-field sources coexistence situation. In addition, the presented algorithm utilizes the orthogonal propagation algorithm for subspace decomposition and uses the total least square solution to replace the orthogonal solution of singular value decomposition, which effectively reduces the computational complexity. The experiment proved the effectiveness of the proposed algorithm.

A Combined Asymptotic Waveform Evaluation and Random Auxiliary Sources Method for Wideband Solutions of General-Purpose EM Problems

An asymptotic waveform evaluation (AWE) formulation for the random auxiliary sources (RAS) method is presented to solve general-purpose electromagnetic (EM) problems. The susceptibility of the RAS-AWE procedure to numerical noise is studied. A singular value decomposition solution is proposed to improve the possible obtained bandwidth. The solution parameters are studied for 2-D and 3-D problems. Also, the limitations of the proposed formulation are discussed in comparison to the method of moments (MoM). Furthermore, several general EM scattering problems are solved using the proposed RAS-AWE procedure to assess the performance and accuracy compared to other formulations. General observation points are discussed to highlight the advantages and disadvantages of the proposed approach. In addition, the core kernel operators are presented for the 2-D, 3-D, and waveport formulations.

SVD solutions to inverse source problems in the time domain: Application to complex point sources

The inverse source problem is formulated in terms of the singular value decomposition (SVD) for scalar sources that radiate time-domain wavefields. Both spatial and temporal constraints are imposed on the sources. The temporal constraint takes the form of a preselected bandlimit that is imposed through the use of sinc-interpolation. To make the inverse source problem numerically tractable, both the field exterior and interior to the source can be specified. A time-domain spherical-harmonics truncation formula is derived for the complex point source with time dependence given by analytic δ−pulses. This pulsed complex point source is then realized using an array of real point sources that are evenly distributed over a smooth array surface. The array surface and the inner surface are bodies of revolution with cross sections in the shape of superellipses. Accurate well-behaved array solutions are obtained in a straightforward manner by appropriate truncation of the singular-value expansion.

Feasibility of approximate broadband estimation of acoustic impedance profile from first principles and band-limited reflection data

Because seismic reflection data are band limited, acoustic impedance profiles derived from them are nonunique. The conventional inversion methods counter the nonuniqueness either by stabilizing the answer with respect to an initial model or by imposing mathematical constraints such as sparsity of the reflection coefficients. By making a nominal assumption of an earth model locally consisting of a stack of homogeneous and horizontal layers, we have formulated a set of linear equations in which the reflection coefficients are the unknowns and the recursively integrated seismic trace constitute the data. Drawing only on first principles, the Zoeppritz equation in this case, the approach makes a frontal assault on the problem of reconstructing reflection coefficients from band-limited data. The local layer-cake assumption and the strategy of seeking a singular value decomposition solution of the linear equations counter the nonuniqueness, provided that the objective is to reconstruct a smooth version of the impedance profile that includes only its crude structures. Tests on synthetic data generated from elementary models and from measured logs of acoustic impedance demonstrated the efficacy of the method, even when a significant amount of noise was added to the data. The emergence of consistent estimates of impedance, approximating the original impedance, from synthetic data generated for several frequency bands has inspired our confidence in the method. The other attractive outputs of the method are as follows: (1) an accurate estimate of the impedance mean, (2) an accurate reconstruction of the direct-current (DC) frequency of the reflectivity, and (3) an acceptable reconstruction of the broad outline of the original impedance profile. These outputs can serve as constraints for either more refined inversions or geologic interpretations. Beginning from the restriction of band-limited data, we have devised a method that neither requires a starting input model nor imposes mathematical constraints on the earth reflectivity and still yielded significant and relevant geologic information.

Algorithmic vs. finite difference Jacobians for infrared atmospheric radiative transfer

Jacobians, i.e. partial derivatives of the radiance and transmission spectrum with respect to the atmospheric state parameters to be retrieved from remote sensing observations, are important for the iterative solution of the nonlinear inverse problem. Finite difference Jacobians are easy to implement, but computationally expensive and possibly of dubious quality; on the other hand, analytical Jacobians are accurate and efficient, but the implementation can be quite demanding. GARLIC, our “Generic Atmospheric Radiation Line-by-line Infrared Code”, utilizes algorithmic differentiation (AD) techniques to implement derivatives w.r.t. atmospheric temperature and molecular concentrations. In this paper, we describe our approach for differentiation of the high resolution infrared and microwave spectra and provide an in-depth assessment of finite difference approximations using “exact” AD Jacobians as a reference. The results indicate that the “standard” two-point finite differences with 1K and 1% perturbation for temperature and volume mixing ratio, respectively, can exhibit substantial errors, and central differences are significantly better. However, these deviations do not transfer into the truncated singular value decomposition solution of a least squares problem. Nevertheless, AD Jacobians are clearly recommended because of the superior speed and accuracy.

Robust reduced-rank filtering for erratic seismic noise attenuation

Singular spectrum analysis (SSA) or Cadzow reduced-rank filtering is an efficient method for random noise attenuation. SSA starts by embedding the seismic data into a Hankel matrix. Rank reduction of this Hankel matrix followed by antidiagonal averaging is utilized to estimate an enhanced seismic signal. Rank reduction is often implemented via the singular value decomposition (SVD). The SVD is a nonrobust matrix factorization technique that leads to suboptimal results when the seismic data are contaminated by erratic noise. The term erratic noise designates non-Gaussian noise that consists of large isolated events with known or unknown distribution. We adopted a robust low-rank factorization that permitted use of the SSA filter in situations in which the data were contaminated by erratic noise. In our robust SSA method, we replaced the quadratic error criterion function that yielded the truncated SVD solution by a bisquare function. The Hankel matrix was then approximated by the product of two lower dimensional factor matrices. The iteratively reweighed least-squares method was used to approximately solve for the optimal robust factorization. Our algorithm was tested with synthetic and real data. In our synthetic examples, the data were contaminated with band-limited Gaussian noise and erratic noise. Then, denoising was carried out by means of [Formula: see text] deconvolution, the classical SSA method, and the proposed robust SSA method. The [Formula: see text] deconvolution and the classical SSA method failed to properly eliminate the noise and to preserve the desired signal. On the other hand, the robust SSA method was found to be immune to erratic noise and was able to preserve the desired signal. We also tested the robust SSA method with a data set from the Western Canadian Sedimentary Basin. The results with this data set revealed improved denoising performance in portions of data contaminated with erratic noise.

Iterative solution of inconsistent systems of linear inequalities

AbstractIn 1980 S.P. Han proposed a finitely terminating (in exact arithmetic) algorithm for solving an inconsistent system of linear inequalities in a least squares sense. This algorithm uses a singular value decomposition of a submatrix of the problem matrix on each iteration, making it impractical for all but only smaller problems. In this paper we show that a modification of Han's algorithm allows us to introduce an iterative approximation to the singular value decomposition solution. (© 2013 Wiley‐VCH Verlag GmbH & Co. KGaA, Weinheim)

Singular Value Decomposition Solution of the Schrödinger Equation in the Presence of Exchange Terms

A new numerical method of solving integro-differential equations appearing in the theory of atomic and nuclear scattering systems has been devised. It is termed Singular Value Decomposition Method (SVD). It consists in expanding the exchange kernel into a number of separable terms by means of the Singular Value Decomposition and then iterating over the remainder. In this paper, we extend our SVD method to the scattering of low energy electron-helium which has been the subject of interest, both theoretically and experimentally. We compare our results with the Moments Method which is widely used. The Moments Method consists of making an expansion of the solution into an especially favorable basis that takes care of the non-exchange part of the Hamiltonian.

Application of regularization methods to damage detection in large scale plane frame structures using incomplete noisy modal data

This paper presents an approach for detecting local damage in large scale frame structures by utilizing regularization methods for ill-posed problems. A direct relationship between the change in stiffness caused by local damage and the measured modal data for the damaged structure is developed, based on the perturbation method for structural dynamic systems. Thus, the measured incomplete modal data can be directly adopted in damage identification without requiring model reduction techniques, and common regularization methods could be effectively employed to solve the developed equations. Damage indicators are appropriately chosen to reflect both the location and severity of local damage in individual components of frame structures such as in brace members and at beam–column joints. The Truncated Singular Value Decomposition solution incorporating the Generalized Cross Validation method is introduced to evaluate the damage indicators for the cases when realistic errors exist in modal data measurements. Results for a 16-story building model structure show that structural damage can be correctly identified at detailed level using only limited information on the measured noisy modal data for the damaged structure.

Plane wave interpolation in direct collocation boundary element method for radiation and wave scattering: numerical aspects and applications

The classical boundary element formulation for the Helmholtz equation is rehearsed, and its limitations with respect to the number of variables needed to model a wavelength are explained. A new type of interpolation for the potential is then described in which the usual boundary element shape functions are modified by the inclusion of a set of plane waves, propagating in a range of directions. This is termed the plane wave basis boundary element method. The modifications needed to the classical procedures, in terms of integration of the element matrices, and location of collocation points are described. The well-known Singular Value Decomposition solution technique, which is adopted here for the solution of the system matrix equation in its complex form, is briefly outlined. The conditioning of the system matrix is analysed for a simple radiation problem. The corresponding diffraction problem is also analysed and results are compared with analytical and classical boundary element solutions. The CHIEF method is adopted to enhance the quality of the solution, particularly in the vicinity of irregular frequencies. The plane wave basis boundary element method is then applied to two problems: scattering of plane waves by an elliptical cylinder and the multiple circular cylinder plane wave scattering problem. In both cases results are compared with analytical solutions. The results clearly demonstrate that the new method is considerably more efficient than the classical approach. For a given number of degrees of freedom, the frequency for which accurate results can be obtained, using the new technique, can be up to three or four times higher than that of the classical method. This makes the method a powerful new addition to our tools for tackling high-frequency radiation and scattering problems.

Iterative refinement of the singular-value decomposition solution to the regression equations

In solving the regression equations, collinearity in the design matrix can result in parameter estimates which are inaccurate. The use of orthogonal matrix transformations such as the singular-value decomposition can reduce the effect of collinearity. Also, estimates of the regression coefficients can sometimes be improved through the process known as iterative refinement. The application of iterative refinement to the singular-value decomposition solution of the regression equations is described. Tests show that iterative refinement using the singular-value decomposition can improve regression coefficient estimates, in cases where the design matrix is highly collinear.

Accuracy of TSVD solutions computed from rank-revealing decompositions

Rank-revealing decompositions are favorable alternatives to the singular value decomposition (SVD) because they are faster to compute and easier to update. Although they do not yield all the information that the SVD does, they yield enough information to solve various problems because they provide accurate bases for the relevant subspaces. In this paper we consider rank-revealing decompositions in computing estimates of the truncated SVD (TSVD) solution to an overdetermined system of linear equations\(A x \approx b\) , where\(A\) is numerically rank deficient. We derive analytical bounds which show how the accuracy of the solution is intimately connected to the quality of the subspaces.

Estimation of resolution and covariance for large matrix inversions

SUMMARY Key advantages of conjugate gradient (CG) methods are that they require far less computer memory than full singular value decomposition (SVD), and that iteration may be stopped at any time to give an approximate solution; this means that they may be used to obtain solutions of problems that are too large for SVD. The disadvantage is that CG does not conveniently provide auxiliary information on the quality of the solution (resolution and covariance matrices). This may be overcome by extensions of Paige and Saunders’ LSQR algorithm, which is one of the family of CG algorithms. The extensions are produced by analogy with SVD; bidiagonalization in LSQR produces orthonormal basis vectors that can be used to construct solutions and estimates of resolution and covariance. For large problems, for which SVD can not be performed, the new method provides approkimate resolution and covariance estimates that asymptotically approach those of the SVD solutions as the number of iterations increases.

Ensemble inference in geophysical inverse problems

Non-invasive studies of the shallow Earth are an essential element in a broad range of earth science disciplines. Such studies are usually accomplished using geophysical techniques, which means that a geophysical inverse problem must be solved. Unfortunately, non-uniqueness is a basic property of almost all solutions to geophysical inverse problems. In this paper we describe a method of dealing with this non-uniqueness which we believe is more direct, more complete, and more informative than previous attempts in this area. Instead of using regularization methods to constrain the non-uniqueness of geophysical inverse problems, we confront directly the non-uniqueness and attempt to describe it. The basic technique is to generate and describe a collection of models that fit the data within acceptable limits based upon observational errors. This collection of models, called an ensemble, is analysed with statistical methods in order to characterize the uncertainty in the solution and also make inferences about properties that are shared by all acceptable models. A by-product of this approach is that it produces basic information about the degree of linearity contained in the problem and maps trade-offs between various parameters. The method proposed is quite general and should be applicable to a broad range of geophysical inverse problems. We illustrate the basic method by applying it to two typical geophysical problems. The first application is to a set of cross-borehole traveltime observations gathered in Kesterson, California. The small deviations of the observed traveltime residuals from those predicted by a uniform half-space indicates that the velocity contrasts are not large. The set of acceptable models appears to form a non-degenerate hyperellipsoid in the model space. The mean of the velocity models converges quite rapidly to a smoothed version of a linearized singular-value decomposition solution. The velocity variations agree very well with the results of a hydrological tracer test conducted prior to the seismic experiment. The second application is to the problem of simultaneously estimating the shape and density of a constant density body using surface gravity measurements. In this case a linear analysis is found to be inadequate. A quadratic formulation is necessary to accurately represent the trade-off between the magnitude of the density perturbation and the depth extent of the body.

On the least-squares determination of lattice dimensions: a modified singular value decomposition approach to ill-conditioned cases

A modification is described of the singular value decomposition (SVD) method suitable for underdetermined linear least squares (LLS). When a set of data to be fitted is incomplete and does not allow an independent determination of all model parameters, the modified method automatically merges a previously available approximate solution into the LLS results. The solution so produced is more appropriate to the particular problem than the usual SVD solution, while still being a LLS estimate of the whole set of parameters. The method is discussed with reference to the LLS determination of unit-cell dimensions during the step-by-step assignment of h, k, l indices of a diffraction pattern.

Singular Value Decomposition of Wintertime Sea Surface Temperature and 500-mb Height Anomalies

Abstract Single field principal component analysis (PCA), direct singular value decomposition (SVD), canonical correlation analysis (CCA), and combined principal component analysis (CPCA) of two fields are applied to a 39-winter dataset consisting of normalized seasonal mean sea surface temperature anomalies over the North Pacific and concurrent 500-mb height anomalies over the same region. The CCA solutions are obtained by linear transformations of the SVD solutions. Spatial patterns and various measures of the variances and covariances explained by the modes derived from the different types of expansions are compared, with emphasis on the relative merits of SVD versus CCA. Results for two different analysis domains (i.e., the Pacific sector versus a full hemispheric domain for the 500-mb height field) are also compared in order to assess the domain dependence of the two techniques. The SVD solution is also compared with the results of 28 Monte Carlo simulations in which the temporal order of the SST gri...

Structural Identification Using Order Overspecified Time-Series Models

An identification method that uses order overspecified time-series models and a truncated singular value decomposition (SVD) solution is studied. The overspecified model reduces the effects of noise during the identification process, but produces extraneous modes. A backwards approach coupled with a minimum norm approximation, using a truncated SVD solution, enables the system modes to be distinguished from the extraneous modes of the model. Experimental data from a large flexible truss is used to study the effects of varying the truncation of the SVD solution and an order recursive algorithm is used to study the effects of model order. Results show that the SVD may be ineffective in separating the data into signal and noise subspaces. However solutions for highly overspecified model orders exhibit solution properties similar to the minimum norm solution and system and computational modes can be discriminated without a truncated solution.

Truncated Singular Value Decomposition Solutions to Discrete Ill-Posed Problems with Ill-Determined Numerical Rank

Tikhonov regularization is a standard method for obtaining smooth solutions to discrete ill-posed problems. A more recent method, based on the singular value decomposition (SVD), is the truncated S...