Singular Value Spectrum Research Articles

Preconditioned flow as a solution to the hierarchical growth problem in the generalized Lefschetz thimble method

The generalized Lefschetz thimble method is a promising approach that attempts to solve the sign problem in Monte Carlo methods by deforming the integration contour using the flow equation. Here we point out a general problem that occurs due to the property of the flow equation, which extends a region on the original contour exponentially to a region on the deformed contour. Since the growth rate for each eigenmode is governed by the singular values of the Hessian of the action, a huge hierarchy in the singular value spectrum, which typically appears for large systems, leads to various technical problems in numerical simulations. We solve this hierarchical growth problem by preconditioning the flow so that the growth rate becomes identical for every eigenmode. As an example, we show that the preconditioned flow enables us to investigate the real-time quantum evolution of an anharmonic oscillator with the system size that can hardly be achieved by using the original flow.

Reduced Tensor Network Formulation for Non-Abelian Gauge Theories in Arbitrary Dimensions

Abstract Formulating non-Abelian gauge theories as a tensor network is known to be challenging due to the internal degrees of freedom that result in the degeneracy in the singular value spectrum. In two dimensions, it is straightforward to “trace out” these degrees of freedom with the use of character expansion, giving a reduced tensor network where the degeneracy associated with the internal symmetry is eliminated. In this work, we show that such an index loop also exists in higher dimensions in the form of a closed tensor network that we call the “armillary sphere”. This allows us to completely eliminate the matrix indices and reduce the overall size of the tensors in the same way as is possible in two dimensions. This formulation allows us to include significantly more representations with the same tensor size, thus making it possible to reach a greater level of numerical accuracy in the tensor renormalization group computations.

Boundary between noise and information applied to filtering neural network weight matrices.

Deep neural networks have been successfully applied to a broad range of problems where overparametrization yields weight matrices which are partially random. A comparison of weight matrix singular vectors to the Porter-Thomas distribution suggests that there is a boundary between randomness and learned information in the singular value spectrum. Inspired by this finding, we introduce an algorithm for noise filtering, which both removes small singular values and reduces the magnitude of large singular values to counteract the effect of level repulsion between the noise and the information part of the spectrum. For networks trained in the presence of label noise, we indeed find that the generalization performance improves significantly due to noise filtering.

Application of VMD–SSA–BiLSTM algorithm to smart grid financial market time series forecasting and sustainable innovation management

Introduction: This paper proposes a deep learning algorithm based on the VMD-SSA-BiLSTM model for time series forecasting in the smart grid financial market. The algorithm aims to extract useful information from power grid signals to improve the timing prediction accuracy and meet the needs of sustainable innovation management.Methods: The proposed algorithm employs the variational mode decomposition (VMD) method to decompose and reduce the dimensionality of historical data, followed by singular spectrum analysis (SSA) to perform singular spectrum analysis on each intrinsic mode function component. The resulting singular value spectrum matrices serve as input to a bidirectional long short-term memory (BiLSTM) neural network, which learns the feature representation and prediction model of the smart grid financial market through forward propagation and backpropagation.Results: The experimental results demonstrate that the proposed algorithm effectively predicts the smart grid financial market's time series, achieving high prediction accuracy and stability. The approach can contribute to sustainable innovation management and the development of the smart grid.Discussion: The VMD-SSA-BiLSTM algorithm's efficiency in extracting useful information from power grid signals and avoiding overfitting can improve the accuracy of timing predictions in the smart grid financial market. The algorithm's broad application prospects can promote sustainable innovation management and contribute to the development of the smart grid.

GFR: Generic feature representations for class incremental learning

Class incremental learning (CIL) aims to continuously learn new classes while maintaining discrimination for old classes with sequentially coming data. Due to the lack of old-class samples, existing CIL methods fail to learn discriminative representations for both old and new classes simultaneously, resulting in a severe performance drop in old classes, which is the well-known catastrophic forgetting phenomenon. Different from most existing works, we facilitate CIL by learning generic feature representations that perform well in seen and unseen classes. Specifically, we prove that representations with a substantial number of significant singular values benefit CIL via better old knowledge reservation. However, the overly uniform singular value spectrum will hurt the discrimination of current tasks. Furthermore, we propose that increasing the embedding dimension can enhance the number of significant singular values and validate this assumption from two perspectives: adopting different pooling techniques and devising a wider network. Meanwhile, we also prove that satisfactory current task accuracy and old knowledge reservation can be achieved simultaneously. Finally, the simple yet effective generic feature representation regulation (GFR) is devised and incorporated into two baselines. Extensive experiments are conducted on CIFAR100, ImageNet-Subset, and ImageNet. The results show that the proposed method boosts the performance of both baselines with a large margin (2.00%-9.58% on CIFAR100, 0.68%-7.10% on ImageNet-Subset and 1.18%-5.04% on ImageNet) which outperforms existing SOTAs.

The Geometry of Feature Space in Deep Learning Models: A Holistic Perspective and Comprehensive Review

As the field of deep learning experiences a meteoric rise, the urgency to decipher the complex geometric properties of feature spaces, which underlie the effectiveness of diverse learning algorithms and optimization techniques, has become paramount. In this scholarly review, a comprehensive, holistic outlook on the geometry of feature spaces in deep learning models is provided in order to thoroughly probe the interconnections between feature spaces and a multitude of influential factors such as activation functions, normalization methods, and model architectures. The exploration commences with an all-encompassing examination of deep learning models, followed by a rigorous dissection of feature space geometry, delving into manifold structures, curvature, wide neural networks and Gaussian processes, critical points and loss landscapes, singular value spectra, and adversarial robustness, among other notable topics. Moreover, transfer learning and disentangled representations in feature space are illuminated, accentuating the progress and challenges in these areas. In conclusion, the challenges and future research directions in the domain of feature space geometry are outlined, emphasizing the significance of comprehending overparameterized models, unsupervised and semi-supervised learning, interpretable feature space geometry, topological analysis, and multimodal and multi-task learning. Embracing a holistic perspective, this review aspires to serve as an exhaustive guide for researchers and practitioners alike, clarifying the intricacies of the geometry of feature spaces in deep learning models and mapping the trajectory for future advancements in this enigmatic and enthralling domain.

Joint Inversion of Receiver Functions and Apparent Incidence Angles to Determine the Crustal Structure of Mars

AbstractRecent estimates of the crustal thickness of Mars show a bimodal result of either ∼20 or ∼40 km beneath the InSight lander. We propose an approach based on random matrix theory applied to receiver functions (RFs) to further constrain the subsurface structure. Assuming a spiked covariance model for our data, we first use the phase transition properties of the singular value spectrum of random matrices to detect coherent arrivals in the waveforms. Examples from terrestrial data show how the method works in different scenarios. We identify three previously undetected converted arrivals in the InSight data, including the first multiple from a deeper third interface. We then use this information to jointly invert RFs with the absolute S‐wave velocity information in the polarization of body waves. Results show a crustal thickness of 43 ± 5 km beneath the lander with two mid‐crustal interfaces at depths of 8 ± 1 and 21 ± 3 km.

Bond-weighting method for the Grassmann tensor renormalization group

Recently, the tensor network description with bond weights on its edges has been proposed as a novel improvement for the tensor renormalization group algorithm. The bond weight is controlled by a single hyperparameter, whose optimal value is estimated in the original work via the numerical computation of the two-dimensional critical Ising model. We develop this bond-weighted tensor renormalization group algorithm to make it applicable to the fermionic system, benchmarking with the two-dimensional massless Wilson fermion. We show that the accuracy with the fixed bond dimension is improved also in the fermionic system and provide numerical evidence that the optimal choice of the hyperparameter is not affected by whether the system is bosonic or fermionic. In addition, by monitoring the singular value spectrum, we find that the scale-invariant structure of the renormalized Grassmann tensor is successfully kept by the bond-weighting technique.

An Approach for Time Synchronization of Wireless Accelerometer Sensors Using Frequency-Squeezing-Based Operational Modal Analysis.

Wireless sensor networks usually suffer from the issue of time synchronization discrepancy due to environmental effects or clock management collapse. This will result in time delays between the dynamic responses collected by wireless sensors. If non-synchronized dynamic response data are directly used for structural modal identification, it leads to the misestimation of modal parameters. To overcome the non-synchronization issue, this study proposes a time synchronization approach to detect and correct asynchronous dynamic responses based on frequency domain decomposition (FDD) with frequency-squeezing processing (FSP). By imposing the expected relationship between modal phase angles extracted from the first-order singular value spectrum, the time lags between different sensors can be estimated, and synchronization can be achieved. The effectiveness of the proposed approach is fully demonstrated by numerical and experimental studies, as well as field measurement of a large-span spatial structure. The results verify that the proposed approach is effective for the time synchronization of wireless accelerometer sensors.

Clutter Suppression Algorithm of Ultrasonic Color Doppler Imaging Based on BP Neural Network

Aiming at the time complexity of singular value spectrum weighted Hankel SVD filtering algorithm, a clutter suppression algorithm for ultrasonic color Doppler imaging based on BP neural network model is proposed in this paper. Firstly, using the PRF data collected by portable ultrasound instrument, we verify the singular value weighted Hankel SVD filtering algorithm, and the results show that the algorithm has high accuracy; Then, the BP neural network model is established based on the input and output data of singular value weighted Hankel-SVD filtering algorithm; Finally, the clutter suppression algorithm of ultrasonic color Doppler imaging based on BP neural network model is established. The experimental results show that compared with Hankel SVD filtering algorithm, the clutter suppression algorithm proposed in this paper greatly shortens the operation time without reducing the accuracy, so as to improve the real-time performance of the filtering algorithm.

Effective diffraction separation using the improved optimal rank-reduction method

Seismic diffractions encoding subsurface small-scale geologic structures have great potential for high-resolution imaging of subwavelength information. Diffraction separation from the dominant reflected wavefields still plays a vital role because of the weak energy characteristics of the diffractions. Traditional rank-reduction (RR) methods based on the low-rank assumption of reflection events have been commonly used for diffraction separation. However, these methods using truncated singular-value decomposition (TSVD) suffer from the problem of reflection-rank selection by singular-value spectrum analysis, especially for complicated seismic data. In addition, the separation problem for the tangent wavefields of reflections and diffractions is challenging. To alleviate these limitations, we have developed an effective diffraction separation strategy using an improved optimal RR (ORR) method to remove the dependence on the reflection rank and improve the quality of separation results. The improved RR method adaptively determines the optimal singular values from the input signals by directly solving an optimization problem that minimizes the Frobenius-norm difference between the estimated and exact reflections instead of the TSVD operation. This improved method can effectively overcome the problem of reflection-rank estimation in the global and local RR methods and adjusts to the diversity and complexity of seismic data. The adaptive data-driven algorithms indicate good performance in terms of the trade-off between high-quality diffraction separation and reflection suppression for the ORR operation. Applications of our strategy to synthetic and field examples demonstrate the superiority of diffraction separation in detecting and revealing subsurface small-scale geologic discontinuities and inhomogeneities.

Bond-weighted tensor renormalization group

We propose an improved tensor renormalization-group (TRG) algorithm, the bond-weighted TRG (BTRG). In BTRG, we generalize the conventional TRG by introducing bond weights on the edges of the tensor network. We show that BTRG outperforms the conventional TRG and the higher-order tensor renormalization group with the same bond dimension, whereas its computation time is almost the same as that of TRG. Furthermore, BTRG can have nontrivial fixed-point tensors at an optimal hyperparameter. We demonstrate that the singular value spectrum obtained by BTRG is invariant under the renormalization procedure in the case of the two-dimensional Ising model at the critical point. This property indicates that BTRG performs the tensor contraction with high accuracy whereas keeping the scale-invariant structure of tensors.

The Advantage Analysis of 3-D Inversion of Airborne Gravity Gradiometry Data With Larger Sampling Interval

Airborne gravity gradiometry is an effective tool for its high efficiency and sensitivity to the interesting buried targets, which has been increasingly introduced in mineral and petroleum exploration. When compared to gravimetry, full tensor gradient (FTG) measurements provide higher resolution and signal-to-noise ratio. In this article, we build and test models with different sampling intervals and compare the performances of individual components or their combinations in minimum structure inversion to evaluate the benefits of FTG combinations. As part of a theoretical study, the singular value spectrum (SVS) and depth-resolution plot (DRP) are examined to determine how much information and resolution different datasets provide in the underdetermined inverse problem. The synthetic examples show that integrating more components in inversion improves the resolution of the recovered model, and the inversion result of gravity or individual component cannot accurately restore the distribution of anomalous bodies. With the increasing sampling interval, joint inversion of FTG data still provides enough information corresponding to the source density distribution, and the limit survey interval is tested through models. In addition, we use a real survey over the Vinton dome as a case study and discover that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$V_{\mathrm {zz}}$ </tex-math></inline-formula> component fails to reconstruct the caprock model at coarse line spacing, whereas FTG combinations recover the acceptable geometry flawlessly. It also reflects the benefit of FTG combinations for inversion processing, which reduces the interference of measured noise on inversion results and allows for longer sampling intervals during the survey, effectively lowering the cost of an airborne survey and improving exploration efficiency

Diffraction Separation and Imaging Using an Improved Singular Value Decomposition Method

Faults, fractures, karst caves, and other small-scale geological targets are critical for carbonate oil and gas exploration. However, seismic responses of these small-scale geological targets are low-energy diffractions and it is difficult for traditional method to image them with high resolution. To identify these targets, the diffraction separation is a key technology. Based on the difference of diffractions and reflections in both kinematic and dynamic properties, the singular value decomposition (SVD) method can separate diffractions and reflections effectively. However, how to select the appropriate singular value sequence for diffractions and reflections is a key issue in the application. Based on the analysis of the singular value spectrum characteristics, we propose a second-order difference spectrum strategy to improve the SVD method. The improved SVD method can separate the diffractions and reflections with minimal error. It is stable when seismic data contain Gaussian noise. Reverse time migration (RTM) method is used in the diffraction imaging because it can keep the true shape of subsurface scatterers when diffractions information is complete. Synthetic examples demonstrate the effectiveness of this method.

Utilizing SVD and VMD for Denoising Non-Stationary Signals of Roller Bearings.

In view of the fact that vibration signals of rolling bearings are much contaminated by noise in the early failure period, this paper presents a new denoising SVD-VMD method by combining singular value decomposition (SVD) and variational mode decomposition (VMD). SVD is used to determine the structure of the underlying model, which is referred to as signal and noise subspaces, and VMD is used to decompose the original signal into several band-limited modes. Then the effective components are selected from these modes to reconstruct the denoised signal according to the difference spectrum (DS) of singular values and kurtosis values. Simulated signals and experimental signals of roller bearing faults have been analyzed using this proposed method and compared with SVD-DS. The results demonstrate that the proposed method can effectively retain the useful signals and denoise the bearing signals in extremely noisy backgrounds.

Tensor renormalization group and the volume independence in 2D U(N) and SU(N) gauge theories

The tensor renormalization group method is a promising approach to lattice field theories, which is free from the sign problem unlike standard Monte Carlo methods. One of the remaining issues is the application to gauge theories, which is so far limited to U(1) and SU(2) gauge groups. In the case of higher rank, it becomes highly nontrivial to restrict the number of representations in the character expansion to be used in constructing the fundamental tensor. We propose a practical strategy to accomplish this and demonstrate it in 2D U(N) and SU(N) gauge theories, which are exactly solvable. Using this strategy, we obtain the singular-value spectrum of the fundamental tensor, which turns out to have a definite profile in the large-N limit. For the U(N) case, in particular, we show that the large-N behavior of the singular-value spectrum changes qualitatively at the critical coupling of the Gross-Witten-Wadia phase transition. As an interesting consequence, we find a new type of volume independence in the large-N limit of the 2D U(N) gauge theory with the θ term in the strong coupling phase, which goes beyond the Eguchi-Kawai reduction.

A nonlocal enhanced Low-Rank tensor approximation framework for 3D Magnetic Resonance image denoising

Magnetic Resonance Imaging (MRI) has become an increasingly essential tool in clinical detection and diagnosis for its ability to disclose the distinctive information of human anatomic structures in vivo. However, the image intensity in magnitude MRI images is frequently corrupted by Rician noise which is inherent to the acquisition process. This study aimed to propose a denoising model for 3D MR image restoration based on low-rank tensor approximation with a Logarithmic-Sum regularization framework. Nonlocal similarity of images was exploited with grouping and block matching techniques, and tensor decomposition approaches were utilized taking consideration of multi-dimensional structural dependence conservation. The low-rank tensor approximation problem was regulated by introducing a Logarithmic-Sum trace norm, which thresholded the singular value spectrum adaptively with different thresholds. Noise removal was processed through the low-rank approximation, and the denoised tensor blocks were aggregated to restore the clear images. Numerical experiments were conducted under comprehensive noise conditions for 3D MR volumetric datasets. Through denoising comparison, the results demonstrated that the proposed algorithm achieves state-of-the-art performance for Rician noise removal with excellent detail preservation..

Automated band selection for Bayesian FFT modal identification Based on RetinaNet

Bayesian FFT algorithm is a popular method to identify modal parameters, e.g., modal frequencies, damping ratios, and mode shapes, of civil structures under operational conditions. It is efficient and provides the identification uncertainty in terms of posterior distribution. However, in utilizing the Bayesian FFT algorithm, it is tedious to manually select frequency bands and initial frequencies. This step requires professional knowledge and costs most of time, which prevents the automation of Bayesian FFT algorithm. Regarding the band selection as an object detection problem, we design a band selection network based on the RetinaNet to automatically select frequency bands and a peak prediction network to predict the initial frequencies. The designed networks are trained using the singular value spectrum of measured ambient vibration data and verified by various data sets. It can achieve the human accuracy with much less operation time, and thus provides a corner stone for the automation of Bayesian FFT algorithm.

Singular value decomposition and similarity renormalization group evolution of nuclear interactions

One of the main challenges for ab initio nuclear many-body theory is the growth of computational and storage costs as calculations are extended to heavy, exotic, and structurally complex nuclei. Here, we investigate the factorization of nuclear interactions as a means to address this issue. We perform Singular Value Decompositions of nucleon-nucleon interactions in partial wave representation and study the dependence of the singular value spectrum on interaction characteristics like regularization scheme and resolution scales. We develop and implement the Similarity Renormalization Group (SRG) evolution of the factorized interaction, and demonstrate that this SVD-SRG approach accurately preserves two-nucleon observables. We find that low-resolution interactions allow the truncation of the SVD at low rank, and that a small number of relevant components is sufficient to capture the nuclear interaction and perform an accurate SRG evolution, while the Coulomb interaction requires special consideration. The rank is uniform across all partial waves, and almost independent of the basis choice in the tested cases. This suggests an interpretation of the relevant singular components as mere representations of a small set of abstract operators that can describe the interaction and its SRG flow. Following the traditional workflow for nuclear interactions, we discuss how the transformation between the center-of-mass and laboratory frames creates redundant copies of the partial wave components when implemented in matrix representation, and we discuss strategies for mitigation. Finally, we test the low-rank approximation to the SRG-evolved interactions in many-body calculations using the In-Medium SRG. By including nuclear radii in our analysis, we verify that the implementation of the SRG using the singular vectors of the interaction does not spoil the evolution of other observables.

Scattered Far-Field Sampling in Multi-Static Multi-Frequency Configuration.

This paper deals with an inverse scattering problem under a linearized scattering model for a multi-static/multi-frequency configuration. The focus is on the determination of a sampling strategy that allows the reduction of the number of measurement points and frequencies and at the same time keeping the same achievable performance in the reconstructions as for full data acquisition. For the sake of simplicity, a 2D scalar geometry is addressed, and the scattered far-field data are collected. The relevant scattering operator exhibits a singular value spectrum that abruptly decays (i.e., a step-like behavior) beyond a certain index, which identifies the so-called number of degrees of freedom (NDF) of the problem. Accordingly, the sampling strategy is derived by looking for a discrete finite set of data points for which the arising semi-discrete scattering operator approximation can reproduce the most significant part of the singular spectrum, i.e., the singular values preceding the abrupt decay. To this end, the observation variables are suitably transformed so that Fourier-based arguments can be used. The arising sampling grid returns several data that is close to the NDF. Unfortunately, the resulting data points (in the angle-frequency domain) leading to a complicated measurement configuration which requires collecting the data at different spatial positions for each different frequency. To simplify the measurement configuration, a suboptimal sampling strategy is then proposed which, by an iterative procedure, enforces the sampling points to belong to a rectangular grid in the angle-frequency domain. As a result of this procedure, the overall data points (i.e., the couples angle-frequency) actually increase but the number of different angles and frequencies reduce and lead to a measurement configuration that is more practical to implement. A few numerical examples are included to check the proposed sampling scheme.