Spectral Regularization Method Research Articles

The Eigenspace Spectral Regularization Method for Solving Discrete Ill-Posed Systems

This paper shows that discrete linear equations with Hilbert matrix operator, circulant matrix operator, conference matrix operator, banded matrix operator, TST matrix operator, and sparse matrix operator are ill-posed in the sense of Hadamard. Gauss least square method (GLSM), QR factorization method (QRFM), Cholesky decomposition method (CDM), and singular value decomposition (SVDM) failed to regularize these ill-posed problems. This paper introduces the eigenspace spectral regularization method (ESRM), which solves ill-posed discrete equations with Hilbert matrix operator, circulant matrix operator, conference matrix operator, and banded and sparse matrix operator. Unlike GLSM, QRFM, CDM, and SVDM, the ESRM regularizes such a system. In addition, the ESRM has a unique property, the norm of the eigenspace spectral matrix operator κ K = K − 1 K = 1 . Thus, the condition number of ESRM is bounded by unity, unlike the other regularization methods such as SVDM, GLSM, CDM, and QRFM.

An inverse problem for a time-fractional advection equation associated with a nonlinear reaction term

Fractional derivative is an important notion in the study of the contemporary mathematics not only because it is more mathematically general than the classical derivative but also it really has applications to understand many physical phenomena. In particular, fractional derivatives are related to long power-law particle jumps, which can be understood as transient anomalous sub-diffusion model (see Sabzikar F, Meerschaert M, Chen J. Tempered fractional calculus. J Comput Phys. 2015;293:14–28; Sokolov IM, Klafter J, Blumen A. Fractional kinetics. Phys Today. 2002;55:48–54; Sokolov IM, Klafter J. Anomalous diffusion spreads its wings. Phys World. 2005;18:19–22; Zhang Y, Meerschaert MM, Packman AI. Linking fluvial bed sediment transport across scales. Geophys Res Lett. 2012;39(20):20404. doi:10.1029/2012GL053476). Based on the models given in Scher H, Montroll EW. Anomalous transit-time dispersion in amorphous solids. Phys Rev B. 1975;12(6):2455–2477 and Zheng GH, Wei T. Spectral regularization method for solving a time-fractional inverse diffusion problem. Appl Math Comput. 2011;218:1972–1990, we study an inverse problem for the advection equation with a nonlinear reaction term in a two-dimensional semi-infinite domain for which we recover the initial distribution from the observation data provided at the final location x = 1. This problem is severely ill-posed in the sense of Hadamard. Thus, we propose a regularization method to construct an approximate solution for the problem. From that, convergence rate of the regularized solution is obtained under some a priori bound assumptions on the exact solution. Eventually, a numerical experiment is given to show the effectiveness of the proposed regularization methods.

SAR Target Classification Based on Sample Spectral Regularization

Synthetic Aperture Radar (SAR) target classification is an important branch of SAR image interpretation. The deep learning based SAR target classification algorithms have made remarkable achievements. But the acquisition and annotation of SAR target images are time-consuming and laborious, and it is difficult to obtain sufficient training data in many cases. The insufficient training data can make deep learning based models suffering from over-fitting, which will severely limit their wide application in SAR target classification. Motivated by the above problem, this paper employs transfer-learning to transfer the prior knowledge learned from a simulated SAR dataset to a real SAR dataset. To overcome the sample restriction problem caused by the poor feature discriminability for real SAR data. A simple and effective sample spectral regularization method is proposed, which can regularize the singular values of each SAR image feature to improve the feature discriminability. Based on the proposed regularization method, we design a transfer-learning pipeline to leverage the simulated SAR data as well as acquire better feature discriminability. The experimental results indicate that the proposed method is feasible for the sample restriction problem in SAR target classification. Furthermore, the proposed method can improve the classification accuracy when relatively sufficient training data is available, and it can be plugged into any convolutional neural network (CNN) based SAR classification models.

Spectral regularization for combating mode collapse in GANs

Generative adversarial networks (GANs) have been enjoying considerable success in recent years. However, mode collapse remains a major unsolved problem in training GANs and is one of the main obstacles hindering progress. In this paper, we present spectral regularization for GANs (SR-GANs), a new and robust method for combating the mode collapse problem in GANs. We first perform theoretical analysis to show that the spectral distributions of the weight matrix in the discriminator affect how the equality of Lipschitz constraint can be fulfilled, thus will have an impact on the performance of the discriminator. Prompted by these analysis, we set out to monitor the spectral distributions in the discriminators of spectral normalized GANs (SN-GANs), and discover a phenomenon which we refer to as spectral collapse, where a large number of singular values of the weight matrices drop dramatically when mode collapse occurs. We show that there are strong evidences linking mode collapse to spectral collapse; and based on this link, we set out to tackle spectral collapse as a surrogate of mode collapse. We have developed a spectral regularization method where we introduce two schemes, one static and one dynamic, to compensate the spectral distributions of the weight matrices to prevent them from collapsing, which in turn successfully prevents mode collapse in GANs. Through gradient analysis, we provide theoretical explanations for why SR-GANs are more stable and can provide better performances than SN-GANs. We also present extensive experimental results and analysis to show that SR-GANs not only always outperform SN-GANs but also always succeed in combating mode collapse where SN-GANs fail. The code is available at https://github.com/max-liu-112/SRGANs-Spectral-Regularization-GANs-

Force reconstruction based on the union of singular spectral analysis and regularization method

To overcome the problem of some ill-posed inverse problem of force reconstruction, which is caused by the noise in the measured responses and small singular values of the structure, a technology of force reconstruction based on a hybrid method of singular spectral analysis (SSA) and the Landweber regularization method is proposed in this study for the first time. The SSA is used to filter the structural response before using Landweber regularization. A new choice method of phase space reconstruction dimension is theoretically proposed, and the minimum embedding dimension is determined by the concept of optimizing difference spectrum theory. The feasibility of this method was demonstrated through three kinds of force reconstructions. The numerical simulation results and an acoustic vibration experiment demonstrated that the proposed method is more effective than the traditional method.Communicated by Wei-Chau Xie.

On the backward problem for parabolic equations with memory

ABSTRACT We study the backward problem for the parabolic equation with memory. We prove that the nonlocal problem is well-posed for 0<t<T and is ill-posed only at the initial time of t=0. This result makes a big difference between the classical backward parabolic equation and the backward parabolic equation with memory. We further propose a spectral regularization method to overcome the ill-posedness at the initial time. The Hölder convergence rate is obtained under both a priori and a posteriori parameter choice rules.

A truncated spectral regularization method for a source identification problem

Abstract inverse source problem of identifying the source function f in the abstract Cauchy problem $$u_t+Au=f(t),\, 0<t<\tau $$ with $$u(0)=\phi _0$$ when the data, the final value, $$u(\tau )=\phi _\tau $$ is noisy is considered, where A is a densly defined self-adjont coercive unbounded operator on a Hilbert space H. This problem is known to be an ill-posed problem. A truncated spectral representation of a mild solution of the above problem is shown to be a regularized approximation, and error analysis is carried out when $$\phi _\tau $$ is noisy as well as exact, and stability estimate is given under appropriate parameter choice strategies.

The Backward Problem for a Nonlinear Riesz-Feller Diffusion Equation

In this paper, we reconstruct the solution u(x,t) of the backward space-fractional diffusion problem with a locally Lipschitzian nonlinear source This problem is severely ill-posed in the Hadamard sense, hence, a regularization is in order. In the paper, we introduce one spectral regularization method and establish stability error estimates with optimal order under an a priori choice of regularization parameter. Finally, numerical implementations are given to show the effectiveness of the proposed regularization methods.

Determination of the unknown source term in a space-fractional diffusion equation

This paper studies an inverse problem of determining the unknown source term in a space-fractional diffusion equation. Three types of spectral regularization method are proposed to deal with the ill-posed problem and the corresponding error estimates are obtained with an a priori strategy to find the regularization parameter. We verify the efficiency of the proposed numerical method with some numerical experiments.

A Source-Type Best Approximation Method for Imaging Applications

In this letter, we embark on solving the inverse source problem for electromagnetic (EM) waves with application to target detection. The best approximate solution is derived in terms of the singular value decomposition (SVD) of the direct integral operator, while avoiding any optimization or iterative scheme. Inherent complications in the process are also expressed. The spectral cutoff regularization method is employed to overcome the ill-posed nature of the integral equation. The algorithm is applied to the detection and localization of infinitely long PEC and dielectric cylinders demonstrating robustness under noise impact.

Directional convergence of spectral regularization method associated to families of closed operators

We consider regularized solutions of linear inverse ill-posed problems obtained with generalized Tikhonov–Phillips functionals with penalizers given by linear combinations of seminorms induced by closed operators. Convergence of the regularized solutions is proved when the vector regularization rule approaches the origin through appropriate radial and differentiable paths. Characterizations of the limiting solutions are given. Finally, examples of image restoration using generalized Tikhonov–Phillips methods with convex combinations of seminorms are shown.

On the existence of global saturation for spectral regularization methods with optimal qualification

Abstract. A family of real functions defining a spectral regularization method with optimal qualification is considered. Sufficient condition on the family and on the optimal qualification guaranteeing the existence of saturation are established. Appropriate characterizations of both the saturation function and the saturation set are found and some examples are provided.

Regularization strategy for the Cauchy problem of Laplace’s equation from the viewpoint of regularization theory

Within the framework of the regularization theory, a spectral regularization method is introduced and analyzed. The convergence estimate under an appropriate choice of regularization parameter is obtained. A numerical implementation is described. Numerical examples show that the proposed method is effective and stable.

Spectral regularization method for solving a time-fractional inverse diffusion problem

Abstract In this paper, we consider an inverse problem for a time-fractional diffusion equation with one-dimensional semi-infinite domain. The temperature and heat flux are sought from a measured temperature history at a fixed location inside the body. We show that such problem is severely ill-posed and further apply a spectral regularization method to solve it based on the solution given by the Fourier method. Convergence estimates are presented under a priori bound assumptions for the exact solution. Finally, numerical examples are given to show that the proposed numerical method is effective.

Adaptive spectral regularizations of high dimensional linear models

This paper focuses on recovering an unknown vector β from the noisy data Y=Xβ+σξ, where X is a known n×p-matrix, ξ is a standard white Gaussian noise, and σ is an unknown noise level. In order to estimate β, a spectral regularization method is used, and our goal is to choose its regularization parameter with the help of the data Y. In this paper, we deal solely with regularization methods based on the so-called ordered smoothers (see [13]) and extend the oracle inequality from [11] to the case, where the noise level is unknown.

Two regularization methods for solving a Riesz–Feller space-fractional backward diffusion problem

In this paper, a backward diffusion problem for a space-fractional diffusion equation (SFDE) in a strip is investigated. Such a problem is obtained from the classical diffusion equation in which the second-order space derivative is replaced with a Riesz–Feller derivative of order β ∊ (0, 2]. We show that such a problem is severely ill-posed and further propose a new regularization method and apply a spectral regularization method to solve it based on the solution given by the Fourier method. Convergence estimates are presented under a priori bound assumptions for the exact solution. Finally, numerical examples are given to show that the proposed numerical methods are effective.

On universal oracle inequalities related to high-dimensional linear models

This paper deals with recovering an unknown vector θ from the noisy data Y = Aθ + σξ, where A is a known (m × n)-matrix and ξ is a white Gaussian noise. It is assumed that n is large and A may be severely ill-posed. Therefore, in order to estimate θ, a spectral regularization method is used, and our goal is to choose its regularization parameter with the help of the data Y. For spectral regularization methods related to the so-called ordered smoothers [see Kneip Ann. Statist. 22 (1994) 835–866], we propose new penalties in the principle of empirical risk minimization. The heuristical idea behind these penalties is related to balancing excess risks. Based on this approach, we derive a sharp oracle inequality controlling the mean square risks of data-driven spectral regularization methods.

Spectral regularization method for the time fractional inverse advection–dispersion equation

In this paper, we consider the time fractional inverse advection–dispersion problem (TFIADP) in a quarter plane. The solute concentration and dispersion flux are sought from a measured concentration history at a fixed location inside the body. Such problem is obtained from the classical advection–dispersion equation by replacing the first-order time derivative by the Caputo fractional derivative of order α(0 < α < 1). We show that the TFIADP is severely ill-posed and further apply a spectral regularization method to solve it based on the solution given by the Fourier method. Convergence estimates are presented under a priori bound assumptions for the exact solution. Finally, numerical examples are given to show that the proposed numerical method is effective.

A Spectral Regularization Method for a Cauchy Problem of the Modified Helmholtz Equation

We consider the Cauchy problem for a modified Helmholtz equation, where the Cauchy data is given at and the solution is sought in the interval . A spectral method together with choice of regularization parameter is presented and error estimate is obtained. Combining the method of lines, we formulate regularized solutions which are stably convergent to the exact solutions.

Spectral regularization method for a Cauchy problem of the time fractional advection–dispersion equation

In this paper, a Cauchy problem for the time fractional advection–dispersion equation (TFADE) is investigated. Such a problem is obtained from the classical advection–dispersion equation by replacing the first-order time derivative by the Caputo fractional derivative of order α ( 0 < α ≤ 1 ) . We show that the Cauchy problem of TFADE is severely ill-posed and further apply a spectral regularization method to solve it based on the solution given by the Fourier method. The convergence estimate is obtained under a priori bound assumptions for the exact solution. Numerical examples are given to show the effectiveness of the proposed numerical method.