Backward Error Research Articles

Complex Palais matrix and a new unitary transform with bounded component norms

Abstract The Palais matrix represents an n n -dimensional rotation between two vectors that is functionally equivalent to the Householder reflection. This study introduces a one-parameter family of unitary transforms, termed the θ \theta transform, which encompasses the transform by the Palais matrix, the Householder reflection, and their unitary extensions. Furthermore, we define the θ ∠ {\theta }_{\angle } transform, a variant of the θ \theta transform featuring bounded component norms. It is demonstrated that the θ ∠ {\theta }_{\angle } transform is computationally efficient and backward stable when one of the vectors has the “one-hot” structure, making it highly valuable for matrix decompositions such as the QR decomposition. In addition, the θ \theta transform exhibits additional characteristics, including its convergence to the identity and the rowwise structure of its backward error.

Computation of matrix [formula omitted]th roots using moments

In this work we present new theoretical and computational aspects about the matrix pth root problem. In particular, we compute a formulation for the backward error of the problem and adapt the Sakurai-Sugiura moment method to the computation of the pth root of a given matrix. Moreover, we use this approach to approximate this pth root via a quadrature rule.

Pseudospectra, stability radii and their relationship with backward error for structured nonlinear eigenvlaue problems

This paper discusses pseudospectra and stability radii for structured nonlinear matrix functions, such as Hermitian, skew‐Hermitian, H‐even, H‐odd, complex symmetric, and complex skew‐symmetric. To compute pseudospectra and stability radii, eigenvalue backward error is required. Hence, we initially present the structured eigenvalue backward error. Subsequently, we compute the structured pseudospectra using the obtained results for the eigenvalue backward error of a class of structured nonlinear matrix functions. Finally, we discuss the stability radii of the above‐structured problems arising in different applications. The paper also generalizes the results on the eigenvalue backward error of matrix polynomials in the literature for the above structures.

Stabilized recovery and model reduction for multivariate exponential polynomials

Recovery of multivariate exponential polynomials, i.e., the multivariate version of Prony's problem, can be stabilized by using more than the minimally needed multiinteger samples of the function. We present an algorithm that takes into account this extra information and prove a backward error estimate for the algebraic recovery method SMILE. In addition, we give a method to approximate data by an exponential polynomial sequence of a given structure as a step in the direction of multivariate model reduction.

Feed Error Prediction and Compensation of CNC Machine Tools Based on Whale Particle Swarm Backpropagation Neural Network

Current modeling methods of machine tool feed error are challenging to meet the demand of high-precision machining when facing complex machining conditions. To enhance the model’s predictive accuracy and the effectiveness of actual compensation, the Whale Particle Swarm Optimization (WPSO) algorithm is proposed to optimize the Backpropagation Neural Network (BPNN). Subsequently, the optimized network incorporates screw elongation and feed position as inputs to establish a feed-error prediction model. Ultimately, the established model was compared with other models and applied to real-time compensation experiments. The research results show that the proposed prediction model outperforms the BPNN model, the particle swarm-optimized BPNN model, and the whale-optimized BPNN model in various indicators. The accuracy of the prediction model was 93.12%, and the errors ranged from −3.80 μm to 4.57 μm with an average error of −0.30 μm. Under different operating conditions, the maximum backward and forward errors are reduced by 33.21% and 87.21%, and the average backward and forward errors are reduced by 57.15% and 84.37%, respectively. The error range is reduced by 67.41%. Beyond elevating prediction accuracy and compensation efficacy, the proposed model offers robust theoretical guidance for practical production.

Structured eigenvalue backward errors for rational matrix functions with symmetry structures

Structured eigenvalue backward errors for rational matrix functions with symmetry structures

Structured backward errors for block three-by-three saddle point systems

In this paper, we analyze the structured backward error for block three-by-three saddle point systems. The computable formulas of structured backward errors are derived, which are useful for testing the stability of practical algorithms.

Backward error analysis and the qualitative behaviour of stochastic optimization algorithms: Application to stochastic coordinate descent

Backward error analysis and the qualitative behaviour of stochastic optimization algorithms: Application to stochastic coordinate descent

Strong backward error analysis of symplectic integrators for stochastic Hamiltonian systems

Backward error analysis is a powerful tool in order to detect the long-term conservative behavior of numerical methods. In this work, we present a long-term analysis of symplectic stochastic numerical integrators, applied to Hamiltonian systems with multiplicative noise. We first compute and analyze the associated stochastic modified differential equations. Then, suitable bounds for the coefficients of such equations are provided towards the computation of long-term estimates for the Hamiltonian deviations occurring along the aforementioned numerical dynamics. This result generalizes Benettin-Giorgilli Theorem to the scenario of stochastic symplectic methods. Finally, specific numerical methods are considered, in order to provide a numerical evidence confirming the effectiveness of the theoretical investigation.

On the accuracy of de Casteljau-type algorithms and Bernstein representations

This paper summarizes interesting results on systematic backward and forward error analyses performed for corner cutting algorithms providing evaluation of univariate and multivariate functions defined in terms of Bernstein and Bernstein related bases. Relevant results on the conditioning of the bases are also recalled. Finally, the paper surveys important advances, lately obtained, for the design of algorithms adapted to the structure of totally positive matrices, allowing the resolution of interpolation and approximation problems with Bernstein-type bases achieving computations to high relative accuracy.

Low‐rank updates of matrix square roots

AbstractModels in which the covariance matrix has the structure of a sparse matrix plus a low rank perturbation are ubiquitous in data science applications. It is often desirable for algorithms to take advantage of such structures, avoiding costly matrix computations that often require cubic time and quadratic storage. This is often accomplished by performing operations that maintain such structures, for example, matrix inversion via the Sherman–Morrison–Woodbury formula. In this article, we consider the matrix square root and inverse square root operations. Given a low rank perturbation to a matrix, we argue that a low‐rank approximate correction to the (inverse) square root exists. We do so by establishing a geometric decay bound on the true correction's eigenvalues. We then proceed to frame the correction as the solution of an algebraic Riccati equation, and discuss how a low‐rank solution to that equation can be computed. We analyze the approximation error incurred when approximately solving the algebraic Riccati equation, providing spectral and Frobenius norm forward and backward error bounds. Finally, we describe several applications of our algorithms, and demonstrate their utility in numerical experiments.

Adversarial ink: componentwise backward error attacks on deep learning

Abstract Deep neural networks are capable of state-of-the-art performance in many classification tasks. However, they are known to be vulnerable to adversarial attacks—small perturbations to the input that lead to a change in classification. We address this issue from the perspective of backward error and condition number, concepts that have proved useful in numerical analysis. To do this, we build on the work of Beuzeville, T., Boudier, P., Buttari, A., Gratton, S., Mary, T. and Pralet S. (2021) Adversarial attacks via backward error analysis. hal-03296180, version 3. In particular, we develop a new class of attack algorithms that use componentwise relative perturbations. Such attacks are highly relevant in the case of handwritten documents or printed texts where, for example, the classification of signatures, postcodes, dates or numerical quantities may be altered by changing only the ink consistency and not the background. This makes the perturbed images look natural to the naked eye. Such ‘adversarial ink’ attacks therefore reveal a weakness that can have a serious impact on safety and security. We illustrate the new attacks on real data and contrast them with existing algorithms. We also study the use of a componentwise condition number to quantify vulnerability.

Computing with B-series

We present BSeries.jl, a Julia package for the computation and manipulation of B-series, which are a versatile theoretical tool for understanding and designing discretizations of differential equations. We give a short introduction to the theory of B-series and associated concepts and provide examples of their use, including method composition and backward error analysis. The associated software is highly performant and makes it possible to work with B-series of high order.

Perturbations of Tensor-Schur decomposition and its applications to multilinear control systems and facial recognitions

Perturbation analysis has been primarily considered to be one of the main issues in many fields. The Schur decomposition can factor a square matrix as the product of a unitary matrix and an upper triangular matrix, which contains eigenvalues of the square matrix. In view of the importance of T-eigenvalue problems, this paper discusses a tensor Schur decomposition (T-Schur), the T-Schur decomposition =U∗T∗U∗, which is based on T-product multiplication of third-order tensors. We present the normwise and componentwise perturbation analysis for the unitary tensor U, the upper triangular tensor T and T-eigenvalues of the tensor A. We explore some applications of the T-Schur decomposition and perform the T-Schur form to solve the tensor pole assignment. We give an algorithm to solve the tensor based T-Sylvester equation and present its perturbation bound and the backward error. The T-Sylvester equation can also be used to estimate the condition of the T-sign function. We apply the T-Schur decomposition to facial recognation and compare it with other types of tensor decompositions.

Shadow energy functionals and potentials in Born-Oppenheimer molecular dynamics.

In Born-Oppenheimer molecular dynamics (BOMD) simulations based on the density functional theory (DFT), the potential energy and the interatomic forces are calculated from an electronic ground state density that is determined by an iterative self-consistent field optimization procedure, which, in practice, never is fully converged. The calculated energies and forces are, therefore, only approximate, which may lead to an unphysical energy drift and instabilities. Here, we discuss an alternative shadow BOMD approach that is based on backward error analysis. Instead of calculating approximate solutions for an underlying exact regular Born-Oppenheimer potential, we do the opposite. Instead, we calculate the exact electron density, energies, and forces, but for an underlying approximate shadow Born-Oppenheimer potential energy surface. In this way, the calculated forces are conservative with respect to the approximate shadow potential and generate accurate molecular trajectories with long-term energy stabilities. We show how such shadow Born-Oppenheimer potentials can be constructed at different levels of accuracy as a function of the integration time step, δt, from the constrained minimization of a sequence of systematically improvable, but approximate, shadow energy density functionals. For each energy functional, there is a corresponding ground state Born-Oppenheimer potential. These pairs of shadow energy functionals and potentials are higher-level generalizations of the original "zeroth-level" shadow energy functionals and potentials used in extended Lagrangian BOMD [Niklasson, Eur. Phys. J. B 94, 164 (2021)]. The proposed shadow energy functionals and potentials are useful only within this extended dynamical framework, where also the electronic degrees of freedom are propagated as dynamical field variables together with the atomic positions and velocities. The theory is quite general and can be applied to MD simulations using approximate DFT, Hartree-Fock, or semi-empirical methods, as well as to coarse-grained flexible charge models.

Long-Term Analysis of Stochastic Hamiltonian Systems Under Time Discretizations

In this paper, we focus our investigation on providing long-term estimates of the Hamiltonian deviation computed along numerical approximations to the solutions of stochastic Hamiltonian systems, of both Itô and Stratonovich types. It is well known that the expected Hamiltonian of an Itô Hamiltonian system with additive noise exhibits a linear drift in time [C. Chen et al., Adv. Comput. Math., 46 (2020), 27], while the Hamiltonian function is conserved along the exact flow of a Stratonovich Hamiltonian system [T. Misawa, Japan J. Indust. Appl. Math., 17 (2000), pp. 119–128; T. Misawa, Math. Probl. Eng., 2010 (2010), 384937]. Here, we focus our attention on providing modified differential equations associated to suitable discretizations for the above problems, by means of weak backward error analysis arguments [K. C. Zygalakis, SIAM J. Sci. Comput., 33 (2011), pp. 102–130]. Then, long-term estimates are provided for both Itô and Stratonovich Hamiltonian systems, revealing the presence of parasitic terms affecting the overall conservation accuracy. Finally, selected numerical experiments are provided to confirm the theoretical analysis.

Learning effective stochastic differential equations from microscopic simulations: Linking stochastic numerics to deep learning.

We identify effective stochastic differential equations (SDEs) for coarse observables of fine-grained particle- or agent-based simulations; these SDEs then provide useful coarse surrogate models of the fine scale dynamics. We approximate the drift and diffusivity functions in these effective SDEs through neural networks, which can be thought of as effective stochastic ResNets. The loss function is inspired by, and embodies, the structure of established stochastic numerical integrators (here, Euler-Maruyama and Milstein); our approximations can thus benefit from backward error analysis of these underlying numerical schemes. They also lend themselves naturally to "physics-informed" gray-box identification when approximate coarse models, such as mean field equations, are available. Existing numerical integration schemes for Langevin-type equations and for stochastic partial differential equations can also be used for training; we demonstrate this on a stochastically forced oscillator and the stochastic wave equation. Our approach does not require long trajectories, works on scattered snapshot data, and is designed to naturally handle different time steps per snapshot. We consider both the case where the coarse collective observables are known in advance, as well as the case where they must be found in a data-driven manner.

Fuzzy Cognitive Map-Driven Comprehensive Time-Series Classification.

This article presents a comprehensive approach for time-series classification. The proposed model employs a fuzzy cognitive map (FCM) as a classification engine. Preprocessed input data feed the employed FCM. Map responses, after a postprocessing procedure, are used in the calculation of the final classification decision. The time-series data are staged using the moving-window technique to capture the time flow in the training procedure. We use a backward error propagation algorithm to compute the required model hyperparameters. Four model hyperparameters require tuning. Two are crucial for the model construction: 1) FCM size (number of concepts) and 2) window size (for the moving-window technique). Other two are important for training the model: 1) the number of epochs and 2) the learning rate (for training). Two distinguishing aspects of the proposed model are worth noting: 1) the separation of the classification engine from pre- and post-processing and 2) the time flow capture for data from concept space. The proposed classifier joins the key advantage of the FCM model, which is the interpretability of the model, with the superior classification performance attributed to the specially designed pre- and postprocessing stages. This article presents the experiments performed, demonstrating that the proposed model performs well against a wide range of state-of-the-art time-series classification algorithms.

Blends have decent numerical properties

A "blend" is a two-point Hermite interpolational polynomial, typically of quite high degree. This note shows that implementing them in a double Horner evaluation scheme has good backward error, and also shows that the Lebesgue constant for a balanced blend or nearly balanced blend on the interval [0,1] is bounded by 2, independently of the grade or degree of the approximation. On [-1,1], which is a more natural interval for comparison, it is of course unbounded, but grows only like 2√(m/π) where 2m+1 is the grade of approximation. I also show that the quadrature schemes for balanced blends amplify errors only by O( ln(m) ).

Algorithms for Sparse Multichannel Blind Deconvolution

In this paper we present two algorithms for sparse multichannel blind deconvolution. The first algorithm is based on a cascade of a forward and a backward prediction error filter (C-PEF). The second consists in an alternating minimization algorithm for estimating both the reflectivity series and the seismic wavelet (AM-SMBD). We also compare the algorithms with other state-of-the-art sparse blind deconvolution algorithms. Simulation results with synthetic data for different SNR levels showed that the AM-SMBD was outperformed (in terms of the Pearson Correlation Coefficient and the Gini Correlation Coefficient) other estimation methods, like the reduced SMBD, the TS-sparseTLS (Toeplitz-structured sparse total least square) and the SMBD-SPG (Sparse multichannel blind deconvolution via spectral projected-gradient). For the same data, the C-PEF was able to provide better results (in terms of the Gini Correlation Coefficient, visual inspection and frequency gain) when compared with the F-SMBD (Fast sparse multichannel blind deconvolution). In a simulation considering reflectivities with different levels of sparsity, the C-PEF seems to be more robust for less sparse data when compared with AM-SMBD and SMBD-SPG (up to a certain degree of sparsity). Finally, simulations considering a real land acquisition show that both algorithms were able to greatly improve the resolution of the seismic data.