Theoretical Error Analysis Research Articles

Convergence analysis of pressure reconstruction methods from discrete velocities

Magnetic resonance imaging allows the measurement of the three-dimensional velocity field in blood flows. Therefore, several methods have been proposed to reconstruct the pressure field from such measurements using the incompressible Navier–Stokes equations, thereby avoiding the use of invasive technologies. However, those measurements are obtained at limited spatial resolution given by the voxel sizes in the image. In this paper, we propose a strategy for the convergence analysis of state-of-the-art pressure reconstruction methods. The methods analyzed are the so-called Pressure Poisson Estimator (PPE) and Stokes Estimator (STE). In both methods, the right-hand side corresponds to the terms that involving the field velocity in the Navier–Stokes equations, with a piecewise linear interpolation of the exact velocity. In the theoretical error analysis, we show that many terms of different order of convergence appear. These are certainly dominated by the lowest-order term, which in most cases stems from the interpolation of the velocity field. However, the numerical results in academic examples indicate that only the PPE may profit of increasing the polynomial order, and that the STE presents a higher accuracy than the PPE, but the interpolation order of the velocity field always prevails. Furthermore, we compare the pressure estimation methods on real MRI data, assessing the impact of different spatial resolutions and polynomial degrees on each method. Here, the results are consistent with the academic test cases in terms of sensitivity to polynomial order as well as the STE showing to be potentially more accurate when compared to reference pressure measurements.

Theoretical bounds of generalization error for generalized extreme learning machine and random vector functional link network

Ensuring the prediction accuracy of a learning algorithm on a theoretical basis is crucial and necessary for building the reliability of the learning algorithm. This paper analyzes prediction error obtained through the least square estimation in the generalized extreme learning machine (GELM), which applies the limiting behavior of the Moore–Penrose generalized inverse (M–P GI) to the output matrix of ELM. ELM is the random vector functional link (RVFL) network without direct input to output links Specifically, we analyze tail probabilities associated with upper and lower bounds to the error expressed by norms. The analysis employs the concepts of the L2 norm, the Frobenius norm, the stable rank, and the M–P GI. The coverage of theoretical analysis extends to the RVFL network. In addition, a criterion for more precise bounds of prediction errors that may give stochastically better network environments is provided. The analysis is applied to simple examples and large-size datasets to illustrate the procedure and verify the analysis and execution speed with big data. Based on this study, we can immediately obtain the upper and lower bounds of prediction errors and their associated tail probabilities through matrices calculations appearing in the GELM and RVFL. This analysis provides criteria for the reliability of the learning performance of a network in real-time and for network structure that enables obtaining better performance reliability. This analysis can be applied in various areas where the ELM and RVFL are adopted. The proposed analytical method will guide the theoretical analysis of errors occurring in DNNs, which employ a gradient descent algorithm.

Fourier Series Expansion Based Filter Parametrization for Equivariant Convolutions.

It has been shown that equivariant convolution is very helpful for many types of computer vision tasks. Recently, the 2D filter parametrization technique has played an important role for designing equivariant convolutions, and has achieved success in making use of rotation symmetry of images. However, the current filter parametrization strategy still has its evident drawbacks, where the most critical one lies in the accuracy problem of filter representation. To address this issue, in this paper we explore an ameliorated Fourier series expansion for 2D filters, and propose a new filter parametrization method based on it. The proposed filter parametrization method not only finely represents 2D filters with zero error when the filter is not rotated (similar as the classical Fourier series expansion), but also substantially alleviates the aliasing-effect-caused quality degradation when the filter is rotated (which usually arises in classical Fourier series expansion method). Accordingly, we construct a new equivariant convolution method based on the proposed filter parametrization method, named F-Conv. We prove that the equivariance of the proposed F-Conv is exact in the continuous domain, which becomes approximate only after discretization. Moreover, we provide theoretical error analysis for the case when the equivariance is approximate, showing that the approximation error is related to the mesh size and filter size. Extensive experiments show the superiority of the proposed method. Particularly, we adopt rotation equivariant convolution methods to a typical low-level image processing task, image super-resolution. It can be substantiated that the proposed F-Conv based method evidently outperforms classical convolution based methods. Compared with pervious filter parametrization based methods, the F-Conv performs more accurately on this low-level image processing task, reflecting its intrinsic capability of faithfully preserving rotation symmetries in local image features.

Theoretical Error Analysis of Hybrid Finite Difference–Asymptotic Interpolation Method for Non-Newtonian Fluid Flow

In this paper, we utilized a hybrid method for the unsteady flow of the non-Newtonian third-grade fluid that combines the finite difference with the asymptotic interpolation method. This hybrid method is used to satisfy the semiunbound domain condition of the fluid flow’s length approaching infinity. The primary issue with this research is how much of the hybrid approach’s error may be accepted to guarantee that the method is significant. This paper discussed theoretical error analysis for numerical solutions, including the range and norm of error. The perturbation method’s concept is used to assess the hybrid method’s error. It is discovered that the hybrid approach’s relative error norm is lower than that of the finite difference method. In terms of the error standard, the hybrid approach is more consistent. Error analysis is performed to check for the accuracy as well as the platform for variable mesh size finite difference method in the future research.

SOLVING LINEAR VOLTERRA INTEGRAL EQUATIONS WITH A PIECEWISE LINEAR MAXIMUM ENTROPY METHOD

Based on the principle of maximum entropy and piecewise linear basis functions, a numerical method for approximating a nonnegative solution of Volterra integral equations is proposed. Then a discrete form of the maximum entropy method is given to reduce the cost. The error analysis and the convergence rate are provided. Numerical experimental results are consistent with the theoretical analysis of errors and the convergence rate.

Theoretical error analysis of spotlight-based instrument localization for retinal surgery.

Retinal surgery is widely considered to be a complicated and challenging task even for specialists. Image-guided robot-assisted intervention is among the novel and promising solutions that may enhance human capabilities therein. In this paper, we demonstrate the possibility of using spotlights for 5D guidance of a microsurgical instrument. The theoretical basis of the localization for the instrument based on the projection of a single spotlight is analyzed to deduce the position and orientation of the spotlight source. The usage of multiple spotlights is also proposed to check the possibility of further improvements for the performance boundaries. The proposed method is verified within a high-fidelity simulation environment using the 3D creation suite Blender. Experimental results show that the average positioning error is 0.029 mm using a single spotlight and 0.025 mm with three spotlights, respectively, while the rotational errors are 0.124 and 0.101, which shows the application to be promising in instrument localization for retinal surgery.

Theoretical analysis of transmission error in rack and pinion systems

Rack and pinion drive systems are widely used in machine tools with long travel distances because the stiffness is independent of the travelled distance, in contrast to other drive systems such as ballscrews. Although the inherent backlash problem of gear systems has been solved by means of utilizing two pinions independently preloaded, the time-varying mesh stiffness causes periodic position differences between the motor encoder and table position, which is known as transmission error, and may lead to vibrations and dynamic load. Few experimental works have analyzed such transmission error, but there are no theoretical approaches in the scientific literature. Therefore, this work aims to first extend previous analytical and approximate equations for mesh stiffness of gearing to rack and pinion systems and validate them with finite element methods. Finally, the effect of geometry variations, such as pressure angle and tooth thickness tolerance, on transmission error is discussed.

An averaged space–time discretization of the stochastic p-Laplace system

We study the stochastic p-Laplace system in a bounded domain. We propose two new space–time discretizations based on the approximation of time-averaged values. We establish linear convergence in space and 1/2 convergence in time. Additionally, we provide a sampling algorithm to construct the necessary random input in an efficient way. The theoretical error analysis is complemented by numerical experiments.

Performance Analysis and Sensor-Target Geometry Optimization for TOA and TDOA-Based Hybrid Source Localization Method

Currently, the performance analysis of positioning algorithms and optimization of ground station deployment schemes are predominantly based on pure TOA or TDOA measurement information, and the relevant theoretical analysis is primarily the geometric analysis of optimal station deployment for fixed point targets, with few placement ranges and amount of station constraints. In practice, however, there are typically several measurements from TOA and TDOA stations, with a focus on positioning precision within a certain region or line trajectory, as well as the necessity for constraints on the ground station placement range. This paper proposes an efficient method for hybrid source localization using TOA and TDOA measurement information, establishes a mathematical model for hybrid source localization based on TOA and TDOA measurement information, derives and simulates the Gauss–Newton iterative localization algorithm with the least squares criterion, and performs a theoretical analysis of the least squares error and CRLB boundary to improve the accuracy of target localization in the aforementioned scenarios. Taking the average CRLB value of target line trajectory positioning error as the objective function, the ground station placement scheme of TOA- and TDOA-receiving sensors is optimized by utilizing a Genetic Algorithm with strong global optimization capability under the constraints of station placement range and station quantities, and a station placement geometry with better performance than typical station placement is obtained. Meanwhile, we summarize the general placement principles for TOA and TDOA hybrid source localization of target line trajectories.

Error analysis of a Haar wavelets-based numerical method with its application to a nonlinear fractional dengue model

Millions of people are infected by Dengue every year. So to understand its mechanism, we have considered one of the recent fractional and nonlinear Dengue models proposed by Diethelm. However, the problem is that its exact solution is not known. Therefore, an extended Haar wavelets based numerical method has been implemented to solve it. Although Haar wavelets-based numerical methods have been used by researchers in recent years its theoretical error analysis over the extended interval , , has never been done before for fractional dynamical systems. It helps to show the convergence of a sequence of approximate solutions to the exact solution. Also, the derived error bound gives a guarantee that error in the considered Haar approximate solution cannot exceed its bound. And any desired accuracy can be obtained by increasing the number of Haar wavelets in the approximate solution.

Top-k Partial Label Machine.

To deal with ambiguities in partial label learning (PLL), the existing PLL methods implement disambiguations, by either identifying the ground-truth label or averaging the candidate labels. However, these methods can be easily misled by the false-positive labels in the candidate label set. We find that these ambiguities often originate from the noise caused by highly correlated or overlapping candidate labels, which leads to the difficulty in identifying the ground-truth label on the first attempt. To give the trained models more tolerance, we first propose the top-k partial loss and convex top-k partial hinge loss. Based on the losses, we present a novel top-k partial label machine (TPLM) for partial label classification. An efficient optimization algorithm is proposed based on accelerated proximal stochastic dual coordinate ascent (Prox-SDCA) and linear programming (LP). Moreover, we present a theoretical analysis of the generalization error for TPLM. Comprehensive experiments on both controlled UCI datasets and real-world partial label datasets demonstrate that the proposed method is superior to the state-of-the-art approaches.

Numerical computation of the equilibrium-reduced density matrix for strongly coupled open quantum systems.

We describe a numerical algorithm for approximating the equilibrium-reduced density matrix and the effective (mean force) Hamiltonian for a set of system spins coupled strongly to a set of bath spins when the total system (system + bath) is held in canonical thermal equilibrium by weak coupling with a "super-bath". Our approach is a generalization of now standard typicality algorithms for computing the quantum expectation value of observables of bare quantum systems via trace estimators and Krylov subspace methods. In particular, our algorithm makes use of the fact that the reduced system density, when the bath is measured in a given random state, tends to concentrate about the corresponding thermodynamic averaged reduced system density. Theoretical error analysis and numerical experiments are given to validate the accuracy of our algorithm. Further numerical experiments demonstrate the potential of our approach for applications including the study of quantum phase transitions and entanglement entropy for long range interaction systems.

Partial Multi-Label Learning via Large Margin Nearest Neighbour Embeddings

To deal with ambiguities in partial multi-label learning (PML), existing popular PML research attempts to perform disambiguation by direct ground-truth label identification. However, these approaches can be easily misled by noisy false-positive labels in the iteration of updating the model parameter and the latent ground-truth label variables. When labeling information is ambiguous, we should depend more on underlying structure of data, such as label and feature correlations, to perform disambiguation for partially labeled data. Moreover, large margin nearest neighbour (LMNN) is a popular strategy that considers data structure in classification. However, due to the ambiguity of labeling information in PML, traditional LMNN cannot be used to solve the PML problem directly. In addition, embedding is an effective technology to decrease the noise information of data. Inspried by LMNN and embedding technology, we propose a novel PML paradigm called Partial Multi-label Learning via Large Margin Nearest Neighbour Embeddings (PML-LMNNE), which aims to conduct disambiguation by projecting labels and features into a lower-dimension embedding space and reorganize the underlying structure by LMNN in the embedding space simultaneously. An efficient algorithm is designed to implement the proposed method and the convergence rate of the algorithm is analyzed. Moreover, we present a theoretical analysis of the generalization error bound for the proposed PML-LMNNE, which shows that the generalization error converges to the sum of two times the Bayes error over the labels when the number of instances goes to infinity. Comprehensive experiments on artificial and real-world datasets demonstrate the superiorities of the proposed PML-LMNNE.

A Fast Time-Stepping Strategy for Dynamical Systems Equipped with a Surrogate Model

Simulation of complex dynamical systems arising in many applications is computationally challenging due to their size and complexity. Model order reduction, machine learning, and other types of surrogate modeling techniques offer cheaper and simpler ways to describe the dynamics of these systems, but are inexact and introduce additional approximation errors. In order to overcome the computational difficulties of the full complex models, on one hand, and the limitations of surrogate models, on the other, this work proposes a new accelerated time-stepping strategy that combines information from both. This approach is based on the multirate infinitesimal general-structure additive Runge-Kutta (MRI-GARK) framework. The inexpensive surrogate model is integrated with a small timestep to guide the solution trajectory, and the full model is treated with a large timestep to occasionally correct for the surrogate model error and ensure convergence. We provide a theoretical error analysis, and several numerical experiments, to show that this approach can be significantly more efficient than using only the full or only the surrogate model for the integration.

One-Shot Distributed Generalized Eigenvalue Problem (DGEP): Concept, Algorithm and Experiments

This paper focuses on the design of a distributed algorithm for generalized eigenvalue problems (GEPs) in one-shot communication. Since existing distributed methods for eigenvalue decomposition cannot be applied to GEP, a general one-shot distributed GEP framework is proposed. The theoretical analysis of the approximation error reveals its relation to the divergence of the data covariance, the eigenvalues of the empirical data covariance, and the number of local servers. If the symmetric data covariance has repeated eigenvalues in GEP, e.g., in canonical component analysis, we further modify the method for better convergence and prove the necessity experimentally. Numerical experiments validate the effectiveness of the proposed algorithms both on synthetic and real-world datasets.

Tensor CUR Decomposition under T-Product and Its Perturbation

In order to process the large-scale data, a useful tool in dimensionality reduction of a matrix, the CUR decomposition has been developed, which can compress the huge matrix with its original elements. Tensor-tensor decompositions have become prevalent and a new multiplication of a tensor based on the T-product has been presented for the tensor computation. Using the T-product, we propose a dimensionality reduction tool of three-order tensor called the T-product CUR decomposition (t-CUR decomposition for short) and analyze its stability of the perturbation. The t-CUR decomposition can reduce the size of a large-scale tensor with its original entries, its perturbation error bound is refined in the first order of the noise tensor under the spectrum norm. Numerical tests are provided to verify the results of our theoretical error analysis as well.

Augmented direct learning for conditional average treatment effect estimation with double robustness

Inferring the heterogeneous treatment effect is a fundamental problem in many applications. In this paper, we focus on estimating the Conditional Average Treatment Effect (CATE), that is, the difference in the conditional mean outcome between treatments given covariates. Traditionally, Q-Learning based approaches estimate each conditional mean outcome. However, they are subject to model misspecification. Recently, flexible one-step methods to directly learn (D-Learning) the CATE without outcome model specifications have been proposed. However, they require a specification of the propensity score. We propose robust direct learning (RD-Learning), to augment D-learning, leading to doubly robust estimators of the treatment effect. The consistency for our CATE estimator is guaranteed if either the main effect model or the propensity score model is correctly specified. The framework can be used in both the binary and the multi-arm settings and is general enough to allow different function spaces and incorporate different generic learning algorithms. We conduct a thorough theoretical analysis of the prediction error of our CATE estimator using statistical learning theory under both linear and non-linear settings. The effectiveness of our proposed method is demonstrated by simulation studies and a real data example about an AIDS Clinical Trials study.

Generalized Large Margin $k$NN for Partial Label Learning

To deal with noises in partial label learning (PLL), existing approaches try to perform disambiguation either by identifying the ground-truth label or by averaging the candidate labels. However, these methods can be easily misled by the false-positive noisy labels in the candidate set, and fail to generalize well in testing. When labeling information is ambiguous, learning paradigms should depend more on underlying data structure. Large margin nearest neighbour (LMNN) is a popular strategy to consider instance and class correlations in supervised learning, but can not be directly used in weakly-supervised PLL due to the ambiguity of labeling information. In this paper, we first define similarly and differently labeled pairs as well as the similarity weight to evaluate the similarties between any two instances. We then propose a novel PLL method called Generalized Large Margin <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> NN for Partial Label Learning (GLMNN-PLL), which adapts the framework of LMNN to PLL by modifying the constraint from ‘the same class’ to ‘similarly-labeled’. GLMNN-PLL aims to learn a new metric and perform disambiguation by reorganizing the underlying data structure, that is, making similarly labeled instances closer to each other while making differently labeled instances seperated by a large margin. As two close instances with shared labels do not necessarily belong to the same class, we put a weight on each instance pair. An efficient algorithm is designed to optimize the proposed method and the convergence is analyzed in this paper. Moreover, we present a theoretical analysis of the generalization error bound for GLMNN-PLL. Comprehensive experiments on controlled UCI datasets as well as real-world partial label datasets from various domains demonstrate the superiorities of the proposed method.

Parallel 3D ADI Scheme for Partially Dimension Reduced Heat Conduction Problem

This model describes the heat equation in 3D domains, and this problem is reduced to a hybrid dimension problem, keeping the initial dimension only in some parts and reducing it to one-dimensional equation within the domains in some distance from the base regions. Such mathematical models are typical in industrial installations such as pipelines. Our aim is to add two additional improvements into this methodology. First, the economical ADI type finite volume scheme is constructed to solve the non-classical heat conduction problem. Special interface conditions are defined between 3D and 1D parts. It is proved that the ADI scheme is unconditionally stable. Second, the parallel factorization algorithm is proposed to solve the obtained systems of discrete equations. Due to both modifications the run-time of computations is reduced essentially. Results of computational experiments confirm the theoretical error analysis and scalability estimates of the parallel algorithm.

An Element-Free Galerkin Method for Time-Fractional Diffusion-Wave Equations

Numerical solution and theoretical error analysis of the element-free Galerkin (EFG) method were presented for the time-fractional diffusion-wave equations in the sense of Caputo. Through discretization of the time variables in the equation with the L1 approximate formula, the time-fractional diffusion-wave equation was transformed into a series of time-independent integer-order differential equations. Then, the penalty method was used to deal with the Dirichlet boundary condition and the EFG method was used to discretize the integer-order differential equations. Error estimates of the EFG method for the time-fractional diffusion-wave equations were derived theoretically. Finally, several numerical examples show the accuracy and effectiveness of the proposed meshless method.