General Smoothness Assumptions Research Articles

Designing truncated priors for direct and inverse Bayesian problems

The Bayesian approach to inverse problems with functional unknowns, has received significant attention in recent years. An important component of the developing theory is the study of the asymptotic performance of the posterior distribution in the frequentist setting. The present paper contributes to the area of Bayesian inverse problems by formulating a posterior contraction theory for linear inverse problems, with truncated Gaussian series priors, and under general smoothness assumptions. Emphasis is on the intrinsic role of the truncation point both for the direct as well as for the inverse problem, which are related through the modulus of continuity as this was recently highlighted by Knapik and Salomond (2018).

Homogenization of elliptic systems with stratified structure revisited

We consider quantitative estimates in periodic homogenization for second-order elliptic systems with stratified structure on bounded Lipschitz domains. Under rather general smoothness assumptions on A and ρ, we establish a sharp-order scale-invariant convergence rate in As a byproduct, a qualitative homogenization result is also derived under more or less sharp conditions on A and ρ. The results obtained here improve our previous results greatly.

Regularized Nyström subsampling in regression and ranking problems under general smoothness assumptions

In the supervised learning, the Nyström type subsampling is considered as a tool for reducing the computational complexity of regularized kernel methods in the big data setting. Up to now, the theoretical analysis of this approach has been done almost exclusively in the context of the regression learning and in the case where the smoothness of the target functions is restricted to the Hölder type source conditions. Such conditions do not cover the case of target functions with high and low smoothness, which are also of practical interest. Moreover, in the case of the Hölder source conditions, there is no need to consider a regularization with high enough qualification because order-optimal learning rates are achieved by the simple Tikhonov regularization known also as the kernel ridge regression. At the same time, this learning method does not improve its performance for any smoothness higher than Hölder ones. Therefore, in this paper, our goal is to extend previous analysis of the Nyström type subsampling to the case of the general source conditions, and to the regularization schemes with high enough qualification. We also show that under rather natural assumption, our results can be easily reformulated in the ranking learning setting.

Sparse Approximation of 3D Meshes Using the Spectral Geometry of the Hamiltonian Operator

The discrete Laplace operator is ubiquitous in spectral shape analysis, since its eigenfunctions are provably optimal in representing smooth functions defined on the surface of the shape. Indeed, subspaces defined by its eigenfunctions have been utilized for shape compression, treating the coordinates as smooth functions defined on the given surface. However, surfaces of shapes in nature often contain geometric structures for which the general smoothness assumption may fail to hold. At the other end, some explicit mesh compression algorithms utilize the order by which vertices that represent the surface are traversed, a property which has been ignored in spectral approaches. Here, we incorporate the order of vertices into an operator that defines a novel spectral domain. We propose a method for representing 3D meshes using the spectral geometry of the Hamiltonian operator, integrated within a sparse approximation framework. We adapt the concept of a potential function from quantum physics and incorporate vertex ordering information into the potential, yielding a novel data-dependent operator. The potential function modifies the spectral geometry of the Laplacian to focus on regions with finer details of the given surface. By sparsely encoding the geometry of the shape using the proposed data-dependent basis, we improve compression performance compared to previous results that use the standard Laplacian basis and spectral graph wavelets.

Balancing principle in supervised learning for a general regularization scheme

We discuss the problem of parameter choice in learning algorithms generated by a general regularization scheme. Such a scheme covers well-known algorithms as regularized least squares and gradient descent learning. It is known that in contrast to classical deterministic regularization methods, the performance of regularized learning algorithms is influenced not only by the smoothness of a target function, but also by the capacity of a space, where regularization is performed. In the infinite dimensional case the latter one is usually measured in terms of the effective dimension. In the context of supervised learning both the smoothness and effective dimension are intrinsically unknown a priori . Therefore we are interested in a posteriori regularization parameter choice, and we propose a new form of the balancing principle. An advantage of this strategy over the known rules such as cross-validation based adaptation is that it does not require any data splitting and allows the use of all available labeled data in the construction of regularized approximants. We provide the analysis of the proposed rule and demonstrate its advantage in simulations. • For smooth kernels, a power decay of effective dimensions may not be appropriate. • Learning rates under general smoothness assumptions on target functions. • Optimal learning rates in all involved norms based on a novel bound. • An adaptive parameter choice which uses the whole dataset without data splitting.

Multi-penalty regularization in learning theory

In this paper we establish the error estimates for multi-penalty regularization under the general smoothness assumption in the context of learning theory. One of the motivation for this work is to study the convergence analysis of two-parameter regularization theoretically in the manifold learning setting. In this spirit, we obtain the error bounds for the manifold learning problem using more general framework of multi-penalty regularization. We propose a new parameter choice rule “the balanced-discrepancy principle” and analyze the convergence of the scheme with the help of estimated error bounds. We show that multi-penalty regularization with the proposed parameter choice exhibits the convergence rates similar to single-penalty regularization. Finally on a series of test samples we demonstrate the superiority of multi-parameter regularization over single-penalty regularization.

Discrepancy principle for statistical inverse problems with application to conjugate gradient iteration**Dedicated to Ulrich Tautenhahn, a friend and co-author, who passed away too early at the age of 60.

The authors discuss the use of the discrepancy principle for statistical inverse problems, when the underlying operator is of trace class. Under this assumption the discrepancy principle is well defined, however a plain use of it may occasionally fail and it will yield sub-optimal rates. Therefore, a modification of the discrepancy is introduced, which corrects both of the above deficiencies. For a variety of linear regularization schemes as well as for conjugate gradient iteration it is shown to yield order optimal a priori error bounds under general smoothness assumptions. A posteriori error control is also possible, however at a sub-optimal rate, in general. This study uses and complements previous results for bounded deterministic noise.

An analysis of Tikhonov regularization for nonlinear ill-posed problems under a general smoothness assumption

In this paper we introduce an adaptive regularization scheme based on algorithms for minimization of the Tikhonov functional to reconstruct the solution x† of nonlinear ill-posed problem F(x) = y, where the right-hand side is replaced by noisy data yδ ∊ Y with ‖y − yδ‖ ⩽ δ, and F : D(F) ⊂ X → Y is a nonlinear operator between Hilbert spaces X and Y. Under the assumption that Fréchet derivative F′ is Lipschitz continuous, a choice of the regularization parameter and the stopping criteria for minimization algorithms are presented. We prove that under a general source condition given in terms of a nonlinear operator F the error ‖x† − xk‖ between the regularized approximation xk and the solution x† is of optimal order.

On the asymptotic equivalence and rate of convergence of nonparametric regression and Gaussian white noise

The experiments of nonparametric regression with equidistant design points and Gaussian white noise are considered. Brown and Low have proven asymptotic equivalence of these models under a quite general smoothness assumption on the parameter space of regression functions. In the present paper we focus on periodic Sobolev classes. We prove asymptotic equivalence of nonparametric regression and white noise with a construction different to Brown and Low. Whereas their original method cannot give a better rate than n−1/2 for the smoothness classes under consideration, even if the underlying function class is actually smoother than just Lipschitz, in the present work a rate of convergence n−β+1/2 for the delta-distance over a Sobolev class with any smoothness index β > 1/2 is derived. Furthermore, the results are constructive and therefore lead to a simple transfer of decision procedures.

Change-Points in Nonparametric Regression Analysis

an otherwise smooth regression model are proposed. The assumptions needed are much weaker than those made in parametric models. The proposed estimators apply as well to the detection of discontinuities in derivatives and therefore to the detection of change-points of slope and of higher order curvature. The proposed estimators are based on a comparison of left and right one-sided kernel smoothers. Weak convergence of a stochastic process in local differences to a Gaussian process is established for properly scaled versions of estimators of the location of a change-point. The continuous mapping theorem can then be invoked to obtain asymptotic distributions and corresponding rates of convergence for change-point estimators. These rates are typically faster than n- 1/2. Rates of global LP convergence of curve estimates with appropriate kernel modifications adapting to estimated change-points are derived as a consequence. It is shown that these rates of convergence are the same as if the location of the change-point was known. The methods are illustrated by means of the well known data on the annual flow volume of the Nile river between 1871 and 1970. 1. Introduction. Nonparametric regression methods are usually applied in order to obtain a smooth fit of a regression curve without having to specify a parametric class of regression functions. Sometimes a generally smooth curve might contain an isolated discontinuity or change-point in the curve or in a (possibly higher order) derivative, and in many cases interest focuses on the occurrence of such change-points. In parametric approaches to the regression change-point problem, simple linear regressions before and after a possible change-point are assumed, and then the possibility of a discontinuity in the form of a jump or of a jump in the first derivative, or, equivalently, a slope change, is incorporated into the model; see for instance Hinkley (1969) and Brown, Durbin and Evans (1975). The analysis of change-points which describe sudden, localized changes typically occurring in economics, medicine and the physical sciences has recently found increasing interest. General smoothness assumptions, allowing for a large class of regression functions to be considered, seem to be more appropriate in a variety of applied problems than parametric modelling. An

Applying temporal constraints to the dynamic stereo problem

An algorithm is presented for the problem of the stereopsis of time-varuing images (the dynamic stereo problem). Dynamic stereopsis is the integration of two problems; static stereopsis and temporal correspondence. Rather than finding the intersection of these problems to be more difficult, it was found that by solving the two problem simultaneously, and thus incorporating the spatio-temporal context within which a scene exists, some of the hard subproblems belonging to stereopsis and temporal correspondence could be avoided. The algorithm relies on a general smoothness assumption to assign both disparity and temporal matches. A simple model of the motion of three-dimensional features is used to guide the matching process and to identify conditional matches which violate a general smoothness assumption. A spatial proximity rule is used to further restrict possible matches. The algorithm has been tested on both synthetic and real input sequences. Input sequences were chosen from three-dimensional moving light displays and from “real” grey-level digitized images.

Tracking three dimensional moving light displays (abstract only)

A method is presented for tracking the three-dimensional motion of points from their changing two-dimensional perspective images as viewed by a nonconvergent binocular vision system. The algorithm relies on a general smoothness assumption to guide the tracking process, and application of the tracking algorithm to a three-dimensional moving light display based on Cutting's Walker program as well as other domains are discussed.Evidence is presented relating the tracking algorithm to certain beliefs about neurophysiological structures in the visual cortex.