Christoffel Function Research Articles

Leveraging the Christoffel function for outlier detection in data streams

Leveraging the Christoffel function for outlier detection in data streams

The Christoffel function: Applications, connections and extensions

The Christoffel function: Applications, connections and extensions

Asymptotics for Christoffel functions associated to continuum Schrödinger operators

We prove asymptotics of the Christoffel function, λL(ξ), of a continuum Schrödinger operator for points in the interior of the essential spectrum under some mild conditions on the spectral measure. It is shown that LλL(ξ) has a limit and that this limit is given by the Radon–Nikodym derivative of the spectral measure with respect to the Martin measure. Combining this with a recently developed local criterion for universality limits at scale λL(ξ), we compute universality limits for continuum Schrödinger operators at scale L and obtain clock spacing of the eigenvalues of the finite range truncations.

Data-Driven Reachability and Support Estimation With Christoffel Functions

We present algorithms for estimating the forward reachable set of a dynamical system using only a finite collection of independent and identically distributed samples. The produced estimate is the sublevel set of a function called an empirical inverse Christoffel function: empirical inverse Christoffel functions are known to provide good approximations to the support of probability distributions. In addition to reachability analysis, the same approach can be applied to general problems of estimating the support of a random variable, which has applications in data science towards detection of novelties and outliers in data sets. In applications where safety is a concern, having a guarantee of accuracy that holds on finite data sets is critical. In this paper, we prove such bounds for our algorithms under the Probably Approximately Correct (PAC) framework. In addition to applying classical Vapnik- Chervonenkis (VC) dimension bound arguments, we apply the PAC-Bayes theorem by leveraging a formal connection between kernelized empirical inverse Christoffel functions and Gaussian process regression models.

A modified Christoffel function and its asymptotic properties

We introduce a certain variant (or regularization) Λ̃nμ of the standard Christoffel function Λnμ associated with a measure μ on a compact set Ω⊂Rd. Its reciprocal is now a sum-of-squares polynomial in the variables (x,ɛ), ɛ>0. It shares the same dichotomy property of the standard Christoffel function, that is, the growth with n of its inverse is at most polynomial inside and exponential outside the support of the measure. Its distinguishing and crucial feature states that for fixed ɛ>0, and under weak assumptions, limn→∞ɛ−dΛ̃nμ(ξ,ɛ)=f(ζɛ) where f (assumed to be continuous) is the unknown density of μ w.r.t. Lebesgue measure on Ω, and ζɛ∈B∞(ξ,ɛ) (and so f(ζɛ)≈f(ξ) when ɛ>0 is small). This is in contrast with the standard Christoffel function where if limn→∞ndΛnμ(ξ) exists, it is of the form f(ξ)/ωE(ξ) where ωE is the density of the equilibrium measure of Ω, usually unknown. At last but not least, the additional computational burden (when compared to computing Λnμ) is just integrating symbolically the monomial basis (xα)α∈Nnd on the box {x:‖x−ξ‖∞<ɛ/2}, so that 1/Λ̃nμ is obtained as an explicit polynomial of (ξ,ɛ).

CAS4DL: Christoffel adaptive sampling for function approximation via deep learning

The problem of approximating smooth, multivariate functions from sample points arises in many applications in scientific computing, e.g., in computational Uncertainty Quantification (UQ) for science and engineering. In these applications, the target function may represent a desired quantity of interest of a parameterized Partial Differential Equation (PDE). Due to the large cost of solving such problems, where each sample is computed by solving a PDE, sample efficiency is a key concern in these applications. Recently, there has been increasing focus on the use of Deep Neural Networks (DNN) and Deep Learning (DL) for learning such functions from data. In this work, we propose an adaptive sampling strategy, CAS4DL (Christoffel Adaptive Sampling for Deep Learning) to increase the sample efficiency of DL for multivariate function approximation. Our novel approach is based on interpreting the second to last layer of a DNN as a dictionary of functions defined by the nodes on that layer. With this viewpoint, we then define an adaptive sampling strategy motivated by adaptive sampling schemes recently proposed for linear approximation schemes, wherein samples are drawn randomly with respect to the Christoffel function of the subspace spanned by this dictionary. We present numerical experiments comparing CAS4DL with standard Monte Carlo (MC) sampling. Our results demonstrate that CAS4DL often yields substantial savings in the number of samples required to achieve a given accuracy, particularly in the case of smooth activation functions, and it shows a better stability in comparison to MC. These results therefore are a promising step towards fully adapting DL towards scientific computing applications.

A disintegration of the Christoffel function

We show that the Christoffel function (CF) factorizes (or can be disintegrated) as the product of two Christoffel functions, one associated with the marginal and the another related to the conditional distribution, in the spirit of “the CF of the disintegration is the disintegration of the CFs”. In the proof one uses an apparently overlooked property (but interesting in its own) which states that any sum-of-squares polynomial is the Christoffel function of some linear form (with a representing measure in the univariate case). The same is true for the convex cone of polynomials that are positive on a basic semi-algebraic set. This interpretation of the CF establishes another bridge between polynomials optimization and orthogonal polynomials.

A Note on Sampling Recovery of Multivariate Functions in the Uniform Norm

We study the recovery of multivariate functions from reproducing kernel Hilbert spaces in the uniform norm. Our main interest is to obtain preasymptotic estimates for the corresponding sampling numbers. We obtain results in terms of the decay of related singular numbers of the compact embedding into $L_2(D,\varrho_D)$ multiplied with the supremum of the Christoffel function of the subspace spanned by the first $m$ singular functions. Here the measure $\varrho_D$ is at our disposal. As an application we obtain near optimal upper bounds for the sampling numbers for periodic Sobolev type spaces with general smoothness weight. Those can be bounded in terms of the corresponding benchmark approximation number in the uniform norm, which allows for preasymptotic bounds. By applying a recently introduced sub-sampling technique related to Weaver's conjecture we mostly lose a $\sqrt{\log n}$ and sometimes even less. Finally we point out a relation to the corresponding Kolmogorov numbers.

Rate of convergence for geometric inference based on the empirical Christoffel function

We consider the problem of estimating the support of a measure from a finite, independent, sample. The estimators which are considered are constructed based on the empirical Christoffel function. Such estimators have been proposed for the problem of set estimation with heuristic justifications. We carry out a detailed finite sample analysis, that allows us to select the threshold and degree parameters as a function of the sample size. We provide a convergence rate analysis of the resulting support estimation procedure. Our analysis establishes that we may obtain finite sample bounds which are comparable to existing rates for different set estimation procedures. Our results rely on concentration inequalities for the empirical Christoffel function and on estimates of the supremum of the Christoffel-Darboux kernel on sets with smooth boundaries, that can be considered of independent interest.

Optimal Sampling and Christoffel Functions on General Domains

We consider the problem of reconstructing an unknown function \(u\in L^2(D,\mu )\) from its evaluations at given sampling points \(x^1,\dots ,x^m\in D\), where \(D\subset {\mathbb {R}}^d\) is a general domain and \(\mu \) a probability measure. The approximation is picked from a linear space \(V_n\) of interest where \(n=\dim (V_n)\). Recent results (Cohen and Migliorati in SMAI J Comput Math 3:181–203, 2017, Doostan and Hampton in Comput Methods Appl Mech Eng 290:73–97, 2015, Jakeman et al. in Math Comput 86:1913–1947, 2017) have revealed that certain weighted least-squares methods achieve near best (or instance optimal) approximation with a sampling budget m that is proportional to n, up to a logarithmic factor \(\ln (2n/\varepsilon )\), where \(\varepsilon >0\) is a probability of failure. The sampling points should be picked at random according to a well-chosen probability measure \(\sigma \) whose density is given by the inverse Christoffel function that depends both on \(V_n\) and \(\mu \). While this approach is greatly facilitated when D and \(\mu \) have tensor product structure, it becomes problematic for domains D with arbitrary geometry since the optimal measure depends on an orthonormal basis of \(V_n\) in \(L^2(D,\mu )\) which is not explicitly given, even for simple polynomial spaces. Therefore, sampling according to this measure is not practically feasible. One computational solution recently proposed in Adcock and Huybrechs (Approximating smooth, multivariate functions on irregular domains, forum of mathematics, sigma, Cambridge University Press, Cambridge, 2020) relies on using the restrictions of an orthonormal basis of \(V_n\) defined on a simpler bounding domain and sampling according to the original probability measure \(\mu \), in turn giving up on the optimal sampling budget \(m\sim n\). In this paper, we discuss practical sampling strategies, which amounts to using a perturbed measure \(\widetilde{\sigma }\) that can be computed in an offline stage, not involving the measurement of u, as recently proposed in Adcock and Cardenas (SIAM J Math Data Sci 2:607–630, 2020) and Migliorati (IMA J Numer Anal, 2020. https://doi.org/10.1093/imanum/draa023). We show that near best approximation is attained by the resulting weighted least-squares method at near-optimal sampling budget and we discuss multilevel approaches that preserve optimality of the cumulated sampling budget when the spaces \(V_n\) are iteratively enriched. These strategies rely on the knowledge of a-priori upper bounds B(n) on the inverse Christoffel function for the space \(V_n\) and the domain D. We establish bounds of the form \(\mathcal O(n^r)\) for spaces \(V_n\) of multivariate algebraic polynomials of given total degree, and for general domains D. The exact growth rate r depends on the regularity of the domain, in particular \(r=2\) for domains with Lipschitz boundaries and \(r=\frac{d+1}{d}\) for smooth domains.

Geometric computation of Christoffel functions on planar convex domains

For an arbitrary planar convex domain, we compute the behavior of Christoffel function up to a constant factor using comparison with other simple reference domains. The lower bound is obtained by constructing an appropriate ellipse contained in the domain, while for the upper bound an appropriate parallelogram containing the domain is constructed.As an application we obtain a new proof that every planar convex domain possesses optimal polynomial meshes.

Designs in Finite Metric Spaces: A Probabilistic Approach

A finite metric space is called here distance degree regular if its distance degree sequence is the same for every vertex. A notion of designs in such spaces is introduced that generalizes that of designs in Q-polynomial distance-regular graphs. An approximation of their cumulative distribution function, based on the notion of Christoffel function in approximation theory is given. As an application we derive limit laws on the weight distributions of binary orthogonal arrays of strength going to infinity. An analogous result for combinatorial designs of strength going to infinity is given.

The moment-SOS hierarchy and the Christoffel–Darboux kernel

We consider the global minimization of a polynomial on a compact set \(\mathbf {B}\). We show that each step of the Moment-SOS hierarchy has a nice and simple interpretation that complements the usual one. Namely, it computes coefficients of a polynomial in an orthonormal basis of \(L^2(\mathbf {B},\mu )\) where \(\mu \) is an arbitrary reference measure whose support is exactly \(\mathbf {B}\). The resulting polynomial is a certain density (with respect to \(\mu \)) of some signed measure on \(\mathbf {B}\). When some relaxation is exact (which generically takes place) the coefficients of the optimal polynomial density are values of orthonormal polynomials at the global minimizer and the optimal (signed) density is simply related to the Christoffel–Darboux (CD) kernel and the Christoffel function associated with \(\mu \). In contrast to the hierarchy of upper bounds which computes positive densities, the global optimum can be achieved exactly as integration against a polynomial (signed) density because the CD-kernel is a reproducing kernel, and so can mimic a Dirac measure (as long as finitely many moments are concerned).

A dual-type problem to Christoffel function

We study a dual-type problem to generalized Christoffel function. The solution is connected with other extremal problems in the Hp space of analytic functions on the unit circle considered by Macintyre, Rogosinski and Shapiro.

Multivariate approximation of functions on irregular domains by weighted least-squares methods

AbstractWe propose and analyse numerical algorithms based on weighted least squares for the approximation of a bounded real-valued function on a general bounded domain $\varOmega \subset \mathbb{R}^d$. Given any $n$-dimensional approximation space $V_n \subset L^2(\varOmega )$, the analysis in Cohen and Migliorati (2017, Optimal weighted least-squares methods. SMAI J. Comput. Math., 3, 181–203) shows the existence of stable and optimally converging weighted least-squares estimators, using a number of function evaluations $m$ of the order $n \ln n$. When an $L^2(\varOmega )$-orthonormal basis of $V_n$ is available in analytic form, such estimators can be constructed using the algorithms described in Cohen and Migliorati (2017, Optimal weighted least-squares methods. SMAI J. Comput. Math., 3, 181–203, Section 5). If the basis also has product form, then these algorithms have computational complexity linear in $d$ and $m$. In this paper we show that when $\varOmega $ is an irregular domain such that the analytic form of an $L^2(\varOmega )$-orthonormal basis is not available, stable and quasi-optimally weighted least-squares estimators can still be constructed from $V_n$, again with $m$ of the order $n \ln n$, but using a suitable surrogate basis of $V_n$ orthonormal in a discrete sense. The computational cost for the calculation of the surrogate basis depends on the Christoffel function of $\varOmega $ and $V_n$. Numerical results validating our analysis are presented.

Perturbations of Christoffel–Darboux Kernels: Detection of Outliers

Two central objects in constructive approximation, the Christoffel–Darboux kernel and the Christoffel function, encode ample information about the associated moment data and ultimately about the possible generating measures. We develop a multivariate theory of the Christoffel–Darboux kernel in $$\mathbb {C}^d$$ , with emphasis on the perturbation of Christoffel functions and their level sets with respect to perturbations of small norm or low rank. The statistical notion of leverage score provides a quantitative criterion for the detection of outliers in large data. Using the refined theory of Bergman orthogonal polynomials, we illustrate the main results, including some numerical simulations, in the case of finite atomic perturbations of area measure of a 2D region. Methods of function theory of a complex variable and (pluri) potential theory are widely used in the derivation of our perturbation formulas.

Data Analysis from Empirical Moments and the Christoffel Function

Spectral features of the empirical moment matrix constitute a resourceful tool for unveiling properties of a cloud of points, among which, density, support and latent structures. It is already well known that the empirical moment matrix encodes a great deal of subtle attributes of the underlying measure. Starting from this object as base of observations we combine ideas from statistics, real algebraic geometry, orthogonal polynomials and approximation theory for opening new insights relevant for Machine Learning (ML) problems with data supported on singular sets. Refined concepts and results from real algebraic geometry and approximation theory are empowering a simple tool (the empirical moment matrix) for the task of solving non-trivial questions in data analysis. We provide (1) theoretical support, (2) numerical experiments and, (3) connections to real world data as a validation of the stamina of the empirical moment matrix approach.

Asymptotics for recurrence coefficients of X1-Jacobi exceptional polynomials and Christoffel function

ABSTRACTComputing asymptotics of the recurrence coefficients of -Jacobi polynomials, we investigate the limit of the Christoffel function. We also study the relation between the normalized counting measure based on the zeros of the modified average characteristic polynomial and the Christoffel function in limit. The proofs of the corresponding theorems with respect to standard orthogonal polynomials are based on the three-term recurrence relation. The main point is that exceptional orthogonal polynomials possess at least five-term formulae and so Christoffel-Darboux formula also fails. It seems that these difficulties can be handled in a combinatorial way.

Christoffel functions on planar domains with piecewise smooth boundary

Up to a constant factor, we compute the Christoffel function on planar domains with boundary consisting of finitely many C2 curves such that each corner point of the boundary has interior angle strictly between 0 and $$\pi $$ . The resulting formula uses the distances from the point of interest to the curves or certain parts of the curves defining the boundary of the domain.

The empirical Christoffel function with applications in data analysis

We illustrate the potential applications in machine learning of the Christoffel function, or, more precisely, its empirical counterpart associated with a counting measure uniformly supported on a finite set of points. Firstly, we provide a thresholding scheme which allows approximating the support of a measure from a finite subset of its moments with strong asymptotic guaranties. Secondly, we provide a consistency result which relates the empirical Christoffel function and its population counterpart in the limit of large samples. Finally, we illustrate the relevance of our results on simulated and real-world datasets for several applications in statistics and machine learning: (a) density and support estimation from finite samples, (b) outlier and novelty detection, and (c) affine matching.