Sparse Operators Research Articles

STile: Searching Hybrid Sparse Formats for Sparse Deep Learning Operators Automatically

Sparse operators, i.e., operators that take sparse tensors as input, are of great importance in deep learning models. Due to the diverse sparsity patterns in different sparse tensors, it is challenging to optimize sparse operators by seeking an optimal sparse format, i.e., leading to the lowest operator latency. Existing works propose to decompose a sparse tensor into several parts and search for a hybrid of sparse formats to handle diverse sparse patterns. However, they often make a trade-off between search space and search time: their search spaces are limited in some cases, resulting in limited operator running efficiency they can achieve. In this paper, we try to extend the search space in its breadth (by doing flexible sparse tensor transformations) and depth (by enabling multi-level decomposition). We formally define the multi-level sparse format decomposition problem, which is NP-hard, and we propose a framework STile for it. To search efficiently, a greedy algorithm is used, which is guided by a cost model about the latency of computing a sub-task of the original operator after decomposing the sparse tensor. Experiments of two common kinds of sparse operators, SpMM and SDDMM, are conducted on various sparsity patterns, and we achieve 2.1-18.0× speedup against cuSPARSE on SpMMs and 1.5 - 6.9× speedup against DGL on SDDMM. The search time is less than one hour for any tested sparse operator, which can be amortized.

Weighted Strong-Type Estimates on Classical Lorentz Spaces

We present new estimates in the setting of weighted classical Lorentz spaces for important operators in Harmonic Analysis such as Calderón-Zygmund operators, sparse operators and the Bochner-Riesz operator among others.

A Multi-resolution Approach to 3D Controlled-source Electromagnetic Modeling

We have introduced a multi-resolution sampling approach for 3D finite-difference controlled-source electromagnetic forward modeling. Inside this scheme, the discrete operators are constructed based on the original sampling points. At the same time, the algorithm reduces the number of discrete electromagnetic fields instead of the number of cells. To avoid the deterioration of the condition number of the coefficient matrix caused by multiplying by different levels of sparse operators, we introduce a novel technique to optimize the condition number of the coefficient matrix. The results show that the new approach improves computational efficiency without compromising the accuracy of the solution.

Challenges of high-fidelity virtual reactor for exascale computing and research progress of China Virtual Reactor

The virtual reactor is a sophisticated software system deployed on supercomputers to simulate physics process or predict component behaviors inside real reactors. It highly relies on advanced modeling and coupling methods, high performance computing and advanced verification and validation (V&V) techniques, which is another important method for reactor engineering design and mechanism research besides experiment. With the advent of exascale computer, it becomes mainstream trend to develop full-core high-fidelity simulation research under the help of exascale computing technology. This paper first summarizes the challenges of developing high-fidelity virtual reactor simulation in exascale computing. The challenges of high-fidelity virtual reactor simulation are introduced from four aspects: science-based modeling methods, high-resolution mesh model, scientific numerical methods , and multiphysics and multiscale coupling. Based on the architecture of exascale computer, the challenges of parallel optimization of virtual reactor for exascale computing are introduced. Furthermore, this paper introduces the recent research progress of China Virtual Reactor (CVR), which is about the parallel optimization for Chinese exascale computers and the functional extensions for achieving full-core high-fidelity simulation. The recent research progress of CVR includes multiscale simulation software of materials, fuel performance simulation software, thermal-hydraulic simulation software, full-core high-resolution meshing tool, and optimization of basic sparse operators.

Causal sparse domination of Beurling maximal regularity operators

We prove boundedness of Calderön–Zygmund operators acting in Banach function spaces on domains, defined by the L1 Carleson functional and Lq (1 < q < ∞) Whitney averages. For such bounds to hold, we assume that the operator maps towards the boundary of the domain. We obtain the Carleson estimates by proving a pointwise domination of the operator, by sparse operators with a causal structure. The work is motivated by maximal regularity estimates for elliptic PDEs and is related to one-sided weighted estimates for singular integrals.

Weighted Gagliardo-Nirenberg interpolation inequalities

In this paper, we prove weighted versions of the Gagliardo-Nirenberg interpolation inequality with Riesz as well as Bessel type fractional derivatives, generalizing the celebrated result with classical derivatives. We use a harmonic analysis approach employing several methods, including the method of domination by sparse operators, to obtain such inequalities for a general class of weights satisfying Muckenhoupt-type conditions. We also obtain improved results for some particular families of weights, including power-law weights |x|α. In particular, we prove an inequality which generalizes both the Stein-Weiss inequality and the Caffarelli-Kohn-Nirenberg inequality. However, our approach is sufficiently flexible to allow as well for non-homogeneous weights and we also prove versions of the inequalities with Japanese bracket weights 〈x〉α=(1+|x|2)α2.

Sparse Bounds for Pseudo-multipliers Associated to Grushin Operators, I

In this article, we prove sharp quantitative weighted $$L^p$$ -estimates for Grushin pseudo-multipliers satisfying Hörmander’s condition as an application of pointwise domination of Grushin pseudo-multipliers by appropriate sparse operators.

Off-Diagonal Two Weight Bumps for Fractional Sparse Operators

In this paper, we continue some recent work on two weight boundedness of sparse operators to the “off-diagonal” setting. We use the new “entropy bumps” introduced in by Treil and Volberg and improved by Lacey and Spencer [11] and the “direct comparison bumps” introduced by Rahm and Spencer [23] and improved by Lerner [14]. Our results are “sharp” in the sense that they are sharp in various particular cases. A feature is that given the current machinery and advances, the proofs are almost trivial.

Bump conditions and two-weight inequalities for commutators of fractional integrals

This paper gives new two-weight bump conditions for sparse operators related to iterated commutators of fractional integrals. As applications, two-weight bounds for iterated commutators of fractional integrals under more general bump conditions are obtain

Paraproducts, Bloom BMO and sparse BMO functions

We address L^p(\mu)\to L^p(\lambda) bounds for paraproducts in the Bloom setting. We introduce certain “sparse BMO” functions associated with sparse collections with no infinitely increasing chains, and use these to express sparse operators as sums of paraproducts and martingale transforms – essentially, as Haar multipliers – as well as to obtain an equivalence of norms between sparse operators \mathcal{A}_{\mathcal{S}} and compositions of paraproducts \Pi^*_a\Pi_b .

Zygmund type and flag type maximal functions, and sparse operators

We prove that the maximal functions associated with a Zygmund dilation dyadic structure in three-dimensional Euclidean space, and with the flag dyadic structure in two-dimensional Euclidean space, cannot be bounded by multiparameter sparse operators associated with the corresponding dyadic grid. We also obtain supplementary results about the absence of sparse domination for the strong dyadic maximal function.

Quantitative weighted estimates for Harmonic analysis operators in the Bessel setting by using sparse domination

In this paper we obtain quantitative weighted L^p-inequalities for some operators involving Bessel convolutions. We consider maximal operators, Littlewood-Paley functions and variational operators. We obtain L^p(w)-operator norms in terms of the A_p-characteristic of the weight w. In order to do this we show that the operators under consideration are dominated by a suitable family of sparse operators in the space of homogeneous type ((0,infty ),|cdot |,x^{2lambda }dx).

Weighted estimates for Bochner–Riesz operators on Lorentz spaces

We present new estimates in the setting of weighted Lorentz spaces of operators satisfying a limited Rubio de Francia condition; namely $T$ is bounded on $L^{p}(v)$ for every $v$ in an strictly smaller class of weights than the Muckenhoupt class $A_p$. Important examples will be the Bochner–Riesz operators $BR_\lambda$ with $0&lt;\lambda &lt;{(n-1)}/2$, sparse operators, Hörmander multipliers with a limited regularity condition and rough operators with $\Omega \in L^{r}(\Sigma )$, $1 &lt; r &lt; \infty$.

DCT and DST Filtering With Sparse Graph Operators

Graph filtering is a fundamental tool in graph signal processing. Polynomial graph filters (PGFs), defined as polynomials of a fundamental graph operator, can be implemented in the vertex domain, and usually have a lower complexity than frequency domain filter implementations. In this paper, we focus on the design of filters for graphs with graph Fourier transform (GFT) corresponding to a discrete trigonometric transform (DTT), i.e., one of 8 types of discrete cosine transforms (DCT) and 8 discrete sine transforms (DST). In this case, we show that multiple sparse graph operators can be identified, which allows us to propose a generalization of PGF design: multivariate polynomial graph filter (MPGF). First, for the widely used DCT-II (type-2 DCT), we characterize a set of sparse graph operators that share the DCT-II matrix as their common eigenvector matrix. This set contains the well-known connected line graph. These sparse operators can be viewed as graph filters operating in the DCT domain, which allows us to approximate any DCT graph filter by a MPGF, leading to a design with more degrees of freedom than the conventional PGF approach. Then, we extend those results to all of the 16 DTTs as well as their 2D versions, and show how their associated sets of multiple graph operators can be determined. We demonstrate experimentally that ideal low-pass and exponential DCT/DST filters can be approximated with higher accuracy with similar runtime complexity. Finally, we apply our method to transform-type selection in a video codec, AV1, where we demonstrate significant encoding time savings, with a negligible compression loss.

Sawyer-type inequalities for Lorentz spaces

The Hardy-Littlewood maximal operator M satisfies the classical Sawyer-type estimate MfvL1,∞(uv)≤Cu,v‖f‖L1(u),\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\begin{aligned} \\left\\| \\frac{Mf}{v}\\right\\| _{L^{1,\\infty }(uv)} \\le C_{u,v} \\Vert f \\Vert _{L^{1}(u)}, \\end{aligned}$$\\end{document}where uin A_1 and uvin A_{infty }. We prove a novel extension of this result to the general restricted weak type case. That is, for p>1, uin A_p^{{mathcal {R}}}, and uv^p in A_infty , MfvLp,∞(uvp)≤Cu,v‖f‖Lp,1(u).\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\begin{aligned} \\left\\| \\frac{Mf}{v}\\right\\| _{L^{p,\\infty }(uv^p)} \\le C_{u,v} \\Vert f \\Vert _{L^{p,1}(u)}. \\end{aligned}$$\\end{document}From these estimates, we deduce new weighted restricted weak type bounds and Sawyer-type inequalities for the m-fold product of Hardy-Littlewood maximal operators. We also present an innovative technique that allows us to transfer such estimates to a large class of multi-variable operators, including m-linear Calderón-Zygmund operators, avoiding the A_infty extrapolation theorem and producing many estimates that have not appeared in the literature before. In particular, we obtain a new characterization of A_p^{{mathcal {R}}}. Furthermore, we introduce the class of weights that characterizes the restricted weak type bounds for the multi(sub)linear maximal operator {mathcal {M}}, denoted by A_{mathbf {P}}^{{mathcal {R}}}, establish analogous bounds for sparse operators and m-linear Calderón-Zygmund operators, and study the corresponding multi-variable Sawyer-type inequalities for such operators and weights. Our results combine mixed restricted weak type norm inequalities, A_p^{{mathcal {R}}} and A_{mathbf {P}}^{{mathcal {R}}} weights, and Lorentz spaces.

Weighted inequalities for q-functions

Let f be a martingale on an arbitrary atomic probability space equipped with a tree-like structure and let S(f,q) denote the associated q-function. The paper is devoted to weighted Lp-estimates, cp,q,w−1‖S(f,q)‖ Lp(w)≤‖f‖Lp(w)≤Cp,q,w‖S(f,q)‖Lp(w),1≤p<∞, for Muckenhoupt weights. Using the combination of the theory of sparse operators, extrapolation, and Bellman function method, we identify the optimal dependence of the constants cp,q,w and Cp,q,w on the Ap characteristics of the weights involved.

Boundedness of Sparse and Rough Operators on Weighted Lorentz Spaces

We present new estimates in the setting of weighted Lorentz spaces for important operators in Harmonic Analysis such as sparse operators, Bochner–Riesz at the critical index, Hormander multipliers and rough singular integrals among others.

Data-Driven Corrections of Partial Lotka-Volterra Models.

In many applications of interacting systems, we are only interested in the dynamic behavior of a subset of all possible active species. For example, this is true in combustion models (many transient chemical species are not of interest in a given reaction) and in epidemiological models (only certain subpopulations are consequential). Thus, it is common to use greatly reduced or partial models in which only the interactions among the species of interest are known. In this work, we explore the use of an embedded, sparse, and data-driven discrepancy operator to augment these partial interaction models. Preliminary results show that the model error caused by severe reductions—e.g., elimination of hundreds of terms—can be captured with sparse operators, built with only a small fraction of that number. The operator is embedded within the differential equations of the model, which allows the action of the operator to be interpretable. Moreover, it is constrained by available physical information and calibrated over many scenarios. These qualities of the discrepancy model—interpretability, physical consistency, and robustness to different scenarios—are intended to support reliable predictions under extrapolative conditions.

Global regularity estimates for non-divergence elliptic equations on weighted variable Lebesgue spaces

This paper is to prove global regularity estimates for solutions to the second-order elliptic equation in non-divergence form with BMO coefficients in a [Formula: see text] domain on weighted variable exponent Lebesgue spaces. Our approach is based on the representations for the solutions to the non-divergence elliptic equations and the domination technique by sparse operators in harmonic analysis.

Sharp off-diagonal weighted weak type estimates for sparse operators

We prove sharp weak type weighted estimates for a class of sparse operators that includes majorants of standard $\alpha$-fractional singular integrals, fractional integral operators, Marcinkiewicz integral operators, and square functions. These bounds are knows to be sharp in many cases, and our main new result is the optimal bound $$[w]_{A_{p,q}}^{\frac{1}{q}}[w^{q}]_{A_{\infty}}^{\frac{1}{2}-\frac{1}{p}}\lesssim[w]_{A_{p,q}}^{\frac{1}{2}-\frac{\alpha}{d}}$$ for proper conditions which satisfy that three index $p$, $q$ and $\alpha$ ensure weak type norm of fractional square functions on $L^{q}(w^{q})$ with $p>2$.