Tensor Algebra Research Articles

Generalized iterated-sums signatures

We explore the algebraic properties of a generalized version of the iterated-sums signature, inspired by previous work of F. Király and H. Oberhauser. In particular, we show how to recover the character property of the associated linear map on the tensor algebra by considering a deformed quasi-shuffle product of words. We introduce three non-linear transformations on iterated-sums signatures, close in spirit to Machine Learning applications, and show some of their properties.

Mosaic: An Interoperable Compiler for Tensor Algebra

We introduce Mosaic, a sparse tensor algebra compiler that can bind tensor expressions to external functions of other tensor algebra libraries and compilers. Users can extend Mosaic by adding new functions and bind a sub-expression to a function using a scheduling API. Mosaic substitutes the bound sub-expressions with calls to the external functions and automatically generates the remaining code using a default code generator. As the generated code is fused by default, users can productively leverage both fusion and calls to specialized functions within the same compiler. We demonstrate the benefits of our dual approach by showing that calling hand-written CPU and specialized hardware functions can provide speedups of up to 206× against fused code in some cases, while generating fused code can provide speedups of up to 3.57× against code that calls external functions in other cases. Mosaic also offers a search system that can automatically map an expression to a set of registered external functions. Both the explicit binding and automatic search are verified by Mosaic. Additionally, the interface for adding new external functions is simple and general. Currently, 38 external functions have been added to Mosaic, with each addition averaging 20 lines of code.

Indexed Streams: A Formal Intermediate Representation for Fused Contraction Programs

We introduce indexed streams, a formal operational model and intermediate representation that describes the fused execution of a contraction language that encompasses both sparse tensor algebra and relational algebra. We prove that the indexed stream model is correct with respect to a functional semantics. We also develop a compiler for contraction expressions that uses indexed streams as an intermediate representation. The compiler is only 540 lines of code, but we show that its performance can match both the TACO compiler for sparse tensor algebra and the SQLite and DuckDB query processing libraries for relational algebra.

Automatic Generation of Spatial Accelerator for Tensor Algebra

Tensor algebra finds applications in various domains including machine learning applications, data analytics and others. Spatial hardware accelerators are widely used to boost the performance of tensor algebra applications. It has a complex hardware architecture and rich design space. Prior approaches based on manual implementation lead to low programming productivity, making it hard to explore the large design space. In this paper, we propose Tensorlib, a framework for generating spatial hardware accelerators for tensor algebra applications. Tensorlib is motivated by the observation that, tensor dataflows can be expressed with linear transformations, and they share common hardware modules which can be reused across different designs. Tensorlib first uses Space-Time Transformation to explore different dataflows, which can compactly represent the hardware dataflow using a transformation matrix. Next, we identify the common structures of different dataflows and build parameterized hardware module templates. Our generation framework can select the needed hardware modules for each dataflow, connect the modules using a specified interconnection pattern, and automatically generate the complete hardware accelerator design. Tensorlib remarkably improves the productivity for the development and optimization of spatial hardware architecture, providing a rich design space with trade-offs in performance, area, and power. Experiments show that Tensorlib can automatically generate hardware designs with different dataflows for a variety of tensor algebra programs. Tensorlib can achieve 318 MHz frequency and 786 GFLOP/s throughput for matrix multiplication kernel on Xilinx VU9P FPGA, which outperforms the state-of-the-art generators.

SDPN: A Slight Dual-Path Network With Local-Global Attention Guided for Medical Image Segmentation.

Accurate identification of lesions is a key step in surgical planning. However, this task mainly exists two challenges: 1) Due to the complex anatomical shapes of different lesions, most segmentation methods only achieve outstanding performance for a specific structure, rather than other lesions with location differences. 2) The huge number of parameters limits existing transformer-based segmentation models. To overcome these problems, we propose a novel slight dual-path network (SDPN) to segment variable location lesions or organs with significant differences accurately. First, we design a dual-path module to integrate local with global features without obvious memory consumption. Second, a novel Multi-spectrum attention module is proposed to pay further attention to detailed information, which can automatically adapt to the variable segmentation target. Then, the compression module based on tensor ring decomposition is designed to compress convolutional and transformer structures. In the experiment, four datasets, including three benchmark datasets and a clinical dataset, are used to evaluate SDPN. Results of the experiments show that SDPN performs better than other start-of-the-art methods for brain tumor, liver tumor, endometrial tumor and cardiac segmentation. To ensure the generalizability, we train the network on Kvasir-SEG and test on CVC-ClinicDB which collected from a different institution. The quantitative analysis shows that the clinical evaluation results are consistent with the experts. Therefore, this model may be a potential candidate for the segmentation of lesions and organs segmentation with variable locations in clinical applications.

Spatiotemporal traffic data imputation by synergizing low tensor ring rank and nonlocal subspace regularization

AbstractSpatiotemporal traffic data usually suffers from missing entries in the data acquisition and transmission process. Existing imputation methods only consider the global/local structure of spatiotemporal traffic data, resulting in insufficient estimation performance. Fortunately, it is found that traffic data admits the nonlocal self‐similarity (NSS) prior. This paper incorporates the global and nonlocal low‐rank priors of traffic data and proposes a tensor completion model for spatiotemporal traffic data imputation. To be specific, the proposed method uses tensor ring (TR) decomposition with an enhanced representation capability to characterize the global low‐TR‐rank prior of traffic data, e.g. the correlation of sensor and time modes of the tensor (i.e. traffic data). An implicit plug‐and‐play (PnP)‐based regularization is further utilized to exploit the NSS prior, which depicts the nonlocal similar traffic data patterns. Furthermore, the proximal alternating minimization algorithm under the PnP framework is derived to solve this model. The experiment results on various datasets and missing scenarios show the superiority of the proposed model.

Covariant predictions for Planck-scale features in primordial power spectra

In this companion to our letter (arXiv:2208.10514), we elaborate the full details of the predicted corrections to the primordial scalar and tensor power spectra that arise from quantum gravity-motivated, natural, covariant ultraviolet cutoffs. We implement these cutoffs by covariantly restricting the fields which are summed over in the path integrals for the primordial correlators, and we discuss in detail the functional analytic techniques necessary for evaluating such path integrals. Our prediction, which is given in terms of measured cosmological parameters and without assuming any particular inflationary potential, is that the corrections take the form of small oscillations which are superimposed on the conventional power spectra. The frequency of these oscillations only depends on the location of the cutoff scale and the first inflationary slow-roll parameter, while the amplitude and phase are also moderately sensitive to how smoothly the cutoff turns on. The specificity of the new predictions offers an opportunity to significantly enhance experimental sensitivity in observations of the cosmic microwave background and large-scale structure. This may be used to place ever higher bounds on the scale at which quantum gravity effects become important in quantum field theory or may even provide positive evidence for quantum gravity effects.

Incomplete Multiview Clustering via Low-Rank Tensor Ring Completion

Since real-world multiview data frequently contains numerous samples that are not observed from some viewpoints, the incomplete multiview clustering (IMC) issue has received a great deal of attention recently. However, most existing IMC methods choose to zero-fill the missing instances, which leads to the failure to exploit information hidden in the missing instances, and high-order interactions between various views. To tackle these problems, we proposed an effective IMC method using low-rank tensor ring completion, which was demonstrated to be powerful in exploiting high-order correlation. Specifically, we first stack the incomplete similarity graphs of all views into a 3rd-order incomplete tensor and then restore it via the tensor ring decomposition. Next, using an adaptive weighting technique, we apply multiview spectral clustering to all entire graphs in order to balance the contributions of different viewpoints and identify the consensus representation for grouping. Finally, we employ the alternating direction method of multipliers (ADMM) to optimize the suggested model. Numerous experimental findings on numerous different datasets show that the suggested approach is superior to other cutting-edge approaches.

Sgap: towards efficient sparse tensor algebra compilation for GPU

Sparse compiler is a promising solution for sparse tensor algebra optimization. In compiler implementation, reduction in sparse-dense hybrid algebra plays a key role in performance. Though GPU provides various reduction semantics that can better utilize the parallel computing and memory bandwidth capacity, the central question is: how to elevate the flexible reduction semantics to sparse compilation theory that assumes serial execution. Specifically, we have to tackle two main challenges: (1) there are wasted parallelism by adopting static synchronization granularity (2) static reduction strategy limits optimization space exploration. We propose Sgap: s egment g roup and a tomic p arallelism to solve these problems. Atomic parallelism captures the flexible reduction semantics to systematically analyze the optimization space of sparse-dense hybrid algebra on GPU. It is a new optimization technique beyond current compiler-based and open-source runtime libraries. Segment group elevates the flexible reduction semantics to suitable levels of abstraction in the sparse compilation theory. It adopts changeable group size and user-defined reduction strategy to solve challenge (1) and (2), respectively. Finally, we use GPU sparse matrix-matrix multiplication (SpMM) on the TACO compiler as a use case to demonstrate the effectiveness of segment group in reduction semantics elevation. We achieve up to $$1.2\,\times$$ speedup over the original TACO’s SpMM kernels. We also apply new optimization techniques found by atomic parallelism to an open-source state-of-the-art SpMM library dgSPARSE. We achieve $$1.6 \, \times \sim 2.3 \, \times$$ speedup on the algorithm tuned with atomic parallelism.

Fusions of tensor powers of Johnson schemes

This paper is a follow-up to [5], in which the first author studied primitive association schemes lying between a tensor power 𝒯 m d of the trivial association scheme and the Hamming scheme ℋ(d,m). A question which arose naturally in that study was whether all primitive fusions of 𝒯 m d lie between 𝒯 m e d/e and ℋ(d/e,m e ) for some e∣d. This note answers this question positively provided that m is large enough. We similarly classify primitive fusions of the dth tensor power of a Johnson scheme on m k points when m is large enough in terms of k and d.

Graph-Regularized Non-Negative Tensor-Ring Decomposition for Multiway Representation Learning.

Tensor-ring (TR) decomposition is a powerful tool for exploiting the low-rank property of multiway data and has been demonstrated great potential in a variety of important applications. In this article, non-negative TR (NTR) decomposition and graph-regularized NTR (GNTR) decomposition are proposed. The former equips TR decomposition with the ability to learn the parts-based representation by imposing non-negativity on the core tensors, and the latter additionally introduces a graph regularization to the NTR model to capture manifold geometry information from tensor data. Both of the proposed models extend TR decomposition and can be served as powerful representation learning tools for non-negative multiway data. The optimization algorithms based on an accelerated proximal gradient are derived for NTR and GNTR. We also empirically justified that the proposed methods can provide more interpretable and physically meaningful representations. For example, they are able to extract parts-based components with meaningful color and line patterns from objects. Extensive experimental results demonstrated that the proposed methods have better performance than state-of-the-art tensor-based methods in clustering and classification tasks.

Robust to Rank Selection: Low-Rank Sparse Tensor-Ring Completion.

Tensor-ring (TR) decomposition was recently studied and applied for low-rank tensor completion due to its powerful representation ability of high-order tensors. However, most of the existing TR-based methods tend to suffer from deterioration when the selected rank is larger than the true one. To address this issue, this article proposes a new low-rank sparse TR completion method by imposing the Frobenius norm regularization on its latent space. Specifically, we theoretically establish that the proposed method is capable of exploiting the low rankness and Kronecker-basis-representation (KBR)-based sparsity of the target tensor using the Frobenius norm of latent TR-cores. We optimize the proposed TR completion by block coordinate descent (BCD) algorithm and design a modified TR decomposition for the initialization of this algorithm. Extensive experimental results on synthetic data and visual data have demonstrated that the proposed method is able to achieve better results compared to the conventional TR-based completion methods and other state-of-the-art methods and, meanwhile, is quite robust even if the selected TR-rank increases.

Kronecker CP Decomposition With Fast Multiplication for Compressing RNNs.

Recurrent neural networks (RNNs) are powerful in the tasks oriented to sequential data, such as natural language processing and video recognition. However, because the modern RNNs have complex topologies and expensive space/computation complexity, compressing them becomes a hot and promising topic in recent years. Among plenty of compression methods, tensor decomposition, e.g., tensor train (TT), block term (BT), tensor ring (TR), and hierarchical Tucker (HT), appears to be the most amazing approach because a very high compression ratio might be obtained. Nevertheless, none of these tensor decomposition formats can provide both space and computation efficiency. In this article, we consider to compress RNNs based on a novel Kronecker CANDECOMP/PARAFAC (KCP) decomposition, which is derived from Kronecker tensor (KT) decomposition, by proposing two fast algorithms of multiplication between the input and the tensor-decomposed weight. According to our experiments based on UCF11, Youtube Celebrities Face, UCF50, TIMIT, TED-LIUM, and Spiking Heidelberg digits datasets, it can be verified that the proposed KCP-RNNs have a comparable performance of accuracy with those in other tensor-decomposed formats, and even 278 219× compression ratio could be obtained by the low-rank KCP. More importantly, KCP-RNNs are efficient in both space and computation complexity compared with other tensor-decomposed ones. Besides, we find KCP has the best potential of parallel computing to accelerate the calculations in neural networks.

Power spectra of slow-roll inflation in the consistent D → 4 Einstein-Gauss-Bonnet gravity

The slow-roll inflation which took place at extremely high energy regimes is in general believed to be sensitive to the high-order curvature corrections to the classical general relativity (GR). In this paper, we study the effects of the high-order curvature term, the Gauss-Bonnet (GB) term, on the primordial scalar and tensor spectra of the slow-roll inflation in the consistent D → 4 Einstein Gauss-Bonnet (4EGB) gravity. The GB term is incorporated into gravitational dynamics via the re-scaling of the GB coupling constant α → α/(D-4) in the limit D → 4. For our purpose, we calculate explicitly the primordial scalar and tensor power spectra with GB corrections accurate to the next-to-leading order in the slow-roll approximation in the slow-roll inflation by using the third-order uniform asymptotic approximation method. The corresponding spectral indices and their runnings of the spectral indices for both the scalar and tensor perturbations as well as the ratio between the scalar and tensor spectra are also calculated up to the next-to-leading order in the slow-roll expansions. These results represent the most accurate results obtained so far in the literature. In addition, by studying the theoretical predictions of the scalar spectral index and the tensor-to-scalar ratio with Planck 2018 constraint in a model with power-law potential, we show that the second-order corrections are important in future measurements.

Scale-invariant enhancement of gravitational waves during inflation

The inflationary 1-loop tensor power spectrum from a spectator scalar field with a sharp peak is calculated. Recent studies on primordial black holes suggest that the inflationary curvature perturbation may be huge on small scales. An enhanced curvature perturbation may arise from a drastic enhancement of spectator scalar field fluctuations. In this paper, using the in-in formalism, we calculate 1-loop quantum corrections to primordial gravitational waves by such a spectator field with a sharp peak in momentum space. We find scale-invariant loop corrections in this full quantum setup, in contrast to the sharply peaked corrections in the previously calculated scalar-induced tensor modes. Especially on super Hubble scales, the primordial gravitational waves are also amplified, which can be understood as a Bogoliubov transformation of the vacuum due to the enhanced scalar field. This mechanism allows us to probe the scalar field properties on extremely short-distance scales with the current and future cosmic microwave background and gravitational wave experiments, opening a novel window for inflationary cosmology.

Separation of spherically and translationally covariant finite quantum spaces within the XXX model

A method of separation of subspaces with definite values of the total spin, its z-projection and quasimomentum, within the state space of the XXX model, i.e. the N-th tensor power of a single qubit, is presented. We construct a complete system of orthogonal projectors onto such subspaces by use of the basis of wavelets and chosen measuring operators based on the total spin. We also demonstrate, on examples of N=6 and 8 magnetic spin rings, that the constructed projectors can be further decomposed into the orthogonal sum of density matrices of eigenstates of the XXX model in the corresponding subspaces. This decomposition exhibits a Galois symmetry, stemming from roots of indecomposable factors of characteristic polynomial of the Heisenberg Hamiltonian. In this way, we propose an alternative for Bethe Ansatz for moderate lengths of spin chains, by a return to the original spectral problem, avoiding the highly nonlinear system of Bethe Ansatz equations. Such results are easy to adapt in quantum information processing.

Tensor decomposition for painting analysis. Part 2: spatio-temporal simulation

In a previous article, we modelled the spectral and temporal dimensions of the photodegradation behaviour of pigments in the painting “A Japanese Lantern” by Oda Krohg. In particular, we extracted the endmembers and spectral fading rate of pigments by applying tensor decomposition on a time-series of spectroscopic point measurements. Now, we capture the same painting with a hyperspectral imaging setup and propose an approach to render the fading effects as 2D images. More precisely, from the hyperspectral image, we compute the concentration maps of each previously identified endmember with a least-squares unmixing method. Subsequently, by using tensor algebra, we multiply the concentration maps with the endmembers and their corresponding fading rate and obtain a 4D tensor where each pixel in the image is described by a spectrum and a fading function. This way, we generate past and future spatio-temporal simulations of the painting’s appearance by reversing and elevating light exposure, respectively.

Classical simulation of variational quantum classifiers using tensor rings

In recent times, Variational Quantum Circuits (VQC) have been widely adopted to different tasks in machine learning such as Combinatorial Optimization and Supervised Learning. With the growing interest, it is pertinent to study the boundaries of the classical simulation of VQCs to effectively benchmark the algorithms. Classically simulating VQCs can also provide the quantum algorithms with a better initialization reducing the amount of quantum resources needed to train the algorithm. Even though Matrix Product State representations have been extensively used for quantum state approximation, their capacity is limited in simulating quantum circuits due to the exponential complexity in circuit depth. This manuscript proposes an algorithm that compresses the quantum state within a circuit using a noisy tensor ring representation which allows for the implementation of VQC based algorithms on a classical simulator at a fraction of the usual storage and computational complexity. Using the tensor ring approximation of the input quantum state, we propose a method that applies the parametrized unitary operations while retaining the low-rank structure of the tensor ring corresponding to the transformed quantum state, providing an exponential improvement of storage and computational time in the number of qubits and layers. This approximation is used to implement the tensor ring VQC (TRVQC) for the task of supervised learning on Iris and MNIST datasets to demonstrate the performance of the proposed method compared with the implementations from classical simulator using Matrix Product States (MPS). TRVQC has a test accuracy of 82.63% compared to the benchmark of 83.68% on Iris dataset whereas the former outperforms the latter on a reduced MNIST dataset with TRVQC having an accuracy of 83.73% compared to the benchmark 81.02%, showcasing the comparable performance of the proposed algorithm with the MPS framework.

TR-ReFloc: A TR-based Framework for Recovering Missed RSS for WiFi indoor positioning in the offline and online phase

The research on WiFi-based fingerprint positioning has become an important topic in the field of indoor positioning due to its low-cost deployment and widespread applications. However, received signal strength (RSS) from access points (APs) may be missed due to factors such as Green WiFi or AP malfunctions. Traditional approach often set the missed RSS to a predetermined default value, which may not be accurate. We propose a fingerprint recovery framework based on tensor-ring nuclear norm minimization (TRNNM) algorithm which uses the correlation of RSS to recover missed fingerprints in both offline and online phases, called TR-ReFloc. In the offline phase, the radio map in tensor form is recovered by tensor ring (TR) decomposition. In the online phase, depending on different user requirements, there are two stages to generate the positioning results. First, rapid results are generated in coarse positioning stage. Next, high-precision results generated in fine positioning stage by recovering the missed fingerprints using the correlation of test points (TPs) are closed to the current TP. Experiments demonstrate that the proposed framework is robust for recovering missed RSS and can maintain localization accuracy within 2 m even in cases of high missing rate, demonstrating significant improvement over traditional methods.

Real-Time Equation-of-Motion Coupled-Cluster Cumulant Green's Function Method: Heterogeneous Parallel Implementation Based on the Tensor Algebra for Many-Body Methods Infrastructure.

We report the implementation of the real-time equation-of-motion coupled-cluster (RT-EOM-CC) cumulant Green's function method [ J. Chem. Phys. 2020, 152, 174113] within the Tensor Algebra for Many-body Methods (TAMM) infrastructure. TAMM is a massively parallel heterogeneous tensor library designed for utilizing forthcoming exascale computing resources. The two-body electron repulsion matrix elements are Cholesky-decomposed, and we imposed spin-explicit forms of the various operators when evaluating the tensor contractions. Unlike our previous real algebra Tensor Contraction Engine (TCE) implementation, the TAMM implementation supports fully complex algebra. The RT-EOM-CC singles (S) and doubles (D) time-dependent amplitudes are propagated using a first-order Adams-Moulton method. This new implementation shows excellent scalability tested up to 500 GPUs using the Zn-porphyrin molecule with 655 basis functions, with parallel efficiencies above 90% up to 400 GPUs. The TAMM RT-EOM-CCSD was used to study core photoemission spectra in the formaldehyde and ethyl trifluoroacetate (ESCA) molecules. Simulations of the latter involve as many as 71 occupied and 649 virtual orbitals. The relative quasiparticle ionization energies and overall spectral functions agree well with available experimental results.