Sparse Tensor Research Articles

An Intermediate Language for General Sparse Format Customization

The inevitable trend of hardware specialization drives an increasing use of custom data formats in processing sparse workloads, which are typically memory-bound. These formats facilitate the automated generation of target-aware data layouts to improve memory access latency and bandwidth utilization. However, existing sparse tensor programming models and compilers offer little or no support for productively customizing the sparse formats. Moreover, since these frameworks adopt an attribute-based approach for format abstraction, they cannot easily be extended to support general format customization. To overcome this deficiency, we propose UniSparse, an intermediate language that provides a unified abstraction for representing and customizing sparse formats. More concretely, we express a sparse format as a map from dense coordinates to a layout tree using a small set of well-defined query and mutation primitives. We also develop a compiler leveraging the MLIR infrastructure, which supports adaptive customization of formats, and automatic code generation of format conversion and compute operations for heterogeneous architectures. We demonstrate the efficacy of our approach through experiments running commonly-used sparse linear algebra operations with hybrid formats on multiple different hardware targets, including an Intel CPU, an NVIDIA GPU, and a simulated processing-in-memory (PIM) device.

Accurate regularized Tucker decomposition for image restoration

We propose a new accurate regularized Tucker decomposition (ARTD) method for image restoration (IR), which considers global low-rankness and local similarity of intrinsic image characteristics. Specifically, global low-rankness is represented by a sparse Tucker core tensor, whereas the local similarity is captured using nonnegative factor matrices and manifold regularization terms. Sparse nonnegative Tucker decomposition (SNTD) and graph nonnegative Tucker decomposition (GNTD) can be considered a special case of ARTD. We propose and implement an effective Alternating Proximal Gradient (APG) based algorithm to solve the ARTD model and deduce the closed-form updating rules. Notably, ARTD does not need to tune the Tucker rank and provides an initialization strategy and a first-order feedback control rule to accelerate its convergence. Experiments on real IR problems show that our method outperforms some existing state-of-the-art methods.

One Is All: Bridging the Gap between Neural Radiance Fields Architectures with Progressive Volume Distillation

Neural Radiance Fields (NeRF) methods have proved effective as compact, high-quality and versatile representations for 3D scenes, and enable downstream tasks such as editing, retrieval, navigation, etc. Various neural architectures are vying for the core structure of NeRF, including the plain Multi-Layer Perceptron (MLP), sparse tensors, low-rank tensors, hashtables and their compositions. Each of these representations has its particular set of trade-offs. For example, the hashtable-based representations admit faster training and rendering but their lack of clear geometric meaning hampers downstream tasks like spatial-relation-aware editing. In this paper, we propose Progressive Volume Distillation (PVD), a systematic distillation method that allows any-to-any conversions between different architectures, including MLP, sparse or low-rank tensors, hashtables and their compositions. PVD consequently empowers downstream applications to optimally adapt the neural representations for the task at hand in a post hoc fashion. The conversions are fast, as distillation is progressively performed on different levels of volume representations, from shallower to deeper. We also employ special treatment of density to deal with its specific numerical instability problem. Empirical evidence is presented to validate our method on the NeRF-Synthetic, LLFF and TanksAndTemples datasets. For example, with PVD, an MLP-based NeRF model can be distilled from a hashtable-based Instant-NGP model at a 10~20X faster speed than being trained the original NeRF from scratch, while achieving a superior level of synthesis quality. Code is available at https://github.com/megvii-research/AAAI2023-PVD.

Enhanced Tensor Low-Rank and Sparse Representation Recovery for Incomplete Multi-View Clustering

Incomplete multi-view clustering (IMVC) has attracted remarkable attention due to the emergence of multi-view data with missing views in real applications. Recent methods attempt to recover the missing information to address the IMVC problem. However, they generally cannot fully explore the underlying properties and correlations of data similarities across views. This paper proposes a novel Enhanced Tensor Low-rank and Sparse Representation Recovery (ETLSRR) method, which reformulates the IMVC problem as a joint incomplete similarity graphs learning and complete tensor representation recovery problem. Specifically, ETLSRR learns the intra-view similarity graphs and constructs a 3-way tensor by stacking the graphs to explore the inter-view correlations. To alleviate the negative influence of missing views and data noise, ETLSRR decomposes the tensor into two parts: a sparse tensor and an intrinsic tensor, which models the noise and underlying true data similarities, respectively. Both global low-rank and local structured sparse characteristics of the intrinsic tensor are considered, which enhances the discrimination of similarity matrix. Moreover, instead of using the convex tensor nuclear norm, ETLSRR introduces a generalized non-convex tensor low-rank regularization to alleviate the biased approximation. Experiments on several datasets demonstrate the effectiveness of our method compared with the state-of-the-art methods.

Sparse Support Tensor Machine with Scaled Kernel Functions

As one of the supervised tensor learning methods, the support tensor machine (STM) for tensorial data classification is receiving increasing attention in machine learning and related applications, including remote sensing imaging, video processing, fault diagnosis, etc. Existing STM approaches lack consideration for support tensors in terms of data reduction. To address this deficiency, we built a novel sparse STM model to control the number of support tensors in the binary classification of tensorial data. The sparsity is imposed on the dual variables in the context of the feature space, which facilitates the nonlinear classification with kernel tricks, such as the widely used Gaussian RBF kernel. To alleviate the local risk associated with the constant width in the tensor Gaussian RBF kernel, we propose a two-stage classification approach; in the second stage, we advocate for a scaling strategy on the kernel function in a data-dependent way, using the information of the support tensors obtained from the first stage. The essential optimization models in both stages share the same type, which is non-convex and discontinuous, due to the sparsity constraint. To resolve the computational challenge, a subspace Newton method is tailored for the sparsity-constrained optimization for effective computation with local convergence. Numerical experiments were conducted on real datasets, and the numerical results demonstrate the effectiveness of our proposed two-stage sparse STM approach in terms of classification accuracy, compared with the state-of-the-art binary classification approaches.

Performance Implication of Tensor Irregularity and Optimization for Distributed Tensor Decomposition

Tensors are used by a wide variety of applications to represent multi-dimensional data; tensor decompositions are a class of methods for latent data analytics, data compression, and so on. Many of these applications generate large tensors with irregular dimension sizes and nonzero distribution. CANDECOMP/PARAFAC decomposition ( Cpd ) is a popular low-rank tensor decomposition for discovering latent features. The increasing overhead on memory and execution time of Cpd for large tensors requires distributed memory implementations as the only feasible solution. The sparsity and irregularity of tensors hinder the improvement of performance and scalability of distributed memory implementations. While previous works have been proved successful in Cpd for tensors with relatively regular dimension sizes and nonzero distribution, they either deliver unsatisfactory performance and scalability for irregular tensors or require significant time overhead in preprocessing. In this work, we focus on medium-grained tensor distribution to address their limitation for irregular tensors. We first thoroughly investigate through theoretical and experimental analysis. We disclose that the main cause of poor Cpd performance and scalability is the imbalance of multiple types of computations and communications and their tradeoffs; and sparsity and irregularity make it challenging to achieve their balances and tradeoffs. Irregularity of a sparse tensor is categorized based on two aspects: very different dimension sizes and a non-uniform nonzero distribution. Typically, focusing on optimizing one type of load imbalance causes other ones more severe for irregular tensors. To address such challenges, we propose irregularity-aware distributed Cpd that leverages the sparsity and irregularity information to identify the best tradeoff between different imbalances with low time overhead. We materialize the idea with two optimization methods: the prediction-based grid configuration and matrix-oriented distribution policy, where the former forms the global balance among computations and communications, and the latter further adjusts the balances among computations. The experimental results show that our proposed irregularity-aware distributed Cpd is more scalable and outperforms the medium- and fine-grained distributed implementations by up to 4.4 × and 11.4 × on 1,536 processors, respectively. Our optimizations support different sparse tensor formats, such as compressed sparse fiber (CSF), coordinate (COO), and Hierarchical Coordinate (HiCOO), and gain good scalability for all of them.

A Heterogeneous Parallel Computing Approach Optimizing SpTTM on CPU-GPU via GCN

Sparse Tensor-Times-Matrix (SpTTM) is the core calculation in tensor analysis. The sparse distributions of different tensors vary greatly, which poses a big challenge to designing efficient and general SpTTM. In this paper, we describe SpTTM on CPU-GPU heterogeneous hybrid systems and give a parallel execution strategy for SpTTM in different sparse formats. We analyze the theoretical computer powers and estimate the number of tasks to achieve the load balancing between the CPU and the GPU of the heterogeneous systems. We discuss a method to describe tensor sparse structure by graph structure and design a new graph neural network SPT-GCN to select a suitable tensor sparse format. Furthermore, we perform extensive experiments using real datasets to demonstrate the advantages and efficiency of our proposed input-aware slice-wise SpTTM. The experimental results show that our input-aware slice-wise SpTTM can achieve an average speedup of 1.310 × compared to ParTI! library on a CPU-GPU heterogeneous system.

A Distributed-GPU Deep Reinforcement Learning System for Solving Large Graph Optimization Problems

Graph optimization problems (such as minimum vertex cover, maximum cut, traveling salesman problems) appear in many fields including social sciences, power systems, chemistry, and bioinformatics. Recently, deep reinforcement learning (DRL) has shown success in automatically learning good heuristics to solve graph optimization problems. However, the existing RL systems either do not support graph RL environments or do not support multiple or many GPUs in a distributed setting. This has compromised the ability of reinforcement learning in solving large-scale graph optimization problems due to lack of parallelization and high scalability. To address the challenges of parallelization and scalability, we develop RL4GO , a high-performance distributed-GPU DRL framework for solving graph optimization problems. RL4GO focuses on a class of computationally demanding RL problems, where both the RL environment and policy model are highly computation intensive. Traditional reinforcement learning systems often assume either the RL environment is of low time complexity or the policy model is small. In this work, we distribute large-scale graphs across distributed GPUs and use the spatial parallelism and data parallelism to achieve scalable performance. We compare and analyze the performance of the spatial parallelism and data parallelism and show their differences. To support graph neural network (GNN) layers that take as input data samples partitioned across distributed GPUs, we design parallel mathematical kernels to perform operations on distributed 3D sparse and 3D dense tensors. To handle costly RL environments, we design a parallel graph environment to scale up all RL-environment-related operations. By combining the scalable GNN layers with the scalable RL environment, we are able to develop high-performance RL4GO training and inference algorithms in parallel. Furthermore, we propose two optimization techniques—replay buffer on-the-fly graph generation and adaptive multiple-node selection—to minimize the spatial cost and accelerate reinforcement learning. This work also conducts in-depth analyses of parallel efficiency and memory cost and shows that the designed RL4GO algorithms are scalable on numerous distributed GPUs. Evaluations on large-scale graphs show that (1) RL4GO training and inference can achieve good parallel efficiency on 192 GPUs, (2) its training time can be 18 times faster than the state-of-the-art Gorila distributed RL framework [ 34 ], and (3) its inference performance achieves a 26 times improvement over Gorila.

Mosaic: An Interoperable Compiler for Tensor Algebra

We introduce Mosaic, a sparse tensor algebra compiler that can bind tensor expressions to external functions of other tensor algebra libraries and compilers. Users can extend Mosaic by adding new functions and bind a sub-expression to a function using a scheduling API. Mosaic substitutes the bound sub-expressions with calls to the external functions and automatically generates the remaining code using a default code generator. As the generated code is fused by default, users can productively leverage both fusion and calls to specialized functions within the same compiler. We demonstrate the benefits of our dual approach by showing that calling hand-written CPU and specialized hardware functions can provide speedups of up to 206× against fused code in some cases, while generating fused code can provide speedups of up to 3.57× against code that calls external functions in other cases. Mosaic also offers a search system that can automatically map an expression to a set of registered external functions. Both the explicit binding and automatic search are verified by Mosaic. Additionally, the interface for adding new external functions is simple and general. Currently, 38 external functions have been added to Mosaic, with each addition averaging 20 lines of code.

Indexed Streams: A Formal Intermediate Representation for Fused Contraction Programs

We introduce indexed streams, a formal operational model and intermediate representation that describes the fused execution of a contraction language that encompasses both sparse tensor algebra and relational algebra. We prove that the indexed stream model is correct with respect to a functional semantics. We also develop a compiler for contraction expressions that uses indexed streams as an intermediate representation. The compiler is only 540 lines of code, but we show that its performance can match both the TACO compiler for sparse tensor algebra and the SQLite and DuckDB query processing libraries for relational algebra.

An Onboard Point Cloud Semantic Segmentation System for Robotic Platforms

Point clouds represent an important way for robots to perceive their environments, and can be acquired by mobile robots with LiDAR sensors or underwater robots with sonar sensors. Hence, real-time semantic segmentation of point clouds with onboard edge devices is essential for robots to apprehend their surroundings. In this paper, we propose an onboard point cloud semantic segmentation system for robotic platforms to overcome the conflict between attaining high accuracy of segmentation results and the limited available computational resources of onboard devices. Our system takes raw a sequence of point clouds as inputs, and outputs semantic segmentation results for each frame as well as a reconstructed semantic map of the environment. At the core of our system is the transformer-based hierarchical feature extraction module and fusion module. The two modules are implemented with sparse tensor technologies to speed up inference. The predictions are accumulated according to Bayes rules to generate a global semantic map. Experimental results on the SemanticKITTI dataset show that our system achieves +2.2% mIoU and 18× speed improvements compared with SOTA methods. Our system is able to process 2.2 M points per second on Jetson AGX Xavier (NVIDIA, Santa Clara, USA), demonstrating its applicability to various robotic platforms.

Dose Images Reconstruction Based on X-ray-Induced Acoustic Computed Tomography

The accurate reconstruction of the in vivo dose is a critical step in radiation therapy. X-ray-induced acoustic imaging is a promising technology for in vivo dose reconstruction, as it enables the nonradiative and noninvasive monitoring of radiation dose. However, current X-ray acoustic imaging methods suffer from several limitations, including high signal-to-noise ratio, poor imaging quality and massive loss of structural information. To address these limitations, we propose a dose image reconstruction method based on tensor sparse dictionary learning. Specifically, we combine tensor coding with compressed sensing data, extend two-dimensional dictionary learning to three-dimensional by using tensor product, and then utilize the spatial information of X-ray acoustic signal more efficiently. To reduce the artifacts of reconstruction images caused by spare sampling, we design the alternate iterative solution of the tensor sparse coefficient and tensor dictionary. In addition, we build the X-ray-induced acoustic dose images reconstruction system, simulate the X-ray acoustic signals based on patients’ information from Sichuan Cancer Hospital, and then create the simulated datasets. Compared to some typical state-of-the art imaging methods, the experimental results demonstrate that our method can significantly improve the quality of reconstructed images and the accuracy of dose distribution.

Sgap: towards efficient sparse tensor algebra compilation for GPU

Sparse compiler is a promising solution for sparse tensor algebra optimization. In compiler implementation, reduction in sparse-dense hybrid algebra plays a key role in performance. Though GPU provides various reduction semantics that can better utilize the parallel computing and memory bandwidth capacity, the central question is: how to elevate the flexible reduction semantics to sparse compilation theory that assumes serial execution. Specifically, we have to tackle two main challenges: (1) there are wasted parallelism by adopting static synchronization granularity (2) static reduction strategy limits optimization space exploration. We propose Sgap: s egment g roup and a tomic p arallelism to solve these problems. Atomic parallelism captures the flexible reduction semantics to systematically analyze the optimization space of sparse-dense hybrid algebra on GPU. It is a new optimization technique beyond current compiler-based and open-source runtime libraries. Segment group elevates the flexible reduction semantics to suitable levels of abstraction in the sparse compilation theory. It adopts changeable group size and user-defined reduction strategy to solve challenge (1) and (2), respectively. Finally, we use GPU sparse matrix-matrix multiplication (SpMM) on the TACO compiler as a use case to demonstrate the effectiveness of segment group in reduction semantics elevation. We achieve up to $$1.2\,\times$$ speedup over the original TACO’s SpMM kernels. We also apply new optimization techniques found by atomic parallelism to an open-source state-of-the-art SpMM library dgSPARSE. We achieve $$1.6 \, \times \sim 2.3 \, \times$$ speedup on the algorithm tuned with atomic parallelism.

Multi-view Ensemble Clustering via Low-rank and Sparse Decomposition: From Matrix to Tensor

As a significant extension of classical clustering methods, ensemble clustering first generates multiple basic clusterings and then fuses them into one consensus partition by solving a problem concerning graph partition with respect to the co-association matrix. Although the collaborative cluster structure among basic clusterings can be well discovered by ensemble clustering, most advanced ensemble clustering utilizes the self-representation strategy with the constraint of low-rank to explore a shared consensus representation matrix in multiple views. However, they still encounter two challenges: (1) high computational cost caused by both the matrix inversion operation and singular value decomposition of large-scale square matrices; (2) less considerable attention on high-order correlation attributed to the pursue of the two-dimensional pair-wise relationship matrix. In this article, based on low-rank and sparse decomposition from both matrix and tensor perspectives, we propose two novel multi-view ensemble clustering methods, which tangibly decrease computational complexity. Specifically, our first method utilizes low-rank and sparse matrix decomposition to learn one common co-association matrix, while our last method constructs all co-association matrices into one third-order tensor to investigate the high-order correlation among multiple views by low-rank and sparse tensor decomposition. We adopt the alternating direction method of multipliers to solve two convex models by dividing them into several subproblems with closed-form solution. Experimental results on ten real-world datasets prove the effectiveness and efficiency of the proposed two multi-view ensemble clustering methods by comparing them with other advanced ensemble clustering methods.

Distributed non-negative RESCAL with automatic model selection for exascale data

With the boom in the development of computer hardware and software, social media, IoT platforms, and communications, there has been exponential growth in the volume of data produced worldwide. Among these data, relational datasets are growing in popularity as they provide unique insights regarding the evolution of communities and their interactions. Relational datasets are naturally non-negative, sparse, and extra-large. Relational data usually contain triples (subject, relation, object) and are represented as graphs/multigraphs, called knowledge graphs, which need to be embedded into a low-dimensional dense vector space. Among various embedding models, RESCAL allows the learning of relational data to extract the posterior distributions over the latent variables and to make predictions of missing relations. However, RESCAL is computationally demanding and requires a fast and distributed implementation to analyze extra-large real-world datasets. Here we introduce a distributed non-negative RESCAL algorithm for heterogeneous CPU/GPU architectures with automatic selection of the number of latent communities (model selection), called pyDRESCALk. We demonstrate the correctness of pyDRESCALk with real-world and large synthetic tensors and the efficacy showing near-linear scaling that concurs with the theoretical complexities. Finally, pyDRESCALk determines the number of latent communities in an 11-terabyte dense and 9-exabyte sparse synthetic tensor.

Pixel-wise mechanical damage detection of waxy maize using spectral–spatial feature extraction and hyperspectral image

Mechanical damage detection on waxy maize is critical to guarantee maize quality and meet the product requirements. A novel spectral–spatial feature extraction enhanced fully connected neural network (SSFE-FCNN) classification method is proposed in this paper to achieve pixel-wise mechanical damage detection of waxy maize using the hyperspectral image (HSI). The proposed SSFE-FCNN utilizes the online tensor dictionary learning algorithm to extract spectral–spatial tensor features from HSI cubes. Moreover, the designed fully connected neural network extracts high-level features from the tensor features and provides classification results. The SSFE-FCNN combines the advantages of sparse tensor representation and neural networks to exploit joint spectral–spatial information in HSI and improve detection performance. Experimental results on the HSI of waxy maize show that the proposed SSFE-FCNN is superior to the support vector machine and the currently popular neural network methods. The overall accuracy of SSFE-FCNN in the pixel-wise classification of waxy maize achieves as high as 98.09%. The false and missing detection rates are 2.08% and 1.96%, respectively. Furthermore, pixel-wise detection can detect slight damage and perform the maize grading operation according to the damage ratio.

Sparse tensor based nuclear gradients for periodic Hartree-Fock and low-scaling correlated wave function methods in the CP2K software package: A massively parallel and GPU accelerated implementation.

The development of novel double-hybrid density functionals offers new levels of accuracy and is leading to fresh insights into the fundamental properties of matter. Hartree-Fock exact exchange and correlated wave function methods, such as second-order Møller-Plesset (MP2) and direct random phase approximation (dRPA), are usually required to build such functionals. Their high computational cost is a concern, and their application to large and periodic systems is, therefore, limited. In this work, low-scaling methods for Hartree-Fock exchange (HFX), SOS-MP2, and direct RPA energy gradients are developed and implemented in the CP2K software package. The use of the resolution-of-the-identity approximation with a short range metric and atom-centered basis functions leads to sparsity, allowing for sparse tensor contractions to take place. These operations are efficiently performed with the newly developed Distributed Block-sparse Tensors (DBT) and Distributed Block-sparse Matrices (DBM) libraries, which scale to hundreds of graphics processing unit (GPU) nodes. The resulting methods, resolution-of-the-identity (RI)-HFX, SOS-MP2, and dRPA, were benchmarked on large supercomputers. They exhibit favorable sub-cubic scaling with system size, good strong scaling performance, and GPU acceleration up to a factor of 3. These developments will allow for double-hybrid level calculations of large and periodic condensed phase systems to take place on a more regular basis.

Accurate and Efficient Simulation of Very High-Dimensional Neural Mass Models with Distributed-Delay Connectome Tensors

This paper introduces methods and a novel toolbox that efficiently integrates high-dimensional Neural Mass Models (NMMs) specified by two essential components. The first is the set of nonlinear Random Differential Equations (RDEs) of the dynamics of each neural mass. The second is the highly sparse three-dimensional Connectome Tensor (CT) that encodes the strength of the connections and the delays of information transfer along the axons of each connection. To date, simplistic assumptions prevail about delays in the CT, often assumed to be Dirac-delta functions. In reality, delays are distributed due to heterogeneous conduction velocities of the axons connecting neural masses. These distributed-delay CTs are challenging to model. Our approach implements these models by leveraging several innovations. Semi-analytical integration of RDEs is done with the Local Linearization (LL) scheme for each neural mass, ensuring dynamical fidelity to the original continuous-time nonlinear dynamic. This semi-analytic LL integration is highly computationally-efficient. In addition, a tensor representation of the CT facilitates parallel computation. It also seamlessly allows modeling distributed delays CT with any level of complexity or realism. This ease of implementation includes models with distributed-delay CTs. Consequently, our algorithm scales linearly with the number of neural masses and the number of equations they are represented with, contrasting with more traditional methods that scale quadratically at best. To illustrate the toolbox's usefulness, we simulate a single Zetterberg-Jansen and Rit (ZJR) cortical column, a single thalmo-cortical unit, and a toy example comprising 1000 interconnected ZJR columns. These simulations demonstrate the consequences of modifying the CT, especially by introducing distributed delays. The examples illustrate the complexity of explaining EEG oscillations, e.g., split alpha peaks, since they only appear for distinct neural masses. We provide an open-source Script for the toolbox.

Smooth low-rank representation with a Grassmann manifold for tensor completion

Utilizing low-rank representation, recent methods have efficiently estimated the low-rank tensor for tensor completion (TC). However, owing to the identifiability issue, these methods inevitably preserve some outliers of the sparse tensor as elements of the low-rank tensor, which leads to an imprecise estimation of the low-rank tensor. In this paper, we propose a smooth low-rank representation with a Grassmann manifold (SLRR-GM) model to tackle the identifiability issue. Specifically, we first smooth the low-frequency and high-frequency discontinuities of the low-rank tensor to sufficiently remove the preserved outliers of the sparse tensor. Accordingly, a smooth low-rank representation model is built to precisely estimate the low-rank tensor. Second, we smooth the discontinuities through a small number of the feature vectors that characterize the original vectors of the low-rank tensor. This uses the Grassmann manifold to effectively reduce the computational complexity of smoothing the discontinuities. Third, we integrate the smooth low-rank representation model with the Grassmann manifold to propose SLRR-GM. Based on the alternating direction method of multipliers (ADMM), an algorithm is proposed to solve the SLRR-GM model. We also investigate the complexity and convergence to validate the algorithmic effectiveness. Experiments in different scenarios demonstrate that the proposed method outperforms several state-of-the-art methods in terms of the quantitative and visual aspects with the acceptable processing time.

Hyperspectral image mixed noise removal via tensor robust principal component analysis with tensor-ring decomposition

ABSTRACT Due to the instability of sensors and other factors, hyperspectral images (HSIs) are inevitably polluted by various types of mixed noise. To explore a better denoising method based on the existing research, combining the denoising advantages of tensor-ring (TR) decomposition and tensor robust principal component analysis (TRPCA), a mixed-noise removal method for HSIs is proposed in this paper. First, TRPCA maintains the tensor structure of the image itself, accurately recovers the low-rank part and the sparse part from their sum and separates the sparse noise in the form of sparse tensors. Then, TR decomposition is introduced to denoise the low-rank tensors. To verify the effectiveness and superiority of this method, experiments are carried out on two simulated data sets and two real data sets. Compared with the traditional denoising methods and several existing improved denoising methods from both visual and quantitative aspects, the proposed TRPCA-TR method provides better denoising results.