Sparse Compression Research Articles

GSC: efficient lossless compression of VCF files with fast query.

With the rise of large-scale genome sequencing projects, genotyping of thousands of samples has produced immense variant call format (VCF) files. It is becoming increasingly challenging to store, transfer, and analyze these voluminous files. Compression methods have been used to tackle these issues, aiming for both high compression ratio and fast random access. However, existing methods have not yet achieved a satisfactory compromise between these 2 objectives. To address the aforementioned issue, we introduce GSC (Genotype Sparse Compression), a specialized and refined lossless compression tool for VCF files. In benchmark tests conducted across various open-source datasets, GSC showcased exceptional performance in genotype data compression. Compared with the industry's most advanced tools (namely, GBC and GTC), GSC achieved compression ratios that were higher by 26.9% to 82.4% over GBC and GTC on the datasets, respectively. In lossless compression scenarios, GSC also demonstrated robust performance, with compression ratios 1.5× to 6.5× greater than general-purpose tools like gzip, zstd, and BCFtools-a mode not supported by either GBC or GTC. Achieving such high compression ratios did require some reasonable trade-offs, including longer decompression times, with GSC being 1.2× to 2× slower than GBC, yet 1.1× to 1.4× faster than GTC. Moreover, GSC maintained decompression query speeds that were equivalent to its competitors. In terms of RAM usage, GSC outperformed both counterparts. Overall, GSC's comprehensive performance surpasses that of the most advanced technologies. GSC balances high compression ratios with rapid data access, enhancing genomic data management. It supports seamless PLINK binary format conversion, simplifying downstream analysis.

Dense or Sparse : Elastic SPMM Implementation for Optimal Big-Data Processing

Many real-world graph datasets can be represented using a sparse matrix format, and they are widely used for various big-data applications. The multiplication of two sparse matrices (SPMM) is a major kernel for various machine learning algorithms when using a sparsely expressed dataset. Apache Spark, a general-purpose big-data processing engine, includes the SPMM operation in its linear algebra package. The default Spark SPMM implementation, however, always converts a right sparse matrix to a dense format before performing multiplication, which can result in significant performance overhead for diverse SPMM scenarios. To address a limitation of the current Spark implementation, we describe an SPMM implementation that keeps the right matrix in a Compressed Sparse Column (CSC) format and propose an SPMM task latency prediction model based on a Deep Neural Network (DNN) architecture. Using the SPMM latency prediction model, we implement an elastic SPMM implementation recommendation service, which we name DoS ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D</b> ense <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">o</b> r <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</b> parse). The proposed DoS recommends an optimal SPMM implementation method of either transforming a right matrix to a dense format or keeping it as a sparse format during the multiplication. Through evaluation of the proposed system using a real-world graph reveals that the proposed service can improve the SPMM latency of default Spark implementation by 2.2 times while shortening the overall execution time.

GPU-Accelerated PCG Method for the Block Adjustment of Large-Scale High-Resolution Optical Satellite Imagery Without GCPs

The precise geo-positioning of high-resolution satellite imagery (HRSI) without ground control points (GCPs) is an important and fundamental step in global mapping, three-dimensional modeling, and so on. In this paper, to improve the efficiency of large-scale bundle adjustment (BA), we propose a combined Preconditioned Conjugate Gradient (PCG) and Graphic Processing Unit (GPU) parallel computing approach for the BA of large-scale HRSI without GCPs. The proposed approach consists of three main components: 1) construction of a BA model without GCPs ; 2) reduction of memory consumption using the Compressed Sparse Row sparse matrix format; and 3) improvement of the computational efficiency by the use of the combined PCG and GPU parallel computing method. The experimental results showed that the proposed method: 1) consumes less memory consumption compared to the conventional full matrix format method; 2) demonstrates higher computational efficiency than the single-core, Ceres-solver and multi-core central processing unit computing methods, with 9.48, 6.82, and 3.05 times faster than the above three methods, respectively; 3) obtains comparable BA accuracy with the above three methods, with image residuals of about 0.9 pixels; and 4) is superior to the parallel bundle adjustment method in the reprojection error.

Heterogeneous Defect Prediction Algorithm Combined With Federated Sparse Compression

Heterogeneous defect prediction (HDP) constructs a defect prediction model through the source project to realize the defect tendency prediction of the target project. HDP based on federated learning can combine multi-party defect data to improve the defect prediction effect under the condition of protecting privacy. However, with the increase of task complexity and model performance requirements, the number of neural network layers gradually deepens, and the model parameters also increase. increase accordingly. In a federated learning scenario with multiple model parameters, limited communication bandwidth, and multiple clients, the amount of data received by the server is very large, causing huge communication pressure and seriously affecting the overall training efficiency. In order to reduce the communication cost, this paper proposes a federated sparse compression (FSC) algorithm. First, in order to improve the generalization performance of the model, each participant performs local training based on CapsNet, and achieves better prediction performance by calculating the relative position information of feature combinations. Then the gradient parameters are encrypted based on differential privacy to ensure data security. In order to reduce the amount of communication bits, the encrypted model parameters are subjected to sparse binary compression, the model training is converted from dense calculation to sparse calculation, and Golomb encoding is selected to be sent to the server for aggregation. Finally, the server side decodes the received data, performs sparse binary compression and encoding and sends it back to each participant. Experiments on nine projects in three public databases (Relink, NASA, and AEEEM) show that FSC can effectively reduce the amount of communication bits.

A novel wind turbine fault diagnosis method based on compressed sensing and DTL-CNN

This paper describes the development of a fault diagnosis method for identifying different fault conditions in the rolling bearings and gears of wind turbines. For the fault signal, the compressed sensing (CS) technology is used to perform noise reduction and feature extraction. The noise reduction process consists of sparse compression and reconstruction of the signal. After the data is processed by the compressed sensing technology, the noise and redundant parts of the signal can be greatly removed, and the real operating state signal of the wind turbine can be restored to the maximum. The fault diagnosis scheme is based on a combination of deep transfer learning and convolutional neural network (DTL-CNN), which is able to perform fault type identification with a small batch of rolling bearing data samples and gear samples. In this study, a new CNN structure was developed and the structure was used to achieve bearing-to-bearing and bearing-to-gear transfer fault diagnosis. Finally, the reliability and superiority of the proposed method in wind turbine rolling bearing and gear fault diagnosis are shown by the experimental results.

Adaptive sparse ternary gradient compression for distributed DNN training in edge computing

In edge computing, though distributed training of Deep Neural Networks (DNNs) is expected to exchange massive gradients between parameter servers and working nodes, the high communication cost constrains the training speed. To break this limitation, gradient compression algorithms expect the ultimate compression ratio at the expense of the accuracy of the trained model. Therefore, new gradient compression techniques are necessary to ensure both communication efficiency and model accuracy. This paper introduces a novel technique—an Adaptive Sparse Ternary Gradient Compression (ASTC) scheme, which relies on the number of gradients in model layers to compress gradients. ASTC establishes the model compression selection criterion by gradients’ amount, compresses the network layer that meets the model’s standard, evaluates the gradients’ importance based on entropy to adaptively perform sparse compression, and finally conducts ternary quantization compression and a lossless code scheme on sparse gradients. Using public datasets (MNIST, CIFAR-10, Tiny ImageNet) and deep learning models (CNN, LeNet5, ResNet18) for experimental evaluation, we exhibit excellent results that the training efficiency of ASTC is about 1.6 times, 1.37 times, and 1.1 times higher than that of Top-1, AdaComp, and SBC, respectively. Furthermore, ASTC can be improved by an average of about 1.9% in training accuracy compared with the above approaches.

CMD: controllable matrix decomposition with global optimization for deep neural network compression

The compression and acceleration of Deep neural networks (DNNs) are necessary steps to deploy sophisticated networks into resource-constrained hardware systems. Due to the weight matrix tends to be low-rank and sparse, several low-rank and sparse compression schemes are leveraged to reduce the overwhelmed weight parameters of DNNs. In these previous schemes, how to make the most of the low-rank and sparse components of weight matrices and how to globally decompose the weight matrix of different layers for efficient compression need to be further investigated. In this paper, in order to effectively utilize the low-rank and sparse characteristics of the weight matrix, we first introduce a sparse coefficient to dynamically control the allocation between the low-rank and sparse components, and an efficient reconstructed network is designed to reduce the inference time. Secondly, since the results of low-rank decomposition can affect the compression ratio and accuracy of DNNs, we establish an optimization problem to automatically select the optimal hyperparameters of the compressed network and achieve global compression for all the layers of network synchronously. Finally, to solve the optimization problem, we present a decomposition-searching algorithm to search the optimal solution. The algorithm can dynamically keep the balance between the compression ratio and accuracy. Extensive experiments of AlexNet, VGG-16 and ResNet-18 on CIFAR-10 and ImageNet are employed to evaluate the effectiveness of the proposed approach. After slight fine-tuning, compressed networks have gained \(1.2\times\) to \(11.3\times\) speedup and our method reduces the size of different networks by \(1.4\times\) to \(14.6\times\).

Financial Volatility Forecasting: A Sparse Multi-Head Attention Neural Network

Accurately predicting the volatility of financial asset prices and exploring its laws of movement have profound theoretical and practical guiding significance for financial market risk early warning, asset pricing, and investment portfolio design. The traditional methods are plagued by the problem of substandard prediction performance or gradient optimization. This paper proposes a novel volatility prediction method based on sparse multi-head attention (SP-M-Attention). This model discards the two-dimensional modeling strategy of time and space of the classic deep learning model. Instead, the solution is to embed a sparse multi-head attention calculation module in the network. The main advantages are that (i) it uses the inherent advantages of the multi-head attention mechanism to achieve parallel computing, (ii) it reduces the computational complexity through sparse measurements and feature compression of volatility, and (iii) it avoids the gradient problems caused by long-range propagation and therefore, is more suitable than traditional methods for the task of analysis of long time series. In the end, the article conducts an empirical study on the effectiveness of the proposed method through real datasets of major financial markets. Experimental results show that the prediction performance of the proposed model on all real datasets surpasses all benchmark models. This discovery will aid financial risk management and the optimization of investment strategies.

Efficient multiscale methods for the semiclassical Schrödinger equation with time-dependent potentials

The semiclassical Schrödinger equation with time-dependent potentials is an important model to study electron dynamics under external controls in the mean-field picture. In this paper, we propose two multiscale finite element methods to solve this problem. In the offline stage, for the first approach, the localized multiscale basis functions are constructed using sparse compression of the Hamiltonian operator at the initial time; for the latter, basis functions are further enriched using a greedy algorithm for the sparse compression of the Hamiltonian operator at later times. In the online stage, the Schrödinger equation is approximated by these localized multiscale basis functions in space and is solved by the Crank–Nicolson method in time. These multiscale basis functions have compact supports in space, leading to the sparsity of the stiffness matrix, and thus the computational complexity of these two methods is comparable to that of the standard finite element method. The spatial mesh size in the multiscale finite element methods is O(ε), while the mesh size in the standard finite element method is o(ε), where ε is the semiclassical parameter. Through a number of numerical examples in 1D and 2D, for approximately the same number of basis, we show that the approximation error of the multiscale finite element method is at least two orders of magnitude smaller than that of the standard finite element method, and the enrichment further reduces the error by another one order of magnitude.

FedOpt: Towards Communication Efficiency and Privacy Preservation in Federated Learning

Artificial Intelligence (AI) has been applied to solve various challenges of real-world problems in recent years. However, the emergence of new AI technologies has brought several problems, especially with regard to communication efficiency, security threats and privacy violations. Towards this end, Federated Learning (FL) has received widespread attention due to its ability to facilitate the collaborative training of local learning models without compromising the privacy of data. However, recent studies have shown that FL still consumes considerable amounts of communication resources. These communication resources are vital for updating the learning models. In addition, the privacy of data could still be compromised once sharing the parameters of the local learning models in order to update the global model. Towards this end, we propose a new approach, namely, Federated Optimisation (FedOpt) in order to promote communication efficiency and privacy preservation in FL. In order to implement FedOpt, we design a novel compression algorithm, namely, Sparse Compression Algorithm (SCA) for efficient communication, and then integrate the additively homomorphic encryption with differential privacy to prevent data from being leaked. Thus, the proposed FedOpt smoothly trade-offs communication efficiency and privacy preservation in order to adopt the learning task. The experimental results demonstrate that FedOpt outperforms the state-of-the-art FL approaches. In particular, we consider three different evaluation criteria; model accuracy, communication efficiency and computation overhead. Then, we compare the proposed FedOpt with the baseline configurations and the state-of-the-art approaches, i.e., Federated Averaging (FedAvg) and the paillier-encryption based privacy-preserving deep learning (PPDL) on all these three evaluation criteria. The experimental results show that FedOpt is able to converge within fewer training epochs and a smaller privacy budget.

Sparse Compression of Expected Solution Operators

We show that the expected solution operator of prototypical linear elliptic PDEs with random coefficients is well approximated by a computable sparse matrix. This result is based on a random locali...

Spatial correlation based channel compression feedback algorithm for massive MIMO systems

Aiming at the non-linear structure of massive multiple-input multiple-output (MIMO) channel data, this paper proposes a channel state information (CSI) compression feedback algorithm based on Laplacian Eigenmaps (LE) non-linear processing for massive MIMO uniform linear array. The spatial correlation of the channel array determines the Laplacian matrix, and the channel compression matrix is obtained by Laplacian matrix eigenvalue decomposition. The simulation results show that the proposed LE algorithm can reduce the feedback overhead, and its bit error rate (BER) performance is better than that of the discrete cosine transform (DCT) sparse compression algorithm. In addition, the proposed LE algorithm computational complexity is higher than DCT, and lower than principal component analysis (PCA) and Karhunen-Loeve transform (KLT) algorithms, but the LE algorithm can achieve higher feedback accuracy when the feedback overhead is slightly lower than DCT.

Bi-dimensional Signal Compression Based on Linear Prediction Coding

The big data phenomenon has gained much attention in the wireless communications field. Addressing big data is a challenging and time-demanding task that requires a large computational infrastructure to ensure successful data processing and analysis. In such a context, data compression helps to reduce the amount of data required to represent redundant information while reliably preserving the original content as much as possible. We here consider Compressed Sensing (CS) theory for extracting critical information and representing it with substantially reduced measurements of the original data. For CS application, it is, however, required to design a convenient sparsifying basis or transform. In this work, a large amount of bi-dimensional (2D) correlated signals are considered for compression. The envisaged application is that of data collection in large scale Wireless Sensor Networks. We show that, using CS, it is possible to recover a large amount of data from the collection of a reduced number of sensors readings. In this way, CS use makes it possible to recover large data sets with acceptable accuracy as well as reduced global scale cost. For sparsifying basis search, in addition to conventional sparsity-inducing methods, we propose a new transformation based on Linear Prediction Coding (LPC) that effectively exploits correlation between neighboring data. The steps of data aggregation using CS include sparse compression basis design and then decomposition matrix construction and recovery algorithm application. Comparisons to the case of one-dimensional (1D) reading and to conventional 2D compression methods show the benefit from the better exploitation of the correlation by herein envisaged 2D processing. Simulation results on both synthetic and real WSN data demonstrate that the proposed LPC approach with 2D scenario realizes significant reconstruction performance enhancement compared to former conventional transformations.

Sparse compression algorithm for MBOK signals

To cope with the limitations of traditional direct-sequence spread spectrum system application, MBOK signals have been considered owing to their high gain, fast rate, and high frequency band transmission capability. However, to address the problems of large search range, slow speed, and large complexity when receiving and searching for MBOK signals, the sparse compression search (SCS) algorithm is proposed. The idea of the SCS algorithm is to compress the frequency dimension in order to improve the frequency compensation efficiency, and to then establish the sparse processing from the code dimension and the frequency dimension, thus improving the search speed. Next, the code and frequency values are calculated by channel mapping matching. The test and analysis results show that the SCS algorithm can quickly estimate the code and frequency values, and the detection probability and the computation speed are much better than other algorithms.

Compressing by Learning in a Low-Rank and Sparse Decomposition Form

Low-rankness and sparsity are often used to guide the compression of convolutional neural networks (CNNs) separately. Since they capture global and local structure of a matrix respectively, we combine these two complementary properties together to pursue better network compression performance. Most existing low-rank or sparse compression methods compress the networks by approximating pre-trained models. However, the optimal solutions to pre-trained models may not be optimal to compressed networks with low-rank or sparse constraints. In this paper, we propose a low-rank and sparse learning framework that trains the compressed network from scratch. Our compressing process can be described as the following three stages. (a) In the structure designing stage, we decompose a weight matrix into sum of low-rank matrix and sparse matrix, and then the low-rank matrix is further factorized into product of two small matrices. (b) In training stage, we add l 1 regularization to the loss function to force the sparse matrix to be sparse. (c) In the post-processing stage, we remove the unimportant connection of sparse matrix according to its energy distribution. The pruning process in the post-processing stage reserves most of capacity of the network and keeps the performance of the network to a great extent. The performance can be further improved with fine-tuning, along with sparse masked convolution. Experiments on several common datasets demonstrate our model is superior to other network compression methods based on low-rankness or sparsity. On CIFAR-10, our method compresses VGGNet-19 to 3.14% and PreActResNet-56 to 29.78% without accuracy drop. 62.43% of parameters of ResNet-50 are reduced with 0.55% top-5 accuracy loss on ImageNet.

Light Field Image Compression via CNN-Based EPI Super-Resolution and Decoder-Side Quality Enhancement

Because of the capacity of capturing both the spatial and angular information of the light rays simultaneously, light field images (LFIs) contain richer scene information compared with conventional images, but at the cost of huge volume. This paper proposes a novel LFI sparse compression framework driven by convolutional neural network (CNN). The epipolar plane image (EPI) super-resolution is for compensating the information loss caused by sparse sampling and the decoder-side sub-aperture images (SAIs) quality enhancement is for compensating the information loss caused by lossy compression. Specifically, we choose those SAIs both in odd rows and odd columns as our key SAIs and compress them using standard video encoder. For those non-key SAIs, we predict them using decompressed key SAIs by taking advantage of the special structure of EPI. The low-resolution EPIs generated from the sparse SAIs are super-resolved by a CNN and the outputs, high-resolution EPIs, are used to rebuild the dense SAIs. Moreover, in order to improve the quality of the predicted SAI, we add decoder-side quality enhancement before prediction. We propose a multi-scale dense residual network (MSDRN) to implement both EPI super-resolution and quality enhancement. Transfer learning strategies are used to improve the training performance of quality enhancement. The experimental results show the superior performance of the proposed framework over existing methods in terms of rate-distortion performance.

A Multiscale Finite Element Method for the Schrödinger Equation with Multiscale Potentials

In recent years, an increasing attention has been paid to quantum heterostructures with tailored functionalities, such as heterojunctions and quantum metamaterials, in which quantum dynamics of electrons can be described by the Schrödinger equation with multiscale potentials. The model, however, cannot be solved by asymptotic-based approaches where an additive form of different scales in the potential term is required to construct the prescribed approximate solutions. In this paper, we propose a multiscale finite element method to solve this problem in the semiclassical regime. The localized multiscale basis functions are constructed using sparse compression of the Hamiltonian operator and thus are “blind" to the specific form of the potential term. After an one-shot eigendecomposition, we solve the resulting system of ordinary differential equations explicitly for the time evolution. In our approach, the spatial mesh size is $ H=\mathcal{O}(\epsilon)$, where $\epsilon$ is the semiclassical parameter and the time stepsize $ k$ is independent of $\epsilon$. Numerical examples in one dimension with a periodic potential, a multiplicative two-scale potential, and a layered potential, and in two dimension with an additive two-scale potential and a checkboard potential are tested to demonstrate the robustness and efficiency of the proposed method. Moreover, first-order and second-order rates of convergence are observed in $H^1$ and $L^2$ norms, respectively.

A sparse matrix‐vector multiplication method with low preprocessing cost

SummarySparse matrix‐vector multiplication (SpMV) is a crucial operation used for solving many engineering and scientific problems. In general, there is no single SpMV method that gives high performance for all sparse matrices. Even though there exist sparse matrix storage formats and SpMV implementations that yield high efficiency for certain matrix structures, using these methods may entail high preprocessing or format conversion costs. In this work, we present a new SpMV implementation, named CSRLenGoto, that can be utilized by preprocessing the Compressed Sparse Row (CSR) format of a matrix. This preprocessing phase is inexpensive enough for the associated cost to be compensated in just a few repetitions of the SpMV operation. CSRLenGoto is based on complete loop unrolling and gives performance improvements in particular for matrices whose mean row length is low. We parallelized our method by integrating it into a state‐of‐the‐art matrix partitioning approach as the kernel operation. We observed up to 2.46× and on the average 1.29× speedup with respect to Intel MKL's SpMV function for matrices with short‐ or medium‐length rows.

Sparse operator compression of higher-order elliptic operators with rough coefficients

We introduce the sparse operator compression to compress a self-adjoint higher-order elliptic operator with rough coefficients and various boundary conditions. The operator compression is achieved by using localized basis functions, which are energy minimizing functions on local patches. On a regular mesh with mesh size h, the localized basis functions have supports of diameter O(hlog (1/h)) and give optimal compression rate of the solution operator. We show that by using localized basis functions with supports of diameter O(hlog (1/h)), our method achieves the optimal compression rate of the solution operator. From the perspective of the generalized finite element method to solve elliptic equations, the localized basis functions have the optimal convergence rate O(h^k) for a (2k)th-order elliptic problem in the energy norm. From the perspective of the sparse PCA, our results show that a large set of Matérn covariance functions can be approximated by a rank-n operator with a localized basis and with the optimal accuracy.

Multigrid with Rough Coefficients and Multiresolution Operator Decomposition from Hierarchical Information Games

We introduce a near-linear complexity (geometric and meshless/algebraic) multigrid/multiresolution method for PDEs with rough ($L^\infty$) coefficients with rigorous a priori accuracy and performance estimates. The method is discovered through a decision/game theory formulation of the problems of (1) identifying restriction and interpolation operators, (2) recovering a signal from incomplete measurements based on norm constraints on its image under a linear operator, and (3) gambling on the value of the solution of the PDE based on a hierarchy of nested measurements of its solution or source term. The resulting elementary gambles form a hierarchy of (deterministic) basis functions of $H^1_0(\Omega)$ (gamblets) that (1) are orthogonal across subscales/subbands with respect to the scalar product induced by the energy norm of the PDE, (2) enable sparse compression of the solution space in $H^1_0(\Omega)$, and (3) induce an orthogonal multiresolution operator decomposition. The operating diagram of the multig...