Deep Compression Research Articles

Joint Graph Attention and Asymmetric Convolutional Neural Network for Deep Image Compression

Recent deep image compression methods have achieved prominent progress by using nonlinear modeling and powerful representation capabilities of neural networks. However, most existing learning-based image compression approaches employ customized convolutional neural network (CNN) to utilize visual features by treating all pixels equally, neglecting the effect of local key features. Meanwhile, the convolutional filters in CNN usually express the local spatial relationship within the receptive field and seldom consider the long-range dependencies from distant locations. This results in the long-range dependencies of latent representations not being fully compressed. To address these issues, an end-to-end image compression method is proposed by integrating graph attention and asymmetric convolutional neural network (ACNN). Specifically, ACNN is used to strengthen the effect of local key features and reduce the cost of model training. Graph attention is introduced into image compression to address the bottleneck problem of CNN in modeling long-range dependencies. Meanwhile, regarding the limitation that existing attention mechanisms for image compression hardly share information, we propose a self-attention approach which allows information flow to achieve reasonable bit allocation. The proposed self-attention approach is in compliance with the perceptual characteristics of human visual system, as information can interact with each other via attention modules. Moreover, the proposed self-attention approach takes into account channel-level relationship and positional information to promote the compression effect of rich-texture regions. Experimental results demonstrate that the proposed method achieves state-of-the-art rate-distortion performances after being optimized by MS-SSIM compared to recent deep compression models on the benchmark datasets of Kodak and Tecnick. The project page with the source code can be found in <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://mic.tongji.edu.cn</uri> .

Sensory and emotional responses to deep pressure stimulation at myofascial trigger points: a pilot study.

Although manual pressure, such as that used during a massage, is often associated with pain, it can simultaneously be perceived as pleasant when applied to certain body areas. We hypothesized that stimulation of myofascial trigger points (TPs) leads to simultaneous pain and pleasure. TPs are hyperirritable points located in the taut band of the skeletal muscle. In this study, we measured the muscle tone, muscle stiffness, and pressure pain threshold of TPs and control points in the left brachioradialis muscle of 48 healthy participants. We also applied deep compression to the two points and collected subjective data on pain, pleasantness, unpleasantness, and relief. Greater muscle stiffness was observed in the TPs versus control points (t = 6.55, p < 0.001), and the pain threshold was significantly lower in the TPs (t = -6.21, p < 0.001). Unpleasantness ratings after deep compression were significantly lower in the TPs compared with control points (t = -2.68, p < 0.05). Participants experienced greater relief at the TPs compared with control points (t = 2.01, p < 0.05), although the perceived pain did not differ between the two types of points. We compared the properties of TPs and control points, and found that deep compression at TPs was associated with higher muscle tone and stiffness, lower unpleasantness ratings, and higher relief ratings compared with the control points. These findings suggest that, at least for some TPs, pain and pleasantness are simultaneously elicited by deep pressure stimulation.

CBANet: Toward Complexity and Bitrate Adaptive Deep Image Compression Using a Single Network.

In this work, we propose a new deep image compression framework called Complexity and Bitrate Adaptive Network (CBANet) that aims to learn one single network to support variable bitrate coding under various computational complexity levels. In contrast to the existing state-of-the-art learning-based image compression frameworks that only consider the rate-distortion trade-off without introducing any constraint related to the computational complexity, our CBANet considers the complex rate-distortion-complexity trade-off when learning a single network to support multiple computational complexity levels and variable bitrates. Since it is a non-trivial task to solve such a rate-distortion-complexity related optimization problem, we propose a two-step approach to decouple this complex optimization task into a complexity-distortion optimization sub-task and a rate-distortion optimization sub-task, and additionally propose a new network design strategy by introducing a Complexity Adaptive Module (CAM) and a Bitrate Adaptive Module (BAM) to respectively achieve the complexity-distortion and rate-distortion trade-offs. As a general approach, our network design strategy can be readily incorporated into different deep image compression methods to achieve complexity and bitrate adaptive image compression by using a single network. Comprehensive experiments on two benchmark datasets demonstrate the effectiveness of our CBANet for deep image compression. Code is released at https://github.com/JinyangGuo/CBANet-release.

Comprehensive Comparisons of Uniform Quantization in Deep Image Compression

In deep image compression, uniform quantization is applied to latent representations obtained by using an auto-encoder architecture for reducing bits and entropy coding. Quantization is a problem encountered in the end-to-end training of deep image compression. Quantization’s gradient is zero, and it cannot backpropagate meaningful gradients. Many methods have been proposed to address the approximations of quantization to obtain gradients. However, there have not been equitable comparisons among them. In this study, we comprehensively compare the existing approximations of uniform quantization. Furthermore, we evaluate possible combinations of quantizers for the decoder and the entropy model, as the approximated quantizers can be different for them. We conduct experiments using three network architectures on two test datasets. The experimental results reveal that the best approximated quantization differs by the network architectures, and the best approximations of the three are different from the original ones used for the architectures. We also show that the combination of quantizers that uses universal quantization for the entropy model and differentiable soft quantization for the decoder is a comparatively good choice for different architectures and datasets.

Enhanced Motion Compensation for Deep Video Compression

Most of the existing deep learning-based video compression frameworks rely on motion estimation and compensation. However, the artifacts of the warped frames after motion compensation, which propagate the errors to the next frame, limit the video coding performance. In this work, we propose enhanced motion compensation for reduced error propagation in deep video compression. More specifically, we incorporate the designed convolutional neural network into Open DVC as the motion compensation enhancement network to remove noise in the predicted frame. With the enhanced frame, we jointly optimize the whole framework with a single loss function by considering the trade-off between bit cost and frame quality. Experiments show that the proposed enhanced motion compensation model reduces error propagation within a group of frames. Compared with Open DVC, our model can achieve 8.94% bit savings on average for standard test videos in terms of PSNR. Regarding MS-SSIM, our model outperforms Open DVC with 5.67% bit rate savings.

Progressive Deep Image Compression for Hybrid Contexts of Image Classification and Reconstruction

Progressive deep image compression (DIC) with hybrid contexts is an under-investigated problem that aims to jointly maximize the utility of a compressed image for multiple contexts or tasks under variable rates. In this paper, we consider the contexts of image reconstruction and classification. We propose a DIC framework, called residual-enhanced mask-based progressive generative coding (RMPGC), designed for explicit control of the performance within the rate-distortion-classification-perception (RDCP) trade-off. Three independent mechanisms are introduced to yield a semantically structured latent representation that can support parameterized control of rate and context adaptation. Experimental results show that the proposed RMPGC outperforms a benchmark DIC scheme using the same generative adversarial nets (GANs) backbone in all six metrics related to classification, distortion, and perception. Moreover, RMPGC is a flexible framework that can be applied to different neural network backbones. Some typical implementations are given and shown to outperform the classic BPG codec and four state-of-the-art DIC schemes in classification and perception metrics, with a slight degradation in distortion metrics. Our proposal of a nonlinear-neural-coded and richly structured latent space makes the proposed DIC scheme well suited for image compression in wireless communications, multi-user broadcasting, and multi-tasking applications.

Contrastive Distillation With Regularized Knowledge for Deep Model Compression on Sensor-Based Human Activity Recognition

Contrastive Distillation With Regularized Knowledge for Deep Model Compression on Sensor-Based Human Activity Recognition

Entropy-Constrained Implicit Neural Representations for Deep Image Compression

Implicit neural representations (INRs) for various data types have gained popularity in the field of deep learning owing to their effectiveness. However, previous studies on INRs have only focused on recovering original representations. This paper investigated an image compression model based on INRs using a model compression technique for entropy-constrained neural networks. Specifically, the proposed model trains a multilayer perceptron (MLP) to overfit a single image and then uses its weights to optimize its compressed representation using additive uniform noise. Accordingly, the proposed model efficiently minimizes the size of the model weight in an end-to-end manner. This training optimization process is fairly desirable for adjusting the rate of distortion for image compression. In contrast to other model compression techniques, the proposed model is implemented without additional training process or memory cost. By introducing entropy loss, this paper demonstrated that the proposed model can be used to preserve high image quality while maintaining smaller model size. The experimental results demonstrated that the proposed model achieved comparable performance to conventional image compression models without incurring high storage costs.

Learned Image Compression With Gaussian-Laplacian-Logistic Mixture Model and Concatenated Residual Modules.

Recently deep learning-based image compression methods have achieved significant achievements and gradually outperformed traditional approaches including the latest standard Versatile Video Coding (VVC) in both PSNR and MS-SSIM metrics. Two key components of learned image compression are the entropy model of the latent representations and the encoding/decoding network architectures. Various models have been proposed, such as autoregressive, softmax, logistic mixture, Gaussian mixture, and Laplacian. Existing schemes only use one of these models. However, due to the vast diversity of images, it is not optimal to use one model for all images, even different regions within one image. In this paper, we propose a more flexible discretized Gaussian-Laplacian-Logistic mixture model (GLLMM) for the latent representations, which can adapt to different contents in different images and different regions of one image more accurately and efficiently, given the same complexity. Besides, in the encoding/decoding network design part, we propose a concatenated residual blocks (CRB), where multiple residual blocks are serially connected with additional shortcut connections. The CRB can improve the learning ability of the network, which can further improve the compression performance. Experimental results using the Kodak, Tecnick-100 and Tecnick-40 datasets show that the proposed scheme outperforms all the leading learning-based methods and existing compression standards including VVC intra coding (4:4:4 and 4:2:0) in terms of the PSNR and MS-SSIM. The source code is available at https://github.com/fengyurenpingsheng.

Paper] Hybrid Spatial and Deep Learning-based Point Cloud Compression with Layered Representation on 3D Shape

Paper] Hybrid Spatial and Deep Learning-based Point Cloud Compression with Layered Representation on 3D Shape

Knowledge Distillation With Feature Self Attention

With the rapid development of deep learning technology, the size and performance of the network continuously grow, making network compression essential for commercial applications. In this paper, we propose a Feature Self Attention (FSA) module that extracts correlation information between the hidden features of a network and a new method for distilling the correlation features to compress the model. FSA does not require a special module or network to match features between the teacher model and the student model. By removing the multi-head structure and the repeated self-attention blocks in the existing self-attention mechanism, it minimizes the addition of parameters. Based on ResNet-18, 34, the added parameters are only 2.00M and the training speed is also the fastest in comparison to benchmark models. It was demonstrated through experiments that the use of interrelationship loss between features can be beneficial for training student models, indicating the importance of considering correlation information in deep neural network compression. And it was verified through training from scratch on the vanilla without the pre-trained weight of the student model.

Deep Compressive Imaging With Meta-Learning

Recently, deep learning-based compressive synthetic aperture radar (SAR) imaging has received increasing interests. However, its performances rely heavily on the training data and could not well adapt to new observations. To solve it, in this article a robust compressive SAR imaging method is proposed under the paradigm of meta-learning. First, a Deep Skip-connected Convolutional Network (DSCN) is constructed, to learn the mapping from Low-Resolution (LR) radar returns to High-Resolution (HR) radar image of the same scene. Then a set of meta tasks are sampled from one or multiple radar systems, to guide the search of DSCN and knowledge transfer to new compressive imaging tasks. This method is non-iterative and does not require any information of radar system. Moreover, it can work for compressive imaging in both range and azimuth. Extensive experiments are taken and the results show that the proposed meta-learning-based DSCN (ML-DSCN) can achieve accurate and rapid resolution enhancement, and is superior to its counterparts in terms of reconstruction speed, robustness, and generalization.

Cardiopulmonary resuscitation: difficulty in maintaining sufficient compression depth at the appropriate rate

Over the last 50 years, the recommended chest compression for cardiopulmonary resuscitation (CPR) has become faster and deeper, but maintaining deep compressions may be difficult at higher rates. Our study aimed to determine whether adequate compression (chest compression at an appropriate depth and rate) is being performed in emergency departments (ED). We also investigated the effect of adequate compression performance on the return of spontaneous circulation (ROSC). This prospective observational study was conducted at the EDs of two urban academic medical centers. We included adult patients (age ≥18 years) with cardiac arrest who underwent CPR in the ED between May and November 2020. We excluded patients with cardiac arrest related to trauma, repeated arrest except the first, and those for whom a monitor-defibrillator (ZOLL X-series) was not used. The following data were obtained from the monitor-defibrillator devices: compression depth, rate, chest compression fraction, CPR time, and percentage of compressions at the recommended rate and, at the recommended depth, at over and below rates, and depth, and at the appropriate depth and rate. Our study included 50 patients, from whom 441 chest compression sequences were obtained and analyzed. The mean compression depth, rate, and fraction were 6.48 ± 0.87 cm, 117 ± 5/min, 92.1 ± 3.70%, respectively. As the compression rate increased, the depth decreased, and most compressions were over-depth. Adequate compression (appropriate depth at recommended rate) was observed in 97 of the 441 compression sequences (21.9%). Below-depth and below-rate percentages were higher in the deceased group than that in the ROSC group (9.7 ± 15.2% vs. 3.3 ± 3.5%, p = 0.27; 2.7 ± 2.6% vs. 1.2± 0.9%, p = 0.06). The global ratio of chest compression showed low compliance with the recommended rate and depth, even when performed by skilled ED staff.

ECG Signals Deep Compressive Sensing Framework Based on Multiscale Feature Fusion and SE Block

ECG Signals Deep Compressive Sensing Framework Based on Multiscale Feature Fusion and SE Block

Optimized video compression with residual split attention and swin-block artifact contraction

Research in video compression has seen significant advancement in the last several years. However, the existing deep learning-based algorithms continue to be plagued by erroneous motion compression and ineffective motion compensation architectures, resulting in compression errors with a lower rate–distortion trade-off. To overcome these challenges, we present an end-to-end purely deep learning-based video compression method through a set of primary operations (e.g., motion estimation, motion compression, motion compensation, residual compression, and artifact contraction) differently. A deep residual attention split (DRAS) block is introduced for motion compression networks to pay more attention to certain image regions to create more effective features for the decoder while boosting the rate–distortion optimization (RDO) efficiency. A channel residual block (CRB) is proposed in motion compensation to yield a more accurate predicted frame, potentially improving the residual frame. To mitigate the compression errors, an artifact contraction module (ACM) by residual swin convolution UNet block is included in this model to improve the reconstruction quality. To improve the final frame, a buffer is added to fine-tune the previous reference frames. These modules combine with a loss function by assessing the trade-off and enhancing the decoded video quality. A comprehensive ablation study demonstrates the effectiveness of the proposed blocks and modules for video compression. Experimental results show the competitive performance of the proposed method on four benchmark datasets.

Knowledge distillation: A novel approach for deep feature selection

High dimensional data in hyperspectral remote sensing leads to computational, analytical, and storage complexities. Dimensionality reduction serves as an efficient tool to remove redundant, irrelevant, and highly correlated features. Recently, deep learning approaches have received remarkable progress in hyperspectral data analysis. In this paper, a new end-to-end deep learning framework based on a teacher-student network inspired by knowledge distillation is proposed for deep feature selection. Initially, a complicated teacher deep neural network is employed on complex high dimensional data to learn its corresponding best low dimensional representation. Then, the knowledge from the network is transferred to a simple student network that performs feature selection. Hence, it eventually leads to deep neural network compression which is of prime concern in hyperspectral remote sensing. Limited studies have been carried out to explore the benefits of knowledge distillation on hyperspectral data. The proposed method could be employed to choose deep features for both supervised and unsupervised tasks. Experimental results reveal the performance of the proposed scheme using limited features. In comparison to 1D and simple autoencoder models, the 2D model based on convolutional autoencoder delivers greater classification accuracies, with a classification accuracy value of 96.15% for the Indian Pines dataset and 97.82% for the Pavia University dataset. A similar trend is reported with unsupervised learning as well. Furthermore, the proposed model has a low degree of sensitivity to parameter selection.

Feature distillation Siamese networks for object tracking

In recent years, Siamese-based trackers have shown remarkable improvement in visual tracking. The general trends are interested in making deeper and more complicated networks to pursue higher accuracy. However, these advances result in cumbersome trackers with respect to size and speed, which hinders the deployment of deep trackers on edge devices. Due to the tracking scenario complexity and temporal coherence, the backbone network of deep trackers emphasizes the target appearance information more. However, the appearance feature is sensitive to parameter variation, making the traditional deep model compression methods hard to compress a deep tracker. To bridge the gap between deep Siamese trackers and practical use, we propose a new feature distillation algorithm suitable for deep trackers in this paper. Firstly, motivated by the concept of divide-and-conquer, we formulate the feature distillation into a stepwise distillation problem and perform distillation on each minimum unit to relieve the hard-to-distill problem of appearance feature. Secondly, we reconstruct the student model into a combination of convolution kernels and a point-wise convolutional layer, which enables the student model to inherit all the parameters of the teacher model during initialization. Finally, we propose a 3-step warm-up training strategy to address the student model’s degradation and structural adaptation problems during training. Extensive experiments on eight benchmarks demonstrate that our proposed method compresses the fully convolutional Siamese Networks (SiamFC) and its variant Siamd and achieves leading tracking performance with only 2.1 MB model size while running at 225 fps.

Harmonic‐tuned high‐efficiency GaN power amplifier with precise AM‐AM and AM‐PM characteristics

AbstractAn optimal balance between high efficiency and high linearity is one of the main performance metrics of modern base stations to handle the deep compression of the power amplifier (PA) module. In principle, the plethora of amplitude‐to‐amplitude (AM‐AM) and amplitude‐to‐phase (AM‐PM) distortions are issues worth exploring. Therefore, this paper presents a linearized harmonic‐tuned PA that operates at 2.4 GHz. The presented PA utilizes a compact input matching network (IMN) and output MN (OMN) with shunt‐connected tunable resonant circuits augmented by the stabilization network, whose presence greatly reduces the transistor parasitics and high‐order effects. Hence, a joint embodiment of each other gives optimal fundamental impedance matching alongside harmonic terminations and precise AM/PM waveform properties. The measurement results have shown that the fabricated PA exhibits peak saturated output power of 40.9 dBm and peak drain efficiency (DE) of 67%. In addition, the AM‐AM and AM‐PM curves under continuous‐wave excitation yield a gain flatness of 0.5 dB over a 37.5 dBm power range and ±3.5° phase distortion, respectively, when the input power level is swept up to the saturation level of 40 dBm. When driven with a 5 MHz 64‐QAM OFDM signal and 6.9 dB power back‐off, the manufactured PA meets the adjacent channel leakage ratio specification of −30 dBc at an average output power of 34.5 dBm.

Research on Deep Compression Method of Expressway Video Based on Content Value

Aiming at the problem that the storage space and network bandwidth of expressway surveillance video are occupied largely due to data redundancy and sparse information, this paper proposes a deep compression method of expressway video depth based on content value. Firstly, the YOLOv4 algorithm is used to analyze the content value of the original video, extract video frames with vehicle information, and eliminate unintentional frames. An improved CNN is then designed by adding Feature Pyramids and the Inception module to accelerate the extraction and fusion of features at all levels and improve the performance of image classification and prediction. Finally, the whole model is integrated into HEVC encoder for compressing the preprocessed video. The experimental results show that at the expense of only a 5.96% increase of BD-BR, and only a 0.19 dB loss of BD-PSNR, the proposed method achieves a 64% compression ratio and can save 62.82% coding time compared with other classic data compression methods based on deep learning.

Medical Image Compression Based on Variational Autoencoder

With the rapid growth of medical image data, it has become a current research hotspot that how to realize the large amounts of the real-time upload and storage of medical images with limited network bandwidth and storage space. However, currently, medical image compression technology cannot perform joint optimization of rate (the degree of compression) and distortion (reconstruction effect). Therefore, this study proposed a medical image compression algorithm based on a variational autoencoder. This algorithm takes rate and distortion as the common optimization goal and uses the residual network module to directly transmit information, which alleviates the contradiction between improving the degree of compression and optimizing the reconstruction effect. At the same time, the algorithm also reduces image loss in the medical image compression process by adding the residual network. The experimental results show that, compared with the traditional medical image compression algorithm and the deep learning compression algorithm, the algorithm in this study has smaller distortion, better reconstruction effect, and can obtain higher quality medical images at the same compression rate.