Compression Framework Research Articles

Compressive Wavelet Domain Deep CNN for Image Classification Using Genetic Algorithm Based Sensing Mask Learning

Using a novel Genetic Algorithm-based Compressive Learning (GACL), a compressed domain-learning framework is proposed that is implemented on the Haar wavelet approximation coefficient images of the standard kaggle RGB cat dog dataset with every images resized to 256x256x3. The compressive sensing (CS) measurements on the selected dataset is achieved by using a simple reduced pixel scheme by retaining only P% of the pixels of the approximation coefficient images and forcing the remaining pixels to 0 using the Primitive Walsh Hadamard (PWH) binary mask and the masked images are used for further learning. A numerical experiment is conducted to analyze the image classification performance of deep convolution neural network (DCNN) learning on compressive sensing (CS) measurements of wavelet approximation coefficient image of the selected dataset. The unmasked wavelet approximation coefficients images are of size only one fourth of the original image, but they visually resembles the original image. The numerical experiment shows that when learning is done on this unmasked wavelet approximation coefficient images a training accuracy of 97% and validation accuracy of 77% are achieved which is remarkable and as good as using the complete spatial domain image. It is found from numerical experiment that, when PWH masking with P=10 is applied only to the test images the validation accuracy falls up to 58% and this fall is due to the fact that the training is done on unmasked images and tested on masked images. On the other hand in a compressive learning framework the DCNN is trained using masked images and when tested using masked images the validation accuracy rises up to 62% due to the fact both the trained images and test images are masked. Further, it is demonstrated that the best PWH masks can be learned by GACL in which the training accuracy increases up to 89% when vertical cross over is used in GACL and increases up to 96% when diagonal crossover is used in GACL. in this case of GACL using diagonal crossover, the 96% of training accuracy using only (10/4=2.5) 2.5% of the image is very remarkable and stand as proof of concept to implement compressive learning framework which need very less pixel thus very less measurement rate (MR) both in training and testing phase minimizing the bandwidth and storing requirement in applications related to IoT and cloud solutions. The average SSIM ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S_avg</i> )and average PSNR ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PSNR_avg</i> )are used as quality measurements, which reduce as P reduces, and it is demonstrated that the average SSIM and average PSNR improves when GACL is used to learn the mask, which is the key reason for the performance improvement.

A Super-Resolution-Based Feature Map Compression for Machine-Oriented Video Coding

Recently, video and image compression methods using neural networks have received much attention. In MPEG standardization, Video Coding for Machine (VCM) is a newly arising topic which attempts to compress features/images for the purpose of machine vision tasks. Especially, compressing features has advantages in terms of privacy protection and computation off-loading. In this paper, we propose an effective feature compression method equipped with a super-resolution (SR) module for features. Our main motivation comes from the observation that features are somewhat robust to spatial distortions (e.g., AWGN, blur, quantization distortions, coding artifacts), which leads us to integrating an SR module into the compression framework. We also further explore the best training strategy of the proposed method, i.e., finding the best combination of various losses and proper input feature shapes. Our comprehensive experiments show that the proposed method outperforms the baseline in the original VCM anchor scenario on various QP values with Versatile Video Coding (VVC). Specifically, the proposed framework achieved up to 50% BD-rate reduction compared to the conventional P-layer feature map compression method for the object detection task on the OpenImage dataset.

An overhead-free region-based JPEG framework for task-driven image compression

An overhead-free region-based JPEG framework for task-driven image compression

3D Holoscopic Image Compression Based on Gaussian Mixture Model

We introduce a Gaussian Mixture Model (GMM) framework for 3D holoscopic image compression in this paper. The elemental-images of the 3D holoscopic image are predicted using GMM and the parameters of GMM are estimated using the common Expectation-Maximization (EM) algorithm. GMM Model Optimization (GMO) is used in this framework to select the optimal number of distributions and avoid local optimum of EM at the same time. A three-dimensional distribution-rotation based decomposition is proposed to change covariance parameters to meaningful features and improve the coding efficiency. The features and the remaining parameters of the GMM are encoded using fixed-length bits. A feature-based dictionary is proposed in this framework to match the similar Gaussian distributions utilizing the similar GMM features. And the offsets of the matched distributions are recorded as motion vectors to replace the similar areas in the elemental-images of the 3D holoscopic image. The residual between the original image and the prediction is encoded using Screen Content Coding Extension of High Efficiency Video Coding (HEVC-SCC). Experimental results show that our method performs better than HEVC-SCC, two coding methods based on pseudo-sequences and a state-of-the-art content-based compression method with Gaussian process regression.

A framework for lossy compression and recovery of battery cell voltage

High-quality historical data of battery is important for state estimation and management. However, limited by bandwidth of 4G network and storage capacity, the cloud only receives low-frequency (LF) data and a few high-frequency (HF) signals. This paper proposes a model-based method to transfer adequate battery data between vehicle and cloud at a low cost. Firstly, a training dataset is built from real-world vehicle data. Then, a multi-task learning (MTL) model under semi-supervised learning (SSL) framework is proposed to learn HF voltage representation of each battery cell. SSL framework uses a large number of unlabeled low-frequency voltages to improve the voltage frequency recovery accuracy. MTL framework enables the model to focus on battery aging and solve the problem of target domain drifting over time. Finally, both real-world vehicle and experimental data are used to compare the results of different methods on the voltage recovering task. The results show that the proposed method can reduce the average voltage recovery error to less than 7mV at various driving conditions.

Light Field Compression With Graph Learning and Dictionary-Guided Sparse Coding

Light field (LF) data are widely used in the immersive representations of the 3D world. To record the light rays along with different directions, an LF requires much larger storage space and transmission bandwidth than a conventional 2D image with similar spatial dimension. In this paper, we propose a novel framework for light field image compression that leverages graph learning and dictionary learning to remove structural redundancies between different views. Specifically, to significantly reduce the bit-rates, only a few key views sampled and encoded, whereas the remaining non-key views are reconstructed via the graph adjacency matrix learned from the angular patch. Furthermore, dictionary-guided sparse coding is developed to compress the graph adjacency matrices and reduce the coding overheads. To the best of our knowledge, this paper is the first to achieve compact representation of cross-view structural information via adaptive learning on graphs. Experimental results demonstrate that the proposed framework achieves better performance than the standardized HEVC-based codec.

A General Framework for Progressive Data Compression and Retrieval.

In scientific simulations, observations, and experiments, the transfer of data to and from disk and across networks has become a major bottleneck for data analysis and visualization. Compression techniques have been employed to tackle this challenge, but traditional lossy methods often demand conservative error tolerances to meet the numerical accuracy requirements of both anticipated and unknown data analysis tasks. Progressive data compression and retrieval has emerged as a promising solution, where each analysis task dictates its own accuracy needs. However, few analysis algorithms inherently support progressive data processing, and adapting compression techniques, file formats, client/server frameworks, and APIs to support progressivity can be challenging. This paper presents a framework that enables progressive-precision data queries for any data compressor or numerical representation. Our strategy hinges on a multi-component representation that successively reduces the error between the original and compressed field, allowing each field in the progressive sequence to be expressed as a partial sum of components. We have implemented this approach with four established scientific data compressors and assessed its effectiveness using real-world data sets from the SDRBench collection. The results show that our framework competes in accuracy with the standalone compressors it is based upon. Additionally, (de)compression time is proportional to the number of components requested by the user. Finally, our framework allows for fully lossless compression using lossy compressors when a sufficient number of components are employed.

Learned Structure-Based Hybrid Framework for Martian Image Compression

Learned Structure-Based Hybrid Framework for Martian Image Compression

Learned Image Compression Using Cross-Component Attention Mechanism.

Learned image compression methods have achieved satisfactory results in recent years. However, existing methods are typically designed for RGB format, which are not suitable for YUV420 format due to the variance of different formats. In this paper, we propose an information-guided compression framework using cross-component attention mechanism, which can achieve efficient image compression in YUV420 format. Specifically, we design a dual-branch advanced information-preserving module (AIPM) based on the information-guided unit (IGU) and attention mechanism. On the one hand, the dual-branch architecture can prevent changes in original data distribution and avoid information disturbance between different components. The feature attention block (FAB) can preserve the important information. On the other hand, IGU can efficiently utilize the correlations between Y and UV components, which can further preserve the information of UV by the guidance of Y. Furthermore, we design an adaptive cross-channel enhancement module (ACEM) to reconstruct the details by utilizing the relations from different components, which makes use of the reconstructed Y as the textural and structural guidance for UV components. Extensive experiments show that the proposed framework can achieve the state-of-the-art performance in image compression for YUV420 format. More importantly, the proposed framework outperforms Versatile Video Coding (VVC) with 8.37% BD-rate reduction on common test conditions (CTC) sequences on average. In addition, we propose a quantization scheme for context model without model retraining, which can overcome the cross-platform decoding error caused by the floating-point operations in context model and provide a reference approach for the application of neural codec on different platforms.

CBANet: Toward Complexity and Bitrate Adaptive Deep Image Compression Using a Single Network.

In this work, we propose a new deep image compression framework called Complexity and Bitrate Adaptive Network (CBANet) that aims to learn one single network to support variable bitrate coding under various computational complexity levels. In contrast to the existing state-of-the-art learning-based image compression frameworks that only consider the rate-distortion trade-off without introducing any constraint related to the computational complexity, our CBANet considers the complex rate-distortion-complexity trade-off when learning a single network to support multiple computational complexity levels and variable bitrates. Since it is a non-trivial task to solve such a rate-distortion-complexity related optimization problem, we propose a two-step approach to decouple this complex optimization task into a complexity-distortion optimization sub-task and a rate-distortion optimization sub-task, and additionally propose a new network design strategy by introducing a Complexity Adaptive Module (CAM) and a Bitrate Adaptive Module (BAM) to respectively achieve the complexity-distortion and rate-distortion trade-offs. As a general approach, our network design strategy can be readily incorporated into different deep image compression methods to achieve complexity and bitrate adaptive image compression by using a single network. Comprehensive experiments on two benchmark datasets demonstrate the effectiveness of our CBANet for deep image compression. Code is released at https://github.com/JinyangGuo/CBANet-release.

Slimmable Multi-Task Image Compression for Human and Machine Vision

In the Internet of Things (IoT) communications, visual data is frequently processed among intelligent devices using artificial intelligence algorithms, replacing humans for analyzing and decision-making while only occasionally requiring human’s scrutiny. However, due to high redundancy of compressive encoders, existing image coding solutions for machine vision are not efficient at runtime. To balance the rate-accuracy performance and efficiency of image compression for machine vision while attaining high-quality reconstructed images for human vision, this paper introduces a novel slimmable multi-task compression framework for human and machine vision in visual IoT applications. Firstly, the image compression for human and machine vision under the constraint of bandwidth, latency, computational resources are modelled as a multi-task optimization problem. Secondly, slimmable encoders are employed to multiple human and machine vision tasks in which the parameters of the sub-encoder for machine vision tasks are shared among all tasks and jointly learned. Thirdly, to solve the feature match between latent representation and intermediate features of deep vision networks, feature transformation networks are introduced as decoders of machine vision feature compression. Finally, the proposed framework is successfully applied to human and machine vision tasks’ scenarios, e.g., object detection and image reconstruction. Experimental results show that the proposed method outperforms baselines and other image compression approaches on machine vision tasks with higher efficiency (shorter latency) in two vision tasks’ scenarios while retaining comparable quality on image reconstruction.

Task-Driven Video Compression for Humans and Machines: Framework Design and Optimization

Learned video compression has developed rapidly and achieved impressive progress in recent years. Despite efficient compression performance, existing signal fidelity oriented or semantic fidelity oriented video compression methods limit the capability to meet the requirements of both machine and human vision. To address this problem, a task-driven video compression framework is proposed to flexibly support vision tasks for both human vision and machine vision. Specifically, to improve the compression performance, the backbone of the video compression framework is optimized by using three novel modules, including multi-scale motion estimation, multi-frame feature fusion, and reference based in-loop filters. Then, based on the proposed efficient compression backbone, a task-driven optimization approach is designed to achieve the trade-off between signal fidelity oriented compression and semantic fidelity oriented compression. Moreover, a post-filter module is employed for the framework to further improve the performance of the human vision branch. Finally, rate-distortion performance, rate-accuracy performance, and subjective quality are employed as the evaluation metrics, and experimental results show the superiority of the proposed framework for both human vision and machine vision. The source code of this work can be found in <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://mic.tongji.edu.cn</uri> .

Enhanced Motion Compensation for Deep Video Compression

Most of the existing deep learning-based video compression frameworks rely on motion estimation and compensation. However, the artifacts of the warped frames after motion compensation, which propagate the errors to the next frame, limit the video coding performance. In this work, we propose enhanced motion compensation for reduced error propagation in deep video compression. More specifically, we incorporate the designed convolutional neural network into Open DVC as the motion compensation enhancement network to remove noise in the predicted frame. With the enhanced frame, we jointly optimize the whole framework with a single loss function by considering the trade-off between bit cost and frame quality. Experiments show that the proposed enhanced motion compensation model reduces error propagation within a group of frames. Compared with Open DVC, our model can achieve 8.94% bit savings on average for standard test videos in terms of PSNR. Regarding MS-SSIM, our model outperforms Open DVC with 5.67% bit rate savings.

Rate-Distortion Optimized Geometry Compression for Spinning LiDAR Point Cloud

Point cloud is a major representation format of 3D objects and scenes. It has been increasingly applied in various applications due to the rapid advances in 3D sensing and rendering technologies. In the field of autonomous driving, point clouds captured by spinning Light Detection And Ranging (LiDAR) devices have become an informative data source for road environment perception and intelligent vehicle control. On the other hand, the massive data volume of point clouds also brings huge challenges to point cloud transmission and storage. Therefore, establishing compression frameworks and algorithms that conform to the characteristics of point cloud data has become an important research topic for both academia and industry. In this paper, a geometry compression method dedicated to spinning LiDAR point cloud was proposed taking advantage of the prior information of the LiDAR acquisition procedure. Rate-distortion optimizations were further integrated into the coding pipeline according to the characteristics of the prediction residuals. Experimental results obtained on different datasets show that the proposed method consistently outperforms the state-of-the-art G-PCC predictive geometry coding method with reduced runtime at both the encoder and decoder sides.

JPEG 2000 Extensions for Scalable Coding of Discontinuous Media.

In this paper we propose novel extensions to JPEG 2000 for the coding of discontinuous media which includes piecewise smooth imagery such as depth maps and optical flows. These extensions use breakpoints to model discontinuity boundary geometry and apply a breakpoint dependent Discrete Wavelet Transform (BP-DWT) to the input imagery. The highly scalable and accessible coding features provided by the JPEG 2000 compression framework are preserved by our proposed extensions, with the breakpoint and transform components encoded as independent bit streams that can be progressively decoded. Comparative rate-distortion results are provided along with corresponding visual examples which highlight the advantages of using breakpoint representations with accompanying BD-DWT and embedded bit-plane coding. Recently our proposed extensions have been adopted and are in the process of being published as a new Part 17 to the JPEG 2000 family of coding standards.

Simultaneously Training and Compressing Vision-and-Language Pre-Training Model

Model compression is an essential step for large-scale pre-training models toward practical application and deployment on the edge device. However, when conventional compression methods following ‘pre-training then compressing’ two-phase pipeline are applied to Vision-and-Language Pre-training (VLP) models, it will lead to a high calculation and memory overhead. In this work, we break the two-phase pipeline and propose an efficient and effective one-phase VLP model compression mechanism, named <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">REDUCER</b> , which stands for ‘simultaneously training and comp <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RE</b> ssing’ VLP model via progressive mo <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DU</b> le repla <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</b> ing and n <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</b> twork <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</b> ewiring. Specifically, REDUCER consists of three insightful designs. Firstly, we design a one-phase compression framework to train and compress the VLP model simultaneously to avoid the extra calculation and memory cost caused by an isolated model compression phase in the conventional two-phase pipeline. Secondly, we propose an adaptive progressive module replacing mechanism to compress the model depth free from explicit knowledge distillation losses, relieving the multi-task optimization problems. Thirdly, we integrate pruning techniques into VLP model compression to simultaneously compress the model in width and depth. Overall, we obtain a lightweight VLP model with only one pre-training phase, and it is the first one-phase compression method for VLP models. Extensive experiments have been conducted on representative VLP models, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , ClipBERT and VICTOR, and the experimental results show a superior trade-off between performance and efficiency.

Detection and prevention of SQLI attacks and developing compressive framework using machine learning and hybrid techniques

A web application is a software system that provides an interface to its users through a web browser on any operating system (OS). Despite their growing popularity, web application security threats have become more diverse, resulting in more severe damage. Malware attacks, particularly SQLI attacks, are common in poorly designed web applications. This vulnerability has been known for more than two decades and is still a source of concern. Accordingly, different techniques have been proposed to counter SQLI attacks. However, the majority of them either fail to cover the entire scope of the problem. The structured query language injection (SQLI) attack is among the most harmful online application attacks and often happens when the attacker(s) alter (modify), remove (delete), read, and copy data from database servers. All facets of security, including confidentiality, data integrity, and data availability, can be impacted by a successful SQLI attack. This paper investigates common SQLI attack forms, mechanisms, and a method of identifying, detecting, and preventing them based on the existence of the SQL query. Here, we have developed a comprehensive framework for detecting and preventing the effectiveness of techniques that address specific issues following the essence of the SQLI attacks by using traditional Navies Bayes (NB), Decision Trees (DT), Support Vectors Machine (SVM), Random Forests (RF), Logistic Regression (LR), and Neural Networks Based on Multilayer Perceptron (MLP), and hybrid approach are used for our study. The machine learning (ML) algorithms were implemented using the Keras library, while the classical methods were implemented using the Tensor Flow-Learn package. For this proposed research work, we gathered 54,306 pieces of data from weblogs, cookies, session usage, and from HTTP (S) request files to train and test our model. The performance evaluation results for training set in metrics such as the hybrid approach (ANN and SVM) perform better accuracies in precision (99.05% and 99.54%), recall (99.65% and 99.61%), f1-score (99.35% and 99.57%), and training set (99.20% and 99.60%) respectively than other ML approaches. However, their training time is too high (i.e., 19.62 and 26.16 s respectively) for NB and RF. Accordingly, the NB technique performs poorly in accuracy, precision, recall, f1-score, training set evaluation metrics, and best in training time. Additionally, the performance evaluation results for test set in metrics such as hybrid approach (ANN and SVM) perform better accuracies in precision (98.87% and 99.20%), recall (99.13% and 99.47%), f1-score (99.00% and 99.33%) and test set (98.70% and 99.40%) respectively than other ML approaches. However, their test time is too high (i.e., 11.76 and 15.33 ms respectively). Accordingly, the NB technique performs poorly in accuracy, precision, recall, f1-score, test set evaluation metrics, and best in training time. Here, among the implemented ML techniques, SVM and ANN are weak learners. The achieved performance evaluation results indicated that the proposed SQLI attack detection and prevention mechanism has been improved over the previously implemented techniques in the theme. Finally, in this paper, we aimed to keep researchers up-to-date, with contributions, and recommendations to the understanding of the intersection between SQLI attacks and prevention in the artificial intelligence (AI) field.

Toward Quantity-of-Interest Preserving Lossy Compression for Scientific Data

Today's scientific simulations and instruments are producing a large amount of data, leading to difficulties in storing, transmitting, and analyzing these data. While error-controlled lossy compressors are effective in significantly reducing data volumes and efficiently developing databases for multiple scientific applications, they mainly support error controls on raw data, which leaves a significant gap between the data and user's downstream analysis. This may cause unqualified uncertainties in the outcomes of the analysis, a.k.a quantities of interest (QoIs), which are the major concerns of users in adopting lossy compression in practice. In this paper, we propose rigorous mathematical theories to preserve four families of QoIs that are widely used in scientific analysis during lossy compression along with practical implementations. Specifically, we first develop the error control theory for univariate QoIs which are essential for computing physical properties such as kinetic energy, followed by multivariate QoIs that are more commonly used in real-world applications. The proposed method is integrated into a state-of-the-art compression framework in a modular fashion, which could easily adapt to new QoIs and new compression algorithms. Experiments on real-world datasets demonstrate that the proposed method provides faithful error control on important QoIs including kinetic energy, regional average, and isosurface without trials and errors, while offering compression ratios that are up to 4X of the compression ratios provided by state-of-the-art compressors.

3D-Pruning: A Model Compression Framework for Efficient 3D Action Recognition

The existing end-to-end optimized 3D action recognition methods often suffer from high computational costs. Observing that different frames and different points in point cloud sequences often have different importance values for the 3D action recognition task, in this work, we propose a fully automatic model compression framework called 3D-Pruning (3DP) for efficient 3D action recognition. After performing model compression by using our 3DP framework, the compressed model can process different frames and different points in each frame by using different computational complexities based on their importance values, in which both the importance value and computational complexity for each frame/point can be automatically learned. Extensive experiments on five benchmark datasets demonstrate the effectiveness of our 3DP framework for model compression.

GAN Compression: Efficient Architectures for Interactive Conditional GANs.

Conditional Generative Adversarial Networks (cGANs) have enabled controllable image synthesis for many vision and graphics applications. However, recent cGANs are 1-2 orders of magnitude more compute-intensive than modern recognition CNNs. For example, GauGAN consumes 281G MACs per image, compared to 0.44G MACs for MobileNet-v3, making it difficult for interactive deployment. In this work, we propose a general-purpose compression framework for reducing the inference time and model size of the generator in cGANs. Directly applying existing compression methods yields poor performance due to the difficulty of GAN training and the differences in generator architectures. We address these challenges in two ways. First, to stabilize GAN training, we transfer knowledge of multiple intermediate representations of the original model to its compressed model and unify unpaired and paired learning. Second, instead of reusing existing CNN designs, our method finds efficient architectures via neural architecture search. To accelerate the search process, we decouple the model training and search via weight sharing. Experiments demonstrate the effectiveness of our method across different supervision settings, network architectures, and learning methods. Without losing image quality, we reduce the computation of CycleGAN by 21×, Pix2pix by 12×, MUNIT by 29×, and GauGAN by 9×, paving the way for interactive image synthesis.