Video Compression Methods Research Articles

Optimized video compression with residual split attention and swin-block artifact contraction

Research in video compression has seen significant advancement in the last several years. However, the existing deep learning-based algorithms continue to be plagued by erroneous motion compression and ineffective motion compensation architectures, resulting in compression errors with a lower rate–distortion trade-off. To overcome these challenges, we present an end-to-end purely deep learning-based video compression method through a set of primary operations (e.g., motion estimation, motion compression, motion compensation, residual compression, and artifact contraction) differently. A deep residual attention split (DRAS) block is introduced for motion compression networks to pay more attention to certain image regions to create more effective features for the decoder while boosting the rate–distortion optimization (RDO) efficiency. A channel residual block (CRB) is proposed in motion compensation to yield a more accurate predicted frame, potentially improving the residual frame. To mitigate the compression errors, an artifact contraction module (ACM) by residual swin convolution UNet block is included in this model to improve the reconstruction quality. To improve the final frame, a buffer is added to fine-tune the previous reference frames. These modules combine with a loss function by assessing the trade-off and enhancing the decoded video quality. A comprehensive ablation study demonstrates the effectiveness of the proposed blocks and modules for video compression. Experimental results show the competitive performance of the proposed method on four benchmark datasets.

Comparative analysis of compression algorithms for four-dimensional light fields

Objectives. The widespread use of systems for capturing light fields is due to the high quality of the reproduced image. This type of capture, although qualitatively superior to traditional methods to capturing volumetric images, generates a huge amount of data needed to reconstruct the original captured 4D light field. The purpose of the work is to consider traditional and extended to four-dimensional image compression algorithms, to perform a comparative analysis and determine the most suitable.Methods. Mathematical methods of signal processing and methods of statistical analysis are used.Results. Algorithms are compared and analyzed in relation to the compression of four-dimensional light fields using the PSNR metric. The selected evaluation criterion is affected not only by the dimension of the compression algorithm, but also by the distance of the baseline of the capture setting, since the difference between images increases with the distance between the optical centers of each camera matrix. Thus, for installations consisting of an array of machine vision cameras located on racks and placed in a room, the obvious choice would be to use conventional image compression methods. Furthermore, based on the assessment of the arbitrariness of video compression methods, it should be noted that the XVC algorithm remains undervalued, although its results are higher. Algorithm AV1 can be considered the next in order of importance. It has been established that the latest compression algorithms show higher performance if compared to their predecessors. It has also been shown that with a small distance between the optical centers of the captured images, the use of video compression algorithms is preferable to the use of image compression algorithms, since they show better results in both three-dimensional and four-dimensional versions.Conclusions. A comparison of the results obtained shows the need to use algorithms from the video compression family (XVC, AV1) on installations with a long baseline (mounted on camera stands). When working with integrated light field cameras (Lytro) and setting the capture with a short baseline, it is recommended to use image compression algorithms (JPEG). In general, video compression algorithms are recommended, in particular XVC, since on average it shows an acceptable level of PSNR in both the case of a short and long installation baseline.

Plenoptic 2.0 Intra Coding Using Imaging Principle

Plenoptic 2.0 videos that are captured by focused plenoptic cameras outperform the traditional plenoptic videos with higher spatial resolution of rendered subaperture images and more effective light field sampling. However, its distinct imaging principle causes different pixel intensity distribution and complex macropixel structures, which results in ineffectiveness in coding them using the existing plenoptic video compression methods. Thus, a novel compression method is proposed in this paper to dramatically improve the intra prediction efficiency for multi-focus plenoptic 2.0 video coding. It consists of two new intra prediction modes: imaging-principle-guided static prediction (ISP) and imaging-principle-guided zoomed prediction (IZP). In ISP, the prediction candidates spatially positioned by plenoptic 2.0 imaging principle are weighted by minimizing boundary error to generate the predicted samples. In IZP, the spatial positions and scales of the prediction candidates vary according to the focusing status and the focal length of the microlens that generates the pixels in the prediction candidates. The combination of two modes generalizes plenoptic 2.0 video coding for multi-focus plenoptic cameras. The proposed approach outperforms high efficiency video coding (HEVC) standard by an average of 15.37% bitrate reduction under All Intra configuration. It also outperforms other state-of-the-art methods with obvious compression efficiency improvements.

A Joint Spatiotemporal Video Compression Based on Stochastic Adaptive Fourier Decomposition

This paper proposes a novel video compression method &#x2013; stochastic adaptive Fourier decomposition (SAFD) based joint spatiotemporal model (JSTM). SAFD is a recently developed sparse representation theory that combines the traditional signal decomposition method with machine learning to adaptively decompose multi-signals into common atoms of predefined interpretable Szeg<inline-formula><tex-math notation="LaTeX">$\ddot{\text{o}}$</tex-math></inline-formula> kernel dictionary. Based on SAFD, this paper firstly proposes a learning-based 3D video to 2D data embedding method. The method learns common atoms based on the frames themselves that require to be compressed, without the need for pre-training on large-scale data like deep learning. Then the embedding-based JSTM and a feasible video compression architecture are developed. The contribution is twofold: introducing SAFD into video compression first time in the literature and developing a new video compression model JSTM. The experimental results are promising.

End-to-End Rate-Distortion Optimized Learned Hierarchical Bi-Directional Video Compression.

Conventional video compression (VC) methods are based on motion compensated transform coding, and the steps of motion estimation, mode and quantization parameter selection, and entropy coding are optimized individually due to the combinatorial nature of the end-to-end optimization problem. Learned VC allows end-to-end rate-distortion (R-D) optimized training of nonlinear transform, motion and entropy model simultaneously. Most works on learned VC consider end-to-end optimization of a sequential video codec based on R-D loss averaged over pairs of successive frames. It is well-known in conventional VC that hierarchical, bi-directional coding outperforms sequential compression because of its ability to use both past and future reference frames. This paper proposes a learned hierarchical bi-directional video codec (LHBDC) that combines the benefits of hierarchical motion-compensated prediction and end-to-end optimization. Experimental results show that we achieve the best R-D results that are reported for learned VC schemes to date in both PSNR and MS-SSIM. Compared to conventional video codecs, the R-D performance of our end-to-end optimized codec outperforms those of both x265 and SVT-HEVC encoders ("veryslow" preset) in PSNR and MS-SSIM as well as HM 16.23 reference software in MS-SSIM. We present ablation studies showing performance gains due to proposed novel tools such as learned masking, flow-field subsampling, and temporal flow vector prediction. The models and instructions to reproduce our results can be found in https://github.com/makinyilmaz/LHBDC/.

CAVLCU: an efficient GPU-based implementation of CAVLC

CAVLC (Context-Adaptive Variable Length Coding) is a high-performance entropy method for video and image compression. It is the most commonly used entropy method in the video standard H.264. In recent years, several hardware accelerators for CAVLC have been designed. In contrast, high-performance software implementations of CAVLC (e.g., GPU-based) are scarce. A high-performance GPU-based implementation of CAVLC is desirable in several scenarios. On the one hand, it can be exploited as the entropy component in GPU-based H.264 encoders, which are a very suitable solution when GPU built-in H.264 hardware encoders lack certain necessary functionality, such as data encryption and information hiding. On the other hand, a GPU-based implementation of CAVLC can be reused in a wide variety of GPU-based compression systems for encoding images and videos in formats other than H.264, such as medical images. This is not possible with hardware implementations of CAVLC, as they are non-separable components of hardware H.264 encoders. In this paper, we present CAVLCU, an efficient implementation of CAVLC on GPU, which is based on four key ideas. First, we use only one kernel to avoid the long latency global memory accesses required to transmit intermediate results among different kernels, and the costly launches and terminations of additional kernels. Second, we apply an efficient synchronization mechanism for thread-blocks (In this paper, to prevent confusion, a block of pixels of a frame will be referred to as simply block and a GPU thread block as thread-block.) that process adjacent frame regions (in horizontal and vertical dimensions) to share results in global memory space. Third, we exploit fully the available global memory bandwidth by using vectorized loads to move directly the quantized transform coefficients to registers. Fourth, we use register tiling to implement the zigzag sorting, thus obtaining high instruction-level parallelism. An exhaustive experimental evaluation showed that our approach is between 2.5times and 5.4times faster than the only state-of-the-art GPU-based implementation of CAVLC.

Deceptive Techniques to Hide a Compressed Video Stream for Information Security.

With the recent development of video compression methods, video transmission on traditional devices and video distribution using networks has increased in various devices such as drones, IP cameras, and small IoT devices. As a result, the demand for encryption techniques such as MPEG-DASH for transmitting streams over networks is increasing. These video stream security methods guarantee stream confidentiality. However, they do not hide the fact that the encrypted stream is being transmitted over the network. Considering that sniffing attacks can analyze the entropy of the stream and scan huge amounts of traffic on the network, to solve this problem, the deception method is required, which appears unencrypted but a confidential stream. In this paper, we propose the new deception method that utilizes standard NAL unit rules of video codec, where the unpromised device shows the cover video and the promised device shows the secret video for deceptive security. This method allows a low encryption cost and the stream to dodge entropy-based sniffing scan attacks. The proposed stream shows that successful decoding using five standard decoders and processing performance was 61% faster than the conventional encryption method in the test signal conformance set. In addition, a network encrypted stream scan method the HEDGE showed classification results that our stream is similar to a compressed video.

Optimal block search mechanism using deep recurrent neural network for enabling the code-efficiency in HEVC

Purpose High-efficiency video coding (HEVC) is the latest video coding standard that has better coding efficiency than the H.264/advanced video coding (AVC) standard. The purpose of this paper is to design and develop an effective block search mechanism for the video compression-HEVC standard such that the developed compression standard is applied for the communication applications. Design/methodology/approach In the proposed method, an rate-distortion (RD) trade-off, named regressive RD trade-off is used based on the conditional autoregressive value at risk (CaViar) model. The motion estimation (ME) is based on the new block search mechanism, which is developed with the modification in the Ordered Tree-based Hex-Octagon (OrTHO)-search algorithm along with the chronological Salp swarm algorithm (SSA) based on deep recurrent neural network (deepRNN) for optimally deciding the shape of search, search length of the tree and dimension. The chronological SSA is developed by integrating the chronological concept in SSA, which is used for training the deep RNN for ME. Findings The competing methods used for the comparative analysis of the proposed OrTHO-search based RD + chronological-salp swarm algorithm (RD + C-SSA) based deep RNN are support vector machine (SVM), fast encoding framework, wavefront-based high parallel (WHP) and OrTHO-search based RD method. The proposed video compression method obtained a maximum peak signal-to-noise ratio (PSNR) of 42.9180 dB and a maximum structural similarity index measure (SSIM) of 0.9827. Originality/value In this research, an effective block search mechanism was developed with the modification in the OrTHO-search algorithm along with the chronological SSA based on deepRNN for the video compression-HEVC standard.

Analisis Perbandingan Performa Codec H.264 &amp; H.265 Video Streaming Dari Segi Quality of Service

Video streaming is a technology that is often used when watching videos on the internet without having to download then video to play it. Some problems that affect the performance of video streaming are such as the large capacity of the video size, the capacity of the video size affects the smoothness when streaming video. In this research, two video compression methods will be compared, namely H.264 codec and H.265 codec. The test is carried out to determine the effect of using the compression method according to changes in the codec ang framerate used. After testing the two codec, it can be concluded that the H.265 codec is more effective to apply than the H.264 codec.

Improved HEVC video compression algorithm using low-complexity frame rate up conversion

The high-efficiency video coding (HEVC) standard can reduce the bit rate of compression video. However, the compression efficiency of HEVC is still challenging current internet bandwidth. This problem can be solved by reducing the number of video frames encoded by HEVC and reconstructing the reduced frames that rely on frame rate up conversion (FRUC). Unfortunately, most existing FRUC methods need to re-estimate the motion information, a step that will lead to a huge increase of video decoder complexity. An improved HEVC video compression algorithm using low-complexity FRUC is proposed based on reusing the motion vector (MV) predicted by HEVC encoding procedure to directly reconstruct the skipped frames. First, an adaptive frame-skipping scheme is used to maximize reducing the number of frames that will encode by HEVC. Second, decoder extracts the MV and residual from the received bit stream. Then an MV reliability checking and rectifying method is proposed according to adaptive residual energy analyzing. Finally, a motion-compensation method and efficient occluded area processing algorithm are proposed to reconstruct skipped frames. These methods all directly utilize the rectified MVs. Experimental results show that due to the reduced frames and decoder that does not need to re-estimate MV, the proposed method can not only significantly decrease the time of all codec procedure when comparing HEVC and other FRUC-based video compression methods but also achieve 9.7022% to 66.0030% reduction in Bjøntegaard delta bit rate at low bit rates when compared to HEVC standard. In addition, the compression of the proposed method is also better than other state-of-the-art methods.

A caching‐enabled light control scheme for aerobics video

The video transmission traffic has account for 60% Internet traffic, and how to optimize the video transmission efficiency has become a major challenge. Especially, the aerobics video is strongly subjected to both temporal locality and spatial locality, and the related contents are also dynamically changeable with the precise and complex movements. Thus, in the age of multimedia, the transmission optimization of aerobics video has the considerable research value. In this paper, a transmission control scheme for aerobics video is proposed. This scheme has the light feature, which is embodied by one heuristic video compression method. Then, this paper realizes the effective congestion control based on the adaptive rate design. Meanwhile, video compression and congestion control are performed in the simultaneous way, and they are completed by using the explored in‐network cache. Given this, the simple cache allocation method is presented at last. Two baselines are compared by evaluating packet loss rate, transmission time and delay jitter, and the experiment results demonstrate the efficiency of the proposed transmission scheme.

Deep Predictive Video Compression Using Mode-Selective Uni- and Bi-Directional Predictions Based on Multi-Frame Hypothesis

Recently, deep learning-based image compression has shown significant performance improvement in terms of coding efficiency and subjective quality. However, there has been relatively less effort on video compression based on deep neural networks. In this paper, we propose an end-to-end deep predictive video compression network, called DeepPVCnet, using mode-selective uni- and bi-directional predictions based on multi-frame hypothesis with a multi-scale structure and a temporal-context-adaptive entropy model. Our DeepPVCnet jointly compresses motion information and residual data that are generated from the multi-scale structure via the feature transformation layers. Recent deep learning-based video compression methods were proposed in a limited compression environment using only P-frame or B-frame. Learned from the lesson of the conventional video codecs, we firstly incorporate a mode-selective framework into our DeepPVCnet with uni- and bi-directional predictive modes in a rate-distortion minimization sense. Also, we propose a temporal-context-adaptive entropy model that utilizes the temporal context information of the reference frames for the current frame coding. The autoregressive entropy models for CNN-based image and video compression is difficult to compute with parallel processing. On the other hand, our temporal-context-adaptive entropy model utilizes temporally coherent context from the reference frames, so that the context information can be computed in parallel, which is computationally and architecturally advantageous. Extensive experiments show that our DeepPVCnet outperforms AVC/H.264, HEVC/H.265 and state-of-the-art methods in an MS-SSIM perspective.

Neural Video Coding Using Multiscale Motion Compensation and Spatiotemporal Context Model

Over the past two decades, traditional block-based video coding has made remarkable progress and spawned a series of well-known standards such as MPEG-4, H.264/AVC and H.265/HEVC. On the other hand, deep neural networks (DNNs) have shown their powerful capacity for visual content understanding, feature extraction and compact representation. Some previous works have explored the learnt video coding algorithms in an end-to-end manner, which show the great potential compared with traditional methods. In this paper, we propose an end-to-end deep neural video coding framework (NVC), which uses variational autoencoders (VAEs) with joint spatial and temporal prior aggregation (PA) to exploit the correlations in intra-frame pixels, inter-frame motions and inter-frame compensation residuals, respectively. Novel features of NVC include: 1) To estimate and compensate motion over a large range of magnitudes, we propose an unsupervised multiscale motion compensation network (MS-MCN) together with a pyramid decoder in the VAE for coding motion features that generates multiscale flow fields, 2) we design a novel adaptive spatiotemporal context model for efficient entropy coding for motion information, 3) we adopt nonlocal attention modules (NLAM) at the bottlenecks of the VAEs for implicit adaptive feature extraction and activation, leveraging its high transformation capacity and unequal weighting with joint global and local information, and 4) we introduce multi-module optimization and a multi-frame training strategy to minimize the temporal error propagation among P-frames. NVC is evaluated for the low-delay causal settings and compared with H.265/HEVC, H.264/AVC and the other learnt video compression methods following the common test conditions, demonstrating consistent gains across all popular test sequences for both PSNR and MS-SSIM distortion metrics.

Enhanced holoentropy-based encoding via whale optimization for highly efficient video coding

High-efficiency video coding (HEVC), a video compression method is considered as the most capable descendant of the extensively deployed advanced VC (AVC). Compared with AVC, HEVC provides about twice the data compression ratio at the similar video quality level or considerably enhanced video quality at an equal bit rate. This paper proposes a novel enhanced holoentropy model for proficient systems for distributed VC (DVC). HEVC standard is considered as an archetypal system. The main contribution of this paper is the accomplishment of the encoding process in the HEVC system by enhanced holoentropy, which is linked with the proposed weighting tansig function. It necessitates considerable development when handling video sequences with high resolution. The pixel deviations under altering frames are grouped based on interest, and the outliers are eliminated with the aid of an enhanced entropy standard known as enhanced holoentropy. Here, the weight of tansig function is optimally tuned by whale optimization algorithm. To next of implementation, the suggested encoding scheme is compared with the conventional schemes concerning the number of compressed bits and computational time. By carrying out the encoding process, it reduces the video size with perceptually improved video quality or PSNR.

A Novel Method for Image and Video Compression Based on Two-Level DCT with Hexadata Coding

In this paper a novel method for 2D image compression is proposed and demonstrated through high quality reconstruction with compression ratios up to 99%. The proposed novel algorithm is based on a two-level discrete cosine transform (DCT) followed by Hexadata coding and arithmetic coding at compression stage. The novel method consists of four main steps: (1) a two-level DCT is applied to an image to reinforce the low frequency coefficients and increase the number of high frequency coefficients to facilitate the compression process; (2) the Hexadata coding algorithm is applied to each high frequency matrix separately through five different keys to reduce each matrix to 1/6 of their original size; (3) build a probability table of original high-frequency data required in the decoding step; and (4) apply arithmetic coding to compress each of the outputs of steps (2) and (3). At decompression stage, arithmetic decoding and a fast matching search algorithm (FMS-Algorithm) decodes the high frequency coefficients of step (2) using the probability table of step (3). Finally, two level inverse DCT is applied to decode the high frequency coefficients to reconstruct the image. The technique is demonstrated on still images including video streaming from YouTube. The results show that the proposed method yields high compression ratios up to 99% with better perceptual quality of reconstructed images as compared with the popular JPEG method.

A signal adaptive diffusion filter for video coding: Mathematical framework and complexity reductions

In this paper we combine video compression and modern image processing methods. We construct novel iterative filter methods for prediction signals based on Partial Differential Equation (PDE) based methods. The mathematical framework of the employed diffusion filter class is given and some desirable properties are stated. In particular, two types of diffusion filters are constructed: a uniform diffusion filter using a fixed filter mask and a signal adaptive diffusion filter that incorporates the structures of the underlying prediction signal. The latter has the advantage of not attenuating existing edges while the uniform filter is less complex. The filters are embedded into a software based on HEVC with additional QTBT (Quadtree plus Binary Tree) and MTT (Multi-Type-Tree) block structure. In this setting, several measures to reduce the coding complexity of the tool are introduced, discussed and tested thoroughly. The coding complexity is reduced by up to 70% while maintaining over 80% of the gain. Overall, the diffusion filter method achieves average bitrate savings of 2.27% for Random Access having an average encoder runtime complexity of 119% and 117% decoder runtime complexity. For individual test sequences, results of 7.36% for Random Access are accomplished.

Learned Video Compression via Joint Spatial-Temporal Correlation Exploration

Traditional video compression technologies have been developed over decades in pursuit of higher coding efficiency. Efficient temporal information representation plays a key role in video coding. Thus, in this paper, we propose to exploit the temporal correlation using both first-order optical flow and second-order flow prediction. We suggest an one-stage learning approach to encapsulate flow as quantized features from consecutive frames which is then entropy coded with adaptive contexts conditioned on joint spatial-temporal priors to exploit second-order correlations. Joint priors are embedded in autoregressive spatial neighbors, co-located hyper elements and temporal neighbors using ConvLSTM recurrently. We evaluate our approach for the low-delay scenario with High-Efficiency Video Coding (H.265/HEVC), H.264/AVC and another learned video compression method, following the common test settings. Our work offers the state-of-the-art performance, with consistent gains across all popular test sequences.

Video compression method on the basis of discrete wavelet transform for application in video information systems with non-standard parameters

Video compression method on the basis of discrete wavelet transform for application in video information systems with non-standard parameters

Steerable-Discrete-Cosine-Transform (SDCT): Hardware Implementation and Performance Analysis.

In the last years, the need for new efficient video compression methods grown rapidly as frame resolution has increased dramatically. The Joint Collaborative Team on Video Coding (JCT-VC) effort produced in 2013 the H.265/High Efficiency Video Coding (HEVC) standard, which represents the state of the art in video coding standards. Nevertheless, in the last years, new algorithms and techniques to improve coding efficiency have been proposed. One promising approach relies on embedding direction capabilities into the transform stage. Recently, the Steerable Discrete Cosine Transform (SDCT) has been proposed to exploit directional DCT using a basis having different orientation angles. The SDCT leads to a sparser representation, which translates to improved coding efficiency. Preliminary results show that the SDCT can be embedded into the HEVC standard, providing better compression ratios. This paper presents a hardware architecture for the SDCT, which is able to work at a frequency of 188, reaching a throughput of 3.00 GSample/s. In particular, this architecture supports 8k UltraHigh Definition (UHD) (7680 × 4320) with a frame rate of 60 , which is one of the best resolutions supported by HEVC.

A Signal Adaptive Prediction Filter for Video Coding Using Directional Total Variation: Mathematical Framework and Parameter Selection.

In this paper we combine video compression and modern image processing methods. Iterative filter methods for prediction signals based on classic inpainting methods are introduced and extensive parameter tests are described. In order to construct an alternative prediction filter for video coding, techniques originally employed for inpainting are applied. Thereby, the structures of the underlying prediction were incorporated into the filter construction making it signal adaptive. The resulting optimization problem is solved using the so-called Alternating Direction Method of Multipliers (ADMM). The undertaken novel parameter tests are described and it is shown that they improve the coding efficiency of the tool. The suggested filter is embedded into a software based on HEVC with additional QTBT (Quadtree plus Binary Tree) and MTT (Multi-Type-Tree) block structure. Overall, the proposed filter method obtains average bitrate savings of 1.35% at an average encoder runtime increase of 28% and decoder runtime increase of 38%. UHD test sequences achieve bitrate savings of up to 3.66% for Random Access.