Video Compression Algorithms Research Articles

Robust Mixed-Rate Region-of-Interest-Aware Video Compressive Sensing for Transmission Line Surveillance Video

Classic video compression methods usually suffer from long encode time and requires large memories, making it hard to deploy on edge devices; thus, video compressive sensing, which requires less resources during encoding, is receiving more attention. We propose a robust mixed-rate ROI-aware video compressive sensing algorithm for transmission line surveillance video compression. The proposed method compresses foreground targets and background frames separately and uses reversible neural network to reconstruct original frames. The result on transmission line surveillance video data shows that the proposed compressive sensing method can achieve 26.47, 34.71 PSNR and 0.6839, 0.9320 SSIM higher than existing methods on 1.5% and 15% measurement rates, and the proposed ROI extraction net can precisely retrieve regions under high noise levels. This research not only demonstrates the potential for a more efficient video compression technique in resource-constrained environments, but also lays a foundation for future advancements in video compressive sensing techniques and their applications in various real-time surveillance systems.

A Comparison of Machine Learning-Based and Conventional Technologies for Video Compression

The growing demand for high-quality video transmission over bandwidth-constrained networks and the increasing availability of video content have led to the need for efficient storage and distribution of large video files. To improve the latter, this article offers a comparison of six video compression methods without loss of quality. Particularly, H.255, VP9, AV1, convolutional neural network (CNN), recurrent neural network (RNN), and deep autoencoder (DAE). The proposed decision is to use a dataset of high-quality videos to implement and compare the performance of classical compression algorithms and algorithms based on machine learning. Evaluations of the compression efficiency and the quality of the received images were made on the basis of two metrics: PSNR and SSIM. This comparison revealed the strengths and weaknesses of each approach and provided insights into how machine learning algorithms can be optimized in future research. In general, it contributed to the development of more efficient and effective video compression algorithms that can be useful for a wide range of applications.

An investigation of machine learning-based video compression techniques

As video technology continues to seamlessly weave itself into the fabric of daily life, there is a growing need for enhanced storage and efficient video transmission. This surge in demand has led to heightened expectations and standards for video compression technology. Machine learning as an up-and-coming technology can play its advantages in the field of video compression. This article reviews the current state of research on combining video compression techniques with machine learning. The article provides an overview of various research avenues for enhancement, spanning from conventional video compression algorithms to the fusion of traditional compression frameworks with machine learning methodologies, and even the development of novel end-to-end compression algorithms. In additional, the article explores the possible various application scenarios of machine learning-based video compression algorithms based on the characteristics of such non-standard and arithmetic demanding algorithms. At the end, the article speculates on the future of video compression algorithms based on the content of the various studies reviewed in the article.

Perceptual Quality Improvement in Videoconferencing Using Keyframes-Based GAN

In the latest years, videoconferencing has taken a fundamental role in interpersonal relations, both for personal and business purposes. Lossy video compression algorithms are the enabling technology for videoconferencing, as they reduce the bandwidth required for real-time video streaming. However, lossy video compression decreases the perceived visual quality. Thus, many techniques for reducing compression artifacts and improving video visual quality have been proposed in recent years. In this work, we propose a novel GAN-based method for compression artifacts reduction in videoconferencing. Given that, in this context, the speaker is typically in front of the camera and remains the same for the entire duration of the transmission, we can maintain a set of reference keyframes of the person from the higher-quality I-frames that are transmitted within the video stream and exploit them to guide the visual quality improvement; a novel aspect of this approach is the update policy that maintains and updates a compact and effective set of reference keyframes. First, we extract multi-scale features from the compressed and reference frames. Then, our architecture combines these features in a progressive manner according to facial landmarks. This allows the restoration of the high-frequency details lost after the video compression. Experiments show that the proposed approach improves visual quality and generates photo-realistic results even with high compression rates. Code and pre-trained networks are publicly available at https://github.com/LorenzoAgnolucci/Keyframes-GAN.

Retracted: Research on Low Frame Rate Video Compression Algorithm in the Context of New Media

Retracted: Research on Low Frame Rate Video Compression Algorithm in the Context of New Media

Video Compression Artifact Reduction by Fusing Motion Compensation and Global Context in a Swin-CNN Based Parallel Architecture

Video Compression Artifact Reduction aims to reduce the artifacts caused by video compression algorithms and improve the quality of compressed video frames. The critical challenge in this task is to make use of the redundant high-quality information in compressed frames for compensation as much as possible. Two important possible compensations: Motion compensation and global context, are not comprehensively considered in previous works, leading to inferior results. The key idea of this paper is to fuse the motion compensation and global context together to gain more compensation information to improve the quality of compressed videos. Here, we propose a novel Spatio-Temporal Compensation Fusion (STCF) framework with the Parallel Swin-CNN Fusion (PSCF) block, which can simultaneously learn and merge the motion compensation and global context to reduce the video compression artifacts. Specifically, a temporal self-attention strategy based on shifted windows is developed to capture the global context in an efficient way, for which we use the Swin transformer layer in the PSCF block. Moreover, an additional Ada-CNN layer is applied in the PSCF block to extract the motion compensation. Experimental results demonstrate that our proposed STCF framework outperforms the state-of-the-art methods up to 0.23dB (27% improvement) on the MFQEv2 dataset.

Investigation on Motion Teaching Video Compression Algorithm Based on Artificial Intelligence

Investigation on Motion Teaching Video Compression Algorithm Based on Artificial Intelligence

Improving Compressed Video Using Single Lightweight Model with Temporal Fusion Module.

Video compression algorithms are commonly used to reduce the number of bits required to represent a video with a high compression ratio. However, this can result in the loss of content details and visual artifacts that affect the overall quality of the video. We propose a learning-based restoration method to address this issue, which can handle varying degrees of compression artifacts with a single model by predicting the difference between the original and compressed video frames to restore video quality. To achieve this, we adopted a recursive neural network model with dilated convolution, which increases the receptive field of the model while keeping the number of parameters low, making it suitable for deployment on a variety of hardware devices. We also designed a temporal fusion module and integrated the color channels into the objective function. This enables the model to analyze temporal correlation and repair chromaticity artifacts. Despite handling color channels, and unlike other methods that have to train a different model for each quantization parameter (QP), the number of parameters in our lightweight model is kept to only about 269 k, requiring only about one-twelfth of the parameters used by other methods. Our model applied to the HEVC test model (HM) improves the compressed video quality by an average of 0.18 dB of BD-PSNR and -5.06% of BD-BR.

Image and video compression of fluid flow data

We study the compression of spatial and temporal features in fluid flow data using multimedia compression techniques. The efficacy of spatial compression techniques, including JPEG and JPEG2000 (JP2), and spatiotemporal video compression techniques, namely H.264, H.265, and AV1, in limiting the introduction of compression artifacts and preserving underlying flow physics are considered for laminar periodic wake around a cylinder, two-dimensional turbulence, and turbulent channel flow. These compression techniques significantly compress flow data while maintaining dominant flow features with negligible error. AV1 and H.265 compressions present the best performance across a variety of canonical flow regimes and outperform traditional techniques such as proper orthogonal decomposition in some cases. These image and video compression algorithms are flexible, scalable, and generalizable holding potential for a wide range of applications in fluid dynamics in the context of data storage and transfer.Graphic abstract

ANALYSIS OF VIDEO DATA COMPRESSION ALGORITHMS AND OPTIMIZED FOR USE IN REMOTELY CONTROLLED DRONES

Due to the widespread use of unmanned aerial vehicles (UAVs) in the civil sphere, there is a need to improve the video image generation quality and transmission technologies, changing the video streams coding to reduce their size and improve quality. Video streaming technologies are moving further towards complicating the video transmission systems used, which helps improve the received video data quality. Higher of transmitted video signal quality – more requirements for frequency band increase used. This fact leads us to use effective video compression algorithms that allow us to combine high image quality with a narrow bandwidth of frequencies used. In this paper, currently existing coding algorithms, their efficiency and computational complexity are studied. Among the algorithms under consideration there will only be those whose effectiveness has been proven by finding their application in modern realities. Based on the research, conclusions will be drawn regarding the feasibility of using each encoding algorithm for transmitting video data in real time.

Video Compression Performance Evaluation Method in Transmission via a Low-Speed Channel

The paper presents an overview of the works devoted to video compression algorithms based on artificial intelligence methods. Two main directions for the development of such algorithms are identified - these are the development of modules for post-processing the results of the work of classical algorithms and the development of algorithms that completely replace existing video codecs. The problems of video compression algorithms performance evaluation criteria are considered. It was found that criteria based on the calculation of the standard deviation, such as PSNR, cannot be used to assess the quality of video data compression algorithms when operating under the influence of radio interference, due to the strong influence on the evaluation result of artifacts that inevitably arise as a result of the effects of this kind of interference. An alternative quality assessment method based on the semantic analysis of frames is proposed, which can be used to assess the noise immunity of video data transmission algorithms. The presented estimation method uses a frame-by-frame comparison of the original and restored video sequence, similar to PSNR. However, unlike the latter, it ignores artifacts that occur during exposure to radio interference, due to the use of semantic frame analysis, which consists in searching for objects in the image using approaches based on artificial intelligence. To compare the found objects on the original and reconstructed frames, an estimate is used based on the geometric position of the found objects in the image. After that, the data indicator is averaged for all frames of the video sequence in order to obtain the resulting similarity indicator of the restored video and the original.

Comparative analysis of compression algorithms for four-dimensional light fields

Objectives. The widespread use of systems for capturing light fields is due to the high quality of the reproduced image. This type of capture, although qualitatively superior to traditional methods to capturing volumetric images, generates a huge amount of data needed to reconstruct the original captured 4D light field. The purpose of the work is to consider traditional and extended to four-dimensional image compression algorithms, to perform a comparative analysis and determine the most suitable.Methods. Mathematical methods of signal processing and methods of statistical analysis are used.Results. Algorithms are compared and analyzed in relation to the compression of four-dimensional light fields using the PSNR metric. The selected evaluation criterion is affected not only by the dimension of the compression algorithm, but also by the distance of the baseline of the capture setting, since the difference between images increases with the distance between the optical centers of each camera matrix. Thus, for installations consisting of an array of machine vision cameras located on racks and placed in a room, the obvious choice would be to use conventional image compression methods. Furthermore, based on the assessment of the arbitrariness of video compression methods, it should be noted that the XVC algorithm remains undervalued, although its results are higher. Algorithm AV1 can be considered the next in order of importance. It has been established that the latest compression algorithms show higher performance if compared to their predecessors. It has also been shown that with a small distance between the optical centers of the captured images, the use of video compression algorithms is preferable to the use of image compression algorithms, since they show better results in both three-dimensional and four-dimensional versions.Conclusions. A comparison of the results obtained shows the need to use algorithms from the video compression family (XVC, AV1) on installations with a long baseline (mounted on camera stands). When working with integrated light field cameras (Lytro) and setting the capture with a short baseline, it is recommended to use image compression algorithms (JPEG). In general, video compression algorithms are recommended, in particular XVC, since on average it shows an acceptable level of PSNR in both the case of a short and long installation baseline.

Deep learning based optimised data transmission over 5G networks with Lagrangian encoder

Video data transmission without buffering is ubiquitous for all contemporary appliances. The increased consumption of video content has created the demand for more efficient video compression algorithms for high-resolution videos. In this study, a video compression technique is presented that uses a Lagrangian encoder (LE) with H.265 protocol. This architecture provides compression with less bandwidth compared to contemporary encoders, and it is compatible with 5 G network transmission speed and latency. Adding LE in the H.265 protocol architecture reduced the buffering delay and increased the efficiency. Along with these parameters, this study focused on the quality of video streaming, which was measured by metrics such as peak signal to noise ratio (PSNR), structural similarity index (SSIM), and video multi-method assessment fusion (VMAF). A comparison of the values of compression ratio and latency with and without H.264 architecture showed improved performance in the architecture. Thus, deep learning-based optimised data transmission can improve accuracy and reduce computational complexity.

Blind quality scalable video compression algorithm for low bit-rate coding

Blind quality scalable video compression algorithm for low bit-rate coding

A Low Power Architectural Framework for Automated Surveillance System with Low Bit Rate Transmission

Abstract The changed security scenario of the modern time has necessitated increased and sophisticated vigilance of the countries' borders. The technological challenges involved in accomplishing such feat of automated security system are many and require research at the components-and-algorithms as well as the architectural levels. This paper proposes an architectural framework for automated video surveillance comprising a network of sensors and closed circuit television cameras as well as proposing algorithmic/component research of software and hardware for the core functioning of the framework, such as: communication protocols, object detection, data-integration, object identification, object tracking, video compression, threat identification, and alarm generation. In this paper, we are addressing some general topological and routing features that would be adopted in our system. There are two types of data with regard to data communication – video stream and object detection. The network is broken down into several disjoint, almost equal zones. A zone have one or more one cluster. A zone manager is chosen among the cluster heads depending on their relative residual energies. There are several levels of control that could be implemented with this arrangement with localized decision made, to get distributed effect at all levels. A cell tracks each target in its zone. If the target moves out of the range of a cell, the cell manager will send the target description to estimated next cell. The next cell starts tracking the target. If the estimated cell is wrongly chosen, corrections will be made by the cluster heads to get the new target-tracking. We also propose bitrate reduction algorithms to accommodate the limited bandwidth. One of the main feature of this paper is introducing a Low-Power Low-Bit rate video compression algorithm to accommodate the low power requirements at sensor nodes, and the low bit rate requirement for the communication protocol. We proposed two algorithms the ALBR and LPHSME. ALBR is addressing low bit rate required for sensors network with limited bandwidth which achieves a reduction in Average number of bits per Iframe by approximately 60% in case of low motion video sequences and 53% in case of fast motion video sequences . LPHSME addresses low power requirements of multi sensor network that has limited power batteries. The performance of the proposed LPHSME algorithm versus full search and three step search indicates a reduction in motion estimation time by approximately 89% in case of low motion video sequences (e.g., Claire ) and 84% in case of fast motion video sequences. The reduced complexity of LPHSME results in low power requirements.

Video Compressed Sensing Reconstruction Based on Multi-Dimensional Reference Frame Multi Hypothesis Rediction

In this paper, a video compressed sensing reconstruction algorithm based on multidimensional reference frames is proposed using the sparse characteristics of video signals in different sparse representation domains. First, the overall structure of the proposed video compressed sensing algorithm is introduced in this paper. The paper adopts a multi-reference frame bidirectional prediction hypothesis optimization algorithm. Then, the paper proposes a reconstruction method for CS frames at the re-decoding end. In addition to using key frames of each GOP reconstructed in the time domain as reference frames for reconstructing CS frames, half-pixel reference frames and scaled reference frames in the pixel domain are also used as CS frames. Reference frames of CS frames are used to obtain higher quality assumptions. The method of obtaining reference frames in the pixel domain is also discussed in detail in this paper. Finally, the reconstruction algorithm proposed in this paper is compared with video compression algorithms in the literature that have better reconstruction results. Experiments show that the algorithm has better performance than the best multi-reference frame video compression sensing algorithm and can effectively improve the quality of slow motion video reconstruction.

BVI-DVC: A Training Database for Deep Video Compression

Deep learning methods are increasingly being applied in the optimisation of video compression algorithms and can achieve significantly enhanced coding gains, compared to conventional approaches. Such approaches often employ Convolutional Neural Networks (CNNs) which are trained on databases with relatively limited content coverage. In this paper, a new extensive and representative video database, BVI-DVC, is presented for training CNN-based video compression systems, with specific emphasis on machine learning tools that enhance conventional coding architectures, including spatial resolution and bit depth up-sampling, post-processing and in-loop filtering. BVI-DVC contains 800 sequences at various spatial resolutions from 270p to 2160p and has been evaluated on ten existing network architectures for four different coding tools. Experimental results show that this database produces significant improvements in terms of coding gains over three existing (commonly used) image/video training databases under the same training and evaluation configurations. The overall additional coding improvements by using the proposed database for all tested coding modules and CNN architectures are up to 10.3% based on the assessment of PSNR and 8.1% based on VMAF.

Design and Implementation of Parallel Bypass Bin Processing for CABAC Encoder

The ever-increasing demand for high-quality digital video requires efficient compression techniques and fast video codecs. It necessitates increased complexity of the video codec algorithms. So, there is a need for hardware accelerators to implement such complex algorithms. The latest video compression algorithms such as High-Efficiency Video Coding (HEVC) and Versatile Video Coding (VVC) have been adopted Context-based Adaptive Binary Arithmetic Coding (CABAC) as the entropy coding method. The CABAC has two main data processing paths: regular and bypass bin path, which can achieve good compression when used with Syntax Elements (SEs) statistics. However, it is highly intrinsic data dependence and has sequential coding characteristics. Thus, it is challenging to parallelize. In this work, a 6-core bypass bin path having high-throughput and low hardware area has been proposed. It is a parallel architecture capable of processing up to 6 bypass bins per clock cycle to improve throughput. Further, the resource-sharing techniques within the binarization and a common controller block have reduced the hardware area. The proposed architecture has been simulated, synthesized, and prototyped on 28 nm Artix 7 Field Programmable Gate Array (FPGA). The implementation of Application Specific Integrated Circuit (ASIC) has been done using 65 nm CMOS technology. The proposed design achieved a throughput of 1.26 Gbin/s at 210 MHz operating frequency with a low hardware area compared to existing architectures. This architecture also supports multi-standard (HEVC/VVC) encoders for Ultra High Definition (UHD) applications.

Research on Low Frame Rate Video Compression Algorithm in the Context of New Media

When using the current method to compress the low frame rate video animation video, there is no frame rate compensation for the video image, which cannot eliminate the artifacts generated in the compression process, resulting in low definition, poor quality, and low compression efficiency of the compressed low frame rate video animation video. In the context of new media, the linear function model is introduced to study the frame rate video animation video compression algorithm. In this paper, an adaptive detachable convolutional network is used to estimate the offset of low frame rate video animation using local convolution. According to the estimation results, the video frames are compensated to eliminate the artifacts of low frame rate video animation. After the frame rate compensation, the low frame rate video animation video is divided into blocks, the CS value of the image block is measured, the linear estimation of the image block is carried out by using the linear function model, and the compression of the low frame rate video animation video is completed according to the best linear estimation result. The experimental results show that the low frame rate video and animation video compressed by the proposed algorithm have high definition, high compression quality under different compression ratios, and high compression efficiency under different compression ratios.

Modified Hilbert Curve for Rectangles and Cuboids and Its Application in Entropy Coding for Image and Video Compression.

In our previous work, by combining the Hilbert scan with the symbol grouping method, efficient run-length-based entropy coding was developed, and high-efficiency image compression algorithms based on the entropy coding were obtained. However, the 2-D Hilbert curves, which are a critical part of the above-mentioned entropy coding, are defined on squares with the side length being the powers of 2, i.e., 2n, while a subband is normally a rectangle of arbitrary sizes. It is not straightforward to modify the Hilbert curve from squares of side lengths of 2n to an arbitrary rectangle. In this short article, we provide the details of constructing the modified 2-D Hilbert curve of arbitrary rectangle sizes. Furthermore, we extend the method from a 2-D rectangle to a 3-D cuboid. The 3-D modified Hilbert curves are used in a novel 3-D transform video compression algorithm that employs the run-length-based entropy coding. Additionally, the modified 2-D and 3-D Hilbert curves introduced in this short article could be useful for some unknown applications in the future.