Video Compression Technology Research Articles

Video compression and optimization technologies - review

Video compression and optimization technologies - review

An investigation of machine learning-based video compression techniques

As video technology continues to seamlessly weave itself into the fabric of daily life, there is a growing need for enhanced storage and efficient video transmission. This surge in demand has led to heightened expectations and standards for video compression technology. Machine learning as an up-and-coming technology can play its advantages in the field of video compression. This article reviews the current state of research on combining video compression techniques with machine learning. The article provides an overview of various research avenues for enhancement, spanning from conventional video compression algorithms to the fusion of traditional compression frameworks with machine learning methodologies, and even the development of novel end-to-end compression algorithms. In additional, the article explores the possible various application scenarios of machine learning-based video compression algorithms based on the characteristics of such non-standard and arithmetic demanding algorithms. At the end, the article speculates on the future of video compression algorithms based on the content of the various studies reviewed in the article.

Implementation of multimedia search & management system based on remote education

In order for remote education using multimedia to be effective, an efficient management technique for video information needs to be developed. Therefore, for real-time processing of moving images, it is necessary to manage and search image data in a compressed state. MPEG-4 is the most widely used video compression technology. In order to process video in real time in distance education using multimedia, it is very important to develop a technique for managing and retrieving video information compressed with MPEG-4. Therefore, in this paper, a multimedia information management system and search technology were developed using MPEG-4 compression technology used for real-time distance education.

Deep Learning-Based Video Compression for Surveillance Footage

The utilization of closed-circuit television (CCTV) monitoring plays a pivotal role in the realm of video processing, providing an effective means for vigilant surveillance. However, a significant challenge associated with this practice is the substantial demand it places on storage space. Traditionally, surveillance footage is stored on hard disk drives, and due to limited storage capacities, it often necessitates periodic deletion. To tackle this issue, we have introduced an innovative method for compressing CCTV video, named “Detection-Based video Compression” (DBVC). Our DBVC model is a two-step process. In the first step, we employ advanced neural network approaches such as Mask-RCNN and YOLOv4 to determine active and idle frames within the surveillance video. These cutting-edge techniques enable precise identification of objects and events of interest in the video feed. In the second step, we construct a new video composed solely of the active frames, eliminating redundant or uneventful segments. After conducting a comprehensive analysis of the experimental results, it is evident that Mask-RCNN stands out with an impressive detection accuracy of 98% on the COCO dataset, making it a robust choice for identifying objects and events in the surveillance footage. Consequently, we chose to leverage the output generated by Mask-RCNN and YOLOv4 for subsequent processing stages. Our DBVC approach is a breakthrough in video compression technology, significantly reducing the storage space required for CCTV footage. In fact, it achieves an average compression ratio of up to 85% when using YOLOv4, surpassing the capabilities of existing state-of-the-art compression methods. This innovation not only optimizes storage efficiency but also maintains a high level of surveillance data integrity, making it a valuable advancement in the field of CCTV video processing and storage management.

Perceptual video quality assessment: the journey continues!

Perceptual Video Quality Assessment (VQA) is one of the most fundamental and challenging problems in the field of Video Engineering. Along with video compression, it has become one of two dominant theoretical and algorithmic technologies in television streaming and social media. Over the last 2 decades, the volume of video traffic over the internet has grown exponentially, powered by rapid advancements in cloud services, faster video compression technologies, and increased access to high-speed, low-latency wireless internet connectivity. This has given rise to issues related to delivering extraordinary volumes of picture and video data to an increasingly sophisticated and demanding global audience. Consequently, developing algorithms to measure the quality of pictures and videos as perceived by humans has become increasingly critical since these algorithms can be used to perceptually optimize trade-offs between quality and bandwidth consumption. VQA models have evolved from algorithms developed for generic 2D videos to specialized algorithms explicitly designed for on-demand video streaming, user-generated content (UGC), virtual and augmented reality (VR and AR), cloud gaming, high dynamic range (HDR), and high frame rate (HFR) scenarios. Along the way, we also describe the advancement in algorithm design, beginning with traditional hand-crafted feature-based methods and finishing with current deep-learning models powering accurate VQA algorithms. We also discuss the evolution of Subjective Video Quality databases containing videos and human-annotated quality scores, which are the necessary tools to create, test, compare, and benchmark VQA algorithms. To finish, we discuss emerging trends in VQA algorithm design and general perspectives on the evolution of Video Quality Assessment in the foreseeable future.

Lightweight Video Super-Resolution for Compressed Video

Video compression technology for Ultra-High Definition (UHD) and 8K UHD video has been established and is being widely adopted by major broadcasting companies and video content providers, allowing them to produce high-quality videos that meet the demands of today’s consumers. However, high-resolution video content broadcasting is not an easy problem to be resolved in the near future due to limited resources in network bandwidth and data storage. An alternative solution to overcome the challenges of broadcasting high-resolution video content is to downsample UHD or 8K video at the transmission side using existing infrastructure, and then utilizing Video Super-Resolution (VSR) technology at the receiving end to recover the original quality of the video content. Current deep learning-based methods for Video Super-Resolution (VSR) fail to consider the fact that the delivered video to viewers goes through a compression and decompression process, which can introduce additional distortion and loss of information. Therefore, it is crucial to develop VSR methods that are specifically designed to work with the compression–decompression pipeline. In general, various information in the compressed video is not utilized enough to realize the VSR model. This research proposes a highly efficient VSR network making use of data from decompressed video such as frame type, Group of Pictures (GOP), macroblock type and motion vector. The proposed Convolutional Neural Network (CNN)-based lightweight VSR model is suitable for real-time video services. The performance of the model is extensively evaluated through a series of experiments, demonstrating its effectiveness and applicability in practical scenarios.

Fast CU Partitioning Algorithm for VVC Based on Multi-Stage Framework and Binary Subnets

VVC is the latest video compression technology available, and while the coding efficiency has improved significantly over the previous generation of standards, it has also led to a dramatic increase in coding complexity. As VVC uses a QTMT division structure, the more flexible division structure also allows for a significant increase in coding time. We have built a multi-stage network framework to solve the above problem by dividing the CU into different stages according to the size of the blocks. The desired features are extracted by dynamically adjusting to the size of the input CU. Secondly, we construct a binary classification subnet to perform the classification task at each stage and can determine the QT and MT division decisions. Finally, the resulting experimental results can demonstrate that our novel two-threshold decision scheme can achieve a balance between RD performance and TS. Our method succeeds in reducing the coding time by 49.08% to 52.56%, while the complexity of the negligible BD-BR increases by only 1.10% to 1.42%.

Perceptual Quality Assessment of HEVC and VVC Standards for 8K Video

With the growing data consumption of emerging video applications and users’ requirement for higher resolutions, up to 8K, a huge effort has been made in video compression technologies. Recently, versatile video coding (VVC) has been standardized by the moving picture expert group (MPEG), providing a significant improvement in compression performance over its predecessor high efficiency video coding (HEVC). In this paper, we provide a comparative subjective quality evaluation between VVC and HEVC standards for 8K resolution videos. In addition, we evaluate the perceived quality improvement offered by 8K over UHD 4K resolution. The compression performance of both VVC and HEVC standards has been conducted in random access (RA) coding configuration, using their respective reference software, VVC test model (VTM-11) and HEVC test model (HM-16.20). Objective measurements, using PSNR, MS-SSIM and VMAF metrics have shown that the bitrate gains offered by VVC over HEVC for 8K video content are around 31%, 26% and 35%, respectively. Subjectively, VVC offers an average of around 41% of bitrate reduction over HEVC for the same visual quality. A compression gain of 50% has been reached for some tested video sequences regarding a Student’s t-test analysis. In addition, for most tested scenes, a significant visual difference between uncompressed 4K and 8K has been noticed.

A Method Based on Curvature and Hierarchical Strategy for Dynamic Point Cloud Compression in Augmented and Virtual Reality System.

As a kind of information-intensive 3D representation, point cloud rapidly develops in immersive applications, which has also sparked new attention in point cloud compression. The most popular dynamic methods ignore the characteristics of point clouds and use an exhaustive neighborhood search, which seriously impacts the encoder’s runtime. Therefore, we propose an improved compression means for dynamic point cloud based on curvature estimation and hierarchical strategy to meet the demands in real-world scenarios. This method includes initial segmentation derived from the similarity between normals, curvature-based hierarchical refining process for iterating, and image generation and video compression technology based on de-redundancy without performance loss. The curvature-based hierarchical refining module divides the voxel point cloud into high-curvature points and low-curvature points and optimizes the initial clusters hierarchically. The experimental results show that our method achieved improved compression performance and faster runtime than traditional video-based dynamic point cloud compression.

FSpot: Fast and Efficient Video Encoding Workloads Over Amazon Spot Instances

HTTP Adaptive Streaming (HAS) of video content is becoming an undivided part of the Internet and accounts for most of today's network traffic. Video compression technology plays a vital role in efficiently utilizing network channels, but encoding videos into multiple representations with selected encoding parameters is a significant challenge. However, video encoding is a computationally intensive and time-consuming operation that requires high-performance resources provided by on-premise infrastructures or public clouds. In turn, the public clouds, such as Amazon elastic compute cloud (EC2), provide hundreds of computing instances optimized for different purposes and clients’ budgets. Thus, there is a need for algorithms and methods for optimized computing instance selection for specific tasks such as video encoding and transcoding operations. Additionally, the encoding speed directly depends on the selected encoding parameters and the complexity characteristics of video content. In this paper, we first benchmarked the video encoding performance of Amazon EC2 spot instances using multiple ×264 codec encoding parameters and video sequences of varying complexity. Then, we proposed a novel fast approach to optimize Amazon EC2 spot instances and minimize video encoding costs. Furthermore, we evaluated how the optimized selection of EC2 spot instances can affect the encoding cost. The results show that our approach, on average, can reduce the encoding costs by at least 15.8% and up to 47.8% when compared to a random selection of EC2 spot instances.

Analisis Perbandingan Teknik Video Codec H.264/AVC, H.265/HEVC, VP9 dan AV1

Video applications consume more energy on the Internet and can be accessed by electronic devices, due to an increase in the consumption of high-resolution and high-quality video content, presenting serious issues to delivery infrastructure that needs higher video compression technologies. The focus of this paper is to evaluate the quality of the most current codec, AV1, to its predecessor codec. The comparison was made experimentally at two video resolutions (1080p and 720p) by sampling video frames with various CRF/CQP values and testing several parameters analyses such as encoding duration, compression ratio, bit rate, Mean Square Error (MSE), and Peak Signal to Noise Ratio (PSNR). The AV1 codec is very great in terms of quality and file size, even though it is slower in terms of compression speed. The H.265/HEVC codec, on the other side, beats the other codec in terms of compression ratio. In conclusion, the H.265/HEVC codec is suggested as a material for obtaining a well compressed video with small file size and a short time.

VVC Complexity and Software Implementation Analysis

A steady increase in available processing power continues to drive advances in video compression technology. The recently completed Versatile Video Coding (VVC) standard aims to double the compression efficiency of HEVC and deliver a same quality of video at half the bitrate. To achieve this goal, VVC includes several new methods that improve coding efficiency at the cost of increased complexity. This paper provides a complexity analysis of VVC and its VTM reference software. Whereas VVC is more complex than HEVC, it remains readily implementable in software on current generation processors. Performance of practical decoders are reported, showing that real-time decoding of 8K content is feasible. An encoder is also presented, showing that most of the compression gains of VVC over HEVC can be obtained at a small fraction of the resources needed by the VTM encoder under common test conditions.

Video quality assessment with dense features and ranking pooling

Benefiting with the rapid development of communication networks, effective video quality assessment (VQA) models which provide guidance for video transmission and compression technologies are highly demanded. This paper proposes a general-purpose full-reference VQA method combining DenseNet with spatial pyramid pooling and RankNet to not only extract high-level distortion representation and global spatial information of samples but also characterize the temporal correlation among frames. Firstly, the pretrained DenseNet is modified and finetuned to extract high-level features of distorBenefiting with the rapid development of communication networks, effective video quality assessment (VQA) models which provide guidance for video transmission and compression technologies are highly demanded. This paper proposes a general-purpose full-reference VQA method combining DenseNet with spatial pyramid pooling and RankNet to not only extract high-level distortion representation and global spatial information of samples but also characterize the temporal correlation among frames. Firstly, the pretrained DenseNet is modified and finetuned to extract high-level features of distorted videos. Then, spatial pyramid pooling is equipped in the DenseNet module to process flexible inputs with arbitrary spatial resolution. Thus, this kind of input which has the same spatial resolution as the original distorted video is processed by the well-trained DenseNet to generate frame-level quality, which considers the global spatial information of videos directly. Finally, learning to rank is introduced to explore the high-level temporal correlation of distorted videos by taking the RankNet as the temporal pooling function. The experimental results on two public VQA databases show that the proposed algorithm performs consistently with human visual perception.

An Overview of the Legacy of Video Compression Technologies and New Advancement in Video Conference Technology

An Overview of the Legacy of Video Compression Technologies and New Advancement in Video Conference Technology

RETRACTED: Optimization of Motion Estimation Algorithm Based on FPGA Hardware System and Video Tracking

Motion Estimation (ME) is a computationally expensive step in video tracking. A thorough search technique produces the best execution time and the best accuracy. Computational burden, many fast search algorithms limit and report the number of places to search. Motion estimation, analysis of image sequences, computer vision, tracking a target, and is a critical process in a wide range of fields and applications such as video coding. Precision motion estimation lowers significantly bit rate, but can require a complexity of higher computational. This is different reference frames, sub-pixel estimation, and video compression standards, including techniques such as variable block size, MPEG, especially true for a new generation. Only the motion estimation in H.261 is limited to an integer pixel accuracy, rather than the pixel grid, not in the object is often located, move to move between pixels. The system proposed in the motion vector optimization motion estimation (OME) pixel accuracy. By estimating the amount of displacement at a finer resolution, improve prediction, it can expect better performance than motion estimation using an integer pixel precision. Optimization motion estimation (OME) and the compensation method is very similar to that employed in other standards. The main is that block-based motion estimation and compensation is suitable for optimization structures in MPEG-4. Motion estimation (ME) is one of the most computationally-intensive companies in the video compression technology. Video compression algorithm, MPEG1, MPEG4, such as H.261 and H.264, a lot of the standard used. Improved compression performance. Energy effective motion estimation technique is reduced to the rest of the frame involved in motion compensation.

Video processing and compression technologies

Video processing and compression technologies

Learned Video Compression via Joint Spatial-Temporal Correlation Exploration

Traditional video compression technologies have been developed over decades in pursuit of higher coding efficiency. Efficient temporal information representation plays a key role in video coding. Thus, in this paper, we propose to exploit the temporal correlation using both first-order optical flow and second-order flow prediction. We suggest an one-stage learning approach to encapsulate flow as quantized features from consecutive frames which is then entropy coded with adaptive contexts conditioned on joint spatial-temporal priors to exploit second-order correlations. Joint priors are embedded in autoregressive spatial neighbors, co-located hyper elements and temporal neighbors using ConvLSTM recurrently. We evaluate our approach for the low-delay scenario with High-Efficiency Video Coding (H.265/HEVC), H.264/AVC and another learned video compression method, following the common test settings. Our work offers the state-of-the-art performance, with consistent gains across all popular test sequences.

A method for reducing information redundancy of digital images for cloud storage

The article is devoted to the topical issue of finding new effective and improving existing widespread compression methods in order to reduce computational complexity and improve the quality of image-renewable image compression images, is important for the introduction of cloud technologies. The article presents a problem To increase the efficiency of cloud storage, it is necessary to determine methods for reducing the information redundancy of digital images by fractal compression of video content, to make recommendations on the possibilities of applying these methods to solve various practical problems. The necessity of storing high-quality video information in new HDTV formats 2k, 4k, 8k in cloud storage to meet the existing needs of users has been substantiated. It is shown that when processing and transmitting high quality video information there is a problem of reducing the redundancy of video data (image compression) provided that the desired image quality is preserved, restored by the user. It has been shown that in cloud storage the emergence of such a problem is historically due to the contradiction between consumer requirements for image quality and the necessary volumes and ways to reduce redundancy of video data, which are transmitted over communication channels and processed in data center servers. The solution to this problem is traditionally rooted in the search for effective technologies for compressing, archiving and compressing video information. An analysis of video compression methods and digital video compression technology has been performed, which reduces the amount of data used to represent the video stream. Approaches to image compression in cloud storage under conditions of preservation or a slight reduction in the amount of data that provide the user with the specified quality of the restored image are shown. Classification of special compression methods without loss and with information loss is provided. Based on the analysis, it is concluded that it is advisable to use special methods of compression with loss of information to store high quality video information in the new formats HDTV 2k, 4k, 8k in cloud storage. The application of video image processing and their encoding and compression on the basis of fractal image compression is substantiated. Recommendations for the implementation of these methods are given.

Global Rate-Distortion Optimization-Based Rate Control for HEVC HDR Coding

High dynamic range (HDR) video compression technology, which is capable of delivering a wider range of luminance and a larger colour gamut than standard dynamic range (SDR) technology, has been widely used in recent years in many fields, including industrial image processing, digital entertainment, and machine vision. Rate control (RC) is of paramount importance to HDR compression and transmission; accordingly, an RC scheme for HDR in High Efficiency Video Coding (HEVC) is proposed in this paper. First, considering the HDR characteristics, we propose an HDR-Visual Difference Predictor (VDP)-2-based rate-distortion (R-D) model to improve the coding performance. Second, we directly utilize $\lambda $ rather than the bit rate in the optimization process to obtain the optimal solution. Finally, we propose a new model parameter estimation method to further reduce the RC errors. According to our experimental results, significant bit rate reductions in terms of HDR-VDP-2, the Video Quality Metric (VQM) and the mean peak-signal-to-noise ratio (mPSNR) can be achieved on average compared with the state-of-the-art algorithm used in HM16.19.

Omnidirectional 360° Video Coding Technology in Responses to the Joint Call for Proposals on Video Compression With Capability Beyond HEVC

Augmented reality (AR) and virtual reality (VR) applications have seen rising popularity in recent years. Omnidirectional 360° video is a video format often used in AR and VR applications. To address the industry needs, a new HEVC edition recently published includes several supplemental enhancement information (SEI) messages to enable the carriage of omnidirectional video using HEVC. However, further improvement in 360° video compression efficiency is needed. In order to address this challenge, the Joint Video Exploration Team (JVET) of ITU-T VCEG and ISO/IEC MPEG has been investigating 360° video coding technologies, including projection formats, pre- and post-processing technologies, as well as 360°-video-specific coding tools since 2016. The joint call for proposals (CfP) recently issued by ITU-T VCEG and ISO/IEC MPEG on video compression technologies beyond HEVC included a category on 360° video. Twelve CfP responses in the 360° video category were received. This paper describes technologies relevant to 360° video for VVC. A summary of projection formats, pre- and post-processing methods, and 360°-video specific coding tool modifications in these proposals is provided.