Video Compression Research Articles

Review on Fractal Video and Image Compression Techniques

Compression of images process is a step in the image processing. It is concerned with the transmission and storage of digitally created images. Fractal coding is a potential image and video compression approach with good reconstruction fidelity and relatively large compression ratios, because of its simplicity and great performance; fractal image compression (FIC) is a particularly popular approach in image compression applications. However, it has a significant disadvantage in the form of a long encoding time. This is because encoding any small bit necessitates a massive similarity search in the original data As a result; the FIC search time is reduced while the quality of the reconstructed images is maintained acceptable level in many introduced paper and other still a study topic in progress. Fractal images are images that are self-similar in that each individual part is the same as the total. This paper will discusses many attempts for more author that working on image and video compression using fractal compression technique based on various approach and with each discuss focuses on the main parameter of compression such compression ratio (CR), peak signal to noise ratio (PSNR) and encoding time, as well as the details of data set the used for testing also writing with each technique to creating fractal video and image compression. Index Terms— FIC, Video compression, parallel processing, Iteration Function System (IFS), Image processing.

Signal transmission optimization moving-vector classification Mpeg standard video coding

The rise of useful multimedia applications has sparked a rush to perfect video compression methods. Over the last decade, there has been a lot of interest in using motion-compensated frame interpolation (MCFI) methods to boost video frame rate during playback in both the academic and consumer electronics sectors. Interpolation refers to the process of defining the values of a function at places that are not represented by any of the examples. It accomplishes this process by fitting a constant function across the discrete input samples. We based our methodology on the fact that color cameras function similarly to the human eye. We have been used to black and white photographs, yet the real world is full of color. Grayscale images were the sole option until the advent of color monitors and cameras. Energy assessment calls for photo capture using a camera. Light's electromagnetic waves provide the observable energy in this scheme. For motion-compensated video vector frame interpolation or frame rate up-conversion and Phase frame interpolation Method, the authors present a novel approach with low vector dispensation method in the conclusion. Using optical flow or other conventional methods, it is necessary to have precise pixel correspondences between pictures in order to compute interpolated frames in a video series. We provide a practical replacement by capitalizing on advances in phase-based approaches, which use the phase shift of individual pixels to express motion. In challenging interpolation conditions, such as major appearance changes, flow-based algorithms frequently create severe visual distortions, yet our solution fails gently. Our approach is particularly useful for retiming and interpolating frames in high-resolution video shot at a high frame rate. Estimating motion for the purpose of improving the quality of the decompressed video requires more research on the nature of the compression problem. a novel way to estimating motion that is analogous to full-search 3ss,4ss, block-matching, and parametric techniques. The primary focus will be research into both error-free compression and the development of novel motion estimation algorithms. The results of the research indicate that the suggested system has the potential to increase visual superiority and is also robust, especially when applied to video sequences with quick movements and challenging portions. By combining phase based and conventional Lagrangian, which is utilized to get better quality results and deal with variations in light more gracefully.

Signal transmission optimization moving-vector classification mpeg standard video coding

The rise of useful multimedia applications has sparked a rush to perfect video compression methods. Over the last decade, there has been a lot of interest in using motion-compensated frame interpolation (MCFI) methods to boost video frame rate during playback in both the academic and consumer electronics sectors. Interpolation refers to the process of defining the values of a function at places that are not represented by any of the examples. It accomplishes this process by fitting a constant function across the discrete input samples. We based our methodology on the fact that color cameras function similarly to the human eye. We have been used to black and white photographs, yet the real world is full of color. Grayscale images were the sole option until the advent of color monitors and cameras. Energy assessment calls for photo capture using a camera. Light's electromagnetic waves provide the observable energy in this scheme. For motion-compensated video vector frame interpolation or frame rate up-conversion and Phase frame interpolation Method, the authors present a novel approach with low vector dispensation method in the conclusion. Using optical flow or other conventional methods, it is necessary to have precise pixel correspondences between pictures to compute interpolated frames in a video series. We provide a practical replacement by capitalizing on advances in phase-based approaches, which use the phase shift of individual pixels to express motion. In challenging interpolation conditions, such as major appearance changes, flow-based algorithms frequently create severe visual distortions, yet our solution fails gently. Our approach is particularly useful for retiming and interpolating frames in high-resolution video shot at a high frame rate. Estimating motion for the purpose of improving the quality of the decompressed video requires more research on the nature of the compression problem. a novel way to estimating motion that is analogous to full-search 3ss,4ss, block-matching, and parametric techniques. The primary focus will be research into both error-free compression and the development of novel motion estimation algorithms. The results of the research indicate that the suggested system has the potential to increase visual superiority and is also robust, especially when applied to video sequences with quick movements and challenging portions. By combining phase based and conventional Lagrangian, which is utilized to get better quality results and deal with variations in light more gracefully.

A Survey on Compression Domain Image and Video Data Processing and Analysis Techniques

A tremendous amount of image and video data are being generated and shared in our daily lives. Image and video data are typically stored and transmitted in compressed form in order to reduce storage space and transmission time. The processing and analysis of compressed image and video data can greatly reduce input data size and eliminate the need for decompression and recompression, thereby achieving significant savings in memory and computation time. There exists a body of research on compression domain data processing and analysis. This survey focuses on the work related to image and video data. The papers cited are categorized based on their target applications, including image and video resizing and retrieval, information hiding and watermark embedding, image and video enhancement and segmentation, object and motion detection, as well as pattern classification, among several other applications. Key methods used for these applications are explained and discussed. Comparisons are drawn among similar approaches. We then point out possible directions of further research.

Machine Learning Based Fast QTMTT Partitioning Strategy for VVenC Encoder in Intra Coding

The newest video compression standard, Versatile Video Coding (VVC), was finalized in July 2020 by the Joint Video Experts Team (JVET). Its main goal is to reduce the bitrate by 50% over its predecessor video coding standard, the High Efficiency Video Coding (HEVC). Due to the new advanced tools and features included in VVC, it actually provides high coding performances—for instance, the Quad Tree with nested Multi-Type Tree (QTMTT) involved in the partitioning block. Furthermore, VVC introduces various techniques that allow for superior performance compared to HEVC, but with an increase in the computational complexity. To tackle this complexity, a fast Coding Unit partition algorithm based on machine learning for the intra configuration in VVC is proposed in this work. The proposed algorithm is formed by five binary Light Gradient Boosting Machine (LightGBM) classifiers, which can directly predict the most probable split mode for each coding unit without passing through the exhaustive process known as Rate Distortion Optimization (RDO). These LightGBM classifiers were offline trained on a large dataset; then, they were embedded on the optimized implementation of VVC known as VVenC. The results of our experiment show that our proposed approach has good trade-offs in terms of time-saving and coding efficiency. Depending on the preset chosen, our approach achieves an average time savings of 30.21% to 82.46% compared to the VVenC encoder anchor, and a Bjøntegaard Delta Bitrate (BDBR) increase of 0.67% to 3.01%, respectively.

Efficient, low complexity 8-point discrete tchebichef transform approximation for signal processing applications

The Discrete Tchebichef Transform (DTT) is widely used in signal processing applications such as image and video compression and digital watermarking due to its coding and decorrelation properties. In embedded systems and real-time applications, the need for fast, low complexity, and energy efficient algorithms is increasingly important. However, there is a limited number of such algorithms available for efficient computation of the DTT compared to the discrete cosine transform. To address this issue, a new DTT approximation is proposed, which provides a low-complexity and efficient solution by exploiting the deviation from orthogonality. The suggested algorithm requires fewer additions and bit-shifting operations compared to the existing ones. The implementation of this DTT approximation on the Xilinx Virtex-6 XC6VSX475T-1FF1759-2 FPGA resulted in a 19.20% decrease in hardware resources and 2% reduction in energy consumption for 1-D 8 inputs compared to the best candidate DTT approximation. With a low deviation from orthogonality, the efficiency of the proposed approximation has been confirmed in image compression and digital watermarking. The experimental results show an average quality improvement of 0.25 dB over the latest DTT approximation.

A Survey on Perceptually Optimized Video Coding

To provide users with more realistic visual experiences, videos are developing in the trends of Ultra High Definition (UHD), High Frame Rate (HFR), High Dynamic Range (HDR), Wide Color Gammut (WCG), and high clarity. However, the data amount of videos increases exponentially, which requires high efficiency video compression for storage and network transmission. Perceptually optimized video coding aims to maximize compression efficiency by exploiting visual redundancies. In this article, we present a broad and systematic survey on perceptually optimized video coding. Firstly, we present problem formulation and framework of the perceptually optimized video coding, which includes visual perception modeling, visual quality assessment, and perceptual video coding optimization. Secondly, recent advances on visual factors, computational perceptual models, and quality assessment models are presented. Thirdly, we review perceptual video coding optimizations from four key aspects, including perceptually optimized bit allocation, rate-distortion optimization, transform and quantization, and filtering and enhancement. In each part, problem formulation, working flow, recent advances, advantages, and challenges are presented. Fourthly, perceptual coding performances of the latest coding standards and tools are experimentally analyzed. Finally, challenging issues and future opportunities are identified.

Impact of Packet Loss Rate on Quality of Compressed High Resolution Videos.

Video delivered over IP networks in real-time applications, which utilize RTP protocol over unreliable UDP such as videotelephony or live-streaming, is often prone to degradation caused by multiple sources. The most significant is the combined effect of video compression and its transmission over the communication channel. This paper analyzes the adverse impact of packet loss on video quality encoded with various combinations of compression parameters and resolutions. For the purposes of the research, a dataset containing 11,200 full HD and ultra HD video sequences encoded to H.264 and H.265 formats at five bit rates was compiled with a simulated packet loss rate (PLR) ranging from 0 to 1%. Objective assessment was conducted by using peak signal to noise ratio (PSNR) and Structural Similarity Index (SSIM) metrics, whereas the well-known absolute category rating (ACR) was used for subjective evaluation. Analysis of the results confirmed the presumption that video quality decreases along with the rise of packet loss rate, regardless of compression parameters. The experiments further led to a finding that the quality of sequences affected by PLR declines with increasing bit rate. Additionally, the paper includes recommendations of compression parameters for use under various network conditions.

SOR-TC: Self-attentive octave ResNet with temporal consistency for compressed video action recognition

Modeling and recognizing video activities from videos are key parts of many promising applications such as visual surveillance, human–computer interaction, and video summarization. However, current approaches mainly suffer from two issues: (a) Short-term local and global spatial features are not well represented. The spatial redundancy and dependency have not been well considered on CNN-based action recognition, which may result in a further increase in both memory and computation cost. (b) Long-term temporal consistency is not well captured. The action consistency across multiple clips has been ignored in video-level action recognition approaches. To address these two issues, we propose a Self-Attentive Octave ResNet with Temporal Consistency (SOR-TC) for compressed video action recognition to better capture the short-term and long-term features in video and improve the efficiency and effectiveness of action recognition. In addition, this paper introduces a consistency hypothesis that adjacent clips should predict similar actions. So the consistency loss function is designed to learn the correlation of clips. Finally, extensive experimental results on two benchmark datasets HMDB-51 and UCF-101 verify the effectiveness of our proposed method.

Coronavirus disease 2019 patients prognostic stratification based on low complex lung ultrasound video compression

In the last years, efforts have been made towards automating semi-quantitative analysis of lung ultrasound (LUS) data. To this end, several methods have been proposed with a focus on frame-level classification. However, no extensive work has been done to evaluate LUS data directly at the video level. This study proposes an effective video compression and classification technique for assessing LUS data. This technique is based on maximum, mean, and minimum intensity projection (with respect to the temporal dimension) of LUS video data. This compression allows preserving hyper- and hypo-echoic regions and results in compressing a LUS video down to three frames, which are then classified using a convolutional neural network (CNN). Results show that this compression not only preserves visual artifacts appearance in the reduced data, but also achieves a promising agreement of 81.61% at the prognostic level. Conclusively, the suggested method reduces the amount of frames needed to assess LUS video down to 3. Note that on average a LUS videos consists of a few hundreds frames. At the same time, state-of-the-art performance at video and prognostic levels are achieved, while significantly reducing the computational cost.

Survey On Parameters of Data Compression

The rapid development in the hardware and the software gives rise to data growth. This data growth has numerous impacts, including the need for a larger storage capacity for storing and transmitting. Data compression is needed in today’s world because it helps to minimize the amount of storage space required to store and transmit data. Performance measures in data compression are used to evaluate the efficiency and effectiveness of data compression algorithms. In recent times, numerous data compression algorithms are developed to reduce data storage and increase transmission speed in this internet era. In order to analyses how data compression performance is measured in terms of text, image, audio, and video compressions. This survey presents discussion made for important data compression parameters according to their data types.

OVQE: Omniscient Network for Compressed Video Quality Enhancement

How to use information from temporal, spatial, and frequency domain dimensions is crucial for the quality enhancement of compressed video. The state-of-the-art methods generally design powerful networks to fuse the spatiotemporal information of the videos. But the spatiotemporal information of the entire video is not fully utilized and effectively fused, resulting in the learned context information that is not closely related to the target frame. In addition, various compressed videos have varying degrees of frequency domain information loss. The previous methods ignored the non-uniform distortion of compressed video in different frequency domains and did not design unique algorithms for different frequency domains, so the real texture details of the video could not be restored. In this paper, we propose an omniscient network, which learns video spatiotemporal and omni-frequency information more effectively. The omniscient network consists of two novel components: a Spatio-Temporal Feature Fusion (STFF) module and an Omni-Frequency Adaptive Enhancement (OFAE) block. The former aims to capture spatiotemporal information in adjacent frames, while the latter aims to adaptively recover different frequency domains of compressed video. The information is designed to be bidirectionally propagated in a grid manner such that the omni-enhanced results can be applied. Extensive experiments show that our method outperforms the state-of-the-art method in terms of objective metrics, subjective visual effects, and model complexity.

RE-MAC: A Hybrid MAC Protocol for Underwater Multimedia Communication System

In this article, we propose a reliable and efficient medium access control (MAC) protocol, RE-MAC, for resource-constraint underwater acoustic multimedia communication systems. RE-MAC achieves scalability by utilizing a hybrid MAC protocol smartly, which takes advantage of contention, reservation, and null data packet (NDP) based channel access mechanisms. The proposed scheme achieves multihop communication and synchronization for video streaming using a transmission opportunity (TXOP) based efficient channel access mechanism at the access point (AP) and relay nodes. TXOP time has been utilized by relay and AP for sending stream of frames with low overhead. To improve the efficiency in multihop communication, RE-MAC finds an optimal frame duration for relays and AP. RE-MAC efficiently places the NDP mechanism replacing traditional ACK-based control packet exchange. We incorporate the TXOP in AP and relay nodes for multimedia streaming and sleep mode for stations. Further, the video compression technique gives the efficiency of our system to improve the transmission of multimedia data. Our proposed schemes help communicate multimedia data without compromising the quality of service. We have demonstrated our proposed scheme experimentally in a prototype. After that, a simulation-based detailed performance analysis is provided to see the scalability of the proposed scheme. Compared to the existing state-of-the-art solutions, RE-MAC significantly delivers the throughput, energy consumption, delay, and packet loss.

Multi-Scale Inter-Communication Spatio-Temporal Network for Video Compression Artifacts Reduction

Video compression artifacts are widespread in online videos, which greatly affects the quality of the videos. In recent years, the deep learning-based methods for video compression artifacts reduction have made impressive achievements. However, most of existing methods ignore the rich multi-scale information contained in video frames. In this brief, we propose a new multi-scale inter-communication spatio-temporal network (MSICSTN) for video compression artifacts reduction by fully exploring contextual multi-scale information. MSICSTN is constructed on the basis of the multi-scale feature inter-communication module (MSFICM) and the source feature selection enhancement module (SFSEM). Specifically, the MSFICM adaptively generates respective attention weights using channel statistical information after the inter-communication of different scale features, and recalibrates the input multi-scale features to enhance the multi-scale features. The SFSEM utilizes the statistical information of the shallow features of the target frame to fuse the shallow features with the deep features for better dense pixel predictions. The experimental results on test videos demonstrate that our method outperforms the existing methods.

A Hardware-Friendlyand High-Efficiency H.265/HEVC Encoder for Visual Sensor Networks.

Visual sensor networks (VSNs) have numerous applications in fields such as wildlife observation, object recognition, and smart homes. However, visual sensors generate vastly more data than scalar sensors. Storing and transmitting these data is challenging. High-efficiency video coding (HEVC/H.265) is a widely used video compression standard. Compare to H.264/AVC, HEVC reduces approximately 50% of the bit rate at the same video quality, which can compress the visual data with a high compression ratio but results in high computational complexity. In this study, we propose a hardware-friendly and high-efficiency H.265/HEVC accelerating algorithm to overcome this complexity for visual sensor networks. The proposed method leverages texture direction and complexity to skip redundant processing in CU partition and accelerate intra prediction for intra-frame encoding. Experimental results revealed that the proposed method could reduce encoding time by 45.33% and increase the Bjontegaard delta bit rate (BDBR) by only 1.07% as compared to HM16.22 under all-intra configuration. Moreover, the proposed method reduced the encoding time for six visual sensor video sequences by 53.72%. These results confirm that the proposed method achieves high efficiency and a favorable balance between the BDBR and encoding time reduction.

Fractal video compression for IOT-based smart cities applications using motion vector estimation

Recently, Internet of the things (IoTs) devices have been used highly in surveillance applications in smart environments, cities, and buildings. Most of the surveillance data recorded by the IoT devices is in the form of video and needs to be transmitted to the admin or server by video streaming technology. Fractal video compression uses a self-similarity concept for video compression, meaning the fractal image contains self-similarity of itself that is explained and denoted by change. Computational cost is more in fractal video coding; different methods have been developed to reduce this computational cost. The proposed work's main objective is to implement a video streaming approach by combining the Diamond-search-pattern- block-matching motion estimation algorithm and hash-based fractal video compression algorithm to reduce the encoding time. It improves both the delay and security of the block matching system and the developed approach measures based on evaluation parameters: Peak signal-to-noise ratio (PSNR), encoding time, decoding time, Mean squared error (MSE), and Compression ratio for Smart City Applications. Compared to other algorithms, the IoTs video data is streamed efficiently in a smart city to analyse the abnormalities.

Multiple targets video tracking based on extended kalman filter in combination with particle swarm optimization for intelligent applications

Multiple targets tracking is a major issue in the intelligent applications. Numerous methods have been presented for the multiple targets tracking to capture the targets trajectory in a video sequence in order to increase intelligence and reduce human error. In this paper, a method is proposed based on combining the Extended Kalman Filter (EKF) and Particle Swarm Optimization (PSO) to construct an intermediate tracker and track targets more accurately. The EKF solves targets collision problem, and PSO reduces the covariance of measured noise. Finally, the Joint Probabilistic Data Association (JPDA) filter is used to reduce the number of multiple hypotheses and create a one-to-one correspondence between targets and measurements. To detect targets, frames subtracting along with background modeling and canny edge detector are used. To reduce running time of the proposed method, number of video frames per second (fps) is reduced from 30 to 10 and the sampling rate is also reduced. Despite of this reduction, simulation reults of the proposed method show the multiple targets tracking with 98% accuracy at an acceptable running time compared to the similar methods. In addition, by using the proposed method, the number of assignment states is reduced in the targets tracking process. Overall, the proposed method not only can be used in the intelligent applications, but also in the video compression applications as well.

Quality Enhancement of Compressed 360-Degree Videos Using Viewport-based Deep Neural Networks

360-degree video provides omnidirectional views by a bounding sphere, thus also called omnidirectional video. For omnidirectional video, people can only see specific content in the viewport through head movement, i.e., only a small portion of the 360-degree content is exposed at a given time. Therefore, the viewport quality is of particular importance for 360-degree videos. In this article, we propose a quality enhancement of compressed 360-degree videos using viewport-based deep neural networks, named V-DNN. V-DNN is mainly composed of two modules: viewport prediction network (VPN) and viewport quality enhancement network (VQEN). VPN based on spherical convolution and 2D convolution generates potential viewports for omnidirectional video. VQEN takes the current viewport and its reference viewports as the input and enhances residual for the current viewport based on bidirectional offset prediction and Spatio-temporal deformable convolutions. Compared with HM16.16 baseline at QP = 37 under the Low Delay P (LDP) configuration, experimental results show that V-DNN achieves an average 0.605 dB and 0.0139 gains in viewport-based ΔPSNR and ΔMS-SSIM, respectively, and is 0.379 dB (59.63%) and 0.0073 (110.61%) higher than the multi-frame quality enhancement (MFQE-2.0) scheme at QP = 37, respectively. Moreover, V-DNN consistently outperforms MFQE-1.0, MFQE-2.0, and HM16.16 baseline at the other QPs in terms of ΔPSNR, ΔWS-PSNR, and ΔMS-SSIM.

MPEG/H256 video encoder with 6T/8T hybrid memory architecture for high quality output at lower supply

The use of Multimedia video content is increased rapidly in the past decade, and most multimedia video content is used by mobile phone users. Multimedia video processing consumes significant power during video compression, and thus low power multimedia video compression is essential for battery operated devices. Moving Picture Experts Group (MPEG) Video encoding is giving a higher compression rate and low bandwidth requirement. Conventional MPEG Video encoding architecture uses the conventional 6T memory cells to store video frames for further compression processing. The failure probability of 6T cells is significantly large (0.0988 at 600 mV supply voltage), leading to a decrease in the output quality of the encoded video. From the hybrid memory matrix formulation, it is calculated that storing higher-order MSB bits in highly stable memory cells will provide high-quality video encoding processing as compared to the conventional technique because the human eye is more susceptible to higher-order luminance bits. Hence, in this research work instant of using conventional 6T memory cells during video encoding processing, a unique Hybrid 6T/8T memory architecture is proposed, where the 8-bit Luminance pixels are stored favourably in consonance with their effect on the output quality. The higher order luminance bits (MSB’s) require high stability and thus these bits are stored in the 8T bit cells and the remaining bits (LSB’s) are stored in the conventional 6T bit cells for high-quality video encoding processing. This research article also proposes a separate memory peripheral circuitry for hybrid memory architecture for video encoding techniques. In addition, this article proposes a unique architecture for parallel video processing with the use of a hybrid pixel memory array. The failure probability of 6T and 8T at the worst failure corner (FS corner for read and SF corner for write) is simulated for 30000 Monte-Carlo simulations points at 45 nm CMOS technology node using CADENCE EDA tool. For the simulation work here, a standard Common Intermediate Format/Quarter Common Intermediate Format (CIF/QCIF) Coastguard video sample is used and for output quality here average PSNR method is used and simulation work is performed using the MATLAB tool.The worst PSNR for conventional 6T memory array and Hybrid memory array at 600 mV supply voltage shows improvement in worst minimum PSNR as 6.43 dB is calculated. 30% less power consumption to conventional memory architecture.

Image and video compression of fluid flow data

We study the compression of spatial and temporal features in fluid flow data using multimedia compression techniques. The efficacy of spatial compression techniques, including JPEG and JPEG2000 (JP2), and spatiotemporal video compression techniques, namely H.264, H.265, and AV1, in limiting the introduction of compression artifacts and preserving underlying flow physics are considered for laminar periodic wake around a cylinder, two-dimensional turbulence, and turbulent channel flow. These compression techniques significantly compress flow data while maintaining dominant flow features with negligible error. AV1 and H.265 compressions present the best performance across a variety of canonical flow regimes and outperform traditional techniques such as proper orthogonal decomposition in some cases. These image and video compression algorithms are flexible, scalable, and generalizable holding potential for a wide range of applications in fluid dynamics in the context of data storage and transfer.Graphic abstract