Compressed Video Frames Research Articles

Phase-based Full-field Displacement Measurement with Phase Nonlinearity Weighting Process

The camera, as a non-contact sensor, has evolved into a robust tool for measuring the full-field vibration of complex structures. It offers distinct advantages over traditional contact sensors, including flexible positioning, simultaneous multi-point tracking, and high spatial resolution. Vibration imaging techniques have been developed to identify structural operational conditions through feature extraction methods. Phase-based vibration imaging techniques can visualize full-field operational shapes or vibrational features in a less-supervised manner. While phase-based optical flow with Gabor filters provides accurate displacement estimation, managing a large amount of image data can pose challenges in extracting full-field displacement efficiently. In this paper, we propose a compressive sensing approach for full-field phase-based optical flow and vibration imaging. Compressive sensing serves as an efficient sampling method that requires significantly fewer frames than traditional approaches, selecting frames at less than one third of the original video. Features are extracted from the displacement of the compressed video frames. Subsequently, the vibration features are transformed into images. The proposed methodology is validated through experiments conducted on an air compressor. Comparative results between vibration imaging from video with compressive sensing and that from the original video demonstrate that vibrational features can be extracted using only a few frames of the original video. Consequently, this approach can reduce processing time and data volume while preserving the performance of vibration feature extraction.

Video Compression Artifact Reduction by Fusing Motion Compensation and Global Context in a Swin-CNN Based Parallel Architecture

Video Compression Artifact Reduction aims to reduce the artifacts caused by video compression algorithms and improve the quality of compressed video frames. The critical challenge in this task is to make use of the redundant high-quality information in compressed frames for compensation as much as possible. Two important possible compensations: Motion compensation and global context, are not comprehensively considered in previous works, leading to inferior results. The key idea of this paper is to fuse the motion compensation and global context together to gain more compensation information to improve the quality of compressed videos. Here, we propose a novel Spatio-Temporal Compensation Fusion (STCF) framework with the Parallel Swin-CNN Fusion (PSCF) block, which can simultaneously learn and merge the motion compensation and global context to reduce the video compression artifacts. Specifically, a temporal self-attention strategy based on shifted windows is developed to capture the global context in an efficient way, for which we use the Swin transformer layer in the PSCF block. Moreover, an additional Ada-CNN layer is applied in the PSCF block to extract the motion compensation. Experimental results demonstrate that our proposed STCF framework outperforms the state-of-the-art methods up to 0.23dB (27% improvement) on the MFQEv2 dataset.

Improving Compressed Video Using Single Lightweight Model with Temporal Fusion Module.

Video compression algorithms are commonly used to reduce the number of bits required to represent a video with a high compression ratio. However, this can result in the loss of content details and visual artifacts that affect the overall quality of the video. We propose a learning-based restoration method to address this issue, which can handle varying degrees of compression artifacts with a single model by predicting the difference between the original and compressed video frames to restore video quality. To achieve this, we adopted a recursive neural network model with dilated convolution, which increases the receptive field of the model while keeping the number of parameters low, making it suitable for deployment on a variety of hardware devices. We also designed a temporal fusion module and integrated the color channels into the objective function. This enables the model to analyze temporal correlation and repair chromaticity artifacts. Despite handling color channels, and unlike other methods that have to train a different model for each quantization parameter (QP), the number of parameters in our lightweight model is kept to only about 269 k, requiring only about one-twelfth of the parameters used by other methods. Our model applied to the HEVC test model (HM) improves the compressed video quality by an average of 0.18 dB of BD-PSNR and -5.06% of BD-BR.

One-Class Double Compression Detection of Advanced Videos Based on Simple Gaussian Distribution Model

Passive video forensics has become an active topic in recent years. Generally, a pristine video obtained from surveillance cameras or other video recording devices is single compressed. At the same time, double lossy compression will certainly be introduced in tampered videos since it needs to go through recompression to perform tampering on a commonly used compressed video. In the video’s double compression, some traces are left due to the intrinsic effects of recompression. Traditional supervised learning is inefficient because it requires two classes (the pristine and the manipulated) that occur in the video to be exhaustively assigned labels. Actually, compared with pristine videos, manipulated videos with labels are more difficult to obtain. To address this problem, in this paper, one-class classification, which is often used for anomaly detection and only needs the target class, is introduced. We first treat all decompressed video frames as still images and extract subtractive pixel adjacency matrix (SPAM) steganalysis features to detect traces left in the double compression process. Then we adopt a Gaussian density-based one-class classifier since SPAM features extracted from pristine video frames approximately subject to the Gaussian distribution. Furthermore, we improve the robustness of the classifier by using ensemble strategy. Experimental results indicate that our proposed method exceeds other more complex one-class classification methods, and outperforms fully-supervised learning methods only by feeding features from single compressed video frames to our one-class classifier.

MFQE 2.0: A New Approach for Multi-Frame Quality Enhancement on Compressed Video.

The past few years have witnessed great success in applying deep learning to enhance the quality of compressed image/video. The existing approaches mainly focus on enhancing the quality of a single frame, not considering the similarity between consecutive frames. Since heavy fluctuation exists across compressed video frames as investigated in this paper, frame similarity can be utilized for quality enhancement of low-quality frames given their neighboring high-quality frames. This task is Multi-Frame Quality Enhancement (MFQE). Accordingly, this paper proposes an MFQE approach for compressed video, as the first attempt in this direction. In our approach, we first develop a Bidirectional Long Short-Term Memory (BiLSTM) based detector to locate Peak Quality Frames (PQFs) in compressed video. Then, a novel Multi-Frame Convolutional Neural Network (MF-CNN) is designed to enhance the quality of compressed video, in which the non-PQF and its nearest two PQFs are the input. In MF-CNN, motion between the non-PQF and PQFs is compensated by a motion compensation subnet. Subsequently, a quality enhancement subnet fuses the non-PQF and compensated PQFs, and then reduces the compression artifacts of the non-PQF. Also, PQF quality is enhanced in the same way. Finally, experiments validate the effectiveness and generalization ability of our MFQE approach in advancing the state-of-the-art quality enhancement of compressed video.

Partition-Aware Adaptive Switching Neural Networks for Post-Processing in HEVC

This paper addresses neural network based post-processing for the state-of-the-art video coding standard, High Efficiency Video Coding (HEVC). We first propose a partition-aware Convolution Neural Network (CNN) that utilizes the partition information produced by the encoder to assist in the post-processing. In contrast to existing CNN-based approaches, which only take the decoded frame as input, the proposed approach considers the coding unit (CU) size information and combines it with the distorted decoded frame such that the artifacts introduced by HEVC are efficiently reduced. We further introduce an adaptive-switching neural network (ASN) that consists of multiple independent CNNs to adaptively handle the variations in content and distortion within compressed-video frames, providing further reduction in visual artifacts. Additionally, an iterative training procedure is proposed to train these independent CNNs attentively on different local patch-wise classes. Experiments on benchmark sequences demonstrate the effectiveness of our partition-aware and adaptive-switching neural networks. The source code can be found at http://min.sjtu.edu.cn/lwydemo/HEVCpostprocessing.html.

A Hybrid Modified Advanced Encryption Standard and Chaos Encryption Algorithm for Securing Compressed Multimedia Data

Multimedia data is being used more widely in the field of Internet communication technology. Over wireless networks, extensive use of multimedia applications can easily be intercepted. The widespread use of multimedia data makes media content security and protection increasingly urgent and necessary. In order to fully protect or maintain security and privacy of multimedia data, it must be secured through encryption before transmission. Multimedia encryption is the core enabling technology that provides confidentiality and end-to-end security preventing unauthorized access of the information. In this paper, an encryption algorithm is proposed to achieve the desired integrity. Compressed video frames usingH.264 with multi-view video coding MVC are encrypted using a hybrid algorithm based on modified advanced encryption standard AES with 1D, and 2D chaotic maps. The encryption process replaces add round key and S-box operations of AES encryption by XOR operation and chaotic map generators on different blocks of input data frames. The results of our simulation using MATLAB show that the proposed algorithm is suitable for multimedia encryption applications.

Evaluation of Data Compression Techniques for Video Transmission over Wireless Sensor Networks

Video transmission over WSN is a challenging task because, video data is inherently huge in size and its transmission requires high bandwidth and processing requires more memory and more power. Presently majority of the research work in this area is focused on improving battery life, optimizing network parameters and increasing energy efficiency. The area of best suitable compression scheme for WSN is rarely being explored. This paper focuses on finding out a suitable compression technique for video transmission over WSN. Wireless sensor network (WSN) is an ad-hoc network of sensor nodes, where each node is able to communicate with every other node in the network in single hop or multi-hop manner. It is a low cost, low power and low bandwidth network with each node having limited battery life and limited memory. In order to overcome these challenges, initially image compression is applied on each frame extracted from given video. To find out most suitable compression technique for WSN, the various existing 2D image transforms like DCT, KLT, Slant, Hartley, Hadamard and DST are compared based upon their energy compaction property. The transform that gives more energy compaction is best suitable for WSN. Thus, after selecting a suitable transform for WSN domain, it is applied to obtain compressed video frames. Zigbee protocol based hardware setup is used for serial transmission of RGB video data frames between the nodes. Different parameters are evaluated for received image frames and transmitted image frames. Experimental evaluation shows that zigbee hardware set up improves the reliability and efficiency of video data transmission. This type of WSN set-up can be used for capturing video in any remote and hard-lying (border, mountains, forest) area, where any other types of networks are not available.

Efficient hybrid framework for transmission enhancement of composite 3D H.264 and H.265 compressed video frames

Three-Dimensional Multi-View Video (3D-MVV) transmission over wireless networks suffers from losses. Therefore, the robust performance of 3D-MVV transmission techniques over wireless channels has become a recent hot research issue due to the restricted resources and the presence of severe channel errors. The 3D-MVV is composed of multiple video stream shots by several cameras around a single object, simultaneously. So, it is an urgent task to achieve high compression ratios to meet future bandwidth constraints. Unfortunately, the highly-compressed 3D-MVV data becomes more sensitive and vulnerable to packet losses, especially in the case of heavy channel errors. Thus, in this paper, we propose the application of a chaotic Baker map interleaving technique with equalization for efficient transmission of composite 3D-MVV compressed frames over an Orthogonal Frequency Division Multiplexing (OFDM) wireless channel. Rayleigh fading and Additive White Gaussian Noise (AWGN) are considered in the real scenario of 3D-MVV transmission. Firstly, the 3D-MVV content is compressed exploiting the intra- and inter-prediction correlations between frames. After that, a composite frame of luminance is generated from each four consecutive frames using Discrete Cosine Transform (DCT), which represents a second level of compression. The resultant composite frame is converted to binary data format. Then, chaotic interleaving is applied on the binary information prior to the modulation process. This chaotic interleaving is used to mitigate the OFDM induced Peak-to-Average Power Ratio (PAPR) problem and to reduce the wireless channel effects on the transmitted bit streams. It also adds a degree of encryption to the transmitted 3D-MVV compressed frames. To evaluate the performance of the proposed hybrid technique, several simulation experiments on different 3D-MVV frames have been executed. The experimental results show that the received 3D-MVV frames have high average Peak Signal-to-Noise Ratio (PSNR) gains up to 4.25 dB and a reduction of the average PAPR values by about 12 dB with the proposed hybrid technique compared to the other traditional techniques.

Chaotic Interleaving for the Transmission of Compressed Video Frames with Self-Embedded Digital Signatures

This paper presents an efficient video compression and transmission scheme based on the Set Partitioning In Hierarchical Trees (SPIHT) algorithm and chaotic Baker map interleaving. The compressed video frames have self-embedded digital signatures, and the transmission is performed over Orthogonal Frequency Division Multiplexing wireless systems with equalization. The video frames compressed with SPIHT algorithm are primarily converted to the luminance and chrominance scheme for self-watermark embedding, and then the binary format is reconstructed again. Prior to the modulation step, the chaotic randomization is performed on the binary data to encrypt the data to be transmitted and reduce the noise impact of the multipath channel. For performance evaluation, both Rayleigh and additive noise channels are considered for various video frames with chaotic interleaving. The simulation results reveal that the received watermarked video frames with chaotic interleaving applied achieve higher Peak Signal-to-Noise Ratios and correlation coefficients compared to other approaches.

Intra Prediction Mode Decision for H.264

Digital video compression has become an integral part of the way we create, consume visual information and communicate, over the last few decades. A robust technique of compression is proposed in this paper for video compression. Reduction of irrelevant or redundant data in order to save the storage space requirements and processing time is nothing but compression. Here H.264/AVC (Advance Video Coding) video coding standard is used for compression. Where I-frames are divided into different macro blocks (MB) and each MB is efficiently compressed using 4 x4 and 16 x16 blocks in H.264/AVC intra prediction. Choosing one of 9 prediction modes for each 4 x4 block with reduced time and less complexity is still a bottleneck. In this paper a gradient based fast intra prediction mode selection method for 4x4 and 16x16 is proposed. The proposed method divides an input frames into variable block sizes based on the texture and performs few mode examinations based on the gradient direction in the given slice. Respective best mode with minimum cost is selected using sum of absolute difference (SAD) and rate distortion optimization (RDO). This process is carried for all the video input frames. Each compressed video frames are combined finally to get a compressed video output.

Motion Detection in Compressed Video Using Macroblock Classification

In this paper, to detect the moving objects between frames in compressed video and to obtain the best compression video and the noiseless video. We describe a video in which frames by classifying macroblocks (MB), and describe motion estimation (ME), motion vector field (MV) and motion compensation (MC). we propose to classify Macroblocks of each video frame into different classes and use this class information to describe the frame content based on the motion vector. MB class information video applications such as shot change detection, motion discontinuity detection, Outlier rejection for global motion estimation. To reduce the noise and to improve the clarity of the compressed video by using contrast limited adaptive histogram equalization (CLAHE) Algorithm.

Data Hiding in Motion Vectors of Compressed Video Based on Their Associated Prediction Error

This paper deals with data hiding in compressed video. Unlike data hiding in images and raw video which operates on the images themselves in the spatial or transformed domain which are vulnerable to steganalysis, we target the motion vectors used to encode and reconstruct both the forward predictive (P)-frame and bidirectional (B)-frames in compressed video. The choice of candidate subset of these motion vectors are based on their associated macroblock prediction error, which is different from the approaches based on the motion vector attributes such as the magnitude and phase angle, etc. A greedy adaptive threshold is searched for every frame to achieve robustness while maintaining a low prediction error level. The secret message bitstream is embedded in the least significant bit of both components of the candidate motion vectors. The method is implemented and tested for hiding data in natural sequences of multiple groups of pictures and the results are evaluated. The evaluation is based on two criteria: minimum distortion to the reconstructed video and minimum overhead on the compressed video size. Based on the aforementioned criteria, the proposed method is found to perform well and is compared to a motion vector attribute-based method from the literature.

Packet loss in peer-to-peer video streaming over the Internet

Peer-to-peer streaming has recently gained attention as an effective solution to support large scale media streaming applications over the Internet. One of the main challenges of peer-to-peer video streaming is the cumulative impact of the Internet packet loss due to the decoding dependency of the compressed video frames. In this paper we study the impact of the Internet packet loss on the performance of peer-to-peer video streaming systems, and analyze the efficiency of various packet loss recovery policies in such systems. Our analytical and simulation results show how the Internet packet loss can affect the performance of peer- to-peer video streaming systems and how different packet loss recovery policies can be effective for such systems. Our analysis results give us some insights that can be used in designing efficient peer-to-peer video streaming systems.