Compressed Domain Video Research Articles

Image processing‐based Athletes' injury prevention mechanism

AbstractThis study proposes a compressed domain video action recognition to prevent athlete injuries. The main idea is to use the motion vector and residual of the compressed video to construct new spatiotemporal features while extracting compressed code stream information. The new spatiotemporal features have the spatiotemporal relationship between motion vectors and residuals and the characteristic of precise object edges. Through verification on mainstream action recognition datasets (HMDB‐51, UCF‐101), the proposed method has a more negligible computational overhead than action recognition based on the traditional pixel domain and higher recognition accuracy than action recognition based on the video compression domain. Experiments have shown that the new spatiotemporal features based on the compressed domain have advantages such as solid spatiotemporal relationships and high information density, which can make action recognition results with high accuracy, detect abnormalities or patterns associated with injuries, and analyze an athlete's movements and provide personalized training recommendations.

Fast Human Pose Estimation in Compressed Videos

Current approaches for human pose estimation in videos can be categorized into per-frame and warping-based methods. Both approaches have their pros and cons. For example, per-frame methods are generally more accurate, but they are often slow. Warping-based approaches are more efficient, but the performance is usually not good. To bridge the gap, in this paper, we propose a novel fast framework for human pose estimation to meet the real-time inference with controllable accuracy degradation in compressed video domain. Our approach takes advantage of the motion representation (called ‘`motion vector’') that is readily available in a compressed video. Pose joints in a frame are obtained by directly warping the pose joints from the previous frame using the motion vectors. We also propose modules to correct possible errors introduced by the pose warping when needed. Extensive experimental results demonstrate the effectiveness of our proposed framework for accelerating the speed of top-down human pose estimation in videos.

Mutual information regularized identity-aware facial expression recognition in compressed video

How to extract effective expression representations that invariant to the identity-specific attributes is a long-lasting problem for facial expression recognition (FER). Most of the previous methods process the RGB images of a sequence, while we argue that the off-the-shelf and valuable expression-related muscle movement is already embedded in the compression format. In this paper, we target to explore the inter-subject variations eliminated facial expression representation in the compressed video domain. In the up to two orders of magnitude compressed domain, we can explicitly infer the expression from the residual frames and possibly extract identity factors from the I frame with a pre-trained face recognition network. By enforcing the marginal independence of them, the expression feature is expected to be purer for the expression and be robust to identity shifts. Specifically, we propose a novel collaborative min-min game for mutual information (MI) minimization in latent space. We do not need the identity label or multiple expression samples from the same person for identity elimination. Moreover, when the apex frame is annotated in the dataset, the complementary constraint can be further added to regularize the feature-level game. In testing, only the compressed residual frames are required to achieve expression prediction. Our solution can achieve comparable or better performance than the recent decoded image-based methods on the typical FER benchmarks with about 3 times faster inference.

A hybrid DST-SBPNRM approach for compressed video steganography

The proposed research work is presenting a novel approach in the field of steganography, especially in the compressed video domain with optimum imperceptibility to secure the secret information. In this approach, the specific secret cover video frames are selected from the sequence of video frame from which the non-dynamic region is separated. Discrete sine transform (DST) transforms this non-dynamic region from spatial domain to frequency domain. The least significant bits (LSBs) of the integer part of DST components are used to conceal the secret data. The H.264 codec is used to construct the compressed stego video using intra-frame, inter-frame prediction, motion vector estimation, transform coefficient, i.e., Discrete Cosine Transform (DCT), quantization, and entropy coding. The efficiency of the proposed hybrid technique “DST- Secret Bit Positions of Non-dynamic Region for Message (DST-SBPNRM)” for video steganography is measured by evaluating imperceptibility, robustness and embedding capacity. Moreover, the proposed technique is experimented on the well-defined video dataset and the obtained results are compared with the related work to validate the significance of the proposed work.

Compressed Domain Deep Video Super-Resolution.

Real-world video processing algorithms are often faced with the great challenges of processing the compressed videos instead of pristine videos. Despite the tremendous successes achieved in deep-learning based video super-resolution (SR), much less work has been dedicated to the SR of compressed videos. Herein, we propose a novel approach for compressed domain deep video SR by jointly leveraging the coding priors and deep priors. By exploiting the diverse and ready-made spatial and temporal coding priors (e.g., partition maps and motion vectors) extracted directly from the video bitstream in an effortless way, the video SR in the compressed domain allows us to accurately reconstruct the high resolution video with high flexibility and substantially economized computational complexity. More specifically, to incorporate the spatial coding prior, the Guided Spatial Feature Transform (GSFT) layer is proposed to modulate features of the prior with the guidance of the video information, making the prior features more fine-grained and content-adaptive. To incorporate the temporal coding prior, a guided soft alignment scheme is designed to generate local attention off-sets to compensate for decoded motion vectors. Our soft alignment scheme combines the merits of explicit and implicit motion modeling methods, rendering the alignment of features more effective for SR in terms of the computational complexity and robustness to inaccurate motion fields. Furthermore, to fully make use of the deep priors, the multi-scale fused features are generated from a scale-wise convolution reconstruction network for final SR video reconstruction. To promote the compressed domain video SR research, we build an extensive Compressed Videos with Coding Prior (CVCP) dataset, including compressed videos of diverse content and various coding priors extracted from the bitstream. Extensive experimental results show the effectiveness of coding priors in compressed domain video SR.

Detection and localization of inter-frame forgeries in videos based on macroblock variation and motion vector analysis

Surveillance videos and footages are the primary sources of evidence for any event or crime in the court of law. However, with the rapid advent of low-cost, computationally cheap video manipulating software and tools, video manipulation has become a no-brainer task today. This introduces a major challenge in authenticating the sanctity/originality of videos before they can be produced in the court, or used in other sensitive application domains. In this paper, we propose a digital forensic technique to detect inter-frame forgeries in surveillance videos. The proposed technique utilizes compressed domain video footprints i.e, prediction footprint variation and variation of motion vectors in videos, for the purpose of video forgery detection and localization. Through this work, we identify the type of forgery that has taken place in a video. We have performed experiment over 43 authentic and 720 inter-frame forged videos. Our experimental results indicate that the proposed technique performs consistently efficiently, irrespective of the group of pictures length and degree of compression in videos.

Video Abstraction in H.264/AVC Compressed Domain

Video abstraction allows searching, browsing and evaluating videos only by accessing the useful contents. Most of the studies are using pixel domain, which requires the decoding process and needs more time and process consuming than compressed domain video abstraction. In this paper, we present a new video abstraction method in H.264/AVC compressed domain, AVAIF. The method is based on the normalized histogram of extracted I-frame prediction modes in H.264 standard. The frames’ similarity is calculated by intersecting their I-frame prediction modes’ histogram. Moreover, fuzzy c-means clustering is employed to categorize similar frames and extract key frames. The results show that the proposed method achieves on average 85% accuracy and 22% error rate in compressed domain video abstraction, which is higher than the other tested methods in the pixel domain. Moreover, on average, it generates video key frames that are closer to human summaries and it shows robustness to coding parameters.

Survey of Compressed Domain Video Summarization Techniques

Video summarization is the method of extracting key frames or clips from a video to generate a synopsis of the content of the video. Generally, video is compressed before storing or transmitting it in most of the practical applications. Traditional techniques require the videos to be decoded to summarize them, which is a tedious job. Instead, compressed domain video processing can be used for summarizing videos by partially decoding them. A classification and analysis of various summarization techniques are presented in this article with special focus on compressed domain techniques along with a discussion on machine-learning-based techniques that can be applied to summarize the videos.<?vsp -1.2pt?>

Toward a secure video watermarking in compressed domain

Collusion attack is a major concern in the field of video watermarking. To overcome this problem, a classic approach is exploiting a content-based key in order to select the embedding locations. This key should be chosen carefully especially in the blind embedding to prevent the possibility of any alteration after inserting the watermark. Extracting a robust content-based key is a more challenging issue in blind compressed domain watermarking since the syntactic elements such as DCT residuals may change after re-encoding even without inserting the watermark. Consequently, it is really desirable to derive a robust content-based key using just the syntactic elements of compressed video sequences. In this paper, we utilize the inter-mode values, as available information, to provide a robust content-based key. The key is evaluated and found to be robust against common compressed domain video watermarking attacks. In addition, the key generation mechanism is appropriate for real-time applications since the full decompression and recompression of the video stream are avoided.

Robust compressed domain watermarking algorithm for video protection and authentication in noisy channels

This paper introduces a robust and noise-resilient compressed domain video watermarking technique for data authentication and copyright protection. In recent years, watermarking has emerged as an essential technique to be equipped with data transmission. The main challenge pertaining to transmission is to protect the watermark from noise introduced by the channel. Here, we address this issue by watermark replication and by using the independent pass coding (INPAC) algorithm for compression. A replicated watermark is embedded into the video by the proposed blind video watermarking algorithm and then the watermarked video is compressed by the INPAC algorithm. The compressed video is transmitted through binary symmetric channels and tested under various bit error rates to evaluate the proposed algorithm in noisy environments. The results obtained suggest that the proposed algorithm is capable of handling noise efficiently and the overhead due to key embedding is observed to be only about 3%.

Compressed Domain Moving Object Detection Based on CRF

This paper aims to present a novel accurate moving object detection method based on the conditional random field (CRF) for high efficiency video coding/H.265 compressed domain video sequences. For each block, the number of consumed bits, motion vectors (MVs), and partitioning modes for a given block is extracted from the compressed bitstream. After removing outlier MVs, compensating MVs are assigned to the I-blocks based on their neighboring blocks. The information, such as MV, partitioning mode, and bit consumption, is used in the potential functions of a CRF model which is updated for every frame to detect the objects. Then, a number of standard test video sequences are used to verify the performance of the model. The results indicated that the model can offer a precision, that is more than 90% on average for the video sequences. The proposed method offers a 1.8 speedup, compared to the latest works in the compressed domain without losing the objects in the I-frames.

Compressed Domain Video Abstraction Based on I-Frame of HEVC Coded Videos

Video abstraction allows indexing, searching, browsing and evaluating a video only by accessing its useful contents. Several studies have been done in this field, but most of them are in pixel domain and require decoding process. It makes these methods more time and process consuming than compressed domain video abstraction. In this paper, we present a new video abstraction method in H.265/HEVC compressed domain, HVAIF. The method is based on the normalized histogram of extracted I-frame prediction modes from an H.265/HEVC coded video. The frames’ similarity is calculated by intersecting their I-frame prediction modes’ histogram. The similarity measure detects and removes redundant key-frames to increase the quality of final video abstraction. Moreover, we employ fuzzy c-means clustering to categorize similar frames and extract key-frames as representatives of the entire video frames. The interpretation of the results shows that using the proposed method achieves on average 86% accuracy and 19% error rate in compressed domain video abstraction which is higher than the other tested methods in the pixel domain. Also, it has an acceptable robustness to coding parameters, and on average it generates video key-frames that are closer to human summaries.

Noise-resilient compressed domain video watermarking system for in-car camera security

In-car camera system emerges as a very useful technology for automobile drivers, as it can provide proper evidence to insurance companies and to police investigators in case of car accidents. Dashcam, being one of the car video storage devices, stores the data in the SD card which is overwritten and can be tampered very easily. Thus, it is important to develop an in-car security system, where data can be stored in the server and can provide privacy to the user. In this paper, we have proposed a noise-resilient compressed domain video watermarking system for in-car security to document the vehicle history which can be a proof for any mischief. We have used Independent Pass Coding (INPAC) compression technique for designing the system. Here, data generated by the system is stored in the server database which is accessible to the authentic registered user and administrator in case of any claim. This system ensures copyright, proprietorship, authentication and security against third party intrusion. The proposed system is implemented in off-line mode to evaluate the robustness and efficiency of the algorithm, and in online mode to verify the system. Graphical user interface is designed for the end user access, to make it user friendly.

Genuine reversible data hiding technology using compensation for H.264 bitstreams

Reversible data hiding technologies have been considered largely impractical because those are, in most cases, applicable to raw video data rather than prevailing compressed data. Even though, many algorithms have been recently developed in the compressed video domain, most of them cannot guarantee the reversibility of cover video due to the lossy characteristics of video compression standards. We suggest completely practical data hiding scheme for H.264 baseline bitstream by achieving genuine reversibility for both I and P frames. Regardless of the data hiding algorithm, the proposed scheme can increase embedding payload by 66.9% and reduce computational complexity by 93%. Also, a novel compensation based difference expansion method with clever coefficient pairing strategy is proposed as a data hiding algorithm and achieved superior embedding payload vs. image quality performance. The proposed algorithm improves payload by 48.9% on average at almost the same video quality distortion.

Review of Research on Shot Boundary Detection Algorithm of the Compressed Video Domain in Content-based Video Retrieval Technique

Based on the background of extensive use of the video retrieval technique in substation monitoring, this paper focuses on the research of the shot boundary detection methods in the video retrieval technique. The research content is mainly involved in the following four kinds of shot boundary detection methods: the shot boundary detection method based on the base tone, the shot boundary detection method based on the macro- block motion information, the shot boundary detection method based on the motion vector and the video shot segmentation method based on clustering. Through the in-depth explanation and comparison of four kinds of shot boundary detection methods, this paper systematically summarizes the characteristics of four kinds of shot boundary detection methods, and also vividly points out the problems of four kinds of shot boundary detection methods, thus pointing out the direction for the future development of video retrieval technique.

Efficient motion estimation methods for fast recognition of activities of daily living

This work proposes a framework for the efficient recognition of activities of daily living (ADLs), captured by static color cameras, applicable in real world scenarios. Our method reduces the computational cost of ADL recognition in both compressed and uncompressed domains by introducing system level improvements in State-of-the-Art activity recognition methods. Faster motion estimation methods are employed to replace costly dense optical flow (OF) based motion estimation, through the use of fast block matching methods, as well as motion vectors, drawn directly from the compressed video domain (MPEG vectors). This results in increased computational efficiency, with minimal loss in terms of recognition accuracy. To prove the effectiveness of our approach, we provide an extensive, in-depth investigation of the trade-offs between computational cost, compression efficiency and recognition accuracy, tested on bench-mark and real-world ADL video datasets.

A fast and efficient saliency detection model in video compressed-domain for human fixations prediction

Research and application of human fixations detection in video compressed-domain have gained an increasing attention in the latest years. However, both prediction accuracy and computational complexity still remain a challenge. This paper addresses the problem of compressed-domain video human fixations prediction based on saliency detection, and presents a fast and efficient algorithm based on Residual DCT Coefficients Norm (RDCN feature) and Operational Block Description Length (OBDL feature). These two features are directly extracted from the compressed bit-stream with partial decoding, and are normalized. After spatial and temporal filtering, the normalized salient maps are fused by the dynamic fusion coefficients with variation of quantization parameters. Then the fused salient map is worked by Gaussian model whose center is determined by the feature values. The proposed saliency detection model for human fixations prediction combines the accuracy of the pixel-domain saliency detections with the computational efficiency of their compressed-domain counterparts. The validation and comparison are made by several accuracy metrics on two ground truth datasets. Experimental results show that the proposed saliency detection model for human fixations prediction obtains superior performances over several state-of-the-art compressed-domain and pixel-domain algorithms on evaluation metrics. Computationally, our algorithm achieves a speed-up of over 10 times as compared to similar algorithms, which illustrates it appropriate for in-camera saliency estimation.

Video processing for panoramic streaming using HEVC and its scalable extensions

Panoramic streaming is a particular way of video streaming where an arbitrary Region-of-Interest (RoI) is transmitted from a high-spatial resolution video, i.e. a video covering a very “wide-angle” (much larger than the human field-of-view – e.g. 360°). Some transport schemes for panoramic video delivery have been proposed and demonstrated within the past decade, which allow users to navigate interactively within the high-resolution videos. With the recent advances of head mounted displays, consumers may soon have immersive and sufficiently convenient end devices at reach, which could lead to an increasing demand for panoramic video experiences. The solution proposed within this paper is built upon tile-based panoramic streaming, where users receive a set of tiles that match their RoI, and consists in a low-complexity compressed domain video processing technique for using H.265/HEVC and its scalable extensions (H.265/SHVC and H.265/MV-HEVC). The proposed technique generates a single video bitstream out of the selected tiles so that a single hardware decoder can be used. It overcomes the scalability issue of previous solutions not using tiles and the battery consumption issue inherent of tile-based panorama streaming, where multiple parallel software decoders are used. In addition, the described technique is capable of reducing peak streaming bitrate during changes of the RoI, which is crucial for allowing a truly immersive and low latency video experience. Besides, it makes it possible to use Open GOP structures without incurring any playback interruption at switching events, which provides a better compression efficiency compared to closed GOP structures.

A Secure and Robust Compressed Domain Video Steganography for Intra- and Inter-Frames Using Embedding-Based Byte Differencing (EBBD) Scheme.

This paper presents a novel secure and robust steganographic technique in the compressed video domain namely embedding-based byte differencing (EBBD). Unlike most of the current video steganographic techniques which take into account only the intra frames for data embedding, the proposed EBBD technique aims to hide information in both intra and inter frames. The information is embedded into a compressed video by simultaneously manipulating the quantized AC coefficients (AC-QTCs) of luminance components of the frames during MPEG-2 encoding process. Later, during the decoding process, the embedded information can be detected and extracted completely. Furthermore, the EBBD basically deals with two security concepts: data encryption and data concealing. Hence, during the embedding process, secret data is encrypted using the simplified data encryption standard (S-DES) algorithm to provide better security to the implemented system. The security of the method lies in selecting candidate AC-QTCs within each non-overlapping 8 × 8 sub-block using a pseudo random key. Basic performance of this steganographic technique verified through experiments on various existing MPEG-2 encoded videos over a wide range of embedded payload rates. Overall, the experimental results verify the excellent performance of the proposed EBBD with a better trade-off in terms of imperceptibility and payload, as compared with previous techniques while at the same time ensuring minimal bitrate increase and negligible degradation of PSNR values.

A Review on Data Hiding Techniques in Compressed Video

This paper presents various Data Hiding techniques of video in compressed domain. Data is embedded into digital media for the purpose of identification, annotation, copy right and tampering detection. The identification purpose mainly falls into the area of cloud computing- making sure that the sender is the one intended; annotation refers to the property of tagging or giving captions to the video; copyright protection, which is used to prevent the recreation of the video; tampering detection is the method in which the receiver checks whether the video is tampered or disturbed during the transmission by a third party. Several factors affect the data hiding process-the quantity of data to be hidden, whether the embedding technique can handle large payloads without affecting the quality of the video, embedding in a lossy compression environment and the extent to which the data hidden is subject to modification or removal by a third party. This paper discusses the various data hiding techniques of video in the compressed environment and compares the techniques based on the quantities mentioned above. Furthermore they are analyzed in terms of their method, robustness and capacity. A comparison graph is plotted based on the efficiency analysis of the data hiding techniques and conclusions are drawn based on the analysis.