MPEG-2 Bitstream Research Articles

Optimising lossless stages in a GPU-based MPEG encoder

Modern GPUs excel in parallel computations, so they are an interesting target to perform matrix transformations such as the DCT, a fundamental part of MPEG video coding algorithms. Considering a system to encode synthetic video (e.g., computer-generated frames), this approach becomes even more appealing, since the images to encode are already in the GPU, eliminating the costs of transferring raw video from the CPU to the GPU. However, after a raw frame has been transformed and quantized by the GPU, the resulting coefficients must be reordered, entropy encoded and framed into the resulting MPEG bitstream. These last steps are essentially sequential and their straightforward GPU implementation is inefficient compared to CPU-based implementations. We present different approaches to implement part of these steps in GPU, aiming for a better usage of the memory bus, compensating the suboptimal use of the GPU with the gains in transfer time. We analyze three approaches to perform the zigzag scan and Huffman coding combining GPU and CPU, and two approaches to assemble the results to build the actual output bitstream both in GPU and CPU memory. Our experiments show that optimising the amount of data transferred from GPU to CPU implementing the last sequential compression steps in the GPU, and using a parallel fast scan implementation of the zigzag scanning improve the overall performance of the system. Savings in transfer time outweigh the extra cost incurred in the GPU.

Data hiding in MPEG-2 bitstream by creating exceptions in code space

In this work, we have developed a file-size preserving algorithm to hide data directly in MPEG-2 bitstreams without violating the syntax. The approach is based on the realization that embedding of a bit directly in the bitstream can be achieved by creating an exception in the code space. An exception is created by inserting code words not occurring in the original content into the bitstream. Because the entropy-coded segment of MPEG-2 is fully used, every valid code word appears somewhere in the video. However, exceptions can still be created by recognizing that there are certain variable length codes (VLC) that never occur together in any video block. These are unused VLC pairs and although they vary from stream to stream, they always exist. It is then possible to embed one bit of data by forcing an unused pair in a block subject to a number of constraints, including transparency, file-size preservation and quality metric. The list of unused pairs constitutes a secure key that can be identified by pointer. This pointer can be efficiently encoded for sharing through a secure download. The content-dependence of the key allows a secure exchange of information because each key is valid only for the corresponding video. Because the cover video still uses standard VLCs, the marked bitstream is transparent to external observers. Application areas for the proposed approach are metadata binding and general data hiding when the size of the key is less important than the security and value of attaching data to content. Examples are surveillance video footage and medical applications.

Low complexity intra prediction algorithm for MPEG-2 to H.264/AVC transcoder

Transcoding MPEG-2 video into H.264/AVC format is highly necessary for the broad distribution of digital video contents to mobile terminals. Since H.264/AVC achieves great coding efficiency by employing computationally demanding algorithms, the computational complexity of the transcoder is one of technically challenging issues. The empirical results show that DCT energy trend in the decoded MB of MPEG-2 bitstream has strong correlation with the Intra prediction modes of H.264/AVC. Based on the relationship, the mode skipping rule of intra prediction mode decision in the H.264/AVC encoder part of the transcoder is proposed in this paper. The simulation results show that the proposed algorithm can achieve on average 72.36% reduction in computational complexity, compared with that of the conventional transcoder.

インタレース映像を対象としたＭＰＥＧ‐２／Ｈ．２６４イントラ再符号化方法の検討

An MPEG-2 to H.264 intra transcoding method for interlace contents is proposed. This method uses the encoding information from an MPEG-2 stream and keeps the DCT coefficients of the original MPEG-2 bitstream as much as possible. Experimental results show that the proposed method improves on the conventional methiod about 0.25 dB in PSNR.

ＤＣＴ成分を極力保存するプログレッシブ映像用ＭＰＥＧ‐２／Ｈ．２６４再符号化手法

An MPEG-2 to H.264 transcoding method preserving DCT information is proposed. This method uses the encoding information from an MPEG-2 stream and keeps the DCT coefficients of the original MPEG-2 bitstream as much as possible. Experimental results show that the proposed method increased the PSNR by about 0.20-1.11 dB compared with the conventional method.

Region-of-interest-based video transcoding from MPEG-2 to H.264 in the compressed domain

A new MPEG-2-to-H.264 video transcoding algorithm based on regions of interest (ROIs) is proposed. The whole transcoding scheme, including the ROI detection, is applied in the compressed domain to alleviate the computational load. The ROI detection uses the motion information from the partially decoded MPEG-2 bitstream. Then the ROIs that are likely to attract the viewer's attention are transcoded with drift compensation to provide better quality, while other regions are transcoded without drift compensation to save computational complexity. Experimental results show that our algorithm provides a better trade-off between computational complexity and picture quality. It achieves a speedup of 2 times over the closed-loop transcoding algorithm in the compressed domain, while giving comparable visual quality.

イントラ符号化されたＭＰＥＧ‐２プログレッシブ映像のＨ．２６４再符号化方法の検討

An MPEG-2 to H.264 intra transcoding method for progressive contents has been developed. This method uses the encoding information from the MPEG-2 stream and keeps as many DCT coefficients of the original MPEG-2 bitstream as possible. Experimental results show that the proposed method improves about 0.53-1.80 dB in PSNR compared with the conventional method.

Motion Flow-Based Video Retrieval

In this paper, we propose the use of motion vectors embedded in MPEG bitstreams to generate so-called ldquomotion flowsrdquo, which are applied to perform video retrieval. By using the motion vectors directly, we do not need to consider the shape of a moving object and its corresponding trajectory. Instead, we simply ldquolinkrdquo the local motion vectors across consecutive video frames to form motion flows, which are then recorded and stored in a video database. In the video retrieval phase, we propose a new matching strategy to execute the video retrieval task. Motions that do not belong to the mainstream motion flows are filtered out by our proposed algorithm. The retrieval process can be triggered by query-by-sketch or query-by-example. The experiment results show that our method is indeed superb in the video retrieval process.

MPEG-4 to H.264 transcoding with frame rate reduction

In this paper, a temporal resolution reduction transcoding method that transforms an MPEG-4 video bitstream into an H.264 video bitstream is proposed. The block modes and motion vectors in the MPEG-4 bitstream are utilized in the H.264 encoder for the block mode conversion and motion vector interpolation methods. Four types of motion vector interpolation methods are proposed in order to avoid the use of brute-force motion estimation in H.264. According to the experimental results, the proposed methods achieve a 3~4 times improvement in the computational complexity compared to the cascade pixel-domain transcoding method, while the PSNR (peak signal to noise ratio) is degraded by 0.2~0.9 dB depending on the bitrates.

A practical foveation-based rate-shaping mechanism for MPEG videos

Foveation is one of the nonuniform resolution properties of the human visual system. Recently, different foveation models are proposed and utilized for image and video coding, for the sake of bit-rate saving with no or minor perceptual quality distortion. In the first part of this paper, we propose an efficient and practical DCT-domain foveation model, which is deduced from existing experimental results. In the second part, we present a foveation-based rate-shaping mechanism for MPEG bitstreams, as an application example of the proposed foveation model. The rate shaper is based on eliminating DCT coefficients embedded in MPEG bitstreams. An efficient rate-shaping mechanism is developed to meet various bit-rate requirements. Our simulation confirmed that the proposed foveation model and the rate-shaping mechanism are practical for real-world usage.

Spatial and temporal error concealment algorithms of shape information for MPEG-4 video

In this paper, new methods for error concealment of shape information are proposed for compressed MPEG-4 bit-streams transmitted over error prone channels. For spatial domain, the proposed approach employs the Bezier curve fitting to approximate and outline the intra-video object plane (I-VOP). For temporal domain, the proposed scheme uses the weighted side matching criterion to reconstruct the inter-video object planes (P-/B-VOPs). The proposed algorithms get better subjective and objective qualities than previous works.

MPEG video markup language and its applications to robust video transmission

In this research, we develop the MPEG video markup language (MPML) to describe MPEG-4 video contents. As an XML designed specifically for MPEG-4 bitstreams, MPML provides a friendly support for random access, portability, interoperability, flexibility, and extendibility. We propose an efficient compression algorithm to reduce the amount of storage space for MPML documents. Then, as two applications, we show how to use MPML for error-resilient video transmission over wireless channels and video multicast over Internet. It is demonstrated that the MPML-based decoding algorithm can protect image quality effectively against transmission errors over noisy channels by using the random access support of MPML.

Dual Level Access Scheme for Digital Video Sequences

In this paper, a dual level access scheme is proposed to provide two levels of access to the broadcast data; one to video signals protected for authorized users, another to extra information e.g. advertisements provided for the remaining users in the network. In the scheme, video signals in MPEG format are considered. The video contents are protected from unauthorized viewing by encrypting the DC coefficients of the luminance component in I-frames, which are extracted from the MPEG bit-stream. An improved direct sequence spread spectrum technique is used to add extra information to non-zero AC coefficients, extracted from the same MPEG bit-stream. The resultant MPEG bit-stream still occupies the same existing bandwidth allocated for a broadcast channel. At the receiver, the extra information is recovered and subtracted from the altered AC coefficients. The result is then combined with the decrypted DC coefficients to restore the original MPEG bit-stream. The experimental results show that less than 2.9% of the size of MPEG bit-stream was required to be encrypted in order to efficiently reduce its commercial value. Also, on average, with a 1.125 Mbps MPEG bit-stream, an amount of extra information up to 1.4kbps could be successfully transmitted, while the video quality (PSNR) was unnoticeably degraded by 2.81 dB.

A real-time video transport system for the best-effort Internet

It is a challenging problem to design an efficient real-time video transport system for the present best-effort Internet because of the strict bandwidth, loss and time constraints and the lack of quality of service (QoS) guarantees from the present IP networks. This paper presents an end-to-end system architecture based on the MPEG-4 Delivery Multimedia Integration Framework (DMIF) for transporting real-time live video over the Internet. The key features of the system include: (1) MPEG-4 DMIF-based terminal architecture; (2) combination of an end-to-end feedback control mechanism and a rate-adaptive encoding algorithm for the best use of the Internet; (3) a robust and efficient packetization scheme for the MPEG-4 bit-stream by using the resynchronization marker approach specified in the MPEG-4 standard; and (4) efficient error control algorithms adopted at the end systems for visual quality enhancement. Simulation and test results using the actual Internet showed that our system is capable of utilizing the available network resource and achieve good perceptual quality at the application level.

Digital watermarking of low bit-rate advanced simple profile MPEG-4 compressed video

A novel MPEG-4 compressed domain video watermarking method is proposed and its performance is studied at video bit rates ranging from 128 to 768 kb/s. The spatial spread-spectrum watermark is embedded directly into compressed MPEG-4 bitstreams by modifying DCT coefficients. A synchronization template combats geometric attacks, such as cropping, scaling, and rotation. The method also features a gain control algorithm that adjusts the embedding strength of the watermark depending on local image characteristics, increasing watermark robustness or, equivalently, reducing the watermark's impact on visual quality. A drift compensator prevents the accumulation of watermark distortion and reduces watermark self-interference due to temporal prediction in inter-coded frames and AC/DC prediction in intra-coded frames. A bit-rate controller maintains the bit rate of the watermarked video within an acceptable limit. The watermark was evaluated and found to be robust against a variety of attacks, including transcoding, scaling, rotation, and noise reduction.

ROI Extraction from MPEG-2 bitstream and its Applicacion to Transcoder

ROI Extraction from MPEG-2 bitstream and its Applicacion to Transcoder

Adaptive rate control algorithm based on logarithmic R–Q model for MPEG-1 to MPEG-4 transcoding

This paper proposes an adaptive rate control algorithm for MPEG-1 to MPEG-4 transcoder for video internetworking where a compressed video is transmitted through networks of different capacities. It is based on an adaptive logarithmic R–Q (rate-quantization) model for controlling the new target bit-rate of a transcoded MPEG-4 bitstream. In R–Q modeling, the R–Q characteristics predicted from a previous picture of the same picture type are used in combination with the R–Q data measured for the current picture. The proposed adaptive model takes into account the effect of picture dependencies arising in motion-compensated video coding. Furthermore, full advantage is taken of the special situation related to transcoding where a priori knowledge of the current R–Q data can be obtained by just counting the number of bits in the input bitstream without re-encoding. It is shown by simulation that the adaptive R–Q model-based rate control can efficiently control the output bit-rate to match the target bit-budget.

地上デジタル放送用TS伝送シームレス切替装置の開発(デジタル放送技術, デジタル設備および一般)

地上デジタル放送用TS伝送シームレス切替装置の開発(デジタル放送技術, デジタル設備および一般)

Automatic TV advertisement detection from MPEG bitstream

The Centre for Digital Video Processing at Dublin City University conducts concentrated research and development in the area of digital video management. The current stage of development is demonstrated on our Web-based digital video system called Fı́schlár (Proceedings of the Content based Multimedia Information Access, RIAO 2000, Vol. 2, Paris, France, 12–14 April 2000, p. 1390), which provides for efficient recording, analysing, browsing and viewing of digitally captured television programmes. Advertisement breaks during or between television programmes are typically recognised by a series of ‘black’ video frames simultaneously accompanying a depression in audio volume which separate each advertisement from one another by recurrently occurring before and after each individual advertisement. It is the regular prevalence of these flags that enables automatic differentiation between what is programme and what is a commercial break. This paper reports on the progress made in the development of this idea into an advertisement detector system that automatically detects the commercial breaks from the bitstream of digitally captured television broadcasts.

STEREOSCOPIC VIDEO CODING USING A FAST AND ROBUST AFFINE MOTION SEARCH

This paper presents a stereoscopic video compression scheme using a novel fast affine motion estimation technique. A temporal scalable approach is used to achieve backward compatibility with a standard definition TV. We use an adaptive mode selection scheme from three temporal locations in both channels. Both block-based and affine-motion based approaches are used to achieve two levels of improvements with different complexities. An innovative motion estimation technique using Gauss–Newton optimization and pyramid processing is implemented to efficiently estimate affine parameters. Unlike other Gauss–Newton approaches, our search technique uses only addition, subtraction and multiplication and it converges within four iterations, which implies great complexity reduction. An efficient and robust affine motion prediction yields significant over the disparity-based approach. Part of the disparity-based approach has been tested in the rigorous MPEG-2 bitstream exchange process, and adopted in the MPEG-2 Multi-View Profile (MVP).