Video Bitstream Research Articles

Error concealment techniques for digital TV

Compressed video bitstreams are intended for real-time transmission over communication networks. Most of the video coding standards employ the temporal and spatial prediction structure to reduce the transmitted video data. Therefore, the coded video bitstreams are highly sensitive to information loss and channel errors. Even a single bit error can lead to disastrous quality degradation in both time and space. This quality deterioration is exacerbated when no error resilient coding mechanism is employed to protect coded video data against the error prone environments. Error concealment is a data recovery technique that enables the decoder to conceal effects of transmission errors by predicting the lost or corrupted video data from the previously reconstructed error-free information. Motion vector recovery and motion compensation with the estimated motion vector is a good approach to conceal the corrupted macroblock data. In this paper, we develop various error concealment algorithms based on motion vector recovery, and compare their performances to those of conventional error concealment methods.

압축상태에서 MPEG2 P 프레임을 H.263 P 프레임으로 변환하기 위한 가이드 탐색 방법 연구

본 논문은 이 기종간의 표준 압축형식을 변환하는데 있어 실시간 변환이 가능하도록 하며 프레임의 특성을 이용하여 압축률을 좀 더 높여 보는데 그 목적을 두고 있다. 이 기종간의 표준 압축형식 변환으로는 압축률이 낮은 MPEG2에서 압축률이 높은 H.263으로의 변환을 시도하였다. MPEG2에서 H.263으로 변환한 후 좀 더 압축률을 높이기 위하여 분석한 결과 MPEG2의 P 프레임내의 인트라 매크로블록 수가 H.263의 P 프레임의 인트라 매크로블록보다 많다는 것을 알 수 있었다. 인트라 매크로블록은 P 프레임을 생성하는 과정에서 움직임 예측 과정의 결과치가 임계치에 미치지 못했을 경우에 생성되는 것으로써 공간적인 압축 과정을 통하여 압축되기 때문에 P 프레임의 압축률에 큰 영향을 준다. 따라서 본 논문에서는 P 프레임의 압축률을 높이기 위하여 MPEG2 P 프레임에 생성된 인트라 매크로블록의 개수를 줄여 보았다. 이를 위한 해결방법으로써 P 프레임내의 인트라 매크로블록 변환 과정에 움직임 예측 과정을 삽입하여 변환하였다. 그리고 움직임 예측 과정의 복잡도를 줄이기 위하여 복원과정에서 알 수 있는 이웃하는 매크로블록의 정보와 움직임 벡터 정보를 이용하여 가이드 탐색 방법을 제안하였다. 실험 결과 MPEG2에서 H.263으로 변환이 실시간으로 이루어짐을 확인하였다. The purpose of the paper is to enable a format transcoding between a heterogeneous compression format in a real time, and to enhance the compression ratio using characteristics of the compressed frame. In this paper, for the heterogeneous format transcoding, we tried to transcode from MPEG2 having a lower compression ratio to H.263 having a higher compression ratio. After analyzing MPEG 2 bit stream and H.263 bit stream of the same original video, we found that the number of intra coded macro blocks in MPEG 2 data is much higher than the number of the intra coded macro blocks in H.263 data. In the process of P frame generation, a intra coded macro block is generated when a motion estimation value representing the similarity between the previous frame and current frame does not meet a threshold. Especially the intra coded macro block has a great impact on the compression ratio. Hence the paper, we tried to minimize the number of intra coded macro blocks in H.263 data stream which is transcoded from MPEG 2 in a compressed domain. For the purpose, we propose a guided search method for transcoding the INTRA coded block into INTER coded block using the information about motion vectors surrounding the intra macro block in order to minimize the complexity of the motion estimation process. The experimental results show that the transcoding of MPEG 2 into H.263 can be done in a real time successfully.

Drift compensation for reduced spatial resolution transcoding

This paper discusses the problem of reduced-resolution transcoding of compressed video bitstreams. An analysis of drift errors is provided to identify the sources of quality degradation when transcoding to a lower spatial resolution. Two types of drift error are considered: a reference picture error, which has been identified in previous works, and error due to the noncommutative property of motion compensation and down-sampling, which is unique to this work. To overcome these sources of error, four novel architectures are presented. One architecture attempts to compensate for the reference picture error in the reduced resolution, while another architecture attempts to do the same in the original resolution. We present a third architecture that attempts to eliminate the second type of drift error and a final architecture that relies on an intrablock refresh method to compensate for all types of errors. In all of these architectures, a variety of macroblock level conversions are required, such as motion vector mapping and texture down-sampling. These conversions are discussed in detail. Another important issue for the transcoder is rate control. This is especially important for the intra-refresh architecture since it must find a balance between number of intrablocks used to compensate for errors and the associated rate-distortion characteristics of the low-resolution signal. The complexity and quality of the architectures are compared. Based on the results, we find that the intra-refresh architecture offers the best tradeoff between quality and complexity and is also the most flexible.

CIF-to-QCIF video bitstream down-conversion in the DCT domain

Many advanced video applications require converting a compressed video bitstream from the Common Intermediate Format (CIF) to the Quarter Common Intermediate Format (QCIF). This task could be performed in the pixel domain or in the discrete cosine transform (DCT) domain. In this paper, we propose a DCT-domain approach, which requires lower complexity than that in the pixel domain. We also propose a scheme for selecting a coding mode and a motion vector for a macroblock (MB) in a QCIF video sequence from those of the corresponding CIF video without motion estimation (ME), which is computationally very expensive. Simulation results show that our DCT-domain approach reduces computational complexity by about 40% compared to the pixel-domain approach while providing comparable picture quality with about 0.4 dB degradation.

Error resilience and recovery in streaming of embedded video

The three-dimensional (3-D) SPIHT coder is a scalable or embedded coder that has proved its efficiency and its real-time capability in compression of video. A forward-error-correcting (FEC) channel (RCPC) code combined with a single automatic repeat request (ARQ) proved to be an effective means for protecting the bitstream. There were two problems with this scheme: the noiseless reverse channel ARQ may not be feasible in practice; and, in the absence of channel coding and ARQ, the decoded sequence was hopelessly corrupted even for relatively clean channels. In this paper, we introduce a new method of partitioning wavelet coefficients into spatio-temporal (s-t) tree blocks to achieve error resilience. Each of these s-t blocks corresponds to the full 3-D image region, because roots of these trees are wavelet coefficients taken at fixed intervals in the root low-frequency subband. Previously, we reported on grouping contiguous root subband coefficients to generate s-t tree blocks that correspond to local 3-D regions. The new procedure brings higher error resilience, since lost coefficients can be concealed with the surrounding coefficients even if some of the coded s-t blocks are totally missing. The bitstreams of the coded s-t blocks are packetized and encoded with a channel code to correct errors and to prevent decoding of erroneous data after errors are detected. Because the separately encoded s-t blocks produce embedded bitstreams, the packets from the bitstreams are interleaved to generate an embedded composite bitstream. The embedded property, whereby successive compressed bits convey successively smaller value information, suggests unequal error protection, where earlier bits are more strongly protected by the channel code than later bits. Therefore, unequal error protection is also incorporated into our video bitstreams to bring an even higher degree of resilience to channel bit errors. Our claims are supported by extensive simulations with decoding of the various 3-D SPIHT bitstreams compared to each other and to MPEG-2. Superiority to MPEG-2 in noiseless and noisy channels, under equal conditions with or without FEC, is clearly demonstrated by the results of these simulations.

New architecture for dynamic frame-skipping transcoder

Transcoding is a key technique for reducing the bit rate of a previously compressed video signal. A high transcoding ratio may result in an unacceptable picture quality when the full frame rate of the incoming video bitstream is used. Frame skipping is often used as an efficient scheme to allocate more bits to the representative frames, so that an acceptable quality for each frame can be maintained. However, the skipped frame must be decompressed completely, which might act as a reference frame to nonskipped frames for reconstruction. The newly quantized discrete cosine transform (DCT) coefficients of the prediction errors need to be re-computed for the nonskipped frame with reference to the previous nonskipped frame; this can create undesirable complexity as well as introduce re-encoding errors. In this paper, we propose new algorithms and a novel architecture for frame-rate reduction to improve picture quality and to reduce complexity. The proposed architecture is mainly performed on the DCT domain to achieve a transcoder with low complexity. With the direct addition of DCT coefficients and an error compensation feedback loop, re-encoding errors are reduced significantly. Furthermore, we propose a frame-rate control scheme which can dynamically adjust the number of skipped frames according to the incoming motion vectors and re-encoding errors due to transcoding such that the decoded sequence can have a smooth motion as well as better transcoded pictures. Experimental results show that, as compared to the conventional transcoder, the new architecture for frame-skipping transcoder is more robust, produces fewer requantization errors, and has reduced computational complexity.

A frame-based MPEG characteristics extraction tool and its application in video transcoding

We present a frame-based characteristics extraction tool, which can retrieve meaningful information from MPEG-compressed bitstreams. Taking advantage of the extracted information, we present a general architecture for MPEG video transcoding, which can transcode a given MPEG video bitstream into various MPEG video bitstreams with different characteristics. We also implement an MPEG-2-to-MPEG-4 video transcoder which can not only change the bitstreams' syntax and bitrate but can also encode each semantic video segment as a group of pictures (GOPs), by using the techniques of frame type conversion and video shot boundary detection. Moreover, the proposed transcoder can intelligently select a motion mode for each macroblock of B-VOPs in an MPEG-4 video to avoid some unpredictable artifacts due to mis-selection of the motion mode. This representative application demonstrates the potential usage of such frame-based characteristics.

Joint error control and power allocation for video transmission over CDMA networks with multiuser detection

Error control and power allocation for transmitting wireless video over CDMA networks are considered in conjunction with multiuser detection. We map a layered video bitstream to several CDMA fading channels and inject multiple source/parity layers into each of these channels at the transmitter. At the receiver, we employ a linear minimum mean-square error (MMSE) multiuser detector in the uplink and two types of blind linear MMSE detectors, i.e., the direct-matrix-inversion blind detector and the subspace blind detector, in the downlink, for demodulating the received data. For given constraints on the available bandwidth and transmit power, the transmitter determines the optimal power allocation among different CDMA fading channels and the optimal number of source and parity packets to send that offer the best video quality. We formulate a combined optimization problem and give the optimal joint rate and power allocation for each of these three receivers. Simulation results show a performance gain of up to 3.5 dB with joint optimization over with rate optimization only.

Local bandwidth constrained fast inverse motion compensation for DCT-domain video transcoding

Discrete cosine transform (DCT) based digital video coding standards, such as MPEG and H.26x, are becoming more widely adopted for multimedia applications. Since the standards differ in their format and syntax, video transcoding, where a compressed video bit-stream is converted from one format to another, is of interest for purposes such as channel bandwidth adaptation and video composition. DCT-domain video transcoding is generally more efficient than spatial-domain transcoding. However, since the data is organized block by block in the DCT domain, inverse motion compensation becomes the bottleneck for DCT-domain methods. We propose a novel local bandwidth constrained fast inverse motion compensation algorithm operating in the DCT domain. Relative to Chang's algorithm, we achieve computational improvement of 25%-55% without visual degradation. A by-product of our algorithm is a reduction of blocking artifacts in very low bit-rate compressed video sequences. The proposed algorithm can be combined with other reported fast methods for more computational savings. We also present a look-up-table (LUT) based implementation method by modeling the statistical distribution of the DCT coefficients in natural images and video sequences. By this method, we obtain a further 31%-48% improvement in computation. The memory requirement of the LUT is about 800 kB, which is reasonable. Moreover, the LUT can be shared by multiple DCT-domain video processing applications running on the same computer or video server.

A full-featured, error-resilient, scalable wavelet video codec based on the set partitioning in hierarchical trees (SPIHT) algorithm

Compressed video bitstreams require protection from channel errors in a wireless channel. The 3-D set partitioning in hierarchical trees (SPIHT) coder has proved its efficiency and its real-time capability in the compression of video. A forward-error-correcting (FEC) channel (RCPC) code combined with a single automatic-repeat request (ARQ) proved to be an effective means for protecting the bitstream. There were two problems with this scheme: (1) the noiseless reverse channel ARQ may not be feasible in practice and (2) in the absence of channel coding and ARQ, the decoded sequence was hopelessly corrupted even for relatively clean channels. We eliminate the need for ARQ by making the 3-D SPIHT bitstream more robust and resistant to channel errors. We first break the wavelet transform into a number of spatio-temporal tree blocks which can be encoded and decoded independently by the 3-D SPIHT algorithm. This procedure brings the added benefit of parallelization of the compression and decompression algorithms, and enables implementation of region-based coding. We demonstrate the packetization of the bitstream and the reorganization of these packets to achieve scalability in bit rate and/or resolution in addition to robustness. Then we encode each packet with a channel code. Not only does this protect the integrity of the packets in most cases, but it also allows detection of packet-decoding failures, so that only the cleanly recovered packets are reconstructed. In extensive comparative tests, the reconstructed video is shown to be superior to that of MPEG-2, with the margin of superiority growing substantially as the channel becomes noisier. Furthermore, the parallelization makes possible real-time implementation in hardware and software.

Pilot-assisted 16-level QAM for wireless video

This paper presents a twin-class transmission system for narrowband radio access channels suitable for handheld video phone and multimedia portable PC applications. The transmission system is comprised of a hierarchical 16-QAM modulation technique and a channel-coding scheme. The formation of dual-priority transmission is due to differing error resiliencies of the bits that make up a given symbol in a Gray-coded 16-QAM. On this basis, a twin-class pilot-assisted fade-estimation technique that can gracefully reduce the power loss caused by the transmission of pilot overhead is developed. The twin-class 16-QAM system is then used to transport a compressed video bitstream, which is partitioned to match the bit-error sensitivity of the transmitted symbol. The partitioning scheme is based on a separation of the variable-length (VL) coded discrete cosine transform (DCT) coefficients within each DCT block. This partitioning scheme is then applied to split the ITU-T H.263-coded bitstream. The scheme is suitable for constant bit-rate transmission (CBR), where the fraction of bits assigned to each of the two partitions can be adjusted according to the requirements of the unequal error protection scheme employed. The distribution of the VL-coded (VLC) information amongst the two partitions is performed adaptively. Finally, the performance of the partitioning scheme for transmission of video signals using our twin-class 16-QAM transmission system is evaluated under multipath fading conditions.

Embedded video coding using invertible motion compensated 3-D subband/wavelet filter bank

Abstract In this paper, we present a new embedded video coding system with local motion compensation. An invertible motion compensated 3-D subband filter bank is utilized for video analysis/synthesis. The efficient embedded image coding scheme EZBC is extended to 3-D coding of video subbands. The coded bitstream is rate scalable and fully embedded thus meeting the increasing need for fine granular scalability in video streaming applications. Our experimental results show the new algorithm outperforms nonscalable standard MPEG-2 by 0.8– 3.3 dB (luminance component) over a broad range of bitrates, while providing an embedded video bitstream. Unlike conventional SNR hybrid coders, this multirate property is achieved without a loss in compression efficiency. Comparisons are also made to two other competing embedded video coders, LZC and 3D-SPIHT, both based on 3-D subband coding as well. The new coder demonstrates a significant advantage for encoding video with high local motion content, thanks to the efficient inclusion of local motion information in the temporal filter bank of our proposed system.

Forward error correction (FEC) codes based multiple description coding for internet video streaming and multicast

In this work, we describe the application of generalized multiple description (MD) source coding architected through the usage of forward error correction (FEC) codes to achieve robust and efficient Internet video streaming and multicast consistent with the existing Internet architecture. We highlight the synergistic interaction between the robust source coding and packetization strategy and an efficient as well as responsive, TCP-friendly congestion control protocol linear increase multiplicative decrease with history (LIMD/H) to achieve robust streaming over the Internet. The proposed source packetization scheme transforms a loss sensitive layered video bitstream into a robust packet stream so as to provide graceful resilience to network packet drops. The proposed congestion control mechanism provides low variation in transmission rate in steady state and at the same time is reactive and provably TCP friendly. Furthermore, exploiting the fundamental property of a multicast tree, namely, the observation that the multicast tree is a degraded channel, we propose an elegant application level architecture for efficient video multicast. We exploit the amenability of the MD coding strategy to fast transcoding. Our extensive simulation studies for the streaming problem and preliminary results for the multicast case, bear out the efficacy of the proposed setup.

On-line Scene Change Detection of Multicast Video

Network-based computing is becoming an increasingly important area of research, whereby computational elements within a distributed infrastructure process/enhance the data that traverses through its path. We refer to these computations as online processing and this paper investigates scene change detection in the context of MBone-based proxies in the network. On-line processing varies from traditional offline processing schemes, where for example, the whole video scope is known as a priori, which allows multiple scans of the stored video files during video processing. The on-line processing system is designed to meet the requirements of real-time video multicasting over the Internet and to utilize the successful video parsing techniques available today. The proposed algorithms do scene change detection and extract key frames from video bitstreams sent through the MBone network. We study several algorithms based on histogram differences and evaluate them with respect to precision, recall, and processing latency.

On packetization of embedded multimedia bitstreams

We study the problem of packetizing embedded multimedia bitstreams to improve the error resilience of source (compression) codes. This problem is important because of the increasing popularity of embedded compression methodology and its suitability for scalable streaming media over IP or/and mobile IP. We study various packetization schemes against packet erasure at both low and high bit rates. Maximizing packetization efficiency for embedded bitstreams is formulated as a discrete optimization problem and globally optimal packetization (OP) algorithms are proposed under different settings. Suboptimal packetization algorithms are also devised to reduce the complexity of the OP algorithms. In order to assess their effectiveness, the proposed packetization algorithms are used to packetize embedded image and video bitstreams with simulated packet loss. Experimental results show that our OP algorithms slightly outperforms suboptimal ones. In addition to confirming the superiority of the OP algorithms, these results also provide justification of heuristic packetization methods published in the literature.

An integrated source transcoding and congestion control paradigm for video streaming in the Internet

In this work we present an end-to-end optimized video streaming system comprising of synergistic interaction between a source packetization strategy and an efficient and responsive, TCP-friendly congestion control protocol [Linear Increase Multiplicative Decrease with History (LIMD/H)]. The proposed source packetization scheme transforms a scalable/layered video bitstream so as to provide graceful resilience to network packet drops. The congestion control mechanism provides low variation in transmission rate in steady state and at the same time is reactive and provably TCP-friendly. While the two constituent algorithms identified above are novel in their own right, a key aspect of this work is the integration of these algorithms in a simple yet effective framework. This application-transport layer interaction approach is used to maximize the expected delivered video quality at the receiver. The integrated framework allows our system to gracefully tolerate and quickly react to sudden changes in the available connection capacity due to the onset of congestion, as verified in our simulations.

An efficient error concealment implementation for MPEG-4 video streams

This paper presents an efficient error concealment implementation for damaged MPEG-4 video bitstreams. The chosen spatial and temporal concealment algorithms are designed to fit in real-time decoders and are advantageously combined in a hybrid spatial/temporal approach to provide visually more plausible pictures than basic concealment techniques. In addition, the encoder impact on the visual quality of the reconstruction in presence of channel errors is highlighted.

Encoding stored video for streaming applications

In streaming video applications, video sequences are encoded off-line and stored in a server. Users may access the server over a constant bit rate channel. Examples of the streaming video applications are video-on-demand, archived video news, and noninteractive distance learning. Before the playback, part of the video bitstream is pre-loaded in the decoder buffer to ensure that every frame can be decoded at the scheduled time. For these streaming video applications, since the video is encoded off-line and the future video frames are available to the encoder, a more sophisticated bit-allocation scheme can be used to achieve better video quality. During the encoding process for streaming video, two requirements need to be considered: the pre-loading time that the video viewers have to wait and the physical buffer-size at the receiver (decoder) side. In this paper, we propose a sliding-window rate-control scheme that uses statistical information of the future video frames as a guidance to generate better video quality for video streaming involving constant bit rate channels. A quantized discrete cosine transform coefficient selection scheme based on the rate-distortion measurement is also used to improve the video quality. Simulation results show video quality improvements over the regular H.263 TMN8 encoder.

Motion vector composition algorithm for spatial scalability in compressed video

Spatial scalable encoding of visual data has several applications, including browsing visual databases, querying multimedia databases, interactive multimedia communications, etc. To convert a compressed video sequence to a lower spatial resolution compressed video, it may decode the original bit-stream, down-sample each frame, and encode. The computation-intensive motion estimation will be included in the transcoder. The re-use of motion vectors extracted from incoming video bit-streams during transcoding has been widely accepted. This paper proposes a new algorithm to estimate the motion vector of the reduced low-resolution video by only using the original motion vectors and DC coefficients in the compressed bit-stream. By taking into account the spatial activity measurement to generate better prediction for the composed motion vector, higher quality and lower complexity than previous works are achieved by the proposed method.

DSP-based transcoding of digital video signals with MPEG-2 format

MPEG-2 video coding is widely used in broadcasting and increasingly in studio applications. For reasons of limited storage and/or transmission capacity, it may be necessary to reduce the bit rate of MPEG-2 video bitstreams. We present an implementation of a frequency domain transcoder (FDTC) on a media processor which reduces the bit rate of an incoming video bitstream in real time. The FDTC has a low complexity and low memory requirements. A rate control with low delay provides a bitstream with constant bit rate (CBR) at the output, independent whether the input bit rate is variable (VBR) or constant (CBR). The FDTC achieves a better or same peak signal-to-noise ratio (PSNR) than a bulky cascade of a complete MPEG-2 video decoder and encoder.