Video Bitstream Research Articles

Error concealment in video coding of arbitrarily shaped objects

Recent trends in video coding technology have included the coding of arbitrarily shaped objects and the improvement of the robustness of video bitstreams when subjected to errors during transmission. In this paper, we examine the interaction of these two technologies, using a coder and decoder based on MPEG 4-video. Effective techniques are proposed for concealing the effects of these transmission errors.

Robust and scalable matching pursuits video transmission using the Bluetooth air interface standard

This paper introduces an error resilient implementation of the matching pursuits algorithm for video coding. The video bitstream is transmitted using a simulation of the Bluetooth air interface standard, which recommends ARQ as a means of overcoming channel errors in the data packets. This approach may be unsuitable for real time and broadcast applications. Therefore, a modified receiver is proposed in this paper, which does not request the retransmission of erroneous packets, but instead passes them to the video decoder to exploit error resilience. This strategy is shown to be superior to a standard compliant system if ARQ cannot be applied. The work confirms that wireless communication standards should support a transparent mode for video applications.

On end-to-end architecture for transporting MPEG-4 video over the Internet

With the success of the Internet and flexibility of MPEG-4, transporting MPEG-4 video over the Internet is expected to be an important component of many multimedia applications in the near future. Video applications typically have delay and loss requirements, which cannot be adequately supported by the current Internet. Thus, it is a challenging problem to design an efficient MPEG-4 video delivery system that can maximize the perceptual quality while achieving high resource utilization. This paper addresses this problem by presenting an end-to-end architecture for transporting MPEG-4 video over the Internet. We present a framework for transporting MPEG-4 video, which includes source rate adaptation, packetization, feedback control, and error control. The main contributions of this paper are: (1) a feedback control algorithm based on the Real Time Protocol (RTP) and the Real Time Control Protocol (RTCP); (2) an adaptive source-encoding algorithm for MPEG-4 video which is able to adjust the output rate of MPEG-4 video to the desired rate; and (3) an efficient and robust packetization algorithm for MPEG video bit-streams at the sync layer for Internet transport. Simulation results show that our end-to-end transport architecture achieves good perceptual picture quality for MPEG-4 video under low bit-rate and varying network conditions and efficiently utilizes network resources.

Minimum-drift digital video downconversion

This paper presents a new technique for decoding a full-resolution video bitstream at low memory cost and displaying the signal at a lower resolution. Existing techniques solve the problem by storing the downconverted blocks into memory instead of the full-resolution blocks. While the memory is reduced, these techniques introduce drift errors because the decoder does not have the same pixels as the encoder in performing motion-compensated prediction. The approach proposed here alleviates the problem by tracking the drift at the decoder. It improves the video quality without any increase in decoder complexity. The effectiveness of the approach is evaluated using both objective and subjective tests. This minimum-drift approach is very simple to implement and can also be applied for memory reduction of a full-resolution HDTV decoder.

Backward compatible user-defined data insertion into MPEG-4 bitstreams

It is often useful to provide information related to a video bitstream that is unsupported by the official standard. A method is described by which data can be inserted in an MPEG-4 bitstream , while still retaining compatibility with standard MPEG-4 decoders. Its effectiveness is demonstrated by results from insertion of cyclic redundancy check (CRC) codes.

Wireless MPEG-4 video communication on DSP chips

We discuss the design and implementation of wireless video communication systems on DSPs. We cover both video coding and the channel coding aspects of the problem. The emphasis of the article is on highlighting the issues involved both from an algorithmic standpoint as well as from a DSP standpoint. We discuss the first DSP implementation, to the authors' knowledge, of the MPEG-4 simple profile video standard. We give an overview of DSPs and highlight the salient features that make them especially well-suited for wireless applications. We describe MPEG-4 simple profile video coding noting that in order to facilitate interoperability, it is important that wireless devices use standardized compression algorithms. We discuss our implementation of the MPEG-4 simple profile video codec on Texas Instruments' TMS320C54x DSP and discuss the issues involved. We highlight some general and application-specific DSP instructions on the TMS320C54x DSP that enable us to implement the core operations in the video coder efficiently, often in a single cycle, with minimal control overhead. We describe the performance of MPEG-4 video compression for a variety of content and formats. We give a description of channel coding with the H.223 standard. Then, we describe several channel coding experiments using both unequal and equal protection on MPEG-4 video sent through a simulated GSM channel. Unequal error protection, which ensures fewer errors in the important portions of the MPEG-4 video bitstream, provides improved quality when compared to equal error protection under harsh error conditions.

Motion adaptive intra refresh for MPEG-4

The sensitivity to error of a video bitstream often changes as the amount of motion within a scene changes. Taking this into account, a scheme that attempts to optimise error robustness by varying the amount of adaptive intra refresh blocks is presented. Simulation results demonstrate the benefits of the proposed technique.

MPEG-2 Coded Video Traces Transmitted Over aSatellite Link: Scalable and Non-Scalable Solutionsin Rain Fading Conditions

The literature is poor in the analyses of the effects produced by corrupted bits in compressed video bitstreams. This paper presents the results of a transmission experiment of MPEG-2 coded video data over a satellite link affected by noise, in order to investigate under which conditions this type of transmission is economically feasible. The signal-to-noise ratio scalability feature of the MPEG-2 encoder was used to produce different bitstreams of the same movie sequence. The scope of the study was to verify which are the best combinations of video and channel codings in the presence of attenuation on the satellite link, in order to optimize the bandwidth utilisation for a requested image quality. The results obtained give indications about the data channel codings to be used to counter the rain fade on the transmission link, which is a non negligible problem especially when satellite transmissions are in the Ka band. Moreover, the results highlight the flexibility of the scalable video coding in the examined scenario.

A method for removing blocking effects in MPEG-2 video by applying a block classification technique using stream information

This paper proposes a method for removing the blocking effects that become evident when decoding MPEG-2 video data. The method classifies each block and detects blocking effects by using the information obtained from the MPEG-2 video bitstream, including DCT coefficients and motion vectors. Blocking effect removal is achieved by applying a filter to the decoded images in response to the detected result. The effectiveness of the method has been verified by computer simulation. Based on subjective judgment results, the image quality encoded at 3 Mbps by using this method was confirmed to be closely comparable with the quality of images encoded at 4 Mbps. Because this method carries out noise elimination by only using the information obtained from the MPEG-2 video bitstream, it can be applied to any application, including storage, broadcasting, communications, etc.

A hybrid approach with drift minimization and correction for a layered H.263 codec

Layered video coding allows the video information to be transmitted in multiple video bitstreams to achieve scalability, and the quality of the base layer is limited by drift. We propose a hybrid approach to eliminate the drift in an H.263-based video codec with two-layer scalability. The base layer decoder takes advantage of the minimal drift approach to obtain the reconstructed services that have the minimum drift. Then, the encoder sends an additional drift correction signal, which is used to eliminate drift completely, to the decoder. In this way, the proposed approach can eliminate the drift completely and reduce the overhead required for the drift correction.

Efficient MPEG compressed video analysis using macroblock type information

Efficient indexing methods are required to handle the rapidly increasing amount of visual information within video databases. Video analysis that partitions the video into clips or extracts interesting frames is an important preprocessing step for video indexing. We develop a novel method for video analysis using the macroblock (MB) type information of MPEG compressed video bitstreams. This method exploits the comparison operations performed in the motion estimation procedure, which results in specific characteristics of the MB type information when scene changes occur or some special effects are applied. Only a simple analysis on MB types of frames is needed to achieve very fast scene change, gradual transition, flashlight, and caption detection. The advantages of this novel approach are its direct extraction from the MPEG bitstreams after VLC decoding, very low complexity analysis, frame-based detection accuracy and high sensitivity.

Adaptive motion-vector resampling for compressed video downscaling

Digital video is becoming widely available in compressed form, such as a motion JPEG or MPEG coded bitstream. In applications such as video browsing or picture-in-picture, or in transcoding for a lower bit rate, there is a need to downscale the video prior to its transmission. In such instances, the conventional approach to generating a downscaled video bitstream at the video server would be to first decompress the video, perform the downscaling operation in the pixel domain, and then recompress it as, say, an MPEG, bitstream for efficient delivery. This process is computationally expensive due to the motion-estimation process needed during the recompression phase. We propose an alternative compressed domain-based approach that computes motion vectors for the downscaled (N/2xN/2) video sequence directly from the original motion vectors for the N/spl times/N video sequence. We further discover that the scheme produces better results by weighting the original motion vectors adaptively. The proposed approach can lead to significant computational savings compared to the conventional spatial (pixel) domain approach. The proposed approach is useful for video severs that provide quality of service in real time for heterogeneous clients.

Design and implementation of a Java-based MPEG-1 video decoder

Although Java has many useful programming language features for developing consumer multimedia applications, it is not widely used for multimedia application development. One of the main reasons for the lack of Java usage in consumer multimedia application development is a concern for the execution speed of Java-based applications. In this paper, we investigate the feasibility of using Java as a language for multimedia applications. As a specific multimedia application example, we have developed a Java-based MPEG-1 video decoder. We describe the design and implementation of the MPEG-1 video decoder and report our experience in optimizing the decoding performance. Based on the performance analysis results from a Java performance profiler, we have applied both general and Java-specific optimization techniques. The final implementation could decode about 28.67 frames per second on a Pentium-II 300 MHz computer for a 240/spl times/170 MPEG-1 video bitstream, a speed-up of 2.8 times over the initial implementation. Our experience strongly suggests that the pure Java-based media processing is a feasible solution.

Real-Time Labeling of MPEG-2 Compressed Video

Digital mass recording devices for video data will enter the consumer market soon. Service providers are reluctant to offer services in digital form because of their fears for unrestricted duplication and dissemination. Therefore adequate copy protection systems should be developed, most likely based on labeling techniques. In this paper we propose two different techniques for the real-time labeling of digital video. Both methods embed the label information directly into an MPEG compressed video bitstream. The first method embeds the label by changing variable length codes in the bitstream. The second method discards some of the high frequency DCT-coefficients of the bitstream to embed the label. The first technique is computationally less expensive than the second one, however, the second one is more robust against attacks to remove the label. We show that the label can still be extracted after MPEG re-encoding at a lower bit-rate.

A study of the burstiness of combined MPEG video and audio bitstreams

Variable bit rate (VBR) video is very bursty and its transportation over ATM networks is problematic because bandwidth has to be reserved in advance. Studies have shown that VBR video exhibits statistical self-similarity: it displays structural similarities across a wide range of time scales. To date, studies on the burstiness of MPEG compressed bitstreams have been conducted on the video part only; the audio part has been neglected because of its constant bit rate nature. Furthermore, the change in burstiness of MPEG data which occurs after they have been converted to MPEG-2 transport and ATM bitstreams has not been investigated; a considerable amount of overhead bytes is introduced during the conversion. This paper reports on a study on the burstiness of combined MPEG-1 audio and video bitstreams as found on commercial Video CDs, and the burstiness of these combined bitstreams after they have been converted to MPEG-2 transport and ATM layer bitstreams. We found that there is no increase in burstiness in both situations.

Dynamic bit allocation in video combining for multipoint conferencing

For multipoint video conferencing over a wide-area network, a video bridge is often used to combine the coded video signals from multiple participants into a single video for display. The video combining can be done in the coded-domain by concatenating coded video bitstreams, or in the pixel-domain by transcoding. In this paper we compare the video quality using the two approaches for a multipoint video conference over a symmetrical wide-area network such as Integrated Service Digital Network (ISDN). We observe that in most multipoint video conferences usually only one or two persons are active at one time, while other participants are just listening with little motion. To achieve similar video quality for all the conference participants, the active persons require higher bit rates than the inactive persons. By doing video combining using the transcoding approach and dynamically allocating more bits to the active persons, we show that the achieved video quality can be much better than that can be achieved using the coded-domain video combining approach.

High-quality bit-rate transformation for the Home Media Station

This paper discusses methods for bit-rate transformation of previously encoded compressed video bitstreams. The proposed high-quality, low complexity bit-rate transformation algorithms are especially suitable for consumer devices such as the Home Media Station-a sophisticated multimedia receiver/decoder/recorder for consumer use.

Motion estimation for high performance transcoding

Traditionally, the re-use of motion vectors extracted from incoming video bit-stream during transcoding has been widely accepted. However, this simple re-use scheme introduces significant quality degradation in many applications including the situation when the frame-rate conversion is needed. In this paper, we analyzed the quantization errors that cause the extracted motion vectors to be non-optimal and we performed simulations to show the quality degradation due to the inaccurate motion vectors during transcoding. To improve the video quality, we proposed an adaptive motion vector refinement. With a highly reduced computational complexity, the proposed adaptive motion vector refinement achieves significant quality improvement in comparison to the conventional motion vector re-use scheme. In addition, the adaptive motion vector refinement is almost as good as performing a new full-scale motion estimation.

Implementing real-time video decoding on multimedia processors by complexity prediction techniques

The implementation of software based video/audio decoders is an advantageous solution over traditional dedicated hardware systems. However, simultaneously guaranteeing real-time performance, needed for processing video/audio bit-streams, and an efficient use of the processing power, remains a very difficult task. In this paper, we briefly discuss the main reasons for such difficulties and we present a new technique able to predict the decoding time of compressed video bit-streams without the need of the actual decoding. This result can be obtained when appropriate statistics about the actual decoding process are included and transmitted with the compressed video data. We also show how such results can be used to implement new efficient processing resource allocation strategies. New possible schemes of intelligent interactions, and in particular an efficient implementation of computational graceful degradation, between the encoder and the real-time OS are also proposed. An example of results obtained implementing the proposed technique for the MPEG-4 VM6 video algorithm is presented. Although the techniques and concepts presented in this paper are of general application, they can only be used for video compression standards capable of transmitting in any appropriate form the associated statistical coding information. This capability has been included in the new defined MPEG-4 standard.

The influence of residual errors on a digital satellite TV decoder

Abstract In spite of the use of convolutional and Reed-Solomon codes in satellite broadcasting to protect the MPEG-2 bitstream, residual errors will be found at the input of audio and video decoders. Concealment methods must be elaborated to preserve image quality. In this paper, we simulate the complete transmission system in order to design residual error models. Then, using these models, errors are injected in audio and video bitstreams and their influence on quality is determined with subjective tests. We show that the audio decoding process is much more robust than the video one. So we focus our attention on video and elaborate two concealment strategies. In the first one, the decoder uses the information provided by the system layer to conceal false packets. In the second, a smart decoder is designed which uses bitstream properties to detect and conceal residual errors. We note that the smart decoder gives better image quality than that of the first decoder. Finally, the benefits of concealment strategy are evaluated in terms of C N .