Lossy Packet Networks Research Articles

Toward bandwidth-efficient and error-robust audio streaming over lossy packet networks

Bandwidth efficiency and error robustness are essential issues for different multimedia streaming applications. This paper presents strategies for high-quality audio streaming based on fragmenting perceptually coded audio frames and shuffling the data components among multiple packets for transportation. This is done to increase robustness against packet loss. We also address the delivery of audio data consisting of components with different proportional priorities. Our approach is rationalized with streaming tests using the MPEG AAC audio codec in a simulated network environment and formal listening tests to evaluate the resulting audio output. According to the results, the proposed schemes improve audio quality significantly with reasonable increase to network resource utilization compared to traditional error robustness measures.

Fast Code-Rate Optimization for Robust Image Transmission Over Lossy Packet Networks

In this paper, we propose an efficient method for the allocation of Reed-Solomon codes to source symbols, for unequal loss protection. The proposed formulation recasts the multivariate optimization problem into a univariate one, dramatically reducing the computational complexity. Results are shown for image transmission over lossy packet networks, employing the JPEG2000 and SPIHT encoders. The proposed algorithm exhibits performance equivalent to previous methods, while providing a significant complexity reduction.

Unequal loss protection for H.263 compressed video

We study the application of unequal loss protection (ULP) algorithms to motion-compensated video over lossy packet networks. In particular, we focus on streaming video applications over the Internet. The original ULP framework applies unequal amounts of forward error correction to embedded data to provide graceful degradation of quality in the presence of increasing packet loss. In this letter, we apply the ULP framework to baseline H.263, a video compression standard that targets low bit rates, by investigating reorderings of the bitstream to make it embedded. The reordering process allows a receiver to display quality video, even at the loss rates encountered in wireless transmissions and the current Internet.

Video Compression for Lossy Packet Networks With Mode Switching and a Dual-Frame Buffer

Video codecs that use motion compensation benefit greatly from the development of algorithms for near-optimal intra/inter mode switching within a rate-distortion framework. A separate development has involved the use of multiple-frame prediction, in which more than one past reference frame is available for motion estimation. In this paper, we show that using a dual-frame buffer (one short-term frame and one long-term frame available for prediction) together with intra/inter mode switching improves the compression performance of the coder. We improve the mode-switching algorithm with the use of half-pel motion vectors. In addition, we investigate the effect of feedback in making more informed and effective mode-switching decisions. Feedback information is used to limit drift errors due to packet losses by synchronizing the long-term frame buffers of both the encoder and the decoder.

Ensuring quality of service for image transmission: hybrid loss protection

We present hybrid loss protection as a new channel coding and packetization scheme for image transmission over nonprioritized lossy packet networks. The scheme employs an interleaver-based structure, and attempts to maximize the expected peak signal-to-noise ratio (PSNR) at the receiver given the constraint that the probability of failure, i.e., the probability that the PSNR of the decoded image is below a given threshold, is upper-bounded by a user-defined value. A new code-allocation algorithm is proposed, which employs Gilbert-Elliot modeling of the network statistics. Experimental results are provided in the case of transmission of images encoded by SPIHT and JPEG 2000 over a wireline, as well as a wireless UMTS-based Internet connection.

Quality measurements for compressed video transmitted over a lossy packet network

We evaluate the degradation in a video sequence that suffers from increasing packet loss ratio (PLR), by using several kinds of quality measures, namely, mean square error (MSE); A max (the root MSE, which is weighted by the maximum signal change between pairs of pixels—the current pixel and eight surrounding neighbors); the video perceptual distortion measure (VPDM); and a perceptual distortion metric for digital color video (VPDM2). Packet losses can occur at different positions in the compressed video sequence, for example, in headers, I frames, P frames, and B frames. Each packet loss results in a different amount of distortion in the received video signal. We demonstrate the differences between two packet loss models—IID (identical independent distribution) and burst (a sequence of packets lost together). We also examine the influence of packet loss in a communication network on MPEG2 compressed digital video quality. Three of these measures emulate features of the human visual system: the VPDM, which considers temporal masking of distortion (the human eye notices less degradation within a frame, if it appears during fast changes between consequent frames, such as scene cuts); the A max , which encompasses spatial masking, resulting in small objects within high-frequency neighborhoods being less noticeable; and the VPDM2, which takes into account the human vision color channels and temporal and spatial masking. The results point out that loss due to burst degrades the quality of the compressed video stream significantly less than loss due to IID. The video quality can still be tolerable for a 0.1 PLR using the burst loss model, but for the IID loss model, even for lower PLRs (i.e., 0.01), the video is not perceptible to the human viewer.

Streaming video and rate scalable compression: what are the challenges for watermarking?

Video streaming, or the real-time delivery of video over a data network, is the underlying technology behind many applications including video conferencing, video on demand, and the delivery of educational and entertainment content. In many applications, particularly ones involving entertainment content, security issues, such as conditional access and copy protection, must be addressed. To resolve these security issues, techniques that include encryption and watermarking need to be developed. Since video sequences will often be compressed using a scalable compression technique and transported over a lossy packet network using the Internet Protocol (IP), security techniques must be compatible with the compression method and data transport and be robust to errors. We address the issues involved in the watermarking of rate-scalable video streams delivered using a practical network. Watermarking is the embedding of a signal (the watermark) into a video stream that is imperceptible when the stream is viewed, but can be detected by a watermark detector. Many watermarking techniques have been proposed for digital images and video, but the issues of streaming have not been fully investigated. A review of streaming video is presented, including scalable video compression and network transport, followed by a brief review of video watermarking and the discussion of watermarking streaming video.

Spatiotemporal error concealment with optimized mode selection and application to H.264

In this paper we propose an error concealment algorithm for video transmission over lossy packet networks. The proposed technique is based on temporal and spatial interpolation. A sophisticated mode selection algorithm decides whether to employ the temporal or the spatial part, or a combination thereof, to estimate a missing macroblock; the selection does not rely on knowledge of the original coding modes. The resulting error concealment algorithm is designed so as to optimize both PSNR and visual quality of the restored video sequence, and employs directional interpolation and texture analysis/synthesis. The technique has been applied to H.264 coded video, providing satisfactory results on a number of test sequences.

Error-resilient video coding using multiple description motion compensation

A new approach for multiple description video coding is proposed. It employs a second-order predictor for motion-compensation, which predicts a current frame from two previously coded frames. The coder generates two descriptions, containing the coded even and odd frames, respectively. When only a single description (say, that containing even frames) is received, the decoder can only use previous even frames for prediction. The mismatch between the predicted frames at the encoder and decoder is explicitly coded to avoid error propagation in the ideal MD channels, where a description is either received intact or lost completely. By using the second-order predictor and coding the mismatch signal, one can also suppress error propagation in packet lossy networks where packets in either description can be lost. The predictor and the mismatch signal quantizer can be varied to achieve a wide range of tradeoffs between coding efficiency and error resilience.

Application Layer Error-Correction Coding for Rate-Distortion Optimized Streaming to Wireless Clients

This paper addresses the problem of streaming packetized media over a lossy packet network to a wireless client, in a rate-distortion optimized way. We introduce an incremental redundancy error-correction scheme that combats the effects of both packet loss and bit errors in an end-to-end fashion, without support from the underlying network or from an intermediate base station. The scheme is employed within an optimization framework that enables the sender to compute which packets it should send, out of all the packets it could send at a given transmission opportunity, in order to meet an average transmission-rate constraint while minimizing the average end-to-end distortion. Experimental results show that our system is robust and maintains quality of service over a wide range of channel conditions. Up to 8 dB performance gains are registered over systems that are not rate-distortion optimized, at bit-error rates as large as 10/sup -2/.

Modulation transfer function as a quality measure for compressed images transmitted over a lossy packet network

We present a study of the grating detuning effect on the transmission and reflection holographic memories recorded in a photopolymer material. By using the Bragg matching condition, we analyze the angular shift and the degradation of the diffraction efficiency of the reconstructed images. Based on these results, a method for precompensating the detuning effect has been proposed.

AMISP: a complete content-based MPEG-2 error-resilient scheme

We address a new error-resilient scheme for broadcast quality MPEG-2 video streams to be transmitted over lossy packet networks. A new scene-complexity adaptive mechanism, namely Adaptive MPEG-2 Information Structuring (AMIS) is introduced. AMIS modulates the number of resynchronization points (i.e., slice headers and intra-coded macroblocks) in order to maximize the perceived video quality, assuming that the encoder is aware of the underlying packetization scheme, the packet loss probability (PLR), and the error-concealment technique implemented at the decoding side. The end-to-end video quality depends both on the encoding quality and the degradation due to data loss. Therefore, AMIS constantly determines the best compromise between the rate allocated to encode pure video information and the rate aiming at reducing the sensitivity to packet loss. Experimental results show that AMIS dramatically outperforms existing structuring techniques, thanks to its efficient adaptivity. We then extend AMIS with a forward-error-correction (FEC)-based protection algorithm to become AMISP. AMISP triggers the insertion of FEC packets in the MPEG-2 video packet stream. Finally, the performances of the AMISP scheme in an MPEG-2 over RTP/UDP/IP scenario are evaluated.

Multiple description perceptual audio coding with correlating transforms

In audio communication over a lossy packet network, concealment techniques are used to mitigate the effects of lost packets. This concealment is markedly improved if the compressed representation retains redundancy to aid in the estimation of lost information. A perceptual audio coder employing multiple description correlating transforms demonstrates this phenomenon.

User-Oriented QoS Analysis in MPEG-2 Video Delivery

We address the problem of video quality prediction and control for high-resolution video transmitted over lossy packet networks. In packet video, the bitstream flows through several subsystems (coder, network, decoder); each of them can impair the information, either by data loss or by introducing some delay. However, each of these subsystems can be fine-tuned in order to minimize these problems and to optimize the quality of the delivered signal, taking into account the available bitrate. The assessment of the end-user quality is a non-trivial issue. We analyse how the user-perceived quality is related to the average encoding bitrate for variable bit rate MPEG-2 video. We then show why simple distortion metrics may lead to inconsistent interpretations. Furthermore, for a given coder setup, we analyse the effect of packet loss on the user-level quality. We then demonstrate that, when jointly studying the impact of coding bit rate and packet loss, the reachable quality is upperbound and exhibits one optimal coding rate for a given packet loss ratio.

Optimal intra coding of blocks for robust video communication over the Internet

Abstract Reliable transmission of compressed video in a packet lossy environment cannot be achieved without error recovery mechanisms. We describe an effective method for increasing error resilience of video transmission over packet lossy networks such as the Internet. Intra coding (without reference to a previous picture) is a well-known technique for eliminating temporal error propagation in a predictive video coding system. Randomly intra coding of blocks increases error resilience to packet loss. However, when the error concealment used by the decoder is known, intra encoding following a method that optimizes the tradeoffs between compression efficiency and error resilience is a better alternative. In this paper, we present a rate-distortion optimized mode selection method for packet lossy environments that takes into account the network conditions and the error concealment method used at the decoder. We present results for different packet loss rates and typical packet sizes of the Internet, that illustrate the advantages of the proposed method.

Application of H.263+ Video Coding Modes in Lossy Packet Network Environments

The quality of real-time audio and video information transmitted via today's Internet suffers severely from often significant packet losses. While this problem is well understood and solved for existing audio coding schemes, support from the video coding standards themselves is required for video streams. This paper presents the newly introduced error resilience mechanisms built into the second version of H.263 (1998), known under its working name H.263+, and addresses the corresponding packetization format issues that together significantly improve the image quality at packet loss rates up to 20%. In particular, it is support from the video coding algorithm itself, paired with appropriate transport layer mechanisms, that leads to significant improvements of perceived image quality for communicative as well as retrieval applications at moderate bit rates up to some 100 kbit/s.

Subband-coded image reconstruction for lossy packet networks

Transmission of digital subband-coded images over lossy packet networks presents a reconstruction problem at the decoder. This paper presents two techniques for reconstruction of lost subband coefficients, one for low-frequency coefficients and one for high-frequency coefficients. The low-frequency reconstruction algorithm is based on inherent properties of the hierarchical subband decomposition. To maintain smoothness and exploit the high intraband correlation, a cubic interpolative surface is fit to known coefficients to interpolate lost coefficients. Accurate edge placement, crucial for visual quality, is achieved by adapting the interpolation grid in both the horizontal and vertical directions as determined by the edges present. An edge model is used to characterize the adaptation, and a quantitative analysis of this model demonstrates that edges can be identified by simply examining the high-frequency bands, without requiring any additional processing of the low-frequency band. High-frequency reconstruction is performed using linear interpolation, which provides good visual performance as well as maintains properties required for edge placement in the low-frequency reconstruction algorithm. The complete algorithm performs well on loss of single coefficients, vectors, and small blocks, and is therefore applicable to a variety of source coding techniques.

Subband filters optimized for lost coefficient reconstruction

Transmission of digital subband coded images over lossy packet networks presents a reconstruction problem at the decoder. This correspondence addresses the design of a family of quadrature mirror filters (QMFs) that minimize the mean-squared error of mean-reconstructed low-frequency subband coefficients. The resulting filters perform well as QMFs as well as provide the desired reconstruction properties in the event of loss.

Transform coded image reconstruction exploiting interblock correlation

Transmission of still images and video over lossy packet networks presents a reconstruction problem at the decoder. Specifically, in the case of block-based transform coded images, loss of one or more packets due to network congestion or transmission errors can result in errant or entirely lost blocks in the decoded image. This article proposes a computationally efficient technique for reconstruction of lost transform coefficients at the decoder that takes advantage of the correlation between transformed blocks of the image. Lost coefficients are linearly interpolated from the same coefficients in adjacent blocks subject to a squared edge error criterion, and the resulting reconstructed coefficients minimize blocking artifacts in the image while providing visually pleasing reconstructions. The required computational expense at the decoder per reconstructed block is less than 1.2 times a non-recursive DCT, and as such this technique is useful for low power, low complexity applications that require good visual performance.