MPEG-4 Research Articles

Dynamic motion estimation for transcoding P frames in H.264 to MPEG-2 transcoders

The efficiency of H.264 video compression together with the wide use of MPEG-2 today necessitates transcoding between these two technologies. The combination of bandwidth efficient technology such as H.264 and the widely deployed MP EG-2 allows for innovative video delivery services that can exploit both these technologies. Using H.264 for video delivery and MPEG-2 for playback on the end user equipment fully leverages the benefits of these technologies. This architecture requires transcoding H.264 video delivered over networks to MPEG-2 format for playback on the end- user device. We present algorithms for transcoding H.264 to MPEG-2 and evaluate the performance of these algorithms. The performance is compared with a baseline transcoder with a full decoding stage and a full encoding stage. The proposed algorithms reduce the complexity by reducing the search space for motion estimation. The results show that the proposed algorithms reduce the motion estimation complexity by over 80% with a negligible loss in PSNR.

Extended MPEG Video Format for Efficient Dynamic Voltage Scaling

We present an extended MPEG video format for efficient Dynamic Voltage Scaling (DVS). DVS technique has been widely researched, but the execution time variation of a periodic task (i. e. MPEG decoding) is still a challenge to be tackled. Unlike previous works, we focus on the data (video stream) rather than the execution code to overcome such limitation. The proposed video format provides the decoding costs of frames to help the precise prediction of their execution times at client machines. The experimental results show that the extended format only increases the data size less than 1% by adding about 10bits representing the decoding cost of each frame. Also, a DVS technique adjusted for the proposed format achieves 90% of efficiency compared to the oracle case, while keeping the run time overhead of the technique negligible.

Real-time shot transition detection in compressed MPEG video streams

Digital libraries became necessary vehicles that provide users with powerful and easy-to-use tools for searching, browsing, and retrieving media information. The starting point for these endeavors is the same: segmentation of video material into shots. The aim of this study is to segment MPEG video streams into shots. A fully automatic detection for both abrupt and gradual transitions (dissolve and fade groups) with minimal decoding in real time is developed. Each detection process was explored through two phases: macroblock type analysis in bidirectional predictive pictures (Bframes), and on-demand intensity information analysis. The abrupt transition detection is explored first by examining the number of forward and backward macroblocks (p- and b-MBs) in consecutive B-frames, and then an intensity histogram comparison is applied to confirm detected transitions. The gradual transition is detected first by examining the intracoded predicted macroblocks (i-MBs) within successive B-frames, and then the detection is confirmed by checking the parabolic shape of the frame variances of the candidate sequence. Results of the study show remarkable detection rate for both abrupt and gradual transitions.

Erratum: Real-time shot transition detection in compressed MPEG video streams

The <i>Journal of Electronic Imaging</i> (JEI), copublished bimonthly with the Society for Imaging Science and Technology, publishes peer-reviewed papers that cover research and applications in all areas of electronic imaging science and technology.

A new framework for MPEG video delivery over heterogeneous networks

The failure of legacy packet markers, such as the Single Rate Three Color Marker (SRTCM) and the Two Rate Three Color Marker (TRTCM), to protect important data from being dropped, renders a poor delivery of video quality over Differentiated Services (DiffServ) networks due to the hierarchical structure of MPEG video. Therefore, we propose a content-aware packet marking scheme, which we call Two Markers System (TMS), to improve the delivery quality of MPEG video streams. TMS places marker modules at both the video source and the edge of a DiffServ network. The source marking modules act at the application-level to set an importance indicator in the IP header of the MPEG frame packet by evaluating the contribution of the packet to the expected picture quality perceived by an end user. Meanwhile, at the edge of the DiffServ network, the Enhanced Token Bucket Three Color Marker (ETBTCM) is proposed to perform packet marking at the network-level in accordance with the pre-assigned importance of individual video frame packets. When the traffic flow is measured as out-of-profile, the less important packets will be marked with higher probability than the more important ones. Therefore, in the event of network congestion, less important frame packets can be dropped before more important ones. Consequently, a better delivered video quality can be provided. Further, due to the convenience offered by wireless networks, more users choose mobile devices to connect to the Internet. Therefore, a Video Differentiator is suggested, placed at the entry to the IEEE 802.11e wireless network, to provide traffic differentiation and better service for video traffic. We use authentic MPEG4 video traffic traces to evaluate this framework. Results show that better video quality can be delivered over our proposed new framework.

Architecture Design of Shape-Adaptive Discrete Cosine Transform and Its Inverse for MPEG-4 Video Coding

This paper presents efficient VLSI architectures of the shape-adaptive discrete cosine transform (SA-DCT) and its inverse transform (SA-IDCT) for MPEG-4. Two of the challenges encountered during the exploitation of more efficient architectures for the SA-DCT and SA-IDCT are addressed. One challenge is to handle the architectural irregularity due to the shape-adaptive nature. The other one is to provide acceptable throughput using minimal hardware. In the algorithm-level optimization, this work exploits the numerical properties found in the transform matrices of various lengths, and derives a fine-grained zero-skipping scheme for the IDCT which can perform 22.6% more zero-skipping than the common vector-based coarse-grained zero-skipping scheme does. In the architecture-level design, the 1-D variable-length DCT/IDCT architectures designed on the basis of the numerical properties are proposed. An auto-aligned transpose memory that aligns the data of different lengths is also incorporated. In addition, a zero-index table is also included in the transpose memory to support the fine-grained zero-skipping in the SA-IDCT. The synthesized designs of the SA-DCT and SA-IDCT are implemented using UMC 0.18-mum technology. The SA-DCT architecture has 26 635 gates, and its average cycle-throughput is 0.66 pixels/cycle, which is comparable to other proposed architectures. On the other hand, the SA-IDCT architecture has 29 960 gates, and its cycle-throughput is 6.42 pixels/cycle. While decoding for CIF@30FPS, the SA-IDCT is clocked at 0.7 MHz, and the power consumption is 0.14 mW. Both the throughput and power consumption of the proposed SA-IDCT architecture are an order better than those of the existing SA-IDCT architectures.

An Evaluation Framework for More Realistic Simulations of MPEG Video Transmission

We present a novel and complete tool-set for evaluating the delivery quality of MPEG video transmissions in simulations of a network environment. This tool-set is based on the EvalVid framework. We extend the connecting interfaces of EvalVid to replace its simple error simulation model by a more general network simulator like NS2. With this combination, researchers and practitioners in general can analyze through simulation the performance of real video streams, i.e. taking into account the video semantics, under a large range of network scenarios. To demonstrate the usefulness of our new tool-set, we point out that it enables the investigation of the relationship between two popular objective metrics for Quality of Service (QoS) assessment of video quality delivery: the PSNR (Peak Signal to Noise Ratio) and the fraction of decodable frames. The results show that the fraction of decodable frames reflects well the behavior of the PSNR metric, while being less time-consuming. Therefore, the fraction of decodable frames can be an alternative metric to objectively assess through simulations the delivery quality of transmission in a network of publicly available video trace files.

Optimal video stream multiplexing through linear programming

This paper presents a new optimal multiplexing scheme for compressed video streams based on their individual e-PCRTT transmission schedules. A linear programming algorithm is proffered, which takes into account the different constraints of each client. The algorithm simultaneously finds the optimum total multiplexed and individual stream schedules that minimize the peak transmission rate. Since the problem is formulated as a linear program it is bounded in polynomial time. It is shown that the algorithm succeeds in obtaining maximum bandwidth utilization with Quality of Service (QoS) guarantees. Simulation results using 10 real MPEG-1 video sequences are presented. The optimal multiplexing linear programming results are compared to the e-PCRTT and Join-the-Shortest-Queue (JSQ) procedures in terms of peak transmission bandwidth, P-loss performance and standard deviation. For several client buffer sizes, the rate obtained by our LP solution when compared to a previous e-PCRTT and JSQ methods resulted in reductions of 47% and 56%, respectively. This implies for a fixed rate problem that the proposed scheme can allow an increase in the number of simultaneously served video streams.

Symbolic quality control for multimedia applications

We present a fine grain quality control method for multimedia applications. The method takes as input an application software composed of actions. The execution times of actions are unknown increasing functions of quality level parameters. The method allows the construction of a Controller which computes adequate action schedules and corresponding quality levels, so as to meet QoS requirements for a given platform. These include requirements for safety (action deadlines are met) as well optimality (maximization and smoothness of quality levels).The Controller consists of a Quality Manager and a Scheduler. For each action, the Controller uses a quality management policy for choosing a schedule and quality levels meeting the QoS requirements. The schedule is selected amongst a set of optimal schedules computed by the Scheduler.We extend and improve results of previous papers providing a solid theoretical basis for designing and implementing the Controller.We propose a symbolic quality management method using speed diagrams, a representation of the controlled system’s dynamics. Instead of numerically computing a quality level for each action, the Quality Manager changes action quality levels based on the knowledge of constraints characterizing control relaxation regions. These are sets of states in which quality management for a given number of computation steps can be relaxed without degrading quality.We study techniques for efficient computation of optimal schedules.We present experimental results including the implementation of the method and benchmarks for an MPEG4 video encoder. The benchmarks show drastic performance improvement for controlled quality with respect to constant quality. They also show that symbolic quality management allows significant reduction of the overhead with respect to numeric quality management. Finally, using optimal schedules can lead to considerable performance gains.

A Fast MB Mode Decision Algorithm for MPEG-2 to H.264 P-Frame Transcoding

The H.264 standard achieves much higher coding efficiency than the MPEG-2 standard, due to its improved inter-and intra-prediction modes at the expense of higher computational complexity. Transcoding MPEG-2 video to H.264 is important to enable gradual migration to H.264. However, given the significant differences between the MPEG-2 and the H.264 coding algorithms, transcoding is a much more complex task and new approaches to transcoding are necessary. The main problems that need to be addressed in the design of an efficient heterogeneous MPEG-2/H.264 transcoder are: the inter-frame prediction, the transform coding and the intra-frame prediction. In this paper, we focus our attention on the inter-frame prediction, the most computationally intensive task involved in the transcoding process. This paper presents a novel macroblock (MB) mode decision algorithm for P-frame prediction based on machine learning techniques to be used as part of a very low complexity MPEG-2 to H.264 video transcoder. Since coding mode decisions take up the most resources in video transcoding, a fast MB mode estimation would lead to reduced complexity. The proposed approach is based on the hypothesis that MB coding mode decisions in H.264 video have a correlation with the distribution of the motion compensated residual in MPEG-2 video. We use machine learning tools to exploit the correlation and construct decision trees to classify the incoming MPEG-2 MBs into one of the several coding modes in H.264. The proposed approach reduces the H.264 MB mode computation process into a decision tree lookup with very low complexity. Experimental results show that the proposed approach reduces the MB mode selection complexity by as much as 95% while maintaining the coding efficiency. Finally, we conduct a comparative study with some of the most prominent fast inter-prediction methods for H.264 presented in the literature. Our results show that the proposed approach achieves the best results for video transcoding applications.

Mining Appearance Models Directly From Compressed Video

In this paper, we propose an approach for learning appearance models of moving objects directly from compressed video. The appearance of a moving object changes dynamically in video due to varying object poses, lighting conditions, and partial occlusions. Efficiently mining the appearance models of objects is a crucial and challenging technology to support content-based video coding, clustering, indexing, and retrieval at the object level. The proposed approach learns the appearance models of moving objects in the spatial-temporal dimension of video data by taking advantage of the MPEG video compression format. It detects a moving object and recovers the trajectory of each macroblock covered by the object using the motion vector present in the compressed stream. The appearances are then reconstructed in the DCT domain along the object's trajectory, and modeled as a mixture of Gaussians (MoG) using DCT coefficients. We prove that, under certain assumptions, the MoG model learned from the DCT domain can achieve pixel-level accuracy when transformed back to the spatial domain, and has a better band-selectivity compared to the MoG model learned in the spatial domain. We finally cluster the MoG models to merge the appearance models of the same object together for object-level content analysis.

WCDMA망상에서 영상통화의 품질향상 기법

영상통화는 3G 이동통신망에서 가장 대표적인 서비스이며, 많은 여러 분야에서 영상통화의 품질을 향상시키기 위한 노력이 있다. 국내에서도 SKT와 KTF에서 3G와 3.5G망으로 WCDMA 방식을 채택해 전국서비스를 시작하였고, 이미 백만 명 이상의 많은 사용자를 확보한 상태이다. 본 논문에서는 WCDMA 망에서의 영상통화방법에 대해 알아보고, 영상품질을 향상시키기 위해 회선망의 물리계층 정보를 이용하여 영상코덱에서 손실확산을 최소화 할 수 있는 슬라이스 그룹을 만드는 새로운 전송기법에 대해 제안하고 실험을 통해 성능을 평가해 보았다. 실험결과, 기존의 방법들보다 좀 더 회선 망과 영상코덱 사이의 동기화가 잘 이루어지고, 이를 통해 손실의 확산을 줄일 수 있는 결과를 보여준다. Video telephony is a representative service in 3G mobile network. And there have been efforts to improve quality of video telephony service in different fields. In Korea, the leading mobile service provider SKTelecom and KTF service the WCDMA network as 3G mobile network. Now, more than a million people is using the network. In this paper, we study about video telephony over WCDMA network. and propose error minimizing algorithm using cross-layer adaptation between physical layer and video codec. We simulated 3G-324M protocol with MPEG-4 video codec, and simulation results show suggested algorithm improve packet transmission rate for improving quality of video telephony service.

Reducing the complexity of MPEG-4 global motion estimation using pixel subsampling

MPEG-4 uses a perspective model for global motion estimation. The region of support for global motion representation consists of the entire image frame. Estimating global motion parameters is very expensive computationally, since all the pixels in the frames are involved in the computation. Therefore, computationally efficient global motion estimation techniques are sought-after. It is shown that real-time global motion estimation in an MPEG-4 video codec can be achieved by using a pixel subsampling method based on regular subsampling patterns. Simulation results demonstrated that the proposed method was capable of speeding up the conventional all-pixel GME approach by more than 11 times, with a subsampling ratio of 1/16, without significant loss in estimation accuracy and compression efficiency.

MEERA: Cross-Layer Methodology for Energy Efficient Resource Allocation in Wireless Networks

In many portable devices, wireless network interfaces consume upwards of 30% of scarce system energy. Reducing the transceiver's power consumption to extend the system lifetime has therefore become a design goal. Our work is targeted at this goal and is based on the following two observations. First, conventional energy management approaches have focused independently on minimizing the fixed energy cost (by shutdown) and on scalable energy costs (by leveraging, for example, the modulation, code-rate and transmission power). These two energy management approaches present a tradeoff. For example, lower modulation rates and transmission power minimize the variable energy component, but this shortens the sleep duration thereby increasing fixed energy consumption. Second, in order to meet the quality of service (QoS) timeliness requirements for multiple users, we need to determine to what extent each system in the network may sleep and scale. Therefore, we propose a two-phase methodology that resolves the sleep-scaling tradeoff across the physical, communications and link layers at design time and schedules nodes at runtime with near optimal energy-efficient configurations in the solution space. As a result, we are able to achieve very low run-time overheads. Our methodology is applied to a case study on delivering a guaranteed QoS for multiple users with MPEG-4 video over a slow-fading channel. By exploiting runtime controllable parameters of actual RF components and a modified 802.11 medium access controller, system lifetime is increased by a factor of 3-to-10 in comparison with conventional techniques.

Shot change detection using a novel histogram feature in compressed video

In this paper, we propose an efficient shot change detection algorithm for MPEG compressed video by using a novel histogram feature essentially different from conventional ones. This algorithm is based on the adjacent pixel intensity difference quantization (APIDQ) histogram method, which had been reliably applied to human face recognition previously. The proposed algorithm has been evaluated by publicly reliable MPEG video database. Experimental results show the APIDQ histogram method can detect shot changes in MPEG video more accurately and robustly than conventional methods.

異種符号化映像フォーマットの総合画質評価

We present a subjective quality assessment scheme for digitally coded motion pictures in multi-formats. Several scanning formats are available in digital broadcasting, and broadcasters can select an adequate delivery format according to the overall picture quality and transmission bit-rate. Thus, the overall quality of digitally coded pictures should be evaluated in terms of both coding degradation and image resolution. This assessment-test scheme enables a direct comparison of picture quality of different scanning formats based on the double stimulus impairment scale (DSIS) method recommended in Rec. ITU-R BT.500. We show results of assessment tests performed on 1080i, 720p, and 480p videos encoded in the MPEG-2 video standard and give some consideration to picture quality in digital broadcasting.

Video Distribution Networks Using Remote Statistical Multiplexing

Statistical techniques have been used to aggregate video content at broadcast headends for some time now. Direct-to-Home systems based on MPEG-2 video encoding are now commonplace thus there is considerable pressure on content providers to make programming available on multiple platforms, with voice and data traffic. Achieving significant reductions in operator costs, to maintain competitive position, is the final consideration facing broadcasters and internet service providers. These demands have led to statistically multiplexed video systems utilizing the IP infrastructure, in addition to dedicated broadcast links, to adopt a distributed topology allowing source encoders to reside at remote broadcaster sites, rather than a centralized headend.

Video Error Concealment Using Fidelity Tracking

We propose a method to prevent the degradation of decoded MPEG pictures caused by video transmission over error-prone networks. In this paper, we focus on the error concealment that is processed at the decoder without using any backchannels. Though there have been various approaches to this problem, they generally focus on minimizing the degradation measured frame by frame. Although this frame-level approach is effective in evaluating individual frame quality, in the sense of human perception, the most noticeable feature is the spatio-temporal discontinuity of the image feature in the decoded video image. We propose a novel error concealment algorithm comprising the combination of i) A spatio-temporal error recovery function with low processing cost, ii) A MB-based image fidelity tracking scheme, and iii) An adaptive post-filter using the fidelity information. It is demonstrated by experimental results that the proposed algorithm can significantly reduce the subjective degradation of corrupted MPEG video quality with about 30 % of additional decoding processing power.

Enhancement of dissolved shot boundary detection with twin-windows amplification method

We propose an algorithm to detect shot transitions of video data in the compressed domain and then extend its effect to the spatial domain. This algorithm is an improvement of the conventional solution, where energy histograms of the DC coefficients are used to calculate the distance between consecutive frames. The DC coefficients of compressed MPEG video data are obtained either through direct extraction [intraframes (I-frames)] or by frame conversion [predicted frames (P-frames)] while those of uncompressed video data are calculated by 8×8 block pixel averaging. The key contribution of this approach is to enhance the detection result by using the ratio between two sliding windows to attenuate the low-pass-filtered frame distance, resulting in the amplification of the transitional regions. The experimental results demonstrate that this algorithm achieves high detection rates with low computational complexity.

An integrated rate control scheme for TCP-friendly MPEG-4 video transmission

Delivering MPEG-4 real-time video over the Internet requires the use of rate control schemes in both the compression and the transport domains. This paper discusses the practical difficulties involved in integrating MPEG-4 scalable rate control (SRC) in the compression domain with TCP-friendly rate control (TFRC) in the transport domain. In addition to the requirement for rate matching between the two schemes to ensure TCP-friendliness, users demand a high-quality, smooth media output under the varying network conditions. In this paper, an integrated rate control scheme is proposed for MPEG-4 video transported by a TFRC protocol. The proposed scheme comprises a rate smoothing control mechanism to meet the rate matching with the controlled transmission buffering delay, and an adaptive rate control for the I-frame designed to reduce the frame skipping. Through the successful integration of the two schemes, the experimental results show that the proposed scheme better preserves the motion smoothness and improves the overall perceptual quality for TCP-friendly MPEG-4 video transmissions to obtain a smooth, high visual quality video experience.