Advanced Video Coding Research Articles

IEEE Standards for Advanced Audio and Video Coding in Emerging Applications

The IEEE Standards Association recently approved a comprehensive set of standards that cover advanced audio and video coding for storage and transmission, targeting emerging applications such as Internet media streaming and smart-camera surveillance. The standards' basic goal is to facilitate reliable and efficient exchange of audiovisual data streams between content creators and target users to provide high-quality, recognition friendly end presentations. They enable audiovisual content storage on a wide range of media and transmission over diverse wired and wireless networks, and offer tools for long-term picture reference, learning based picture generation, and object-based description and coding. In addition, they establish audio and video synchronization guarantees, guidelines for operating in error prone environments, and conventions for embedding metadata to describe content and make it discoverable. The standards conform to IEEE's patent policy, providing access to essential patent licensing at a reasonable cost.

Next-Generation 3D Formats with Depth Map Support

This article reviews the most recent extensions to the Advanced Video Coding (AVC) and High Efficiency Video Coding (HEVC) coding standards, which integrate depth video to support advanced multiview and 3D video functionalities. All the extensions provide single-view compatibility, while some extensions add depth support on top of conforming stereoscopic bitstreams. To achieve the highest gains in coding efficiency, depth information is utilized in coding the texture views. The compression formats described in this article can be used to support emerging auto-stereoscopic displays and free-viewpoint video functionalities.

Quadtree-based non-local Kuan’s filtering in video compression

Transform coding has been widely used in video coding standards, such as H.264 advanced video coding (H.264/AVC) and high efficiency video coding (HEVC). But the coded video sequences suffer from annoying coding artifacts, such as blocking and ringing artifacts. In this paper, we propose the quadtree-based non-local Kuan’s (QNLK) filter to suppress the quantization noise optimally and improve the objective and subjective quality of the reconstructed frame simultaneously. The proposed filter takes advantage of the non-local Kuan’s (NLK) filter to restore the quantized signal in transform domain. Restored coefficients are then projected onto designed quantization constraint sets (QCS). Quadtree-based signaling strategy is used at the end of QNLK for adaptive filtering on/off control. Experimental results of QNLK show that the proposed method achieves significant objective coding gain and visual quality improvement, compared with both H.264/AVC high profile and HEVC.

The H.264 Video Coding Standard

Since 1990, the Video Coding Expert Group (VCEG) and the Moving Pictures Expert Group (MPEG) have focused their research in video coding techniques for different applications. The International Telecommunication Union (ITU-T) and the MPEG groups joined in a single group to form the Joint Video Team (JVT) and worked together on a new video standard called H.264, H.264/AVC Advanced Video Coding, or MPEG-4 part 10, which achieves more coding efficiency with simpler syntax specifications than the previous standards, more integration with all network protocols, and multiplex architectures. The H.264 is the state-ofthe-art codec and covers a wide range of applications with excellent results such as videoconferencing, video streaming, and video transmission over fixed and wireless networks with different transport protocols among others.

A PERFORMANCE COMPARISON ON DIFFERENTIAL PREDICTION BASED ALGORITHMS IN H.264/AVC

H.264 is an Advanced Video Coding standard that gives better coding efficiency than the previous standards. In H.264, video compression is carried out in many ways such as Interframe prediction and Intraframe prediction. Intraframe prediction is carried out by the process of motion estimation by calculating motion vectors. Intraframe prediction reduces spatial redundancy (ie) similarity between pixels and Interframe prediction reduces the temporal redundancy (ie) change in video content from one frame to the next frame. In this paper we compare these two compression schemes of H.264 in terms of compression ratio, PSNR and memory bandwidth. Finally we conclude that intra prediction method provides better video quality with average compression ratio 1.034 and the percentage of memory saving is 96.26% but the average PSNR value is less compared to motion estimation.

H.264‐based hierarchical two‐layer lossless video coding method

An efficient lossless coding technique is very important for storage and transmission applications of error sensitive information such as medical, seismic and digital artistic data. In this study, the authors proposed an H.264-based advance video coding (H.264/AVC)-based hierarchical lossless coding method, where the input video will be firstly encoded by H.264/AVC coder with a quantisation parameter (QP) selector in the base layer and the coded error is encoded by a QP-adaptive Rice coder in the enhancement layer. To reduce encoding time, the QP selector can be simplified to select the nearly optimal QP. Simulation results show that the proposed hierarchical lossless coding architecture achieves better compression ratio than the traditional H.264/AVC-based lossless coding systems. Since the proposed system could provide both lossy and lossless coding services at the same time, the proposed lossless video coding system has advantages of efficiency and flexibility for practical applications. Experimental results show that the proposed coding system can provide less coding bits and reduce coding complexity compare with H.264-differential pulse code modulation.

Background-Modeling-Based Adaptive Prediction for Surveillance Video Coding.

The exponential growth of surveillance videos presents an unprecedented challenge for high-efficiency surveillance video coding technology. Compared with the existing coding standards that were basically developed for generic videos, surveillance video coding should be designed to make the best use of the special characteristics of surveillance videos (e.g., relative static background). To do so, this paper first conducts two analyses on how to improve the background and foreground prediction efficiencies in surveillance video coding. Following the analysis results, we propose a background-modeling-based adaptive prediction (BMAP) method. In this method, all blocks to be encoded are firstly classified into three categories. Then, according to the category of each block, two novel inter predictions are selectively utilized, namely, the background reference prediction (BRP) that uses the background modeled from the original input frames as the long-term reference and the background difference prediction (BDP) that predicts the current data in the background difference domain. For background blocks, the BRP can effectively improve the prediction efficiency using the higher quality background as the reference; whereas for foreground-background-hybrid blocks, the BDP can provide a better reference after subtracting its background pixels. Experimental results show that the BMAP can achieve at least twice the compression ratio on surveillance videos as AVC (MPEG-4 Advanced Video Coding) high profile, yet with a slightly additional encoding complexity. Moreover, for the foreground coding performance, which is crucial to the subjective quality of moving objects in surveillance videos, BMAP also obtains remarkable gains over several state-of-the-art methods.

Study of H.264/Avc Algorithm and It’s Implementation In Matlab

H.264/AVC(Advanced Video Coding) is a video coding standard of the ITU-T(International Telecommunication Union for Telecommunication Standardization Sector) Video Coding Experts Group and the ISO/IEC(International Standards Organization/International Electrotechnical Commission) Moving Picture Experts Group. The main goals of the H.264/AVC standardization effort have been enhanced compression performance and provision of a network friendly video representation addressing conversational (video telephony) and no conversational (storage, broadcast, or streaming) applications. H.264/AVC has achieved a significant improvement in rate-distortion efficiency relative to existing standards. This article provides an overview of the technical features of H.264/AVC, describes profiles and applications for the standard, and compared it with H.261 and H.263.

Video Traffic Characteristics of Modern Encoding Standards: H.264/AVC with SVC and MVC Extensions and H.265/HEVC

Video encoding for multimedia services over communication networks has significantly advanced in recent years with the development of the highly efficient and flexible H.264/AVC video coding standard and its SVC extension. The emerging H.265/HEVC video coding standard as well as 3D video coding further advance video coding for multimedia communications. This paper first gives an overview of these new video coding standards and then examines their implications for multimedia communications by studying the traffic characteristics of long videos encoded with the new coding standards. We review video coding advances from MPEG-2 and MPEG-4 Part 2 to H.264/AVC and its SVC and MVC extensions as well as H.265/HEVC. For single-layer (nonscalable) video, we compare H.265/HEVC and H.264/AVC in terms of video traffic and statistical multiplexing characteristics. Our study is the first to examine the H.265/HEVC traffic variability for long videos. We also illustrate the video traffic characteristics and statistical multiplexing of scalable video encoded with the SVC extension of H.264/AVC as well as 3D video encoded with the MVC extension of H.264/AVC.

The application of neural networks to improve the quality of experience of video transmission over IP networks

The transmission of real-time multimedia streams requires service guarantees, such as limited packet loss, minimum bandwidth and low delay and jitter, to ensure a good quality of experience (QoE) for viewers. The spatial and temporal redundancy of videos is addressed by coding algorithms that reduce the amount of information necessary to represent the images. As a consequence, multimedia traffic commonly presents variable bit rate behavior and self-similar characteristics. Although the reduction in bandwidth requirements is highly desirable, the burstiness of traffic leads to problems in network design and performance prediction. Even a low level of packet loss could severely affect the viewer QoE. In this paper, we propose a real-time packet payload classifier, implemented with artificial neural network (ANN) to be used at network routers. A priority packet discard strategy can be implemented to avoid discarding packets that carry the most relevant information for image reconstruction, thus improving the perceived quality. This approach does not require changes at the video source to classify outgoing packets. The ANN was employed because of its good capacity in temporal series recognition and the possibility of its implementation in real-time systems due to its low computational complexity. The video traces used for training and validation were encoded with H.264/MPEG-4 Advanced Video Coding and are publicly available. The priority packet discard strategy was tested through computational simulations. The QoE was estimated comparing the peak signal-to-noise ratio (PSNR) of original and the received frames of video, and the results indicate that the proposed method improves the QoE. The implementation does not require packet payload processing and can be performed with network layer information only.

A PMC-driven methodology for energy estimation in RVC-CAL video codec specifications

In this paper, an energy estimation methodology based on performance monitor counters (PMC) is proposed to estimate the energy consumption of RVC-CAL video codec specifications. The proposed PMC-driven methodology is able to automatically identify the most appropriate events and training data to cover the main application characteristics. In addition, knowledge of the hardware platform employed is not required. Therefore, this methodology can be easily implemented on other PMC-available systems while keeping the estimation accuracy. It is worth noting that this is an attractive asset to analyze the energy consumption of RVC-CAL codec specifications. Besides, the methodology reduces the PMC redundancy and, thus, the overhead introduced when applied to on-line power management. Experimenting on two RVC-CAL decoders, H.264 and MPEG4 Part2 SP, a coarse estimation model based on instructions per cycle (IPC) and the proposed PMC-driven model are compared. The results show that the PMC-driven model can achieve for the H.264 and MPEG4 Part2 SP decoders average estimation errors of 5.95% and 5.01%, respectively, in comparison to the 17.11% and 13.65% average errors obtained with the IPC-based model. As a consequence, this methodology is suggested to be combined into the RVC framework to help the designer to have an overview of the energy consumption of the specification actors at earlier design stages.

MPEG Reconfigurable Video Coding: From specification to a reconfigurable implementation

This paper demonstrates that it is possible to produce automatic, reconfigurable, and portable implementations of multimedia decoders onto platforms with the help of the MPEG Reconfigurable Video Coding (RVC) standard. MPEG RVC is a new formalism standardized by the MPEG consortium used to specify multimedia decoders. It produces visual representations of decoder reference software, with the help of graphs that connect several coding tools from MPEG standards. The approach developed in this paper draws on Dataflow Process Networks to produce a Minimal and Canonical Representation (MCR) of MPEG RVC specifications. The MCR makes it possible to form automatic and reconfigurable implementations of decoders which can match any actual platforms. The contribution is demonstrated on one case study where a generic decoder needs to process a multimedia content with the help of the RVC specification of the decoder required to process it. The overall approach is tested on two decoders from MPEG, namely MPEG-4 part 2 Simple Profile and MPEG-4 part 10 Constrained Baseline Profile. The results validate the following benefits on the MCR of decoders: compact representation, low overhead induced by its compilation, reconfiguration and multi-core abilities.

Multi-symbol QIM video watermarking

This paper introduces the theoretical framework allowing for the binary quantization index modulation (QIM) embedding techniques to be extended towards multiple-symbol QIM (m-QIM, where m stands for the number of symbols on which the mark is encoded prior to its embedding). The underlying detection method is optimized with respect to the minimization of the average error probability, under the hypothesis of white, additive Gaussian behavior for the attacks. This way, for prescribed transparency and robustness constraints, the data payload is increased by a factor of log2m.m-QIM is experimentally validated under the frameworks of the MEDIEVALS French national project and of the SPY ITEA2 European project, related to MPEG-4 AVC robust and semi-fragile watermarking applications, respectively. The experiments are three-folded and consider the data payload–robustness–transparency tradeoff. In the former case, the main benefit is the increase of data payload by a factor of log2m while keeping fixed robustness (variations lower than 3% of the bit error rate after additive noise, transcoding and Stirmark random bending attacks) and transparency (set to average PSNR=45dB and 65dB for SD and HD encoded content, respectively). The experiments consider 1h of video content. In the semi-fragile watermarking case, the m-QIM main advantage is a relative gain factor of 0.11 of PSNR for fixed robustness (against transcoding), fragility (to content alteration) and the data payload. The experiments consider 1h 20min of video content.

Two-step regionally focused algorithm for H.264/AVC encoder

H.264, MPEG-4 Part 10, is the latest digital video coding standard that achieves very high data compression by using several new coding features. One of the key features is the intra prediction based on the rate-distortion optimization (RDO) procedure, which requires a true encoding of the intra block by trying all intra prediction modes. The computational burden of intra prediction with the RDO procedure is extremely high for real-time applications. In this paper, we propose a classified block type algorithm that selects the appropriate intra block size and further our proposed two-step regionally focused algorithm selected block that reduces a number of candidate mode required for RDO which significantly reduces the encoder complexity. Experimental results show that the proposed fast mode decision algorithm reduces the encoding time by about 68%, with negligible loss of video quality.

Multiview Video Coding Optimization using SIMD on Portable Devices

Multimedia support on hand-held devices is growing rapidly. The introduction of 3D videos in cinemas and on home entertainment systems has made it appealing on mobile environment devices. MVC (Multiview Video Coding) provides a compressed representation of multiple views in a single scene. However, handheld devices have hardware constraints that makes it challenging to support highdefinition content and increases the complexity of the algorithms used. Due to this, an advanced processor with the capabilities of low power consumption and high instruction computation is needed. Compression standards such as H.264/MPEG-4 and its amendment, AVC (Advanced Video Coding), have had great success in video compression. However, more complex algorithms and significantly higher processing power are necessary. As an extension to this standard, MVC was introduced for 3D stereo video support. Conventional processors contain coprocessors that support Single Instruction and Multiple Data technology. Some arithmetic operations are also supported on the main processor. Therefore these features can reduce the decoding time of the MVC decoding process. General Terms Multiview video coding, Video codec, SIMD, H.264 codec, Decoder, Standards, Optimization

Fast mode decision algorithm for H.264/SVC enhancement layer

Scalable video coding (SVC) has been standardized to extend the capabilities of the H.264 advanced video coding (AVC). The SVC can compress several video sequences of various resolutions as a single bit-stream. In the SVC enhancement layer, for Joint Scalable Video Model (JSVM) software implementation, an exhaustive mode decision process based on the base layer mode predictions is performed to obtain the best mode for each macroblock (MB). This technique may achieve a higher coding efficiency; however, it induces a significant computational complexity in the encoding engine. In order to speedup the SVC encoder, a fast mode decision algorithm was proposed in this paper. In other words, our aim was to decrease the number of candidate modes to reduce the computational complexity and maintain the same level of coding efficiency, this approach used the spatial and temporal correlation between MB situated at the enhancement layer and its co-located MB at the base layer. Our statistical analyses were made using several HD sequences with different motion characteristics. Experimental results show a significant improvement in terms of time encoding which is a major constraint for any real-time implementation. However, this gain is accompanied with an acceptable loss in video quality and a tolerable bit rate increase for most media supports. In fact, our proposed algorithm permits a major improvement that can reach up to 64.9 % in terms of computational effort. This gain will induce an average loss yield to 10.5 or 13.87 % that is comparable to the 13.12 % of the He Li's algorithm with an acceptable loss in terms of subjective video quality.

Multiview-Video-Plus-Depth Coding Based on the Advanced Video Coding Standard

This paper presents a multiview-video-plus-depth coding scheme, which is compatible with the advanced video coding (H.264/AVC) standard and its multiview video coding (MVC) extension. This scheme introduces several encoding and in-loop coding tools for depth and texture video coding, such as depth-based texture motion vector prediction, depth-range-based weighted prediction, joint inter-view depth filtering, and gradual view refresh. The presented coding scheme is submitted to the 3D video coding (3DV) call for proposals (CfP) of the Moving Picture Experts Group standardization committee. When measured with commonly used objective metrics against the MVC anchor, the proposed scheme provides an average bitrate reduction of 26% and 35% for the 3DV CfP test scenarios with two and three views, respectively. The observed bitrate reduction is similar according to an analysis of the results obtained for the subjective tests on the 3DV CfP submissions.

Exploiting task and data parallelism for advanced video coding on hybrid CPU + GPU platforms

Considering the prevalent usage of multimedia applications on commodity computers equipped with both CPU and GPU devices, the possibility of simultaneously exploiting all parallelization capabilities of such hybrid platforms for high performance video encoding has been highly quested for. Accordingly, a method to concurrently implement the H.264/ advanced video coding (AVC) inter-loop on hybrid GPU + CPU platforms is proposed in this manuscript. This method comprises dynamic dependency aware task distribution methods and real-time computational load balancing over both the CPU and the GPU, according to an efficient dynamic performance modeling. With such optimal balance, the set of rather optimized parallel algorithms that were conceived for video coding on both the CPU and the GPU are dynamically instantiated in any of the existing processing devices, to minimize the overall encoding time. The proposed model does not only provide an efficient task scheduling and load balancing for H.264/AVC inter-loop, but it also does not introduce any significant computational burden to the time-limited video coding application. Furthermore, according to the presented set of experimental results, the proposed scheme has proved to provide speedup values as high as 2.5 when compared with highly optimized GPU-only encoding solutions or even other state of the art algorithm. Moreover, by simply using the existing computational resources that usually equip most commodity computers the proposed scheme is able to achieve inter-loop encoding rates as high as 40 fps at a HD 1920 × 1080 resolution.

Interactive scheduling for mobile multimedia service in M2M environment

Computational load of motion estimation in advanced video coding (AVC) standard is significantly high and its more true for HDTV sequences. In this paper, video processing algorithm is mapped onto a learning method to improve machine to machine (M2M) architecture, namely, the parallel reconfigurable computing (PRC) architecture, which consists of multiple units, First, we construct a directed acyclic graph (DAG) to represent the video coding algorithms comprising motion estimation. In the future trillions of devices are connected (M2M) together to provide services and that time power management would be a challenge. Computation aware scheme for different machine is reduced by dynamically scheduling usage of multi-core processing environment for video sequence depending up complexity of the video. And different video coding algorithm is selected depending upon the nature of the video. Simulation results show the effectiveness of the proposed method.

The New High-Efficiency Video Coding Standard

Recent developments in camera and display technologies are supporting increasingly larger resolutions. At the same time, studies predict that the share of video data in overall consumer Internet traffic will become larger than 75%. These developments require a more efficient coding of digital video than the current video-coding standard H.264/ MPEG-4 AVC is capable of. The new High-Efficiency Video Coding (HEVC) standard achieves this by introducing new coding tools as well as by improving techniques already known from H.264/MPEG- 4 AVC. Recent performance analyses comparing HEVC with H.264/MPEG-4 AVC High Profile show average bitrate reductions around 35 to 40% for a given objective quality while subjective test results indicate that a 50% bitrate reduction for a comparable visual quality can be achieved. Besides techniques for substantially improving coding efficiency, large efforts have been put on facilitating parallel encoding and decoding in HEVC. This article presents in detail all these different aspects of HEVC.