Advanced Video Coding Research Articles

Efficient Foreground Extraction From HEVC Compressed Video for Application to Real-Time Analysis of Surveillance 'Big' Data.

While surveillance video is the biggest source of unstructured Big Data today, the emergence of high-efficiency video coding (HEVC) standard is poised to have a huge role in lowering the costs associated with transmission and storage. Among the benefits of HEVC over the legacy MPEG-4 Advanced Video Coding (AVC), is a staggering 40 percent or more bitrate reduction at the same visual quality. Given the bandwidth limitations, video data are compressed essentially by removing spatial and temporal correlations that exist in its uncompressed form. This causes compressed data, which are already de-correlated, to serve as a vital resource for machine learning with significantly fewer samples for training. In this paper, an efficient approach to foreground extraction/segmentation is proposed using novel spatio-temporal de-correlated block features extracted directly from the HEVC compressed video. Most related techniques, in contrast, work on uncompressed images claiming significant storage and computational resources not only for the decoding process prior to initialization but also for the feature selection/extraction and background modeling stage following it. The proposed approach has been qualitatively and quantitatively evaluated against several other state-of-the-art methods.

Fast Quadtree Structure Decision for HEVC Intra Coding Using Histogram Statistics

The final draft of the latest video coding standard, High Efficiency Video Coding (HEVC), was approved in January 2013. The coding efficiency of HEVC surpasses its predecessor, H.264/MPEG-4 Advanced Video Coding (AVC), by using only half of the bitrate to encode the same sequence with similar quality. However, the complexity of HEVC is sharply increased compared to H.264/AVC. In this paper, a method is proposed to decrease the complexity of intra coding in HEVC. Early pruning and an early splitting strategy are applied to the quadtree structure of coding tree units (CTU) and residual quadtree (RQT). According to our experiment, when our method is applied to sequences from Class A to Class E, the coding time is decreased by 44% at the cost of a 1.08% Bjontegaard delta rate (BD-rate) increase on average.

NR-Bitstream video quality metrics for SSIM using encoding decisions in AVC and HEVC coded videos

We propose a no-reference compressed video quality model to predict the full-reference SSIM metrics for AVC (Advanced Video Coding, H.264) and HEVC (High Efficiency Video Coding) videos. The model we use is support vector regression (SVR) model. We use only encoding decisions made during motion estimation to perform the prediction, and do not need the information from pixel domain. We show that the Block-Partition-related features have great importance in SSIM prediction, especially for HEVC videos, due to its partition decisions being more complex than those of AVC. The proposed SVR model trained by data of two different encoding configurations can predict SSIM well for AVC videos (0.78 correlation) and for HEVC videos (0.88 correlation). The proposed models are also compared with a state-of-the-art no-reference-bitstream-pixel SSIM prediction model. We show that the proposed methods provide higher prediction correlation (as high as 13.13% improvement in correlation) with much lower complexity.

Ultra-High-Throughput VLSI Architecture of H.265/HEVC CABAC Encoder for UHDTV Applications

Ultra high definition television (UHDTV) imposes extremely high throughput requirement on video encoders based on High Efficiency Video Coding (H.265/HEVC) and Advanced Video Coding (H.264/AVC) standards. Context-adaptive binary arithmetic coding (CABAC) is the entropy coding component of these standards. In very-large-scale integration implementation, CABAC has known difficulties in being effectively pipelined and parallelized, due to the critical bin-to-bin data dependencies in its algorithm. This paper addresses the throughput requirement of CABAC encoding for UHDTV applications. The proposed optimizations including prenormalization, hybrid path coverage and lookahead rLPS to reduce the critical path delay of binary arithmetic encoding (BAE) by exploiting the incompleteness of data dependencies in rLPS updating. Meanwhile, the number of bins BAE delivers per clock cycle is increased by the proposed bypass bin splitting technique. The context modeling and binarization components are also optimized. As a result, our CABAC encoder delivers an average of 4.37 bins per clock cycle. Its maximum clock frequency reaches 420 MHz when synthesized in 90 nm. The corresponding overall throughput is 1836 Mbin/s that is 62.5% higher than the state-of-the-art architecture.

Efficient and Reliable MPEG-4 Multicast MAC Protocol for Wireless Networks

Multicasting is the transmission of data to a group of nodes identified by a single destination address. Furthermore, multicasting is considered as an appropriate transmission method for multimedia services. Multimedia applications are expected to become more prevalent over mobile ad hoc networks in the near future. Therefore, achieving reliability in multimedia communications is an important task. Video compression technologies are about reducing and removing redundant video data so that a digital video file can be effectively sent over a network. With modern compression standards, such as Moving Picture Expert Group 2 (MPEG-2), MPEG-4, and H.264, which is also known as advanced video coding MPEG-4 part 10, losses of different frames have different impacts on video quality. In this paper, we propose a leader-based reliable multicast medium-access control layer protocol for multimedia applications to enhance video quality. We present a Markov chain model and numerical formulation of our proposed protocol. Simulation results show that our proposed method is better than other protocols in terms of the number of decodable frames, peak signal-to-noise ratio (PSNR), and video quality.

State of HEVC Bitrates in 2014: Comparing HEVC, H.264, and MPEG-2

This paper presents the results of empirical testing of high-efficiency video coding (HEVC) for standard-definition (SD), high-definition (HD), and ultra-high-definition (UHD), including data from hundreds of encoding tests across a variety of bitrates. The paper then compares HEVC quality results with the quality achieved by common Motion Picture Experts Group 2 (MPEG-2) and advanced video coding (AVC) distribution profiles to determine HEVC bitrate recommendations for SD, HD, and UHD distribution. The paper presents the following recommended bitrates for HEVC encoding: SD at 860 kbits/sec, HD at 2703 kbits/sec, and UHD at 23,922 kbits/sec. These represent a 45% bitrate savings, compared to AVC. HEVC encoding was done using x265 version 1.2, and all quality measurements were done using the peak signal-to-noise ratio and structural similarity index. The paper uses the HEVC Main (8-bit) profile to allow quality matching with H.264 and MPEG-2 8-bit encoding.

H.264 SVC Complexity Reduction Based on Likelihood Mode Decision.

H.264 Advanced Video Coding (AVC) was prolonged to Scalable Video Coding (SVC). SVC executes in different electronics gadgets such as personal computer, HDTV, SDTV, IPTV, and full-HDTV in which user demands various scaling of the same content. The various scaling is resolution, frame rate, quality, heterogeneous networks, bandwidth, and so forth. Scaling consumes more encoding time and computational complexity during mode selection. In this paper, to reduce encoding time and computational complexity, a fast mode decision algorithm based on likelihood mode decision (LMD) is proposed. LMD is evaluated in both temporal and spatial scaling. From the results, we conclude that LMD performs well, when compared to the previous fast mode decision algorithms. The comparison parameters are time, PSNR, and bit rate. LMD achieve time saving of 66.65% with 0.05% detriment in PSNR and 0.17% increment in bit rate compared with the full search method.

A Hybrid Scheme Based on Pipelining and Multitasking in Mobile Application Processors for Advanced Video Coding

One of the key requirements for mobile devices is to provide high-performance computing at lower power consumption. The processors used in these devices provide specific hardware resources to handle computationally intensive video processing and interactive graphical applications. Moreover, processors designed for low-power applications may introduce limitations on the availability and usage of resources, which present additional challenges to the system designers. Owing to the specific design of the JZ47x series of mobile application processors, a hybrid software-hardware implementation scheme for H.264/AVC encoder is proposed in this work. The proposed scheme distributes the encoding tasks among hardware and software modules. A series of optimization techniques are developed to speed up the memory access and data transferring among memories. Moreover, an efficient data reusage design is proposed for the deblock filter video processing unit to reduce the memory accesses. Furthermore, fine grained macroblock (MB) level parallelism is effectively exploited and a pipelined approach is proposed for efficient utilization of hardware processing cores. Finally, based on parallelism in the proposed design, encoding tasks are distributed between two processing cores. Experiments show that the hybrid encoder is 12 times faster than a highly optimized sequential encoder due to proposed techniques.

Scalable Bit Allocation Between Texture and Depth Views for 3-D Video Streaming Over Heterogeneous Networks

In the multiview video plus depth (MVD) coding format, both texture and depth views are jointly compressed to represent the 3-D video content. The MVD format enables synthesis of virtual views through depth-image-based rendering; hence, distortion in the texture and depth views affects the quality of the synthesized virtual views. Bit allocation between texture and depth views has been studied with some promising results. However, to the best of our knowledge, most of the existing bit-allocation methods attempt to allocate a fixed amount of total bit rate between texture and depth views; that is, to select appropriate pair of quantization parameters for texture and depth views to maximize the synthesized view quality subject to a fixed total bit rate. In this paper we propose a scalable bit-allocation scheme, where a single ordering of texture and depth packets is derived and used to obtain optimal bit allocation between texture and depth views for any total target rates. In the proposed scheme, both texture and depth views are encoded using the quality scalable coding method; that is, medium grain scalable (MGS) coding of the Scalable Video Coding (SVC) extension of the Advanced Video Coding (H.264/AVC) standard. For varying target total bit rates, optimal bit truncation points for both texture and depth views can be obtained using the proposed scheme. Moreover, we propose to order the enhancement layer packets of the H.264/SVC MGS encoded depth view according to their contribution to the reduction of the synthesized view distortion. On one hand, this improves the depth view packet ordering when considered the rate-distortion performance of synthesized views, which is demonstrated by the experimental results. On the other hand, the information obtained in this step is used to facilitate optimal bit allocation between texture and depth views. Experimental results demonstrate the effectiveness of the proposed scalable bit-allocation scheme for texture and depth views.

Impact of Random and Burst Packet Losses on H.264 Scalable Video Coding

This paper presents an encoder-based prediction model to study the impact of packet loss on H.264 scalable video coding (SVC). A Markov Chain (MC) with 2 N states is developed to describe the error propagation process inside a group of pictures (GOP). Using the packet loss rate reported by RTCP (RTP Control Protocol) and analyzing the inter-frame prediction rules, we evaluate packet-loss-induced quality degradation in SVC as the probability that a given scalable layer can be correctly received. Based on the proposed model, the performance of the SVC hierarchical B-frame structure, the zero delay encoding/decoding structure, and the advance video coding (AVC) IPPP structure (compatible base layer in SVC) are evaluated and compared under both random and burst packet loss events.

QoS-Optimized Adaptive Multi-layer (OQAM) architecture of wireless network for high quality digital video transmission

Quality of Service (QoS) optimization is an important design goal in wireless video transmission. The application (APP) layer 802.11e medium access control (MAC) layer, and physical (PHY) layer of the wireless protocol stack can be jointly designed for the exchange of information. This will optimize the performance of wireless network for real-time digital video transmission. This paper proposes an innovative ‘QoS-Optimized Adaptive Multi-layer (OQAM)’ architecture. It ensures reliable and high-quality video transmission over communication channels. The channel exhibits wide variability in throughput, delay, and packet loss. The simulation is performed using EvalVid and Network Simulator-2. Enhanced Intra Prediction (EnIP) algorithm with H.264/MPEG4 Advanced Video Coding (AVC) is proposed at the APP layer. This H.264/MPEG4 AVC is a non-scalable video enCOder/DECoder (CODEC). Improved MAC Adaptive Retry Limit (IMALr) is proposed as smart-packet drop mechanism at the 802.11e MAC layer. It uses packet overflow drop (Pov) and expired-time packet discard (Pex) algorithm. Enhanced Adaptive Forward Error Correction (EnAFEC) is proposed at the PHY layer. The aforementioned algorithms are jointly considered in the proposed OQAM architecture, which increases the coding efficiency, reduces end-to-end delay and increases reliability of wireless network for real-time Video over Wireless (VoW) transmission.

View Synthesis Prediction in the 3-D Video Coding Extensions of AVC and HEVC

Advanced multiview video systems are able to generate intermediate viewpoints of a 3-D scene. To enable low-complexity free view generation, texture and its associated depth are used as input data for each viewpoint. To improve the coding efficiency of such content, view synthesis prediction (VSP) is proposed to further reduce interview redundancy in addition to traditional disparity compensated prediction. This paper describes and analyzes rate-distortion optimized VSP designs, which were adopted in the 3-D extensions of both Advanced Video Coding (AVC) and High Efficiency Video Coding (HEVC). In particular, we propose a novel backward-VSP scheme using a derived disparity vector, as well as efficient signalling methods in the context of AVC and HEVC. In addition, we put forward a novel depth-assisted motion vector prediction method to optimize the coding efficiency. A thorough analysis of coding performance is provided using different VSP schemes and configurations. Experimental results demonstrate average bit rate reductions of 2.5% and 1.2% in AVC and HEVC coding frameworks, respectively, with up to 23.1% bit rate reduction for dependent views.

Offset Compensation Method for Skip Mode in Hybrid Video Coding

The skip mode has been adopted to reduce the bitrate in hybrid video coding such as H.264/MPEG-4 Advanced Video Coding (AVC). To improve the coding performance by reducing the dc distortion in the skip mode, we propose a skip with offset method, which uses offset compensation with the estimated offset only in the skip mode. In the proposed method, no flag bit is used for offset because incorrectly estimated offsets are mostly filtered out through the rate distortion mode decision. Only a semantic change is needed at the skip mode in the proposed method. The existing skip mode can therefore be easily replaced with the proposed method. The proposed method shows better coding performance than previous methods, including the weighted prediction method, in H.264/AVC. Furthermore, the proposed method shows a significant coding gain of -7.60% and -5.81% in its averaged bitrate differences compared, respectively, with baseline and high profiles of H.264/AVC at 720p high definition sequences.

Efficient Mode Decision Schemes for HEVC Inter Prediction

The emerging High Efficiency Video Coding (HEVC) standard reduces the bit rate by almost 40% over the preceding state-of-the-art Advanced Video Coding (AVC) standard with the same objective quality but at about 40% encoding complexity overhead. The main reason for HEVC complexity is inter prediction that accounts for 60%-70% of the whole encoding time. This paper analyzes the rate-distortion-complexity characteristics of the HEVC inter prediction as a function of different block partition structures and puts the analysis results into practice by developing optimized mode decision schemes for the HEVC encoder. The HEVC inter prediction involves three different partition modes: square motion partition, symmetric motion partition (SMP), and asymmetric motion partition (AMP) out of which the decision of SMPs and AMPs are optimized in this paper. The key optimization techniques behind the proposed schemes are: 1) a conditional evaluation of the SMP modes; 2) range limitations primarily in the SMP sizes and secondarily in the AMP sizes; and 3) a selection of the SMP and AMP ranges as a function of the quantization parameter. These three techniques can be seamlessly incorporated in the existing control structures of the HEVC reference encoder without limiting its potential parallelization, hardware acceleration, or speed-up with other existing encoder optimizations. Our experiments show that the proposed schemes are able to cut the average complexity of the HEVC reference encoder by 31%-51% at a cost of 0.2%-1.3% bit rate increase under the random access coding configuration. The respective values under the low-delay B coding configuration are 32%-50% and 0.3%-1.3%.

The IEEE 1857 Standard: Empowering Smart Video Surveillance Systems

The IEEE 1857 Standard for Advanced Audio and Video Coding was released as IEEE 1857-2013 in June 2013. Despite consisting of several different groups, the most significant feature of IEEE 1857-2013 is its Surveillance Groups, which can not only achieve at least twice the coding efficiency on surveillance videos as H.264/AVC High Profile, but it's the most analysis-friendly video coding standard. This article presents an overview of IEEE 1857 Surveillance Groups, highlighting background model-based coding technology and analysis-friendly functionalities. IEEE 1857-2013 will present new opportunities and drive research in smart video surveillance communities and industries.

Unified transform architecture for AVC, AVS, VC-1 and HEVC high-performance codecs

A unified architecture for fast and efficient computation of the set of two-dimensional (2-D) transforms adopted by the most recent state-of-the-art digital video standards is presented in this paper. Contrasting to other designs with similar functionality, the presented architecture is supported on a scalable, modular and completely configurable processing structure. This flexible structure not only allows to easily reconfigure the architecture to support different transform kernels, but it also permits its resizing to efficiently support transforms of different orders (e.g. order-4, order-8, order-16 and order-32). Consequently, not only is it highly suitable to realize high-performance multi-standard transform cores, but it also offers highly efficient implementations of specialized processing structures addressing only a reduced subset of transforms that are used by a specific video standard. The experimental results that were obtained by prototyping several configurations of this processing structure in a Xilinx Virtex-7 FPGA show the superior performance and hardware efficiency levels provided by the proposed unified architecture for the implementation of transform cores for the Advanced Video Coding (AVC), Audio Video coding Standard (AVS), VC-1 and High Efficiency Video Coding (HEVC) standards. In addition, such results also demonstrate the ability of this processing structure to realize multi-standard transform cores supporting all the standards mentioned above and that are capable of processing the 8k Ultra High Definition Television (UHDTV) video format (7,680 × 4,320 at 30 fps) in real time.

Evaluation of Long Term Evolution Cellular Network Performance when Transmitting Multi-view Video Content

The Long Term Evolution (LTE) cellular technology provides higher data rates than its predecessor technologies. This advancement paves the way for more data services, including improved multimedia services. Three-dimensional (3D) video transmission is one such service that can benefit from LTE deployment. For a positive uptake of 3D video transmission, the network must provide a good Quality of Service (QoS). In this paper the authors evaluate the LTE network's performance when transmitting Multi-view Video Coding (MVC) using simulcast and inter-view prediction coding. Furthermore, the authors evaluate the system using both the H.264/AVC (Advanced Video Coding) and the more recent High Efficiency Video Coding (HEVC) and their MVC extensions. Results show that, in an urban environment, LTE can accommodate a maximum of 93 users per cell, with adequate QoS, when transmitting 3D HEVC video at Common Intermediate Format (CIF) resolution. Moreover, cross-layer techniques can be used to reduce the QoS degradation as the user moves away from the eNodeB by transmitting lower resolution video.

View Synthesis Distortion Estimation for AVC- and HEVC-Compatible 3-D Video Coding

This paper presents an efficient view synthesis distortion estimation method for 3-D video. It also introduces the application of this method to Advanced Video Coding (AVC)- and High Efficiency Video Coding (HEVC)-compatible 3-D video coding. Although the proposed view synthesis distortion scheme is generic, its use for actual 3-D video codec systems addresses the many issues caused by different video-coding formats and restrictions. The solutions for these issues are herein proposed. The simulation results show that the proposed scheme can achieve approximately 5.4% and 10.2% coding gains for AVC- and HEVC-compatible 3-D coding, respectively. In addition, the results show the remarkable complexity reduction of the scheme compared to the view synthesis optimization method currently used in 3-D-HEVC. The proposed method has been adopted into the presently developing AVC- and HEVC-compatible test model reference software.

EFFICIENT DOCUMENT COMPRESSION USING INTRA FRAME PREDICTION TECTHNIQUE

Compression is the reduction of the irrelevant or redundant data, in order to save space or transmission time. Scanned document is compressed to reduce the requirement of the storage space. This paper proposes, an Advanced Document Coding (ADC), which was a H.264/AVC (Advance Video Coding) video coding standard for compressing the scanned document. During encoding phase, intra-frame and inter-frame prediction technique is used to reduce the redundancy of the image data. Intra-frame prediction technique is performed relative to information that is contained only within the current frame and not relative to any other frame. The Inter-frame prediction technique means only changes in the image from the previous subpages. This prediction generates residual data that can be transformed and encoded using Context Adaptive Binary Arithmetic Coding (CABAC). In the decoding process, Arithmetic decoding, inverse transform are used to assemble the final document page. The proposed work is analyzed by Peak Signal to Noise Ratio (PSNR), which is calculated by using global Mean Square Error (MSE). The result reveals that, ADC improves the encoding efficiency than the JPEG2000 in terms of PSNR value.

FAST CALCULATION OF 8 × 8 INTEGER DCT IN THE SOFTWARE IMPLEMENTATION OF H.264/AVC

In order to achieve higher compression performance the fidelity range extension (FRExt) amendment was added to the H.264 advanced video coding (AVC) standard. It uses both 4 × 4 and 8 × 8 integer discrete cosine transform (DCT) adaptively in the high profiles. This led to additional complexity of the initial version of the H.264/AVC encoder which had substantially high computational complexity. In this paper, we propose a new algorithm which reduces the computational complexity for software implementation of horizontal 8 × 8 integer DCT by more than 25%. Simulation results indicate 22% reduction in the computation time by using the proposed algorithm.