Adjacent Macroblocks Research Articles

Computationally Efficient Intra and Inter Mode Decision in H.264/AVC

Abstract The variable block size mode decision employed in H.264/AVC improves its coding efficiency, which also increases the computational complexity when rate distortion optimization (RDO) is used. To reduce the computational complexity of H.264/AVC a fast intra and inter mode decision technique is implemented. The fast intra mode decision is achieved by eliminating the less likely prediction modes from the rate distortion computation, for which a method based on directions of prediction and the Sum of Absolute Transformed Difference (SATD) is used for mode selection. The fast inter mode decision algorithm employs early skip mode detection, adaptive thresholds and the correlation of adjacent macroblocks to reduce the computational complexity. The experimental results show that the proposed method can reduce the coding time by 50.73% and thereby improve the real time performance, with slight video quality degradation of 0.13dB and bit-rate increment of 1.13%.

Spatial Error Concealment Algorithm Based on Adaptive Edge Threshold and Directional Weight

In order to improve the H.264/AVC compressed video stream error resilience in wireless channel transmission, this paper presents a spatial error concealment algorithm based on adaptive edge threshold and directional weight. Firstly, this algorithm makes use of Sobel gradient operator of image edge detection to detect the edge of adjacent macro blocks; secondly, according to specific information of adjacent macro-block of damaged macro-block, it can set gradient adaptive threshold; thirdly, it makes the direction weighted interpolation to damaged macro-block with the Sobel gradient operator of image edge detection. Experiments show that the image reconstruction quality is greatly improved by using this algorithm, which has higher application value for different video sequence as compared to the traditional spatial error concealment algorithms. This algorithm not only improves the quality of image restoration, but also has higher application value.

Motion vector recovery for video error concealment based on the plane fitting

In a wireless transmission, the compressed video stream can result in lower quality of image reconstruction with the motion vector mismatch. A motion vector recovery for video error concealment based on the plane fitting algorithm is proposed. According to the characteristics of the H.264 video standard, the adjacent macro blocks and the multi reference frame have the correlation. The motion vector which surrounds the lost macro-block in each divided block is projected by one plane which adopts one point to present the MV changing tendency in small regions. This algorithm reconstructs the motion vector of the damaged macro-block by the fitting plane algorithm based on the improved boundary matching algorithm, and then the optimal motion vector is selected to reconstruct the damaged images. This algorithm not only avoids the block effects that caused by the original algorithm, but also in different probability of RTP packet losing, the peak signal to noise ratio of the algorithm has improved 0.3 ~ 2.5 dB compared to the adaptive intra-frame error concealment algorithm .

Video temporal error concealment using improved directional boundary matching algorithm

Nowadays some systems such as multimedia systems try to present a high quality of digital videos every day. Because of the possible errors in communication channels, compressed video data would be damaged in the sending process. Error concealment is a useful technique for concealing the effects of sending errors at the decoder. In this paper, an improved directional boundary matching algorithm is presented, in which by using adjacent macroblocks (MBs) of the damaged MB a direction is identified for each boundary. Then every boundary of candidate MBs is compared to an identified direction. Finally, the candidate motion vector (MV) that has the minimum improved directional matching function is selected as the MV of the damaged MB. Furthermore, to increase the accuracy in damaged MV estimation, a specific weight is given to the boundaries of adjacent MBs. Conforming to the experimental results, the proposed algorithm not only outperforms subjective visual evaluation compared to classic boundary matching, outer boundary matching, and directional boundary matching algorithms, but also it is able to improve the objective quality of reconstructed video frames effectively.

A novel Cooperative Motion Estimation Algorithm based on Particle Swarm Optimization and its multicore implementation

A cooperative motion estimation (ME) scheme using a modified Particle Swarm Optimization (PSO) algorithm is presented. The proposed algorithm is based on a multi-swarm PSO model where a swarm of PSO particles is defined for each macroblock (MB) in the frame. Motion estimation is then performed in a cooperative manner concurrently for all the MBs in the frame. Cooperation between neighboring MBs during the motion estimation process is allowed through a communication step to exchange information about the motion vectors found so far in the estimation process. This synergic relationship between the swarms of adjacent MBs allows refining the motion search and leads to both a faster convergence of the PSO process and an improvement in the resulting motion vectors. Several techniques are also proposed to improve the search capacity and computational complexity of the PSO iterations. A novel PSO initialization scheme that exploits the existing temporal correlation is proposed to remove dependency between adjacent MBs. A fitness function history preservation mechanism is also presented to prevent redundant repeated calculations of the fitness function of a given search point by the PSO particles which dramatically decreases the computational complexity. Moreover, the maximum allowed velocity of the particles is adaptively varied during the PSO iterative process which provides a balance between search exploration and exploitation. The proposed scheme exhibits a high level of data parallelism since it is capable of performing motion estimation for all the MBs of the frame in parallel rather than serially. As a result, the presented algorithm is amenable to parallel processing techniques. In this paper, a multicore implementation of our proposed algorithm is performed using the MATLAB® Parallel Computing Toolbox™ (PCT). Extensive simulations are performed to analyze the performance of the presented algorithm. It is found that the presented scheme provides improvements in terms of accuracy and computational complexity as compared to conventional fast motion estimation techniques and two state-of-the-art PSO-based ME schemes. An analysis of the parallel performance shows that the presented scheme is highly scalable and that the parallel efficiency increases with the increase in video resolution. The multicore implementation of the proposed algorithm using MATLAB could achieve a speedup of 6.21 on eight CPU cores for high-definition (HD) video sequences. The multicore performance of the proposed scheme is also compared with existing parallel algorithms in the literature and is shown to give superior results.

Efficient two-stage early SKIP mode termination for depth video coding

The video plus depth format has been widely-used in multi-view video systems. Therefore, low complexity depth video coding becomes very essential. For that, an efficient two-stage early SKIP mode termination (TESMT) algorithm is proposed in this paper. First, by using texture-depth correlation, our approach first checks whether current macroblock (MB) is motionless or slow-motion based on motion activity of corresponding region in texture video. If so, SKIP mode is selected as optimal mode and mode decision process is early terminated. Otherwise, our approach further checks whether the rate-distortion cost of SKIP mode is below an adaptive threshold, which is derived by exploiting spatial-temporal correlation between current MB and its adjacent MBs in depth video. Experimental results show that proposed algorithm significantly reduces computational complexity while keeping almost the same coding efficiency of depth video and quality of synthesized view, compared with exhaustive mode decision in multi-view video coding.

Fast Mode Decision for Multiview Video Coding Using Mode Correlation

Exhaustive mode decision has been exploited in multiview video coding for effectively improving the coding efficiency, but at the expense of yielding much higher computational complexity. In this paper, a fast mode decision algorithm, called the mode correlation-based mode decision (MCMD), is proposed to speed up the encoding process by reducing the number of the modes required to be checked. In our approach, all the prediction modes are first categorized into five motion-activity classes, and only one of them will be chosen to identify the optimal mode in a hierarchical manner, as follows. For each macroblock (MB), the proposed MCMD algorithm always begins with checking whether the rate-distortion cost computed at the SKIP mode (i.e., Class 1) is below an adaptive threshold for providing a possible early termination chance. If this early termination condition is not met, one of the remaining four motion-activity classes will be chosen for further mode checking according to the analysis of the predicted motion vector (PMV) of the current MB. The above-mentioned adaptive threshold and PMV are derived by exploiting the mode correlation between the current MB and a set of adjacent MBs (i.e., region of support) in the current view and its neighboring view. Experimental results have shown that compared with exhaustive mode decision, which is a default approach set in the joint multiview video model (JMVM) reference software, the proposed MCMD algorithm achieves a reduction of the computational complexity by 73.39% on average, while incurring only 0.07 dB loss in peak signal-to-noise ratio (PSNR) and 2.22% increment on the total bit rate.

A Fast Normalized Cross Correlation-Based Block Matching Algorithm Using Multilevel Cauchy-Schwartz Inequality

This paper presents a fast block-matching algorithm based on the normalized cross-correlation, where the elimination order is determined based on the gradient magnitudes of subblocks in the current macroblock. Multilevel Cauchy-Schwartz inequality is derived to skip unnecessary block-matching calculations in the proposed algorithm. Also, additional complexity reduction is achieved re-using the normalized cross correlation values for the spatially neighboring macroblock because the search areas of adjacent macroblocks are overlapped. Simulation results show that the proposed algorithm can improve the speed-up ratio up to about 3 times in comparison with the existing algorithm.

Design and Memory Optimization of Motion Estimation Module for H.264/AVC Encoder

H.264/AVC standard has been widely used in video compression at various kinds of application domain. Motion estimation takes the most calculation workload of H.264/AVC encoder. Memory optimization has played an even more important role in encoder design. Firstly, dependency relation between motion vectors was analyzed and removed at a little cost of estimation accuracy decrement, and then a 3-stage macro-block level pipeline architecture was proposed to increase parallel process ability of motion estimation. Then an optimized memory organization strategy of reference frame data was put forward, aiming at avoiding row changing frequently in SDRAM access. Finally, based on the 3-stage pipeline structure, a shared cyclic search window memory was proposed: 1) data relativity between adjacent macro-block was analyzed, 2) and search window memory size was elaborated, 3) and then a slice based structure and the work process were discussed. Analysis and experiment result show that 50% of on chip memory resource and cycles for off chip SDRAM access can be saved. The whole design was implemented with Verilog HDL and integrated into a H.264 encoder, which can demo 1280*720@30 video successfully at frequency of 120MHz under a cyclone III FPGA development board.

Inter mode decision of complexity scalability

In order to resolve the problem that computation complexity of H.264/AVC can not satisfy the diversity of computing resources of terminal equipment,the authors proposed a novel algorithm of inter mode decision.The algorithm utilized the spatial correlation and time correlation of adjacent macroblocks to predict the modes of current macroblocks,and controlled the number of modes involved in the prediction by scalability factor;hence the complexity can change flexibly from 20% to 100%.The experimental results show that the proposed method can achieve the tradeoff between the accuracy of motion estimation and its computational complexity,fit for the different computing power of different equipments.

On parallelizing H.264/AVC rate-distortion optimization baseline profile encoder

A H.264/AVC encoder can incorporate many coding schemes, such as rate-distortion optimization (RDO), into its design to improve its compression performance, dramatically raising computational complexity. With the H.264/AVC RDO encoder, computation time is primarily spent calculating the rate-distortion cost in choosing the optimal coding mode for both inter and intra coding modes. Parallel computation is one of the ways to speed up the encoder. However, calculating rate-distortion costs requires a great amount of reference data obtained from coded adjacent macroblocks in order to maintain the coding efficiency established by the JM encoder. This is an undesirable property for any parallel computing strategy. The transmission of such a large amount of reference data, as well as the frequency of transmission between processing nodes, reduces the speed of the entire encoding process. Thus, it may become necessary to drop part of the reference data and decrease the frequency of transmission in order to reduce the traffic. In the investigation of this problem, this study uses three different parallel schemes for the implementation of the H.264/AVC RDO encoder. These schemes are each run over a software DSM-based (distributed shared memory) PC cluster system consisting of 1 to 5 PC computers (one master node, with or without one to several slave processing nodes). The amount of data to be exchanged among processing nodes is analyzed for each scheme. In addition, the PSNR performance and the number of speedup results are provided for each scheme. Experiments show that considerable reduction in coding gain is expected, as more information is dropped. In lower bit rate cases, performance is reduced to the level of a regular H.264 encoder. Nevertheless, this paper provides a good reference for implementing such an encoder utilizing a cluster computing system.

MPEG-2 Transcoding Method for Reducing Re-quantization Noise Based on a Two-tiered Quantizer Matrix

We developed an MPEG-2 transcoding method on the basis of a two-tiered quantizer matrix that reduces re-quantization noise. The proposed method changes the quantization matrix to decrease the re-quantization noise, instead of changing the value of the quantization parameters as conventional methods do. Using this two-tiered matrix makes it possible to reduce re-quantization noise in the high-frequency domain. The main feature of the proposed method is that it does not cause any differences in image quality between adjacent macroblocks, which the conventional method does. Experimental results show that our proposed method produces transcoding scores of structural similarity (SSIM) and video quality model (VQM) that are 0.044 and 0.0938, respectively, better than those of the best conventional method. The proposed method always produced superior SSIM and VQM scores when the bit reduction ratio was smaller than 0.1015 for all the tested sequences.

Fast Mode Decision for H.264/AVC Based on Macroblock Motion Activity

The intra-mode and inter-mode predictions have been made available in H.264/AVC for effectively improving coding efficiency. However, exhaustively checking for all the prediction modes for identifying the best one (commonly referred to as exhaustive mode decision) greatly increases computational complexity. In this paper, a fast mode decision algorithm, called the motion activity-based mode decision (MAMD), is proposed to speed up the encoding process by reducing the number of modes required to be checked in a hierarchical manner, and is as follows. For each macroblock, the proposed MAMD algorithm always starts with checking the rate-distortion (RD) cost computed at the SKIP mode for a possible early termination, once the RD cost value is below a predetermined ldquolowrdquo threshold. On the other hand, if the RD cost exceeds another ldquohighrdquo threshold, then this indicates that only the intra modes are worthwhile to be checked. If the computed RD cost falls between the above-mentioned two thresholds, the remaining seven modes, which are classified into three motion activity classes in our work, will be examined, and only one of the three classes will be chosen for further mode checking. The above-mentioned motion activity can be quantitatively measured, which is equal to the maximum city-block length of the motion vector taken from a set of adjacent macroblocks (i.e., region of support, ROS). This measurement is then used to determine the most possible motion-activity class for the current macroblock. Experimental results have shown that, on average, the proposed MAMD algorithm reduces the computational complexity by 62.96%, while incurring only 0.059 dB loss in PSNR (peak signal-to-noise ratio) and 0.19% increment on the total bit rate compared to that of exhaustive mode decision, which is a default approach set in the JM reference software.

Low complexity intra MB encoding in AVC/H.264

In this paper we introduce and evaluate a novel machine learning based approach to reduce the complexity of Intra macroblock (MB) coding. The proposed approach is based on the hypothesis that MB coding mode decisions in H.264/AVC video have a correlation with the intensities of adjacent MBs and sub-MBs. This paper also discusses and analyzes different approaches of using machine learning in Intra prediction. We discuss, amongst other features, slices, Intra prediction scheme for H.264 and data mining. We use data mining algorithms to develop decision trees for H.264 coding mode decisions. The proposed approach reduces the H.264/AVC MB mode computation process into a decision tree lookup with very low complexity. The proposed algorithm is implemented in reference software by modifying the source code and is compared with the JM reference software for H.264/AVC.

Layered Motion Compensation for Moving Image Compression

Block-based motion compensation began with MPEG-1 and H.261 and has continued into MPEG-2, MPEG-4_part 2, MPEG-4_AVC_part 10, H.264, and SMPTE VC1. The general principle used by these conventional motion compensation systems is that there is one motion vector for a square or rectangular group of pixels. The group of pixels associated with the motion vector is generally referred as a “macroblock.” It is common in MPEG and other compression systems to use reduced resolution for Red-Y (U) and Blue-Y (V) compared to Y (luminance), where Y usually equals approximately .59Green + .29Red + .12Blue. Intra-coded frames and the motion-compensated-difference frames use a common quantized discrete cosine transform (DCT). The macroblock structure provides a common boundary for the Y, U, and V DCT-coded regions. The DCT size is usually 8 × 8 pixels, but may be anywhere from 4 × 4 up to 16 × 16 (e.g., in MPEG-4 AVC_part 10 and H.264). The nature of the DCT is that it is a self-contained regional transform that does not extend to any pixel outside its block (usually 8 × 8). In a sense, quantization errors in the DCT wrap from one edge to the other (i.e., left to right, and top to bottom) of the DCT block. Differing quantization errors in adjacent DCT blocks yield block edge discontinuities. Thus, because macroblock boundaries coincide with DCT block boundaries, they share a common edge. When a motion vector differs between adjacent macroblocks, the inherent edge discontinuity does not appear within the DCT block, but rather at its edge. The DCT block edge “wraps around” to see its opposite edges, so the DCT is quite tolerant of the edges inherent in macroblock-based motion compensation.

MPEG-2 Transcoding Method for Reducing Re-quantization Noise Based on a Two-tiered Quantizer Matrix

We developed an MPEG-2 transcoding method on the basis of a two-tiered quantizer matrix that reduces re-quantization noise. The proposed method changes the quantization matrix to decrease the re-quantization noise, instead of changing the value of the quantization parameters as conventional methods do. Using this two-tiered matrix makes it possible to reduce re-quantization noise in the high-frequency domain. The main feature of the proposed method is that it does not cause any differences in image quality between adjacent macroblocks, which the conventional method does. Experimental results show that our proposed method produces transcoding scores of structural similarity (SSIM) and video quality model (VQM) that are 0.044 and 0.0938, respectively, better than those of the best conventional method. The proposed method always produced superior SSIM and VQM scores when the bit reduction ratio was smaller than 0.1015 for all the tested sequences.

Fast motion estimation algorithm using motion adaptive search

Most existing video compression standards use block-matching motion estimation to exploit temporal correlation between frames. To reduce the costly computation of exhaustive searches on all possible motion displacements, researchers have developed many fast algorithms. Some of these successfully proposed algorithms take advantage of the motion correlations between adjacent macroblocks. On the basis of the idea of second-order motion correlation between macroblocks in particular, we developed a new algorithm that applies a set of adaptive search patterns to benefit the statistical model of motions. In addition, adaptive early termination rules are used to prevent the waste of unnecessary computation. Simulation results show that the proposed algorithm outperforms most other existing algorithms in the areas of speed and visual quality.

A Fast Multi-Resolution Block Matching Algorithm for Multiple-Frame Motion Estimation

In this paper, we propose a fast multi-resolution block matching algorithm with three resolution levels (upper, middle, and lower levels) for multiple-frame motion estimation (MFME). The main concept of the algorithm is to perform a fast search while maintaining a PSNR performance similar to a full search block matching algorithm (FSBMA). The algorithm combines motion vector prediction using the spatial correlation of motion vectors and a multiple candidate search based on a multi-resolution search. To further reduce the computational complexity, we propose two temporal reduction schemes. To reduce the number of previous reference frames to be processed, the first scheme is applied to the upper level by using the information obtained from the search results of the spatio-temporally adjacent macroblocks (MBs) and the result from the current MB in the middle level of the first reference frame. The other scheme is applied to the lower level by using statistical information. Experimental results show that the proposed algorithm guarantees an average PSNR loss of less than 0.23 dB with dramatically reduced computational complexity as compared to the FSBMA. In particular, for sequences with fast motion or frame skipping, the proposed method provides a more prominent PSNR performance than those of existing fast schemes with a comparable computational complexity.

A perceptual two-pass VBR MPEG-2 video encoder

This paper presents a perceptual two-pass VBR MPEG-2 video encoder suitable for digital storage applications. The proposed algorithm aims to achieve perceptually uniform distortion among adjacent macroblocks within a frame and smoothen the inter-frame visual quality fluctuation as soon as possible. In the first-pass coding, we analyze the video sequence and collect some useful coding parameters to accurately measure the frame coding complexity. In the second-pass coding, the target bits are allocated to all frames optimally according to the coding complexity, certainly with decoder buffer considered. The human visual system characteristics including both the spatial and temporal masking effects are employed to improve the proposed adaptive quantization algorithm. Furthermore, an improved iteration based Qp selection algorithm is proposed to search for the optimal frame-level reference Qp resulting in satisfactory iteration speed and iteration precision. Experimental results show that the proposed algorithm can achieve superior visual quality both objectively and subjectively compared with the reference algorithms.

Direct mode coding for bipredictive slices in the H.264 standard

The new H.264 (MPEG-4 AVC) video coding standard can achieve considerably higher coding efficiency compared to previous standards. This is accomplished mainly due to the consideration of variable block sizes for motion compensation, multiple reference frames, intra prediction, but also due to better exploitation of the spatiotemporal correlation that may exist between adjacent Macroblocks, with the SKIP mode in predictive (P) slices and the two DIRECT modes in bipredictive (B) slices. These modes, when signaled, could in effect represent the motion of a macroblock (MB) or block without having to transmit any additional motion information required by other inter-MB types. This property also allows these modes to be highly compressible especially due to the consideration of run length coding strategies. Although spatial correlation of motion vectors from adjacent MBs is used for SKIP mode to predict its motion parameters, until recently, DIRECT mode considered only temporal correlation of adjacent pictures. In this letter, we introduce alternative methods for the generation of the motion information for the DIRECT mode using spatial or combined spatiotemporal correlation. Considering that temporal correlation requires that the motion and timestamp information from previous pictures are available in both the encoder and decoder, it is shown that our spatial-only method can reduce or eliminate such requirements while, at the same time, achieving similar performance. The combined methods, on the other hand, by jointly exploiting spatial and temporal correlation either at the MB or slice/picture level, can achieve even higher coding efficiency. Finally, improvements on the existing Rate Distortion Optimization related to B slices within the H.264 codec are also presented, which can lead to improvements of up to 16% in bit rate reduction or, equivalently, more than 0.7 dB in PSNR.