Current Macroblock Research Articles

Lossy and Lossless Video Frame Compression: A Novel Approach for High-Temporal Video Data Analytics

The smart city concept has attracted high research attention in recent years within diverse application domains, such as crime suspect identification, border security, transportation, aerospace, and so on. Specific focus has been on increased automation using data driven approaches, while leveraging remote sensing and real-time streaming of heterogenous data from various resources, including unmanned aerial vehicles, surveillance cameras, and low-earth-orbit satellites. One of the core challenges in exploitation of such high temporal data streams, specifically videos, is the trade-off between the quality of video streaming and limited transmission bandwidth. An optimal compromise is needed between video quality and subsequently, recognition and understanding and efficient processing of large amounts of video data. This research proposes a novel unified approach to lossy and lossless video frame compression, which is beneficial for the autonomous processing and enhanced representation of high-resolution video data in various domains. The proposed fast block matching motion estimation technique, namely mean predictive block matching, is based on the principle that general motion in any video frame is usually coherent. This coherent nature of the video frames dictates a high probability of a macroblock having the same direction of motion as the macroblocks surrounding it. The technique employs the partial distortion elimination algorithm to condense the exploration time, where partial summation of the matching distortion between the current macroblock and its contender ones will be used, when the matching distortion surpasses the current lowest error. Experimental results demonstrate the superiority of the proposed approach over state-of-the-art techniques, including the four step search, three step search, diamond search, and new three step search.

Early DIRECT mode decision based on all‐zero block and rate distortion cost for multiview video coding

The exhaustive variable-block-size mode decision can efficiently remove the redundancies among the multiview videos, while it also leads to significant increase of computational complexity in the multiview video coding (MVC) encoder, and the high encoding complexity becomes a bottleneck for the MVC encoder to achieve real-time multimedia applications. To address this bottleneck, many fast mode decision methods have been proposed. However, most of them are only suitable for optimising the encoding complexity of the odd views of the MVC encoder. In this study, based on the property of the all-zero block and rate distortion (RD) cost of the DIRECT mode as well as the correlations between the current macroblock (MB) and its spatial–temporal nearby MBs, an early DIRECT mode decision method is proposed for reducing the encoding complexity of the MVC. Experimental results show that the proposed method achieves 48.25 and 55.64% on average encoding time saving for the even and odd views, respectively, whereas the RD performance degradation is quite acceptable. In summary, the proposed method efficiently reduces the encoding complexity for the MVC encoder.

Complexity reduction in the H.264/AVC using highly adaptive fast mode decision based on macroblock motion activity

The H.264/AVC video coding standard is used in a wide range of applications from video conferencing to high-definition television according to its high compression efficiency. This efficiency is mainly acquired from the newly allowed prediction schemes including variable block modes. However, these schemes require a high complexity to select the optimal mode. Consequently, complexity reduction in the H.264/AVC encoder has recently become a very challenging task in the video compression domain, especially when implementing the encoder in real-time applications. Fast mode decision algorithms play an important role in reducing the overall complexity of the encoder. In this paper, we propose an adaptive fast intermode algorithm based on motion activity, temporal stationarity, and spatial homogeneity. This algorithm predicts the motion activity of the current macroblock from its neighboring blocks and identifies temporal stationary regions and spatially homogeneous regions using adaptive threshold values based on content video features. Extensive experimental work has been done in high profile, and results show that the proposed source-coding algorithm effectively reduces the computational complexity by 53.18% on average compared with the reference software encoder, while maintaining the high-coding efficiency of H.264/AVC by incurring only 0.097 dB in total peak signal-to-noise ratio and 0.228% increment on the total bit rate.

A Selection Mechanism for Multiple Error Concealment Techniques in Scalable Video Coding

To cope with the error problem for scalable video coding, this paper proposes a selection method of error concealment algorithms for whole frame loss in the enhancement layer. By analyzing the relationship between inter-layer and temporal domains, the proposed algorithm decides which error concealment algorithm adopted by H.264/SVC reference software is suitable for concealing the current erroneous macroblock in the enhancement layer. Through the proposed algorithm, different erroneous macroblocks can be properly concealed by different error concealment algorithms by considering the characteristics of each algorithm. Simulation results demonstrate that our proposed algorithm outperforms other error concealment algorithms in terms of PSNR and visual quality measurements.

A fast mode decision algorithm applied in medium-grain quality scalable video coding

To increase the flexibility of bit stream adaptation in the H.264/Scalable Video Coding, Medium-Grain Scalable Video Coding (MGS), a variation of Coarse-Grain Scalable Video Coding (CGS), is included. The encoding structure of MGS is significantly different from that of CGS, and the encoding procedure of MGS is very complex. It is important to reduce encoding complexity without sacrificing MGS coding performance. In this paper, a fast mode decision algorithm is proposed for the enhancement layer of MGS based on its specific coding structure. First, the candidate modes and the coding sequence of the current macro-block (MB) are predicted based on mode correlations and correlation degrees. Next, early termination strategies, including Direct Mode, inter-layer, layer and spatial, and spatial-only, are proposed based on the coding structure of MGS and correlations. Finally, MBs are encoded in the order predicted with the four proposed early termination strategies to improve the coding speed. Experimental results show that the algorithm can achieve an average time saving of up to 85.44% with strong robustness and negligible loss in coding efficiency.

Research on effective coding algorithm for multi-view video

Based on deeply analyzing and studying of multi-view video coding technology in JMVC (Joint Multi-view Video Coding), an effective coding algorithm in views of model decision and fast block searching was put forward in this paper. First, we use different search algorithms for interview prediction and frame prediction, respectively, to reduce search time and improve search efficiency. Second, we analyze the relationship among the model of current macroblock, the model of adjacent coded macroblocks and the corresponding position of reference frame. According to the result we can optimize the model decision algorithm for current coding macroblock, and reduce the search time for refined small block. The improved fast search algorithm is effective to the sequences containing fast motion area, and the improved model decision algorithm is effective to the sequences containing slow motion. Therefore, we combine the two improved algorithms, and apply them to sequences with different characteristics. The experimental results show that the improved algorithms get significant improvement in coding effect. Compared with the original algorithm in JMVC, the improved algorithm can reduce the coding time by 84.5% under the premise of insuring reconstruction video quality and coding overhead.

Efficient fast mode decision using mode complexity for multi-view video coding

The variable block-size motion estimation (ME) and disparity estimation (DE) are adopted in multi-view video coding (MVC) to achieve high coding efficiency. However, much higher computational complexity is also introduced in coding system, which hinders practical application of MVC. An efficient fast mode decision method using mode complexity is proposed to reduce the computational complexity. In the proposed method, mode complexity is firstly computed by using the spatial, temporal and inter-view correlation between the current macroblock (MB) and its neighboring MBs. Based on the observation that direct mode is highly possible to be the optimal mode, mode complexity is always checked in advance whether it is below a predefined threshold for providing an efficient early termination opportunity. If this early termination condition is not met, three mode types for the MBs are classified according to the value of mode complexity, i.e., simple mode, medium mode and complex mode, to speed up the encoding process by reducing the number of the variable block modes required to be checked. Furthermore, for simple and medium mode region, the rate distortion (RD) cost of mode 16×16 in the temporal prediction direction is compared with that of the disparity prediction direction, to determine in advance whether the optimal prediction direction is in the temporal prediction direction or not, for skipping unnecessary disparity estimation. Experimental results show that the proposed method is able to significantly reduce the computational load by 78.79% and the total bit rate by 0.07% on average, while only incurring a negligible loss of PSNR (about 0.04 dB on average), compared with the full mode decision (FMD) in the reference software of MVC.

Fast macroblock encoding algorithm based on rate-distortion activity for multiview video coding

Multiview video coding (MVC) is the appendix H of H.264/AVC, and it requires a great amount of time to compress multiple viewpoints׳ video with complex prediction structures. To reduce the whole computational complexity of MVC, this paper proposes a fast macroblock (MB) encoding algorithm based on rate-distortion (RD) activity, and it includes the fast mode decision and the fast motion/disparity estimation. First, the RD activity type of the current MB is calculated by utilizing the Skip/Direct RD cost and the average RD costs of classified MB modes. Then, through utilizing the RD activity type and RD costs of the estimated modes, the selection of candidate modes, the early decision of Skip/Direct mode, and the reduction of Inter8×8 mode estimation are all presented in the fast mode decision. By using the RD activity type and the correlations of vectors, the selection of search center and the prediction of search range are introduced in the fast motion/disparity estimation. In addition, the proposed algorithm can be applied to temporal and inter-view views as well as anchor and non-anchor frames. An experiment with a wide range of video scenes, camera setups and quantization parameters was implemented, and the results confirmed that the proposed algorithm can reduce the encoding time significantly while maintaining a similar RD performance as the original MVC encoder. Compared to the state-of-the-art algorithms, the proposed algorithm also demonstrated better performances in the various test cases.

An Efficient Hardware-Oriented Algorithm of Spatial Motion Vector Prediction for AVS HD Video Encoder

Motion Vector Prediction (MVP) plays an important role in improving coding efficiency in HEVC, H.264/AVC and AVS video coding standard. MVP is implemented by exploiting redundancy of adjacent-block optimal coding information under the constraint that MVP must be performed in a serial way. The constraint prevents parallel processing and MB pipeline based on LevelC+. In multi-stage pipeline, to some extent, adjacent-block best mode-decision information can hardly be obtained. In this paper, we propose a new hardware-oriented method to improve the coding performance at a cost of few hardware resources. When adjacent block is not available, spatial motion vector prediction (SMVP) for integer motion estimation (IME) and fraction motion estimation (FME) will take the IME best mode information and FME best mode information of left block as best information to derive PMV (Predicted Motion Vector) for current macro-block or block. Experimental results shows that the method we propose can achieve a better performance than the existing methods by 0.1db for the cases with intense movement and a non-degrading performance for flat cases.

Efficient multi-view video coding using inter-view information

Multi-view video coding (MVC) has been extended from H.264/AVC to improve the coding efficiency of multi-view video. This paper proposes a fast mode decision algorithm which can make an early decision on the correct mode partition to solve the issue of the enormous computational complexity. The best modes of the reference views are utilized to determine the complexity of the macroblock (MB) in the current view, the mode candidates needed to be calculated can then be obtained according to the complexity. If the complexity is low or medium, the search range can be reduced. The threshold of the rate-distortion cost for the current MB is calculated using the co-located and neighboring MBs in previously coded view and is utilized as the criterion for early termination. The motion vector difference in the reference view is applied to dynamically adjust the search range in the current MB. Experimental results prove that the proposed algorithm achieves a time saving of 81.05% for a fast TZ search and 87.85% for full search, and still maintains quality performance and bitrate.

Reducing 3D video coding complexity through more efficient disparity estimation

3D video coding for transmission exploits the Disparity Estimation (DE) to remove the inter-view redundancies present within both the texture and the depth map multi-view videos. Good estimation accuracy can be achieved by partitioning the macro-block into smaller subblocks partitions. However, the DE process must be performed on each individual sub-block to determine the optimal mode and their disparity vectors, in terms of ratedistortion efficiency. This vector estimation process is heavy on computational resources, thus, the coding computational cost becomes proportional to the number of search points and the inter-view modes tested during the rate-distortion optimization. In this paper, a solution that exploits the available depth map data, together with the multi-view geometry, is proposed to identify a better DE search area; such that it allows a reduction in its search points. It also exploits the number of different depth levels present within the current macro-block to determine which modes can be used for DE to further reduce its computations. Simulation results demonstrate that this can save up to 95% of the encoding time, with little influence on the coding efficiency of the texture and the depth map multi-view video coding. This makes 3D video coding more practical for any consumer devices, which tend to have limited computational power.

A Fast Algorithm for Motion and Disparity Estimation Based on the Reference Directions of Neighboring Macroblocks

As for JMVC (Joint Multi-view Video Coding), when executing motion and disparity estimation simultaneously, the implicit correlation of reference directions between current and neighboring macroblocks is neglected, and this will lead to useless searching. To reduce this redundancy, we propose an estimation algorithm which sets search ranges properly by exploiting the mentioned correlation. The proposed algorithm first gets the reference directions of macroblocks in the left, top and top right direction of current macroblock, and compares them with current reference direction, then sets search ranges by the comparison results, executes motion and disparity estimation in the new ranges at last. Experimental results show that the proposed algorithm can save 28.98%- 46.30% of coding time without degradation of coding quality comparing to JMVC.

Efficient two-stage early SKIP mode termination for depth video coding

The video plus depth format has been widely-used in multi-view video systems. Therefore, low complexity depth video coding becomes very essential. For that, an efficient two-stage early SKIP mode termination (TESMT) algorithm is proposed in this paper. First, by using texture-depth correlation, our approach first checks whether current macroblock (MB) is motionless or slow-motion based on motion activity of corresponding region in texture video. If so, SKIP mode is selected as optimal mode and mode decision process is early terminated. Otherwise, our approach further checks whether the rate-distortion cost of SKIP mode is below an adaptive threshold, which is derived by exploiting spatial-temporal correlation between current MB and its adjacent MBs in depth video. Experimental results show that proposed algorithm significantly reduces computational complexity while keeping almost the same coding efficiency of depth video and quality of synthesized view, compared with exhaustive mode decision in multi-view video coding.

A Regular VLSI Architecture of Motion Vector Prediction for Multiple-Standard MPEG-Like Video Codec

Motion vector (MV) prediction and residue coding technique is adopted to fully utilize the motion field redundancy in the prevailing video standards, and MV prediction is desired in both video encoder and decoder. The computation burden for MV prediction is not very high. However, there is high irregularity in raw MV prediction algorithm with two-stage and four-level hierarchical tree control flows. It makes efficient VLSI architecture implementation challenging. The high irregularity is mainly derived from the abundant inter prediction modes including variable block size partition and temporal prediction direction, as well as the irregular control flow of the MV prediction algorithm. This paper proposes a highly regular architecture to implement MV prediction for multi-standard video codec. Complex control logic is simplified by regularly table look-up of the control parameters predefined and stored in on-chip tables. The parameters of the current macroblock (MB) and its neighboring blocks are initialized and refreshed in a regular manner. Moreover, pipelining and parallelism are employed in the proposed architecture to improve throughput efficiency and tradeoff between hardware cost and efficiency. Simulation results verify the effectiveness of the proposed design.

Efficient early direct mode decision for multi-view video coding

Multi-view video coding (MVC) uses various prediction modes and exhaustive mode decision to achieve high coding efficiency. However, the introduced heavy computational complexity becomes the bottleneck of the practical application of MVC. For this, an efficient early Direct mode decision for MVC is proposed in this paper. Based on the observation that the Direct mode is highly possible to be the optimal mode, the proposed method first computes the rate distortion (RD) cost of the Direct mode and compares this RD cost value with an adaptive threshold for providing an early termination chance as follows. If this RD cost value is smaller than the adaptive threshold, the Direct mode will be selected as the optimal mode and the checking process of the remaining modes will be skipped; otherwise, all the modes will be checked to select the one with the minimum RD cost as the optimal mode. Note that the above-mentioned adaptive threshold is determined as the median prediction value of a set of thresholds, which are derived by using the spatial, temporal and inter-view correlations between the current macroblock (MB) and its neighboring MBs, respectively. Experimental results have demonstrated that the proposed method is able to significantly reduce the computational complexity of MVC with negligible loss of coding efficiency, compared with the exhaustive mode decision in MVC.

Mode decision acceleration for scalable video coding through coded block pattern

This paper presents a fast mode decision to accelerate the encoding process of scalable video coding (SVC) through coded block pattern (CBP) analysis. CBP contains residual information of each macroblock (MB) and provide cues for mode prediction in encoding process. The proposed algorithm makes use of the existing mode information in base layer (BL), enhancement layer (EL) and CBP to remove invalid modes for the mode prediction of the current MB in EL coding. Also, the EL’s modes in the highest temporal level of a group of picture are determined by high correlation exists in adjacent frames. Experimental results show that the proposed algorithm saves an average of 68% of EL motion estimation time with minor degradation in PSNR and with few bitrate increase of SVC quality and spatial scalabilities when compared to JSVM 9.19.14. Our algorithm outperforms the best-known method by 17% EL motion estimation time saving in average.

Advanced error concealment for 1Seg video communication considering error propagation inside a frame

AbstractTransmission of compressed video over error‐prone channels may result in packet losses or errors, which can significantly degrade the image quality. Such degradation becomes even worse in 1Seg video broadcasting, which is widely used in Japan and Brazil for mobile phone TV services recently, where errors are drastically increased and the lost areas are contiguous. Therefore, the errors in the earlier concealed MBs (macro blocks) may propagate to the MBs later to be concealed inside the same frame (spatial domain). The error concealment (EC) is used to recover the lost data by the redundancy in videos. Aiming at spatial error propagation (SEP) reduction, this paper proposes an SEP‐reduction‐based EC (SEPEC). In SEPEC, besides the mismatch distortion in the current MB, the potential propagated mismatch distortion in the following to be concealed MBs is also minimized. Also, two extensions of SEPEC, i.e. SEPEC with refined search and SEPEC with multiple layer match, are discussed. Compared with previous work, the experiments show SEPEC achieves much better performance of video recovery and excellent trade‐off between quality and computation in 1Seg broadcasting. © 2012 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Implicit Line-Based Intra 16×16 Prediction for H.264/AVC High-Quality Video Coding

In this paper, we propose an implicit line-based linear intra 16×16 prediction method for high-quality video coding. Typically, in H.264/AVC intra 16×16 vertical and horizontal prediction modes, 256 pixels in the current macroblock are predicted using 16 adjacent boundary pixels of reconstructed neighboring macroblocks. One problem of such a block-based intra prediction is that the prediction error increases as the distance between the reference and current pixels increases. Thus, the prediction accuracy of intra 16×16 vertical and horizontal modes is not sufficient for the mode decision stage. To ensure that the pixels are close to their predictor, we propose a new implicit intra prediction scheme, which makes full use of the correlation between lines instead of blocks. In the proposed method, we perform prediction, transformation, quantization, and reconstruction using the line-of-pixels (LOP) to improve the prediction accuracy. Experimental results show that the proposed algorithm provides an approximately 6.42 % bit-rate reduction compared to the H.264/AVC FRExt high profile, while maintaining the same decoding quality.

Motion vector extrapolation for parallel motion estimation on GPU

The powerful parallel computing ability of Graphics Processing Unit (GPU) has shown its striking superiority for motion estimation acceleration in conventional hybrid video encoding process. Unfortunately, the motion information of the neighboring macroblocks is not available for current macroblock, such that parallel motion estimation using GPU is not very favored. To tackle this problem while achieving high acceleration ration, motion vector cost is always ignored in most existing solutions, which inevitably causes severe rate-distortion loss. In this paper, a novel motion vector extrapolation based approach (MVEA) is presented for enhancing rate-distortion performance of parallel motion estimation on GPU, which is based on the study of motion vector recovery strategies for frame loss error concealment. Furthermore, the efficient implementation of MVEA on Computing Unified Device Architecture (CUDA) is also investigated. Simulation results show that MVEA can achieve a maximum peak Signal-to-Noise ratio enhancement of 0.8 dB with ignorable computational cost increase.

A Noise Robust Mode Decision Algorithm Based on Macroblock-level Motion Detection for H.264/AVC

When video noise increases, the coding efficiency of the H.264/AVC considerably decreases. Therefore, we propose a novel mode decision algorithm based on early macroblock-level motion detection. According to the macroblock motion levels, four different mode decision strategies are chosen for current macroblock coding. Experimental results show that the proposed algorithm can significantly reduce the encoding time with similar coding efficiency when weighed up against the H.264/AVC.