Macroblock Research Articles

A fast mode decision algorithm applied in medium-grain quality scalable video coding

To increase the flexibility of bit stream adaptation in the H.264/Scalable Video Coding, Medium-Grain Scalable Video Coding (MGS), a variation of Coarse-Grain Scalable Video Coding (CGS), is included. The encoding structure of MGS is significantly different from that of CGS, and the encoding procedure of MGS is very complex. It is important to reduce encoding complexity without sacrificing MGS coding performance. In this paper, a fast mode decision algorithm is proposed for the enhancement layer of MGS based on its specific coding structure. First, the candidate modes and the coding sequence of the current macro-block (MB) are predicted based on mode correlations and correlation degrees. Next, early termination strategies, including Direct Mode, inter-layer, layer and spatial, and spatial-only, are proposed based on the coding structure of MGS and correlations. Finally, MBs are encoded in the order predicted with the four proposed early termination strategies to improve the coding speed. Experimental results show that the algorithm can achieve an average time saving of up to 85.44% with strong robustness and negligible loss in coding efficiency.

A novel Cooperative Motion Estimation Algorithm based on Particle Swarm Optimization and its multicore implementation

A cooperative motion estimation (ME) scheme using a modified Particle Swarm Optimization (PSO) algorithm is presented. The proposed algorithm is based on a multi-swarm PSO model where a swarm of PSO particles is defined for each macroblock (MB) in the frame. Motion estimation is then performed in a cooperative manner concurrently for all the MBs in the frame. Cooperation between neighboring MBs during the motion estimation process is allowed through a communication step to exchange information about the motion vectors found so far in the estimation process. This synergic relationship between the swarms of adjacent MBs allows refining the motion search and leads to both a faster convergence of the PSO process and an improvement in the resulting motion vectors. Several techniques are also proposed to improve the search capacity and computational complexity of the PSO iterations. A novel PSO initialization scheme that exploits the existing temporal correlation is proposed to remove dependency between adjacent MBs. A fitness function history preservation mechanism is also presented to prevent redundant repeated calculations of the fitness function of a given search point by the PSO particles which dramatically decreases the computational complexity. Moreover, the maximum allowed velocity of the particles is adaptively varied during the PSO iterative process which provides a balance between search exploration and exploitation. The proposed scheme exhibits a high level of data parallelism since it is capable of performing motion estimation for all the MBs of the frame in parallel rather than serially. As a result, the presented algorithm is amenable to parallel processing techniques. In this paper, a multicore implementation of our proposed algorithm is performed using the MATLAB® Parallel Computing Toolbox™ (PCT). Extensive simulations are performed to analyze the performance of the presented algorithm. It is found that the presented scheme provides improvements in terms of accuracy and computational complexity as compared to conventional fast motion estimation techniques and two state-of-the-art PSO-based ME schemes. An analysis of the parallel performance shows that the presented scheme is highly scalable and that the parallel efficiency increases with the increase in video resolution. The multicore implementation of the proposed algorithm using MATLAB could achieve a speedup of 6.21 on eight CPU cores for high-definition (HD) video sequences. The multicore performance of the proposed scheme is also compared with existing parallel algorithms in the literature and is shown to give superior results.

Intra Prediction Mode Decision for H.264

Digital video compression has become an integral part of the way we create, consume visual information and communicate, over the last few decades. A robust technique of compression is proposed in this paper for video compression. Reduction of irrelevant or redundant data in order to save the storage space requirements and processing time is nothing but compression. Here H.264/AVC (Advance Video Coding) video coding standard is used for compression. Where I-frames are divided into different macro blocks (MB) and each MB is efficiently compressed using 4 x4 and 16 x16 blocks in H.264/AVC intra prediction. Choosing one of 9 prediction modes for each 4 x4 block with reduced time and less complexity is still a bottleneck. In this paper a gradient based fast intra prediction mode selection method for 4x4 and 16x16 is proposed. The proposed method divides an input frames into variable block sizes based on the texture and performs few mode examinations based on the gradient direction in the given slice. Respective best mode with minimum cost is selected using sum of absolute difference (SAD) and rate distortion optimization (RDO). This process is carried for all the video input frames. Each compressed video frames are combined finally to get a compressed video output.

Adaptive mode decision for multiview video coding based on macroblock position constraint model

Multiview video coding (MVC) exploits mode decision, motion estimation and disparity estimation to achieve high compression ratio, which results in an extensive computational complexity. This paper presents an efficient mode decision approach for MVC using a macroblock (MB) position constraint model (MPCM). The proposed approach reduces the number of candidate modes by utilizing the mode correlation and rate distortion cost (RD cost) in the previously encoded frames/views. Specifically, the mode correlations both in the temporal-spatial domain and the inter-view are modeled with MPCM. Then, MPCM is exploited to select the optimal prediction direction for the current encoding MB. Finally, the inter mode is early determined in the optimal prediction direction. Experimental results show that the proposed method can save 86.03 % of encoding time compared with the exhaustive mode decision used in the reference software of joint multiview video coding, with only 0.077 dB loss in Bjontegaard delta peak signal-to-noise ratio (BDPSNR) and 2.29 % increment of the total Bjontegaard delta bit rate (BDBR), which is superior to the performances of state-of-the-art approaches.

Switching error concealment algorithm based on optimal decisions for performance and complexity

The proposed paper discusses two switching error concealment algorithms: SECA (Switching Error Concealment Algorithm) and OSECA (Optimal Switching Error Concealment Algorithm) to efficiently switch among three existing error concealment methods for optimal performance and complexity tradeoff. SECA uses the motion vector statistics in the neighborhood of the lost MB (Macro Block) to make the switching decision. SECA is better than the best-PSNR method by 0.14 dB in PSNR and better than the best-win-rate method by 1.49 % in win rate, with 63 % time complexity reduction. OSECA further considers the correlation between surrounding pixels of the lost MB and the pixels in the reference frame. An optimization problem is formulated to derive the optimal decision thresholds. For OSECA, the PSNR performance is further improved by 0.24 dB from SECA and the time complexity reduction from the existing work can increase to 69 %. On different testing sets of the lost MBs, the proposed SECA and OSECA perform consistently against existing methods. Compared to SECA in the testing sets, OSECA has larger PSNR improvement (by up to 0.23 dB) against the best-PSNR method, and has larger time complexity reduction (by up to 71 %) against existing method.

Pixel-level Matching Based Multi-hypothesis Error Concealment Modes for Wireless 3D H.264/MVC Communication

3D multi-view video (MVV) is multiple video streams shot by several cameras around a single scene simultaneously. Therefore it is an urgent task to achieve high 3D MVV compression to meet future bandwidth constraints while maintaining a high reception quality. 3D MVV coded bit-streams that are transmitted over wireless network can suffer from error propagation in the space, time and view domains. Error concealment (EC) algorithms have the advantage of improving the received 3D video quality without any modifications in the transmission rate or in the encoder hardware or software. To improve the quality of reconstructed 3D MVV, we propose an efficient adaptive EC algorithm with multi-hypothesis modes to conceal the erroneous Macro-Blocks (MBs) of intra-coded and inter-coded frames by exploiting the spatial, temporal and inter-view correlations between frames and views. Our proposed algorithm adapts to 3D MVV motion features and to the error locations. The lost MBs are optimally recovered by utilizing motion and disparity matching between frames and views on pixel-by-pixel matching basis. Our simulation results show that the proposed adaptive multi-hypothesis EC algorithm can significantly improve the objective and subjective 3D MVV quality.

Human centered perceptual adaptation for video coding

Traditional visual saliency based video compression methods try to encode the image with higher quality in the region of saliency. However, the saliency feature changes according to persons, viewpoints, and distances. In this paper, we propose to apply a technique of human centered perceptual computation to improve video coding in the region of human centered perception. To detect the region of interest (ROI) of human body, upper body, frontal face, and profile face, we construct Harr and histogram of oriented gradients features based combo of detectors to analyze a video in the first frame (intra-frame). From the second frame (inter-frame) onward, the optical flow image is computed in the ROI area of the first frame. The optical flow in human centered ROI is then used for macroblock (MB) quantization adjustment in H.264/AVC. For each MB, the quantization parameter (QP) is optimized with density value of optical flow image. The QP optimization process is based on a MB mapping model, which can be calculated by an inverse of the inverse tangent function. The Lagrange multiplier in the rate distortion optimization is also adapted so that the MB distortion at human centered region is minimized. We apply our technique to the H.264 video encoder to improve coding visual quality. By evaluating our scheme with the H.264 reference software, our results show that the proposed algorithm can improve the visual quality of ROI by about 1.01 dB while preserving coding efficiency.

A Practical Video Fragment Identification System

In this paper, we present a practical identification approach of video fragment for digital video files. Before analyzing the video content, we must decode it based on its encoding format first. In order to effectively identify the format of a fragment, a format classification is performed before the format identification. The methods of format classification and identification are discriminative subspace clustering (DiSC) and the Knearest neighbor (KNN).Because of losing the meta-information, we add a maximum similar header (MSH) to the front of the fragment to recover the video content. We adopt a simple key frame detection method using standard deviation and mean value. Motion vectors of macro blocks are utilized to classify the video features for effectively identifying the video. Several edges of frames are accumulated and compose a video feature. The experimental results show the evaluations of the video format classification and identification, fragment recovery, and content identification.

Fast Intra prediction algorithm for quality scalable video coding

The coding efficiency of currently scalable video coding is very high at the cost of high coding complexity. Therefore, it is extremely important to improve the coding speed. By making use of the advantage of the quality scalable video coding (QSVC) structure, we propose a new fast Intra coding algorithm for QSVC. Based on the coding structure of QSVC, temporal, spatial and inter-layer correlations are employed together to predict candidate macro-block (MB) modes and their encoding orders to exclude low possible modes, and then three new early termination methods are proposed. In addition to MB modes, temporal, spatial and inter-layer correlations are also adopted together to estimate weight values and orders of all directional modes. Experimental results demonstrate that the coding speed of the proposed algorithm can be improved by 63.58 % in average with negligible coding loss.

Super-resolution with adaptive macro-block topology applied to a multi-camera system

Super-Resolution (SR) consists in processing an image or a set of images in order to enhance the resolution of a video sequence or a single frame. In this paper, fusion SR techniques are considered, where High-Resolution (HR) images are constructed from several observed Low-Resolution (LR) images, thereby increasing the high-frequency components and removing the degradations caused by the recording process of LR imaging acquisition devices. This paper follows a strategy combining the selection of the most appropriate frames and adaptive sized Macro-Blocks (MBs) together with a Multi-Camera (MC) system. This proposal optimizes the spatial and temporal correlations between the recorded sequences, and minimizes the appearance of annoying artifacts at the same time, improving the quality of the super-resolved HR sequence and reducing the computational cost by more than a factor of two. This type of image enhancement systems has many applications in Consumer Electronics appliances related to imaging. More specifically, many camera suppliers are incorporating Super-Resolution in their high-end products.

A Video Watermarking Algorithm based on Motion Vector

This paper proposes a new fragile video watermarking algorithm based on motion vector, we first use the frame difference method to detect moving targets, determine the motion macro blocks. Then in the macroblock by the diamond search, the fragile watermark is embedded in the P-frame motion vectors for inter prediction, so that by changing the mapping of pixel precision motion vector is to be embedded watermark information embedding watermark information. Experimental results show little effect on the video quality of the algorithm.

An Efficient Block Mode Detection Algorithm for Scalable Video Coding using Probability Model

H.264 / AVC expansion is H.264 / SVC which is applicable in environments that demand video streaming. This paper delivers an algorithm to shorten computational complexity and extend coding efficiency by determining the mode speedily. In this writing, the authors talk a fast mode resolution algorithm with less complexity unlikely the traditional joint scalable video model (JSVM). Their algorithm end mode hunt by a probability model defined. This model is address for both intra-mode and inter-mode predictions of base layer and enhancement layers in a macro block (MB). The estimated rate distortion cost (RDC) for modes among layers is custom to determine the best mode of each MB. The experimental results show that the authors' algorithm realizes 26.9% of encoding time when compared with the JSVM reference software with smallest reduction in peak signal to noise ratio (PSNR).

Joint Adaptive Pre-processing Resilience and Post-processing Concealment Schemes for 3D Video Transmission

3D video transmission over erroneous networks is still a considerable issue due to restricted resources and the presence of severe channel errors. Efficiently compressing 3D video with low transmission rate, while maintaining a high quality of received 3D video, is very challenging. Since it is not plausible to re-transmit all the corrupted macro-blocks (MBs) due to real time applications and limited resources. Thus it is mandatory to retrieve the lost MBs at the decoder side using sufficient post-processing schemes, such as error concealment (EC). In this paper, we propose an adaptive multi-mode EC (AMMEC) algorithm at the decoder based on utilizing pre-processing flexible macro-block ordering error resilience (FMO-ER) technique at the encoder; to efficiently conceal the erroneous MBs of intra and inter coded frames of 3D video. Experimental simulation results show that the proposed FMO-ER/AMMEC schemes can significantly improve the objective and subjective 3D video quality.

Super-resolution with selective filter based on adaptive window and variable macro-block size

Super-resolution (SR) covers a set of techniques whose objective is to improve the spatial resolution of a video sequence or a single frame. In this scope, fusion SR techniques obtain high-resolution (HR) frames taking as a reference several low-resolution (LR) frames contained in a video sequence. This paper is based on a selective filter to decide the best LR frames to be used in the super-resolution process. Additionally, each frame division into macro-blocks (MBs) is analyzed both in a fixed block size approach, which decides which MBs should be used in the process, and in a variable block size approach with an adaptive MB size, which has been developed to set an appropriate frame division into MBs with variable size. These contributions not only improve the quality of video sequences, but also reduce the computational cost of a baseline SR algorithm, avoiding the incorporation of non-correlated data. Furthermore, this paper explains the way in which the enhanced algorithm proposed in it outperforms the quality of typical SR applications, such as underwater imagery, surveillance video, or remote sensing. The results are provided in a test environment to objectively compare the image quality enhancement obtained by bilinear interpolation, by the baseline SR algorithm, and by the proposed methods, thus presenting a quantitative comparison based on peak signal-to-noise ratio (PSNR) and Structural SIMilarity (SSIM) index parameters. The comparison has also been extended to other relevant proposals of the state of the art. The proposed algorithm significantly speeds up the previous ones, allowing real-time execution under certain conditions.

Region-of-interest based rate control algorithm for H.264/AVC video coding

Conventional rate control algorithms allocate bits for every macroblock (MB) without consider whether it needs encoding, and they choose encoding mode only from the set provided by H.264/AVC standard. While, according to the human visual system (HVS) research, human eyes can only focus on one area in a frame, which is called region-of-interest (ROI). This phenomenon gives a chance to code all MBs unequally, especially for NROI MB, we may do not need encoded. In this paper, a ROI-based rate control algorithm for H.264/AVC video coding is proposed to improve the coding efficiency. After the ROI segmentation, the target bits are unequally allocated for each MB. For non-region-of-interest (NROI) MB coding, we proposed the Active MB Concealment (AMC) Mode in rate-distortion optimization (RDO). The AMC gives a tradeoff between the rate and distortion and improve the quality of the ROI MBs at the cost of the quality decreasing of the NROI MBs. Compared with the conventional rate control algorithms, the experimental results demonstrate that the proposed algorithm gives the better performance in terms of the average peak signal-to-noise ratio (PSNR) and the subjective evaluations, and maintains the smoothness of video quality.

Dynamic macroblock wavefront parallelism for parallel video coding

Wavefront parallelism is effective for parallel video encoding thanks to its merits of low latency, no quality loss and high degree of parallelism. In traditional video encoders, macroblock row wavefront (MRW) parallelism was widely adopted. However the performance of MRW is limited by workload unbalance and computing resource unbalance among multiple work threads. This paper proposes a new dynamic macroblock wavefront (DMW) parallelism to alleviate the limitations of MRW. In DMW, the available macroblocks (MBs) are scheduled to work threads MB by MB other than MB row by MB row; and after one MB is encoded by a work thread, the MB on its right (if available) has the highest priority to be scheduled to avoid synchronization delay. Experimental results demonstrate that video encoder with traditional MRW can be accelerated by more than 10% with our proposed DMW. When hyper-threading is used, the advantage of DMW is more prominent.

A Hybrid Scheme Based on Pipelining and Multitasking in Mobile Application Processors for Advanced Video Coding

One of the key requirements for mobile devices is to provide high-performance computing at lower power consumption. The processors used in these devices provide specific hardware resources to handle computationally intensive video processing and interactive graphical applications. Moreover, processors designed for low-power applications may introduce limitations on the availability and usage of resources, which present additional challenges to the system designers. Owing to the specific design of the JZ47x series of mobile application processors, a hybrid software-hardware implementation scheme for H.264/AVC encoder is proposed in this work. The proposed scheme distributes the encoding tasks among hardware and software modules. A series of optimization techniques are developed to speed up the memory access and data transferring among memories. Moreover, an efficient data reusage design is proposed for the deblock filter video processing unit to reduce the memory accesses. Furthermore, fine grained macroblock (MB) level parallelism is effectively exploited and a pipelined approach is proposed for efficient utilization of hardware processing cores. Finally, based on parallelism in the proposed design, encoding tasks are distributed between two processing cores. Experiments show that the hybrid encoder is 12 times faster than a highly optimized sequential encoder due to proposed techniques.

A Programming Environment for Visual Block-Based Domain-Specific Languages

Abstract Visual block-based programming is useful for various users such as novice programmers because it provides easy operations and improves the readability of programs. Also, in programming education, it is known to be effective to initially present basic language features and then gradually make more advanced features available. However, the cost of implementing such visual block-based languages remains a challenge. In this paper, we present a programming environment for providing visual block-based domain- specific languages (visual DSLs) that are translatable into various programming languages. In our environment, programs are built by combining visual blocks expressed in a natural language. Blocks represent program elements such as operations and variables. Tips represent snippets, and macro blocks represent procedures. Using Tips and macros make code more abstract, and reduce the number of blocks in code. Visual DSLs can be a front-end for various languages. It can be easily restricted and extended by adding and deleting blocks. We applied our programming environment to Processing, an educational programming language for media art. We show that the environment is useful for novice programmers who learn basic concepts of programming and the features of Processing.

A Rate Control Algorithm for Low Delay Video Coding

For the standard definition video, the inherent delay produced by the existing coding standards is above 260ms. The delay is too big to apply in real-time video processing system. In order to reduce codec system delay, this paper gives a low delay coding structure. Firstly, the input video is divided into a number of slices for reducing the acquisition delay, and then the slice is coded as a basic unit. Secondly, In order to solve the problem of bit allocation with low delay coding process, this paper presents a constant bit rate control algorithm based on slices; frame and macro block line rate allocation strategy is given. The experimental results show that, compared with the frame/field codec system, new slice coding structure and rate control algorithm can keep the quality of the reconstructed image and effectively reduce the inherent codec system delay. The delay of the proposed algorithm is about 100ms and it can satisfy the real-time requirement.

Research on No-reference Video Quality Assessment Based on H.264/AVC Bitstream

It is an important part to efficiently evaluate the video quality for user experience in video systems. Traditional video evaluation criterion is strict for testing environment, and is not appropriate for real-time evaluation on network vid- eo. No-reference video quality assessment doesn't require any information of original video, which makes it meet the characteristics of high real-time evaluation on network video image quality and small resource reduction. By extracting key parameters distorted by characteristic video in H.264/AVC video stream flow including quantification parameter, en- coding rate and bits of each macro block of coding video, the paper establishes no-reference video quality assessment model from the space complexity and time complexity of video. For the model, not only the model is not complicated, and can meet the requirement of real-time quality assessment of video, the simulation test but also proves that the correlation between the algorithm and subjective assessment value is high, which meets visual perception feature of human eye.