Macroblock Level Research Articles

A Novel Macroblock-Level Filtering Upsampling Architecture for H.264/AVC Scalable Extension

The scalable extension of the H.264/AVC standard, also called H.264/AVC SVC standard or simply SVC standard uses spatial upsampling in the spatial scalability modes. This work presents a novel upsampling architecture designed for operation at macroblock level and dyadic upsampling ratio with QVGA as the base layer resolution and VGA as the enhancement resolution.The adoption of a macroblock-level solution translates into a more efficient use of hardware resources with savings of approximately 25% in the number of ALUTs and DLRs and using about two hundred times less memory bits, when compared to previously published works. The designed architecture was synthesized targeting four FPGAs: Altera Cyclone III and Stratix IV and Xilinx Spartan 3E and Virtex 4. The best throughput was reached by the Xilinx Virtex 4 device, with a processing rate of 506 VGA frames per second. The worst result was reached by the Xilinx Spartan 3E FPGA, with 249 VGA frames per second. All target FPGAs surpasses the necessary throughput to decode VGA videos in real time. This very high throughput is important especially when low power applications are considered, since with low operation frequencies (9.34MHz) it is possible to reach real time (30 frames per second) for all target FPGAs.

Mitigation of H.264 and H.265 Video Compression for Reliable PRNU Estimation

The photo-response non-uniformity (PRNU) is a distinctive image sensor characteristic, and an imaging device inadvertently introduces its sensor's PRNU into all media it captures. Therefore, the PRNU can be regarded as a camera fingerprint and used for source attribution. The imaging pipeline in a camera, however, involves various processing steps that are detrimental to PRNU estimation. In the context of photographic images, these challenges are successfully addressed and the method for estimating a sensor's PRNU pattern is well established. However, various additional challenges related to generation of videos remain largely untackled. With this perspective, this work introduces methods to mitigate disruptive effects of widely deployed H.264 and H.265 video compression standards on PRNU estimation. Our approach involves an intervention in the decoding process to eliminate a filtering procedure applied at the decoder to reduce blockiness. It also utilizes decoding parameters to develop a weighting scheme and adjust the contribution of video frames at the macroblock level to PRNU estimation process. Results obtained on videos captured by 28 cameras show that our approach increases the PRNU matching metric up to more than five times over the conventional estimation method tailored for photos.

Enhanced Macroblock Features for Dynamic Background Modeling in H.264/AVC Video Encoded at Low Bitrate

While H.264 is a well-established standard for video surveillance, its high-profile implementation, in particular, has the unique capabilities that pack more visual detail into a given bitrate. Several algorithms exist that detect moving objects from the H.264 bitstream, most of which end up incorrectly classifying the nonstationary or dynamic components (e.g., jitter camera, waving trees, ripples, etc.) of the background as foreground. Moreover, the coarse quantization schemes adopted at constrained bitrates pose new challenges for direct identification of moving objects/targets from the bitstream. This paper focuses on dynamic background modeling for the H.264 video encoded at very low bitrates. To this end, enhanced features of the fundamental coding unit, i.e., the macroblock (MB) are proposed. Based on the temporal statistics of the MB features gathered from initial frames, MBs that potentially contain parts of moving objects are selected in subsequent frames. The selected MBs constitute a coarse segmentation map of the foreground at the MB level. Finally, pixel-level segmentation of the foreground is obtained by comparing pixels constituting the selected MBs with the colocated counterparts of a background frame. Experimental results showing the comparison of bitstreams encoded at strikingly low bitrates are obtained over a diverse set of standardized surveillance sequences.

Intra Frame Coding In Advanced Video Coding Standard (H.264) to Obtain Consistent PSNR and Reduce Bit Rate for Diagonal Down Left Mode Using Gaussian Pulse

Intra prediction process of H.264 video coding standard used to code first frame i.e. Intra frame of video to obtain good coding efficiency compare to previous video coding standard series. More benefit of intra frame coding is to reduce spatial pixel redundancy with in current frame, reduces computational complexity and provides better rate distortion performance. To code Intra frame it use existing process Rate Distortion Optimization (RDO) method. This method increases computational complexity, increases in bit rate and reduces picture quality so it is difficult to implement in real time applications, so the many researcher has been developed fast mode decision algorithm for coding of intra frame. The previous work carried on Intra frame coding in H.264 standard using fast decision mode intra prediction algorithm based on different techniques was achieved increased in bit rate, degradation of picture quality(PSNR) for different quantization parameters. Many previous approaches of fast mode decision algorithms on intra frame coding achieved only reduction of computational complexity or it save encoding time and limitation was increase in bit rate with loss of quality of picture. In order to avoid increase in bit rate and loss of picture quality a better approach was developed. In this paper developed a better approach i.e. Gaussian pulse for Intra frame coding using diagonal down left intra prediction mode to achieve higher coding efficiency in terms of PSNR and bitrate. In proposed method Gaussian pulse is multiplied with each 4x4 frequency domain coefficients of 4x4 sub macro block of macro block of current frame before quantization process. Multiplication of Gaussian pulse for each 4x4 integer transformed coefficients at macro block levels scales the information of the coefficients in a reversible manner. The resulting signal would turn abstract. Frequency samples are abstract in a known and controllable manner without intermixing of coefficients, it avoids picture getting bad hit for higher values of quantization parameters. The proposed work was implemented using MATLAB and JM 18.6 reference software. The proposed work measure the performance parameters PSNR, bit rate and compression of intra frame of yuv video sequences in QCIF resolution under different values of quantization parameter with Gaussian value for diagonal down left intra prediction mode. The simulation results of proposed algorithm are tabulated and compared with previous algorithm i.e. Tian et al method. The proposed algorithm achieved reduced in bit rate averagely 30.98% and maintain consistent picture quality for QCIF sequences compared to previous algorithm i.e. Tian et al method.

Improved macroblock level rate control for the spatially scalable extension of H.264/AVC

An improved rate control algorithm, designed for scalable video coders incorporating interlayer prediction, is proposed. Firstly, a Rate Distortion (RD) model for interlayer prediction involving the spatial enhancement layers is devised. An optimised Mean Absolute Difference (MAD) prediction model for the spatial enhancement layers that considers both the MAD from the spatial base layer in the same frame and the MAD from the corresponding macroblock in previous frames is also proposed. Simulation results show that the resulting algorithm produces accurate rate control with an average bit rate error of less than 0.26%. Compared with the JVT-W043 default rate control algorithm of the JSVM, the proposed algorithm improves the average PSNR by up to 0.53dB or reduces the bit rate by an average of 10.95%. Furthermore, the proposed algorithm can be combined with the existing rate control scheme for H.264/AVC, resulting in further improvements.

Human Visual System-based Unequal Error Protection for Robust Video Coding

To increase the overall visual quality of the video services without increasing data rate, a human visual system-based video coding, founded on a hierarchy of the video stream in different levels of importance, is developed. Determining these importance levels takes in count three classification criteria: the position of current image in the group of images (image level), the importance of the motion vectors of macroblocks in the current image (macroblock level) and belonging or not of a pixel in a spatial region of interest (pixel level). At the end of this classification process, an interpolation between the results of the three-level selection allows to establish an index of importance for each macroblock of the image to be encoded. This index determines the type of channel coding to be applied to the corresponding macroblock. Tests have shown that the technique presented in this paper achieves better results in PSNR and SSIM (structural similarity) than an equal error protection technique.

A receiver aware H.264/AVC encoder for decoder complexity control in mobile applications

Due to the power limitations of mobile devices, high-quality video decoding is still a main concern, because it quickly drains battery. In this paper, an H.264/AVC receiver aware encoder has been designed that (1) takes into account all of the decoder modules of a receiver, unlike existing RAEs that only consider some of these modules and are therefore sub optimal, and (2) is independent of decoder implementations and platforms. Furthermore, a decoder complexity controller has been proposed that reduces the complexity of different decoder modules, while minimum distortion is achieved. Finally, we formulate and solve a generic RAE optimization problem, and apply this solution to control the computational resource allocation at the macroblock level of a RAE. Our experiments indicate that the proposed approach can reduce the complexity of different modules by up to 10 % with no quality degradation. In addition, the average error of the proposed complexity controller is 0.8 %, making the accuracy of the system very close to 1.

A low-complexity MPEG-2 to H.264/AVC wavefront intra-frame transcoder architecture

We present a reconfigurable high-throughput MPEG-2 to H.264/AVC intra-frame transcoding architecture with wavefront data processing. The proposed FPGA implementation deals with the highly data-dependent critical path of the transcoder, and with low-complexity synchronization of the encoding and decoding stages. Furthermore, the implementation is applicable for reconfiguration and limits the communication bandwidth to the surrounding system through use of on-chip memory. The computationally demanding units of the MPEG-2 decoder have been implemented by processing 8 pixels in parallel, whereas H.264/AVC encoder engine utilizes a 4 × 4 block-level pipeline. The synchronous communication between the stages and full pipeline of the system is achieved by an intermediate memory buffer mechanism. A wavefront macroblock level scanning order based on the on-the-fly processing of consecutive macroblocks and on-chip memory organization are proposed. Achieved results represent a significant reduction of minimal required frequency compared to the state of the art for resolutions CIF, SD and HD1080p. Furthermore, the proposed transcoding core with encoder stage in full pipeline has maximal throughput of 1744 Mpixels/s that corresponds to processing of UHD 4320p resolution at 30 fps.

Architecture Design of the H.264/AVC Encoder Based on Rate-Distortion Optimization

Rate-distortion (RD) optimization (RDO) allows video encoders to achieve better compression efficiencies due to more reliable mode decisions. On the other hand, this technique involves a high computational cost when actual rates and distortions are estimated. To achieve real-time processing, video encoders apply simplified algorithms for mode decisions. This paper presents the H.264/Advanced Video Coding (AVC) encoder architecture, which benefits from the RDO and achieves high throughputs. To access actual motion vector predictors, the motion estimation is placed at the same macroblock stage as the RD mode analysis. The analysis is performed at the partition and macroblock levels. The reconstruction loop and the RDO modules have the high throughput of 32 samples/coefficients per clock cycle, which enables the analysis of a significant number of preselected candidate modes. The high-throughput simplified rate estimation introduces slight quality losses and supports both entropy coding methods available in H.264/AVC. The average quality for the two coding methods is decreased by 0.07 and 0.17 dB compared with the JM17 software using the RDO. The compression efficiency can be traded for throughput. The architecture is verified in the real-time FPGA hardware encoder. The synthesis results show that the architecture consumes 551k gates and 148.63-kB memories. It can support 1080p at 60 frames/s encoding for 90-nm TSMC technology.

Video rate control strategies for cloud gaming

Cloud gaming, also called gaming on demand, is a new kind of service that provides real-time video game experience to the players over the Internet. Although cloud gaming services are getting more and more popular recently, its performance is highly limited by the network bandwidth and latency. This work makes use of the unique characteristics of human visual system (HVS) of video game players to improve bandwidth efficiency. In this work, discussions about the characteristics of game players’ HVS are conducted. The discussions can be further extended to all interactive video on demand systems. Then, some schemes of extracting region of interest and key frames from gaming videos are raised. Based on that, a Macro-block level rate control scheme is proposed based on region of interest and scene-change detection. The simulation results show that, under the same bandwidth constraint, the video quality of proposed method outperforms other methods.

A Hybrid Scheme Based on Pipelining and Multitasking in Mobile Application Processors for Advanced Video Coding

One of the key requirements for mobile devices is to provide high-performance computing at lower power consumption. The processors used in these devices provide specific hardware resources to handle computationally intensive video processing and interactive graphical applications. Moreover, processors designed for low-power applications may introduce limitations on the availability and usage of resources, which present additional challenges to the system designers. Owing to the specific design of the JZ47x series of mobile application processors, a hybrid software-hardware implementation scheme for H.264/AVC encoder is proposed in this work. The proposed scheme distributes the encoding tasks among hardware and software modules. A series of optimization techniques are developed to speed up the memory access and data transferring among memories. Moreover, an efficient data reusage design is proposed for the deblock filter video processing unit to reduce the memory accesses. Furthermore, fine grained macroblock (MB) level parallelism is effectively exploited and a pipelined approach is proposed for efficient utilization of hardware processing cores. Finally, based on parallelism in the proposed design, encoding tasks are distributed between two processing cores. Experiments show that the hybrid encoder is 12 times faster than a highly optimized sequential encoder due to proposed techniques.

A Spatiotemporal Attention-Based Method for Geo-Referenced Video Coding

This paper focuses on the problem that video-GIS has a huge amount of data, which leads to high transmission resource consumption, and introduced attention calculation model to video GIS coding to optimize the coding method so the compression efficiency will be improved as well. Specifically, on the one hand, it will use optical flow technique to calculate the specific location and motion vector of the movement point set of each frame, and reduce the amount of the movement point set integrate with the features of video GIS, then calculate the mask matrix of the foreground movement, and finally compute the Discrete Cosine Transform (DCT) compression integrate with the mask matrix to realize the optimization of the frame macro block level coding. On the other hand, it will calculate the attention of the video frame using the specific location and the motion vector of the movement point set as the primary indicator, and optimize the coding of the video of frame-level according to the calculation result. By experimental verification, the efficiency of video GIS compression is enhanced by the introduction of the features of the video GIS and the cognitive attention theory.

Window-based rate control for video quality optimization with a novel INTER-dependent rate-distortion model

Most model-based rate control schemes use independent rate-distortion (R–D) models at macroblock (MB) level to represent the relationship among bit rate, distortion and encoding complexity. However the correlations between frames (INTER-dependency) are not well considered for distortion, bit allocation and quantization parameter (QP) decision. In this paper, a novel INTER-dependent R–D model is proposed based on the theoretical analysis of the relationship between the predicted residual of one frame and the distortion of its reference frame. To achieve both bit rate accuracy and consistent video quality, a window-based rate control scheme with two sliding windows is introduced. One window is to group certain previously encoded frames and current frame to control the bit rate and buffer delay; the other is to group certain future encoding frames to optimize the fluctuation of video quality. Furthermore, the optimization of Lagrange multiplier is also discussed under the INTER-dependent situation. Experimental results demonstrate that the proposed window-based rate control scheme with INTER-dependent R–D model can achieve accurate target bit rate and improve PSNR performance, meanwhile the variation of PSNR is the smallest compared with other three benchmark algorithms. This one-pass rate control scheme is highly practical for the real-time video coding applications.

Scalable video encoding with macroblock-level parallelism

H.264 video codec provides a wide range of compression options and is popularly implemented over various video recording standards. The compression complexity increases when low-bit-rate video is required. Hence, the encoding time is often a major issue when processing a large number of video files. One of the methods to decrease the encoding time is to employ a parallel algorithm on a multicore system. In order to exploit the capability of a multicore processor, a scalable algorithm is proposed in this paper. Most of the parallelization methods proposed earlier suffer from the drawbacks of limited scalability, memory, and data dependency issues. In this paper, we present the results obtained using data-level parallelism at the macroblock (MB) level for encoder. The key idea of using MB-level parallelism is due to its less memory requirement. This design allows the encoder to schedule the sequences into the available logical cores for parallel processing. A load balancing mechanism is added to allow the encoding with respect to macroblock index and, hence, eliminating the need of a coordinator thread. In our implementation, a dynamic macroblock scheduling technique is used to improve the speedup. Also, we modify some of the pointers with advanced data structures to optimize the memory. The results show that with the proposed MB-level parallelism, higher speedup values can be achieved.

Implementation of AVS Jizhun decoder with HW/SW partitioning on a coarse-grained reconfigurable multimedia system

In this paper, a TPP (Task-based Parallelization and Pipelining) scheme is proposed to implement AVS (Audio Video coding Standard) video decoding algorithm on REMUS (REconfigurable MUltimedia System), which is a coarse-grained reconfigurable multimedia system. An AVS decoder has been implemented with the consideration of HW/SW optimized partitioning. Several parallel techniques, such as MB (Macro-Block)-based parallel and block-based parallel techniques, and several pipeline techniques, such as MB level pipeline and block level pipeline techniques are adopted by hardware implementation, for performance improvement of the AVS decoder. Also, most computation-intensive tasks in AVS video standards, such as MC (Motion Compensation), IP (Intra Prediction), IDCT (Inverse Discrete Cosine Transform), REC (REConstruct) and DF (Deblocking Filter), are performed in the two RPUs (Reconfigurable Processing Units), which are the major computing engines of REMUS. Owing to the proposed scheme, the decoder introduced here can support AVS JP (Jizhun Profile) 1920×1088@39fps streams when exploiting a 200 MHz working frequency.

A novel macroblock level rate control method for stereo video coding.

To compress stereo video effectively, this paper proposes a novel macroblock (MB) level rate control method based on binocular perception. A binocular just-notification difference (BJND) model based on the parallax matching is first used to describe binocular perception. Then, the proposed rate control method is performed in stereo video coding with four levels, namely, view level, group-of-pictures (GOP) level, frame level, and MB level. In the view level, different proportions of bitrates are allocated for the left and right views of stereo video according to the prestatistical rate allocation proportion. In the GOP level, the total number of bitrates allocated to each GOP is computed and the initial quantization parameter of each GOP is set. In the frame level, the target bits allocated to each frame are computed. In the MB level, visual perception factor, which is measured by the BJND value of MB, is used to adjust the MB level bit allocation, so that the rate control results in line with the human visual characteristics. Experimental results show that the proposed method can control the bitrate more accurately and get better subjective quality of stereo video, compared with other methods.

Multiple Description Video Coding Using Inter- and Intra-Description Correlation at Macro Block Level

Multiple Description Video Coding Using Inter- and Intra-Description Correlation at Macro Block Level

Power-aware Regions-of-Interest Computational Resource Allocation for Mobile Sign Language Video Encoding

The primary challenge to design a mobile sign language communication system is overcoming the limitation of battery power. A scheme which allocates the computational resource of the encoder adaptive to available battery power and deaf people’s visual system is proposed. In the scheme, encoding levels which determine number of reference frames and search range are adaptively selected according to the battery power and frame complexity at frame level. Then possible partition mode and quantization parameter are adaptively adjusted at the macro block (MB) level according to the relative priority of each MB. Experimental results show that the proposed algorithm obtains better peak-signal-noise-rate of face and hands that improves the intelligibility of sign language video. Encoding time can be saved by up to 89.4% compared to the standard H.264 encoder. Increased bit rate of the proposed algorithm is less than 5%, which is negligible.

Architectural Decomposition of Video Decoders by Meansof an Intermediate Data Stream Format

The microprocessor industry trend towards many-core architectures introduced the necessity of devising appropriately scalable applications. While implementing software based video decoding, the main challenges are the optimized partitioning of decoder operations, efficient tracking of dependencies and resource synchronization for multiple parallel units. The same applies for hardware implementations of video decoders where monolithic approaches anticipate scalability of the design and reusability of already implemented core components.In this paper, we propose an intermediate data stream format (Meta Format Stream) which is suited for architectural decomposition of video decoding by replacing the conventional monolithic decoder architecture design with a pipelined structure. The Meta Format is forward-oriented and self contained and multistandard capable, so that processing of Meta Streams is independent of the originating bit stream. Our approach does not require special coding settings and is applicable to accelerated decoding of any standards-compliant bit stream. A H.264/AVC multiprocessing proposal is presented as a case study for the potential our our concept. The case study combines coarse grained frame-level parallel decoding of the bit stream with fine-grained macroblock level parallelism in the image processing stage.The proposed H.264 decoder achieved speedup factors of up to 7.6 on an 8 core machine with 2-way SMT. We are reporting actual decoding speeds of up to 150 frames per second in 2160p-resolution.

Visual saliency guided video compression algorithm

Recently Saliency maps from input images are used to detect interesting regions in images/videos and focus on processing these salient regions. This paper introduces a novel, macroblock level visual saliency guided video compression algorithm. This is modelled as a 2 step process viz. salient region detection and frame foveation. Visual saliency is modelled as a combination of low level, as well as high level features which become important at the higher-level visual cortex. A relevance vector machine is trained over 3 dimensional feature vectors pertaining to global, local and rarity measures of conspicuity, to yield probabilistic values which form the saliency map. These saliency values are used for non-uniform bit-allocation over video frames. To achieve these goals, we also propose a novel video compression architecture, incorporating saliency, to save tremendous amount of computation. This architecture is based on thresholding of mutual information between successive frames for flagging frames requiring re-computation of saliency, and use of motion vectors for propagation of saliency values.