Video Encoder Research Articles

Energy‐efficient system‐on‐chip reconfigurable architecture design for sum of absolute difference computation in motion estimation process of H.265/HEVC video encoding

SummaryMotion estimation is the important and computationally intensive part of any video encoding. The objective of this paper is to design and analyze the coarse and fine reconfiguration of processing element‐based hardware design for block matching–based motion estimation in H.265/HEVC video processing. Sum of absolute difference (SAD) is the commonly used criteria for block matching in the motion estimation process. User input is taken as the parameter for coarse reconfiguration, and the threshold value in SAD is considered for the fine reconfiguration. Processing elements are the hardware units designed for performing SAD calculation. In this work, the hardware for block‐based motion estimation for H.265/HEVC encoder is designed using multiple processing elements, which will calculate the SAD values. A system‐on‐chip architecture is implemented and verified the optimization in terms of power and area utilization by reconfiguring the architecture dynamically based on the input video quality requirement.

Efficient Privacy-Preserving Anomaly Detection and Localization in Bitstream Video

In cloud computing, videos may be in an encrypted format to protect privacy. Therefore, encrypted video processing is an important application in secure cloud computing. In this paper, we focus on parameter estimation and anomaly detection in an encrypted video bitstream. By analyzing the common properties of video encoding frameworks and the format-compliant encryption schemes, we propose an anomaly detection scheme for encrypted video bitstream with format-compliant encryption. From the encrypted bitstream, we extract three types of complementary features, i.e., the macroblock sizes, the macroblock partitions, and the motion vector difference magnitude, and then propose a method to combine these three features. The proposed detection and localization scheme does not involve video decryption, full decompression, or an interactive protocol, which makes it efficient. Our scheme is also compatible with different video encryption methods. To accelerate the running time, we develop a parallel implementation for our scheme. The experimental results show that our method achieves good running time and detection rate performance.

A fast temporal and hybrid SHVC encoder

In the last few years new devices with different computational capabilities and network connections has emerged, and this leads to the need for more adaptable video streams. The last two video coding standards, namely H.264/Advanced Video Coding and High Efficiency Video Coding, have defined scalable versions of themselves that provide this adaptability by the use of several layers with different levels of quality, frame rates, and resolutions. Moreover, HEVC introduced hybrid scalability, which allows a non-HEVC base layer (e.g., H.264/AVC) in combination with HEVC enhancement layers, what provides backwards compatibility with older devices. Nevertheless, scalable video coding is very computationally expensive, making it necessary to accelerate the video encoding. This paper presents a fast Coding Unit size algorithm for the combination of hybrid and temporal scalabilities that is based on machine learning. Experimental results show that an acceleration of 54% is achieved at the cost of a slight increment in the bit rate.

Exploring correlation information for image compression of four-dimensional computed tomography.

Nowadays four-dimensional computed tomography (4DCT) is popularly used in evaluating respiration-related organ motion for patients under radiotherapy. As consisting of multiple subsets of CT images, a larger storage space is needed for 4DCT. In this study, the correlations information within these subsets was explored and the popular video encoders were used for 4DCT image compression. The images of 4DCT subsets were arranged in an order and put into a sequence for the input of video encoder. The effects of two ordering methods on the compression performance of video encoder were compared. One ordering method, phase-prioritized (PP) sequence, arranged 4DCT images according to their respiration phases. Another ordering method, location-prioritized (LP) sequence, arranged 4DCT images by their slice locations. Three popular video encoders were selected including one lossless compression algorithm and two lossy compression algorithms. Based on a publicly available database consisting of 82 4DCT datasets of 20 lung cancer patients, the performance of two ordering methods and three video encoders was quantitatively assessed. The highest compression ratios were 310 and 260 for LP and PP ordering methods respectively and achieved by one video encoder employing inter-frame prediction coding algorithm. The highest compression ratios for the other two video encoders employing intra-frame coding algorithms were 16 and 7. The LP ordering method showed less inter-frame variation and higher inter-frame similarity over the PP ordering method. The LP ordering method would result in higher compression ratio than the PP ordering method for 4DCT image compression. The compression performance of video encoder employing inter-frame prediction coding algorithm is superior to those of video encoders employing intra-frame coding algorithms. The video encoder with inter-frame prediction coding algorithm and LP ordering method would be highly suitable for 4DCT image compression.

High-Performance Algorithm Adaptations and Hardware Architecture for HEVC Intra Encoders

High computational complexity and heavy data/timing dependency in high efficiency video coding (HEVC) hinder the throughput of video encoding. In order to address both design challenges, we present high-performance algorithm adaptations and high-throughput hardware architecture for the HEVC intra encoders. Particularly, the chroma intra prediction includes the derived luma mode, which greatly improves the worst-case video compression performance. Compared with the HM-15.0 reference software, the proposed algorithms reduce the computational workload by 22%, while the average BD-Rate and BD-PSNR for classes A-F and 4K are 2.79% and -0.13 dB, respectively. This BD-Rate is lower than existing designs in the literature with the same video resolution. The field-programmable gate array (FPGA) implementation of proposed design shows it operates at 120 MHz and supports 45 f/s of 1080P video sequences using 196K logic elements and 127-KB on-chip SRAM. The application-specified integrated circuit (ASIC) implementation of the proposed design in TSMC 90 nm technology shows that its clock frequency reaches 320 MHz with a hardware gate count of 2186K and power consumption of 290 mW, and it supports real-time encoding of 4K videos at 30 f/s.

Time-Constrained Video Delivery Using Adaptive Coding Parameters

In some applications, video content needs to be encoded and uploaded to a remote destination within a pre-defined amount of time. In order to guarantee that the overall processing time does not exceed certain time constraints, a system that performs joint encoding and uploading time control is needed. Such a system requires the flexibility to control both the time spent in the encoding process and the bit-rate. This is because the latter significantly influences the transmission time, especially for transmissions under low-bandwidth constraints. This paper proposes a novel approach to address this challenge by adapting the quantization parameters (QP) of a video encoder in order to meet overall processing time requirements, including both encoding and uploading time. The proposed QP adaptation approach relies on mechanisms to accurately predict the encoding time and bit-rate during the encoding process for the incoming group of pictures. This in turn allows adequate QP selections that result in accurately meeting the overall time constraints. A comprehensive experimental evaluation shows that the proposed QP adaptation approach can accurately meet the overall time constraints by efficiently adapting the encoding process to different target times and bandwidth conditions.

Network-on-Chip Especially for Video Coding Applications Using Multilayer Mesh Topology

Background: Network on chip is proposed as new reusable and scalable communication system for applications with important number of IPs. The NoC architecture characteristics are based on several factors: the implementation strategy of IPs, the power dissipation, the placement of IPs, data transfer time, the requirements of the given application, etc. The N×M Mesh topology combined with the XY routing algorithm are generally chosen in many studies. Hardware IPs proposed in the literature, for various applications as example video encoders, operates at different frequencies and generally implemented according to several strategies and different bus sizes. Connecting these IPs using the same communication system is very difficult. Methods: In this paper, we present a new topology based on multi-layer mesh topology and adapted for video coding applications. The proposed topology exploits the video coding information regarding groups of cores that communicate through two cores only. The idea is to use a specific NoC for each group of cores and connect the NoCs with bridge in the positions of two communication cores. The choice of parameters in each NoC depends on the characteristic of IPs in the same group in order to maximize communication adaptivity and performance. Results: Synthesis results show that the proposed multi-layer mesh topology NoC uses much less resources than the traditional NxM mesh topology NoC. Conclusion: This reduction in term of resources is assured by the considerable reduction in the length and number of global interconnects, resulting in an increase in the performance and decrease in the power consumption and area of wire limited circuits.

Optimal Adaptive Quantization Based on Temporal Distortion Propagation Model for HEVC.

Optimal adaptive quantization is one of the key points to optimize the coding efficiency of video encoders. The latest block-based video compression standards, such as high-efficiency video coding (HEVC), extensively use predictive coding techniques that create dependencies between blocks and increase the complexity of optimal block quantizers search. Specifically, the motion compensation is responsible for a dependency network connecting all blocks of the same GOP together. In this paper, this dependency network is estimated by a temporal distortion propagation model and an accurate estimation of Inter and Skip modes probabilities. Optimal quantizers are then designed per block in order to achieve global optimization in terms of rate-distortion efficiency. By implementing the algorithm into the HEVC reference model (HM), we report -16.51% PSNR-based and -26.26% SSIM-based average bitrate savings compared to no adaptive quantization. The proposed algorithm outperforms several related methods from the state-of-the-art. Moreover, along with the demonstration of an optimal quantizer solution, we propose an in-depth analysis of the algorithm behavior. This analysis includes, among others, the relative distribution of rates between frames and the control of quantizers dynamic range.

Fast Coding Unit Partition Decision for HEVC Using Support Vector Machines

Despite the several speedup methods proposed in the literature, the computational complexity of High Efficiency Video Coding (HEVC) video encoding is still a problem. This paper proposes a fast coding unit (CU) partition decision for use in HEVC encoders based on support vector machine (SVM)-trained offline. The SVM classifiers, features, and training procedures are described in detail, and a justification for the use of SVMs is provided. The trained classifiers are incorporated into a modified reference encoder in the form of a fast CU partition decision algorithm, which decides if the exhaustive search for the best partition is continued or terminated prematurely. Using the proposed method, an average complexity reduction of 48% is achieved with a 0.48% Bjontegaard-Delta bitrate (BD-BR) loss using the random access coding configuration, 44% reduction with a 0.62% BD-BR loss for the Low Delay B, and a 41% reduction with a 0.6% BD-BR loss for the Low Delay P configuration. We also tested our approach under constant bitrate conditions, achieving a 47% reduction in encoding time with a 1.11% loss in the BD-BR. In addition, a decision threshold adaptation is also proposed to allow adjusting the rate-distortion/complexity trade-off of our solution. With this approach, the computational complexity reduction can be varied from 34.9% (with a 0.13% loss in the BD-BR) up to 52.4% (with a 1.11% BD-BR loss) using the random access configuration. Compared with the state-of-the-art solutions, our decision scheme outperforms the related works in terms of combined rate distortion and complexity.

Optimal Bit Allocation at Frame Level for Rate Control in HEVC

Rate control plays an important role in video coding systems, which makes the output bitrate of a video encoder equal to the target bitrate while minimizing the distortion of the compressed video. However, most of the existing rate control schemes achieve accurate bitrate control at the loss of rate-distortion (R-D) performance. This paper proposes an effective frame level bit allocation method to improve the R-D performance whilst maintaining the high accuracy of the bitrate control in HEVC. First, an improved R-D model is presented at the frame level, which, by making use of the information of encoded frames more completely, achieves higher accuracy with lower bitrate mismatch rate. Second, a group of picture (GOP) level bit allocation approach is introduced with the consideration of the temporal R-D dependency among different GOPs, which can further enhance the R-D performance. Finally, to achieve optimal R-D performance globally, a formulation for optimal frame level bit allocation is developed with a GOP level Lagrange multiplier introduced, which takes into account the coding effects of a current frame on the other frames within the same GOP. A scheme of recursive Taylor expansion is employed to find the GOP level Lagrange multiplier. Experimental results demonstrate that the proposed method can achieve considerable R-D performance improvement and accurate bitrate control. Specifically, our method shows 3.9% and 3.8% BD-rate saving in average compared against the HEVC reference software HM16.7 with rate control in the low-delay B and low-delay P coding structures, respectively.

A Novel Gaming Video Encoding Process Using In-Game Motion Vectors

In this paper, we propose a motion preprocessing method for the use in the game application pipeline, which is composed of two stages: the proposed preprocessing method and the High Efficiency Video Coding (HEVC) encoder. The method accepts the object information from the game application, preprocesses the motion vectors of objects, and pass the preprocessed motion data to the HEVC encoder. The HEVC encoder takes the motion data as the initial (or the dedicated) value of motion estimation. Therefore, the traditional diamond search can be skipped and hence increase the encoding performance of the HEVC encoder. In the motion preprocessing method, the following three steps are taken: a coordination system transformation, determining motion vectors for $4\times 4$ -checkerboard blocks [Atomic Block (AB)], and the selection of proper motion vectors for all varieties of prediction units in the encoder. With a focus on the special issues, such as move-out zone and bi-directional prediction, we are able to further optimize the performance of the encoder. We examined two types of 2D gaming scenes in our experiments. The experimental results show that, as compared with the original diamond search method provided by the encoder, our algorithm is able to achieve up to 49.0% time reduction of video encoding. The Bjontegaard Delta bit rate can achieve up to −17.0% in the random_access mode while combining with the $\times 265$ encoder and up to −26.2% in the lowdelay mode while combining with the HM-16 encoder.

Pseudorandom number generator based on enhanced Hénon map and its implementation

This paper presents a pseudorandom number generator (PRNG) based on enhanced Hénon map (EHM) and its implementation in software and hardware for chaos-based cryptosystems with high processing such as image or video encryption. The proposed EHM presents better statistical properties and higher key sensitivity in comparison with classic Hénon map (CHM) by means of numerical tests such as bifurcation diagrams, largest Lyapunov exponent, Gottwald-Melbourne test, and histograms. The proposed 8-bit PRNG-EHM algorithm is implemented in MATLAB (software) and in FPGA technology (hardware) for experimental results. In hardware implementation, we use VHDL language and the Altera DE2-115 FPGA board with RS-232 serial port communication for data extraction, which are analyzed with MATLAB. In both software and hardware level, the proposed PRNG-EHM passes the randomness NIST 800-22 statistical tests. For first time in literature, a comprehensive security analysis from a cryptographic point of view is presented for hardware implementation such as key space analysis, key sensitivity, floating frequency, histograms, autocorrelation, correlation, entropy, and performance. Comparisons of proposed PRNG-EHM with recent similar schemes show main advantages in security capabilities for cryptographic applications. According with the results, the proposed scheme can be used in chaos-based cryptographic applications at software or hardware implementation.

Dağıtık Sıkıştırmalı Video Algılama için Basit ve Uyarlanabilir bir Veri Hızı Kontrolü Tasarımı

Providing fast video encoding, Compressive Sensing has been very suitable for the schemes which require power-constraint devices such as in wireless sensor networks. However, resulted data load on the network is extremely high and accomplishing rate control on these compressive sensing codecs by a simple scheme is crucial. In this paper we propose a simple scheme of data rate control mechanism for an existing compressive sensing codec for distributed video coding which has been reported as an efficient framework in terms of video reconstruction quality. Our approach does not use a feedback channel from decoder which avoids encoder to wait for high-complexity optimization problem. Embedding the proposed scheme to the codec architecture, we have obtained ~43% gain in data rate with an acceptable decrease (6%) in Video PSNR.

An integrated scheme for streaming scalable encoded video-on-demand over CR networks

In this paper, we propose an adaptive scheme to deliver scalable VoD streams over a cognitive radio network (CRN) that consists of a base station (BS) and multiple secondary users (SUs). The proposed scheme aims at improving the perceptual quality of the received videos at the SUs end while maintaining a continuous playback. To do so, the scheme opportunistically exploits the idle primary channel(s) as sensed by the BS. Moreover, to reduce the impact of the intermittent availability nature of the CR channels, we propose a channel allocation algorithm that fairly allocates the idle channel(s) among the active SUs while jointly taking into consideration the status of the playback buffers at the SUs as well as the quality of the available channels. Additionally, the proposed scheme schedules the video frames on the allocated slots by exploiting the flexibility of the scalable video encoder as well as the adaption of the used modulation level based on the channel conditions to meet a predetermined target bit error rate (BER). Simulation results demonstrate the efficiency of the proposed scheme in exploiting the scalability feature of the transmitted video sequences to efficiently use the dynamic CR resources while achieving improved peak signal-to-noise ratio of the reconstructed video streams with uninterrupted playback.

Improved Perceptual Video Encryption Technique using Residue Number System

In this paper, we propose an enhanced perceptual video encryption technique to speed-up and secure cipher video transmitted across networks using Residue Number System (RNS). The technique proposes a new reverse converter with smaller dynamic range using the moduli set {2n-1, 2n, 2n+1} that is integrated into our previous work of [2]. After encryption, cipher video is encoded into three residual videos that have smaller pixel values and ideal for transmission across networks. Instead of transmitting the three (3) residual videos, the technique effectively transmits and decodes only two (2) of them back into the original video with same visual quality. Experimental results show that the technique enhances transmission speed and security of cipher video across networks.

Efficient Absolute Difference Circuit for SAD Computation On FPGA

Video Compression is very essential to meet the technological demands such as low power, less memory and fast transfer rate for different range of devices and for various multimedia applications. Video compression is primarily achieved by Motion Estimation (ME) process in any video encoder which contributes to significant compression gain. Sum of Absolute Difference (SAD) is used as distortion metric in ME process. In this paper, efficient Absolute Difference(AD)circuit is proposed which uses Brent Kung Adder(BKA) and a comparator based on modified 1’s complement principle and conditional sum adder scheme. Results shows that proposed architecture reduces delay by 15% and number of slice LUTs by 42% as compared to conventional architecture. Simulation and synthesis are done on Xilinx ISE 14.2 using Virtex 7 FPGA.

Impacts of Equirectangular 360-degrees Videos in the Intra-Frame Prediction of HEVC

Recent technological advancements allowed videos to come from a simple sequence of 2D images to be displayed in a flat screen display into spherical representations of one’s surroundings, capable of creating a realistic immersive experience when allied to head-mounted displays. In order to explore the existing infrastructure for video coding, 360-degrees videos are pre-processed and then encoded by conventional video coding standards. However, the flattened version of 360-degrees videos present some peculiarities which are not present in conventional videos, and therefore, may not be properly exploited by conventional video coders. Aiming to find evidence that conventional video encoders can be adapted to perform better over 360-degrees videos, this work performs an evaluation on the intra-frame prediction performed by the High Efficiency Video Coding over 360-degrees videos in the equirectangular projection. Experimental results point that 360-degrees videos present spatial properties that make some regions of the frame likely to be encoded using a reduced set of prediction modes and block sizes. This behavior could be used in the development of fast decision and energy saving algorithms by evaluating a reduced set of prediction modes and block sizes depending on the regions of the frame being encoded.

Considering User Behavior in the Quality of Experience Cycle: Towards Proactive QoE-Aware Traffic Management

The concept of Quality of Experience (QoE) of Internet services is widely recognized by service providers and network operators. They strive to deliver the best experience to their customers in order to increase revenues and avoid churn. Therefore, QoE is increasingly considered as an integral part of the reactive traffic management cycle of network operators. In addition, QoE also constitutes a cycle of its own, which includes user behavior and service requirements. This letter describes this QoE cycle, which is not widely taken into account yet, discusses the interactions of the two cycles, and derives implications toward an improved and proactive QoE-aware traffic management. A showcase on how network operators can obtain hints on the change of network requirements from detecting user behavior in encrypted video traffic is also presented.

A Rate Control Model of MPEG-4 Encoder for Video Transmission over Wireless Sensor Network

Recently, multimedia application has a lot of attention in the research community, especially when transmitting video over IEEE 802.15.4 standard. This is due to the capability of providing low complexity with low cost, but still maintaining the quality of video in term of packet received. However, transmitting video over Wireless Sensor Network (WSN) posed a new research challenges with high bandwidth demand and energy constrained of sensor nodes. MPEG-4 video codec is one of the compression techniques that used to decrease the amount of bandwidth required to meet WSN environment. Therefore, video encoding is a useful tool for rate control to control the video bit rate and maintaining the video quality especially in real-time communication applications. Video bit rate is affected by quantization scale, frame rate, and Group of Picture (GOP) size. A rate control model called enhanced Video Motion Classification based (e-ViMoC) model is proposed in this paper to produce the desired bit rate that complies to the IEEE 802.15.4 standard, while at the same time preserving the video quality. The analysis has shown that, the video transmission using e-ViMoC rate control achieves enhancement in delivery ratio, energy consumption and video quality (PSNR) when compared to video transmission using uncompressed video.

EFFICIENT ABSOLUTE DIFFERENCE CIRCUIT FOR SAD COMPUTATION ON FPGA

Video Compression is very essential to meet the technological demands such as low power, less memory and fast transfer rate for different range of devices and for various multimedia applications. Video compression is primarily achieved by Motion Estimation (ME) process in any video encoder which contributes to significant compression gain. Sum of Absolute Difference (SAD) is used as distortion metric in ME process. In this paper, efficient Absolute Difference(AD)circuit is proposed which uses Brent Kung Adder(BKA) and a comparator based on modified 1’s complement principle and conditional sum adder scheme. Results shows that proposed architecture reduces delay by 15% and number of slice LUTs by 42% as compared to conventional architecture. Simulation and synthesis are done on Xilinx ISE 14.2 using Virtex 7 FPGA.