Video Codec Research Articles

Review on Architectures of Motion Estimation for Video Coding Standards

Various video coding standards like H.264 and H.265 are used for video compression and decompression. These coding standards use multiple modules to perform video compression. Motion Estimation (ME) is one of the critical blocks in the video codec which requires extensive computation. Hence it is computationally complex, it critically consumes a massive amount of time to process the video data. Motion Estimation is the process which improves the compression efficiency of these coding standards by determining the minimum distortion between the current frame and the reference frame. For the past two decades, various Motion Estimation algorithms are implemented in hardware and research is still going on for realizing an optimized hardware solution for this critical module. Efficient implementation of ME in hardware is essential for high-resolution video applications such as HDTV to increase the decoding throughput and to achieve high compression ratio. A review and analysis of various hardware architectures of ME used for H.264 and H.265 coding standards is presented in this paper.

Evaluation and Analysis of Rate Control Methods for H.264/AVC and MPEG-4 Video Codec

<p class="Default"><span>Audio, image and video signals produce a vast amount of data. The only solution of this problem is to compress data before storage and transmission. In general there is the three crucial terms as, Bit Rate Reduction, Fast Data Transfer and Reduction in Storage. Rate control is a vigorous factor in video coding. In video communications, rate control must ensure the coded bitstream can be transmitted effectively and make full use of the narrow bandwidth. There are various test models usually suggested by a standard during the development of video codes models in order to video coding which should be suffienciently be efficient based on H.264 at very low bit rate. These models are Test Model Number 5 (TMN5), Test Model Number 8 for H.263, and Verification Model 8 (VM8) for MPEG-4 and H.264 etc. In this work, Rate control analysis for H.264, MPEG-4 performed. For Rate control analysis test model verification model version 8.0 is adopted.</span></p>

Efficient Inter-View Motion Vector Prediction in Multi-View HEVC

When multi-view video sequences are encoded, the efficient motion vector prediction (MVP) is one of the most important techniques to increase the coding performance. The predictive MV should be highly correlated with the actual MV of the current block to be encoded. In this paper, we explain the inefficiency of the MVP algorithm in the multi-view high efficiency video codec (MV-HEVC) standard. In order to solve the inefficiency problem, we propose an enhanced advanced motion vector prediction algorithm in which the geometry interrelation between two neighboring views is derived based on epipolar geometry, similarity transform, and affine transform, and then predicted motion vectors (PMVs) for efficient MV coding are generated using obtained geometry relation. The experiment results show that the proposed scheme outperforms conventional MV-HEVC with a 1.32%–1.19% coding gain because it provides more efficient PMVs than conventional algorithms.

Low Power Motion Estimation Algorithm and Architecture of HEVC/H.265 for Consumer Applications

High-quality videos like high-definition (HD) and ultra HD became an essential requirement in recent applications such as security surveillance, television system, etc. However, due to increase in resolution of the videos, the volume of visual information data increases significantly, which became a challenge for storage, transmission and processing the HD video data. The new video compression standard, high efficient video coding (HEVC), achieved two-fold video efficiency improvements as compared to H.264/AVC using efficient compression techniques. Motion estimation (ME) is one of the computationally intensive blocks in video CODEC. In HEVC, the complexity of ME further increases due to a large processing unit and flexible partitioning of the prediction unit (PU). In this paper, we proposed a low power ME algorithm and architecture of the HEVC for consumer applications. The proposed algorithm and architecture utilizes sub-sampling, data reuse, pixel truncation and adaptive search range techniques for reducing the computational power. Simulations result shows that the proposed ME algorithm requires an average of 53.82% fewer search points as compared to the reference software HM with a small degradation in PSNR and little increment in bit-rate. The proposed architecture is simulated and synthesized using standard 90 nm technology. The proposed ME architecture can process $3840\times2160$ @ 30 fps video sequences with only 4.5193 mm2 of the area and 8.192 KB of SRAM. The operating frequency of the proposed architecture is 250 MHz with 151.7619 mW of power.

Cross-Layer Resource Allocation Using Video Slice Header Information for Wireless Transmission Over LTE

In this paper, cross-layer resource allocation methods for wireless video transmission are proposed. We propose two practical metrics to measure the relative importance of each forward error correction (FEC) block to the user’s quality of experience. They can be obtained from the header information, and require no overhead bits or amendment of the header format of the existing standard video codec. We discuss adaptive modulation and coding scheme allocation, and adaptive energy allocation for the FEC blocks of a group of pictures. We also provide low-complexity resource allocation solutions. In both approaches, we consider two types of slice packetization, which affect the priority of FEC blocks. The proposed cross-layer resource allocation methods have significant performance gain over equal error protection, and similar performance to more complicated algorithms.

Mixed-resolution HEVC based multiview video codec for low bitrate transmission

There has been increasing demand for multiview video transmission over band limited channel over past years and various techniques have been proposed to fulfil this need. In this paper, a High Efficiency Video Codec (HEVC) based spatial resolution scaling type of mixed resolution coding model, MRHEVC-MVC, for frame interleaved multiview videos is presented. However, enabling the HEVC to encode video with different frame resolutions is a challenge due to the coding tree partitioning used by the codec. This has been overcome by super-imposing the low resolution replica of each full resolution frame on their respective decoded picture buffer and setting the remaining space of the frame buffer to zero. The codec’s reference frames structure is designed to efficiently encode frame interleaved multiview videos using a HEVC based mixed resolution codec. The proposed MRHEVC-MVC codec has been tested against the standard multiview extension of high efficiency video codec (MV-HEVC) for “Balloon”, “Newspaper1”, “Undo_Dancer”, “Kendo” and ““Poznan_Street” standard multiview video sequences. Results show that the proposed codec gives significantly higher coding performance to that of the MV-HEVC codec at low bitrate both subjectively and objectively.

GPU-accelerated depth codec for real-time, high-quality light field reconstruction

Pre-calculated depth information is essential for efficient light field video rendering, due to the prohibitive cost of depth estimation from color when real-time performance is desired. Standard state-of-the-art video codecs fail to satisfy such performance requirements when the amount of data to be decoded becomes too large. In this paper, we propose a depth image and video codec based on block compression, that exploits typical characteristics of depth streams, drawing inspiration from S3TC texture compression and geometric wavelets. Our codec offers very fast hardware-accelerated decoding that also allows partial extraction for view-dependent decoding. We demonstrate the effectiveness of our codec in a number of multi-view 360 degree video datasets, with quantitative analysis of storage cost, reconstruction quality and decoding performance.

S-EMG Signal Compression in One-Dimensional and Two-Dimensional Approaches.

This paper presents algorithms designed for one-dimensional (1-D) and 2-D surface electromyographic (S-EMG) signal compression. The 1-D approach is a wavelet transform based encoder applied to isometric and dynamic S-EMG signals. An adaptive estimation of the spectral shape is used to carry out dynamic bit allocation for vector quantization of transformed coefficients. Thus, an entropy coding is applied to minimize redundancy in quantized coefficient vector and to pack the data. In the 2-D approach algorithm, the isometric or dynamic S-EMG signal is properly segmented and arranged to build a 2-D representation. The high efficient video codec is used to encode the signal, using 16-bit-depth precision, all possible coding/prediction unit sizes, and all intra-coding modes. The encoders are evaluated with objective metrics, and a real signal data bank is used. Furthermore, performance comparisons are also shown in this paper, where the proposed methods have outperformed other efficient encoders reported in the literature.

Efficient Transmission of Multiple Broadcasting Services Using LDM and SHVC

In this paper, we present an efficient transmission method of multiple broadcasting services using layered division multiplexing (LDM) and scalable high efficiency video coding (SHVC) based on the next generation terrestrial digital broadcasting standard, advanced television systems committee 3.0. In a two-layer LDM system, each layer forms a physical layer pipe that carries base layer and enhancement layer video streams of SHVC, respectively. This combination of physical and presentation layers technologies can maximize channel utilization when multiple broadcasting services are delivered in a single radio frequency channel with different robustness and reception conditions. An intensive performance analysis is provided when the proposed combination of LDM and SHVC is compared with other approaches, such as time division multiplexing and/or high efficiency video coding simulcast broadcasting. Several performance measurements are also introduced, such as constant data rate measurement and constant quality measurement, which are used to evaluate the performance of video codecs in conjunction with different physical layer system parameters. A prototype hardware system with LDM and SHVC capable of selecting different parameter combinations is tested in the laboratory and in real field environments to verify the performance and feasibility of the proposed LDM and SHVC combination. Results show that the proposed LDM and SHVC combination provides significant gains on video service quality as well as reception robustness.

Efficient decoding algorithm for 3D video over wireless channels

In wireless communication networks, additive channel noise and time varying multipath fading are considered amongst the main challenges and these effects will bring significant quality degradation to the transmitted 2D/3D video streams. This paper presents an efficient decoding algorithm for Distributed Video Coding (DVC) for enhanced performance of colour and depth based 3D video over error prone wireless channels. The maximum a-posteriori (MAP) algorithm for turbo decoding is modified considering the effects of the channel errors on both Wyner-Ziv and key frame bit streams of colour and depth videos. The proposed codec is simulated using a W-CDMA wireless channel model and the results are analyzed to determine the effect of the modifications. The performance of the state-of-the-art H.264/AVC video codec is also presented for comparison under similar conditions. The results show that the proposed modifications provide a significant improvement in the DVC codec performance under unfavourable channel conditions for colour plus depth based 3D video.

Pixelwise adaptive prediction-based lossless reference frame compression algorithm for video coding

With the increasing size of video sequences, the dramatically increased access bandwidth and power consumption for fetching and storing reference frames with external memory during motion estimation become the bottleneck of realizing a practical video codec system. To reduce the access bandwidth and power consumption with no quality loss for reference frames, a lossless reference frame compression method based on pixelwise adaptive prediction is proposed. The algorithm allocates pixels from the first line and first column with horizontal prediction and vertical prediction separately. Meanwhile, two direction coefficients of horizontal and vertical are calculated to evaluate the similarity between two adjacent samples. For pixels in the rest locations, prediction value is assigned to every pixel adaptively from left, upper, and upper left adjacent reconstructed pixels according to the appropriate mode decision scheme based on similarity coefficients. At last, residuals are compressed by Huffman encoding process to generate bit-stream stored into external memory. By implementing the algorithm with high-efficiency video coding (HEVC) reference software HEVC test model-16.5 (HM-16.5), the experimental results show that the method can obtain 63.83% data reduction ratio averagely for luminance component.

Intelligent video surveillance for real time energy savings in smart buildings using HEVC compressed domain features

This work presents a unique approach to utilize the existing video surveillance infrastructure to optimize electricity consumption in large indoor spaces such as library reading halls, waiting rooms, indoor sports complex, large dormitories etc. The proposed method extracts features from the encoder loop of the digital video recorder (DVR) to analyze the foreground activity. High Efficiency Video Coding (HEVC) is chosen as the target video codec, as it is going to be the most widely deployed video codec in the future DVRs. The bit rate efficiency of HEVC is almost double in comparison to H.264/AVC for same video quality [17]. The proposed method utilizes only compressed domain parameters from video stream such as motion vectors clustering information, block partitioning modes etc., which allows to develop algorithms with relatively lower computational complexity and memory requirement than pixel based methods thus achieving real time performance.

Light-Weight Video Codec at Terminal for Video Coding in the Cloud

This paper proposes a new video transmission strategy that effectively transfers the computational complexity of video coding from the terminal to cloud environment, and this network architecture can generate an observably result: Light-weight Video Coding (LVC) codec at the terminal. The proposed network architecture effectively integrates Wyner-Ziv Video Coding (WZVC), which develops to construct a simple encoder in the uplink. And also we use Decoding-Friendly Encoder Design (DFED) in the downlink, which implements a decoder is lower complexity than the existing decoders. The experimental results show that the computational loading of the encoder is lower than that of H.264/AVC intra-frame coding. Meanwhile, the proposed WZVC architecture shows better performance than the state-of-the-art WZVC (DISCOVER) by maximum up to 4 dB. Moreover, DFED decreases a maximum 2% of the computational loading at the decoder, as compared with traditional H.264/AVC video decoding.

Performance assessment of three-dimensional video codecs in mobile terminals

Understanding the most relevant factors influencing the end user experience is key to design and manage multimedia services in mobile networks. Thus, user demand can be dealt with efficient strategies of resource management. In this paper, a comprehensive analysis is carried out to evaluate the Quality of Experience (QoE) obtained by three-dimensional (3D) video codecs in mobile terminals. The analysis covers the most widespread 3D video coding formats, namely the frame-compatible Side-by-Side (SbS) and Multiview Video Coding (MVC) schemes. The analysis considers both subjective and objective measurements over compression strategies that combine bit rate reduction and frame-rate dropping. Subjective tests are done by presenting 3D video sequences with different coding parameters to users who judge image quality, depth perception and visual comfort. For this purpose, a mobile phone with autostereoscopic screen is used. Then, mean opinion scores are compared with objective measurements obtained by a video quality measurement tool. Results show that MVC outperforms SbS in terms of image quality. However, when internal parameter settings are set to very restrictive configurations, the impairment of the original image are similar for both codecs. Likewise, results show that the reduction of the video bit rate is the key parameter controlling video quality, and that the use of frame-rate dropping is not enough to counteract the impairment introduced by bit rate reduction. Finally, it is confirmed that a simple QoE indicator for 3D video can be obtained from objective measurements.

AN INVESTIGATION OF VIDEO COMMUNICATION OVER BANDWIDTH LIMITED PUBLIC SAFETY NETWORK

Application performance of video-based services for public safety and security (PSS) systems is of high importance in order to protect citizens, organizations, and institutions against threats to their well-being. Video based services help PSS personnel to efficiently perform their rescue operations. Real-time video streaming of incident-based services over the PSS communication networks facilitates in realistic assessment of the situations and saves many precious lives. However, video communication over PSS networks, such as Terrestrial Trunked Radio (TETRA), is a big challenge due to the narrowband and the hostile environment. Thus, specialized codecs have been developed considering the constraints of the PSS networks. This paper presents performance comparisons of two video codecs, namely H.264/AVC and MPEG-4, over TEDS links. The significance of the results is evaluated using SPSS (t-test) to critically analyse the data. It is observed that video quality offered by H.264/AVC over TEDS links is better than the one offered by MPEG-4. Moreover, it is also noticed that TEDS bandwidth limitations substantially affect the quality of reconstructed video containing frequent changes of scenes. Based on the findings of our analyses, we propose a set of applications which are suitable for TEDS networks. The findings of this study would help the researchers to further explore the possibilities of video communication over TEDS links by optimizing the lower level network technologies.

Experimental performance analysis of multipath TCP path characteristics in streaming UHD HEVC-dash over wireless networks

The multiple interfaces in modern devices today have given great promise in enhancing multimedia service delivery over wireless networks. However, the IP coupled nature of the TCP/IP protocol inhibits the simultaneous use of these interfaces. Multipath TCP (MPTCP), a protocol undergoing IETF standardization has been developed to simultaneously use multiple interfaces for service delivery over the Internet. In this paper, we studied the effect of MPTCP paths characteristics in streaming Ultra High Definition, High Efficiency Video Codec (UHD HEVC-DASH) while connected to multiple wireless access points. The findings though specific instead of general, reveal that under balanced and unbalanced network paths, MPTCP offers good Quality of Experience (QoE) compared to SPTCP. However, with variability in latency, and packet loss between paths, MPTCP underperforms compared to SPTCP in terms of video buffering in the unbalanced network case, and high packet retransmission rate in either balanced or unbalanced network paths.

Parallel Content-Aware Adaptive Quantization-Oriented Lossy Frame Memory Recompression for HEVC

Since the development of ultrahigh-definition video, the huge bandwidth and power requirements of external memory have hindered the development of video encoder applications. Power constraints have become a particularly serious problem for portable video codec systems. With high-rate configurations [quantization parameter (QP) ≤ 22 in HEVC test model (HM) reference software], the compression performance of the existing lossless compression algorithms noticeably degrades, because the reference frames are becoming rich of textures. On the other hand, the mathematical analysis of this paper revealed that more quantization noises can be endured by the texture-rich area. Therefore, we develop an adaptive quantization-oriented parallel lossy frame memory recompression algorithm. The contributions of this paper include the following. First, a content-aware adaptive quantization method is devised to achieve a stable high compression ratio that does not deteriorate for highly quality texture-rich pictures. When QP $\in $ [12, 22], a data reduction ratio improvement of up to 14% is obtained compared with the best lossless algorithm. Furthermore, it can reduce the quality loss by 0.49–3.36 dB in terms of Bjontegaard delta peak signal-to-noise rate (BD-PSNR) compared with the fixed length quantization method. Second, to solve the low throughput problem caused by the pixel-grain prediction method, a parallel directional prediction scheme is developed. It can double or quadruple the throughput with a prediction accuracy loss of only 1.7% or 3.3%, respectively. Using the above-mentioned methods, bandwidth and memory requirements are reduced up to 70.6% and 41.0%, respectively, with a corresponding savings of 59.3% in the dynamic power consumption of the off-chip dynamic random access memory, while the BD-PSNR is −0.04 dB, or, equivalently, Bjontegaard delta bit rate (BD-BR) is 1.27%. Using TSMC 65-nm CMOS technology, the proposed frame memory compressor and decompressor can achieve the throughputs of up to 2.89 and 2.26 Gpixels/s, respectively. It is applicable to a Super Hi-Vision(8K)@68-frames/s real-time encoding with a Level D reference data reuse scheme.

Performance Evaluation of WebRTC-based Video Conferencing

WebRTC has quickly become popular as a video conferencing platform, partly due to the fact that many browsers support it. WebRTC utilizes the Google Congestion Control (GCC) algorithm to provide congestion control for realtime communications over UDP. The performance during a WebRTC call may be influenced by several factors, including the underlyingWebRTC implementation, the device and network characteristics, and the network topology. In this paper, we perform a thorough performance evaluation of WebRTC both in emulated synthetic network conditions as well as in real wired and wireless networks. Our evaluation shows that WebRTC streams have a slightly higher priority than TCP flows when competing with cross traffic. In general, while in several of the considered scenarios WebRTC performed as expected, we observed important cases where there is room for improvement. These include the wireless domain and the newly added support for the video codecs VP9 and H.264 that does not perform as expected.

Efficiency improvement of coding method by intraframe prediction in H.265 / HEVC standard

The paper presents the study of video data encoding methods by intraframe prediction in N. 265/HEVC standard. A rapid decision-making scheme for intraframe predictions is proposed. According to the proposed scheme the complexity of the texture block coding is firstly calculated and the threshold value for rapid selection of the block coding size is set. That reduces the number of possible prediction modes. The existing and proposed encoding methods were tested on several video sequences by modeling in Visual C++ environment. To evaluate the methods criteria of video sequence quality (PSNR), bitrate and encoding time were used. The results showed that compared with the reference software of HEVC – HM 16.0, the proposed scheme gives the possibility to reduce the encoding time by approximately 13.6%, while the scheme suffers from a negligible average PSNR loss (dB 0.0033). The application of the proposed approach can reduce significantly the encoding time instead of the traditional encoding in the H. 265/HEVC video codec and can be recommended for the further study by the JCT-VC (Joint Collaborative Team on video Coding).

A High-Speed VLSI Architecture for Motion Estimation Using Modified Adaptive Rood Pattern Search Algorithm

The paper presents an efficient VLSI architecture for fast Motion Estimation in video codec using modified Adaptive Rood Pattern Search Algorithm. The proposed architecture uses an interleaved memory arrangement and an early check technique to compute the Sum of Absolute Differences. The proposed design can process High Definition (1080p) video frames in real time while optimizing the hardware area. The architecture has been implemented in verilog HDL and mapped to 45 nm FPGA. It uses only 6.8K gates for the implementation of the datapath and the controller. It achieves a maximum frequency of 120 MHz. However, working at 100 MHz, it is able to process 60 HD ( $$1920\times 1080$$ ) frames per second while consuming 39 mW of power. The proposed architecture achieves premium speed with an optimum power and area requirements and can be suitably incorporated in light-weight video-intensive devices like smart-phones, tablet computers.