Wavelet-based Scalable Video Coding Research Articles

Content Adaptive Lagrange Multiplier Selection for Rate-Distortion Optimization in 3-D Wavelet-Based Scalable Video Coding.

Rate-distortion optimization (RDO) plays an essential role in substantially enhancing the coding efficiency. Currently, rate-distortion optimized mode decision is widely used in scalable video coding (SVC). Among all the possible coding modes, it aims to select the one which has the best trade-off between bitrate and compression distortion. Specifically, this tradeoff is tuned through the choice of the Lagrange multiplier. Despite the prevalence of conventional method for Lagrange multiplier selection in hybrid video coding, the underlying formulation is not applicable to 3-D wavelet-based SVC where the explicit values of the quantization step are not available, with on consideration of the content features of input signal. In this paper, an efficient content adaptive Lagrange multiplier selection algorithm is proposed in the context of RDO for 3-D wavelet-based SVC targeting quality scalability. Our contributions are two-fold. First, we introduce a novel weighting method, which takes account of the mutual information, gradient per pixel, and texture homogeneity to measure the temporal subband characteristics after applying the motion-compensated temporal filtering (MCTF) technique. Second, based on the proposed subband weighting factor model, we derive the optimal Lagrange multiplier. Experimental results demonstrate that the proposed algorithm enables more satisfactory video quality with negligible additional computational complexity.

Quality Scalability Aware Watermarking for Visual Content.

Scalable coding-based content adaptation poses serious challenges to traditional watermarking algorithms, which do not consider the scalable coding structure and hence cannot guarantee correct watermark extraction in media consumption chain. In this paper, we propose a novel concept of scalable blind watermarking that ensures more robust watermark extraction at various compression ratios while not effecting the visual quality of host media. The proposed algorithm generates scalable and robust watermarked image code-stream that allows the user to constrain embedding distortion for target content adaptations. The watermarked image code-stream consists of hierarchically nested joint distortion-robustness coding atoms. The code-stream is generated by proposing a new wavelet domain blind watermarking algorithm guided by a quantization based binary tree. The code-stream can be truncated at any distortion-robustness atom to generate the watermarked image with the desired distortion-robustness requirements. A blind extractor is capable of extracting watermark data from the watermarked images. The algorithm is further extended to incorporate a bit-plane discarding-based quantization model used in scalable coding-based content adaptation, e.g., JPEG2000. This improves the robustness against quality scalability of JPEG2000 compression. The simulation results verify the feasibility of the proposed concept, its applications, and its improved robustness against quality scalable content adaptation. Our proposed algorithm also outperforms existing methods showing 35% improvement. In terms of robustness to quality scalable video content adaptation using Motion JPEG2000 and wavelet-based scalable video coding, the proposed method shows major improvement for video watermarking.

An improved 3D wavelet-based scalable video coding codec for MC-EZBC

With the rapid growth of modern multimedia applications, 3D wavelet-based scalable video coding (SVC) codec has received considerable attention lately because of its high coding performance and flexibility in bitstream scalability. It combines the motion-compensated temporal filtering (MCTF) together with the spatial decomposition to produce an embedded bitstream offering various levels of video quality over the heterogeneous networks. However, in the existing 3D wavelet-based SVC schemes, where the block types for block matching algorithms are limited, weighting matrices for block-wise motion compensation are fixed, and variations in activities of temporal subbands are not considered in the selection of the Lagrange multiplier for mode decision. In this paper, our major contribution is to provide some recent extensions to the well-known scalable subband/wavelet video codec Motion-Compensated Embedded Zero Block Coding (MC-EZBC) using three novel and content adaptive algorithms. Firstly, the enhanced hierarchical variable size block matching (Enhanced HVSBM) algorithm is proposed for the variable block size motion estimation. Then, the rate-distortion optimization (RDO) based adaptive Lagrange multiplier selection model for mode decision is presented. Finally, we introduce the adaptive weighting matrices design for overlapped block motion compensation (OBMC). Experimental results show that all the three proposed algorithms significantly improve the overall coding performance of MC-EZBC. Comparisons with other popular wavelet-based SVC codecs demonstrate the effectiveness of our improved codec in terms of both video quality assessment and computational complexity.

An Efficient VLSI Architecture of Embedded Sub-Band Coding with Optimal Truncation for Scalable Video Coding

This work presents a new VLSI hardware architecture of Embedded Sub-band Coding with Optimal Truncation (ESCOT) with Pass Concurrent Context Modeling (PCCM) scheme for wavelet-based Scalable Video Coding (SVC). PCCM can merge the three-pass process of bit-plane coding into a single pass process. It improves the efficiency of the ESCOT algorithm and reduces the frequencies of memory access, which can reduce the power consumption. Furthermore we use the parallel architecture scheme of PCCM to encode four samples concurrently, which improves the operation speed and can reduce 40% of internal memory requirement. As a result, indicate that PCCM can have an operation speedup of 9.4 compared to the standard context modeling of ESCOT, and it can support high processing rates for 1080p with a processing rate of 125 MHz, a frame rate of 30 fps. Due to the characteristics of low memory size and high operation speed, it is suitable for VLSI implementation and can support various real-time video applications such as SVC.

움직임 벡터 필드의 상관도 향상을 통한 효과적인 MCTF 방법

본 논문에서는 웨이블릿 변환 기반의 스케일러블 비디오 부호화의 핵심 요소인 움직임 보상 시간적 필터링 (motion compensated temporal filtering; MCTF)의 성능향상을 위한 효과적인 움직임 예측 기법을 제안한다. MCTF의 성능향상과 밀접한 관련이 있는 픽셀 연결성을 개선하기 위하여 변형 중간값 연산 및 주변 블록의 움직임 벡터를 이용한 움직임 예측 방법으로 움직임 벡터 필드를 평탄화한다. 또한 영상의 복잡도 등의 특성에 따라 가변 블록크기로 움직임 예측을 수행하여 움직임 벡터 필드의 상관도를 더욱 향상시킨다. 제안하는 방법은 시간적 필터링 과정에서 고주파 프레임으로 분해되는 에너지를 줄여 MCTF 및 전체 스케일러블 부호기의 성능을 향상시킨다. 실험결과 기존의 고정 블록크기 전역탐색 방법과 비교하여 제안하는 방법이 시간적 고주파 부대역 프레임에 포함된 에너지를 최대 30.33%까지 감소시키는 것을 확인하였다. This paper presents an effective motion estimation to improve the performance of the motion compensated temporal filtering (MCTF) which is a core part of the wavelet-based scalable video coding. The proposed scheme makes the motion vector field uniform by the modified median operation and the search strategies using adjacent motion vectors, in order to enhance the pixel connectivity which is significantly relevant to the performance of the MCTF. Moreover, the motion estimation with variable block sizes that reflects the features of frames is introduced for further correlation improvement of the motion vector field. Experimental results illustrate that the proposed method reduces the decomposed energy on the temporal high frequency subband frame up to 30.33% in terms of variance compared to the case of the full search with fixed block sizes.

Transcoding From Hybrid Nonscalable to Wavelet-Based Scalable Video Codecs

Scalable video coding (SVC) enables low complexity adaptation of compressed video, providing an efficient solution for content delivery through heterogeneous networks and to diverse displays. However, legacy video and most commercially available content capturing devices use conventional nonscalable coding, e.g., H.264/AVC. This paper proposes an efficient transcoder from H.264/AVC to a wavelet-based SVC. It aims at exploiting the advantages offered by fine granularity SVC technology when dealing with conventional coders and legacy video. The proposed transcoder was developed to cope with important functionalities of H.264/AVC, such as flexible reference frame (RF) selection. It is able to work with different coding configurations of H.264/AVC, including IPP or IBBP with multiple RFs. Moreover, many of the techniques presented in this paper are generic in the sense that they can be used for transcoding with many popular wavelet-based and hybrid-based video coding architectures. To reduce the transcoder's complexity, motion information and residual data extracted from a compressed H.264/AVC stream are exploited. Experimental results show a very good performance of the proposed transcoder in terms of decoded video quality and system complexity.

Rate-Distortion Optimized Bitstream Extractor for Motion Scalability in Wavelet-Based Scalable Video Coding

Motion scalability is designed to improve the coding efficiency of a scalable video coding framework, especially in the medium to low range of decoding bit rates and spatial resolutions. In order to fully benefit from the superiority of motion scalability, a rate-distortion optimized bitstream extractor, which determines the optimal motion quality layer for any specific decoding scenario, is required. In this paper, the determination process first starts off with a brute force searching algorithm. Although guaranteed by the optimal performance within the search domain, it suffers from high computational complexities. Two properties, i.e., the monotonically nondecreasing property and the unimodal property, are then derived to accurately describe the rate-distortion behavior of motion scalability. Based on these two properties, modified searching algorithms are proposed to reduce the complexity (up to five times faster) and to achieve the global optimality, even for those decoding scenarios outside the search domain.

Lagrange multiplier selection in wavelet-based scalable video coding for quality scalability

In this paper, a method for Lagrange multiplier selection is proposed in the context of rate-distortion optimisation for wavelet-based scalable video coding targeting quality scalability. Despite the prevalence of the conventional method for Lagrange multiplier selection in hybrid video coding, the underlying formulation is not applicable to wavelet-based scalable video coding. To address the inherent challenges, a thorough analysis of the rate-distortion models for transform video coding is provided with regard to low and middle-to-high bit-rates, respectively. Based on the analysis, the models are consolidated according to experimental observations and the consolidated rate-distortion models serve as the basis for the derivation of the Lagrange multiplier. Considering the influence of the open-loop prediction structure on the rate-distortion performance, the Lagrange multiplier is initially derived for a single-targeted bit-rate. Moreover, the method for Lagrange multiplier selection in scalable video coding aiming at multiple-targeted bit-rates is proposed in a general sense of bit-rate range, varying from low to high bit-rates, building on the initially derived Lagrange multiplier for a single-targeted bit-rate. The proposed Lagrange multiplier is content adaptive and well suited for wavelet-based scalable video coding where quantisation steps are unavailable. Detailed performance evaluation of the proposed method for wavelet-based scalable video coding is provided with regard to a given targeted bit-rate and multiple-targeted bit-rates, respectively. The experimental results have demonstrated the effectiveness of the proposed Lagrange multiplier for rate-distortion optimisation considering quality scalability in wavelet-based scalable video coding.

Visually Lossless Accuracy of Motion Vector in Overcomplete Wavelet-based Scalable Video Coding

In this paper, we propose a visually lossless accuracy model for scalable coding of motion vectors in overcomplete wavelet-based scalable video codec. By exploiting theory of stationary random process, we first estimate motion compensation errors in spatial domain due to inaccurate motion vectors. We then extend the results to overcomplete wavelet domain, and further derive the errors caused by fraction-pixel motion vectors. Finally, combining with a visibility model of wavelet coefficient errors, we propose a novel algorithm to estimate the accuracy threshold of motion vectors with which motion compensation errors will be invisible. Experimental results show that our algorithm is effective in estimating visually lossless accuracy threshold of motion vectors. The proposed algorithm can also be used in scalable motion estimation in wavelet domain. It can accelerate motion estimation speed by stopping halfway at the accuracy that will not cause any visible errors.

Influence of downsampling filter characteristics on compression performance in wavelet-based scalable video coding

The application of different downsampling filters in video coding directly models visual information at lower resolutions and influences the compression performance of a chosen coding system. In wavelet-based scalable video coding the spatial scalability is achieved by the application of wavelets as downsampling filters. However, characteristics of different wavelets influence the performance at targeting spatio-temporal decoding points. An analysis of different downsampling filters in popular wavelet-based scalable video coding schemes is presented. Evaluation is performed for both intra- and inter-coding schemes using wavelets and standard downsampling strategies. On the basis of the obtained results a new concept of inter-resolution prediction is proposed, which maximises the average performance using a combination of standard downsampling filters and wavelet-based coding.

Fast multiresolution motion estimation algorithms for wavelet-based scalable video coding

Motion estimation and compensation in wavelet domain have received much attention recently. To overcome the inefficiency of motion estimation in critically sampled wavelet domain, the low-band-shift (LBS) method and the complete-to-overcomplete discrete wavelet transform (CODWT) method are proposed for motion estimation in shift-invariant wavelet domain. However, a major disadvantage of these methods is the computational complexity. Although the CODWT method has reduced the computational complexity by skipping the inverse wavelet transform and making the direct link between the critically sampled subbands and the shift-invariant subbands, the full search algorithm (FSA) increases it. In this paper, we proposed two fast multiresolution motion estimation algorithms in shift-invariant wavelet domain: one is the wavelet matching error characteristic based partial distortion search (WMEC-PDS) algorithm, which improves computational efficiency of conventional partial distortion search algorithms while keeping the same estimate accuracy as the FSA; another is the anisotropic double cross search (ADCS) algorithm using multiresolution-spatio-temporal context, which provides a significantly computational load reduction while only introducing negligible distortion compared with the FSA. Due to the multiresolution nature, both the proposed approaches can be applied to wavelet-based scalable video coding. Experimental results show the superiority of the proposed fast motion estimation algorithms against other fast algorithms in terms of speed-up and quality.

Bit-stream allocation methods for scalable video coding supporting wireless communications

The demand for video access services through wireless networks, as important parts of larger heterogeneous networks, is constantly increasing. To cope with this demand, flexible compression technology to enable optimum coding performance, especially at low bit-rates, is required. In this context, scalable video coding emerges as the most promising technology. A critical problem in wavelet-based scalable video coding is bit-stream allocation at any bit-rate and in particular when low bit-rates are targeted. In this paper two methods for bit-stream allocation based on the concept of fractional bit-planes are reported. The first method assumes that minimum rate-distortion (R–D) slope of the same fractional bit-plane within the same bit-plane across different subbands is higher than or equal to the maximum R–D slope of the next fractional bit-plane. This method is characterised by a very low complexity since no distortion evaluation is required. Contrasting this approach, in the second method the distortion caused by quantisation of the wavelet coefficients is considered. Here, a simple yet effective statistical distortion model that is used for estimation of R–D slopes for each fractional bit-plane is derived. Three different strategies are derived from this method. In the first one it is assumed that the used wavelet is nearly orthogonal, i.e. the distortion in the transform domain is treated as being equivalent to the distortion in the signal domain. To reduce the error caused by direct distortion evaluation in the wavelet domain, the weighting factors are applied to the used statistical distortion model in the second strategy. In the last strategy, the derived statistical model is used during the bit-plane encoding to determine optimal position of the fractional bit-plane corresponding to refinement information in the compressed bit-stream. Results of selected experiments measuring peak signal to noise ratio (PSNR) of decoded video at various bit-rates are reported. Additionally, the PSNR of decoded video at various bit-rates is measured for two specific cases: when the methods for bit-stream allocation are used to assign quality layers in the compressed bit-stream, and when quality layers are not assigned.