Wavelet-based Video Research Articles

Wavelet-Based Fast Decoding of 360 ° Videos.

In this paper, we propose a wavelet-based video codec specifically designed for VR displays that enables real-time playback of high-resolution 360° videos. Our codec exploits the fact that only a fraction of the full 360° video frame is visible on the display at any time. To load and decode the video viewport-dependently in real time, we make use of the wavelet transform for intra- as well as inter-frame coding. Thereby, the relevant content is directly streamed from the drive, without the need to hold the entire frames in memory. With an average of 193 frames per second at 8192 × 8192 -pixel full-frame resolution, the conducted evaluation demonstrates that our codec's decoding performance is up to 272% higher than that of the state-of-the-art video codecs H.265 and AV1 for typical VR displays. By means of a perceptual study, we further illustrate the necessity of high frame rates for a better VR experience. Finally, we demonstrate how our wavelet-based codec can also directly be used in conjunction with foveation for further performance increase.

Analysis of Video Retinal Angiography With Deep Learning and Eulerian Magnification

Objective: The aim of this research is to present a novel computer-aided decision support tool in analyzing, quantifying, and evaluating the retinal blood vessel structure from fluorescein angiogram (FA) videos. Methods: The proposed method consists of three phases: i) image registration for large motion removal from FA videos, followed by ii) retinal vessel segmentation, and lastly, iii) segmentation-guided video magnification. In the image registration phase, individual frames of the FA video are spatiotemporally aligned using a novel wavelet-based registration approach to compensate for the global camera and patient motion. In the second phase, a capsule-based neural network architecture is employed to perform the segmentation of retinal vessels for the first time in the literature. In the final phase, a segmentation-guided Eulerian video magnification is proposed for magnifying subtle changes in the retinal video produced by blood flow through the retinal vessels. The magnification is applied only to the segmented vessels, as determined by the capsule network. This minimizes the high levels of noise present in FA videos and maximizes useful information, enabling ophthalmologists to more easily identify potential regions of pathology. Results: The collected FA video dataset consists of 1402 frames from 10 normal subjects (prospective study). Experimental results for retinal vessel segmentation show that the capsule-based algorithm outperforms a state-of-the-art CNN (U-Net), obtaining a higher dice coefficient (85.94%) and sensitivity (92.36%) while using just 5% of the network parameters. Qualitative analysis of the FA videos was performed after the final phase by expert ophthalmologists, supporting the claim that artificial intelligence assisted decision support tool can be helpful for providing a better analysis of blood flow dynamics. Conclusions: The authors introduce a novel computational tool, combining a wavelet-based video registration method with a deep learning capsule-based retinal vessel segmentation algorithm and a Eulerian video magnification technique to quantitatively and qualitatively analyze FA videos. To authors' best knowledge, this is the first-ever development of such a computational tool to assist ophthalmologists with analyzing blood flow in FA videos.

A secure communication approach in OFDM using optimized interesting region-based video watermarking

PurposeThe technique for hiding confidential data in specific digital media by enhancing the graphical contents is known as watermarking. The dissemination of information over a secure channel is essential for multimedia applications. The purpose of this study is to develop a secure communication approach for OFDM system.Design/methodology/approachThis paper exploits a secure communication in the orthogonal frequency division multiplexing (OFDM) system using wavelet-based video watermarking technique. In this work, the Chronological-MS algorithm is used for securing the data communication in the OFDM system. Here, the secret message is embedded in video frames using wavelet transform for hiding sensitive information and the hidden information is transmitted over the OFDM system. The Chronological-MS algorithm is used for selecting the optimal regions in the video for embedding secret message. In embedding phase, wavelet coefficients are obtained by applying wavelet transform on the frame for embedding the secret message. Meanwhile, in extraction phase, the inverse wavelet transform is applied to extract the secret message.FindingsConsidering number of frames, the maximum Peak signal-to-noise ratio (PSNR) value is attained by proposed Wavelet + Chronological MS method for Video 2 with value 73.643 dB, respectively. Meanwhile, the minimum mean squared error (MSE) attained by the proposed Wavelet + Chronological MS method is when considering number of frames with MSE values as 0.001 for both Videos 1 and 2. The minimum bit error rate (BER) value is attained by the proposed method with value 0.00009 considering random noise with Video 1. Thus, the proposed Wavelet + Chronological MS have shown better results than the existing techniques.Originality/valueThis work proposes a wavelet-based watermarking method using Chronological-MS, for initiating secured communication over an OFDM. One of the main advantages of wavelets is that they offer a simultaneous localization in time and frequency domain. Hence, the proposed method offers the highly secured data transmission over the OFDM.

Content Adaptive Lagrange Multiplier Selection for Rate-Distortion Optimization in 3-D Wavelet-Based Scalable Video Coding.

Rate-distortion optimization (RDO) plays an essential role in substantially enhancing the coding efficiency. Currently, rate-distortion optimized mode decision is widely used in scalable video coding (SVC). Among all the possible coding modes, it aims to select the one which has the best trade-off between bitrate and compression distortion. Specifically, this tradeoff is tuned through the choice of the Lagrange multiplier. Despite the prevalence of conventional method for Lagrange multiplier selection in hybrid video coding, the underlying formulation is not applicable to 3-D wavelet-based SVC where the explicit values of the quantization step are not available, with on consideration of the content features of input signal. In this paper, an efficient content adaptive Lagrange multiplier selection algorithm is proposed in the context of RDO for 3-D wavelet-based SVC targeting quality scalability. Our contributions are two-fold. First, we introduce a novel weighting method, which takes account of the mutual information, gradient per pixel, and texture homogeneity to measure the temporal subband characteristics after applying the motion-compensated temporal filtering (MCTF) technique. Second, based on the proposed subband weighting factor model, we derive the optimal Lagrange multiplier. Experimental results demonstrate that the proposed algorithm enables more satisfactory video quality with negligible additional computational complexity.

Wavelet-based frame video coding algorithms using fovea and SPECK

Current video coding standards such as the H.264 or Advanced Video Coding (AVC) and the H.265 or High Efficiency Video Coding (HEVC) are based on the Integer Discrete Cosine Transform (IDCT). However, it has been shown that wavelet-based algorithms have better performance than IDCT-based algorithms for still images. Also, exploiting the human visual system characteristics in stationary point of view image and video transmissions like the fovea aliasing can improve the quality of the reconstructed image. In this paper, two wavelet-based video coding approaches called SPECK-based Codec (SP-Codec) and Adaptive Wavelet/Fovea-based Codec (AWFV-Codec) are proposed. The proposed SP-Codec approach applies the Set Partitioned Embedded Block Codec (SPECK) wavelet-based compression algorithm in order to increase the intra-frame coding quality. The second proposed approach AWFV-Codec uses the Adaptive Fovea Set Partitioned Embedded Block Codec (AFV-SPECK) fovea wavelet-based compression algorithm for intra-frame coding. Fovea based compression allows to increase the quality of the reconstructed frames over regions of interest (ROI). A comparison of the proposed algorithms against the HEVC based on IDCT approach shows how the proposed wavelet-based schemes achieve higher compression ratios and higher reconstruction quality. When fixation point of view is stationary, the proposed wavelet/fovea-based algorithm achieves better quality at lower compression ratios than the wavelet-based only proposed algorithm.

An Efficient VLSI Architecture of Embedded Sub-Band Coding with Optimal Truncation for Scalable Video Coding

This work presents a new VLSI hardware architecture of Embedded Sub-band Coding with Optimal Truncation (ESCOT) with Pass Concurrent Context Modeling (PCCM) scheme for wavelet-based Scalable Video Coding (SVC). PCCM can merge the three-pass process of bit-plane coding into a single pass process. It improves the efficiency of the ESCOT algorithm and reduces the frequencies of memory access, which can reduce the power consumption. Furthermore we use the parallel architecture scheme of PCCM to encode four samples concurrently, which improves the operation speed and can reduce 40% of internal memory requirement. As a result, indicate that PCCM can have an operation speedup of 9.4 compared to the standard context modeling of ESCOT, and it can support high processing rates for 1080p with a processing rate of 125 MHz, a frame rate of 30 fps. Due to the characteristics of low memory size and high operation speed, it is suitable for VLSI implementation and can support various real-time video applications such as SVC.

A Novel Motion Field Anchoring Paradigm for Highly Scalable Wavelet-Based Video Coding.

Existing video coders anchor motion fields at frames that are to be predicted. In this paper, we demonstrate how changing the anchoring of motion fields to reference frames has some important advantages over conventional anchoring. We work with piecewise-smooth motion fields, and use breakpoints to signal discontinuities at moving object boundaries. We show how discontinuity information can be used to resolve double mappings arising when motion is warped from reference to target frames. We present an analytical model that allows to determine weights for texture, motion, and breakpoints to guide the rate-allocation for scalable encoding. Compared with the conventional way of anchoring motion fields, the proposed scheme requires fewer bits for the coding of motion; furthermore, the reconstructed video frames contain fewer ghosting artefacts. The experimental results show the superior performance compared with the traditional anchoring, and demonstrate the high scalability attributes of the proposed method.

Transcoding From Hybrid Nonscalable to Wavelet-Based Scalable Video Codecs

Scalable video coding (SVC) enables low complexity adaptation of compressed video, providing an efficient solution for content delivery through heterogeneous networks and to diverse displays. However, legacy video and most commercially available content capturing devices use conventional nonscalable coding, e.g., H.264/AVC. This paper proposes an efficient transcoder from H.264/AVC to a wavelet-based SVC. It aims at exploiting the advantages offered by fine granularity SVC technology when dealing with conventional coders and legacy video. The proposed transcoder was developed to cope with important functionalities of H.264/AVC, such as flexible reference frame (RF) selection. It is able to work with different coding configurations of H.264/AVC, including IPP or IBBP with multiple RFs. Moreover, many of the techniques presented in this paper are generic in the sense that they can be used for transcoding with many popular wavelet-based and hybrid-based video coding architectures. To reduce the transcoder's complexity, motion information and residual data extracted from a compressed H.264/AVC stream are exploited. Experimental results show a very good performance of the proposed transcoder in terms of decoded video quality and system complexity.

Denoising algorithm for the 3D depth map sequences based on multihypothesis motion estimation

This article proposes an efficient wavelet-based depth video denoising approach based on a multihypothesis motion estimation aimed specifically at time-of-flight depth cameras. We first propose a novel bidirectional block matching search strategy, which uses information from the luminance as well as from the depth video sequence. Next, we present a new denoising technique based on weighted averaging and wavelet thresholding. Here we take into account the reliability of the estimated motion and the spatial variability of the noise standard deviation in both imaging modalities. The results demonstrate significantly improved performance over recently proposed depth sequence denoising methods and over state-of-the-art general video denoising methods applied to depth video sequences.

Scalable Video Compression Framework With Adaptive Orientational Multiresolution Transform and Nonuniform Directional Filterbank Design

Although wavelet-based scalable video coding becomes the state-of-the-art video compression engine for its adaptability to heterogeneous networks and clients, a large number of attempts have been made to integrate local directionality onto discrete wavelet transform to explore the intrinsic geometrical structures. Taking into consideration that the contours and textures scattered in different scales change their directional resolutions as their curvatures change, we investigate adaptive directional resolutions along scales to achieve the dual (scale and orientation) multiresolution transform. This paper proposes nonuniform directional frequency decompositions for video representation and approximation, and exploits the nonuniformity of orientation multiresolution distribution and designs nonuniform directional filter banks to make the geometrical transform more sparse and efficient. The nonuniform directional frequency decomposition under arbitrary scales is fulfilled by a non-symmetric binary tree (NSBT) topology structure with nonuniform directional filterbank design. In turn, the proposed scalable video coding framework, called DMSVC, is enriched with the dual multiresolution transform. Each temporal subband through motion compensated temporal filtering is further decomposed into multiscale subbands, and the highpass wavelet subspaces are divided into an arbitrary number of directional subspaces in alignment with the orientation distribution via phase congruency to establish NSBT. The paraunitary perfect reconstruction condition is provided through a polyphase identical form of filter bank. Comparing with the isolated wavelet basis, our transform provides a greater correlated set of localized and anisotropic basis functions. The spatio-temporal subband coefficients are coded by a 3-D ESCOT entropy coding algorithm which is adopted to match the structure of NSBT. Experimental results show that the reconstructed video frames DMSVC in the proposed DMSVC scheme have better visual quality than existing scalable video coding schemes. It could produce higher compression ratio on video sequences full of directional edges and textures.

Methods and Tools for Wavelet-Based Scalable Multiview Video Coding

A wavelet-based multiview video coding scheme is presented in this paper. It uses a 4-D wavelet transform, which is composed of a 1-D temporal wavelet transform, namely motion compensated temporal filtering, a 1-D view-directional wavelet transform, namely disparity compensated view filtering and a 2-D spatial wavelet transform. Since the presented framework can make use of the inherent scalability properties of the wavelet transforms involved, it allows full scalability of the coded bitstream in the temporal, view, spatial, and quality dimensions. Coding performance close to the H.264/advanced video coding based standard multiview video codec is shown. Enhancements of the view transform, in order to better account for brightness and color variations across views are introduced. Additionally, the use of a signal adaptive anisotropic wavelet packet (WP) transform as a generalization of WP transforms for the spatial decomposition is proposed. Both enhancements lead to a decrease of bit rate of up to 11% compared with the baseline version of the codec.

Reliable transmission of wavelet-based scalable video over wireless networks using cross-layer approach

The conventional network layers are mainly designed for fixed networks and are not suitable to cope the challenges offered by the modern wireless networks. Moreover, certain functionalities can be implemented simultaneously in more than one layer. For example, unequal error protection (UEP) can be performed at the application layer, using forward error correction (FEC), but it is bandwidth inefficient. On the other hand, UEP can also be performed at the physical layer using hierarchical modulation to protect higher priority bits more strongly without any additional bandwidth, but at the cost of increased errors in the lower priority bits. The authors propose a cross-layer UEP approach to enhance reliability of scalable video by combining adaptive FEC at the application layer and hierarchical quadrature amplitude modulation (HQAM) at the physical layer. The idea is to slightly reduce the FEC protection of higher priority bits and increase the FEC protection of low-priority bits in such a fashion that the overall data rate at the application layer remains unchanged. The higher priority bits are additionally protected using HQAM at the physical layer. Simulation results show significant improvements in performance when the proposed scheme is used.

Rate-Distortion Optimized Bitstream Extractor for Motion Scalability in Wavelet-Based Scalable Video Coding

Motion scalability is designed to improve the coding efficiency of a scalable video coding framework, especially in the medium to low range of decoding bit rates and spatial resolutions. In order to fully benefit from the superiority of motion scalability, a rate-distortion optimized bitstream extractor, which determines the optimal motion quality layer for any specific decoding scenario, is required. In this paper, the determination process first starts off with a brute force searching algorithm. Although guaranteed by the optimal performance within the search domain, it suffers from high computational complexities. Two properties, i.e., the monotonically nondecreasing property and the unimodal property, are then derived to accurately describe the rate-distortion behavior of motion scalability. Based on these two properties, modified searching algorithms are proposed to reduce the complexity (up to five times faster) and to achieve the global optimality, even for those decoding scenarios outside the search domain.

Interactive region of interest scalability for wavelet based scalable video coder

The current wavelet-based scalable video coder supports quality, spatial and temporal scalabilities required for recent multimedia applications. One of the key scalability required by the Joint Video Team (JVT) is Interactive Region of Interest (IROI) scalability. This paper proposes a technique to interactively extract ROI from scalable sub bit-streams. At the encoder, after sub-band decomposition the 3D wavelet tree is divided into N sub tree, each of these has spatial relevance in the original video, which can be decoded interactively. A generalized model for optimal value of N is proposed, that satisfies minimum requirement of the JVT about IROI support at reduced cost of the encoder. The ROI request from decoder along with resolution and bandwidth are mapped to that in the sub-band domain and respective sub-bit-streams are identified. The extractor assigns more bandwidth to ROI and remaining to non-ROI regions. Hence the decoder reconstructs the video with highest quality at the ROI. The proposed IROI functionality enhances the scalability of video coding by additional dimension.

Performance evaluation of wavelet-based distributed video coding schemes

In this paper we propose and compare different distributed video coding (DVC) schemes based on the use of the wavelet transform, which naturally allows for spatial and other forms of scalability. In particular, we propose a hybrid encoder which utilizes channel codes, and evaluate its performance in the absence of a feedback channel. The proposed scheme uses statistical models for the estimation of the required bitrate at the encoder. We also propose a scheme that is based on a modulo reduction procedure and does not use channel codes at the receiver/transmitter. These schemes are compared with more conventional coders that do not or only partially exploit the distributed coding paradigm. Experimental results show that the considered schemes have good performance when compared with similar asymmetric video compression schemes, and that DVC can be an interesting option in appropriate scenarios.

AN EFFICIENT RATE CONTROL ALGORITHM FOR A WAVELET VIDEO CODEC

Rate control plays an essential role in video coding and transmission to provide the best video quality at the receiver's end given the constraint of certain network conditions. In this paper, a rate control algorithm using the Quality Factor (QF) optimization method is proposed for the wavelet-based video codec and implemented on an open source Dirac video encoder. A mathematical model which we call Rate-QF (R - QF) model is derived to generate the optimum QF for the current coding frame according to the target bitrate. The proposed algorithm is a complete one pass process and does not require complex mathematical calculation. The process of calculating the QF is quite simple and further calculation is not required for each coded frame. The experimental results show that the proposed algorithm can control the bitrate precisely (within 1% of target bitrate in average). Moreover, the variation of bitrate over each Group of Pictures (GOPs) is lower than that of H.264. This is an advantage in preventing the buffer overflow and underflow for real-time multimedia data streaming.

Lagrange multiplier selection in wavelet-based scalable video coding for quality scalability

In this paper, a method for Lagrange multiplier selection is proposed in the context of rate-distortion optimisation for wavelet-based scalable video coding targeting quality scalability. Despite the prevalence of the conventional method for Lagrange multiplier selection in hybrid video coding, the underlying formulation is not applicable to wavelet-based scalable video coding. To address the inherent challenges, a thorough analysis of the rate-distortion models for transform video coding is provided with regard to low and middle-to-high bit-rates, respectively. Based on the analysis, the models are consolidated according to experimental observations and the consolidated rate-distortion models serve as the basis for the derivation of the Lagrange multiplier. Considering the influence of the open-loop prediction structure on the rate-distortion performance, the Lagrange multiplier is initially derived for a single-targeted bit-rate. Moreover, the method for Lagrange multiplier selection in scalable video coding aiming at multiple-targeted bit-rates is proposed in a general sense of bit-rate range, varying from low to high bit-rates, building on the initially derived Lagrange multiplier for a single-targeted bit-rate. The proposed Lagrange multiplier is content adaptive and well suited for wavelet-based scalable video coding where quantisation steps are unavailable. Detailed performance evaluation of the proposed method for wavelet-based scalable video coding is provided with regard to a given targeted bit-rate and multiple-targeted bit-rates, respectively. The experimental results have demonstrated the effectiveness of the proposed Lagrange multiplier for rate-distortion optimisation considering quality scalability in wavelet-based scalable video coding.

Decoupled 3-D Zerotree Structure for Wavelet-Based Video Coding

A novel data structure is proposed for magnitude-ordering 3-D wavelet transform coefficients. This data structure consists of temporal 1-D orientation trees followed by spatial 2-D orientation trees. Based on this data structure, a coding algorithm is developed for the embedded coding of 3-D wavelet transform coefficients. Experiment results confirm that the proposed coding algorithm outperforms a conventional algorithm that uses asymmetric 3-D orientation trees.

Influence of downsampling filter characteristics on compression performance in wavelet-based scalable video coding

The application of different downsampling filters in video coding directly models visual information at lower resolutions and influences the compression performance of a chosen coding system. In wavelet-based scalable video coding the spatial scalability is achieved by the application of wavelets as downsampling filters. However, characteristics of different wavelets influence the performance at targeting spatio-temporal decoding points. An analysis of different downsampling filters in popular wavelet-based scalable video coding schemes is presented. Evaluation is performed for both intra- and inter-coding schemes using wavelets and standard downsampling strategies. On the basis of the obtained results a new concept of inter-resolution prediction is proposed, which maximises the average performance using a combination of standard downsampling filters and wavelet-based coding.

A Cross-Layer Approach for Maximizing Visual Entropy Using Closed-Loop Downlink MIMO

We propose an adaptive video transmission scheme to achieve unequal error protection in a closed loop multiple input multiple output (MIMO) system for wavelet-based video coding. In this scheme, visual entropy is employed as a video quality metric in agreement with the human visual system (HVS), and the associated visual weight is used to obtain a set of optimal powers in the MIMO system for maximizing the visual quality of the reconstructed video. For ease of cross-layer optimization, the video sequence is divided into several streams, and the visual importance of each stream is quantified using the visual weight. Moreover, an adaptive load balance control, named equal termination scheduling (ETS), is proposed to improve the throughput of visually important data with higher priority. An optimal solution for power allocation is derived as a closed form using a Lagrangian relaxation method. In the simulation results, a highly improved visual quality is demonstrated in the reconstructed video via the cross-layer approach by means of visual entropy.