Pyramid Coding Research Articles

Pyramid coding based rate control for variable bit rate video streaming

Multi-pass encoding is preferred over single-pass encoding for applications such as television broadcasting and video on demand to obtain a consistent visual quality media. This study presents a novel pyramid coding based two-pass rate control scheme for variable bit rate video streaming. The proposed algorithm uses the statistics collected in the first pass to derive an optimal quantisation parameter for encoding the frames in the second pass. Simple linear estimation models are employed along with Gaussian filtering to update the frame quantisation parameter values in an adaptive manner, resulting in a smoother variation among these values and a better visual quality. The experimental results demonstrate that the proposed rate control scheme exhibits a better performance in terms of the peak signal-to-noise ratio and consistent perceptual visual quality in comparison with the existing rate control scheme in the joint model reference software while achieving the target bit rate.

SAR image target recognition via Complementary Spatial Pyramid Coding

Many works have been recently presented to extract efficient features for automatic target recognition of synthetic aperture radar (SAR) images. However, they are limited in the discriminative ability of similar targets and robustness to the remarkable speckle noises and background clutters existed in images. In this paper, we propose a Complementary Spatial Pyramid Coding (CSPC) approach in the framework of Spatial Pyramid Matching (SPM). Both the coding coefficients and coding residuals are explored to develop more discriminative and robust features for representing SAR images. Multiple codebooks are first built from some training example images, where each codebook is formulated by local features of a certain class of samples. Then multiple sparse coding models are developed to derive features of a target under these codebooks. Additionally, these coding residuals are further sparsely encoded in the same way to that of local features. Finally, the encoded local features and the residual features are pooled according to spatial pyramid respectively, then concatenated to form the complementary features for the subsequent classification. The experiments on Moving and Stationary Target Acquisition and Recognition (MSTAR) public database verify the superior performance of the proposed method to some related approaches.

Pyramid coding based rate control for constant bit rate video streaming

In this paper, a novel pyramid coding based rate control scheme is proposed for video streaming applications constrained by a constant channel bandwidth. To achieve the target bit rate with the best quality, the initial quantization parameter (QP) is determined by the average spatio-temporal complexity of the sequence, its resolution and the target bit rate. Simple linear estimation models are then used to predict the number of bits that would be necessary to encode a frame for a given complexity and QP. The experimental results demonstrate that the proposed rate control scheme significantly outperforms the existing rate control scheme in the Joint Model (JM) reference software in terms of Peak Signal to Noise Ratio (PSNR) and consistent perceptual visual quality while achieving the target bit rate. Finally, the proposed scheme is validated through experimental evaluation over a miniature test-bed.

Spatio-Temporal Laplacian Pyramid Coding for Action Recognition

We present a novel descriptor, called spatio-temporal Laplacian pyramid coding (STLPC), for holistic representation of human actions. In contrast to sparse representations based on detected local interest points, STLPC regards a video sequence as a whole with spatio-temporal features directly extracted from it, which prevents the loss of information in sparse representations. Through decomposing each sequence into a set of band-pass-filtered components, the proposed pyramid model localizes features residing at different scales, and therefore is able to effectively encode the motion information of actions. To make features further invariant and resistant to distortions as well as noise, a bank of 3-D Gabor filters is applied to each level of the Laplacian pyramid, followed by max pooling within filter bands and over spatio-temporal neighborhoods. Since the convolving and pooling are performed spatio-temporally, the coding model can capture structural and motion information simultaneously and provide an informative representation of actions. The proposed method achieves superb recognition rates on the KTH, the multiview IXMAS, the challenging UCF Sports, and the newly released HMDB51 datasets. It outperforms state of the art methods showing its great potential on action recognition.

Low-bit-rate color image encoder using an adaptive morphological pyramid decomposition

Interest in multiresolution techniques for image processing and coding is increasing steadily. An attractive instance is pyramid decomposition schemes due to their low computational complexity, simple parallel implementation, and ability to produce acceptable color images at moderate data rates. We present an approach combining a modified pyramidal algorithm and mathematical morphology for color image coding applications denoted as the adaptive morphological pyramid coding (AMPC) algorithm. The proposed AMPC pyramidal strategy involves (1) building a multilevel pyramid based on a developed adaptive morphological filter that preserves the significant details (without shape bias) and does not require very high computational cost, and then (2) the resulting pyramid levels (images) are applied to suitable coding techniques including an adaptive 3-D vector quantization (AVQ). The proposed technique gives an average peak SNR (PSNR) in the range 46.78 to 50.7 dB when no additional coding is applied. When AVQ and differential pulse code modulation (DPCM) are applied to the pyramid levels an average PSNR varies in the range 22.2 to 37.8 dB) at bit rates in the range 0.07 to 1.51 bpp is obtained. When compared with JPEG/JPEG2000, embedded zero-tree wavelet (EZW) coding and set partition in hierarchical trees (SPIHT), the proposed algorithm provides comparable reconstruction quality.

A pyramidal image coder using generalized rank-ordered prediction filter

This paper presents a lossy image compression scheme that employs a generalized rank-ordered prediction filter for pyramidal image coding. The proposed prediction method renders significantly reduced variance of the quantizer input. Consequently, the quality of the decompressed image is much enhanced due to the greatly reduced quantization distortion. Both analytical and simulation results show that the proposed scheme yields high-quality performance.

A low bit rate hybrid coding scheme for progressive image transmission

In this paper, a hybrid coding scheme for progressive image transmission (PIT) based on two components, that is the smooth and edge components, is presented. The image is first decimated by using the pyramid coding technique with a decimation factor of 4:1. A multistage predictive locally adaptive vector quantization (MSPVQ) progressively transmits the decimated image that represents the smooth component of the scene. The first stage of the proposed (VQ) gives a general grasp of the Lena image at a compression ratio as high as 80:1. An adaptive VQ (LAVQ) encodes the difference image, which contains the quantization and edge error of the smooth component. By adding the encoded image components, an excellent reproduction is achieved at a low bit rate of only 0.2 bit per pixel (bpp), i.e. compression ratio of 40:1, with peak signal-to-noise ratio (SNR) of 36.92 dB. It is also shown that, for the Lena image and for the same PSNR of about 34 dB, the bit rate using the proposed algorithm is very low; only 0.2 bpp. This bit rate is lower than about 0.6 bpp using the JPEG algorithm, and much lower than that using BTC-DCT or SB-DCT algorithms. Thus the proposed algorithm results in a saving of about 0.4 in the bit rate for the same visual quality.

Digital Television Broadcasting: Nondisruptive Improvement over Time

The Federal Communications Commission (FCC) decided, at an early date, that it was important for the digital broadcasting system to be upgradeable over time. Therefore, it established, as an important desideratum for the system to be chosen, that it could be improved in a nondisruptive manner i.e., without making the original equipment obsolete. An effective way to do this is to formulate a single baseline system that produces standard-definition imagery, and to carry out any later upgrading exclusively by adding one or more enhancement signals. Enhanced receivers would extract all data streams to create improved imagery. This approach to upgrading permits the manufacture of digital receivers (and set-top converters for NTSC receivers) of the lowest possible cost since high-definition decoding and signal-processing capability is unnecessary If the enhancement signals are transmitted by means of nonuniform constellation, then they will appear to be random noise to baseline receivers, guaranteeing that such receivers will continue to be usable regardless of the exact nature of the enhancement signal(s). While it would be preferable for the baseline system to be progressively scanned, it would be possible to permit both interlaced (I) and progressive (P) baseline formats to be used and yet to design enhanced receivers in such a way that they would operate properly with both. The source coding required in such a system is already well known by such names as pyramid coding, layered coding, or multiresolution coding. Channel-coding methods are also described that permit receivers to extract data from the transmitted signal in accordance with the local SNR and their signal-processing capabilities.

Eigen design of quadrature mirror filters

This work presents a method for the design of quadrature mirror filters. A new frequency weighted stopband energy function is introduced, which leads to considerable flexibility in the design process. Unlike other techniques which involve searches and nonlinear optimization, our formulation reduces the design equations to an eigenvector problem. Furthermore, the resulting filters are regular and have additional desirable properties for applications to pyramidal coding of images, which together with DPCM, PCM, vector quantization, or zero-tree wavelet coding schemes leads to high compression ratios.

Compression of stereoscopic images using pyramid and prune DCT encoding

This paper details a compression system for stereoscopic (3-D) images that takes advantage of the disparity compensation effect by heavily compressing one image and a lightly compressing the other. Two methods were tried for the creation of the heavily compressed image: Lossy Pyramid coding and Pruned Discrete Cosine Transform (PDCT) with Variable Length Coding VLC. Both versions of the program used a lightly quantized PDCT image for the “clear” image.

Optimum bit rate pyramid coding with low computational and memory requirements

This paper presents the optimum bit allocation for fixed rate pyramid coding structures which use a lowpass filter with nominal bandwidth of /spl pi//4 for predecimation filtering. This technique results in about 32% less in the computational complexity and 20% less in the memory requirements than the conventional filters used for pyramid coding. Three different algorithms are introduced. In the first algorithm the decimated image is transmitted at 8 bpp and the difference image is DCT-encoded. In the second algorithm the decimated image is encoded using BTC and the difference image is DCT-encoded, while in the third algorithm the decimated image is BTC-encoded with further quantization of the DCT coefficients of the difference image. These algorithms achieve high compression rate and allow at the same time a low complexity encoder. >

Low entropy image pyramids for efficient lossless coding

An efficient image source coding technique gives good compression performance at low computational complexity. This research introduces an efficient coding technique, based on pyramid coding, that involves transforming an image into an equivalent lower entropy form prior to lossless coding. The proposed method is also a multiresolution technique that facilitates progressive image transmission.

Color image coding using morphological pyramid decomposition

Presents a new algorithm that utilizes mathematical morphology for pyramidal coding of color images. The authors obtain lossy color image compression by using block truncation coding at the pyramid levels to attain reduced data rates. The pyramid approach is attractive due to low computational complexity, simple parallel implementation, and the ability to produce acceptable color images at moderate data rates. In many applications, the progressive transmission capability of the algorithm is very useful. The authors show experimental results for color images at data rates of 1.89 bits/pixel.

Minimal error drift in frequency scalability for motion-compensated DCT coding

The authors mathematically analyze the drift of low-resolution images obtained from a smaller IDCT of a subset of the DCT coefficients of the full-resolution images in motion-compensated hybrid predictive/DCT coding such as MPEG-2, which allows for frequency scalability. Using this mathematical structure, they derive a low-resolution decoder that has the theoretically minimum possible drift, and propose techniques for implementation that produce substantial improvement in real sequences. The minimum drift can also be used as a milestone, to be compared with other techniques of drift reduction (of worse performance but lower complexity). For the case where leakage is used to reduce the drift, the authors determine a minimum-energy non-uniform DCT-domain leakage matrix which is no more complex than uniform leakage, but gives a substantial improvement. Finally, they note that DCT-based pyramidal coding is essentially the same as the drift case, and thus they use the same mathematical structure to derive the theoretically-best upward predictor in pyramidal coding.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

相補ピラミッド画像符号化方式

相補ピラミッド画像符号化方式

Multiresolution coding techniques for digital television: A review

Multiresolution decompositions for video coding are reviewed. Both nonrecursive and recursive coding schemes are considered. In nonrecursive schemes, it is shown that pyramid structures have certain advantages over subband or wavelet techniques, and a specific spatiotemporal pyramid coding of HDTV is discussed in some detail. It is shown that recursive, DPCM like schemes will incur a slight loss of optimality due to a restricted form of prediction if multiresolution decomposition with compatible decoding is required. Compatibility and transmission issues are also discussed. Multiresolution transmission for digital broadcast TV is introduced. This, when combined with multiresolution source coding, achieves spectrum efficiency, robustness and graceful degradation under channel impairments.

Comments on "Interpolative multiresolution coding of advanced television with compatible subchannels" (and reply and additional comments)

Recently, Uz et al. (ibid., vol.1., no.1, p.86-99, 1991) analyzed the propagation of quantization noise in a pyramid (with feedback) and subband decomposition schemes. In this study each band was independently quantized by a scalar quantizer of equal step size. The resulting reconstruction error spectrum indicated that in both the pyramid (without feedback) and subband coding schemes, noise was building up in lower frequencies. The commenters show that a quantizer assignment method, using mean-square-error (MSE) optimal bit allocation, avoids the problem. The authors argue that the problem is more involved than just MSE optimal bit allocation, and that compatible coding was the focus, as well as guaranteed quality, effects of numerical computations, and perceptual effects. The commenters further support their argument. >

Multiple-rate structured vector quantization of image pyramids

A recently introduced tree growth algorithm, the marginal returns algorithm, is used to grow multiple-rate tree-structured vector quantizers for the pyramid coding of rectangularly as well as hexagonally sampled images. The use of a structured multirate codebook solves two problems that normally arise in vector quantization of subbands. The multiple-rate codebook can operate over a wide range of rates, thus dispensing with the need to transmit the codebook as overhead, while the tree structure reduces the search complexity. Search complexity is a crucial issue even in low-rate pyramid coding since subbands with more information content are coded at high rates. In addition, the design technique makes it possible to tune the coder to the spectral properties of the image by optimally allocating rate to the different subbands. It has been shown in an earlier paper that the marginal returns algorithm yields codebooks that are optimal for sources that meet the law of diminishing marginal returns. However, even for sources that do not satisfy these conditions, the algorithm gives coders that perform close to the optional. Image coding results at rates below 1 bit per pixel are presented.

A class of fast Gaussian binomial filters for speech and image processing

The authors present an efficient, in-place algorithm for the batch processing of linear data arrays. These algorithms are efficient, easily scaled, and have no multiply operations. They are suitable as front-end filters for a bank of quadrature mirror filters and for pyramid coding of images. In the latter application, the binomial filter was used as the low-pass filter in pyramid coding of images and compared with the Gaussian filter devised by P.J. Burt (Comput. Graph. Image Processing, vol.16, p.20-51, 1981). The binomial filter yielded a slightly larger signal-to-noise ratio in every case tested. More significantly, for an (L+1)*(L+1) image array processed in (N+1)*(N+1) subblocks, the fast Burt algorithm requires a total of 2(L+1)/sup 2/N adds and 2(L+1)/sup 2/ (N/2+1) multiplies. The binomial algorithm requires 2L/sup 2/N adds and zero multiplies. >

Sketch based coding of grey level images

We present an algorithm for data compression of grey level images based on coding geometric and grey level information of the contours in the image. From this sketch information alone an intelligible grey level image can be reconstructed. Compression ratios will be dependent on the amount of detail in the image and the quality of the reconstruction. For a human face image we present results with compression ratio in the range 60–75, with an intelligible reconstructed image. The reconstructed image lacks texture information however. This is therefore coded separately by taking the residual between the original and the sketch reconstructed image and using the Laplacian pyramid coding algorithm. The pyramid coding seems to be a natural complement to the sketch coding algorithm since the resolution of the texture to be added can be varied easily by just adding certain levels of the pyramid coded residual. Compared to traditional wave form coding the use of sketch coding only, seems to be advantageous mainly at high compression ratios in the range 30–75 giving intelligible reconstructed images. For lower compression ratios and high quality reconstructions the advantages are correspondingly less. Using sketch coding and residual pyramid coding gives visible improvement over a larger range of compression ratios compared to pyramid coding only. The main factor that limits the full exploitation of the possibilities of a sketch based coding algorithm seems to be the extraction of the perceptually essential sketch information. If this can be achieved in a more effective way the advantages compared to wave form coding will certainly increase.