Stereo Image Compression Research Articles

MASIC: Deep Mask Stereo Image Compression

Stereo image compression (SIC) aims to simultaneously compress a pair of left and right stereoscopic images, which can achieve higher compression efficiency than single image compression. In this paper, to benefit the SIC tasks, we collect a large real-world stereo image dataset, namely Palace, which is composed of hundreds of stereo image pairs at high-resolution. More importantly, we propose a novel mask stereo image compression network, namely MASIC, which can jointly compress the stereo images with high compression efficiency. Specifically, we first estimate the homography matrix between the stereo images through a regression model. Then, the left image is spatially transformed by the homography matrix, so that only the residual information needs to be encoded for the right image. To avoid the wrong guidance between stereo image pair, we propose a mask prediction module (MPM) to generate a multi-channel guided mask to navigate both the encoding and decoding processes. Based on the guided mask, we introduce a new mask conditional stereo entropy (MCSE) model, to fully explore the correlation between the stereo images in entropy coding. In the decoder, we develop a stereo decoding module to simultaneously decode the stereo images and enhance their compression quality. Experimental results show that our MASIC significantly advances the performance of SIC both quantitatively and qualitatively on a variety of datasets, and is robust to the change of parallax level between stereo images. The software codes are available at https://github.com/eecoder-dyf/MASIC.

Анализ методов сжатия стереоскопических изображений

Virtual Reality (VR) and Augmented Reality (AR) Head-Mounted Displays (HMDs) have been emerging in the last years and they are gaining an increased popularity in many industries. HMDs are generally used in entertainment, social interaction, education, but their use for work is also increasing in domains such as medicine, modeling and simulation. Despite the recent release of many types of HMDs, two major problems are hindering their widespread adoption in the mainstream market: the extremely high costs and the user experience issues [1]. The illusion of a 3D display in HMDs is achieved with a technique called stereoscopy. Applications of stereoscopic imagining are such that data transfer rates and—in mobile applications—storage quickly become a bottleneck. Therefore, efficient image compression techniques are required. Standard image compression techniques are not suitable for stereoscopic images due to the discrete differences that occur between the compressed and uncompressed images. The issue is that the loss in lossy image compression may blur the minute differences between the left-eye and right-eye images that are crucial in establishing the illusion of 3D perception. However, in order to achieve more efficient coding, there are various coding techniques that can be adapted to stereoscopic images. Stereo image compression techniques that can be found in the literature utilize discrete Wavelet transformation and the morphological compression algorithm applied to the transform coefficients. This paper provides an overview and comparison of available techniques for the compression of stereoscopic images, as there is still no technique that is accepted as best for all criteria. We want to test the techniques with users who would actually be potential users of HMDs and therefore would be exposed to these techniques. Also, we focused our research on low-priced, consumer grade HMDs which should be available for larger population.

Subjective quality assessment of stereo image compression based on stereoscopic fusion in binocular vision

Stereo image transmission needs a nominal doubling of the bandwidth of the video signal, but the additional savings in bandwidth can be obtained by utilizing properties of the human visual system. Embarking from human visual perception and binocular stereo imaging mechanism, this study proposes a new subjective assessment method of the quality of stereo image compression on the basis of the characteristics of human binocular vision and the psychophysics of stereoscopic fusion. Specifically, the present study explored the response of the human visual system to mixed-compressed stereo image sequences, in which one eye view was spatially and temporally compressed but the depth perception was unaffected. This method exploited the correlation between the left and right frames of the given sequence that can not only help achieving certain efficiency of stereo image transmission but also facilitate the consistency of the directly intuitive output evaluation value and the sensitivity of stereoscopic perception, which could also overcome the shortcomings of the traditional subjective quality evaluation method that restricted by subjective rating system, exhibits good feasibility and effectiveness.

Improved Method for Disparity Estimation and Compression of Stereo Images with Varied Angle of Separation

Improved Method for Disparity Estimation and Compression of Stereo Images with Varied Angle of Separation

Recurrent pattern matching based stereo image coding using linear predictors

The Multidimensional Multiscale Parser (MMP) is a pattern-matching-based generic image encoding solution which has been investigated earlier for the compression of stereo images with successful results. While first MMP-based proposals for stereo image coding employed dictionary-based techniques for disparity compensation, posterior developments have demonstrated the advantage of using predictive methods. In this paper, we focus on recent investigations on the use of predictive methods in the MMP algorithm and propose a new prediction framework for efficient stereo image coding. This framework comprises an advanced intra directional prediction model and a new linear predictive scheme for efficient disparity compensation. The linear prediction model is the main novelty of this work, combining adaptive linear models estimated by least-squares algorithm with fixed linear models provided by the block-matching algorithm. The performance of the proposed intra prediction and disparity compensation methods when applied in an MMP encoder has been evaluated experimentally. Comparisons with the current stereo image coding standards showed that the proposed MMP algorithm significantly outperforms the Stereo High Profile of H.264/AVC standard. In addition, it presents a competitive performance relative to the MV-HEVC standard. These results also suggest that current stereo image coding standards may benefit from the proposed linear prediction scheme for disparity compensation, as an extension to the omnipresent block-matching solution.

Image Segmentation using Weighted Average Local Histogram

The prime objective of this paper is to implement an efficient improved color image segmentation method using local histogram and region merging technique. The goal of image segmentation is to cluster pixels into salient image regions, i.e. regions corresponding to individual surfaces, objects or natural parts of objects. Segmentation can be used for object recognition, occlusion boundary estimation within motion or stereo systems, image compression, image editing or image database look-up. Usually image segmentation is an initial and vital step in a series of processes aimed at overall image understanding. There are various techniques for image segmentation. In this research paper, a thorough work has done on the average local histogram of three different color spaces RGB, HSV & Lab. After that k-means clustering and labeling have done on the image for final segmentation. General Terms Image Segmentation, Weighted Average Local Histogram.

A mesh-based disparity representation method for view interpolation and stereo image compression

This paper proposes a mesh-based representation method for the disparity map of stereo images. The proposed method is designed to concentrate mainly on applications of view interpolation and stereo image compression. To obtain high image quality in the view interpolation and compression of stereo images, we formulate the view-interpolation error and prediction error. In the formulation, the view-interpolation and prediction errors depend not only on the accuracy of the disparity map, but also on the gradient of the stereo images. The proposed representation method for the disparity map is based on a triangular mesh structure, which minimizes the formulated interpolation and prediction errors. The experimental results show that the proposed method yields higher quality view-interpolated images and also has better performance in stereo image compression than the conventional methods.

Multiscale recurrent patterns applied to stereo image coding

For the compression of stereo images, several techniques employing the concept of disparity compensation have been developed. In this paper, we present a new method for stereo-image compression which is based on a different paradigm. It relies on approximate matching of multiscale recurrent patterns, which is an extension of ordinary pattern matching, where an image block is represented using dilated, contracted, displaced, and deformed versions of blocks already processed. To achieve this, the input image is segmented into variable-sized blocks. Each segment is represented by an element in a dictionary. The segmentation is performed according to a rate /spl times/ distortion criterion, and the dictionary is updated with the concatenation, contraction, expansion, displacement, and deformation of previously coded elements. Unlike most other methods, in this approach the disparity map is not transmitted. That is, the generation of the error images, as well as their coding and transmission, is not necessary. In brief, the proposed method presents high-quality results at the decoder end, compared to other coding methods for stereo images that transmit the disparity map.

Stereo image compression using wavelet coefficients morphology

In this paper, we propose a new stereo image compression scheme that is based on the wavelet transform of both images and the disparity estimation between the stereo pair subbands. The two images are decomposed by using a Discrete Wavelet Transform (DWT) and coded by employing the morphological representation of the wavelet coefficients, which is a technique that exploits the intraband–interband statistical properties of them. The progressive pixel-to-pixel evaluation of the disparity has been incorporated to the morphological coder so that a dense disparity field to be formed for every subband. The proposed method demonstrates very good performance as far as PSNR measures and visual quality are concerned and low complexity.

Disparity estimation window size

Disparity estimation plays a crucial role in many stereo image compression techniques. To reduce computational complexity most methods limit the estimation search area to a limited window. The performance of the disparityestimation depends on the choice of the limited search window. Most techniques use a predetermined value for the window size, which is not optimal over a wide range of images. We show how the choice of the window size affects the performance of the stereo image compression algorithm and propose a method to obtain a better search window size. Our simulation results indicate an improvement of up to 1.81 dB over rigid window size selection and with performance very close to the optimal selection.

A wavelet implementation of the pioneering block-based disparity compensated predictive coding algorithm for stereo image pair compression

In this paper, we propose a novel, discrete wavelet transform (DWT) domain implementation of our previously proposed, pioneering block-based disparity compensated predictive coding algorithm for stereo image compression. Under the present research context we perform predictive coding in the form of pioneering block search in the sub-band domain. The resulting transform domain predictive error image is subsequently converted to a so-called wavelet-block representation, before being quantized and entropy coded by a JPEG-like CODEC. We show that the proposed novel implementation is able to effectively transfer the inherent advantages of DWT-based image coding technology to efficient stereo image pair compression. At equivalent bit rates, the proposed algorithm achieves peak signal to noise ratio gains of up to 5.5 dB, for reconstructed predicted images, as compared to traditional and state of the art DCT and DWT-based predictive coding algorithms.

Residual image coding for stereo image compression

One main focus of research in stereo image coding has been disparity estimation, a technique used to reduce the coding rate by taking advantage of the redundancy in a stereo image pair. Significantly less effort has been put into the coding of the residual image. These images display characteristics that are different from that of natural images. We propose a new method for the coding of residual images that takes into account the properties of residual images. Particular attention is paid to the effects of occlusion and the correlation properties of residual images that result from block-based disparity estimation. The embedded, progressive nature of our coder enables one to stop decoding at any time. We demonstrate that it is possible to achieve good results with a computationally simple method.

Wavelet-transform-based stereo residual image compression

A new stereo image compression algorithm is described in which the residual image, extracted from the stereo image by the disparity-compensated prediction method, is compressed using the wavelet transform in consideration of the inter and intra correlation between subbands. Contrasting the conventional algorithms such as efficient pyramid image code (EPIC), embedded predictive wavelet image coder (EPWIC), and JPEG, compression performance of the suggested method is significantly improved through computer simulation. Finally, it is suggested that the stereo image having a good 3-D effect can be reconstructed from the compressed image data using a new algorithm described.

A hybrid scheme for low bit-rate coding of stereo images

In this paper, we propose a hybrid scheme to implement an object driven, block based algorithm to achieve low bit-rate compression of stereo image pairs. The algorithm effectively combines the simplicity and adaptability of the existing block based stereo image compression techniques with an edge/contour based object extraction technique to determine appropriate compression strategy for various areas of the right image. Unlike the existing object-based coding such as MPEG-4 developed in the video compression community, the proposed scheme does not require any additional shape coding. Instead, the arbitrary shape is reconstructed by the matching object inside the left frame, which has been encoded by standard JPEG algorithm and hence made available at the decoding end for those shapes in right frames. Yet the shape reconstruction for right objects incurs no distortion due to the unique correlation between left and right frames inside stereo image pairs and the nature of the proposed hybrid scheme. Extensive experiments carried out support that significant improvements of up to 20% in compression ratios are achieved by the proposed algorithm in comparison with the existing block-based technique, while the reconstructed image quality is maintained at a competitive level in terms of both PSNR values and visual inspections.

A 2-D filtering scheme for stereo image compression using sequential orthogonal subspace updating

Stereo image compression involves estimating the disparity vectors that represent the amount of binocular parallax. The mismatching problems between the left and right images greatly impact the accuracy of the reconstructed image, and hence the visual effects of the reproduced 3-D image. This paper presents a new method for compensating the mismatching effects in stereo image pairs. This 2-D filtering-based scheme uses a sequential orthogonal subspace updating (SOSU) process to project an image block onto a subset of best-basis vectors. The basis vectors are selected one by one from the neighboring blocks, as well as some typical edge blocks, forming an image-dependent set of basis vectors. This leads to the optimal representation of an image block with fewer coefficients. Simulation results on two different image pairs demonstrate the effectiveness of the SOSU scheme when compared to those of the standard least squares 2-D filtering and the hybrid disparity-compensated discrete cosine transform residual encoding schemes.

A wavelet multiresolution compression technique for 3D stereoscopic image sequence based on mixed-resolution psychophysical experiments

Three-dimensional visual communications, made through the use of stereoscopic images, are able to achieve total display realism. In order to create a 3D system with two images (left and right) that should be transmitted simultaneously, the large amount of information contained in the images should be reduced, and an efficient coding appropriate for stereoscopic images is developed. Stereoscopic image sequence compression involves the exploitation of the spatial redundancy between the left and right image frames to achieve the compression ratio higher than that are by the independent compression of the two frames. In order to further achieve the higher compression ratio, in this paper, we employ the mixed-resolution psychophysical experiments to the stereo image compression. The experiments have shown that a stereo image pair with one high-resolution image and one lower-resolution image is sufficient to provide good stereoscopic depth perception. Thus, one image sequence is compressed independent of the other sequence using the motion compensation, while the other sequence is estimated at a lower resolution from this stream using the low-resolution disparity compensation. To implement the mixed-resolution coding, a wavelet multiresolution framework has been adopted to facilitate such an estimation of motion and disparity vectors at different resolutions. Experimental results indicate that the compression ratio for a typical stereoscopic image sequence is about 90, without any significant loss in the perceived 3D stereoscopic image quality.

A least-squares-based 2-D filtering scheme for stereo image compression

A two-dimensional (2-D) least squares (LS)-based filtering scheme for high fidelity stereo image compression applications is introduced in this correspondence. This method removes the effects of mismatching in a stereo image pair by applying the left image as the reference input to a 2-D transversal filter while the right image is used as the desired output. The weights of the filter are computed using a block-based LS method. A reduced order filtering scheme is also introduced to find the optimum number of filter coefficients. The principal coefficients and the disparity vectors are used together with left image to reconstruct the right image at the receiver, The proposed schemes are examined on a real stereo image pair for 3D-TV applications and the results were benchmarked against those of the block-matching method.

Edge-preserving directional regularization technique for disparity estimation of stereoscopic images

Due to the advances in visual technology three dimensional (3D) imaging systems are becoming popular. One way of stimulating 3D perception is to use stereo pairs, a pair of images of the same scene acquired from different perspectives. Since there is an inherent redundancy between the images of a stereo pair, data compression algorithms should be employed to efficiently represent stereo pairs. Disparity estimation of stereoscopic image is an important stage for the stereo image compression. We propose a new disparity estimation method using a directional regularization technique to preserve edges well for stereoscopic image coding. This method smooths disparity vectors in smooth regions and preserves edges in object boundaries without over-smoothing problem. The experimental results show that the proposed method achieves close matches between a left image and a right image as well as the improved coding efficiency. We use DPCM technique followed by Huffman entropy coding for disparity map coding. In addition, the proposed disparity estimation method can be applied to the high-quality intermediate view reconstruction.

Transform coding of stereo image residuals

Stereo image compression is of growing interest because of new display technologies and the needs of telepresence systems. Compared to monoscopic image compression, stereo image compression has received much less attention. A variety of algorithms have appeared in the literature that make use of the cross-view redundancy in the stereo pair. Many of these use the framework of disparity-compensated residual coding, but concentrate on the disparity compensation process rather than the post compensation coding process. This paper studies specialized coding methods for the residual image produced by disparity compensation. The algorithms make use of theoretically expected and experimentally observed characteristics of the disparity-compensated stereo residual to select transforms and quantization methods. Performance is evaluated on mean squared error (MSE) and a stereo-unique metric based on image registration. Exploiting the directional characteristics in a discrete cosine transform (DCT) framework provides its best performance below 0.75 b/pixel for 8-b gray-scale imagery and below 2 b/pixel for 24-b color imagery, In the wavelet algorithm, roughly a 50% reduction in bit rate is possible by encoding only the vertical channel, where much of the stereo information is contained. The proposed algorithms do not incur substantial computational burden beyond that needed for any disparity-compensated residual algorithm.

A novel predictive coding algorithm for 3-D image compression

We propose two predictive coding algorithms for 3-D stereo image compression which make use of inter-frame and intra-frame correlation present in stereo image pairs. The predictive coding is applied to one of the two frames associated with each stereo image pair on top of DCT-quantization based compression technology to achieve extra data compression. The technology can also be applied in addition to any other compression technology such as, wavelets based, fractal based, etc. This novel coding technique eliminates the necessity of using overhead bits in reconstructing the predicted image.