Image Sequence Coding Research Articles

Learning a deep predictive coding network for a semi-supervised 3D-hand pose estimation

In this paper we present a CNN based approach for a real time 3D-hand pose estimation from the depth sequence. Prior discriminative approaches have achieved remarkable success but are facing two main challenges: Firstly, the methods are fully supervised hence require large numbers of annotated training data to extract the dynamic information from a hand representation. Secondly, unreliable hand detectors based on strong assumptions or a weak detector which often fail in several situations like complex environment and multiple hands. In contrast to these methods, this paper presents an approach that can be considered as semi-supervised by performing predictive coding of image sequences of hand poses in order to capture latent features underlying a given image without supervision. The hand is modelled using a novel latent tree dependency model &#x0028 LDTM &#x0029 which transforms internal joint location to an explicit representation. Then the modeled hand topology is integrated with the pose estimator using data dependent method to jointly learn latent variables of the posterior pose appearance and the pose configuration respectively. Finally, an unsupervised error term which is a part of the recurrent architecture ensures smooth estimations of the final pose. Experiments on three challenging public datasets, ICVL, MSRA, and NYU demonstrate the significant performance of the proposed method which is comparable or better than state-of-the-art approaches.

Heuristic Algorithms of Coincidence for the Estimation of Movements in Compression of Images

The NP-Completeness theory states that exact and efficient algorithms are unlikely to exist for the class of NP-difficult problems. One way to deal with NP hardness is to relax the optimality requirement and look for solutions instead that are close to the optimum. This is the main idea behind the approximation algorithms, which are called heuristic or metaheuristic. The problem of motion estimation is a process with a high degree of computational complexity, it requires sufficient memory space and execution time. It represents the cost of static, dynamic and video image sequence coding. The main task is to minimize the distortion rate and improve visual quality. This makes research in the field of coding, image compression and video focus on finding efficient algorithms to carry out the estimation of movement in a reasonable time. If a list of images of n elements is analyzed, there are feasible solutions. So, an exhaustive search is too slow, even for small values of the solution space. Therefore, from a practical point of view, it is crucial to have efficient and fast heuristic algorithms that avoid thorough search. In this investigation we design and implement heuristic algorithms, based on the frequency domain, which are applied on the coefficients of the discrete transform of the cosine and wavelets. Also, we propose temporal domain algorithms such as block-matching algorithms, which focus your search on the maximum coincidence of the current image with the reference one. The algorithms used during the implementation of this research work were written with the mathematical programming language MATLAB. In addition, we review the basic concepts of image processing, video, compression algorithms and motion estimation frequently used. The evaluation of the algorithms was carried out with a set of images provided by a previous acquisition system. We show the improvement of visual quality, the amount of compressed or reconstructed information and the behavior of the methods in the search for similarities between pixels or images. Finally, we contribute to the dissemination of new lines of scientific research that lead to the expansion and improvement of the study, the generation of new knowledge, since it is a young area within the Education discipline of Nicaragua.

Learning hand latent features for unsupervised 3D hand pose estimation

<p align="justify">Recent hand pose estimation methods require large numbers of annotated training data to extract the dynamic information from a hand representation. Nevertheless, precise and dense annotation on the real data is difficult to come by and the amount of information passed to the training algorithm is significantly higher. This paper presents an approach to developing a hand pose estimation system which can accurately regress a 3D pose in an unsupervised manner. The whole process is performed in three stages. Firstly, the hand is modelled by a novel latent tree dependency model (LTDM) which transforms internal joints location to an explicit representation. Secondly, we perform predictive coding of image sequences of hand poses in order to capture latent features underlying a given image without supervision. A mapping is then performed between an image depth and a generated representation. Thirdly, the hand joints are regressed using convolutional neural networks to finally estimate the latent pose given some depth map. Finally, an unsupervised error term which is a part of the recurrent architecture ensures smooth estimations of the final pose. To demonstrate the performance of the proposed system, a complete experiment is conducted on three challenging public datasets, ICVL, MSRA, and NYU. The empirical results show the significant performance of our method which is comparable or better than state-of-the-art approaches.</p>

A Doubly Error Resilient Coder of Image Sequences

Many scenarios require single frame random access and error resilience in coding, transmission, and decoding of image sequences. We propose in this paper a SPIHT coder that encodes a sequence of images one frame at a time and uses two methods together to achieve strong error resilience. The first method groups wavelet coefficients into a number of tree blocks and the second one encodes these tree blocks independently using SPIHT with progressive significance maps. Because a substantial part of a progressive significance map contains a fixed-length code stream called the complementary significance map (comp-sigmap), transmission bit errors do not propagate within the compsig-map. Furthermore, since the block trees are encoded independently, no error in a given bitstream propagates to another bitstream. Simulations show that the two methods together show stronger error resilience than either one separately. Furthermore, because the groups are formed by interleaved spatial orientation trees, faulty reconstruction from bit errors affecting a given treeblock can be concealed by estimating the true wavelet coefficients from neighboring coefficients in adjacent error-free spatial orien-

A survey of image compression methods for low depth-of-field images and image sequences

The popularity of multimedia applications has resulted in development of lossless and lossy compression techniques. Many image compression methods clustered under these two compression techniques are discussed briefly in this article. In addition to this context, the survey paper gives a study of the recent algorithms that are available for coding low Depth-of-Field (DOF) images and also covers its extension for depth map image sequence coding. Motivation behind this work is to provide a detailed analysis of these algorithms such as the methodology used, merits and demerits, and the objective and subjective comparison of these algorithms with the standard compression algorithms like JPEG, JPEG 2000, H.261/AVC etc. Further, the paper concludes with a guideline for the new researchers in this field which concerns the design of an efficient compression method for low DOF images and depth map image sequences.

ENCODING VIDEO SEQUENCES IN FRACTAL-BASED COMPRESSION

With the rapid increase in the use of computers and the Internet, the demand for higher transmission and better storage is increasing as well. This paper describes the different techniques for data (image-video) compression in general and, in particular, the new compression technique called fractal image compression. Fractal image compression is based on self-similarity, where one part of an image is similar to the other part of the same image. Low bit rate color image sequence coding is very important for video transmission and storage applications. The most significant aspect of this work is the development of color images using fractal-based color image compression, since little work has been done previously in this area. The results obtained show that the fractal-based compression works for the color images works as well as for the gray-scale images. Nevertheless, the encoding of the color images takes more time than the gray-scale images. Color images are usually compressed in a luminance-chrominance coordinate space, with the compression performed independently for each coordinate by applying the monochrome image processing techniques. For image sequence compression, the design of an accurate and efficient algorithm for computing motion to exploit the temporal redundancy has been one of the most active research areas in computer vision and image compression. Pixel-based motion estimation algorithms address pixel correspondence directly by identifying a set of local features and computing a match between these features across the frames. These direct techniques share the common pitfall of high computation complexity resulting from the dense vector fields produced. For block matching motion estimation algorithms, the quad-tree data structure is frequently used in image coding to recursively decompose an image plane into four non-overlapping rectangular blocks.

Block-based motion field segmentation for video coding

In the past few years, motion compensation has been widely used in the coding of image sequences. Most of motion estimation and compensation schemes belong to block-based framework. The framework simplifies the complexity of motion estimation, but gives over constraints to the motion field, which results in worse accuracy on the boundary of moving objects. This paper presents a novel technique for raising motion field accuracy. It uses several pre-defined pattern types to segment the motion fields of the previous frame of a sequence. The segmentation is based on the MAP framework that uses iterative method to obtain the solution. In addition, we develop a predictive scheme to predict the location of motion field discontinuities in the current frame, which further reduces the side information for the representation of segmentation.

Adaptive joint subband vector quantisation codec for handheld videophone applications

A new wavelet-based video codec is presented, which joins vectors in the same subbands of different frames in a frame group and uses the properties of the human visual system. Perceptual weights are designed and used in the vector selection and quantisation of the wavelet coefficients. Results indicate a significant improvement in frame quality compared to JPEG2000 image sequence coder.

Rate—distortion optimal fast thresholding for MPEG—2 image sequence coding

AbstractA new Rate—Distortion (R—D) optimized image sequence encoding scheme is introduced, fully compatible with the standard MPEG—2 decoder. R—D optimal DCT coefficient thresholding is used, so that best image reproduction quality is achieved under the constraint of a “local” target bit rate for every image. The proposed technique combines R—D optimal coefficient thresholding and some elements of the MPEG—2 rate control model “Test Model 5” and results in a new video encoding scheme, which offers in most cases a higher representation quality than “Test Model 5”. The performance of the proposed technique is experimentally evaluated for the coding of standard test image sequences.

Spatiotemporal segmentation and tracking of objects for visualization of videoconference image sequences

A procedure is described for the segmentation, content-based coding, and visualization of videoconference image sequences. First, image sequence analysis is used to estimate the shape and motion parameters of the person facing the camera. A spatiotemporal filter, taking into account the intensity differences between consequent frames, is applied, in order to separate the moving person from the static background. The foreground is segmented in a number of regions in order to identify the face. For this purpose, we propose the novel procedure of K-means with connectivity constraint algorithm as a general segmentation algorithm combining several types of information including intensity, motion and compactness. In this algorithm, the use of spatiotemporal regions is introduced since a number of frames are analyzed simultaneously and as a result, the same region is present in consequent frames. Based on this information, a 3-D ellipsoid is adapted to the person's face using an efficient and robust algorithm. The rigid 3-D motion is estimated next using a least median of squares approach, Finally, a virtual reality modeling language (VRML) file is created containing all the above information; this file may be viewed by using any VRML 2.0 compliant browser.

3D object articulation and motion estimation in model-based stereoscopic videoconference image sequence analysis and coding

This paper describes a procedure for model-based analysis and coding of both left and right channels of a stereoscopic image sequence. The proposed scheme starts with a hierarchical dynamic programming technique for matching across the epipolar line for efficient disparity/depth estimation. Foreground/background segmentation is initially based on depth estimation and is improved using motion and luminance information. The model is initialised by the adaptation of a wireframe model to the consistent depth information. Robust classification techniques are then used to obtain an articulated description of the foreground of the scene (head, neck, shoulders). The object articulation procedure is based on a novel scheme for the segmentation of the rigid 3D motion fields of the triangle patches of the 3D model object. Spatial neighbourhood constraints are used to improve the reliability of the original triangle motion estimation. The motion estimation and motion field segmentation procedures are repeated iteratively until a satisfactory object articulation emerges. The rigid 3D motion is then re-computed for each sub-object and finally, a novel technique is used to estimate flexible motion of the nodes of the wireframe from the rigid 3D motion vectors computed for the wireframe triangles containing each specific node. The performance of the resulting analysis and compression method is evaluated experimentally.

A fast quadtree motion segmentation for image sequence coding

A new fast quadtree-based variable size block matching scheme is devised for image sequence coding. The novelty of the method lies in the quadtree-based block matching with significantly less computations. In the proposed method, the computational requirements for matching are largely transferred from the evaluation of the mean absolute difference to the computations of the sum norms of the blocks which can be easily obtained. The fast quadtree decomposition scheme partitions an image frame into uniform motion blocks (larger blocks) and non-uniform motion blocks (small blocks). Standard VQ-based interframe coding is applied to uniform motion blocks while non-uniform blocks are intraframe coded. Implementation of this method shows a significantly lower computational requirement and a good-quality reconstruction of images at low bit-rates.

Three-dimensional video compression using subband/wavelet transform with lower buffering requirements

Three-dimensional (3-D) video compression using wavelets decomposition along the temporal axis dictates that a number of video frames must be buffered to allow for the temporal decomposition. Buffering of frames allows for the temporal correlation to be made use of, and the larger the buffer the more effective the decomposition. One problem inherent in such a set up in interactive applications such as video conferencing, is that buffering translates into a corresponding time delay. We show that 3-D coding of such image sequences can be achieved in the true sense of temporal direction decomposition but with much less buffering requirements. For a practical coder, this can be achieved by introducing an approximation to the way the transform coefficients are encoded. Applying wavelet decomposition using some types of filters may introduce edge errors, which become more prominent in short signal segments. We also present a solution to this problem for the Daubechies (1988) family of filters.

Object-based coding of stereoscopic and 3D image sequences

The author begins by discussing what object based coding is and goes on to consider the structure of object based stereoscopic coders. Different techniques for object-based stereoscopic image sequence coding are reviewed. These techniques basically differ in the way they define models and estimate model parameters. We review the various models used for representing motion and structure. Then we review segmentation techniques, and discuss coding of object parameters and image synthesis.

Image and video compression: A survey

At the present time, we stand upon the threshold of a revolution in the means available to us for the widespread dissemination of information in visual form through the rapidly increasing use of international standards for image and video compression. Yet, such standards, as observed by the casual user, are only the tip of the coding iceberg. Something like half a century of scientific and technological development has contributed to a vast body of knowledge concerning techniques for coding still and moving pictures, and the present article presents a survey of developments which have taken place since the first (predictive) coding algorithms were implemented in the 1950s. Initially, we briefly review the characteristics of the human eye which influence how we approach the design of coding algorithms; then we examine the still picture techniques of major interest—predictive and transform coding, vector quantization, and subband and wavelet multiresolution approaches. Recognizing that other forms of algorithm have also been of interest during this period, we next consider such techniques as quadtree decomposition and segmentation before looking at the problems arising from the presence of motion and its compensation in the coding of video signals. In the next section, various approaches to the coding of image sequences are reviewed, and we concentrate upon the now universally used hybrid motion compensated transform algorithm before examining more advanced techniques such as model and object-based coding. Of course, the key to widespread acceptance of any technological development is the establishment of standards, and all major proposals—JPEG, MPEG-I, -II, and -IV, H.261, and H.263, are considered with emphasis on the way in which the coding algorithm is implemented rather than on protocol and syntax considerations. Finally, comments are offered in respect of the future viability of coding standards, of less well-researched algorithms, and the overall position of image and video compression techniques in the rapidly developing field of visual information provision. © 1999 John Wiley & Sons, Inc. Int J Imaging Syst Technol 10: 20–32, 1999

Inter-frame vertex selection scheme for object-based image sequence coder

This paper proposes a new boundary coding algorithm suitable for object-based image sequence coding system. The boundary is approximated by a polygon which can be encoded with the smallest number of bits for maximum distortion. The proposed scheme selects the vertex points considering the shape information from the past frames. The temporal redundancy between two successive frames is efficiently removed with the proposed scheme, resulting in lower bit-rate than the conventional algorithms.

Adaptive Fractal Image Sequence Coding by Variable Shape Decomposition

A two-phase fractal image sequence compression system is proposed. In the classification phase, according to the texture attribution a testing solid image block is assigned to its corresponding texture class. The texture attribution is derived from the tomographic block projection classification for the finite projection directions at the three-dimensional (3D) space. In the adaptive coding phase, both the algorithm of the 3D projection classification and the 3D variable shape decomposition are incorporated into the variable shape block transformation for image sequence. By applying this variable shape block transformation algorithm to fractal image sequence coding scheme, we can obtain a promising performance.

New techniques on deformed image motion estimation and compensation

In this paper, new techniques for deformed image motion estimation and compensation using variable-size block-matching are proposed, which can be applied to an image sequence compression system or a moving object recognition system. The motion estimation and compensation techniques have been successfully applied in the area of image sequence coding. Many research papers on improving the performance of these techniques have been published; many directions are proposed, which can all lead to better performance than the conventional techniques. Among them, both generalized block-matching and variable-size block-matching are successfully applied in reducing the data rate of compensation error and motion information, respectively. These two algorithms have their merits, but suffer from their drawbacks. Moreover, reducing the data rate in compensation error is sometimes increasing the data rate in motion information, or vice versa. Based on these two algorithms, we propose and examine several algorithms which are effective in reducing the data rate. We then incorporate these algorithms into a system, in which they work together to overcome the disadvantages to individual and keep their merits at the same time. The proposed system can optimally balance the amount of data rate in two aspects (i.e., compensation error and motion information). Experimental results show that the proposed system outweighs the conventional techniques. Since we propose a recovery operation which tries to recover the incorrect motion vectors from the global motion, this proposed system can also be applied to the moving object recognition in image sequences.

Contour simplification for segmented still image and video coding: algorithms and experimental results

This paper describes some new algorithms for Segmented Still Image and Image Sequence Coding. A new filter is introduced for reducing the number of contour points produced by segmentation, and also the number of bits needed to code them, leading to higher compression ratios without visible subjective degradation of the image quality. The impact of using such a contour points' reduction filter in image sequence compression is studied. The inclusion of the filter in a new video codec scheme leads to higher frame rates for almost the same subjective video quality, especially when coding complex scenes. A simple way to control the video bit-rate is also described.

Applications of kriging to image sequence coding

Abstract Kriging is a linear interpolation method used by geostatisticians. In this paper we show that it can be successfully applied to image sequence coding. First we apply it to texture coding and we improve these preliminary results with an optimization method that we have named ‘inverse kriging’. Then we apply kriging and inverse kriging to motion vector fields, which are by essence smooth within an object. We show an application of these tools in the framework of the active mesh coding scheme developed within the RACE/Morpheco project.