A Survey on Coding of Static and Dynamic 3D Meshes

Rate-distortion-optimized predictive compression of dynamic 3D mesh sequences

3D triangle meshes are a common form for representing the geometry of static and dynamic 3D objects. They are employed already in many areas, e.g. e-commerce, video games, online museums, CGI or 3D animated films, etc. Static triangle meshes represent only a piecewise linear approximation of complex 3D objects. As a consequence the approximation error can be unacceptably high unless the number of triangles is sufficiently large. On the other hand a large number of triangles makes these meshes cumbersome to handle and expensive to store or to transmit. Consequently, there exists a demand for techniques for efficient compression of static and dynamic 3D meshes. In this article we start with basics on 3D meshes. Thereafter, we explain the key ideas behind different mesh compression approaches for static and dynamic 3D meshes, and highlight their similarities and differences. Finally, we introduce the upcoming MPEG standard for compression of dynamic 3D meshes, which is referred to as FAMC (Frame-based Animated Mesh Compression), and show comparative compression results.

In this paper, we propose a Multiple Description Coding (MDC) method for reliable transmission of compressed time consistent 3D dynamic meshes. It trades off reconstruction quality for error resilience to provide the best expected reconstruction of 3D mesh sequence at the decoder side. The method is based on partitioning the mesh vertices into two sets and encoding each set independently by a 3D dynamic mesh coder. The encoded independent bitstreams or socalled descriptions are transmitted independently. The 3D dynamic mesh coder is based on predictive coding with spatial and temporal layered decomposition. In addition, the proposed method allows for different redundancy allocations by duplicating a number of encoded spatial layers in both sets. The algorithm is evaluated with redundancy-rate-distortion curves and flexible trade-off between redundancy and side distortions can be achieved.

In this paper, we propose a Multiple Description Coding (MDC) method for reliable transmission of compressed time consistent 3D dynamic meshes. It trades off reconstruction quality for error resilience to provide the best expected reconstruction of 3D mesh sequence at the decoder side. The method is based on partitioning the mesh frames into two sets by temporal subsampling and encoding each set independently by a 3D dynamic mesh coder. The encoded independent bitstreams or so-called descriptions are transmitted independently. The 3D dynamic mesh coder is based on predictive coding with spatial and temporal layered decomposition. In addition, the proposed method allows for different redundancy allocations by including a number of encoded spatial layers of the frames in the other set. The algorithm is evaluated with redundancy-rate-distortion curves and it is shown that, when one of the descriptions is lost, acceptable quality can be achieved with around 50% redundancy.

Application of 3D mesh model coding is first presented in this chapter. We then survey the typical existing algorithms in the area of compression of static and dynamic 3D meshes. In an introductory sub-section we introduce basic concepts of 3D mesh models, including data representations, model formats, data acquisitions and 3D display technologies. Furthermore, we introduce several typical 3D mesh formats and give an overview to coding principles of mesh compression algorithms in general, followed by describing the quantitative measures for 3D mesh compression. Then we describe some typical and state-of-the-art algorithms in 3D mesh compression. Compression and streaming of gigantic 3D models are specially introduced. At last, the MPEG4 3D mesh model coding standard is briefed. We conclude this chapter with a discussion providing an overall picture of developments in the mesh coding area and pointing out directions for future research.

Recently, spectral methods have been extensively used in the processing of 3D meshes. They usually take advantage of some unique properties that the eigenvalues and the eigenvectors of the decomposed Laplacian matrix have. However, despite their superior behavior and performance, they suffer from computational complexity, especially while the number of vertices of the model increases. In this work, we suggest the use of a fast and efficient spectral processing approach applied to dense static and dynamic 3D meshes, which can be ideally suited for real-time denoising and compression applications. To increase the computational efficiency of the method, we exploit potential spectral coherence between adjacent parts of a mesh and then we apply an orthogonal iteration approach for the tracking of the graph Laplacian eigenspaces. Additionally, we present a dynamic version that automatically identifies the optimal subspace size that satisfies a given reconstruction quality threshold. In this way, we overcome the problem of the perceptual distortions, due to the fixed number of subspace sizes that is used for all the separated parts individually. Extensive simulations carried out using different 3D models in different use cases (i.e., compression and denoising), showed that the proposed approach is very fast, especially in comparison with the SVD based spectral processing approaches, while at the same time the quality of the reconstructed models is of similar or even better reconstruction quality. The experimental analysis also showed that the proposed approach could also be used by other denoising methods as a preprocessing step, in order to optimize the reconstruction quality of their results and decrease their computational complexity since they need fewer iterations to converge.

Adaptive representation of dynamic 3D meshes for low-latency applications

This article presents a new compression scheme for 3D dynamic meshes, referred to as Video and Subdivision based Mesh Coding (VSMC). The VSMC approach combines a displaced subdivision surface model with video-based coding in order to achieve efficient compression performance and real-time, low-power decoding and playback. In addition, VSMC supports a rich set of functionalities including scalability (spatial, temporal, and quality) and progressive transmission. The proposed scheme [1] was shown to outperform the anchor for the MPEG Call for Proposals on Dynamic Mesh coding [2] and was recently selected by the ISO MPEG 3D Graphics Coding group as the basis for the upcoming Video-based Dynamic Mesh Coding standard.

Impact of vertex clustering on registration-based 3D dynamic mesh coding

Shape skeleton extraction is a fundamental pre-processing task in shape-based pattern recognition. This paper presents a new algorithm for fast and precise extraction of kinematic skeletons of 3D dynamic surface meshes. Unlike previous approaches, surface motions are characterized by the mesh edge-length deviation induced by its transformation through time. Then a static skeleton extraction algorithm based on Reeb graphs exploits this latter information to extract the kinematic skeleton. This hybrid static and dynamic shape analysis enables the precise detection of objects? articulations as well as shape topological transitions corresponding to possibly-articulated immobile objects? features. Experiments show that the proposed algorithm is faster than previous techniques and still achieves better accuracy.

We introduce an efficient algorithm for real-time compression of temporally consistent dynamic 3D meshes. The algorithm uses mesh connectivity to determine the order of compression of vertex locations within a frame. Compression is performed in a frame to frame fashion using only the last decoded frame and the partly decoded current frame for prediction. Following the predictive coding paradigm, local temporal and local spatial dependencies between vertex locations are exploited. In this framework we present a novel angle preserving predictor and evaluate its performance against other state of the art predictors. It is shown that the proposed algorithm improves up to 25% upon the current state of the art for compression of temporally consistent dynamic 3D meshes.

In this paper, we propose a distributed 3D dynamic mesh coding system. The system is based on Slepian and Wolf's and Wyner and Ziv's information-theoretic results. Our system extends the ideas in distributed video coding to 3D dynamic meshes with constant connectivity. The connectivity of the sequence and key frames are encoded and decoded by a conventional static mesh coder. The Wyner-Ziv frames are encoded independent of key frames but decoded jointly with decoded key frames. The joint decoding is performed by the low density parity check codes and the side information generated by linear interpolation of decoded key frames. Experimental results show that better rate-distortion performance is obtained compared to encoding each frame by a static mesh coder.

Dynamic 3D mesh compression is of great practical important issues in computer graphics and multimedia applications. In this paper, an efficient compression algorithm is proposed to represent animated mesh sequences in a compact way, so that the storage and transmission of dynamic 3D meshes can be accomplished efficiently. The focus of this paper is on the animated mesh sequences with shared connectivity. The proposed method first computes coarse models (low frequency modes) of the animated sequence using the graph Laplacian matrix. Obtained coordinate weights are used at the decoder to reconstruct the coarse models of the sequence. Then, a novel approach is proposed to extract fixed details (high frequency modes or finer features) of the animated mesh. Finally, a details restoration process is applied at the decoder to add details back to the coarse models of the reconstructed sequence. The superiority of the proposed method to the current state of the arts is demonstrated in terms of low data rates for a given degree of perceived distortion.

Dynamic 3D mesh sequences, also called 3D animation, have been widely used in the movie and gaming industries. However, the huge storage requirements of dynamic mesh data make it problematic for a number of applications such as rendering, transmission over a network, and displaying on mobile devices. This paper proposes a multi resolution representation of 3D animation that results displaying 3D animation progressively. The proposed method transforms traditional 3D animation representation into progressive representation, which takes up less storage and memory space. The progressive representation is constructed by compressing the animation into a base animation with sequenced refining operators. The base animation is viewable and can be reconstructed by transmitting for only a few seconds, which is also called thumbnail animation in this paper, the more detailed animation can be refined smoothly with applying refining operators. As the results, the resolution of animation can be free to change at real-time, where the resolution can be increased automatically or even controlled by user. Moreover, the progressive animation is more suitable for transmitting and displaying through the network.

In this paper, we explore various approaches to model the hemodynamic changes during cardiac contraction in the presence of a mitral paravalvular leak. Using computational fluid dynamics and large deformation diffeomorphic metric mapping, we conducted simulations that represented ventricular motion in four distinct ways. Taking tomography data into account, we developed a heart model that accurately reproduced the actual heart structure. Two simplifications for ventricular geometry to streamline the modeling process were proposed: a static mesh and a universal geometry. The simulation results from the most intricate variant, the CT-based, real model with dynamic mesh, were compared with the outcomes from the simplified approaches, universal geometry and static mesh. The simulations described unsteady flow dynamics during contraction, using a non-Newtonian Carreau-Yasuda blood rheological model. As expected, the hemodynamic conditions and parameter values derived from the hemolysis criterion (shear stresses exceeding 300 Pa) demonstrated no significant discrepancies between the various models under scrutiny. This suggests that the analysis of this phenomenon can be simplified to employ a static and universal ventricular mesh, eliminating the necessity for patient-specific medical imaging of the ventricle. Such a simplification can significantly reduce preprocessing and computational time, making this model more practical for routine medical diagnostics.