Error propagation control in Laplacian mesh compression

Chapter 14 - Coding of dynamic 3D meshes

Recent developments in the compression of dynamic meshes or mesh sequences have shown that the statistical dependencies within a mesh sequence can be exploited well by predictive coding approaches. Coders introduced so far use experimentally determined or heuristic thresholds for tuning the algorithms. In video coding rate-distortion (RD) optimization is often used to avoid fixing of thresholds and to select a coding mode. We applied these ideas and present here an RD-optimized mesh coder. It includes different prediction modes as well as an RD cost computation that controls the mode selection across all possible spatial partitions of a mesh to find the clustering structure together with the associated prediction modes. The structure of the RD-optimized D3DMC coder is presented, followed by comparative results with mesh sequences at different resolutions.

ISO/IEC JTC1 SC29, also called MPEG, has been working on a compression standard for dynamic meshes since couple of year now and it has released a Call for Proposals (CfP) for Dynamic Mesh Coding in October 2021. One of the goals of the future standard is to utilize the Visual Volumetric Video-based Coding (V3C) framework, defined in ISO/IEC 23090-5, that is already used for dynamic point cloud compression and volumetric video. In this paper, the authors described their vision of how dynamic mesh compression could be achieved, which corresponds to their technical response to the CfP. The presented objective and subjective results will show that the proposed solution outperforms the anchor in terms of objective metrics and subjective perceived visual quality for low bit rate use cases.

The latest wavelet based 3D mesh coding schemes convert an irregular mesh into a semi-regular mesh and directly apply the zerotree-like image coders to compress the wavelet vectors generated in the remeshing process. The major problem of such type of approaches is that the particular properties of semi-regular meshes are not being considered in the zerotree-like image coders. In this paper, we propose an improved wavelet based 3D mesh coder. The basic idea is to introduce a preprocessing step to scale up the vector wavelets generated in remeshing so that the inherent dependency of wavelets can be truly understood by the zerotree-like image compression algorithms. The weights used in the scaling process are carefully designed through thoroughly analyzing the distortions of wavelets at different refinement levels. Experimental results show that our proposed mesh coder significantly outperforms the state-of-the-art wavelet based 3D mesh compression scheme.

3D triangle meshes are a common form for representing the geometry of static and dynamic 3D objects. They are employed already in many areas, e.g. e-commerce, video games, online museums, CGI or 3D animated films, etc. Static triangle meshes represent only a piecewise linear approximation of complex 3D objects. As a consequence the approximation error can be unacceptably high unless the number of triangles is sufficiently large. On the other hand a large number of triangles makes these meshes cumbersome to handle and expensive to store or to transmit. Consequently, there exists a demand for techniques for efficient compression of static and dynamic 3D meshes. In this article we start with basics on 3D meshes. Thereafter, we explain the key ideas behind different mesh compression approaches for static and dynamic 3D meshes, and highlight their similarities and differences. Finally, we introduce the upcoming MPEG standard for compression of dynamic 3D meshes, which is referred to as FAMC (Frame-based Animated Mesh Compression), and show comparative compression results.

In neural speech enhancement, a mismatch exists between the training objective, i.e., Mean-Square Error (MSE), and perceptual quality evaluation metrics, i.e., perceptual evaluation of speech quality and short-time objective intelligibility. We propose a novel reinforcement learning algorithm and network architecture, which incorporate a non-differentiable perceptual quality evaluation metric into the objective function using a dynamic filter module. Unlike the traditional dynamic filter implementation that directly generates a convolution kernel, we use a filter generation agent to predict the probability density function of a multivariate Gaussian distribution, from which we sample the convolution kernel. Experimental results show that the proposed reinforcement learning method clearly improves the perceptual quality over other supervised learning methods with the MSE objective function.

Rate-distortion-optimized predictive compression of dynamic 3D mesh sequences

Perception-driven adaptive compression of static triangle meshes

Motion field coding is an essential technique for exploiting temporal correlations in the ongoing standard of Video-based Dynamic Mesh Coding (V-DMC), which is crucial for efficient dynamic mesh compression. In V-DMC, the motion field comprises a set of motion vectors (MVs) that represent the positional changes of vertices between corresponding decimated meshes (called base meshes in V-DMC) of current and reference frames. This paper presents an enhanced motion field coding approach that effectively reduces the number of MVs required. The proposed method leverages the observation that base meshes often contain duplicate vertices with identical MVs. By integrating these vertices, the method achieves a more compact representation, while distinct MVs for the few duplicate vertices are explicitly coded to preserve mesh quality. Experimental results using MPEG test sequences demonstrate that the proposed approach can decrease the MV data size by up to 20% without compromising the quality of the reconstructed meshes.

Subdivision-based wavelet coding techniques yield state-of-the-art performance in scalable compression of semi-regular meshes. However, all these codecs make use of the L-2 distortion metric, which gives only a good approximation of the global error produced by lossy coding of the wavelet coefficients. The L-infinite metric has been proven to be a suitable metric for applications where controlling the local, maximum error on each vertex is of critical importance. In this context, an upper bound formulation for the L-infinite distortion for a wavelet-based coding scheme operating on semi-regular meshes is derived. In addition, we propose a rate-distortion optimization algorithm that minimizes the rate for any target L-infinite distortion. It is shown that our L-infinite coding system outperforms the state-of-the-art and that an L-2 driven coding approach for semi-regular meshes loses ground to its L-infinite driven version when the goal is to have a tight control on the local reconstruction error.

Application of 3D mesh model coding is first presented in this chapter. We then survey the typical existing algorithms in the area of compression of static and dynamic 3D meshes. In an introductory sub-section we introduce basic concepts of 3D mesh models, including data representations, model formats, data acquisitions and 3D display technologies. Furthermore, we introduce several typical 3D mesh formats and give an overview to coding principles of mesh compression algorithms in general, followed by describing the quantitative measures for 3D mesh compression. Then we describe some typical and state-of-the-art algorithms in 3D mesh compression. Compression and streaming of gigantic 3D models are specially introduced. At last, the MPEG4 3D mesh model coding standard is briefed. We conclude this chapter with a discussion providing an overall picture of developments in the mesh coding area and pointing out directions for future research.

In the last decade, the extensive evolution of 3D graphic applications has induced developpers to enhance the 3D mesh compression techniques. Therefore, 3D models are compressed, transmitted and rendered more and more in real-time and with high quality. Moreover, many out-of-core algorithms are proposed to process complex objects. In this paper, we review the major existing technologies for the single-rate and progressive 3D mesh compression. Then, representative out-of-core approaches are surveyed in details. Finally, we represent some parallel schemes exploiting the evolution of many-core GPU architecture.

Summary3D meshes are widely used in graphical and simulation applications for approximating 3D objects. When representing complex shapes in raw data format, meshes consume a large amount of space. Applications calling for compact storage and fast transmission of 3D meshes have motivated the multitude of algorithms developed to compress these datasets efficiently. In this paper we survey recent developments in compression of 3D surface meshes. We survey the main ideas and intuition behind techniques for single-rate and progressive mesh coding. Where possible, we discuss the theoretical results obtained for asymptotic behaviour or optimality of the approach. We also list some open questions and directions for future research.

An adaptive bit-allocation method based on perceptual quality smoothing for wireless video communications is proposed. A visual attention model is build by considering skin, motion and position. Then the perceptual video quality distortion is predicted using the visual attention model and channel state information. Finally the MB target bits are allocated according to the predicted perceptual video quality distortion. Experimental results show that, when wireless channel state varies rapidly, comparing with the H.264 standard encoding method, the proposed method can reduce the variation of attention area's Peak Signal Noise Ratio (PSNR) and the variation of the frame Structural Similarity (SSIM). As a result, the perceptual quality smoothing is enhanced and the whole video perceptual quality is also improved.

Improving image resolution, especially spatial resolution, has been one of the most important concerns on remote sensing research communities. An efficient solution for improving spatial resolution is by using algorithm, known as super-resolution (SR). The super-resolution technique that received special attention recently is super-resolution based on deep learning. In this paper, we propose deep learning approach based on generative adversarial network (GAN) for remote sensing images super resolution. We used residual dense network (RDN) as generator network. Generally, deep learning with residual dense network (RDN) gives high performance on classical (objective) evaluation metrics meanwhile generative adversarial network (GAN) based approach shows a high perceptual quality. Experiment results show that combination of residual dense network generator with generative adversarial network training is found to be effective. Our proposed method outperforms the baseline method in terms of objective and perceptual quality evaluation metrics.