Coarse Shape Research Articles

Learning with Explicit Shape Priors for Medical Image Segmentation.

Medical image segmentation is a fundamental task for medical image analysis and surgical planning. In recent years, UNet-based networks have prevailed in the field of medical image segmentation. However, convolutional neural networks (CNNs) suffer from limited receptive fields, which fail to model the long-range dependency of organs or tumors. Besides, these models are heavily dependent on the training of the final segmentation head. And existing methods can not well address aforementioned limitations simultaneously. Hence, in our work, we proposed a novel shape prior module (SPM), which can explicitly introduce shape priors to promote the segmentation performance of UNet-based models. The explicit shape priors consist of global and local shape priors. The former with coarse shape representations provides networks with capabilities to model global contexts. The latter with finer shape information serves as additional guidance to relieve the heavy dependence on the learnable prototype in the segmentation head. To evaluate the effectiveness of SPM, we conduct experiments on three challenging public datasets. And our proposed model achieves state-of-the-art performance. Furthermore, SPM can serve as a plug-and-play structure into classic CNNs and Transformer-based backbones, facilitating the segmentation task on different datasets. Source codes are available at https://github. com/AlexYouXin/Explicit-Shape-Priors.

3D Shape Completion on Unseen Categories: A Weakly-Supervised Approach.

3D shapes captured by scanning devices are often incomplete due to occlusion. 3D shape completion methods have been explored to tackle this limitation. However, most of these methods are only trained and tested on a subset of categories, resulting in poor generalization to unseen categories. In this paper, we propose a novel weakly-supervised framework to reconstruct the complete shapes from unseen categories. We first propose an end-to-end prior-assisted shape learning network that leverages data from the seen categories to infer a coarse shape. Specifically, we construct a prior bank consisting of representative shapes from the seen categories. Then, we design a multi-scale pattern correlation module for learning the complete shape of the input by analyzing the correlation between local patterns within the input and the priors at various scales. In addition, we propose a self-supervised shape refinement model to further refine the coarse shape. Considering the shape variability of 3D objects across categories, we construct a category-specific prior bank to facilitate shape refinement. Then, we devise a voxel-based partial matching loss and leverage the partial scans to drive the refinement process. Extensive experimental results show that our approach is superior to state-of-the-art methods by a large margin. We will make the source code publicly available at https://github.com/ltwu6/WSSC.

Robust 3D Tracking with Quality-Aware Shape Completion

3D single object tracking remains a challenging problem due to the sparsity and incompleteness of the point clouds. Existing algorithms attempt to address the challenges in two strategies. The first strategy is to learn dense geometric features based on the captured sparse point cloud. Nevertheless, it is quite a formidable task since the learned dense geometric features are with high uncertainty for depicting the shape of the target object. The other strategy is to aggregate the sparse geometric features of multiple templates to enrich the shape information, which is a routine solution in 2D tracking. However, aggregating the coarse shape representations can hardly yield a precise shape representation. Different from 2D pixels, 3D points of different frames can be directly fused by coordinate transform, i.e., shape completion. Considering that, we propose to construct a synthetic target representation composed of dense and complete point clouds depicting the target shape precisely by shape completion for robust 3D tracking. Specifically, we design a voxelized 3D tracking framework with shape completion, in which we propose a quality-aware shape completion mechanism to alleviate the adverse effect of noisy historical predictions. It enables us to effectively construct and leverage the synthetic target representation. Besides, we also develop a voxelized relation modeling module and box refinement module to improve tracking performance. Favorable performance against state-of-the-art algorithms on three benchmarks demonstrates the effectiveness and generalization ability of our method.

High-Resolution Volumetric Reconstruction for Clothed Humans

We present a novel method for reconstructing clothed humans from a sparse set of, e.g., 1–6 RGB images. Despite impressive results from recent works employing deep implicit representation, we revisit the volumetric approach and demonstrate that better performance can be achieved with proper system design. The volumetric representation offers significant advantages in leveraging 3D spatial context through 3D convolutions, and the notorious quantization error is largely negligible with a reasonably large yet affordable volume resolution, e.g., 512. To handle memory and computation costs, we propose a sophisticated coarse-to-fine strategy with voxel culling and subspace sparse convolution. Our method starts with a discretized visual hull to compute a coarse shape and then focuses on a narrow band nearby the coarse shape for refinement. Once the shape is reconstructed, we adopt an image-based rendering approach, which computes the colors of surface points by blending input images with learned weights. Extensive experimental results show that our method significantly reduces the mean point-to-surface (P2S) precision of state-of-the-art methods by more than 50% to achieve approximately 2mm accuracy with a 512 volume resolution. Additionally, images rendered from our textured model achieve a higher peak signal-to-noise ratio (PSNR) compared to state-of-the-art methods.

Visceral stromal sarcoma in a dog: Case report

Visceral stromal sarcoma is a tumor that may cause no symptoms and, when it does, symptoms are unspecific. The tumor is usually an incidental finding, as it is detected while treating unrelated conditions. In this study, we aimed to highlight the importance of complementary exams in diagnosing stromal sarcoma in dogs. A nine-year-old Lhasa Apso dog was admitted into a veterinary hospital with a distended abdomen and decreased defecation. Ultrasound and hematological exams were requested. The hematology test revealed thrombocytosis, neutrophilia, and monocytosis. The ultrasound exam showed the spleen with increased dimensions. Presence of a neoformation occupying the entire left and right mesogastric abdominal topography was observed, apparently originating in the splenic tail, with irregular and coarse shape, mixed echogenicity, vascularized in some portions, and with cavitary areas and irregular walls in others, with anechoic content and echogenic dots in suspension measuring across the vascularized and cavitary portions. Associated with the formation, the mesentery and peritoneum were visible with a slight increase in echogenicity and coarse echotexture. These findings suggested that initial treatment should be surgical, which confirmed that the tumor did not originate in the spleen but in the mesentery. The tumor was removed, and an end-to-end anastomosis was performed at the site the tumor was adhered to. The tumor was taken for a biopsy and multiple nodular mass fragments were found, together measuring 10.0 × 9.0 × 2.5 cm. The inner surface of fragments of the fragments was observed to be whitish-beige, with yellowish areas, smooth and irregular. Biopsy confirmed a diagnosis of visceral stromal sarcoma and the patient was referred to an oncologist. Surgery was the most appropriate treatment in this case due to the size of the tumor, which could compromise the function of other systems. The complementary exam was critical in determining the diagnosis and deciding on the best course of action. In veterinary clinics where the complementary exams are not available, this case could have gone unnoticed.

SANS Spectra with PLUMED: Implementation and Application to Metainference.

Using small-angle scattering with either X-ray or neutron sources has become common in the investigation of soft-matter systems. These experiments provide information about the coarse shape of the scattered objects, but obtaining more-detailed information can usually only be achieved with the aid of molecular simulations. In this Application Note, we report the implementation of an extension in PLUMED to compute the small-angle neutron scattering (SANS), which can be used for data processing as well for enhanced sampling, in particular with the metainference method to bias simulations and sample structures with a resulting spectrum in agreement with an experimental reference. Our implementation includes a resolution function that can be used to smear the SANS intensities according to beamline error sources and is compatible with both all-atom and coarse-grained simulations. Scripts to aid in the calculation of the scattering lengths when the system is coarse-grained and to aid in preparing the inputs are provided. We illustrate the use of the implementation with metainference by performing coarse-grained simulations of beta-octylglucoside and dodecylphosphocholine micelles in water. With different software and different Hamiltonians, we show that the metainference SANS bias can drive micelles to be split and to change shapes to achieve a better agreement with the experimental reference.

ED2IF2-Net: Learning Disentangled Deformed Implicit Fields and Enhanced Displacement Fields from Single Images Using Pyramid Vision Transformer

There has emerged substantial research in addressing single-view 3D reconstruction and the majority of the state-of-the-art implicit methods employ CNNs as the backbone network. On the other hand, transformers have shown remarkable performance in many vision tasks. However, it is still unknown whether transformers are suitable for single-view implicit 3D reconstruction. In this paper, we propose the first end-to-end single-view 3D reconstruction network based on the Pyramid Vision Transformer (PVT), called ED2IF2-Net, which disentangles the reconstruction of an implicit field into the reconstruction of topological structures and the recovery of surface details to achieve high-fidelity shape reconstruction. ED2IF2-Net uses a Pyramid Vision Transformer encoder to extract multi-scale hierarchical local features and a global vector of the input single image, which are fed into three separate decoders. A coarse shape decoder reconstructs a coarse implicit field based on the global vector, a deformation decoder iteratively refines the coarse implicit field using the pixel-aligned local features to obtain a deformed implicit field through multiple implicit field deformation blocks (IFDBs), and a surface detail decoder predicts an enhanced displacement field using the local features with hybrid attention modules (HAMs). The final output is a fusion of the deformed implicit field and the enhanced displacement field, with four loss terms applied to reconstruct the coarse implicit field, structure details through a novel deformation loss, overall shape after fusion, and surface details via a Laplacian loss. The quantitative results obtained from the ShapeNet dataset validate the exceptional performance of ED2IF2-Net. Notably, ED2IF2-Net-L stands out as the top-performing variant, exhibiting the highest mean IoU, CD, EMD, ECD-3D, and ECD-2D scores, reaching impressive values of 61.1, 7.26, 2.51, 6.08, and 1.84, respectively. The extensive experimental evaluations consistently demonstrate the state-of-the-art capabilities of ED2IF2-Net in terms of reconstructing topological structures and recovering surface details, all while maintaining competitive inference time.

A Note on Finite Coarse Shape Groups

In this paper, we investigate properties concerning some recently introduced finite coarse shape invariants—the k-th finite coarse shape group of a pointed topological space and the k-th relative finite coarse shape group of a pointed topological pair. We define the notion of finite coarse shape group sequence of a pointed topological pair X,X0,x0 as an analogue of homotopy and (coarse) shape group sequences and show that for any pointed topological pair, the corresponding finite coarse shape group sequence is a chain. On the other hand, we construct an example of a pointed pair of metric continua whose finite coarse shape group sequence fails to be exact. Finally, using the aforementioned pair of metric continua together with a pointed dyadic solenoid, we show that finite coarse-shape groups, in general, differ from both shape and coarse-shape groups.

Pixel2Mesh++: 3D Mesh Generation and Refinement From Multi-View Images.

We study the problem of shape generation in 3D mesh representation from a small number of color images with or without camera poses. While many previous works learn to hallucinate the shape directly from priors, we adopt to further improve the shape quality by leveraging cross-view information with a graph convolution network. Instead of building a direct mapping function from images to 3D shape, our model learns to predict series of deformations to improve a coarse shape iteratively. Inspired by traditional multiple view geometry methods, our network samples nearby area around the initial mesh's vertex locations and reasons an optimal deformation using perceptual feature statistics built from multiple input images. Extensive experiments show that our model produces accurate 3D shapes that are not only visually plausible from the input perspectives, but also well aligned to arbitrary viewpoints. With the help of physically driven architecture, our model also exhibits generalization capability across different semantic categories, and the number of input images. Model analysis experiments show that our model is robust to the quality of the initial mesh and the error of camera pose, and can be combined with a differentiable renderer for test-time optimization.

Progressive DeepSSM: Training Methodology for Image-To-Shape Deep Models.

Statistical shape modeling (SSM) is an enabling quantitative tool to study anatomical shapes in various medical applications. However, directly using 3D images in these applications still has a long way to go. Recent deep learning methods have paved the way for reducing the substantial preprocessing steps to construct SSMs directly from unsegmented images. Nevertheless, the performance of these models is not up to the mark. Inspired by multiscale/multiresolution learning, we propose a new training strategy, progressive DeepSSM, to train image-to-shape deep learning models. The training is performed in multiple scales, and each scale utilizes the output from the previous scale. This strategy enables the model to learn coarse shape features in the first scales and gradually learn detailed fine shape features in the later scales. We leverage shape priors via segmentation-guided multi-task learning and employ deep supervision loss to ensure learning at each scale. Experiments show the superiority of models trained by the proposed strategy from both quantitative and qualitative perspectives. This training methodology can be employed to improve the stability and accuracy of any deep learning method for inferring statistical representations of anatomies from medical images and can be adopted by existing deep learning methods to improve model accuracy and training stability.

Generating Melodies from Melodic Outlines Towards an Improvisation Support System for Non-musicians

One promising approach for supporting non-musicians’ music improvisation is to enable them to input the coarse shape of melodies they want to play. Therefore, drawing a melodic outline is adopted as an input method of existing improvisation support systems. However, generating melodies from melodic outlines with deep neural networks has not been fully exploited. In this paper, we report an attempt to generate melodies from melodic outlines using a convolutional neural network. Given a melodic outline, it is reduced with two convolution layers and then coverted to a sequence of notes with two deconvolution layers. Objective and subjective evaluations imply that our model with sufficient filter width and channels could generate melodies of moderate, close quality to human melodies.

SPCNet: Stepwise Point Cloud Completion Network

AbstractHow will you repair a physical object with large missings? You may first recover its global yet coarse shape and stepwise increase its local details. We are motivated to imitate the above physical repair procedure to address the point cloud completion task. We propose a novel stepwise point cloud completion network (SPCNet) for various 3D models with large missings. SPCNet has a hierarchical bottom‐to‐up network architecture. It fulfills shape completion in an iterative manner, which 1) first infers the global feature of the coarse result; 2) then infers the local feature with the aid of global feature; and 3) finally infers the detailed result with the help of local feature and coarse result. Beyond the wisdom of simulating the physical repair, we newly design a cycle loss to enhance the generalization and robustness of SPCNet. Extensive experiments clearly show the superiority of our SPCNet over the state‐of‐the‐art methods on 3D point clouds with large missings. Code is available at https://github.com/1127368546/SPCNet.

The finite coarse shape groups

In this paper we define a new pointed finite coarse shape invariant - k-th finite coarse shape group of a pointed topological space and compare it with the corresponding shape and coarse shape group. We also define a functor πˇk⊛ from the pointed finite coarse shape category Sh⋆⊛ to the category Grp of all groups (for k=0 to the category Set⋆ of all pointed sets) associating with every pointed topological space (X,x0) a k-th finite coarse shape group πˇk⊛(X,x0) and with every finite coarse shape morphism F⊛:(X,x0)→(Y,y0) a homomorphism (a base point preserving function) πˇk⊛(F⊛):πˇk⊛(X,x0)→πˇk⊛(Y,y0). We show that, in a case of a pointed compact polyhedral system, functor πˇk⊛ is continuous, i.e., commutes with the inverse limit. Finally, finite coarse shape groups of some interesting spaces (dyadic solenoid and arbitrary stable pointed space) are calculated.

The finite coarse shape - inverse systems approach and intrinsic approach

Given an arbitrary category \(\mathcal{C}\), a category \(pro^{*^f}\)-\(\mathcal{C}\) is constructed such that the known \(pro\)-\(\mathcal{C}\) category may be considered as a subcategory of \(pro^{*^f}\)-\(\mathcal{C}\) and that \(pro^{*^f}\)-\(\mathcal{C}\) may be considered as a subcategory of \(pro^*\)-\(\mathcal{C}\). Analogously to the construction of the shape category \(Sh_{(\mathcal{C},\mathcal{D})}\) and the coarse category \(Sh^*_{(\mathcal{C},\mathcal{D})}\), an (abstract) finite coarse shape category \(Sh^{*^f}_{(\mathcal{C},\mathcal{D})}\) is obtained. Between these three categories appropriate faithful functors are defined. The finite coarse shape is also defined by an intrinsic approach using the notion of the \(\epsilon\)-continuity. The isomorphism of the finite coarse shape categories obtained by these two approaches is constructed. Besides, an overview of some basic properties related to the notion of the \(\epsilon\)-continuity is given.

High fidelity virtual try-on network via semantic adaptation and distributed componentization

Image-based virtual try-on systems have significant commercial value in online garment shopping. However, prior methods fail to appropriately handle details, so are defective in maintaining the original appearance of organizational items including arms, the neck, and in-shop garments. We propose a novel high fidelity virtual try-on network to generate realistic results. Specifically, a distributed pipeline is used for simultaneous generation of organizational items. First, the in-shop garment is warped using thin plate splines (TPS) to give a coarse shape reference, and then a corresponding target semantic map is generated, which can adaptively respond to the distribution of different items triggered by different garments. Second, organizational items are componentized separately using our novel semantic map-based image adjustment network (SMIAN) to avoid interference between body parts. Finally, all components are integrated to generate the overall result by SMIAN. A priori dual-modal information is incorporated in the tail layers of SMIAN to improve the convergence rate of the network. Experiments demonstrate that the proposed method can retain better details of condition information than current methods. Our method achieves convincing quantitative and qualitative results on existing benchmark datasets.

Enriched Pro-Categories and Shapes

Given a category C and a directed partially ordered set J, a certain category proJ-C on inverse systems in C is constructed such that the ordinary pro-category pro-C is the most special case of a singleton J ≡ {1}. Further, the known pro*-category pro*-C becomes proN-C. Moreover, given a pro-reflective category pair (C, D), the J-shape category ShJ(C, D) and the corresponding J-shape functor SJ are constructed which, in mentioned special cases, become the well known ones. Among several important properties, the continuity theorem for a J-shape category is established. It implies the “J-shape theory” is a genuine one such that the shape and the coarse shape theory are its very special instances.

The role of the eye region for familiar face recognition: Evidence from spatial low-pass filtering and contrast negation.

What information is used for familiar face recognition? While previous research suggests a particular importance of the eye region, information from the rest of the face also needs to be integrated. What type of information is used in conjunction with the eyes is largely unclear. In three experiments, participants were asked to recognise so-called face chimeras, in which the eye region was not manipulated, while the rest of the face was either presented in negative contrast (contrast chimeras) or low-pass filtered (blur chimeras). We show (1) that both chimeras are recognised substantially better than fully blurred faces, (2) that the recognition advantage for blur chimeras is specific to the eye region but cannot be explained by cues available in this part of the face alone, and (3) that a combination of negative contrast and blurring outside of the eye region eliminates the chimera advantage. We conclude that full-frequency but distorted surface reflectance cues (in contrast chimeras) or coarse shape information (in blur chimeras) can be used in combination with the eye region for effective face recognition. Our findings further suggest that the face recognition system can flexibly use both types of information, depending on availability.

Numerical approach to synthesizing realistic asteroid surfaces from morphological parameters

Context. The complex shape of asteroids and comets is a critical parameter in many scientific and operational studies. From the global irregular shape down to the local surface details, these topographies reflect the formation and evolutionary processes that remould the celestial body. Furthermore, these processes control how the surface will continue to evolve: from mass wasting on high slopes to spin-up due to anisotropic re-emission of thermal radiation. In addition, for space missions, the irregular coarse shape and complex landscape are a hazard to navigation, which must be accounted for in the planning phase. Aims. In this paper, we propose a novel method to synthesize physically correct 3D shape models of small celestial bodies, such as asteroids, to support the testing of a wide range of parameters in scientific and operational studies. Methods. We modeled virtual asteroid shapes using non-uniform sphere packings to represent the coarse shape, define an implicit surface, and then synthesize high-resolution topography with user-defined, locally controlled spot noise models. This effectively replaces the random noise model (e.g., Perlin noise) used in traditional approaches and allows us to construct a morphology based on actual physical shapes of the most common features observed on asteroids and comets. As an example of such a feature, we propose several kernel functions to add virtual craters to the coarse shape of the asteroid, of which the spatial distribution is controlled by typical crater production functions (e.g., a power law). Results. We demonstrate how this technique can be used to generate a variety of asteroid shapes and topographies using different cratering parameters and distributions. We apply our technique to artificially increase the resolution of existing models of the Didymos-Dimorphos system, the target of the Double Asteroid Redirection Test, and Hera missions. We show that our approach generates models that are suitable for typical analysis relying on detailed asteroid shapes, as well as operational scenarios for space missions. The meshes created with our algorithm can be directly used with existing visualization software and operations or science pipelines to generate data suitable for mission planning and to validate data analysis techniques.

Joint 3D facial shape reconstruction and texture completion from a single image

Recent years have witnessed significant progress in image-based 3D face reconstruction using deep convolutional neural networks. However, current reconstruction methods often perform improperly in self-occluded regions and can lead to inaccurate correspondences between a 2D input image and a 3D face template, hindering use in real applications. To address these problems, we propose a deep shape reconstruction and texture completion network, SRTC-Net, which jointly reconstructs 3D facial geometry and completes texture with correspondences from a single input face image. In SRTC-Net, we leverage the geometric cues from completed 3D texture to reconstruct detailed structures of 3D shapes. The SRTC-Net pipeline has three stages. The first introduces a correspondence network to identify pixel-wise correspondence between the input 2D image and a 3D template model, and transfers the input 2D image to a U-V texture map. Then we complete the invisible and occluded areas in the U-V texture map using an inpainting network. To get the 3D facial geometries, we predict coarse shape (U-V position maps) from the segmented face from the correspondence network using a shape network, and then refine the 3D coarse shape by regressing the U-V displacement map from the completed U-V texture map in a pixel-to-pixel way. We examine our methods on 3D reconstruction tasks as well as face frontalization and pose invariant face recognition tasks, using both in-the-lab datasets (MICC, MultiPIE) and in-the-wild datasets (CFP). The qualitative and quantitative results demonstrate the effectiveness of our methods on inferring 3D facial geometry and complete texture; they outperform or are comparable to the state-of-the-art.

Fine Wrinkling on Coarsely Meshed Thin Shells

We propose a new model and algorithm to capture the high-definition statics of thin shells via coarse meshes. This model predicts global, fine-scale wrinkling at frequencies much higher than the resolution of the coarse mesh; moreover, it is grounded in the geometric analysis of elasticity, and does not require manual guidance, a corpus of training examples, nor tuning of ad hoc parameters. We first approximate the coarse shape of the shell using tension field theory, in which material forces do not resist compression. We then augment this base mesh with wrinkles, parameterized by an amplitude and phase field that we solve for over the base mesh, which together characterize the geometry of the wrinkles. We validate our approach against both physical experiments and numerical simulations, and we show that our algorithm produces wrinkles qualitatively similar to those predicted by traditional shell solvers requiring orders of magnitude more degrees of freedom.