3D Shape Research Articles

Hyperspectral imaging in animal coloration research: A user-friendly pipeline for image generation, analysis, and integration with 3D modeling.

Hyperspectral imaging-a technique that combines the high spectral resolution of spectrophotometry with the high spatial resolution of photography-holds great promise for the study of animal coloration. However, applications of hyperspectral imaging to questions about the ecology and evolution of animal color remain relatively rare. The approach can be expensive and unwieldy, and we lack user-friendly pipelines for capturing and analyzing hyperspectral data in the context of animal color. Fortunately, costs are decreasing and hyperspectral imagers are improving, particularly in their sensitivity to wavelengths (including ultraviolet) visible to diverse animal species. To highlight the potential of hyperspectral imaging for animal coloration studies, we developed a pipeline for capturing, sampling, and analyzing hyperspectral data (here, in the 325 nm to 700 nm range) using avian museum specimens. Specifically, we used the pipeline to characterize the plumage colors of the King bird-of-paradise (Cicinnurus regius), Magnificent bird-of-paradise (C. magnificus), and their putative hybrid, the King of Holland's bird-of-paradise (C. magnificus x C. regius). We also combined hyperspectral data with 3D digital models to supplement hyperspectral images of each specimen with 3D shape information. Using visual system-independent methods, we found that many plumage patches on the hybrid King of Holland's bird-of-paradise are-to varying degrees-intermediate relative to those of the parent species. This was true of both pigmentary and structurally colored plumage patches. Using visual system-dependent methods, we showed that only some of the differences in plumage patches among the hybrid and its parent species would be perceivable by birds. Hyperspectral imaging is poised to become the gold standard for many animal coloration applications: comprehensive reflectance data-across the entire surface of an animal specimen-can be obtained in a matter of minutes. Our pipeline provides a practical and flexible roadmap for incorporating hyperspectral imaging into future studies of animal color.

Mapping genes for human face shape: Exploration of univariate phenotyping strategies

Human facial shape, while strongly heritable, involves both genetic and structural complexity, necessitating precise phenotyping for accurate assessment. Common phenotyping strategies include simplifying 3D facial features into univariate traits such as anthropometric measurements (e.g., inter-landmark distances), unsupervised dimensionality reductions (e.g., principal component analysis (PCA) and auto-encoder (AE) approaches), and assessing resemblance to particular facial gestalts (e.g., syndromic facial archetypes). This study provides a comparative assessment of these strategies in genome-wide association studies (GWASs) of 3D facial shape. Specifically, we investigated inter-landmark distances, PCA and AE-derived latent dimensions, and facial resemblance to random, extreme, and syndromic gestalts within a GWAS of 8,426 individuals of recent European ancestry. Inter-landmark distances exhibit the highest SNP-based heritability as estimated via LD score regression, followed by AE dimensions. Conversely, resemblance scores to extreme and syndromic facial gestalts display the lowest heritability, in line with expectations. Notably, the aggregation of multiple GWASs on facial resemblance to random gestalts reveals the highest number of independent genetic loci. This novel, easy-to-implement phenotyping approach holds significant promise for capturing genetically relevant morphological traits derived from complex biomedical imaging datasets, and its applications extend beyond faces. Nevertheless, these different phenotyping strategies capture different genetic influences on craniofacial shape. Thus, it remains valuable to explore these strategies individually and in combination to gain a more comprehensive understanding of the genetic factors underlying craniofacial shape and related traits.

Image-guided optogenetic spatiotemporal tissue patterning using μPatternScope

In the field of tissue engineering, achieving precise spatiotemporal control over engineered cells is critical for sculpting functional 2D cell cultures into intricate morphological shapes. In this study, we engineer light-responsive mammalian cells and target them with dynamic light patterns to realize 2D cell culture patterning control. To achieve this, we developed μPatternScope (μPS), a modular framework for software-controlled projection of high-resolution light patterns onto microscope samples. μPS comprises hardware and software suite governing pattern projection and microscope maneuvers. Together with a 2D culture of the engineered cells, we utilize μPS for controlled spatiotemporal induction of apoptosis to generate desired 2D shapes. Furthermore, we introduce interactive closed-loop patterning, enabling a dynamic feedback mechanism between the measured cell culture patterns and the light illumination profiles to achieve the desired target patterning trends. Our work offers innovative tools for advanced tissue engineering applications through seamless fusion of optogenetics, optical engineering, and cybernetics.

3D human avatar reconstruction with neural fields: A recent survey

3D human avatar reconstruction aims to reconstruct the 3D geometric shape and appearance of the human body from various data inputs, such as images, videos, and depth information, acting as a key component in human-oriented 3D vision in the metaverse. With the progress in neural fields for 3D reconstruction in recent years, significant advancements have been made in this research area for shape accuracy and appearance quality. Meanwhile, substantial efforts on dynamic avatars with the representation of neural fields have exhibited their effect. Although significant improvements have been achieved, challenges still exist in in-the-wild and complex environments, detailed shape recovery, and interactivity in real-world applications. In this survey, we present a comprehensive overview of 3D human avatar reconstruction methods using advanced neural fields. We start by introducing the background of 3D human avatar reconstruction and the mainstream paradigms with neural fields. Subsequently, representative research studies are classified based on their representation and avatar partswith detailed discussion. Moreover, we summarize the commonly used available datasets, evaluation metrics, and results in the research area. In the end, we discuss the open problems and highlight the promising future directions, hoping to inspire novel ideas and promote further research in this area.

Impact of solar-wind turbulence on a planetary bow shock

Context. The interaction of the solar-wind plasma with a magnetized planet generates a bow-shaped shock ahead of the wind. Over recent decades, near-Earth spacecraft observations have provided insights into the physics of the bow shock, and the findings suggest that solar-wind intrinsic turbulence influences the bow shock dynamics. On the other hand, theoretical studies, primarily based on global numerical simulations, have not yet investigated the global three-dimensional (3D) interaction between a turbulent solar wind and a planetary magnetosphere. This paper addresses this gap for the first time by presenting an investigation of the global dynamics of this interaction that provides new perspectives on the underlying physical processes. Aims. We use the newly developed numerical code MENURA to examine how the turbulent nature of the solar wind influences the 3D structure and dynamics of magnetized planetary environments, such as those of Mercury, Earth, and magnetized Earth-like exoplanets. Methods. We used the hybrid particle-in-cell code MENURA to conduct 3D simulations of the turbulent solar wind and its interaction with an Earth-like magnetized planet through global numerical simulations of the magnetosphere and its surroundings. MENURA runs in parallel on graphics processing units, enabling efficient and self-consistent modeling of turbulence. Results. By comparison with a case in which the solar wind is laminar, we show that solar-wind turbulence globally influences the shape and dynamics of the bow shock, the magnetosheath structures, and the ion foreshock dynamics. Also, a turbulent solar wind disrupts the coherence of foreshock fluctuations, induces large fluctuations on the quasi-perpendicular surface of the bow shock, facilitates the formation of bubble-like structures near the nose of the bow shock, and modifies the properties of the magnetosheath region. Conclusions. The turbulent nature of the solar wind impacts the 3D shape and dynamics of the bow shock, magnetosheath, and ion foreshock region. This influence should be taken into account when studying solar-wind-planet interactions in both observations and simulations. We discuss the relevance of our findings for current and future missions launched into the heliosphere.

3D Reconstruction From a Single Sketch via View-Dependent Depth Sampling.

Reconstructing a 3D shape based on a single sketch image is challenging due to the inherent sparsity and ambiguity present in sketches. Existing methods lose fine details when extracting features to predict 3D objects from sketches. Upon analyzing the 3D-to-2D projection process, we observe that the density map, characterizing the distribution of 2D point clouds, can serve as a proxy to facilitate the reconstruction process. In this work, we propose a novel sketch-based 3D reconstruction model named SketchSampler. It initiates the process by translating a sketch through an image translation network into a more informative 2D representation, which is then used to generate a density map. Subsequently, a two-stage probabilistic sampling process is employed to reconstruct a 3D point cloud: first, recovering the 2D points (i.e., the x and y coordinates) by sampling the density map; and second, predicting the depth (i.e., the z coordinate) by sampling the depth values along the ray determined by each 2D point. Additionally, we convert the reconstructed point cloud into a 3D mesh for wider applications. To reduce ambiguity, we incorporate hidden lines in sketches. Experimental results demonstrate that our proposed approach significantly outperforms other baseline methods.

Hybrid dynamically cross-linked polyurethanes with room temperature self-healing ability for light-responsive actuator

Light-responsive actuators (LRAs) with cross-linked networks that demonstrate fast response and super mechanical strength are essential for practical applications. However, the permanent cross-linked structure typically stabilizes polymer networks and makes them difficult to reprogram or self-heal. Herein, a novel crosslinked light-responsive polyurethane actuator (CPU-DA1SS2) with room temperature self-healing properties was prepared by constructing a hybrid physical and chemical cross-linked polymer network via dynamic DA bonds, SS bonds, and hydrogen bonds. The CPU-DA1SS2 light-responsive actuator exhibits high tensile strength (25.52 MPa) and excellent self-healing efficiency (90.16 %) at room temperature. Moreover, the hybrid dynamic cross-linked network endows the LRAs with liquid-state remold ability and solid-state plasticity, which further contributes to diverse 3D geometric shape remolding. Such a hybrid, dynamic, cross-linked strategy could be utilized to design smart materials that are self-healing and recyclable without sacrificing their mechanical properties, helping to broaden the application areas of LRA materials.

High-throughput 3D shape completion of potato tubers on a harvester

Potato yield is an important metric for farmers to further optimize their cultivation practices. Potato yield can be estimated on a harvester using an RGB-D camera that can estimate the three-dimensional (3D) volume of individual potato tubers. A challenge, however, is that the 3D shape derived from RGB-D images is only partially completed, underestimating the actual volume. To address this issue, we developed a 3D shape completion network, called CoRe++, which can complete the 3D shape from RGB-D images. CoRe++ is a deep learning network that consists of a convolutional encoder and a decoder. The encoder compresses RGB-D images into latent vectors that are used by the decoder to complete the 3D shape using the deep signed distance field network (DeepSDF). To evaluate our CoRe++ network, we collected partial and complete 3D point clouds of 339 potato tubers on an operational harvester in Japan. On the 1425 RGB-D images in the test set (representing 51 unique potato tubers), our network achieved a completion accuracy of 2.8 mm on average. For volumetric estimation, the root mean squared error (RMSE) was 22.6 ml, and this was better than the RMSE of the linear regression (31.1 ml) and the base model (36.9 ml). We found that the RMSE can be further reduced to 18.2 ml when performing the 3D shape completion in the center of the RGB-D image. With an average 3D shape completion time of 10 ms per tuber, we can conclude that CoRe++ is both fast and accurate enough to be implemented on an operational harvester for high-throughput potato yield estimation. CoRe++’s high-throughput and accurate processing allows it to be applied to other tuber, fruit and vegetable crops, thereby enabling versatile, accurate and real-time yield monitoring in precision agriculture. Our code, network weights and dataset are publicly available at https://github.com/UTokyo-FieldPhenomics-Lab/corepp.git.

Phase-shifting profilometry for 3D shape measurement of moving objects on production lines

It is challenging for phase-shifting profilometry to measure objects in motion on production lines due to two main issues. Firstly, the movement of the object causes a mismatch in its position across different fringe patterns. Additionally, there is a violation of the ideal phase shift between these fringe patterns. To address these problems, we propose a method that takes advantage of the known moving direction of objects on production lines. By aligning the isophase direction of fringe patterns with the known direction of the moving objects, our method essentially solves the well-known phase error problem caused by object motion. Furthermore, we have developed a projection and imaging technique to track the measured object that is covered with different fringe patterns. Both of the previously listed issues have been resolved in this manner. The proposed method is validated by comparisons with state-of-the-art methods, proving its simplicity, broader applicability, and improved measurement accuracy.

Memory Color Study: Effects of Chromatic Texture and 3D shape

Memory Color Study: Effects of Chromatic Texture and 3D shape

Generation and Editing of 2D Shapes Using a Branched Representation

In this article, we propose a new planar shape representation, the medial branch graph representation (MBGR) which allows to easily generate, vary and edit all-new sorts of parametric shapes. Each MBGR shape is described by a collection of connected control circles organized as a graph. Each pair of connected circles generates a branch of the shape. The boundaries of the branches are designed using cubic Bézier curves which enable shape representation in a compact SVG format. Each circle is associated with a regularity parameter locally controlling the smoothness of the branch connections. In this way, we can manage shapes having both sharp corners and smooth boundaries. To illustrate the potential of the MBGR representation, we have created an online facility ( https://mbgrg.github.io/mbgrg/ ) where the user can create/edit MBGR shapes automatically or interactively.

Characterization of Direct Ink Writing carbon fiber composite structures with serial sectioning and DREAM.3D

Direct Ink Writing (DIW) combines the flexibility of 3D printing with increased material applications such as thermoset carbon fiber composites, ceramic composites, and metals. The usefulness of direct ink writing, like many additive manufacturing (AM) processes, remains limited for reasons ranging from quality control to lack of process parameter optimization. This study looks to introduce a methodology for characterizing direct ink written carbon fiber composites to facilitate exploration into the relationships between process parameters and material structure. The presented study utilized nine 3D specimens of direct ink writing carbon fiber composites printed with varying process parameters – speed differential, layer height, step-over distance, and nozzle diameter – as the data set. The data was collected with an automatic serial sectioning system, LEROY, from the Air Force Research Laboratory. The collected data was processed in DREAM.3D and analyzed with statistical comparisons of 2D orientation distributions of the fibers, 2D size distributions of the voids, and 2D shape distributions of the voids.

Development of a tumor organoid culture system with peptide-based hydrogels

Peptide-based hydrogel, the polymer materials with a special network structure, are widely used in various fields of biomedicine due to their stable properties and biocompatibility. Environment-responsive self-assembled peptide aqueous solutions can respond to environment changes by the self-assembly of peptides into nanofiber networks. Peptide-based hydrogels well simulate the extracellular matrix and cell growth microenvironment, being suitable for 3D cell culture and organoid culture. To establish a tumor organoid culture system with peptide-based hydrogels, we cultured Panc-1, U87, and H358 cells in a 3D spherical manner using CulX Ⅱ peptide-based hydrogels in 24-well plates for 15 days. The organoids showed a 3D spherical shape, and their sizes increased with the extension of the culture time, with a final diameter ranging from 150 to 300 μm. The organoids had a large number, varying sizes, good cell viability, clear edges, and a good shape, which indicated successful organoid construction. The tumor organoid culture system established in this study with CulX Ⅱ peptide-based hydrogels provides a model for studying tumor pathogenesis, drug development, and tumor suppression.

Gold nanotube self-assemblage from tubular W2@Au16 cluster.

While the pure Au16 cluster tends to exist in a 3D cage shape, the doubly doped W2@Au16 is found to prefer a tubular form containing three five-membered Au rings stabilized by the W2 dimer vertically placed inside a tube-like Au16 framework. Formation of self-assembled nanotubes containing five-membered Au rings from the tube-like W2@Au16 cluster is also feasible, and such a tubular structure constitutes an ideal building block for generating gold-assembled nanotubes, providing us with a useful approach for designing some novel tailor-made materials with interesting optoelectronic properties. Density functional theory (DFT) calculations using the TPSS functional and the cc-pVDZ-PP basis set are employed to determine some noticeable effects of the doping of the W2 dimer on the structure, stability, and optical properties of pure Au16 gold clusters.

A novel framework for predicting 3D scene infrared radiation characteristics through AI-enhanced thermodynamic modeling

In the evolving landscape of modern warfare, virtual battlefield construction technology significantly impacts fields such as weapon development and tactical formulation. Currently, 3D infrared scene simulation face challenges like high computational resource demands, difficulty in depicting interaction mechanisms between dynamic targets and environments, and a lack of reliable validation for algorithms. In this paper, we propose a three-dimensional infrared characteristics prediction framework (3DICPF) that integrates data-driven and theoretical algorithms, capable of rapidly constructing realistic 3D infrared scenes based on limited input information. Considering the extremely high cost of data acquisition, a step-by-step construction approach is adopted to reduce dependence on data. For the aligned image inputs, a 3D shape reconstruction network is utilized to obtain the mesh calculation domain of the target. Subsequently, 3DICPF employs two temperature field reconstruction networks to obtain the temperature distribution of the target. Finally, by using Monte Carlo ray tracing techniques, 3DICPF calculates the radiative interactions between the target and the background, achieving real-time infrared scene rendering. Moreover, confirmatory experiments with a shape-similar tank replica were conducted, demonstrating that 3DICPF could simulate infrared scene images with an average similarity above 88.13%. Overall, our research presents a significant advancement in virtual warfare technology, offering an efficient tool for enhancing strategic military planning and decision-making.

Binocular composite grayscale fringe projection profilometry based on deep learning for single-shot 3D measurements

The efficiency and precision of 3D shape reconstruction has been a focus in the fringe projection profilometry (FPP). However, achieving high-quality 3D measurement for isolated or overlapping objects from single fringe image is still a challenging task in the field. In this paper, a binocular composite grayscale fringe projection profilometry (BCGFPP) based on deep learning is proposed, in which a two-stage one-to-three network (TONet) is trained to predict the images required for phase unwrapping. The obtained absolute phase map exhibits high precision and eliminates deviation error and periodic ambiguity typically encountered in traditional sinusoidal composite fringe coding scheme. Haar transform principle is employed to form a Haar-like composite fringe pattern (HCFP), which consists of three different frequencies, serving as the input. TONet architecture is designed to predict images required by the tri-frequency four-step phase-shifting method (TFPM). Further, the absolute phase is calculated and the disparity map is obtained by matching the absolute phase of the left and right cameras. Finally, the 3D shape of the object can be restored by the system calibration parameters. Experimental results demonstrate the approach can greatly reduce the number of fringes required and achieve the accuracy of the absolute phase close to the training set.

Relation Constrained Capsule Graph Neural Networks for Non-Rigid Shape Correspondence

Non-rigid 3D shape correspondence aims to establish dense correspondences between two non-rigidly deformed 3D shapes. However, the variability and symmetry of non-rigid shapes usually lead to mismatches due to shape deformation, topological changes, or data with severe noise. To finding an accurate correspondence between 3D dynamic shapes for the local deformation complexity, this article proposes a Relation Constrained Capsule Graph Network (RC-CGNet), which combines global and local features by encouraging the relation constraints between the embedding feature space and the input shape space based on the functional maps framework. Specifically, we design a Diffusion Graph Attention Network (DGANet) to segment the surface into parts with correct edge boundary between two regions. The Minimum Spanning Tree (MST) of geodesic curves among the singularities obtained from the segmented parts is added as relation constraints, which can compute isometric correspondences in both direct and symmetric directions. Besides that, the relation-and-attention constrained neural networks are designed to learn the shape correspondence via attention-aware CapsNet and functional maps under relation constraints. To improve the convergence speed and matching accuracy, we propose an optimized residual network structure based on the Nesterov Accelerated Gradient (NAG) to extract local features, and use graph convolution structure to extract global features. Moreover, a lightweight Gated Attention Module (GAM) is designed to fuse global and local features to obtain a richer feature representation. Since the capsule network has better spatial reasoning ability than the traditional convolutional neural network, our novel network architecture is a dual-route capsule network based on Routing Attention Fusion Block (RAFB), filtering low-discriminative capsules from a holistic view by exploiting geometric hierarchical relationships of semantic parts. Experiments on open datasets show that our method has excellent accuracy and wide adaptability.

Enhancing the Aesthetics of 3D Shapes via Reference-based Editing

While there have been previous works that explored methods to enhance the aesthetics of images, the automated beautification of 3D shapes has been limited to specific shapes such as 3D face models. In this paper, we introduce a framework to automatically enhance the aesthetics of general 3D shapes. Our approach employs a reference-based beautification strategy. We first performed data collection to gather the aesthetics ratings of various 3D shapes to create a 3D shape aesthetics dataset. Then we perform reference-based editing to edit the input shape and beautify it by making it look more like some reference shape that is aesthetic. Specifically, we propose a reference-guided global deformation framework to coherently deform the input shape such that its structural proportions will be closer to those of the reference shape. We then optionally transplant some local aesthetic parts from the reference to the input to obtain the beautified output shapes. Comparisons show that our reference-guided 3D deformation algorithm outperforms existing techniques. Furthermore, quantitative and qualitative evaluations demonstrate that the performance of our aesthetics enhancement framework is consistent with both human perception and existing 3D shape aesthetics assessment.

MATTopo: Topology-preserving Medial Axis Transform with Restricted Power Diagram

We present a novel topology-preserving 3D medial axis computation framework based on volumetric restricted power diagram (RPD), while preserving the medial features and geometric convergence simultaneously, for both 3D CAD and organic shapes. The volumetric RPD discretizes the input 3D volume into sub-regions given a set of medial spheres. With this intermediate structure, we convert the homotopy equivalency between the generated medial mesh and the input 3D shape into a localized contractibility checking for each restricted element (power cell, power face, power edge), by checking their connected components and Euler characteristics. We further propose a fractional Euler characteristic algorithm for efficient GPU-based computation of Euler characteristic for each restricted element on the fly while computing the volumetric RPD. Compared with existing voxel-based or point-cloud-based methods, our approach is the first to adaptively and directly revise the medial mesh without globally modifying the dependent structure, such as voxel size or sampling density, while preserving its topology and medial features. In comparison with the feature preservation method MATFP [Wang et al. 2022], our method provides geometrically comparable results with fewer spheres and more robustly captures the topology of the input 3D shape.

Bijective Volumetric Mapping via Star Decomposition

A method for the construction of bijective volumetric maps between 3D shapes is presented. Arbitrary shapes of ball-topology are supported, overcoming restrictions of previous methods to convex or star-shaped targets. In essence, the mapping problem is decomposed into a set of simpler mapping problems, each of which can be solved with previous methods for discrete star-shaped mapping problems. Addressing the key challenges in this endeavor, algorithms are described to reliably construct structurally compatible partitions of two shapes with constraints regarding star-shapedness and to compute a parsimonious common refinement of two triangulations.