3D Datasets Research Articles

TAG-SPARK: Empowering High-Speed Volumetric Imaging With Deep Learning and Spatial Redundancy.

Two-photon high-speed fluorescence calcium imaging stands as a mainstream technique in neuroscience for capturing neural activities with high spatiotemporal resolution. However, challenges arise from the inherent tradeoff between acquisition speed and image quality, grappling with a low signal-to-noise ratio (SNR) due to limited signal photon flux. Here, a contrast-enhanced video-rate volumetric system, integrating a tunable acoustic gradient (TAG) lens-based high-speed microscopy with a TAG-SPARK denoising algorithm is demonstrated. The former facilitates high-speed dense z-sampling at sub-micrometer-scale intervals, allowing the latter to exploit the spatial redundancy of z-slices for self-supervised model training. This spatial redundancy-based approach, tailored for 4D (xyzt) dataset, not only achieves >700% SNR enhancement but also retains fast-spiking functional profiles of neuronal activities. High-speed plus high-quality images are exemplified by in vivo Purkinje cells calcium observation, revealing intriguing dendritic-to-somatic signal convolution, i.e., similar dendritic signals lead to reverse somatic responses. This tailored technique allows for capturing neuronal activities with high SNR, thus advancing the fundamental comprehension of neuronal transduction pathways within 3D neuronal architecture.

Semantic Segmentation-Driven Integration of Point Clouds from Mobile Scanning Platforms in Urban Environments

Precise and complete 3D representations of architectural structures or industrial sites are essential for various applications, including structural monitoring or cadastre. However, acquiring these datasets can be time-consuming, particularly for large objects. Mobile scanning systems offer a solution for such cases. In the case of complex scenes, multiple scanning systems are required to obtain point clouds that can be merged into a comprehensive representation of the object. Merging individual point clouds obtained from different sensors or at different times can be difficult due to discrepancies caused by moving objects or changes in the scene over time, such as seasonal variations in vegetation. In this study, we present the integration of point clouds obtained from two mobile scanning platforms within a built-up area. We utilized a combination of a quadruped robot and an unmanned aerial vehicle (UAV). The PointNet++ network was employed to conduct a semantic segmentation task, enabling the detection of non-ground objects. The experimental tests used the Toronto 3D dataset and DALES for network training. Based on the performance, the model trained on DALES was chosen for further research. The proposed integration algorithm involved semantic segmentation of both point clouds, dividing them into square subregions, and performing subregion selection by checking the emptiness or when both subregions contained points. Parameters such as local density, centroids, coverage, and Euclidean distance were evaluated. Point cloud merging and augmentation enhanced with semantic segmentation and clustering resulted in the exclusion of points associated with these movable objects from the point clouds. The comparative analysis of the method and simple merging was performed based on file size, number of points, mean roughness, and noise estimation. The proposed method provided adequate results with the improvement of point cloud quality indicators.

Methods for calculating gliding-box lacunarity efficiently on large datasets

Lacunarity has proven to be a useful, multifaceted tool for image analysis in several different scientific fields, from geography to virology, which has lent increasing importance to the lacunarity analysis of large datasets. It can be most reliably calculated with the so-called gliding-box method, but the evaluation process can be exceedingly time-consuming and unviable as this algorithm is not designed to operate on large datasets. Here we introduce two novel methods that can calculate gliding-box lacunarity orders of magnitude faster than the original method without any loss of accuracy. We compare these methods with the original as well as with two already existing optimized methods based on runtime memory usage and complexity. The application of all five methods for both 2D and 3D datasets analysis confirms that each of the four optimized methods are orders of magnitude faster than the original one, but each has its advantages and limitations.

Skeleton-guided 3D convolutional neural network for tubular structure segmentation.

Accurate segmentation of tubular structures is crucial for clinical diagnosis and treatment but is challenging due to their complex branching structures and volume imbalance. The purpose of this study is to propose a 3D deep learning network that incorporates skeleton information to enhance segmentation accuracy in these tubular structures. Our approach employs a 3D convolutional network to extract 3D tubular structures from medical images such as CT volumetric images. We introduce a skeleton-guided module that operates on extracted features to capture and preserve the skeleton information in the segmentation results. Additionally, to effectively train our deep model in leveraging skeleton information, we propose a sigmoid-adaptive Tversky loss function which is specifically designed for skeleton segmentation. We conducted experiments on two distinct 3D medical image datasets. The first dataset consisted of 90 cases of chest CT volumetric images, while the second dataset comprised 35 cases of abdominal CT volumetric images. Comparative analysis with previous segmentation approaches demonstrated the superior performance of our method. For the airway segmentation task, our method achieved an average tree length rate of 93.0%, a branch detection rate of 91.5%, and a precision rate of 90.0%. In the case of abdominal artery segmentation, our method attained an average precision rate of 97.7%, a recall rate of 91.7%, and an F-measure of 94.6%. We present a skeleton-guided 3D convolutional network to segment tubular structures from 3D medical images. Our skeleton-guided 3D convolutional network could effectively segment small tubular structures, outperforming previous methods.

Striking the perfect balance: Multi-objective optimization for minimizing deployment cost and maximizing coverage with Harmony Search

In the Internet of Things (IoT) era, wireless sensor networks play a critical role in communication systems. One of the most crucial problems in wireless sensor networks is the sensor deployment problem, which attempts to provide a strategy to place the sensors within the surveillance area so that two fundamental criteria of wireless sensor networks, coverage and connectivity, are guaranteed. In this paper, we look to solve the multi-objective deployment problem so that area coverage is maximized and the number of nodes used is minimized. Since Harmony Search is a simple yet suitable algorithm for our work, we propose Harmony Search algorithm along with various enhancement proposals, including heuristic initialization, random sampling of sensor types, weighted fitness evaluation, and using different components in the fitness function, to provide a solution to the problem of sensor deployment in a heterogeneous wireless sensor network where sensors have different sensing ranges. On top of that, the probabilistic sensing model is used to reflect how the sensors work realistically. We also provide the extension of our solution to 3D areas and propose a realistic 3D dataset to evaluate it. The simulation results show that the proposed algorithms solve the area coverage problem more efficiently than previous algorithms. Our best proposal demonstrates significant improvements in coverage ratio by 10.20% and cost saving by 27.65% compared to the best baseline in a large-scale evaluation.

The NOAH HSQC-COSY module revisited: A theoretical and practical comparison of pulse sequences

NMR supersequences, as exemplified by the NOAH (NMR by Ordered Acquisition using 1H detection) technique, are a powerful way of acquiring multiple 2D data sets in much shorter durations. This is accomplished through targeted excitation and detection of the magnetisation belonging to specific isotopologues (‘magnetisation pools’). Separately, the HSQC-COSY experiment has recently seen an increase in popularity due to the high signal dispersion in the indirect dimension and the removal of ambiguity traditionally associated with HSQC-TOCSY experiments. Here, we describe how the HSQC-COSY experiment can be integrated as a ‘module’ within NOAH supersequences. The benefits and drawbacks of several different pulse sequence implementations are discussed, with a particular focus on how sensitivities of other modules in the same supersequence are affected.

Deep learning for 3D object recognition: A survey

With the growing availability of extensive 3D datasets and the rapid progress in computational power, deep learning (DL) has emerged as a highly promising approach for learning from 3D data, addressing critical tasks like object detection, segmentation, and recognition. Despite the unique challenges in processing geometry data with deep neural networks, recent advancements in DL for 3D object recognition have shown remarkable success, with various methods proposed to tackle different issues. This paper aims to stimulate future research by providing a comprehensive review of recent progress in DL techniques for 3D object recognition, which are systematically categorized based on their learning behavior. We discuss the advantages, limitations, and application of each approach, highlighting their performance in 3D object classification on benchmark datasets such as ModelNet, ScanObjectNN, and Sydney Urban Object. The survey offers insightful observations and inspires future research directions.

Estimating the effects of smolt size and migration timing on salmon marine survival using a multivariate mixed‐effect model

Abstract When the conditions encountered by a migratory species are highly variable, the timing of migration and migrant condition are critical to survival. Spring/summer‐run Chinook salmon in the Snake River of the Northwestern United States are listed as threatened under the US Endangered Species Act because of decades of poor adult returns; juvenile smolt‐to‐adult survival is often 0.5% or less. Juveniles pass through eight dams en route to the ocean, and despite changes to the hydropower system, there has been little recent improvement in survival. We analysed the effect of migration date, fish length and their interaction on the survival to adulthood of 409,747 wild juvenile Chinook salmon as they migrated through a large, dynamic riverine ecosystem and through the ocean, measured over 20 years. Parametric 2D smoothers are often used to quantify unexplained heterogeneity across space; however, these same statistical models can be applied to 2D datasets to uncover patterns of covariance of two biological processes important to management actions. We employ Gaussian Markov random fields (GMRF) and a 2D smoother to estimates of fish survival to identify the nature of variation in survival with respect to length and date. We found significant effects of fish length, migration date and the date‐by‐length interaction on resulting riverine and marine survival. Unlike the gradual shifts in survival in relation to length and timing that marginal estimates demonstrate, we show that within year, changes in survival with length and timing are often more abrupt than previously thought. Synthesis and applications: Inter‐annually, we identified distinct combinations of fish length and migration timing leading to elevated smolt‐to‐adult survival. Often these ‘hotspots’ are compressed into fish sizes greater than 100 mm and passage timings of only a week or two, demonstrating that even modest shifts in body size and migration timing can have a significant effect on survival. Further research can concentrate on these distinct annual periods to identify sources of mortality that have previously been elusive.

Characterizing the dynamics of multi-scale global high impact weather events

The quantitative characterization and prediction of localized severe weather events that emerge as coherences generated by the highly non-linear interacting multivariate dynamics of global weather systems poses a significant challenge whose solution is increasingly important in the face of climate change where weather extremes are on the rise. As weather measurement systems (multiband satellite, radar, etc) continue to dramatically improve, increasingly complex time-dependent multivariate 3D datasets offer the potential to inform such problems but pose an increasingly daunting computational challenge. Here we describe the application to global weather systems of a novel computational method called the Entropy Field Decomposition (EFD) capable of efficiently characterizing coherent spatiotemporal structures in non-linear multivariate interacting physical systems. Using the EFD derived system configurations, we demonstrate the application of a second novel computational method called Space-Time Information Trajectories (STITs) that reveal how spatiotemporal coherences are dynamically connected. The method is demonstrated on the specific phenomenon known as atmospheric rivers (ARs) which are a prime example of a highly coherent, in both space and time, severe weather phenomenon whose generation and persistence are influenced by weather dynamics on a wide range of spatial and temporal scales. The EFD reveals how the interacting wind vector field and humidity scalar field couple to produce ARs, while the resulting STITS reveal the linkage between ARs and large-scale planetary circulations. The focus on ARs is also motivated by their devastating social and economic effects that have made them the subject of increasing scientific investigation to which the EFD may offer new insights. The application of EFD and STITs to the broader range of severe weather events is discussed.

3D displacement measurement using a single-camera and mesh deformation neural network

Computer vision-based methods for civil structure’s vibration displacement measurement have emerged as useful tools in the recent years. These methods offer several benefits including non-contact measurements, cost-effectiveness, and the ability to capture full-field displacement. Yet, there remain certain challenges. Measuring vibration displacement in 3D typically requires multiple cameras, adding complexity to camera configurations. Moreover, existing methods relied heavily on physical markers or natural key points. Placing physical markers on structures is often impractical, and natural key points are difficult to detect on structures with few distinct features or during rapid movements. Contrary to previous approaches, this paper presents a novel technique that uses a monocular camera for 3D displacement measurements. This technique obviates the need for physical markers or the reliance on natural key points, representing a significant advancement. Central to the method is a deep neural network designed to predict 3D mesh deformation directly from a single image input, combined with an initial 3D cube mesh input. A synthetic 3D dataset is generated to train the neural network. In the testing phase for real structures, advanced video segmentation method is employed to remove the background in order to enhance the prediction accuracy. The practical efficacy of this methodology is validated in a laboratory through a series of experimental tests on beam structures, demonstrating reliable results and application potentials.

3D genomic features across >50 diverse cell types reveal insights into the genomic architecture of childhood obesity.

The prevalence of childhood obesity is increasing worldwide, along with the associated common comorbidities of type 2 diabetes and cardiovascular disease in later life. Motivated by evidence for a strong genetic component, our prior genome-wide association study (GWAS) efforts for childhood obesity revealed 19 independent signals for the trait; however, the mechanism of action of these loci remains to be elucidated. To molecularly characterize these childhood obesity loci we sought to determine the underlying causal variants and the corresponding effector genes within diverse cellular contexts. Integrating childhood obesity GWAS summary statistics with our existing 3D genomic datasets for 57 human cell types, consisting of high-resolution promoter-focused Capture-C/Hi-C, ATAC-seq, and RNA-seq, we applied stratified LD score regression and calculated the proportion of genome-wide SNP heritability attributable to cell type-specific features, revealing pancreatic alpha cell enrichment as the most statistically significant. Subsequent chromatin contact-based fine-mapping was carried out for genome-wide significant childhood obesity loci and their linkage disequilibrium proxies to implicate effector genes, yielded the most abundant number of candidate variants and target genes at the BDNF, ADCY3, TMEM18 and FTO loci in skeletal muscle myotubes and the pancreatic beta-cell line, EndoC-BH1. One novel implicated effector gene, ALKAL2 - an inflammation-responsive gene in nerve nociceptors - was observed at the key TMEM18 locus across multiple immune cell types. Interestingly, this observation was also supported through colocalization analysis using expression quantitative trait loci (eQTL) derived from the Genotype-Tissue Expression (GTEx) dataset, supporting an inflammatory and neurologic component to the pathogenesis of childhood obesity. Our comprehensive appraisal of 3D genomic datasets generated in a myriad of different cell types provides genomic insights into pediatric obesity pathogenesis.

3D chromatin-based variant-to-gene maps across 57 human cell types reveal the cellular and genetic architecture of autoimmune disease susceptibility.

A portion of the genetic basis for many common autoimmune disorders has been uncovered by genome-wide association studies (GWAS), but GWAS do not reveal causal variants, effector genes, or the cell types impacted by disease-associated variation. We have generated 3D genomic datasets consisting of promoter-focused Capture-C, Hi-C, ATAC-seq, and RNA-seq and integrated these data with GWAS of 16 autoimmune traits to physically map disease-associated variants to the effector genes they likely regulate in 57 human cell types. These 3D maps of gene cis-regulatory architecture are highly powered to identify the cell types most likely impacted by disease-associated genetic variation compared to 1D genomic features, and tend to implicate different effector genes than eQTL approaches in the same cell types. Most of the variants implicated by these cis-regulatory architectures are highly trait-specific, but nearly half of the target genes connected to these variants are shared across multiple autoimmune disorders in multiple cell types, suggesting a high level of genetic diversity and complexity among autoimmune diseases that nonetheless converge at the level of target gene and cell type. Substantial effector gene sharing led to the common enrichment of similar biological networks across disease and cell types. However, trait-specific pathways representing potential areas for disease-specific intervention were identified. To test this, we pharmacologically validated squalene synthase, a cholesterol biosynthetic enzyme encoded by the FDFT1 gene implicated by our approach in MS and SLE, as a novel immunomodulatory drug target controlling inflammatory cytokine production by human T cells. These data represent a comprehensive resource for basic discovery of gene cis-regulatory mechanisms, and the analyses reported reveal mechanisms by which autoimmune-associated variants act to regulate gene expression, function, and pathology across multiple, distinct tissues and cell types.

NCNet: Deformable medical image registration network based on neighborhood cross-attention combined with multi-resolution constraints

Objective. Existing registration networks based on cross-attention design usually divide the image pairs to be registered into patches for input. The division and merging operations of a series of patches are difficult to maintain the topology of the deformation field and reduce the interpretability of the network. Therefore, our goal is to develop a new network architecture based on a cross-attention mechanism combined with a multi-resolution strategy to improve the accuracy and interpretability of medical image registration. Approach. We propose a new deformable image registration network NCNet based on neighborhood cross-attention combined with multi-resolution strategy. The network structure mainly consists of a multi-resolution feature encoder, a multi-head neighborhood cross-attention module and a registration decoder. The hierarchical feature extraction capability of our encoder is improved by introducing large kernel parallel convolution blocks; the cross-attention module based on neighborhood calculation is used to reduce the impact on the topology of the deformation field and double normalization is used to reduce its computational complexity. Main result. We performed atlas-based registration and inter-subject registration tasks on the public 3D brain magnetic resonance imaging datasets LPBA40 and IXI respectively. Compared with the popular VoxelMorph method, our method improves the average DSC value by 7.9% and 3.6% on LPBA40 and IXI. Compared with the popular TransMorph method, our method improves the average DSC value by 4.9% and 1.3% on LPBA40 and IXI. Significance. We proved the advantages of the neighborhood attention calculation method compared to the window attention calculation method based on partitioning patches, and analyzed the impact of the pyramid feature encoder and double normalization on network performance. This has made a valuable contribution to promoting the further development of medical image registration methods.

A parametric and feature-based CAD dataset to support human-computer interaction for advanced 3D shape learning

3D shape learning is an important research topic in computer vision, in which the datasets play a critical role. However, most of the existing 3D datasets use voxels, point clouds, mesh, and B-rep, which are not parametric and feature-based. Thus they can not support the generation of real-world engineering computer-aided design (CAD) models with complicated shape features. Furthermore, they are based on 3D geometry results without human-computer interaction (HCI) history. This work is the first to provide a full parametric and feature-based CAD dataset with a selection mechanism to support HCI in 3D learning. First, unlike existing datasets, mainly composed of simple features (typical sketch and extrude), we devise complicated engineering features, such as fillet, chamfer, mirror, pocket, groove, and revolve. Second, different from the monotonous combination of features, we invent a select mechanism to mimic how human focuses on and selects a particular topological entity. The proposed mechanism establishes the relationships among complicated engineering features, which fully express the design intention and design knowledge of human CAD engineers. Therefore, it can process advanced 3D features for real-world engineering shapes. The experiments show that the proposed dataset outperforms existing CAD datasets in both reconstruction and generation tasks. In quantitative experiment, the proposed dataset demonstrates better prediction accuracy than other parametric datasets. Furthermore, CAD models generated from the proposed dataset comply with semantics of the human CAD engineers and can be edited and redesigned via mainstream industrial CAD software.

Correcting angular distortions in Bragg coherent X-ray diffraction imaging.

Bragg coherent X-ray diffraction imaging (BCDI) has emerged as a powerful technique for strain imaging and morphology reconstruction of nanometre-scale crystals. However, BCDI often suffers from angular distortions that appear during data acquisition, caused by radiation pressure, heating or imperfect scanning stages. This limits the applicability of BCDI, in particular for small crystals and high-flux X-ray beams. Here, we present a pre-processing algorithm that recovers the 3D datasets from the BCDI dataset measured under the impact of large angular distortions. We systematically investigate the performance of this method for different levels of distortion and find that the algorithm recovers the correct angles for distortions up to 16.4× (1640%) the angular step size dθ = 0.004°. We also show that the angles in a continuous scan can be recovered with high accuracy. As expected, the correction provides marked improvements in the subsequent phase retrieval.

Image scanning microscopy reconstruction by autocorrelation inversion

Abstract Confocal laser scanning microscopy (CLSM) stands out as one of the most widely used microscopy techniques thanks to its three-dimensional imaging capability and its sub-diffraction spatial resolution, achieved through the closure of a pinhole in front of a single-element detector. However, the pinhole also rejects useful photons, and beating the diffraction limit comes at the price of irremediably compromising the signal-to-noise ratio (SNR) of the data. Image scanning microscopy (ISM) emerged as the rational evolution of CLSM, exploiting a small array detector in place of the pinhole and the single-element detector. Each sensitive element is small enough to achieve sub-diffraction resolution through the confocal effect, but the size of the whole detector is large enough to guarantee excellent collection efficiency and SNR. However, the raw data produced by an ISM setup consists of a 4D dataset, which can be seen as a set of confocal-like images. Thus, fusing the dataset into a single super-resolved image requires a dedicated reconstruction algorithm. Conventional methods are multi-image deconvolution, which requires prior knowledge of the system point spread functions (PSFs), or adaptive pixel reassignment (APR), which is effective only on a limited range of experimental conditions. In this work, we describe and validate a novel concept for ISM image reconstruction based on autocorrelation inversion. We leverage unique properties of the autocorrelation to discard low-frequency components and maximize the resolution of the reconstructed image without any assumption on the image or any knowledge of the PSF. Our results push the quality of the ISM reconstruction beyond the level provided by APR and open new perspectives for multi-dimensional image processing.

Bidirectional Efficient Attention Parallel Network for Segmentation of 3D Medical Imaging

Currently, although semi-supervised image segmentation has achieved significant success in many aspects, further improvement in segmentation accuracy is necessary for practical applications. Additionally, there are fewer networks specifically designed for segmenting 3D images compared to those for 2D images, and their performance is notably inferior. To enhance the efficiency of network training, various attention mechanisms have been integrated into network models. However, these networks have not effectively extracted all the useful spatial or channel information. Particularly for 3D medical images, which contain rich spatial and channel information with tightly interconnected relationships between them, there remains a wealth of spatial and channel-specific information waiting to be explored and utilized. This paper proposes a bidirectional and efficient attention parallel network (BEAP-Net). Specifically, we introduce two modules: Supreme Channel Attention (SCA) and Parallel Spatial Attention (PSA). These modules aim to extract more spatial and channel-specific feature information and effectively utilize it. We combine the principles of consistency training and entropy regularization to enable mutual learning among sub-models. We evaluate the proposed BEAP-Net on two public 3D medical datasets, LA and Pancreas. The network outperforms the current state of the art in eight algorithms and is better suited for 3D medical images. It achieves the new best semi-supervised segmentation performance on the LA database. Ablation studies further validate the effectiveness of each component of the proposed model. Moreover, the SCA and PSA modules proposed can be seamlessly integrated into other 3D medical image segmentation networks to yield significant performance gains.

A deep learning-based convolutional spatiotemporal network proxy model for reservoir production prediction

Accurate production prediction is crucial in the field of reservoir management and production optimization. Traditional models often struggle with the complexities of nonlinear relationships and high-dimensional data, which hinders their ability to capture the variability of the production process efficiently and results in time-consuming calculations. To overcome these limitations, this paper introduces an innovative proxy modeling technique employing a convolutional spatiotemporal neural network. This method utilizes convolutional neural networks to extract spatial features from high-dimensional data, while the Transformer is used to model and predict complex temporal dynamics in production. To validate the effectiveness of the proposed proxy model, two case studies involving four injection and nine production wells within two-dimensional (2D) and three-dimensional (3D) non-homogeneous reservoirs were conducted, with the R2 coefficient serving as the primary evaluation metric. As the number of training iterations and data volume increase, the proxy model demonstrates rapid convergence. In tests conducted on the 2D and 3D datasets, the average R2 value exceeded 0.96 and 0.94. These results confirm the accuracy and stability of the proxy model. It also shows that the proxy model can accurately describe the geological and fluid seepage characteristics of the reservoir, which in turn can achieve a highly accurate match with the real data. In addition, the computational time is reduced by two orders of magnitude compared to traditional models. Compared with the long short-term memory method, the accuracy of the prediction results is increased by 30%, which greatly enhances efficiency and accuracy. To some extent, the presented proxy model can provide some guidance for the efficient history match of production data.

Spatial differences of dissolved organic matter composition and humification in an artificial lake.

The depth-dependent dynamics of dissolved organic matter (DOM) structure and humification in an artificial lake limits the understanding of lake eutrophication and carbon cycling. Using fluorescence regional integration (FRI) and parallel factor analysis (PARAFAC) models to analyze the 3D fluorescence spectroscopy dataset, we revealed the depth-dependent structure and vertical distribution of DOM in the estuarine and center regions of Lake Hongfeng. The percentage fluorescence response (Pi,n) showed humic acid is an important part of DOM in Lake Hongfeng. Fluorescence results show that the fulvic-like and protein-like materials in HF1-DOM located at the estuarine position showed greater variation in the middle stage, probably due to human influence and sediment suspension. Fluorescence index (PI+II+IV,n/PIII+V,n and FIC4/FIC3) can be used to indicate the degree of humification of DOM in artificial lakes. Results of each index show that the estuary is more affected by human activities, and the humification degree is significantly lower than that of the center of the lake. The evaluation index system of the humification degree of artificial lake established in this study can effectively predict the eutrophication state of the typical area of artificial lake and deeply understand the possible important influence of human activities on the carbon cycle of lake.

Audio2Gestures: Generating Diverse Gestures From Audio.

People may perform diverse gestures affected by various mental and physical factors when speaking the same sentences. This inherent one-to-many relationship makes co-speech gesture generation from audio particularly challenging. Conventional CNNs/RNNs assume one-to-one mapping, and thus tend to predict the average of all possible target motions, easily resulting in plain/boring motions during inference. So we propose to explicitly model the one-to-many audio-to-motion mapping by splitting the cross-modal latent code into shared code and motion-specific code. The shared code is expected to be responsible for the motion component that is more correlated to the audio while the motion-specific code is expected to capture diverse motion information that is more independent of the audio. However, splitting the latent code into two parts poses extra training difficulties. Several crucial training losses/strategies, including relaxed motion loss, bicycle constraint, and diversity loss, are designed to better train the VAE. Experiments on both 3D and 2D motion datasets verify that our method generates more realistic and diverse motions than previous state-of-the-art methods, quantitatively and qualitatively. Besides, our formulation is compatible with discrete cosine transformation (DCT) modeling and other popular backbones (i.e., RNN, Transformer). As for motion losses and quantitative motion evaluation, we find structured losses/metrics (e.g. STFT) that consider temporal and/or spatial context complement the most commonly used point-wise losses (e.g. PCK), resulting in better motion dynamics and more nuanced motion details. Finally, we demonstrate that our method can be readily used to generate motion sequences with user-specified motion clips on the timeline.