Complex 3D Images Research Articles

3D image processing technology based on interactive entertainment application in cultural and creative product design

3D printing technology has many advantages that traditional production models do not have, greatly accelerating the speed of product design and development. In order to achieve the application of machine learning based 3D image processing technology in cultural and creative product design, this article collects a large amount of 3D image data as training samples and uses machine learning algorithms to establish a training model. Considering the characteristics of the data and the processing objectives, these 3D image data are used as a training set and trained through machine learning algorithms to learn the features and patterns of the image. After the training is completed, use this model to process the new 3D image. By inputting new image data, the model automatically extracts image features and predicts and processes them based on previously learned patterns. Evaluate the advantages and disadvantages of machine learning based methods in processing complex 3D images by comparing experimental results. The experimental results show that machine learning based 3D image processing technology has high accuracy and efficiency in the design of cultural and creative products. Compared with traditional methods, this method can better process complex 3D images and provide better design effects.

Finite element modelling of complex 3D image data with quantification and analysis

Abstract The purpose of this study was to examine how to model aggregated material microstructure and its meshing volumic generation that is provided by any data from 3D tomographic image data. The accurate reconstruction of 3D geometry structures from tomographic images is a powerful method in various application areas of materials science. The resulting mesh can be voxelized or conforming based on volumetric tetrahedral meshing. We investigate this creation depending on improving multiple materials marching cubes algorithm (M3C) with smoothing and remeshing algorithms. Then, a strategy for generating good-quality meshing and its robustness is presented, and this is performed with numerical tests. The novelty of this study is to generate a conforming mesh from complicated topology structures, particularly, when the interfaces of bi-materials are connected. This leads to a reduction in the node count in the generated mesh. The influence of some parameters involved in this algorithm is explored during different levels of meshing. In this work, the numerical homogenization approach from various spherical inclusions in the two-phase system using the algorithm M3C is considered to estimate the effective elastic properties. We created the framework with all the associated information, such as inputs in the format .inp files, to make it possible to run it over the Abaqus solver. Then, the Abaqus model based on the finite element method (FEM) was executed in this case for various material microstructures such as polycrystalline, composite, and fiber. We show the main workflow for providing desired results by visualizing the FEM analysis. We also demonstrate the capabilities of meshing methodology in the solver for these material models. The validation of the local mechanical environment from FEM with loading scenarios is achieved to predict displacements and deformations. Mechanical compression tests are performed to investigate the compressive behavior. Finally, stress-strain curves provided a comparison between simulations and experimental data for materials, and a good agreement is obtained.

2.5D transfer deep learning model for segmentation of contrast-enhancing lesions on brain magnetic resonance imaging of multiple sclerosis and neuromyelitis optica spectrum disorder.

Multiple sclerosis (MS) and neuromyelitis optica spectrum disorder (NMOSD) are the two mimic autoimmune diseases of the central nervous system, which are rare in East Asia. Quantitative detection of contrast-enhancing lesions (CELs) on contrast-enhancing T1-weighted magnetic resonance (MR) images is of great significance for assessing the disease activity of MS and NMOSD. However, it is challenging to develop automatic segmentation algorithms due to the lack of data. In this work, we present an automatic segmentation model of CELs based on Fully Convolutional with Attention DenseNet (FCA-DenseNet) and transfer learning strategy to address the challenge of CEL quantification in small-scale datasets. A transfer learning approach was employed in this study, whereby pretraining was conducted using 77 MS subjects from the open access datasets (MICCAI 2016, MICCAI 2017, ISBI 2015) for white matter hyperintensity segmentation, followed by fine-tuning using 24 MS and NMOSD subjects from the local dataset for CEL segmentation. The proposed FCA-DenseNet combined the Fully Convolutional DenseNet and Convolutional Block Attention Module in order to improve the learning capability. A 2.5D data slicing strategy was used to process complex 3D MR images. U-Net, ResUNet, TransUNet, and Attention-UNet are used as comparison models to FCA-DenseNet. Dice similarity coefficient (DSC), positive predictive value (PPV), true positive rate (TPR), and volume difference (VD) are used as evaluation metrics to evaluate the performances of different models. FCA-DenseNet outperforms all other models in terms of all evaluation metrics, with a DSC of 0.661±0.187, PPV of 0.719±0.201, TPR of 0.680±0.254, and VD of 0.388±0.334. Transfer learning strategy has achieved success in building segmentation models on a small-scale local dataset where traditional deep learning approaches fail to train effectively. The improved FCA-DenseNet, combined with transfer learning strategy and 2.5D data slicing strategy, has successfully addressed the challenges in constructing deep learning models on small-scale datasets, making it conducive to clinical quantification of brain CELs and diagnosis of MS and NMOSD.

MicroMagnify: A Multiplexed Expansion Microscopy Method for Pathogens and Infected Tissues.

Super-resolution optical imaging tools are crucial in microbiology to understand the complex structures and behavior of microorganisms such as bacteria, fungi, and viruses. However, the capabilities of these tools, particularly when it comes to imaging pathogens and infected tissues, remain limited. MicroMagnify (µMagnify) is developed, a nanoscale multiplexed imaging method for pathogens and infected tissues that are derived from an expansion microscopy technique with a universal biomolecular anchor. The combination of heat denaturation and enzyme cocktails essential is found for robust cell wall digestion and expansion of microbial cells and infected tissues without distortion. µMagnify efficiently retains biomolecules suitable for high-plex fluorescence imaging with nanoscale precision. It demonstrates up to eightfold expansion with µMagnify on a broad range of pathogen-containing specimens, including bacterial and fungal biofilms, infected culture cells, fungus-infected mouse tone, and formalin-fixed paraffin-embedded human cornea infected by various pathogens. Additionally, an associated virtual reality tool is developed to facilitate the visualization and navigation of complex 3D images generated by this method in an immersive environment allowing collaborative exploration among researchers worldwide. µMagnify is a valuable imaging platform for studying how microbes interact with their host systems and enables the development of new diagnosis strategies against infectious diseases.

An end-to-end pipeline based on open source deep learning tools for reliable analysis of complex 3D images of ovaries.

Computational analysis of bio-images by deep learning (DL) algorithms has made exceptional progress in recent years and has become much more accessible to non-specialists with the development of ready-to-use tools. The study of oogenesis mechanisms and female reproductive success has also recently benefited from the development of efficient protocols for three-dimensional (3D) imaging of ovaries. Such datasets have a great potential for generating new quantitative data but are, however, complex to analyze due to the lack of efficient workflows for 3D image analysis. Here, we have integrated two existing open-source DL tools, Noise2Void and Cellpose, into an analysis pipeline dedicated to 3D follicular content analysis, which is available on Fiji. Our pipeline was developed on larvae and adult medaka ovaries but was also successfully applied to different types of ovaries (trout, zebrafish and mouse). Image enhancement, Cellpose segmentation and post-processing of labels enabled automatic and accurate quantification of these 3D images, which exhibited irregular fluorescent staining, low autofluorescence signal or heterogeneous follicles sizes. In the future, this pipeline will be useful for extensive cellular phenotyping in fish or mammals for developmental or toxicology studies.

Airborne sparse flight array SAR 3D imaging based on compressed sensing in frequency domain

In airborne array synthetic aperture radar (SAR), the three-dimensional (3D) imaging performance and cross-track resolution depends on the length of the equivalent array. In this paper, Barker sequence criterion is used for sparse flight sampling of airborne array SAR, in order to obtain high cross-track resolution in as few times of flights as possible. Under each flight, the imaging algorithm of back projection (BP) and the data extraction method based on modified uniformly redundant arrays (MURAs) are utilized to obtain complex 3D image pairs. To solve the side-lobe noise in images, the interferometry between each image pair is implemented, and compressed sensing (CS) reconstruction is adopted in the frequency domain. Furthermore, to restore the geometrical relationship between each flight, the phase information corresponding to negative MURA is compensated on each single-pass image reconstructed by CS. Finally, by coherent accumulation of each complex image, the high resolution in cross-track direction is obtained. Simulations and experiments in X-band verify the availability.

A Framework for Providing Augmented Reality as a Service Provided by Cloud Computing for E-Learning

The main objective of the proposed study is to develop an e-learning system using augmented reality technology one of the main problems faces using AR in education is the huge computational power needed to transfer 2D animation to enrich learning facilities. Such problem increases when using smart mobile devices that suffer from hardware limitation. A promising framework is used to utilize cloud services to support augmented reality applications on the cloud. Such method significantly reduces consumption of memory and processing units when dealing with large size videos or images. Hence the augmented reality processing is speeded up to meet the requirements of E-learning systems.
 The proposed work was conducted on 100 students from different academic levels in the first semester of the year 2022. Three experiments were conducted for different fields of education including two-dimensional images using Unity Program (3D Software) to draw 3D objects and Vufoira software development kit. The experimental results showed promising results as the application has the flexibility to work on different platforms. Moreover the consumed memory to run the application is reduced significantly. The results also showed high performance for the application when drawing complex 3D images and when dealing with different animations.
 The study supported with a detailed questioner that proofs the importance of AR in the field of E-learning.

Detecting High-Rise Buildings from Sentinel-2 Data Based on Deep Learning Method

High-rise buildings (HRBs) as a modern and visually distinctive land use play an important role in urbanization. Large-scale monitoring of HRBs is valuable in urban planning and environmental protection and so on. Due to the complex 3D structure and seasonal dynamic image features of HRBs, it is still challenging to monitor large-scale HRBs in a routine way. This paper extends our previous work on the use of the Fully Convolutional Networks (FCN) model to extract HRBs from Sentinel-2 data by studying the influence of seasonal and spatial factors on the performance of the FCN model. 16 Sentinel-2 subset images covering four diverse regions in four seasons were selected for training and validation. Our results indicate the performance of the FCN-based method at the extraction of HRBs from Sentinel-2 data fluctuates among seasons and regions. The seasonal change of accuracy is larger than that of the regional change. If an optimal season can be chosen to get a yearly best result, F1 score of detected HRBs can reach above 0.75 for all regions with most errors located on the boundary of HRBs. FCN model can be trained on seasonally and regionally combined samples to achieve similar or even better overall accuracy than that of the model trained on an optimal combination of season and region. Uncertainties exist on the boundary of detected results and may be relieved by revising the definition of HRBs in a more rigorous way. On the whole, the FCN based method can be largely effective at the extraction of HRBs from Sentinel-2 data in regions with a large diversity in culture, latitude, and landscape. Our results support the possibility to build a powerful FCN model on a larger size of training samples for operational monitoring HRBs at the regional level or even on a country scale.

An Analysis of the Vulnerability of Two Common Deep Learning-Based Medical Image Segmentation Techniques to Model Inversion Attacks.

Recent research in computer vision has shown that original images used for training of deep learning models can be reconstructed using so-called inversion attacks. However, the feasibility of this attack type has not been investigated for complex 3D medical images. Thus, the aim of this study was to examine the vulnerability of deep learning techniques used in medical imaging to model inversion attacks and investigate multiple quantitative metrics to evaluate the quality of the reconstructed images. For the development and evaluation of model inversion attacks, the public LPBA40 database consisting of 40 brain MRI scans with corresponding segmentations of the gyri and deep grey matter brain structures were used to train two popular deep convolutional neural networks, namely a U-Net and SegNet, and corresponding inversion decoders. Matthews correlation coefficient, the structural similarity index measure (SSIM), and the magnitude of the deformation field resulting from non-linear registration of the original and reconstructed images were used to evaluate the reconstruction accuracy. A comparison of the similarity metrics revealed that the SSIM is best suited to evaluate the reconstruction accuray, followed closely by the magnitude of the deformation field. The quantitative evaluation of the reconstructed images revealed SSIM scores of and for the U-Net and the SegNet, respectively. The qualitative evaluation showed that training images can be reconstructed with some degradation due to blurring but can be correctly matched to the original images in the majority of the cases. In conclusion, the results of this study indicate that it is possible to reconstruct patient data used for training of convolutional neural networks and that the SSIM is a good metric to assess the reconstruction accuracy.

GPU-accelerated calculation of computer-generated holograms for line-drawn objects.

The heavy computational burden of computer-generated holograms (CGHs) has been a significant issue for three-dimensional (3D) display systems using electro-holography. Recently, fast CGH calculation methods of line-drawn objects for electro-holography were proposed, which are targeted for holography-based augmented reality/virtual reality devices because of their ability to project object contours in space with a small computational load. However, these methods still face shortcomings, namely, they cannot draw arbitrary curves with graphics processing unit (GPU) acceleration, which is an obstacle for replaying highly expressive and complex 3D images. In this paper, we propose an effective algorithm for calculating arbitrary line-drawn objects at layers of different depths suitable for implementation of GPU. By combining the integral calculation of wave propagation with an algebraic solution, we successfully calculated CGHs of 1, 920 × 1, 080 pixels within 1.1 ms on an NVIDIA Geforce RTX 2080Ti GPU.

Rapid 3D phenotypic analysis of neurons and organoids using data-driven cell segmentation-free machine learning.

Phenotypic profiling of large three-dimensional microscopy data sets has not been widely adopted due to the challenges posed by cell segmentation and feature selection. The computational demands of automated processing further limit analysis of hard-to-segment images such as of neurons and organoids. Here we describe a comprehensive shallow-learning framework for automated quantitative phenotyping of three-dimensional (3D) image data using unsupervised data-driven voxel-based feature learning, which enables computationally facile classification, clustering and advanced data visualization. We demonstrate the analysis potential on complex 3D images by investigating the phenotypic alterations of: neurons in response to apoptosis-inducing treatments and morphogenesis for oncogene-expressing human mammary gland acinar organoids. Our novel implementation of image analysis algorithms called Phindr3D allowed rapid implementation of data-driven voxel-based feature learning into 3D high content analysis (HCA) operations and constitutes a major practical advance as the computed assignments represent the biology while preserving the heterogeneity of the underlying data. Phindr3D is provided as Matlab code and as a stand-alone program (https://github.com/DWALab/Phindr3D).

Fast Computation of 3D Discrete Invariant Moments Based on 3D Cuboid for 3D Image Classification

The use of 3D discrete orthogonal invariant moments as descriptors of images constitutes one of the hot topics in the field of 3D image analysis, especially in the recognition and classification of deformed objects, due to their high numerical precision and low computing complexity. The TSR (translation, scale and rotation) invariant functions of discrete orthogonal moments can be obtained by expressing them as a linear combination of the corresponding invariant of geometric moments. This classical method is very expensive in time due to the long time allocated to compute the polynomial coefficients when complex or large 3D images are used. This main drawback eliminates the use of these moments in the recognition and classification fields of 3D objects. In this article, we propose a fast and efficient method for calculating 3D discrete orthogonal invariant moments of Hahn, Tchebichef, Krawtchouk, Meixner and Charlier of 3D image. This proposed method is based on the extraction of the 3D discrete orthogonal invariant moments from the 3D geometric invariant moments of the entire 3D image, using the 3D image cuboid representation (ICR) strategy, where each cuboid is treated independently. The proposed method has the advantage of accelerating the process of extracting the 3D discrete invariant moments compared with the classical method. The performance of the proposed method is evaluated in terms of the calculation time and classification accuracy using a set of 3D images. Instead of some examples of using the discrete orthogonal invariant moments of Hahn, Tchebichef, Krawtchouk, Meixner and Charlier as pattern features for pattern classification are also provided.

Dynamic image reconstruction and synthesis framework using deep learning algorithm

This research study shows an effective deformable complex 3D image reconstruction and image synthesis technique by consolidating needed high-level features from a deep convolutional neural network (CNN) system. By recognising the inherent deep features in image patches lead to information discovery in medicinal imaging. Utilising the ADNI and LONI imaging datasets, the performance of the proposed deep learning algorithm image reconstruction and synthesis performance was verified. For validation, various performance indices obtained with the proposed deep learning algorithm were compared with two conventional algorithms namely support vector machine and CNN. Likewise, to reveal the adaptability of the proposed image reconstruction and synthesis system, synthesis and reconstruction experiments were directed on the 7 T cerebrum magnetic resonance image. As presented in the study outcomes, the proposed method can accomplish predominant performance compared with other cutting-edge techniques with either low- or high-level features in terms of the synthesis and reconstruction rate. The proposed algorithm has a training time of 5 s with a structural similarity index of 0.97. In all investigations, the outcome shows that the proposed image reconstruction framework reliably exhibited progressively precise outcomes when contrasted with best in class. Hence, it can be used for possible precise image reconstruction and synthesis related applications.

High‐performance dynamic magnetic resonance image reconstruction and synthesis employing deep feature learning convolutional networks

AbstractThis research paper shows an effective deformable complex 3D image reconstruction and image synthesis technique by consolidating needed high‐level features from convolutional Neural Network (CNN) system. By recognize inherent deep feature representations in image patches for morphological changes in medicinal imaging information discovery. Various performance measurements, High Frequency Error Norm (HFEN), Mean Squared Error MSE, peak Signal‐to‐noise‐ratio (PSNR), Structural Similarity Index (SSI), are utilized to inspect different dataset. As presented in the paper outcomes, the proposed method can accomplish predominant performance compared with other cutting‐edge techniques with either low or high‐level features in terms of the synthesis and reconstruction rate. The proposed deep learning algorithm has a training time of 5 seconds with 0.97 SSI and 0.15 HFEN. In all investigations, the outcome shows that the proposed image synthesis and reconstruction framework reliably exhibited progressively precise outcomes when contrasted with best in class.

Breaking crosstalk limits to dynamic holography using orthogonality of high-dimensional random vectors.

Holography is the most promising route to true-to-life 3D projections, but the incorporation of complex images with full depth control remains elusive. Digitally synthesised holograms1–7, which do not require real objects to create a hologram, offer the possibility of dynamic projection of 3D video8,9. Extensive efforts aimed 3D holographic projection10–17, however available methods remain limited to creating images on a few planes10–12, over a narrow depth-of-field13,14 or with low resolution15–17. Truly 3D holography also requires full depth control and dynamic projection capabilities, which are hampered by high crosstalk9,18. The fundamental difficulty is in storing all the information necessary to depict a complex 3D image in the 2D form of a hologram without letting projections at different depths contaminate each other. Here, we solve this problem by preshaping the wavefronts to locally reduce Fresnel diffraction to Fourier holography, which allows inclusion of random phase for each depth without altering image projection at that particular depth, but eliminates crosstalk due to near-orthogonality of large-dimensional random vectors. We demonstrate Fresnel holograms that form on-axis with full depth control without any crosstalk, producing large-volume, high-density, dynamic 3D projections with 1000 image planes simultaneously, improving the state-of-the-art12,17 for number of simultaneously created planes by two orders of magnitude. While our proof-of-principle experiments use spatial light modulators, our solution is applicable to all types of holographic media.

MODELLING METHOD FOR EDUCATION PURPOSE: A FAST APPROACH TO ILLUSTRATION OF POINT DEFECTS IN SEMICONDUCTORS

Point defects in semiconductors cause many considerable behaviours of these materials. This article introduces a procedure for modelling of point defects using a structural approach often referred to as a bond-valence method. This method minimalizes the computation cost and facilitates a contruction of complex 3D images illustrating the point defects, including the charge distribution map at contact interfaces. This method also provides a convenient way to estimate the carrier density and defect locations, it visualizes a dynamics of valence relocation under thermal fluctuation

3D observations of the hydraulic fracturing process for a model non-cemented horizontal well under true triaxial conditions using an X-ray CT imaging technique

To investigate the spatial evolution of fractures in non-cemented horizontal wells, cubic cement specimens were fractured in triaxial experiments to reproduce hydraulic fracturing in field applications. Systematic experiments were conducted on specimens with different azimuth angles (0°, 30°, 45°, 60° and 90°), horizontal principal stress anisotropies (10 MPa and 4 MPa) and fracturing fluid viscosities (60 mPa·s and 5 mPa·s). All the tests were performed by injecting fluid at a constant rate of 9 cc/min. High-resolution non-destructive 3D X-ray microtomography was used to record the internal evolution of the induced fractures. The X-ray computed tomography (CT) imaging technique can produce complex 3D images of fracture systems with little artificial noise. The complex fracture geometries observed are a result of the rapidly changing stress conditions at the wellbore wall during injection and of the mechanical interaction among the fractures. The magnitudes of the well azimuth angle, principal stress anisotropy and injection fluid viscosity play significant roles in the evolution of the induced fractures. For a high horizontal differential stress of 10 MPa, the fracture morphologies are primarily determined by the in situ stresses, and the fractures extend in the direction of the maximum principal stress. However, when the horizontal differential stress is 4 MPa, the fracture geometry is influenced by the combined effects of the induced stress from the open-hole wellbore, fracture internal pressure distribution, far-field stresses and azimuth angle. In general, an induced fracture is planar at both a low (0°) and high (90°) azimuth angles, while twisted fractures form at intermediate azimuth angles (30°, 45° and 60°). In addition, the injection fluid with a low viscosity of 5 mPa·s easily infiltrates the formation, inducing a fracture network, in contrast to the smooth fracture plane induced by the injection fluid with a high viscosity of 60 mPa·s. Finally, we propose a new parameter, called the dimensionless net pressure, which can reflect the combined effects of the principal horizontal stress difference, well azimuth angle and injection fluid parameters on the evolution of the hydraulic fractures. The complexity of the fracture geometries is strongly and positively correlated with the dimensionless net pressure.

A 3 dimensional Digital Image Skeltonization using 3x3x3 Structuring Element

Image Skeletonization promises to be a powerful complexity-cutting tool for compact shape description, pattern recognition, robot vision, animation, petrography pore space fluid flow analysis, model/analysis of bone/lung/circulation, and image compression for telemedicine. The existing image thinning/skeletonization techniques using boundary erosion, distance coding, and Voronoi diagram are first overviewed to assess/compare their feasibility of extending from 2D to 3D. Previously, skeletons have been a common tool for identifying shape components in a solid object. However, obtaining skeletons of a grayscale volume poses new challenges due to the lack of a clear boundary between object and background. In this paper we propose a fast, efficient and robust algorithm to generate the skeleton of large, complex 3D images such as CT, MRI data which make use of 3X3X3 structuring elements for processing. This algorithm has been developed in the frame work of cellular logic array processing. Cellular logic array processing is a logico mathematical paradigm developed using the fundamental notions of normal algorithms and cellular automata. The algorithm provides a straightforward computation which is robust and not sensitive to noise or object boundary complexity. Because 3D skeleton may not be unique, several application-dependent skeletonization options will be explored for meeting specific quality/speed requirements.

Postembryonic Development of the Central Nervous System in the Wolf Spider. Arctosa kwangreungensis (Araneae: Lycosidae)

Microstructural characteristics of the central nervous system from postembryo to early instar spiderings in the wolf spider Arctosa kwangreungensis were studied using paraffine embedded tissue preparation and three dimensional image reconstruction techniques. The CNS of the postembryo was fully functional, despite its primitive microstructure. The CNS consists of a supraesophageal ganglion (brain) and a subesophageal mass, and they are joined with circumesophageal connectives. The brain of 1st instar spidering after a molt of the postembryo is also made up of a protocerebral and tritocerebral ganglion. Moreover, the subesophageal ganglionic mass is composed of a single pair of pedipalpal ganglia, four pairs of appendage ganglia, and a fused mass of abdominal neuromeres. For image processing and reconstruction, serially sectioned paraffin blocks were scanned with research complex microscopy and 3D images were reconstructed from the brain sections. The image stacks of the central complex and surrounding neuropils obtained from the brain sections were processed using Amira 5.3.2 software.

A MULTISCALE APPROACH TO THE REPRESENTATION OF 3D IMAGES, WITH APPLICATION TO POLYMER SOLAR CELLS

A multiscale approach to the description of geometrically complex 3D image data is proposed which distinguishes between morphological features on a ‘macro-scale’ and a ‘micro-scale’. Since our method is mainly tailored to nanostructures observed in composite materials consisting of two different phases, an appropriate binarization of grayscale images is required first. Then, a morphological smoothing is applied to extract the structural information from binarized image data on the ‘macro-scale’. A stochastic algorithm is developed for the morphologically smoothed images whose goal is to find a suitable representation of the macro-scale structure by unions of overlapping spheres. Such representations can be interpreted as marked point patterns. They lead to an enormous reduction of data and allow the application of well-known tools from point-process theory for their analysis and structural modeling. All those voxels which have been ‘misspecified’ by the morphological smoothing and subsequent representation by unions of overlapping spheres are interpreted as ‘micro-scale’ structure. The exemplary data sets considered in this paper are 3D grayscale images of photoactive layers in hybrid solar cells gained by electron tomography. These composite materials consist of two phases: a polymer phase and a zinc oxide phase. The macro-scale structure of the latter is represented by unions of overlapping spheres.