Self-attention Module Research Articles

Pancreatic cancer pathology image segmentation with channel and spatial long-range dependencies

Based on deep learning, pancreatic cancer pathology image segmentation technology effectively assists pathologists in achieving improved treatment outcomes. However, compared to traditional image segmentation tasks, the large size of tissues in pathology images requires a larger receptive field. While methods based on dilated convolutions or attention mechanisms can enhance the receptive field, they cannot capture long-range feature dependencies. Directly applying self-attention mechanisms to capture long-range dependencies results in intolerable computational complexity. To address these challenges, we introduce a channel and spatial self-attention (CS) Module designed for efficiently capturing both channel and spatial long-range feature dependencies in pancreatic cancer pathological images. Specifically, the channel and spatial self-attention module consists of an adaptive channel self-attention module and a window-shift spatial self-attention module. The adaptive channel self-attention module adaptively pools features to a fixed size to capture long-range feature dependencies. While the window-shift spatial self-attention module captures spatial long-range dependencies in a window-based manner. Additionally, we propose a re-weighted cross-entropy loss to mitigate the impact of long-tail distribution on performance. Our proposed method surpasses state-of-the-art on both our Pancreatic Cancer Pathology Image (PCPI) dataset and the GlaS challenge dataset. The mDice and mIoU have achieved 73.93% and 59.42% in our PCPI dataset.

Point Cloud Segmentation Network Based on Attention Mechanism and Dual Graph Convolution

To overcome the limitations of inadequate local feature representation and the underutilization of global information in dynamic graph convolutions, we propose a network that combines attention mechanisms with dual graph convolutions. Firstly, we construct a static graph based on the dynamic graph using the K-nearest neighbors algorithm and geometric distances of point clouds. This integration of dynamic and static graphs forms a dual graph structure, compensating for the underutilization of geometric positional relationships in the dynamic graph. Next, edge convolutions are applied to extract edge features from the dual graph structure. To further enhance the capturing ability of local features, we employ attention pooling, which combines max pooling and average pooling operations. Secondly, we introduce channel attention modules and spatial self-attention modules to improve the representation ability of global features and enhance semantic segmentation accuracy in our network. Experimental results on the S3DIS dataset demonstrate that compared to dynamic graph convolution alone, our proposed approach effectively utilizes both semantic and geometric relationships between point clouds using dual graph convolutions while addressing limitations related to insufficient local feature extraction. The introduction of attention mechanisms helps mitigate underutilization issues with global information, resulting in significant improvements in model performance.

JSMNET: IMPROVING INDOOR POINT CLOUD SEMANTIC AND INSTANCE SEGMENTATION THROUGH SELF-ATTENTION AND MULTISCALE FUSION

Abstract. The semantic understanding of indoor 3D point cloud data is crucial for a range of subsequent applications, including indoor service robots, navigation systems, and digital twin engineering. Global features are crucial for achieving high-quality semantic and instance segmentation of indoor point clouds, as they provide essential long-range context information. To this end, we propose JSMNet, which combines a multi-layer network with a global feature self-attention module to jointly segment three-dimensional point cloud semantics and instances. To better express the characteristics of indoor targets, we have designed a multi-resolution feature adaptive fusion module that takes into account the differences in point cloud density caused by varying scanner distances from the target. Additionally, we propose a framework for joint semantic and instance segmentation by integrating semantic and instance features to achieve superior results. We conduct experiments on S3DIS, which is a large three-dimensional indoor point cloud dataset. Our proposed method is compared against other methods, and the results show that it outperforms existing methods in semantic and instance segmentation and provides better results in target local area segmentation. Specifically, our proposed method outperforms PointNet (Qi et al., 2017a) by 16.0% and 26.3% in terms of semantic segmentation mIoU in S3DIS (Area 5) and instance segmentation mPre, respectively. Additionally, it surpasses ASIS (Wang et al., 2019) by 6.0% and 4.6%, respectively, as well as JSPNet (Chen et al., 2022) by a margin of 3.3% for semantic segmentation mIoU and a slight improvement of 0.3% for instance segmentation mPre.

Multi-scale attention fusion network for semantic segmentation of remote sensing images

ABSTRACT In the realm of high-resolution remote sensing image (HRSI) segmentation, convolutional neural networks have shown their effectiveness and superiority. However, there are still two problems in the segmentation model that generally adopts the encoder-decoder structure in the face of HRSI: 1) Fusing high-level feature maps and low-level feature maps directly in the decoder will make spatial detail features easy to mask; 2) Although self-attention has been used to capture the long-distance dependence of features, the consumption of computing power and memory makes it have many restrictions in practical applications. Aiming at these two problems, this paper proposes a new HRSI segmentation model (named MLWNet). First, the introduction of the maximum pooling module improves the quality of the feature map and obtains the receptive field of the whole map and rich global semantic information. Then, based on a new linear complexity self-attention mechanism, we design a multi-scale linear self-attention module to abstract the correlation between contexts. Finally, the weighted feature fusion helps the feature map restore spatial details and refine the segmentation results. On the two HRSI datasets of ISPRS Potsdam and ISPRS Vaihingen, MLWNet achieved mIOU segmentation accuracy of 78.19% and 71.61%, respectively, which not only outperforms other mainstream segmentation models but also has only 17.423 M parameters. The segmentation model in this study has high precision and small parameters, which can provide decision information for real-time use of remote sensing images.

A lightweight segmentation network for endoscopic surgical instruments based on edge refinement and efficient self-attention.

In robot-assisted surgical systems, surgical instrument segmentation is a critical task that provides important information for surgeons to make informed decisions and ensure surgical safety. However, current mainstream models often lack precise segmentation edges and suffer from an excess of parameters, rendering their deployment challenging. To address these issues, this article proposes a lightweight semantic segmentation model based on edge refinement and efficient self-attention. The proposed model utilizes a lightweight densely connected network for feature extraction, which is able to extract high-quality semantic information with fewer parameters. The decoder combines a feature pyramid module with an efficient criss-cross self-attention module. This fusion integrates multi-scale data, strengthens focus on surgical instrument details, and enhances edge segmentation accuracy. To train and evaluate the proposed model, the authors developed a private dataset of endoscopic surgical instruments. It containing 1,406 images for training, 469 images for validation and 469 images for testing. The proposed model performs well on this dataset with only 466 K parameters, achieving a mean Intersection over Union (mIoU) of 97.11%. In addition, the model was trained on public datasets Kvasir-instrument and Endovis2017. Excellent results of 93.24% and 95.83% were achieved on the indicator mIoU, respectively. The superiority and effectiveness of the method are proved. Experimental results show that the proposed model has lower parameters and higher accuracy than other state-of-the-art models. The proposed model thus lays the foundation for further research in the field of surgical instrument segmentation.

MSDSE: Predicting drug-side effects based on multi-scale features and deep multi-structure neural network

Unexpected side effects may accompany the research stage and post-marketing of drugs. These accidents lead to drug development failure and even endanger patients' health. Thus, it is essential to recognize the unknown drug-side effects. Most existing methods in silico find the answer from the association network or similarity network of drugs while ignoring the drug-intrinsic attributes. The limitation is that they can only handle drugs in the maturation stage.To be suitable for early drug-side effect screening, we conceive a multi-structural deep learning framework, MSDSE, which synthetically considers the multi-scale features derived from the drug. MSDSE can jointly learn SMILES sequence-based word embedding, substructure-based molecular fingerprint, and chemical structure-based graph embedding. In the preprocessing stage of MSDSE, we project all features to the abstract space with the same dimension. MSDSE builds a bi-level channel strategy, including a convolutional neural network module with an Inception structure and a multi-head Self-Attention module, to learn and integrate multi-modal features from local to global perspectives. Finally, MSDSE regards the prediction of drug-side effects as pair-wise learning and outputs the pair-wise probability of drug-side effects through the inner product operation. MSDSE is evaluated and analyzed on benchmark datasets and performs optimally compared to other baseline models. We also set up the ablation study to explain the rationality of the feature approach and model structure. Moreover, we select model partial prediction results for the case study to reveal actual capability. The original data are available at http://github.com/yuliyi/MSDSE.

CFM-YOLOv5:CFPNet moudle and muti-target prediction head incorporating YOLOv5 for metal surface defect detection

Aiming at the problem of low efficiency of manual detection in the field of metal surface defect detection, a deep learning defect detection method based on improved YOLOv5 algorithm is proposed. Firstly, in the feature enhancement part, we replace the multi-head self-attention module of the standard transformer encoder with the EVC module to improve the feature extraction ability. Second, in the prediction part, adding a small target detection head can reduce the negative impact of drastic object scale changes and improve the accuracy and stability of detection. Finally, the performance of the algorithm is verified by ablation experiments and analogy experiments. The experimental results show that the improved algorithm has greatly improved mAP and FPS on the data set, and can quickly and accurately identify the types of metal surface defects, which has reference significance for practical industrial applications.

Intelligent Fault Diagnosis of Marine Diesel Engines Based on Efficient Channel Attention-Improved Convolutional Neural Networks

With the rapid development of smart ships, the ship maintenance model is also changing. In order to extract the fault characteristics of diesel engine thermal parameters more easily, reduce the model’s complexity and improve the model’s accuracy, a new approach is proposed: first, the traditional convolutional neural networks (improved convolutional neural networks (ICNN)) are improved by using Meta-ACON as the activation function, improved AdamP as the optimizer, and label smoothing regularization (LSR) as the loss function, which enhances the stability of the model. Secondly, efficient channel attention (ECA) is added to achieve the mastery of global feature information, reduce the complexity of the traditional self-attention module, and enhance the model’s feature extraction ability. Lastly, the accuracy and reliability of the model are verified through ablation and comparison experiments. The accuracy rate reaches 97.6%, which is significantly improved by 32.1% compared with the original model, and the robustness of the model is verified through the introduction of noise. The experimental results demonstrate the applicability of the model in the field of diesel engine fault diagnosis.

Surface defect segmentation of magnetic tiles based on cross self-attention module

The detection of magnetic tile quality is an essential link before the assembly of permanent magnet motor. In order to meet the high standard of magnetic tile surface defect detection and realize the rapid and automatic segmentation of magnetic tile defects, a magnetic tile surface defect segmentation algorithm based on cross self-attention model (CSAM) is proposed. It adopts high-low level semantic feature fusion method to build the dependency relationship between the deep and shallow features. Multiple auxiliary loss functions are used to constrain the network and reduce the noise in the deep features. In addition, an image enhancement method is also designed to solve the problem of insufficient annotated data. The experimental results show that the network can achieve 79.6% mIoU and 98.5% PA, which can meet the high standard requirements of magnetic tile manufacturing.

Mutually Beneficial Transformer for Multimodal Data Fusion

Multimodal feature fusion representation, e.g., hyperspectral image and light detection and ranging (HSI-LiDAR) fusion, is an essential topic for fusion perception. However, existing networks tend to employ mandatory feature stacking or local context fusion strategies between multiple modalities, ignoring the power of globally mutual-guided feature transmission. Therefore, this paper develops a mutually beneficial transformer method for multimodal data fusion (MBFormer), which contains the following steps. First, a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">spatial constraint-based self-attention (SCS) module</i> . In this module, spectralwise attention and a spatialwise convolution are applied to HSI and LiDAR data individually, and then a spatial guide mask generated from LiDAR elevation information is used as an agent to bridge with HSI for spatial feature constraints. Second, a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">channel diversity-based transformer (CDT) module</i> . On the basis of local spectral embedding explorations, an adaptive token-mixer mechanism is conducted on the groupwise classification token of HSI and individual LiDAR data for global information connectivity and transitivity. At last, the selected features are embedded into a classification layer for the final result calculation. Experimental results show that the proposed MBFormer can obtain 97.76% and 98.62% classification accuracies on Houston and Trento datasets, respectively, indicating the advantages and competitiveness of the MBFormer over the compared state-of-the-art methods.

Self-Supervised Triplet Contrastive Learning for Classifying Endometrial Histopathological Images.

Early identification of endometrial cancer or precancerous lesions from histopathological images is crucial for precise endometrial medical care, which however is increasing hampered by the relative scarcity of pathologists. Computer-aided diagnosis (CAD) provides an automated alternative for confirming endometrial diseases with either feature-engineered machine learning or end-to-end deep learning (DL). In particular, advanced self-supervised learning alleviates the dependence of supervised learning on large-scale human-annotated data and can be used to pre-train DL models for specific classification tasks. Thereby, we develop a novel self-supervised triplet contrastive learning (SSTCL) model for classifying endometrial histopathological images. Specifically, this model consists of one online branch and two target branches. The second target branch includes a simple yet powerful augmentation module named random mosaic masking (RMM), which functions as an effective regularization by mapping the features of masked images close to those of intact ones. Moreover, we add a bottleneck Transformer (BoT) model into each branch as a self-attention module to learn the global information by considering both content information and relative distances between features at different locations. On public endometrial dataset, our model achieved four-class classification accuracies of 77.31 ± 0.84, 80.87 ± 0.48 and 83.22 ± 0.87% using 20, 50 and 100% labeled images, respectively. When transferred to the in-house dataset, our model obtained a three-class diagnostic accuracy of 96.81% with 95% confidence interval of 95.61-98.02%. On both datasets, our model outperformed state-of-the-art supervised and self-supervised methods. Our model may help pathologists to automatically diagnose endometrial diseases with high accuracy and efficiency using limited human-annotated histopathological images.

ZeroBind: a protein-specific zero-shot predictor with subgraph matching for drug-target interactions

Existing drug-target interaction (DTI) prediction methods generally fail to generalize well to novel (unseen) proteins and drugs. In this study, we propose a protein-specific meta-learning framework ZeroBind with subgraph matching for predicting protein-drug interactions from their structures. During the meta-training process, ZeroBind formulates training a protein-specific model, which is also considered a learning task, and each task uses graph neural networks (GNNs) to learn the protein graph embedding and the molecular graph embedding. Inspired by the fact that molecules bind to a binding pocket in proteins instead of the whole protein, ZeroBind introduces a weakly supervised subgraph information bottleneck (SIB) module to recognize the maximally informative and compressive subgraphs in protein graphs as potential binding pockets. In addition, ZeroBind trains the models of individual proteins as multiple tasks, whose importance is automatically learned with a task adaptive self-attention module to make final predictions. The results show that ZeroBind achieves superior performance on DTI prediction over existing methods, especially for those unseen proteins and drugs, and performs well after fine-tuning for those proteins or drugs with a few known binding partners.

Multi-layered self-attention mechanism for weakly supervised semantic segmentation

Weakly Supervised Semantic Segmentation (WSSS) provides efficient solutions for semantic image segmentation using image-level annotations. WSSS requires no pixel-level labeling that Fully Supervised Semantic Segmentation (FSSS) does, which is time-consuming and label-intensive. Most WSSS approaches have leveraged Class Activation Maps (CAM) or Self-Attention (SA) to generate pseudo pixel-level annotations to perform semantic segmentation tasks coupled with fully supervised approaches (e.g., Fully Convolutional Network). However, those approaches often provides incomplete supervision that mainly includes discriminative regions from the last convolutional layer. They may fail to capture regions of low- or intermediate-level features that may not be present in the last convolutional layer. To address the issue, we proposed a novel Multi-layered Self-Attention (Multi-SA) method that applies a self-attention module to multiple convolutional layers, and then stack feature maps from the self-attention layers to generate pseudo pixel-level annotations. We demonstrated that integrated feature maps from multiple self-attention layers produce higher coverage in semantic segmentation than using only the last convolutional layer through intensive experiments using standard benchmark datasets.

Pixel Self-Attention Guided Real-Time Instance Segmentation for Group Raised Pigs

Instance segmentation is crucial to modern agriculture and the management of pig farms. In practical farming environments, challenges arise due to the mutual adhesion, occlusion, and dynamic changes in body posture among pigs, making accurate segmentation of multiple target pigs complex. To address these challenges, we conducted experiments using video data captured from varying angles and non-fixed lenses. We selected 45 pigs aged between 20 and 105 days from eight pens as research subjects. Among these, 1917 images were meticulously labeled, with 959 images designated for the training set, 192 for validation, and 766 for testing. To enhance feature utilization and address limitations in the fusion process between bottom-up and top-down feature maps within the feature pyramid network (FPN) module of the YOLACT model, we propose a pixel self-attention (PSA) module, incorporating joint channel and spatial attention. The PSA module seamlessly integrates into multiple stages of the FPN feature extraction within the YOLACT model. We utilized ResNet50 and ResNet101 as backbone networks and compared performance metrics, including AP0.5, AP0.75, AP0.5-0.95, and AR0.5-0.95, between the YOLACT model with the PSA module and YOLACT models equipped with BAM, CBAM, and SCSE attention modules. Experimental results indicated that the PSA attention module outperforms BAM, CBAM, and SCSE, regardless of the selected backbone network. In particular, when employing ResNet101 as the backbone network, integrating the PSA module yields a 2.7% improvement over no attention, 2.3% over BAM, 2.4% over CBAM, and 2.1% over SCSE across the AP0.5-0.95 metric. We visualized prototype masks within YOLACT to elucidate the model’s mechanism. Furthermore, we visualized the PSA attention to confirm its ability to capture valuable pig-related information. Additionally, we validated the transfer performance of our model on a top-down view dataset, affirming the robustness of the YOLACT model with the PSA module.

MFA-UNet: a vessel segmentation method based on multi-scale feature fusion and attention module

IntroductionThe accurate segmentation of retinal vessels is of utmost importance in the diagnosis of retinal diseases. However, the complex vessel structure often leads to poor segmentation performance, particularly in the case of microvessels.MethodsTo address this issue, we propose a vessel segmentation method composed of preprocessing and a multi-scale feature attention network (MFA-UNet). The preprocessing stage involves the application of gamma correction and contrast-limited adaptive histogram equalization to enhance image intensity and vessel contrast. The MFA-UNet incorporates the Multi-scale Fusion Self-Attention Module(MSAM) that adjusts multi-scale features and establishes global dependencies, enabling the network to better preserve microvascular structures. Furthermore, the multi-branch decoding module based on deep supervision (MBDM) replaces the original output layer to achieve targeted segmentation of macrovessels and microvessels. Additionally, a parallel attention mechanism is embedded into the decoder to better exploit multi-scale features in skip paths.ResultsThe proposed MFA-UNet yields competitive performance, with dice scores of 82.79/83.51/84.17/78.60/81.75/84.04 and accuracies of 95.71/96.4/96.71/96.81/96.32/97.10 on the DRIVE, STARE, CHASEDB1, HRF, IOSTAR and FIVES datasets, respectively.DiscussionIt is expected to provide reliable segmentation results in clinical diagnosis.

Auditory stimulus reconstruction from ECoG with DNN and self-attention modules

Auditory stimulus reconstruction is a technique that predicts sound from neural activities. Decoding speech from human brains using neuronal networks allows us to understand auditory information processing and study hearing abilities effectively. However, current approaches produce low-quality reconstruction of speech which is problematic for applications of the reconstruction technology to practical uses such as evaluation of hearing characteristics of individuals or functional preservation during neurosurgical procedures. To tackle this issue, we used a self-attention (SA) module which are showing promising advances in deep neural network studies. We investigated the effect of including neural activity from a longer span on the reconstruction accuracy, and the learning behavior of self-attention modules. Our results showed that self-attention modules that focus on the presence or absence of auditory stimuli used relationships between features over a longer period (over 1000 ms) and produced better reconstructions than a convoluted neural network (CNN) model using features over a shorter period (300 ms) and a multilayer perceptron (MLP) model not using the features and reconstructed with attention to presence and absence of auditory stimuli. These findings imply that long and sparse relationships of time-series information on neural activities derived from SA modules improve the reconstruction performance.

A Novel Lightweight Object Detection Network with Attention Modules and Hierarchical Feature Pyramid

Object detection methods based on deep learning typically require devices with ample computing capabilities, which limits their deployment in restricted environments such as those with embedded devices. To address this challenge, we propose Mini-YOLOv4, a lightweight real-time object detection network that achieves an excellent trade-off between speed and accuracy. Based on CSPDarknet-Tiny as the backbone network, we enhance the detection performance of the network in three ways. We use a multibranch structure embedded in an attention module for simultaneous spatial and channel attention calibration. We design a group self-attention block with a symmetric structure consisting of a pair of complementary self-attention modules to mine contextual information, thereby ensuring that the detection accuracy is improved without increasing the computational cost. Finally, we introduce a hierarchical feature pyramid network to fully exploit multiscale feature maps and promote the extraction of fine-grained features. The experimental results demonstrate that Mini-YOLOv4 requires only 4.7 M parameters and has a billion floating point operations (BFLOPs) value of 3.1. Compared with YOLOv4-Tiny, our approach achieves a 3.2% improvement in mean accuracy precision (mAP) for the PASCAL VOC dataset and obtains a significant improvement of 3.5% in overall detection accuracy for the MS COCO dataset. In testing with an embedded platform, Mini-YOLOv4 achieves a real-time detection speed of 25.6 FPS on the NVIDIA Jetson Nano, thus meeting the demand for real-time detection in computationally limited devices.

Tek görüntü üretimi için öz-dikkat modüllü koşulsuz üretken bir model

Generative models have recently become a prominent research topic in the field of artificial intelligence. Among these models, Generative Adversarial Networks (GAN) have revolutionized the field of deep learning by enabling the production of high-quality synthetic data that is very similar to real-world data. However, the effectiveness of GANs largely depends on the size and quality of training data. In many real-world applications, collecting large amounts of high-quality training data is impractical, time-consuming, and expensive. Accordingly, in recent years, there has been intense interest in the development of GAN models that can work with limited data. These models are particularly useful in scenarios where available data is sparse, such as medical imaging, or in creative applications such as creating new works of art. In this study, we propose a GAN model that can learn from a single training image. Our model is based on the principle of multiple GANs operating sequentially at different scales. At each scale, the GAN learns the features of the training image in different dimensions and transfers them to the next GAN. Samples produced by the GAN at the finest scale are images that have the characteristics of the training image but have different realistic structures. In our model, we utilized a self-attention module to increase the realism and quality of the generated images. Additionally, we used a new scaling method to increase the success of the model. The quantitative and qualitative results we obtained from our experimental studies show that our model performs image generation successfully. In addition, we demonstrated the robustness of our model by testing its success in different image manipulation applications. As a result, our model can successfully produce realistic, high-quality, diverse images from a single training image, providing short training time, memory efficiency, and good training stability. Our model is flexible enough to be used in areas where limited data needs to be worked on.

Multi-level knowledge-driven feature representation and triplet loss optimization network for image–text retrieval

Image–text retrieval plays a considerable role in associating vision and language. Existing mainstream approaches focus on fine-grained alignment while ignoring the influence of prior knowledge on model performance and the limitation of using a fixed margin in the triplet loss. In this paper, we propose a Multi-level Knowledge-driven feature representation and Triplet Loss Optimization Network (MKTLON) that exploits prior knowledge to enhance visual and textual feature representations and utilizes an adaptive margin of the triplet loss to optimize network training. Specifically, we first present an Enhanced feature Representation scheme based on the Self-Attention (ERSA) module, which incorporates the prior knowledge randomly initialized by uniform distribution into the matrices K and V in the self-attention mechanism. Subsequently, we adopt cascaded ERSA modules to encode images and texts to obtain multi-level visual and textual features with prior knowledge. Furthermore, we develop an adaptive margin optimization strategy that models the relevance scores of positive and negative samples as two independent Gaussian distributions, and obtain the optimized margin by minimizing the intersection of these two distributions. Extensive experiments on two benchmarks, Flickr30K (155,000 image–text pairs) and MSCOCO (616,435 image–text pairs), show the proposed MKTLON achieves 5.7% and 4.3% improvements on rSum, respectively, compared to the state-of-the-art method. The source code will be released at https://github.com/FlyCuteBird/MKTLON.

Three-dimensional reconstruction of granular porous media based on deep generative models.

Reconstruction of microstructure in granular porous media, which can be viewed as granular assemblies, is crucial for studying their characteristics and physical properties in various fields concerned with the behavior of such media, including petroleum geology and computational materials science. In spite of the fact that many existing studies have investigated grain reconstruction, most of them treat grains as simplified individuals for discrete reconstruction, which cannot replicate the complex geometrical shapes and natural interactions between grains. In this work, a hybrid generative model based on a deep-learning algorithm is proposed for high-quality three-dimensional (3D) microstructure reconstruction of granular porous media from a single two-dimensional (2D) slice image. The method extracts 2D prior information from the given image and generates the grain set as a whole. Both a self-attention module and effective pattern loss are introduced in a bid to enhance the reconstruction ability of the model. Samples with grains of varied geometrical shapes are utilized for the validation of our method, and experimental results demonstrate that our proposed approach can accurately reproduce the complex morphology and spatial distribution of grains without any artificiality. Furthermore, once the model training is complete, rapid end-to-end generation of diverse 3D realizations from a single 2D image can be achieved.