3D Image Segmentation Research Articles

Automated 3D semantic segmentation of PCB X-ray CT images and netlist extraction

Printed Circuit Board (PCB) design reconstruction is essential for addressing part obsolescence, intellectual property recovery, compliance, quality assurance, and enhancing national capabilities. Traditional methods for PCB design extraction, both non-geometry-based and geometry-based, have limitations in accuracy, efficiency, and scalability. This paper presents an automated approach, combining image processing and machine learning, to achieve 3D semantic segmentation of PCB X-ray Computed Tomography (X-ray CT) images and subsequent netlist extraction. By employing a 3D U-Net architecture with a ResNet-18 backbone and training on synthetic data, we introduce a first-of-its-kind method for direct 3D semantic segmentation, significantly improving over previous efforts. Our approach eliminates the need for extensive labeled datasets by using inherently labeled synthetic data. Further, this method enhances ease of segmentation by significantly reducing or eliminating the preprocessing effort required for 2D image stacks. It also improves universality by expanding the scope of application beyond images with specific 2D stack criteria, segmenting the 3D image in its entirety. Additionally, this method enables the processing of images of PCBs that have undergone bending, which is common among PCBs with a thickness below a certain threshold. The implications of this approach extend beyond PCBs, finding applications in various physical and biological sciences where 3D image segmentation is crucial. This methodology includes high-resolution 3D imaging, watershed segmentation, machine learning-based semantic segmentation, and netlist extraction. Validation with both synthetic and real-world PCB datasets shows high accuracy and robustness, offering a scalable solution for PCB design reconstruction.

Posttraining Network Compression for 3D Medical Image Segmentation: Reducing Computational Efforts via Tucker Decomposition.

"Just Accepted" papers have undergone full peer review and have been accepted for publication in Radiology: Artificial Intelligence. This article will undergo copyediting, layout, and proof review before it is published in its final version. Please note that during production of the final copyedited article, errors may be discovered which could affect the content. Purpose To investigate whether the computational effort of 3D CT-based multiorgan segmentation with TotalSegmentator can be reduced via Tucker decomposition-based network compression. Materials and Methods In this retrospective study, Tucker decomposition was applied to the convolutional kernels of the TotalSegmentator model, an nnU-Net model trained on a comprehensive CT dataset for automatic segmentation of 117 anatomic structures. The proposed approach reduced the floating-point operations (FLOPs) and memory required during inference, offering an adjustable trade-off between computational efficiency and segmentation quality. This study utilized the publicly available TotalSegmentator dataset containing 1228 segmented CTs and a test subset of 89 CTs, employing various downsampling factors to explore the relationship between model size, inference speed, and segmentation accuracy, evaluated using the Dice score. Results The application of Tucker decomposition to the TotalSegmentator model substantially reduced the model parameters and FLOPs across various compression ratios, with limited loss in segmentation accuracy. Up to 88% of the model's parameters were removed, with no evidence of differences in performance compared with the original model for 113 of 117 classes after fine-tuning. Practical benefits varied across different graphics processing unit architectures, with more distinct speed-ups on less powerful hardware. Conclusion The study demonstrates that posthoc network compression via Tucker decomposition presents a viable strategy for reducing the computational demand of medical image segmentation models without substantially impacting model accuracy. ©RSNA, 2025.

Semantic segmentation via fusing 2D image and 3D point cloud data with shared multi-layer perceptron

ABSTRACT Three-dimensional (3D) semantic segmentation based on point clouds has become a critical technology in the field of intelligent spatial perception and scene understanding. However, most existing methods cannot fully exploit the spatial feature of 3D point cloud data and the orderliness of two-dimensional (2D) image information, resulting in an inability to effectively improve segmentation accuracy. This study proposes a novel 2D image and 3D point cloud fusion method named shared multi-layer perceptron fusion semantic segmentation (SMFusionSeg) for semantic segmentation. The rich features of 2D image segmentation and spatial information of 3D point clouds are integrated by designing a fusion architecture that facilitates the transfer of 2D semantics to 3D semantics. Specifically, the fusion architecture concatenates the features from both 2D and 3D domains, which are then further integrated through a shared multi-layer perceptron (MLP) and optimized using an attention mechanism to enhance feature effectiveness and distinctiveness. Moreover, a knowledge distillation strategy is introduced to improve the learning capability and segmentation accuracy in parsing complex scenes. Experimental results on the SemanticKITTI dataset demonstrate that the proposed SMFusionSeg achieves a mean Intersection over Union ( mIoU ) of 65.4%, significantly outperforming traditional single-modal methods and many existing segmentation techniques, thereby effectively enhancing the precision and robustness of geographic feature recognition.

A spheroid whole mount drug testing pipeline with machine-learning based image analysis identifies cell-type specific differences in drug efficacy on a single-cell level

BackgroundThe growth and drug response of tumors are influenced by their stromal composition, both in vivo and 3D-cell culture models. Cell-type inherent features as well as mutual relationships between the different cell types in a tumor might affect drug susceptibility of the tumor as a whole and/or of its cell populations. However, a lack of single-cell procedures with sufficient detail has hampered the automated observation of cell-type-specific effects in three-dimensional stroma-tumor cell co-cultures.MethodsHere, we developed a high-content pipeline ranging from the setup of novel tumor-fibroblast spheroid co-cultures over optical tissue clearing, whole mount staining, and 3D confocal microscopy to optimized 3D-image segmentation and a 3D-deep-learning model to automate the analysis of a range of cell-type-specific processes, such as cell proliferation, apoptosis, necrosis, drug susceptibility, nuclear morphology, and cell density.ResultsThis demonstrated that co-cultures of KP-4 tumor cells with CCD-1137Sk fibroblasts exhibited a growth advantage compared to tumor cell mono-cultures, resulting in higher cell counts following cytostatic treatments with paclitaxel and doxorubicin. However, cell-type-specific single-cell analysis revealed that this apparent benefit of co-cultures was due to a higher resilience of fibroblasts against the drugs and did not indicate a higher drug resistance of the KP-4 cancer cells during co-culture. Conversely, cancer cells were partially even more susceptible in the presence of fibroblasts than in mono-cultures.ConclusionIn summary, this underlines that a novel cell-type-specific single-cell analysis method can reveal critical insights regarding the mechanism of action of drug substances in three-dimensional cell culture models.

Exploiting 2D Neural Network Frameworks for 3D Segmentation Through Depth Map Analytics of Harvested Wild Blueberries (Vaccinium angustifolium Ait.).

This study introduced a novel approach to 3D image segmentation utilizing a neural network framework applied to 2D depth map imagery, with Z axis values visualized through color gradation. This research involved comprehensive data collection from mechanically harvested wild blueberries to populate 3D and red-green-blue (RGB) images of filled totes through time-of-flight and RGB cameras, respectively. Advanced neural network models from the YOLOv8 and Detectron2 frameworks were assessed for their segmentation capabilities. Notably, the YOLOv8 models, particularly YOLOv8n-seg, demonstrated superior processing efficiency, with an average time of 18.10 ms, significantly faster than the Detectron2 models, which exceeded 57 ms, while maintaining high performance with a mean intersection over union (IoU) of 0.944 and a Matthew's correlation coefficient (MCC) of 0.957. A qualitative comparison of segmentation masks indicated that the YOLO models produced smoother and more accurate object boundaries, whereas Detectron2 showed jagged edges and under-segmentation. Statistical analyses, including ANOVA and Tukey's HSD test (α = 0.05), confirmed the superior segmentation performance of models on depth maps over RGB images (p < 0.001). This study concludes by recommending the YOLOv8n-seg model for real-time 3D segmentation in precision agriculture, providing insights that can enhance volume estimation, yield prediction, and resource management practices.

Hybrid-ctunet: a double complementation approach for 3D medical image segmentation

Hybrid-ctunet: a double complementation approach for 3D medical image segmentation

Volumetric medical image segmentation via fully 3D adaptation of Segment Anything Model

The Segment Anything Model (SAM) exhibits exceptional generalization capabilities in diverse domains, owing to its interactive learning mechanism designed for precise image segmentation. However, applying SAM to out-of-distribution domains, especially in 3D medical image segmentation, poses challenges. Existing methods for adapting 2D segmentation models to 3D medical data treat 3D volumes as a mere stack of 2D slices. The essential inter-slice information, which is pivotal to faithful 3D medical image segmentation tasks, is unfortunately neglected. In this work, we present the 3D Medical SAM-Adapter (3DMedSAM), a pioneer cross-dimensional adaptation, leveraging SAM’s pre-trained knowledge while accommodating the unique characteristics of 3D medical data. Firstly, to bridge the dimensional gap from 2D to 3D, we design a novel module to replace SAM’s patch embedding, ensuring a seamless transition into 3D image processing and recognition. Besides, we incorporate a 3D Adapter while maintaining the majority of pre-training parameters frozen, enriching deep features with abundant 3D spatial information and achieving efficient fine-tuning. Given the diverse scales of anomalies present in medical images, we also devised a multi-scale 3D mask decoder to elevate the network’s proficiency in medical image segmentation. Through various experiments, we showcase the effectiveness of 3DMedSAM in achieving accurate and robust 3D segmentation on both single-target segmentation and multi-organ segmentation tasks, surpassing the limitations of current methods.

An attention mechanism-based lightweight UNet for musculoskeletal ultrasound image segmentation.

Accurate musculoseletal ultrasound (MSKUS) image segmentation is crucial for diagnosis and treatment planning. Compared with traditional segmentation methods, deploying deep learning segmentation methods that balance segmentation efficiency, accuracy, and model size on edge devices has greateradvantages. This paper aims to design a MSKUS image segmentation method that has fewer parameters, lower computation complexity and higher segmentationaccuracy. In this study, an attention mechanism-based lightweight UNet (AML-UNet) is designed to segment target muscle regions in MSKUS images. To suppress the transmission of redundant feature, Channel Reconstruction and Spatial Attention Module is designed in the encoding path. In addition, considering the inherent characteristic of MSKUS image, Multiscale Aggregation Module is developed to replace the skip connection architecture of U-Net. Deep supervision is also introduced to the decoding path to refine predicted masks gradually. Our method is evaluated on two MSKUS 2D-image segmentation datasets, including 3917 MSKUS and 1534 images respectively. In the experiments, a five-fold cross-validation method is adopted in ablation experiments and comparison experiments. In addition, Wilcoxon Signed-Rank Test and Bonferroni correction are employed to validate the significance level. 0.01 was used as the statistical significance level in ourpaper. AML-UNet yielded a mIoU of 84.17% and 90.14% on two datasets, representing a 3.38% ( ) and 3.48% ( ) over the Unext model. The number of parameters and FLOPs are only 0.21M and 0.96G, which are 1/34 and 1/29 of those in comparison withUNet. Our proposed model achieved superior results with fewer parameters while maintaining segmentation efficiency and accuracy compared to othermethods.

A lightweight multi-scale multi-angle dynamic interactive Transformer-CNN fusion model for 3D medical image segmentation

Combining Convolutional Neural Network(CNN) and Transformer has become one of the mainstream methods for three-dimensional (3D) medical image segmentation. However, the complexity and diversity of target forms in 3D medical images require models to capture complex feature information for segmentation, resulting in an excessive number of parameters which are not conducive to training and deployment. Therefore, we have developed a lightweight 3D multi-target semantic segmentation model. In order to enhance contextual texture connections and reinforce the expression of detailed feature information, we designed a multi-scale and multi-angle feature interaction module to enhance feature representation by interacting multi-scale features from different perspectives. To address the issue of attention collapse in Transformers, leading to the neglect of other detailed feature learning, we utilized local features as dynamic parameters to interact with global features, dynamically grouping and learning critical features from global features, thereby enhancing the model's ability to learn detailed features. While ensuring the segmentation capability of the model, we aimed to keep the model lightweight, resulting in a total of 9.63M parameters. Extensive experiments were conducted on public datasets ACDC and Brats2018, as well as a private dataset, Temporal Bone CT. The results indicate that our proposed model is more competitive compared to the latest techniques in 3D medical image segmentation.

Deep-DM: Deep-driven deformable model for 3D image segmentation using limited data.

Objective - Medical image segmentation is essential for several clinical tasks, including diagnosis, surgical and treatment planning, and image-guided interventions. Deep Learning (DL) methods have become the state-of-the-art for several image segmentation scenarios. However, a large and well-annotated dataset is required to effectively train a DL model, which is usually difficult to obtain in clinical practice, especially for 3D images. Methods - In this paper, we proposed Deep-DM, a learning-guided deformable model framework for 3D medical imaging segmentation using limited training data. In the proposed method, an energy function is learned by a Convolutional Neural Network (CNN) and integrated into an explicit deformable model to drive the evolution of an initial surface towards the object to segment. Specifically, the learning-based energy function is iteratively retrieved from localized anatomical representations of the image containing the image information around the evolving surface at each iteration. By focusing on localized regions of interest, this representation excludes irrelevant image information, facilitating the learning process. Results and conclusion - The performance of the proposed method is demonstrated for the tasks of left ventricle and fetal head segmentation in ultrasound, left atrium segmentation in Magnetic Resonance, and bladder segmentation in Computed Tomography, using different numbers of training volumes in each study. The results obtained showed the feasibility of the proposed method to segment different anatomical structures in different imaging modalities. Moreover, the results also showed that the proposed approach is less dependent on the size of the training dataset in comparison with state-of-the-art DL-based segmentation methods, outperforming them for all tasks when a low number of samples is available. Significance - Overall, by offering a more robust and less data-intensive approach to accurately segmenting anatomical structures, the proposed method has the potential to enhance clinical tasks that require image segmentation strategies.

PFormer: An efficient CNN-Transformer hybrid network with content-driven P-attention for 3D medical image segmentation

Medical imaging, particularly medical image segmentation, is pivotal in modern medicine. Transformer architectures have gained significant attention in the field of medical image segmentation. Both pure transformer architectures and hybrid architectures combining transformer with CNN have been extensively proposed, demonstrating impressive performance in medical image segmentation. However, whether embedding the transformer in the higher layers of CNN networks or enhancing attention mechanisms, these approaches have inherent limitations in fully harnessing the potential of the transformer within the constraints of low computational cost. This paper introduces an efficient hybrid network structure that combines CNN and transformer for 3D medical image segmentation. This network utilizes depth-wise separable convolutions (DWconvs) to extract local fine-grained details and introduces a content-driven attention mechanism, P-attention, to replace self-attention within the transformer. P-attention enables global feature modeling in both channel and spatial dimensions. By integrating local information with global dependencies, the network achieves more precise segmentation results with a reduced computational burden. Extensive experiments on Synapse and Automated Cardiac Diagnosis Challenge (ACDC) datasets, utilizing various evaluation metrics such as dice similarity coefficients (DSC), 95% percentile Hausdorff Distance (HD95), average symmetric surface distance (ASSD), precision (PRE) and sensitivity (SE), were conducted to assess the performance of PFormer. The experimental results demonstrate that, compared to other state-of-the-art methods, PFormer achieves superior segmentation performance under lower parameters and floating point of operations (FLOPs), providing strong evidence of the effectiveness of our method. Codes and models of PFormer are available at https://github.com/BitGyy/PFormer.

A novel method: YOLO-CE and 3D point cloud-based feature extraction for welding seams of tower bases

Abstract Robotic automated welding of non-standard steel structures presents significant challenges, particularly for electric power tower bases. This study introduces a novel approach that integrates the You Only Look Once—Compact Invert Block and Efficient Local Attention (YOLO-CE) model, an enhanced version of YOLOV8 for 2D image segmentation, with 3D point cloud technology. The YOLO-CE model is used to accurately extract point cloud data from the target area, which is then processed using the MSAC algorithm for efficient plane segmentation. Weld lines are identified through plane equations, allowing for initial weld point cloud extraction. To further refine accuracy, an optimized evaluation equation is developed that accounts for both the distance between the weld point cloud and the fitted plane, and the angle between their normal vectors. This enables precise classification of the weld point cloud. From this classification, key weld feature points are identified, and their exact positions are determined by calculating the distances between these points and their intersections with three planes. The reliability of the proposed method was validated using a robot for precise measurements, with a total error margin of less than 1.5084 mm, demonstrating high accuracy and stability. Post-operation inspections confirmed that the welds were filled and free from defects, meeting all process requirements. The YOLO-CE model achieved a mIoU of 96.38% and a precision of 99.8%, highlighting its effectiveness. This method provides an efficient and precise solution for the automated welding of non-standard steel structural components and has promising application potential.

TPAFNet: Transformer-Driven Pyramid Attention Fusion Network for 3D Medical Image Segmentation.

The field of 3D medical image segmentation is witnessing a growing trend in the utilization of combined networks that integrate convolutional neural networks and transformers. Nevertheless, prevailing hybrid networks are confronted with limitations in their straightforward serial or parallel combination methods and lack an effective mechanism to fuse channel and spatial feature attention. To address these limitations, we present a robust multi-scale 3D medical image segmentation network, the Transformer-Driven Pyramid Attention Fusion Network, which is denoted as TPAFNet, leveraging a hybrid structure of CNN and transformer. Within this framework, we exploit the characteristics of atrous convolution to extract multi-scale information effectively, thereby enhancing the encoding results of the transformer. Furthermore, we introduce the TPAF block in the encoder to seamlessly fuse channel and spatial feature attention from multi-scale feature inputs. In contrast to conventional skip connections that simply concatenate or add features, our decoder is enriched with a TPAF connection, elevating the integration of feature attention between low-level and high-level features. Additionally, we propose a low-level encoding shortcut from the original input to the decoder output, preserving more original image features and contributing to enhanced results. Finally, the deep supervision is implemented using a novel CNN-based voxel-wise classifier to facilitate better network convergence. Experimental results demonstrate that TPAFNet significantly outperforms other state-of-the-art networks on two public datasets, indicating that our research can effectively improve the accuracy of medical image segmentation, thereby assisting doctors in making more precise diagnoses.

Deep learning enables the fully automated quantification of high-resolution cardiac CT images: human in loop for model development

Abstract Introduction Cardiac computed tomography (CCT) images provide vast spatial and temporal information. Segmentation of the structure of interest is necessary to provide quantitative information. However, manual segmentation of cardiac substructures is highly time-consuming. Objective This study aims to develop a robust deep-learning segmentation model for cardiac substructure segmentation in 3D CCT images. Method We initially developed a 3D U-Net architecture, incorporating extensive augmentation, for the segmentation of the Left Ventricle Myocardium (LVM), the blood cavities of the Left and Right Ventricles (LV, RV), and the Left and Right Atria (LA, RA). This model utilized CT images from 20 patients who had available manual segmentation. Subsequently, we applied this initial model to datasets from two centers, including different generations of scanners. We then visually inspected the segmentation, selected the highest-quality segmentations, and modify it if necessary, and excluded the low-quality data. We continued fine-tuning the model with the new dataset until we reached a count of 456 patients. As there were multiple images from one patient during this model's development, we generated datasets to develop a new model using the entire new dataset to avoid any data leakage in the reported metrics. The model was developed using data from Center 1 (5-fold cross-validation) and then evaluated on an external test set from Center 2 (126 patients) and also 4D images from Center 1. Various metrics were calculated to assess the model's performance. Results In the internal test set, utilizing 5-fold CV, the model demonstrated high segmentation accuracy, achieving an overall Dice coefficient of 0.98, with individual scores of 0.96 for LVM, 0.98 for LV, 0.98 for RV, 0.99 for RA, and 0.98 for LA. We assessed the automatic quantification capability of the segmentation model from 256 patients, resulting in an R2 value greater than 0.92 for both LV end-diastolic and systolic volumes. In contrast, the model maintained robust performance on the external dataset, achieving an overall Dice coefficient of 0.96, with individual scores of 0.95 for LVM, 0.98 for LV, 0.97 for RV, 0.98 for RA, and 0.94 for LA. Conclusion We provided a large cohort of ground truth segmentations for different cardiac CT substructures using a human-in-the-loop strategy. Subsequently, we developed a deep learning model for highly accurate automated segmentation of these substructures. This model allows for fast and fully automatic segmentation and quantification of 3D and 4D cardiac CT images, from which various quantitative metrics can be extracted from different cardiac substructures.

Temporomandibular joint CBCT image segmentation via multi-view ensemble learning network.

Accurate segmentation of the temporomandibular joint (TMJ) from cone beam CT (CBCT) images holds significant clinical value for diagnosing temporomandibular joint osteoarthrosis (TMJOA) and related conditions. Convolutional neural network-based medical image segmentation methods have achieved state-of-the-art performance in various segmentation tasks. However, 3D medical images segmentation requires substantial global context and rich spatial semantic information, demanding much more GPU memory and computational resources. To address these challenges in 3D medical image segmentation, we propose a novel network- the MVEL-Net (Multi-view Ensemble Learning Network) for TMJ CBCT image segmentation. By resampling images along three dimensions, we generate multiple weak learners with different spatial semantic information. A subsequent strong learning network effectively integrates the outputs from these weak learners to achieve more accurate segmentation results. We evaluated our network model using a clinical dataset comprising 88 subjects with TMJ CBCT images. The average Dice similarity coefficient (DSC) was 0.9817 ± 0.0049, the average surface distance was 0.0540 ± 0.0179mm, and the 95% Hausdorff distance was 0.1743 ± 0.0550mm. Our proposed MVEL-Net demonstrates excellent segmentation performance on TMJ from CBCT images, while using fewer GPU memory resources compared to other 3D networks. The effectiveness of this method in capturing spatial context could be leveraged for tasks like organ segmentation from volumetric scans. This may facilitate wider adoption of AI-based solutions for automated analysis of 3D medical images.

Fast‐MedNeXt: Accelerating the MedNeXt Architecture to Improve Brain Tumour Segmentation Efficiency

ABSTRACTWith the rapid development of medical imaging technology, 3D image segmentation technology has gradually become a mainstream method, especially in brain tumour detection and diagnosis showing its unique advantages. The technique makes full use of 3D spatial information to locate and analyze tumours more accurately, thus playing an important role in improving diagnostic accuracy, optimising treatment planning and promoting research. However, it also suffers from significant computational expenditure and delayed processing pace. In this paper, we propose an innovative optimisation scheme to address this problem. We thoroughly investigate the MedNeXt network and propose Fast‐MedNeXt, which aims to increase the processing speed while maintaining accuracy. First, we introduce the partial convolution (PConv) technique, which replaces the deep convolutional layers in the network. This improvement effectively reduces computation and memory requirements while maintaining efficient feature extraction. Second, based on PConv, we propose PConv‐Down and PConv‐Up modules, which are applied to the up‐sampling and down‐sampling modules to further optimise the network structure and improve efficiency. To confirm the efficacy of the approach, we carried out a sequence of tests in the multimodal brain tumour segmentation challenge 2021 (BraTS2021). By comparing with the MedNeXt series network, the Fast‐MedNeXt reduced the latency by 22.1%, 20.5%, 15.8%, and 11.4% respectively, while the average accuracy also increased by 0.475% and 0.2% respectively. These significant performance improvements demonstrate the effectiveness of Fast‐MedNeXt in 3D medical image segmentation tasks and provide a new and more efficient solution for the field.

Translation Consistent Semi-supervised Segmentation for 3D Medical Images.

3D medical image segmentation methods have been successful, but their dependence on large amounts of voxel-level annotated data is a disadvantage that needs to be addressed given the high cost to obtain such annotation. Semi-supervised learning (SSL) solves this issue by training models with a large unlabelled and a small labelled dataset. The most successful SSL approaches are based on consistency learning that minimises the distance between model responses obtained from perturbed views of the unlabelled data. These perturbations usually keep the spatial input context between views fairly consistent, which may cause the model to learn segmentation patterns from the spatial input contexts instead of the foreground objects. In this paper, we introduce the Translation Consistent Co-training (TraCoCo) which is a consistency learning SSL method that perturbs the input data views by varying their spatial input context, allowing the model to learn segmentation patterns from foreground objects. Furthermore, we propose a new Confident Regional Cross entropy (CRC) loss, which improves training convergence and keeps the robustness to co-training pseudo-labelling mistakes. Our method yields state-of-the-art (SOTA) results for several 3D data benchmarks, such as the Left Atrium (LA), Pancreas-CT (Pancreas), and Brain Tumor Segmentation (BraTS19). Our method also attains best results on a 2D-slice benchmark, namely the Automated Cardiac Diagnosis Challenge (ACDC), further demonstrating its effectiveness. Our code, training logs and checkpoints are available at https://github.com/yyliu01/ TraCoCo.

HEDN: multi-oriented hierarchical extraction and dual-frequency decoupling network for 3D medical image segmentation.

Previous 3D encoder-decoder segmentation architectures struggled with fine-grained feature decomposition, resulting in unclear feature hierarchies when fused across layers. Furthermore, the blurred nature of contour boundaries in medical imaging limits the focus on high-frequency contour features. To address these challenges, we propose a Multi-oriented Hierarchical Extraction and Dual-frequency Decoupling Network (HEDN), which consists of three modules: Encoder-Decoder Module (E-DM), Multi-oriented Hierarchical Extraction Module (Multi-HEM), and Dual-frequency Decoupling Module (Dual-DM). The E-DM performs the basic encoding and decoding tasks, while Multi-HEM decomposes and fuses spatial and slice-level features in 3D, enriching the feature hierarchy by weighting them through 3D fusion. Dual-DM separates high-frequency features from the reconstructed network using self-supervision. Finally, the self-supervised high-frequency features separated by Dual-DM are inserted into the process following Multi-HEM, enhancing interactions and complementarities between contour features and hierarchical features, thereby mutually reinforcing both aspects. On the Synapse dataset, HEDN outperforms existing methods, boosting Dice Similarity Score (DSC) by 1.38% and decreasing 95% Hausdorff Distance (HD95) by 1.03 mm. Likewise, on the Automatic Cardiac Diagnosis Challenge (ACDC) dataset, HEDN achieves 0.5% performance gains across all categories.

Dual Channel‐Spatial Self‐Attention Transformer and CNN synergy network for 3D medical image segmentation

Even though the Vision Transformer leverages the self-attention mechanism to capture long-range dependencies, showing significant potential in medical image segmentation, the limited annotations in the image dataset make it difficult for the Transformer model to extract different global features, resulting in attention collapse and generating similar or identical attention maps. Previous studies have attempted to solve the problem by integrating convolutional neural layers into Transformer-based architectures. However, improper integration may lead to the inability of the model to effectively capture local and global information in both spatial and channel dimensions. To address the above issue, we propose a hybrid architecture using the Dual Channel-Spatial Self-Attention Transformer and CNN Synergy Network (DTC-SUNETR) for medical image segmentation. Specifically, we redesigned the self-attention mechanism. A novel Channel-Spatial Self-Attention (CSSA) block is introduced to integrate the enhanced channel and spatial self-attention mechanism to capture the global relationship and local structure among image features. This helps the model to more comprehensively understand the inter-dependencies between different channels and capture the relationships between different pixels, thus enhancing the feature representation of the corresponding dimensions. Simultaneously, it also improves the overall computational efficiency of the network. Extensive experiments on four different medical image segmentation datasets, including Synapse, ACDC, Brain Tumor, and Lung Tumor, demonstrate the superiority of the proposed DTC-SUNETR over state-of-the-art methods.

Quantitative cytoarchitectural phenotyping of deparaffinized human brain tissues.

Advanced 3D imaging techniques and image segmentation and classification methods can profoundly transform biomedical research by offering deep insights into the cytoarchitecture of the human brain in relation to pathological conditions. Here, we propose a comprehensive pipeline for performing 3D imaging and automated quantitative cellular phenotyping on Formalin-Fixed Paraffin-Embedded (FFPE) human brain specimens, a valuable yet underutilized resource. We exploited the versatility of our method by applying it to different human specimens from both adult and pediatric, normal and abnormal brain regions. Quantitative data on neuronal volume, ellipticity, local density, and spatial clustering level were obtained from a machine learning-based analysis of the 3D cytoarchitectural organization of cells identified by different molecular markers in two subjects with malformations of cortical development (MCD). This approach will grant access to a wide range of physiological and pathological paraffin-embedded clinical specimens, allowing for volumetric imaging and quantitative analysis of human brain samples at cellular resolution. Possible genotype-phenotype correlations can be unveiled, providing new insights into the pathogenesis of various brain diseases and enlarging treatment opportunities.