3D Medical Images Research Articles

Developing a phantom for simulating robotic-assisted complete mesocolic excision using 3D printing and medical imaging

BackgroundRobotic-assisted complete mesocolic excision is an advanced procedure mainly because of the great variability in anatomy. Phantoms can be used for simulation-based training and assessment of competency when learning new surgical procedures. However, no phantoms for robotic complete mesocolic excision have previously been described. This study aimed to develop an anatomically true-to-life phantom, which can be used for training with a robotic system situated in the clinical setting and can be used for the assessment of surgical competency.MethodsEstablished pathology and surgical assessment tools for complete mesocolic excision and specimens were used for the phantom development. Each assessment item was translated into an engineering development task and evaluated for relevance. Anatomical realism was obtained by extracting relevant organs from preoperative patient scans and 3D printing casting moulds for each organ. Each element of the phantom was evaluated by two experienced complete mesocolic excision surgeons without influencing each other’s answers and their feedback was used in an iterative process of prototype development and testing.ResultsIt was possible to integrate 35 out of 48 procedure-specific items from the surgical assessment tool and all elements from the pathological evaluation tool. By adding fluorophores to the mesocolic tissue, we developed an easy way to assess the integrity of the mesocolon using ultraviolet light. The phantom was built using silicone, is easy to store, and can be used in robotic systems designated for patient procedures as it does not contain animal-derived parts.ConclusionsThe newly developed phantom could be used for training and competency assessment for robotic-assisted complete mesocolic excision surgery in a simulated setting.

Chaos-Enhanced Archimede Algorithm for Global Optimization of Real-World Engineering Problems and Signal Feature Extraction

Optimization algorithms play a crucial role in a wide range of fields, from designing complex systems to solving mathematical and engineering problems. However, these algorithms frequently face major challenges, such as convergence to local optima, which limits their ability to find global, optimal solutions. To overcome these challenges, it has become imperative to explore more efficient approaches by incorporating chaotic maps within these original algorithms. Incorporating chaotic variables into the search process offers notable advantages, including the ability to avoid local minima, diversify the search, and accelerate convergence toward optimal solutions. In this study, we propose an improved Archimedean optimization algorithm called Chaotic_AO (CAO), based on the use of ten distinct chaotic maps to replace pseudorandom sequences in the three essential components of the classical Archimedean optimization algorithm: initialization, density and volume update, and position update. This improvement aims to achieve a more appropriate balance between the exploitation and exploration phases, offering a greater likelihood of discovering global solutions. CAO performance was extensively validated through the exploration of three distinct groups of problems. The first group, made up of twenty-three benchmark functions, served as an initial reference. Group 2 comprises three crucial engineering problems: the design of a welded beam, the modeling of a spring subjected to tension/compression stresses, and the planning of pressurized tanks. Finally, the third group of problems is dedicated to evaluating the efficiency of the CAO algorithm in the field of signal reconstruction, as well as 2D and 3D medical images. The results obtained from these in-depth tests revealed the efficiency and reliability of the CAO algorithm in terms of convergence speeds, and outstanding solution quality in most of the cases studied.

A multi-center distributed learning approach for Parkinson's disease classification using the traveling model paradigm.

Distributed learning is a promising alternative to central learning for machine learning (ML) model training, overcoming data-sharing problems in healthcare. Previous studies exploring federated learning (FL) or the traveling model (TM) setup for medical image-based disease classification often relied on large databases with a limited number of centers or simulated artificial centers, raising doubts about real-world applicability. This study develops and evaluates a convolution neural network (CNN) for Parkinson's disease classification using data acquired by 83 diverse real centers around the world, mostly contributing small training samples. Our approach specifically makes use of the TM setup, which has proven effective in scenarios with limited data availability but has never been used for image-based disease classification. Our findings reveal that TM is effective for training CNN models, even in complex real-world scenarios with variable data distributions. After sufficient training cycles, the TM-trained CNN matches or slightly surpasses the performance of the centrally trained counterpart (AUROC of 83% vs. 80%). Our study highlights, for the first time, the effectiveness of TM in 3D medical image classification, especially in scenarios with limited training samples and heterogeneous distributed data. These insights are relevant for situations where ML models are supposed to be trained using data from small or remote medical centers, and rare diseases with sparse cases. The simplicity of this approach enables a broad application to many deep learning tasks, enhancing its clinical utility across various contexts and medical facilities.

MS-TCNet: An effective Transformer–CNN combined network using multi-scale feature learning for 3D medical image segmentation

Medical image segmentation is a fundamental research problem in the field of medical image processing. Recently, the Transformer have achieved highly competitive performance in computer vision. Therefore, many methods combining Transformer with convolutional neural networks (CNNs) have emerged for segmenting medical images. However, these methods cannot effectively capture the multi-scale features in medical images, even though texture and contextual information embedded in the multi-scale features are extremely beneficial for segmentation. To alleviate this limitation, we propose a novel Transformer–CNN combined network using multi-scale feature learning for three-dimensional (3D) medical image segmentation, which is called MS-TCNet. The proposed model utilizes a shunted Transformer and CNN to construct an encoder and pyramid decoder, allowing six different scale levels of feature learning. It captures multi-scale features with refinement at each scale level. Additionally, we propose a novel lightweight multi-scale feature fusion (MSFF) module that can fully fuse the different-scale semantic features generated by the pyramid decoder for each segmentation class, resulting in a more accurate segmentation output. We conducted experiments on three widely used 3D medical image segmentation datasets. The experimental results indicated that our method outperformed state-of-the-art medical image segmentation methods, suggesting its effectiveness, robustness, and superiority. Meanwhile, our model has a smaller number of parameters and lower computational complexity than conventional 3D segmentation networks. The results confirmed that the model is capable of effective multi-scale feature learning and that the learned multi-scale features are useful for improving segmentation performance. We open-sourced our code, which can be found at https://github.com/AustinYuAo/MS-TCNet.

Validated respiratory drug deposition predictions from 2D and 3D medical images with statistical shape models and convolutional neural networks.

For the one billion sufferers of respiratory disease, managing their disease with inhalers crucially influences their quality of life. Generic treatment plans could be improved with the aid of computational models that account for patient-specific features such as breathing pattern, lung pathology and morphology. Therefore, we aim to develop and validate an automated computational framework for patient-specific deposition modelling. To that end, an image processing approach is proposed that could produce 3D patient respiratory geometries from 2D chest X-rays and 3D CT images. We evaluated the airway and lung morphology produced by our image processing framework, and assessed deposition compared to in vivo data. The 2D-to-3D image processing reproduces airway diameter to 9% median error compared to ground truth segmentations, but is sensitive to outliers of up to 33% due to lung outline noise. Predicted regional deposition gave 5% median error compared to in vivo measurements. The proposed framework is capable of providing patient-specific deposition measurements for varying treatments, to determine which treatment would best satisfy the needs imposed by each patient (such as disease and lung/airway morphology). Integration of patient-specific modelling into clinical practice as an additional decision-making tool could optimise treatment plans and lower the burden of respiratory diseases.

Towards the Emergence of the Medical Metaverse: A Pilot Study on Shared Virtual Reality for Orthognathic–Surgical Planning

Three-dimensional (3D) medical images are used for diagnosis and in surgical operation planning. Computer-assisted surgical simulations (CASS) are essential for complex surgical procedures that are often performed in an interdisciplinary manner. Traditionally, the participants study the designs on the same display. In 3D virtual reality (VR) environments, the planner is wearing a head-mounted display (HMD). The designs can be then examined in VR by other persons wearing HMDs, which is a practical use case for the medical metaverse. A multi-user VR environment was built for the planning of an orthognathic–surgical (correction of facial skeleton) operation. Four domain experts (oral and maxillofacial radiologists) experimented with the pilot system and found it useful. It enabled easier observation of the model and a better understanding of the structures. There was a voice connection and co-operation during the procedure was natural. The planning task is complex, leading to a certain level of complexity in the user interface.

A review of research and development of semi-supervised learning strategies for medical image processing

Accurate and robust segmentation of organs or lesions from medical images plays a vital role in many clinical applications such as diagnosis and treatment planning. With the massive increase in labeled data, deep learning has achieved great success in image segmentation. However, for medical images, the acquisition of labeled data is usually expensive because generating accurate annotations requires expertise and time, especially in 3D images. To reduce the cost of labeling, many approaches have been proposed in recent years to develop a high-performance medical image segmentation model to reduce the labeling data. For example, combining user interaction with deep neural networks to interactively perform image segmentation can reduce the labeling effort. Self-supervised learning methods utilize unlabeled data to train the model in a supervised manner, learn the basics and then perform knowledge transfer. Semi-supervised learning frameworks learn directly from a limited amount of labeled data and a large amount of unlabeled data to get high quality segmentation results. Weakly supervised learning approaches learn image segmentation from borders, graffiti, or image-level labels instead of using pixel-level labeling, which reduces the burden of labeling. However, the performance of weakly supervised learning and self-supervised learning is still limited on medical image segmentation tasks, especially on 3D medical images. In addition to this, a small amount of labeled data and a large amount of unlabeled data are more in line with actual clinical scenarios. Therefore, semi-supervised learning strategies become very important in the field of medical image processing.

Bilateral Context Modeling for Residual Coding in Lossless 3D Medical Image Compression.

Residual coding has gained prevalence in lossless compression, where a lossy layer is initially employed and the reconstruction errors (i.e., residues) are then losslessly compressed. The underlying principle of the residual coding revolves around the exploration of priors based on context modeling. Herein, we propose a residual coding framework for 3D medical images, involving the off-the-shelf video codec as the lossy layer and a Bilateral Context Modeling based Network (BCM-Net) as the residual layer. The BCM-Net is proposed to achieve efficient lossless compression of residues through exploring intra-slice and inter-slice bilateral contexts. In particular, a symmetry-based intra-slice context extraction (SICE) module is proposed to mine bilateral intra-slice correlations rooted in the inherent anatomical symmetry of 3D medical images. Moreover, a bi-directional inter-slice context extraction (BICE) module is designed to explore bilateral inter-slice correlations from bi-directional references, thereby yielding representative inter-slice context. Experiments on popular 3D medical image datasets demonstrate that the proposed method can outperform existing state-of-the-art methods owing to efficient redundancy reduction. Our code will be available on GitHub for future research.

CAISeg: A Clustering-Aided Interactive Network for Lesion Segmentation in 3D Medical Imaging.

Accurate lesion segmentation in medical imaging is critical for medical diagnosis and treatment. Lesions' diverse and heterogeneous characteristics often present a distinct long-tail distribution, posing difficulties for automatic methods. Currently, interactive segmentation approaches have shown promise in improving accuracy, but still struggle to deal with tail features. This triggers a demand of effective utilizing strategies of user interaction. To this end, we propose a novel point-based interactive segmentation model called Clustering-Aided Interactive Segmentation Network (CAISeg) in 3D medical imaging. A customized Interaction-Guided Module (IGM) adopts the concept of clustering to capture features that are semantically similar to interaction points. These clustered features are then mapped to the head regions of the prompted category to facilitate more precise classification. Meanwhile, we put forward a Focus Guided Loss function to grant the network an inductive bias towards user interaction through assigning higher weights to voxels closer to the prompted points, thereby improving the responsiveness efficiency to user guidance. Evaluation across brain tumor, colon cancer, lung cancer, and pancreas cancer segmentation tasks show CAISeg's superiority over the state-of-the-art methods. It outperforms the fully automated segmentation models in accuracy, and achieves results comparable to or better than those of the leading point-based interactive methods while requiring fewer prompt points. Furthermore, we discover that CAISeg possesses good interpretability at various stages, which endows CAISeg with potential clinical application value.

UniMiSS+: Universal Medical Self-Supervised Learning From Cross-Dimensional Unpaired Data.

Self-supervised learning (SSL) opens up huge opportunities for medical image analysis that is well known for its lack of annotations. However, aggregating massive (unlabeled) 3D medical images like computerized tomography (CT) remains challenging due to its high imaging cost and privacy restrictions. In our pilot study, we advocated bringing a wealth of 2D images like chest X-rays as compensation for the lack of 3D data, aiming to build a universal medical self-supervised representation learning framework, called UniMiSS. Especially, we designed a pyramid U- like medical Transformer (MiT) as the backbone to make UniMiSS possible to perform SSL with both 2D and 3D images. Consequently, the predecessor UniMiSS has two obvious merits compared to current 3D-specific SSL: (1) more effective - superior to learning strong representations, benefiting from more and diverse data; and (2) more versatile - suitable for various downstream tasks without the restriction on the dimensionality barrier. Unfortunately, UniMiSS did not dig deeply into the intrinsic anatomy correlation between 2D medical images and 3D volumes due to the lack of paired multi-modal/dimension patient data. In this extension paper, we propose the UniMiSS+, in which we introduce the digitally reconstructed radiographs (DRR) technology to simulate X-ray images from a CT volume to access paired CT and X-ray data. Benefiting from the paired group, we introduce an extra pair- wise constraint to boost the cross-modality correlation learning, which also can be adopted as a cross-dimension regularization to further improve the representations. We conduct expensive experiments on multiple 3D/2D medical image analysis tasks, including segmentation and classification. The results show that the proposed UniMiSS+ achieves promising performance on various downstream tasks, not only outperforming the ImageNet pre-training and other advanced SSL counterparts substantially but also improving the predecessor UniMiSS pre-training. Code is available at: https://github.com/YtongXie/UniMiSS-code.

One-Shot Weakly-Supervised Segmentation in 3D Medical Images.

Deep neural networks typically require accurate and a large number of annotations to achieve outstanding performance in medical image segmentation. One-shot and weakly-supervised learning are promising research directions that reduce labeling effort by learning a new class from only one annotated image and using coarse labels instead, respectively. In this work, we present an innovative framework for 3D medical image segmentation with one-shot and weakly-supervised settings. Firstly a propagation-reconstruction network is proposed to propagate scribbles from one annotated volume to unlabeled 3D images based on the assumption that anatomical patterns in different human bodies are similar. Then a multi-level similarity denoising module is designed to refine the scribbles based on embeddings from anatomical- to pixel-level. After expanding the scribbles to pseudo masks, we observe the miss-classified voxels mainly occur at the border region and propose to extract self-support prototypes for the specific refinement. Based on these weakly-supervised segmentation results, we further train a segmentation model for the new class with the noisy label training strategy. Experiments on three CT and one MRI datasets show the proposed method obtains significant improvement over the state-of-the-art methods and performs robustly even under severe class imbalance and low contrast. Code is publicly available at https://github.com/LWHYC/OneShot_WeaklySeg.

ContraSurv: Enhancing Prognostic Assessment of Medical Images via Data-Efficient Weakly Supervised Contrastive Learning.

Prognostic assessment remains a critical challenge in medical research, often limited by the lack of well-labeled data. In this work, we introduce ContraSurv, a weakly-supervised learning framework based on contrastive learning, designed to enhance prognostic predictions in 3D medical images. ContraSurv utilizes both the self-supervised information inherent in unlabeled data and the weakly-supervised cues present in censored data, refining its capacity to extract prognostic representations. For this purpose, we establish a Vision Transformer architecture optimized for our medical image datasets and introduce novel methodologies for both self-supervised and supervised contrastive learning for prognostic assessment. Additionally, we propose a specialized supervised contrastive loss function and introduce SurvMix, a novel data augmentation technique for survival analysis. Evaluations were conducted across three cancer types and two imaging modalities on three real-world datasets. The results confirmed the enhanced performance of ContraSurv over competing methods, particularly in data with a high censoring rate.

Sparse Coding Inspired LSTM and Self-Attention Integration for Medical Image Segmentation.

Accurate and automatic segmentation of medical images plays an essential role in clinical diagnosis and analysis. It has been established that integrating contextual relationships substantially enhances the representational ability of neural networks. Conventionally, Long Short-Term Memory (LSTM) and Self-Attention (SA) mechanisms have been recognized for their proficiency in capturing global dependencies within data. However, these mechanisms have typically been viewed as distinct modules without a direct linkage. This paper presents the integration of LSTM design with SA sparse coding as a key innovation. It uses linear combinations of LSTM states for SA's query, key, and value (QKV) matrices to leverage LSTM's capability for state compression and historical data retention. This approach aims to rectify the shortcomings of conventional sparse coding methods that overlook temporal information, thereby enhancing SA's ability to do sparse coding and capture global dependencies. Building upon this premise, we introduce two innovative modules that weave the SA matrix into the LSTM state design in distinct manners, enabling LSTM to more adeptly model global dependencies and meld seamlessly with SA without accruing extra computational demands. Both modules are separately embedded into the U-shaped convolutional neural network architecture for handling both 2D and 3D medical images. Experimental evaluations on downstream medical image segmentation tasks reveal that our proposed modules not only excel on four extensively utilized datasets across various baselines but also enhance prediction accuracy, even on baselines that have already incorporated contextual modules. Code is available at https://github.com/yeshunlong/SALSTM.

6D Hyperchaotic Encryption Model for Ensuring Security to 3D Printed Models and Medical Images

In the 6G era, where ultra-fast and reliable communication is expected to be ubiquitous, encryption shall continue to play a crucial role in ensuring the security and privacy of data. Encryption and decryption of medical images and 3D printed models using 6D hyperchaotic function is proposed in this research work for ensuring security in data transfer. Here we envisage using a six-dimensional hyperchaotic system for encryption purposes which shall offer a high level of security due to its complex and unpredictable dynamics with multiple positive Lyapunov exponents. This system can potentially enhance the encryption process for 3D objects and medical images, ensuring the protection of sensitive data and preventing unauthorized access. A hyperchaotic system is a type of dynamical system characterized by exhibiting more than one positive Lyapunov exponent, which indicates strong sensitivity to initial conditions. These systems have more degrees of freedom and complex and intricate dynamics compared to standard chaotic systems. The security of the encryption scheme depends on the complexity of the hyperchaotic system and the randomness of the secret key. The parameters of a 6D hyperchaotic system shall be used as an encryption key with six dimensions, each with its range of values, and shall provide many possible keys. In this work, we implemented a 6D hyperchaotic system for the encryption of the 3D printed model and medical images. The performance evaluation was done by metrics entropy, correlation, Number of Pixels Change Rate (NPCR), and Unified Averaged Changed Intensity (UACI) which revealed the robustness of the encryption model in ensuring security. Hyperchaotic systems can be efficiently implemented in parallel computing architectures, which allow faster encryption and decryption processes.

A Review of 3D Printing and Surgical Planning

Three-Dimensional Printing (3DP), also known as rapid prototyping or additive manufacturing, has revolutionized the field of medicine, particularly in preoperative planning. By transforming intricate Two-Dimensional (2D) medical imaging data into tangible, patient-specific Three-Dimensional (3D) models, 3DP enhances surgical precision and patient care. This technology facilitates detailed preoperative visualization, enabling surgeons to interact with accurate anatomical models, plan complex procedures, and anticipate potential challenges. The evolution of 3DP, driven by advances in Fused Deposition Modeling (FDM), Selective Laser Sintering (SLS), and Stereolithography (SLA), has democratized access to affordable and sophisticated 3D printers, spurring innovation in medical applications. In this study, a comprehensive review of the literature on the integration of 3DP technology into preoperative planning is conducted. Applications across various surgical disciplines, including cardiovascular, orthopedic, craniofacial, neurosurgical, oncological, transplant, and pediatric surgeries are examined. Benefits of 3DP in providing tactile feedback, creating patient-specific surgical guides and implants, and enhancing surgical simulation and training are evaluated. Additionally, challenges associated with the high costs, time-intensive processes, and material limitations of 3DP are assessed. Findings demonstrate that 3DP significantly reduces operative times, improves surgical outcomes, and lowers complication rates. The study highlights the transformative impact of 3DP on surgical practice and underscores its potential for future advancements. With ongoing research and innovation in materials science, bioprinting, artificial intelligence, and augmented reality, 3DP is poised to further revolutionize preoperative planning, offering more personalized, precise, and patient-centered healthcare solutions.

Reconstruction of a 3D medical image from pre-processed 2D DICOM slices: clinical application

Reconstruction of a 3D medical image from pre-processed 2D DICOM slices: clinical application

An Efficient Deep Neural Network to Classify Large 3D Images With Small Objects.

3D imaging enables accurate diagnosis by providing spatial information about organ anatomy. However, using 3D images to train AI models is computationally challenging because they consist of 10x or 100x more pixels than their 2D counterparts. To be trained with high-resolution 3D images, convolutional neural networks resort to downsampling them or projecting them to 2D. We propose an effective alternative, a neural network that enables efficient classification of full-resolution 3D medical images. Compared to off-the-shelf convolutional neural networks, our network, 3D Globally-Aware Multiple Instance Classifier (3D-GMIC), uses 77.98%-90.05% less GPU memory and 91.23%-96.02% less computation. While it is trained only with image-level labels, without segmentation labels, it explains its predictions by providing pixel-level saliency maps. On a dataset collected at NYU Langone Health, including 85,526 patients with full-field 2D mammography (FFDM), synthetic 2D mammography, and 3D mammography, 3D-GMIC achieves an AUC of 0.831 (95% CI: 0.769-0.887) in classifying breasts with malignant findings using 3D mammography. This is comparable to the performance of GMIC on FFDM (0.816, 95% CI: 0.737-0.878) and synthetic 2D (0.826, 95% CI: 0.754-0.884), which demonstrates that 3D-GMIC successfully classified large 3D images despite focusing computation on a smaller percentage of its input compared to GMIC. Therefore, 3D-GMIC identifies and utilizes extremely small regions of interest from 3D images consisting of hundreds of millions of pixels, dramatically reducing associated computational challenges. 3D-GMIC generalizes well to BCS-DBT, an external dataset from Duke University Hospital, achieving an AUC of 0.848 (95% CI: 0.798-0.896).

SASAN: Spectrum-Axial Spatial Approach Networks for Medical Image Segmentation.

Ophthalmic diseases such as central serous chorioretinopathy (CSC) significantly impair the vision of millions of people globally. Precise segmentation of choroid and macular edema is critical for diagnosing and treating these conditions. However, existing 3D medical image segmentation methods often fall short due to the heterogeneous nature and blurry features of these conditions, compounded by medical image clarity issues and noise interference arising from equipment and environmental limitations. To address these challenges, we propose the Spectrum Analysis Synergy Axial-Spatial Network (SASAN), an approach that innovatively integrates spectrum features using the Fast Fourier Transform (FFT). SASAN incorporates two key modules: the Frequency Integrated Neural Enhancer (FINE), which mitigates noise interference, and the Axial-Spatial Elementum Multiplier (ASEM), which enhances feature extraction. Additionally, we introduce the Self-Adaptive Multi-Aspect Loss ( LSM ), which balances image regions, distribution, and boundaries, adaptively updating weights during training. We compiled and meticulously annotated the Choroid and Macular Edema OCT Mega Dataset (CMED-18k), currently the world's largest dataset of its kind. Comparative analysis against 13 baselines shows our method surpasses these benchmarks, achieving the highest Dice scores and lowest HD95 in the CMED and OIMHS datasets. Our code is publicly available at https://github.com/IMOP-lab/SASAN-Pytorch.

A 3D Medical Image Segmentation Method Based on MCRAIP-Net

A 3D Medical Image Segmentation Method Based on MCRAIP-Net

Semi-Supervised 3D Medical Image Segmentation Using Multi-Consistency Learning With Fuzzy Perception-Guided Target Selection

Semi-Supervised 3D Medical Image Segmentation Using Multi-Consistency Learning With Fuzzy Perception-Guided Target Selection