Medical Image Data Research Articles

Editorial: Artificial intelligence and multimodal medical imaging data fusion for improving cardiovascular disease care

Editorial: Artificial intelligence and multimodal medical imaging data fusion for improving cardiovascular disease care

Self-supervised learning for chest computed tomography: training strategies and effect on downstream applications.

Self-supervised pre-training can reduce the amount of labeled training data needed by pre-learning fundamental visual characteristics of the medical imaging data. We investigate several self-supervised training strategies for chest computed tomography exams and their effects on downstream applications. We benchmark five well-known self-supervision strategies (masked image region prediction, next slice prediction, rotation prediction, flip prediction, and denoising) on 15M chest computed tomography (CT) slices collected from four sites of the Mayo Clinic enterprise, United States. These models were evaluated for two downstream tasks on public datasets: pulmonary embolism (PE) detection (classification) and lung nodule segmentation. Image embeddings generated by these models were also evaluated for prediction of patient age, race, and gender to study inherent biases in models' understanding of chest CT exams. The use of pre-training weights especially masked region prediction-based weights, improved performance, and reduced computational effort needed for downstream tasks compared with task-specific state-of-the-art (SOTA) models. Performance improvement for PE detection was observed for training dataset sizes as large as with a maximum gain of 5% over SOTA. The segmentation model initialized with pre-training weights learned twice as fast as the randomly initialized model. While gender and age predictors built using self-supervised training weights showed no performance improvement over randomly initialized predictors, the race predictor experienced a 10% performance boost when using self-supervised training weights. We released self-supervised models and weights under an open-source academic license. These models can then be fine-tuned with limited task-specific annotated data for a variety of downstream imaging tasks, thus accelerating research in biomedical imaging informatics.

Real-time location of acupuncture points based on anatomical landmarks and pose estimation models

IntroductionPrecise identification of acupuncture points (acupoints) is essential for effective treatment, but manual location by untrained individuals can often lack accuracy and consistency. This study proposes two approaches that use artificial intelligence (AI) specifically computer vision to automatically and accurately identify acupoints on the face and hand in real-time, enhancing both precision and accessibility in acupuncture practices.MethodsThe first approach applies a real-time landmark detection system to locate 38 specific acupoints on the face and hand by translating anatomical landmarks from image data into acupoint coordinates. The second approach uses a convolutional neural network (CNN) specifically optimized for pose estimation to detect five key acupoints on the arm and hand (LI11, LI10, TE5, TE3, LI4), drawing on constrained medical imaging data for training. To validate these methods, we compared the predicted acupoint locations with those annotated by experts.ResultsBoth approaches demonstrated high accuracy, with mean localization errors of less than 5 mm when compared to expert annotations. The landmark detection system successfully mapped multiple acupoints across the face and hand even in complex imaging scenarios. The data-driven approach accurately detected five arm and hand acupoints with a mean Average Precision (mAP) of 0.99 at OKS 50%.DiscussionThese AI-driven methods establish a solid foundation for the automated localization of acupoints, enhancing both self-guided and professional acupuncture practices. By enabling precise, real-time localization of acupoints, these technologies could improve the accuracy of treatments, facilitate self-training, and increase the accessibility of acupuncture. Future developments could expand these models to include additional acupoints and incorporate them into intuitive applications for broader use.

Finite element modeling of concentrated impact loads on the masticatory muscles at the head

AbstractNumerical simulation has great potential to provide a more comprehensive understanding of human impact response to injury mechanisms. Finite Element (FE) models are used as a tool to study human injuries in greater detail, for example, the THUMS (Total Human Model Safety of TOYOTA) model, which is widely used as a reliable human model in different fields to predict human injuries such as fractures, internal organ damage, and brain tissue injuries. However, no available FE model can be used to simulate human‐robot collisions based on standards ISO/TS 15066 and the biomechanical characteristics of human soft tissues in vivo. The authors have developed a head model based on the structures (dimensions and anatomy) of the THUMS head model, specifically designed to simulate impact loads on the masticatory muscles. Based on medical imaging (MRI) data, the soft tissues at the location of the masticatory muscles in the THUMS head are transformed from monolayer to multilayer, that is, a composite geometry of skin‐fat‐muscle each with its own material model and parameters. The model was optimized and validated using the experimental data from the Fraunhofer IFF subjects study, which determined biomechanical thresholds for specific body locations in ISO/TS 15066 under dynamic collisions.

A Review on Kidney Stone Detection using ML and DL Techniques

Kidney stone detection is a critical healthcare challenge, as timely and accurate diagnosis can prevent complications. The motivation behind this review is the increasing prevalence of kidney stones and the need for more effective, non-invasive detection methods. Machine learning (ML) and deep learning (DL) techniques offer promising solutions, leveraging medical imaging data to enhance diagnostic accuracy. However, limitations such as high computational cost and reliance on large datasets hinder their full potential. The aim of this review is to analyze the latest advancements in kidney stone detection using ML and DL techniques. The objective is to compare existing methodologies, highlight their strengths and weaknesses, and suggest future research directions, particularly in integrating transfer learning and fine-tuning techniques to enhance performance.

Deep hashing and attention mechanism-based image retrieval of osteosarcoma scans for diagnosis of bone cancer

BackgroundDue to its intricate nature and substantial data size, microscopic image data of osteosarcoma often present a significant obstacle to the effectiveness of conventional image retrieval methods. Therefore, this study investigates a new approach for medical image retrieval using advanced deep hashing techniques and attention mechanisms to address these challenges more effectively. MethodThe proposed algorithm significantly improves osteosarcoma cell microscopic image retrieval efficiency and accuracy using deep hashing and attention mechanisms. Image preprocessing includes adaptive histogram equalization and dataset augmentation to enhance quality and diversity. Feature extraction employs the WRN-AM model to map high-dimensional features to a low-dimensional hash code space, improving retrieval efficiency. Finally, similarity matching via Hamming distance allows rapid and precise identification of similar images. ResultsThe study shows notable advancements: the WRN-AM model achieves 93.2% classification accuracy and 97.09% mAP using 64-bit hash codes. These findings underscore the technique’s effective performance in extracting and categorizing diverse microscopic cell data efficiently and reliably. ConclusionsThis innovative approach provides a robust solution for retrieving and classifying microscopic data of osteosarcoma cells and other cell types, speeding up clinical diagnosis and medical research. It facilitates quicker access and analysis of patient image data, enhancing diagnostic precision and treatment planning for healthcare professionals. Concurrently, it supports researchers in leveraging medical image data more efficiently, fostering progress and innovation in the medical field.

Comparative Analysis of Advanced Data Mining Methods for Enhancing Medical Diagnosis and Prognosis

Accurate and early diagnosis, coupled with precise prognosis, is critical for improving patient outcomes in various medical conditions. This paper focuses on leveraging advanced data mining techniques to address two key medical challenges: diagnosis and prognosis. Diagnosis involves differentiating between benign and malignant conditions, while prognosis aims to predict the likelihood of recurrence after treatment. Despite significant advances in medical imaging and clinical data collection, achieving high accuracy in both diagnosis and prognosis remains a challenge. This study provides a comprehensive review of state-of-the-art machine learning and data mining techniques used for medical diagnosis and prognosis, including Neural Networks, K-Nearest Neighbors (KNN), Naïve Bayes, Logistic Regression, Decision Trees, and Support Vector Machines (SVM). These methods are evaluated on their ability to process large, complex datasets and produce actionable insights for medical practitioners.We conducted a thorough comparative analysis based on key performance metrics such as accuracy, Area Under the Curve (AUC), precision, recall, and specificity. Our findings reveal that Neural Networks consistently outperform other techniques in terms of diagnostic accuracy and predictive capacity, demonstrating their robustness in handling high-dimensional and nonlinear medical data. This research underscores the potential of advanced machine learning algorithms in revolutionizing early diagnosis and effective prognosis, thus facilitating more personalized treatment plans and improved healthcare outcomes.

Advances in the diagnosis of prostate cancer based on image fusion

Image fusion currently plays an important role in the diagnosis of prostate cancer (PCa). Selecting and developing a good image fusion algorithm is the core task of achieving image fusion, which determines whether the fusion image obtained is of good quality and can meet the actual needs of clinical application. In recent years, it has become one of the research hotspots of medical image fusion. In order to make a comprehensive study on the methods of medical image fusion, this paper reviewed the relevant literature published at home and abroad in recent years. Image fusion technologies were classified, and image fusion algorithms were divided into traditional fusion algorithms and deep learning (DL) fusion algorithms. The principles and workflow of some algorithms were analyzed and compared, their advantages and disadvantages were summarized, and relevant medical image data sets were introduced. Finally, the future development trend of medical image fusion algorithm was prospected, and the development direction of medical image fusion technology for the diagnosis of prostate cancer and other major diseases was pointed out.

Source free domain adaptation for kidney and tumor image segmentation with wavelet style mining

Kidney cancer is a serious malignant disease, and early diagnosis along with precise segmentation are crucial for effective treatment. However, due to the scarcity of labelled medical image data, the development of the intelligent diagnosis of kidney cancer is restricted. To address this challenge, we propose a novel unsupervised domain adaptation (UDA) framework specifically designed for kidney and tumor CT image segmentation. Our framework consists of a generation phase and an adaptation phase. In the generation phase, we employ a wavelet-based style mining generator to create class-specific source-like images, facilitating domain alignment. In the adaptation phase, we introduce contrastive domain extraction and compact-aware domain consistency modules, enhancing feature-level and output-level adaptability through data augmentation techniques. Experimental results demonstrate that our method performs better in kidney and tumor segmentation tasks, exhibiting higher accuracy and generalization capability than state-of-the-art domain adaptation methods. This indicates that our approach has significant advantages in medical image segmentation for kidneys and tumors.

3D LVCN: A Lightweight Volumetric ConvNet

ABSTRACTIn recent years, with the significant increase in the volume of three‐dimensional medical image data, three‐dimensional medical models have emerged. However, existing methods often require a large number of model parameters to deal with complex medical datasets, leading to high model complexity and significant consumption of computational resources. In order to address these issues, this paper proposes a 3D Lightweight Volume Convolutional Neural Network (3D LVCN), aiming to achieve efficient and accurate volume segmentation. This network architecture combines the design principles of convolutional neural network modules and hierarchical transformers, using large convolutional kernels as the basic framework for feature extraction, while introducing 1 × 1 × 1 convolutional kernels for deep convolution. This improvement not only enhances the computational efficiency of the model but also improves its generalization ability. The pro‐posed model is tested on three challenging public datasets, namely spleen, liver, and lung, from the medical segmentation decathlon. Experimental results show that the proposed model performance has in‐creased from 0.8315 to 0.8673, with a reduction in parameters of approximately 5%. This indicates that compared to currently advanced model structures, our proposed model architecture exhibits significant advantages in segmentation performance.

Enhancing Brain Tumor Detection Through Custom Convolutional Neural Networks and Interpretability-Driven Analysis

Brain tumor detection is crucial for effective treatment planning and improved patient outcomes. However, existing methods often face challenges, such as limited interpretability and class imbalance in medical-imaging data. This study presents a novel, custom Convolutional Neural Network (CNN) architecture, specifically designed to address these issues by incorporating interpretability techniques and strategies to mitigate class imbalance. We trained and evaluated four CNN models (proposed CNN, ResNetV2, DenseNet201, and VGG16) using a brain tumor MRI dataset, with oversampling techniques and class weighting employed during training. Our proposed CNN achieved an accuracy of 94.51%, outperforming other models in regard to precision, recall, and F1-Score. Furthermore, interpretability was enhanced through gradient-based attribution methods and saliency maps, providing valuable insights into the model’s decision-making process and fostering collaboration between AI systems and clinicians. This approach contributes a highly accurate and interpretable framework for brain tumor detection, with the potential to significantly enhance diagnostic accuracy and personalized treatment planning in neuro-oncology.

Identification of middle cerebral artery stenosis in transcranial Doppler using a modified VGG-16.

The diagnosis of intracranial atherosclerotic stenosis (ICAS) is of great significance for the prevention of stroke. Deep learning (DL)-based artificial intelligence techniques may aid in the diagnosis. The study aimed to identify ICAS in the middle cerebral artery (MCA) based on a modified DL model. This retrospective study included two datasets. Dataset1 consisted of 3,068 transcranial Doppler (TCD) images of the MCA from 1,729 patients, which were assessed as normal or stenosis by three physicians with varying levels of experience, in conjunction with other medical imaging data. The data were used to improve and train the VGG16 models. Dataset2 consisted of TCD images of 90 people who underwent physical examination, which were used to verify the robustness of the model and compare the consistency between the model and human physicians. The accuracy, precision, specificity, sensitivity, and area under curve (AUC) of the best model VGG16 + Squeeze-and-Excitation (SE) + skip connection (SC) on dataset1 reached 85.67 ± 0.43(%),87.23 ± 1.17(%),87.73 ± 1.47(%),83.60 ± 1.60(%), and 0.857 ± 0.004, while those of dataset2 were 93.70 ± 2.80(%),62.65 ± 11.27(%),93.00 ± 3.11(%),100.00 ± 0.00(%), and 0.965 ± 0.016. The kappa coefficient showed that it reached the recognition level of senior doctors. The improved DL model has a good diagnostic effect for MCV stenosis in TCD images and is expected to help in ICAS screening.

EFFICIENCY AND ACCURACY OF CONVOLUTIONAL AND FOURIER TRANSFORM LAYERS IN NEURAL NETWORKS FOR MEDICAL IMAGE CLASSIFICATION

In an era where information flow is moving at a rapid pace, image data processing is becoming increasingly important as technology advances, including in healthcare. Convolutional Neural Network (CNN) has been a common approach in image classification, but the larger the volume of data and the complexity of the task, the more expensive the computational cost of CNN. With the rapid growth in the amount of image data, efficiency in data processing is becoming increasingly important. In this study, the performance of neural network models using the convolution layer and Fourier transform layer in medical image data classification was compared. The results show that models with a Fourier transform layer tend to provide higher accuracy and better Area Under Curve (AUC) compared to models using a convolution layer. In addition, the model with the Fourier transform layer also shows faster execution time per epoch, which indicates efficiency in data processing. However, the convolution layer has an advantage in terms of model size, although it is not significantly different from the Fourier transform layer. In conclusion, the Fourier transform layer has an advantage in the classification of medical image data.

Application of 3D-Slicer Software in the Treatment of Gliomas.

To explore the application of 3D-Slicer multimodal image fusion technology in the surgical treatment of gliomas. A retrospective analysis of clinical data from 48 cases of intracranial gliomas treated with 3D-Slicer multimodal image fusion technology from September 2021 to February 2023. Preoperative cranial CT, MRI, TOF-MRA, DTI, and MRV sequence examinations were completed. DICOM format medical imaging data were processed using the 3D-Slicer software. Multimodal fusion and 3-dimensional reconstruction of the tumor and surrounding structures were conducted to clarify the relationship between the tumor and surrounding structures. The surgery path was simulated through 3D virtual visualization images, enabling precise preoperative planning and assisting in surgical treatment. Out of 48 patients, complete tumor resection was achieved in 45 cases. One case encountered severe postoperative cerebral edema, no infections were reported, 1 patient had slightly reduced muscle strength after surgery compared with before, and no cases of severe neurological function damage or surgical death were recorded. 3D-Slicer multimodal image fusion technology can assist in detailed preoperative planning for the surgical treatment of intracranial glioma, reducing surgical complications, improving the rate of total tumor resection, and reducing postoperative complications.

Medical image segmentation method based on confidence learning and collaborative training

Confidence learning plays an important role in the training of low-quality annotated data of medical images. However, the current applications of confidence learning are based on the average teacher model, and the possibility of confidence learning on other networks has not been explored. To address this problem, a segmentation model based on confidence learning and co-training is proposed. The model uses two different networks to encourage the output results of the two networks to be consistent. Then, the output of one of the networks is compared with the original low-quality annotations using confidence learning, and the low-quality annotated data is corrected to provide an effective training reference. The proposed model is compared on three medical image datasets of different modalities. Experimental results show that the segmentation effect of the proposed model is better than that of the existing confidence learning model.

Unsupervised few shot learning architecture for diagnosis of periodontal disease in dental panoramic radiographs.

In the domain of medical imaging, the advent of deep learning has marked a significant progression, particularly in the nuanced area of periodontal disease diagnosis. This study specifically targets the prevalent issue of scarce labeled data in medical imaging. We introduce a novel unsupervised few-shot learning algorithm, meticulously crafted for classifying periodontal diseases using a limited collection of dental panoramic radiographs. Our method leverages UNet architecture for generating regions of interest (RoI) from radiographs, which are then processed through a Convolutional Variational Autoencoder (CVAE). This approach is pivotal in extracting critical latent features, subsequently clustered using an advanced algorithm. This clustering is key in our methodology, enabling the assignment of labels to images indicative of periodontal diseases, thus circumventing the challenges posed by limited datasets. Our validation process, involving a comparative analysis with traditional supervised learning and standard autoencoder-based clustering, demonstrates a marked improvement in both diagnostic accuracy and efficiency. For three real-world validation datasets, our UNet-CVAE architecture achieved up to average 14% higher accuracy compared to state-of-the-art supervised models including the vision transformer model when trained with 100 labeled images. This study not only highlights the capability of unsupervised learning in overcoming data limitations but also sets a new benchmark for diagnostic methodologies in medical AI, potentially transforming practices in data-constrained scenarios.

Development of brain tumor radiogenomic classification using GAN-based augmentation of MRI slices in the newly released gazi brains dataset

Significant progress has been made recently with the contribution of technological advances in studies on brain cancer. Regarding this, identifying and correctly classifying tumors is a crucial task in the field of medical imaging. The disease-related tumor classification problem, on which deep learning technologies have also become a focus, is very important in the diagnosis and treatment of the disease. The use of deep learning models has shown promising results in recent years. However, the sparsity of ground truth data in medical imaging or inconsistent data sources poses a significant challenge for training these models. The utilization of StyleGANv2-ADA is proposed in this paper for augmenting brain MRI slices to enhance the performance of deep learning models. Specifically, augmentation is applied solely to the training data to prevent any potential leakage. The StyleGanv2-ADA model is trained with the Gazi Brains 2020, BRaTS 2021, and Br35h datasets using the researchers’ default settings. The effectiveness of the proposed method is demonstrated on datasets for brain tumor classification, resulting in a notable improvement in the overall accuracy of the model for brain tumor classification on all the Gazi Brains 2020, BraTS 2021, and Br35h datasets. Importantly, the utilization of StyleGANv2-ADA on the Gazi Brains 2020 Dataset represents a novel experiment in the literature. The results show that the augmentation with StyleGAN can help overcome the challenges of working with medical data and the sparsity of ground truth data. Data augmentation employing the StyleGANv2-ADA GAN model yielded the highest overall accuracy for brain tumor classification on the BraTS 2021 and Gazi Brains 2020 datasets, together with the BR35H dataset, achieving 75.18%, 99.36%, and 98.99% on the EfficientNetV2S models, respectively. This study emphasizes the potency of GANs for augmenting medical imaging datasets, particularly in brain tumor classification, showcasing a notable increase in overall accuracy through the integration of synthetic GAN data on the used datasets.

A chest imaging diagnosis report generation method based on dual-channel transmodal memory network

Medical imaging is critical for clinical diagnosis. In the past few years, medical imaging data has been steadily growing at an annual rate of 30%, while the annual growth rate in the number of radiologists is only 4. 1%. The traditional manual methods of chest imaging reports result in high information load and workload with low relevance of images and texts. Here we propose a chest CT imaging diagnosis report generation network to effectively integrate image and text feature information, improve the relevance of the cross-modal networks and automatically generate imaging diagnosis reports. The network includes three major modules: a novel ResNetII for information extraction, an image–text dual-channel transmodal memory network (DCTMN), and a dual-channel decoder module combined with LSTM gate. We validate the effectiveness of the proposed network on the IU X-ray and the MIMIC-CXR datasets. Our analysis indicates that the proposed method can strengthen the relationship between chest imaging and text information, ultimately achieving the automatic generation of chest imaging diagnosis reports.

DBCvT: Double Branch Convolutional Transformer for Medical Image Classification

Convolutional Neural Networks (CNNs) are extensively utilized in medical disease diagnosis, demonstrating the prominent performance in most cases. However, medical image processing based on deep learning faces some challenges. The limited availability and time-consuming annotations of medical image data restrict the scale and accuracy of model training. Data diversity and complexity further complicate these challenges. In order to address these issues, we introduce the Double Branch Convolutional Transformer (DBCvT), a hybrid CNN-Transformer feature extractor, which can better capture diverse fine-grained features and remain suitable for small datasets. In this model, separable downsampling convolution (SDConv) is used to mitigate excessive information loss during downsampling in standard convolutions. Additionally, we propose the Dual branch Channel Efficient multi-head Self-Attention (DCESA) mechanism to enhance the self-attention efficiency, consequently elevating network performance and effectiveness. Moreover, we introduce a novel convolutional channel-enhanced Attention mechanism to strengthen inter-channel relationships within feature maps post self-attention. The experiments of DBCvT on various medical image datasets have demonstrated the outstanding classification performance and generalization capability of the proposed model.

MIDC: Medical image dataset cleaning framework based on deep learning

Deep learning technology is widely used in the field of medical imaging. Among them, Convolutional Neural Networks (CNNs) are the most widely used, and the quality of the dataset is crucial for the training of CNN diagnostic models, as mislabeled data can easily affect the accuracy of the diagnostic models. However, due to medical specialization, it is difficult for non-professional physicians to judge mislabeled medical image data. In this paper, we proposed a new framework named medical image dataset cleaning (MIDC), whose main contribution is to improve the quality of public datasets by automatically cleaning up mislabeled data. The main innovations of MIDC are: firstly, the framework innovatively utilizes multiple public datasets of the same disease, relying on different CNNs to automatically recognize images and remove mislabeled data to complete the data cleaning process. This process does not rely on annotations from professional physicians and does not require additional datasets with more reliable labels; Secondly, a novel grading rule is designed to divide the datasets into high-accuracy datasets and low-accuracy datasets, based on which the data cleaning process can be performed; Thirdly, a novel data cleaning module based on CNN is designed to identify and clean low-accuracy datasets by using high-accuracy datasets. In the experiments, the validity of the proposed framework was verified by using four kinds of datasets diabetic retinal, viral pneumonia, breast tumor, and skin cancer, with results showing an increase in the average diagnostic accuracy from 71.18 % to 85.13 %, 82.50 %to 93.79 %, 85.59 %to 93.45 %, and 84.55 %to 94.21 %, respectively. The proposed data cleaning framework MIDC could better help physicians diagnose diseases based on the dataset with mislabeled data.