Medical Image Analysis Research Articles

Clustering-Guided Twin Contrastive Learning for Endomicroscopy Image Classification.

Learning better representations is essential in medical image analysis for computer-aided diagnosis. However, learning discriminative semantic features is a major challenge due to the lack of large-scale well-annotated datasets. Thus, how can we learn a well-structured categorizable embedding space in limited-scale and unlabeled datasets? In this paper, we proposed a novel clustering-guided twin-contrastive learning framework (CTCL) that learns the discriminative representations of probe-based confocal laser endomicroscopy (pCLE) images for gastrointestinal (GI) tumor classification. Compared with traditional contrastive learning, in which only two randomly augmented views of the same instance are considered, the proposed CTCL aligns more semantically related and class-consistent samples by clustering, which improved intra-class tightness and inter-class variability to produce more informative representations. Furthermore, based on the inherent properties of CLE (geometric invariance and intrinsic noise), we proposed to regard CLE images with any angle rotation and CLE images with different noises as the same instance, respectively, for increased variability and diversity of samples. By optimizing CTCL in an end-to-end expectation-maximization framework, comprehensive experimental results demonstrated that CTCL-based visual representations achieved competitive performance on each downstream task as well as more robustness and transferability compared with existing state-of-the-art SSL and supervised methods. Notably, CTCL achieved 75.60%/78.45% and 64.12%/77.37% top-1 accuracy on the linear evaluation protocol and few-shot classification downstream tasks, respectively, which outperformed the previous best results by 1.27%/1.63% and 0.5%/3%, respectively. The proposed method holds great potential to assist pathologists in achieving an automated, fast, and high-precision diagnosis of GI tumors and accurately determining different stages of tumor development based on CLE images.

Many birds, one stone: Medical image segmentation with multiple partially labeled datasets

Medical image segmentation is fundamental in the field of medical image analysis and has wide clinical applications in disease diagnosis and surgical planning etc. Current prevalent solution is to train a deep network in a fully supervised way with a large-scale fully labeled dataset. However, due to the high labor cost and requirement on medical expertise, such dataset is always absent. Instead, there are multiple partially labeled datasets which are originally established for specific purposes. To make full use of these partially labeled datasets, we propose a novel partially supervised segmentation network, named PSSNet, which consists of a task-specific feature learning network followed by a cross-task attention module (xTA) to exploit task dependencies to enhance task-specific features. To solve the challenges raised by unlabeled classes and domain shift across datasets, we propose an adversarial self-training strategy. We conduct experiments on two medical image segmentation tasks. One is the fine-grained fundus image segmentation aiming to simultaneously segment four-class lesions, OD and OC, and vessels. Validation on seven datasets demonstrates that our PSSNet performs the best among three baselines and three state-of-the-arts. The other is the multiple abdominal organ segmentation in CT images. Our PSSNet is trained on three partially labeled datasets, i.e., LiTS, KiTS and Spleen. Validation on one fully labeled dataset, i.e., BTCV, demonstrates that our PSSNet achieves better performances than four state-of-the-arts. The code is publicly available at https://github.com/CVIU-CSU/PSSNet.

SSiT: Saliency-Guided Self-Supervised Image Transformer for Diabetic Retinopathy Grading.

Self-supervised Learning (SSL) has been widely applied to learn image representations through exploiting unlabeled images. However, it has not been fully explored in the medical image analysis field. In this work, Saliency-guided Self-Supervised image Transformer (SSiT) is proposed for Diabetic Retinopathy (DR) grading from fundus images. We novelly introduce saliency maps into SSL, with a goal of guiding self-supervised pre-training with domain-specific prior knowledge. Specifically, two saliency-guided learning tasks are employed in SSiT: 1) Saliency-guided contrastive learning is conducted based on the momentum contrast, wherein fundus images' saliency maps are utilized to remove trivial patches from the input sequences of the momentum-updated key encoder. Thus, the key encoder is constrained to provide target representations focusing on salient regions, guiding the query encoder to capture salient features. 2) The query encoder is trained to predict the saliency segmentation, encouraging the preservation of fine-grained information in the learned representations. To assess our proposed method, four publicly-accessible fundus image datasets are adopted. One dataset is employed for pre-training, while the three others are used to evaluate the pre-trained models' performance on downstream DR grading. The proposed SSiT significantly outperforms other representative state-of-the-art SSL methods on all downstream datasets and under various evaluation settings. For example, SSiT achieves a Kappa score of 81.88% on the DDR dataset under fine-tuning evaluation, outperforming all other ViT-based SSL methods by at least 9.48%.

Multi-strategy learning-based particle swarm optimization algorithm for COVID-19 threshold segmentation

With advancements in science and technology, the depth of human research on COVID-19 is increasing, making the investigation of medical images a focal point. Image segmentation, a crucial step preceding image processing, holds significance in the realm of medical image analysis. Traditional threshold image segmentation proves to be less efficient, posing challenges in selecting an appropriate threshold value. In response to these issues, this paper introduces Inner-based multi-strategy particle swarm optimization (IPSOsono) for conducting numerical experiments and enhancing threshold image segmentation in COVID-19 medical images. A novel dynamic oscillatory weight, derived from the PSO variant for single-objective numerical optimization (PSOsono) is incorporated. Simultaneously, the historical optimal positions of individuals in the particle swarm undergo random updates, diminishing the likelihood of algorithm stagnation and local optima. Moreover, an inner selection learning mechanism is proposed in the update of optimal positions, dynamically refining the global optimal solution. In the CEC 2013 benchmark test, PSOsono demonstrates a certain advantage in optimization capability compared to algorithms proposed in recent years, proving the effectiveness and feasibility of PSOsono. In the Minimum Cross Entropy threshold segmentation experiments for COVID-19, PSOsono exhibits a more prominent segmentation capability compared to other algorithms, showing good generalization across 6 CT images and further validating the practicality of the algorithm.

Lung Cancer Detection Using CNN VGG19 + Model

In the last many years, lung cancer has become a major public health concern. To examine cell breakdown in the lungs in its starting stages, doctors often use imaging modalities such as X-ray chest films, CT scans, MRIs, etc. The timing of diagnosis determines the course of therapy. Artificial intelligence (AI) is a hotspot for developing computational models of human intellect. In this research, we aim to enhance the detection and classification of lung nodules from CT images using a novel deep learning approach. Our study builds upon an extensive review of existing lung cancer detection methods, highlighting their strengths and weaknesses. This LIDC/IDRI dataset has been used in over half of the most recent research on lung cancer diagnosis. While several Convolutional Neural Networks (CNNs) architecture are adequate to process medical field data using technology solutions to discover and diagnose Lung cancer, here we proposed model VGG-19+ model. It is known for its strong feature extraction capabilities. It outflanks the ongoing model on a few exhibition measures, including exactness, accuracy, responsiveness/review, and f1-score.This research using VGG-19+ model could lead to more widespread utilization in cancer diagnosis by enhancing early lung cancer detection and developing the field of medical image analysis.

Development of a Secure Web-Based Medical Imaging Analysis Platform: The AWESOMME Project.

Precision medicine research benefits from machine learning in the creation of robust models adapted to the processing of patient data. This applies both to pathology identification in images, i.e., annotation or segmentation, and to computer-aided diagnostic for classification or prediction. It comes with the strong need to exploit and visualize large volumes of images and associated medical data. The work carried out in this paper follows on from a main case study piloted in a cancer center. It proposes an analysis pipeline for patients with osteosarcoma through segmentation, feature extraction and application of a deep learning model to predict response to treatment. The main aim of the AWESOMME project is to leverage this work and implement the pipeline on an easy-to-access, secure web platform. The proposed WEB application is based on a three-component architecture: a data server, a heavy computation and authentication server and a medical imaging web-framework with a user interface. These existing components have been enhanced to meet the needs of security and traceability for the continuous production of expert data. It innovates by covering all steps of medical imaging processing (visualization and segmentation, feature extraction and aided diagnostic) and enables the test and use of machine learning models. The infrastructure is operational, deployed in internal production and is currently being installed in the hospital environment. The extension of the case study and user feedback enabled us to fine-tune functionalities and proved that AWESOMME is a modular solution capable to analyze medical data and share research algorithms with in-house clinicians.

Design an Effective, Faster Region-Based Convolutional Neural Network with Optimization for the Human Tracking System

Nowadays, real-time Human Tracking System (HTS) is a crucial topic in computer vision and image processing with applications like robotic perception, scene understanding, video surveillance, image compression, medical image analysis, and augmented reality, among many others. In this paper, we design a Faster Region-based Convolutional Neural Network (FR-CNN) with Crow Search Optimization (FR-CNN-CSO) architecture to improve computational complexity and enhance the performance of HTS. The system is implemented in a Python environment with video input. Remove unnecessary data from the gathered datasets during preprocessing. Next, feature extraction is processed using Histograms of Oriented Gradients (HOG). Then update, the extracted features into a designed FR-CNN model for identifying and tracking a person using crow search fitness. The main goal of the developed approach is to attain accurate prediction results and improve the computational complexity by achieving less execution time. Finally, the experimental outcomes show the reliability of the designed system by other conventional techniques in terms of accuracy, precision, recall, F-measure, and execution time.

Ad-Tl: Alzheimer’s Disease Prediction Using Transfer Learning

Although many statistical methods and machine learning algorithms have been explored in both clinical and research settings to extract these patterns from neuroimaging data, differentiating between Alzheimer's disease and healthy brain data in older adults (age > 75) has proven challenging due to highly similar patterns of brain atrophy and image intensities. Medical image analysis is just one field that has benefited from the widespread use of deep learning technologies in recent years. This research paper proposed AD prediction using transfer learning (AD-TL) methods. The MRI dataset has been normalized using the Multi-Layer Perception model (MLP) with the CNN algorithm. In order to improve images, the CLAHE (Contrast-Limited Adaptive Histogram Equalization) technique has been used. Image segmentation has been done with Watershed Image segmentation. The training has been done with the Residual network (ResNet 50) with Alex net. Finally, the classification has been done with the Deep Convolutional neural network (DCNN) algorithm.According to the experimental data, the classification accuracy of the technique provided in this study may reach 99%.

Machine Learning-driven Lung Cancer Detection

Machine Learning-driven Lung Cancer Detection

A hybrid approach based on multipath Swin transformer and ConvMixer for white blood cells classification

White blood cells (WBC) play an effective role in the body’s defense against parasites, viruses, and bacteria in the human body. Also, WBCs are categorized based on their morphological structures into various subgroups. The number of these WBC types in the blood of non-diseased and diseased people is different. Thus, the study of WBC classification is quite significant for medical diagnosis. Due to the widespread use of deep learning in medical image analysis in recent years, it has also been used in WBC classification. Moreover, the ConvMixer and Swin transformer models, recently introduced, have garnered significant success by attaining efficient long contextual characteristics. Based on this, a new multipath hybrid network is proposed for WBC classification by using ConvMixer and Swin transformer. This proposed model is called Swin Transformer and ConvMixer based Multipath mixer (SC-MP-Mixer). In the SC-MP-Mixer model, firstly, features with strong spatial details are extracted with the ConvMixer. Then Swin transformer effectively handle these features with self-attention mechanism. In addition, the ConvMixer and Swin transformer blocks consist of a multipath structure to obtain better patch representations in the SC-MP-Mixer. To test the performance of the SC-MP-Mixer, experiments were performed on three WBC datasets with 4 (BCCD), 8 (PBC) and 5 (Raabin) classes. The experimental studies resulted in an accuracy of 99.65% for PBC, 98.68% for Raabin, and 95.66% for BCCD. When compared with the studies in the literature and the state-of-the-art models, it was seen that the SC-MP-Mixer had more effective classification results.

CGIHE-VDSR: Color global image histogram equalization with very deep super resolution networks for color image super resolution

Medical and satellite image analysis require incredibly high resolution. Super-resolution combines several low-resolution images of the same scene to generate a high-resolution image. The Super resolution employing deep learning techniques still has an illumination issue. This paper proposes a novel CGIHE-VDSR algorithm that integrates the Very Deep Super Resolution (VDSR) Network with Color Global Image Histogram Equalization (CGIHE) to improve image resolution. In the proposed method, the low-resolution image is first histogram equalized using the CGIHE algorithm. Then, the VDSR network is applied to the histogram equalized image for super-resolution. The comparison of real-time data with the benchmark images is done using the proposed algorithm in the MATLAB platform. The PSNR and SSIM metrics demonstrate that the super resolution image obtained using the proposed method is significantly better than the existing methods.

Transforming oral cancer care: The promise of deep learning in diagnosis

The diagnosis and treatment of oral cancer present significant challenges, including delayed diagnosis at advanced stages and limited access to healthcare. Deep learning (DL), a subset of artificial intelligence, holds promise for transforming medical image analysis and predictive analytics. In this perspective, we examine the applications of DL in oral cancer. Specifically, we explore the efficacy of convolutional neural networks (CNNs) and recurrent neural networks (RNNs) in diagnosing and predicting the prognosis of oral cancer in the last five years. Additionally, we underscore the importance of oral cancer databases in advancing research and clinical practice. However, DL methods face constraints related to input variability and model interpretability. Addressing these issues is crucial to harnessing the full potential of DL in oral cancer treatment. In summary, this article underscores the innovative contributions of DL in revolutionizing oral cancer management and advocating for precision medicine in oncology.

Comparative Analysis of Transformer Integration in U-net Networks for Enhanced Medical Image Segmentation

Transformer is popular in Natural Language Processing (NLP) and is a cornerstone of large models. Transformer has been used by researchers to address the limitations of Convolutional Neural Networks (CNNs) in medical picture segmentation models. Through an extensive literature review and case studies, this paper comparatively analyzes the performance of different models in this field, summarizes different methods of integrating transformers into U-net, and points out existing gaps and challenges. Research has found that the Transformer model can significantly improve the accuracy and efficiency of medical image analysis. The paper discusses the advantages, disadvantages, innovations, performance, and complexity of various models in detail, and shows how to enhance performance by integrating the Transformer structure into the U-net network. In particular, the paper also analyzes the advantages of Transformers that are most suitable for integration into the encoder part and highlights the balance that needs to be made between improving performance and computational cost. The conclusion shows that although there is no perfect model, optimal performance and efficiency can be achieved by selecting different combinations of Transformer and U-net according to the actual situation. It can be seen from the networks’ performance that the mixed use of a U-shaped convolutional network and Transformer module has good development prospects and high research significance.

Navigating the Depths: A Comprehensive study of Deep Learning and its Advancements in Medical Image Analysis

Medical imaging is the technique of capturing images of internal organs for diagnostic purposes, contributing to the identification and study of diseases. The primary goal of medical image analysis is to enhance the effectiveness of clinical research and treatment options. Deep learning has revolutionized this field, demonstrating remarkable outcomes in tasks such as registration, segmentation, feature extraction, and classification. The increased availability of computational resources and the resurgence of deep convolutional neural networks are key drivers for these advancements. Deep learning techniques excel in uncovering concealed patterns within images, aiding clinicians in achieving precise diagnoses. They have proven to be particularly effective in organ segmentation, cancer detection, disease categorization, and computer-assisted diagnosis. Numerous deep learning approaches have been proposed for various diagnostic purposes in the analysis of medical images. A significant hurdle in integrating deep learning models into the medical domain is the scarcity of training data, primarily attributed to the challenges associated with collecting and accurately labelling data, a task requiring expertise. To address this limitation, transfer learning (TL) has emerged as a valuable strategy, leveraging pre-trained state-of-the-art models to tackle various medical imaging tasks. This comprehensive review highlights the methodologies, including preprocessing, segmentation, feature extraction, and classification, and evaluates the performance of various DL models and also the recent advancements in the deep learning like transfer learning approaches in medical image processing.

A universal medically assisted model for anatomical landmark detection in radioactive images

AbstractAnatomical landmark detection is a critical task in medical image analysis with significant research and practical value. The analysis of landmarks in radiological images is beneficial for its diagnosis and treatment. Currently, most methods are applied to datasets from specific anatomical regions, with only a few deep neural network models designed for mixed datasets. Meanwhile, the precise and fast localization of landmarks remains a significant challenge due to interdependencies among landmarks and the demanding accuracy requirements in medical clinical applications. In this work, we leverage the advantages of attention mechanisms for focusing on specific information, and propose a universal landmark detection model trained on mixed‐domain radioactive image datasets. The model consists of three main components: the local network, the global network, and the attention feature extraction module. The local network employs a U‐Net architecture with depthwise separable convolutions to extract local image features. Notably, we introduce partial depthwise separable convolutions as an option for further lightweight models. The global network comprises a sequence of dilated convolutions with varying dilation rates, expanding the receptive field to extract global features. The attention feature extraction module efficiently utilizes intermediate outputs to refine the precise location information of landmarks. Additionally, we introduce the regional attention loss function to provide the model with landmark offset information. We conduct experiments on three publicly available x‐ray image datasets covering head, hand, and chest regions. The experimental results demonstrate that the proposed universal model achieves fast convergence and excellent performance across multiple datasets.

Cross and local optimal avoidance of RIME algorithm: A segmentation study for COVID-19 X-ray images

Image segmentation is a crucial technique in analyzing X-ray medical images as it aids in uncovering relevant information concealed within a patient's body, a pivotal aspect of the diagnostic process. The effectiveness of computer-aided diagnosis systems largely depends on the accuracy of the image processing methods. In recent years, multi-threshold image segmentation methods have found widespread application in medical image analysis. Despite the effectiveness of some renowned methods for binary threshold segmentation, the field still faces challenges due to the high cost of threshold computation. Metaheuristic algorithms hold the potential to address this issue as they can produce sufficiently reasonable solutions with manageable computational overheads. While some similar methods have been proposed, the imbalance between exploration and exploitation results in instability as the number of thresholds increases. Consequently, these solutions suffer from reduced efficiency in computing thresholds. In this study, a variant of the latest RIME algorithm, termed SLCRIME, is proposed. This paper replaces the pseudo-random method with low-discrepancy Sobol sequences for solution initialization. Additionally, two methods aimed at avoiding local optima and promoting information exchange within the solution set are introduced, further enhancing its capability to search for optimal threshold sets for IS systems. Subsequently, a multi-threshold image segmentation model based on SLCRIME is proposed and applied to segment 6 COVID-19 X-ray images. In the experiments, SLCRIME is compared with 6 peer algorithms, and the results are evaluated using image segmentation accuracy, feature similarity index metrics (FSIM), peak signal-to-noise ratio (PSNR), and structural similarity index metrics (SSIM). The analysis indicates that SLCRIME achieves optimal thresholds at reasonable computational costs and outperforms other algorithms in terms of performance.

Avrupa Meme Görüntüleme Diploması Çoktan Seçmeli Örnek Soruları: Büyük Dil Modellerinin Yeteneklerinin Değerlendirilmesi

Dear editor, In pursuit of a fresh outlook, my objective was to gauge the abilities of a multimodal large language model (LLM) against sample questions from the European Diploma in Breast Imaging (EDBI) test, an initiative by the European Society of Breast Imaging. Large language models are pushing the potential in radiology, from interpreting text and medical images to generating reports (Bhayana, 2024). Generative Pre-trained Transformer 4 (GPT-4) has notably passed a national mammography board exam with clarity (Almeida et al, 2024). As the latest version among multimodal LLM types, GPT-4 is capable of answering questions requiring both lower-order and higher-order thinking. Three written sample questions, where multiple choices could be correct, were evaluated. It was noted that there was no negative marking for incorrect answers (https://www.eusobi.org/european-diploma-in-breast-imaging-edbi/). The scoring system was adapted from the European Diploma in Radiology scoring guidelines (https://www.myebr.org/edir-scoring-faqs). Data was obtained from Google Gemini, GPT-3.5, and GPT-4 in March 2024. When assigning a value of 1 point to each question, GPT-4 reached an accuracy of 78%, GPT-3.5 achieved 50%, and Google Gemini scored 22.2%. This notable success in the sample questions from the EDBI particularly emphasizes GPT-4's potential in aiding clinical decision-making. Future studies may assess its performance in questions requiring medical image analysis, such as mammography, breast ultrasound, or breast magnetic resonance imaging.

Affine Collaborative Normalization: A shortcut for adaptation in medical image analysis

The paradigm of “pretraining-then-finetuning” (PT-FT) has been extensively explored to enhance the performance of clinical applications with limited annotations. A major impediment to applying such workflow to various medical imaging modalities is the lack of parameter-free approaches boosting the transferability against notable domain shifts. The success of normalization techniques in domain adaptation provides a promising solution, while inaccessible source data in finetuning (FT) poses new challenges. In this paper, we revisit the Batch Normalization module in PT-FT and propose a unified framework for both fast transferability estimation and transferability-aware finetuning. We discovered that the vanilla FT suffers from the issue that feature patterns of corresponding channels could be misaligned across domains. Hence, we present an Affine Collaborative Normalization (AC-Norm) to dynamically recalibrate the channels in the target model according to the cross-domain channel-wise correlations without any source data and extra parameters. AC-Norm provides very competitive results compared to state-of-the-art FT methods in six classification/segmentation tasks. We also demonstrate the capability of AC-Norm in estimating the transferability of pretrained models in only single-step backpropagation. Our code is available at https://github.com/EndoluminalSurgicalVision-IMR/ACNorm.

Co-ordinate-based positional embedding that captures resolution to enhance transformer’s performance in medical image analysis

Vision transformers (ViTs) have revolutionized computer vision by employing self-attention instead of convolutional neural networks and demonstrated success due to their ability to capture global dependencies and remove spatial biases of locality. In medical imaging, where input data may differ in size and resolution, existing architectures require resampling or resizing during pre-processing, leading to potential spatial resolution loss and information degradation. This study proposes a co-ordinate-based embedding that encodes the geometry of medical images, capturing physical co-ordinate and resolution information without the need for resampling or resizing. The effectiveness of the proposed embedding is demonstrated through experiments with UNETR and SwinUNETR models for infarct segmentation on MRI dataset with AxTrace and AxADC contrasts. The dataset consists of 1142 training, 133 validation and 143 test subjects. Both models with the addition of co-ordinate based positional embedding achieved substantial improvements in mean Dice score by 6.5% and 7.6%. The proposed embedding showcased a statistically significant advantage p-value< 0.0001 over alternative approaches. In conclusion, the proposed co-ordinate-based pixel-wise positional embedding method offers a promising solution for Transformer-based models in medical image analysis. It effectively leverages physical co-ordinate information to enhance performance without compromising spatial resolution and provides a foundation for future advancements in positional embedding techniques for medical applications.

Multimodal Machine Learning in Image-Based and Clinical Biomedicine: Survey and Prospects

Machine learning (ML) applications in medical artificial intelligence (AI) systems have shifted from traditional and statistical methods to increasing application of deep learning models. This survey navigates the current landscape of multimodal ML, focusing on its profound impact on medical image analysis and clinical decision support systems. Emphasizing challenges and innovations in addressing multimodal representation, fusion, translation, alignment, and co-learning, the paper explores the transformative potential of multimodal models for clinical predictions. It also highlights the need for principled assessments and practical implementation of such models, bringing attention to the dynamics between decision support systems and healthcare providers and personnel. Despite advancements, challenges such as data biases and the scarcity of “big data” in many biomedical domains persist. We conclude with a discussion on principled innovation and collaborative efforts to further the mission of seamless integration of multimodal ML models into biomedical practice.