Quadruple-Consistency Vision Transformer for Medical Image Segmentation with Limited Number of Sparse Annotations

  • Abstract
  • Literature Map
  • Similar Papers
Abstract
Translate article icon Translate Article Star icon

Deep learning has significantly advanced the field of medical image segmentation but typically relies on extensive, densely annotated datasets, which are both costly and time-consuming to prepare. In response to the need for reducing annotation efforts, this study investigates a novel supervision approach named Semi-Scribble Supervised Learning, which utilizes a combination of semi-supervised (SSL) and weaklysupervised learning (WSL) techniques. This approach leverages both a large volume of unlabeled data and a smaller set of sparsely annotated, scribble-based labels. We introduce the Quadruple-Consistency Vision Transformer (4C-ViT), which capitalizes on the recent success of Vision Transformers in capturing intricate image features. Specifically, the proposed 4C-ViT employs an advanced consistency training strategy that incorporates quadruple perturbations at both the data and network levels, enhancing the network’s robustness and performance. The efficacy of 4 C -ViT is demonstrated on a publicly available MRI cardiac segmentation benchmark, where it outperforms other baseline methods across several evaluation metrics. The proposed $4 \mathrm{C}-\mathrm{ViT}$, alongside all baseline methods and the challenging yet realistic dataset, is made public available at https://github.com/ziyangwang007/CVSSL-MIS.

Similar Papers
  • Research Article
  • Cite Count Icon 2
  • 10.35629/5252-0612125135
Role of Image Segmentation and Deep Learning in Medical Imaging
  • Dec 1, 2024
  • International Journal of Advances in Engineering and Management
  • Ayuns Luz + 1 more

The rapid advancements in medical imaging technologies have significantly enhanced diagnostic accuracy and clinical decision-making in modern healthcare. Image segmentation and deep learning have emerged as transformative tools among these advancements. This article explores the pivotal role of image segmentation and deep learning in medical imaging, detailing their methodologies, applications, challenges, and future directions. Deep learning, particularly Convolutional Neural Networks (CNNs), has revolutionized medical imaging by automating the analysis of complex datasets and improving diagnostic precision. Image segmentation, a fundamental component of medical imaging, allows for delineating specific structures such as organs, tissues, and pathological regions. Together, these technologies have been applied in diverse fields, including oncology, cardiology, neurology, and ophthalmology, enabling applications such as tumor detection, organ segmentation, disease progression monitoring, and treatment planning. However, despite its transformative potential, the integration of deep learning into medical imaging faces several challenges. These include data scarcity, privacy concerns, interpretability issues, and regulatory hurdles. The article discusses various strategies to address these challenges, such as data augmentation, transfer learning, and the development of explainable AI models to ensure transparency and trustworthiness. Evaluation metrics, such as accuracy, sensitivity, specificity, and Dice Similarity Coefficient (DSC), are essential for assessing model performance. Rigorous clinical validation and regulatory approval are crucial to integrating deep learning systems into clinical workflows effectively. Looking ahead, the future of deep learning in medical imaging holds immense promise. Innovations like multimodal imaging, personalized medicine, and AI-driven automation are set to further revolutionize the field, enhancing the efficiency and accuracy of diagnostics. Collaborative efforts between clinicians, researchers, and AI developers will play a vital role in overcoming current limitations and driving progress. This article concludes by emphasizing the transformative potential of deep learning and image segmentation in medical imaging, highlighting their ability to improve diagnostic accuracy, streamline clinical workflows, and ultimately, enhance patient care. By addressing current challenges and continuing to innovate, these technologies are poised to redefine the landscape of medical diagnostics and treatment in the years to come.

  • Research Article
  • Cite Count Icon 4
  • 10.1016/j.vrih.2024.04.001
A review of medical ocular image segmentation
  • Jun 1, 2024
  • Virtual Reality & Intelligent Hardware
  • Lai Wei + 1 more

A review of medical ocular image segmentation

  • Research Article
  • Cite Count Icon 153
  • 10.1007/s10278-021-00556-w
Systematic Review of Generative Adversarial Networks (GANs) for Medical Image Classification and Segmentation.
  • Jan 12, 2022
  • Journal of digital imaging
  • Jiwoong J Jeong + 5 more

In recent years, generative adversarial networks (GANs) have gained tremendous popularity for various imaging related tasks such as artificial image generation to support AI training. GANs are especially useful for medical imaging-related tasks where training datasets are usually limited in size and heavily imbalanced against the diseased class. We present a systematic review, following the PRISMA guidelines, of recent GAN architectures used for medical image analysis to help the readers in making an informed decision before employing GANs in developing medical image classification and segmentation models. We have extracted 54 papers that highlight the capabilities and application of GANs in medical imaging from January 2015 to August 2020 and inclusion criteria for meta-analysis. Our results show four main architectures of GAN that are used for segmentation or classification in medical imaging. We provide a comprehensive overview of recent trends in the application of GANs in clinical diagnosis through medical image segmentation and classification and ultimately share experiences for task-based GAN implementations.

  • Research Article
  • Cite Count Icon 7
  • 10.1109/access.2019.2950960
Segmentation Algorithm of Medical Exercise Rehabilitation Image Based on HFCNN and IoT
  • Jan 1, 2019
  • IEEE Access
  • Liang Ding + 1 more

Deep learning has achieved great success in the field of computer vision, and the precision in image classification and image detection has surpassed humans. Therefore, this paper combines deep learning and medical image segmentation, focusing on how to improve the accuracy and speed of segmentation algorithm of medical exercise rehabilitation image. Aiming at the shortcomings of traditional medical image recognition methods, a medical exercise rehabilitation image segmentation algorithm based on hierarchical features of convolutional neural networks is proposed, this paper calls it as hierarchical features of convolutional neural networks (HFCNN). The algorithm firstly samples the convolution output of multiple layers in the convolutional neural network to a unified scale and combines them to construct a hierarchical feature. This hierarchical feature combines the structural information of objects contained in the shallow layer of the network with the semantic information of objects contained in the deep layers of the network, so it has a strong ability to express. Secondly, the image can be segmented into multiple super pixels by the super pixel segmentation algorithm. The classifier is trained using the hierarchical features of the super pixel, and then the classification result is mapped back to the pixel. Finally, a fully connected conditional random field algorithm including one-potential potential energy and paired potential energy is constructed. The corresponding energy function is used to smooth the classification result of the pixel, and the regional consistency and continuity of the pixel mark are improved. Compared with many classical convolutional neural network algorithms, this algorithm not only accelerates the network convergence speed, shortens the training time, but also significantly improves the accuracy of segmentation algorithm of medical exercise rehabilitation image, showing good practical value.

  • Research Article
  • Cite Count Icon 53
  • 10.1016/j.eswa.2023.119939
DSEU-net: A novel deep supervision SEU-net for medical ultrasound image segmentation
  • Mar 22, 2023
  • Expert Systems with Applications
  • Gongping Chen + 6 more

DSEU-net: A novel deep supervision SEU-net for medical ultrasound image segmentation

  • Research Article
  • Cite Count Icon 6
  • 10.14257/ijsip.2015.8.7.21
Medical and Natural Image Segmentation Algorithm using M-F based Optimization Model and Modified Fuzzy Clustering: A Novel Approach
  • Jul 31, 2015
  • International Journal of Signal Processing, Image Processing and Pattern Recognition
  • Bingquan Huo + 2 more

In this paper, we propose and present a novel algorithm for medical image segmentation (MIS). By analyzing the current state-of-the-art related algorithms, we introduce the multi-band active contour model based limit function to make the multilayer segmentation available. With the development of image segmentation technology, the development of medical image segmentation technology also got very big, because there is no find common, accepted effect ideal is suitable for medical image segmentation method, almost existing each kind of segmentation method has application in the field of medical image segmentation. Furtherly, with the optimized aims of being robust to the noise and avoiding the bad effluence on the result, we adopt the kernel method and new initialization curve. This model suffers from low noise robustness, and model algorithm is difficult to achieve. Integrated segmentation technology refers to two or more technology is used, combined with their own advantages, so they can on the accuracy or efficiency to achieve better performance than when using a single. A new penalty term is introduced to improve numerical stability and the step length is increased to improve efficiency. As far as the robustness and effectiveness are concerned, our method is better than the existing medical image segmentation algorithms. Experimental analysis verifies the success of our method.

  • PDF Download Icon
  • Research Article
  • Cite Count Icon 34
  • 10.1155/2019/6134942
Medical Image Segmentation Algorithm Based on Feedback Mechanism CNN
  • Aug 1, 2019
  • Contrast Media & Molecular Imaging
  • Feng-Ping An + 1 more

With the development of computer vision and image segmentation technology, medical image segmentation and recognition technology has become an important part of computer-aided diagnosis. The traditional image segmentation method relies on artificial means to extract and select information such as edges, colors, and textures in the image. It not only consumes considerable energy resources and people's time but also requires certain expertise to obtain useful feature information, which no longer meets the practical application requirements of medical image segmentation and recognition. As an efficient image segmentation method, convolutional neural networks (CNNs) have been widely promoted and applied in the field of medical image segmentation. However, CNNs that rely on simple feedforward methods have not met the actual needs of the rapid development of the medical field. Thus, this paper is inspired by the feedback mechanism of the human visual cortex, and an effective feedback mechanism calculation model and operation framework is proposed, and the feedback optimization problem is presented. A new feedback convolutional neural network algorithm based on neuron screening and neuron visual information recovery is constructed. So, a medical image segmentation algorithm based on a feedback mechanism convolutional neural network is proposed. The basic idea is as follows: The model for obtaining an initial region with the segmented medical image classifies the pixel block samples in the segmented image. Then, the initial results are optimized by threshold segmentation and morphological methods to obtain accurate medical image segmentation results. Experiments show that the proposed segmentation method has not only high segmentation accuracy but also extremely high adaptive segmentation ability for various medical images. The research in this paper provides a new perspective for medical image segmentation research. It is a new attempt to explore more advanced intelligent medical image segmentation methods. It also provides technical approaches and methods for further development and improvement of adaptive medical image segmentation technology.

  • Research Article
  • Cite Count Icon 969
  • 10.1016/j.media.2020.101693
Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation.
  • Apr 3, 2020
  • Medical Image Analysis
  • Nima Tajbakhsh + 5 more

Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation.

  • Research Article
  • Cite Count Icon 262
  • 10.1016/j.inffus.2022.09.031
Medical image segmentation using deep semantic-based methods: A review of techniques, applications and emerging trends
  • Oct 8, 2022
  • Information Fusion
  • Imran Qureshi + 7 more

Medical image segmentation using deep semantic-based methods: A review of techniques, applications and emerging trends

  • Research Article
  • 10.26483/ijarcs.v2i6.887
Optimum Regularized Joint Registration and Segmentation Method for Medical Brain Images
  • Jan 1, 2011
  • International Journal of Advanced Research in Computer Science
  • N Usha Rani + 1 more

Image registration and segmentation are the two important processes that are frequently used in medical image processing and computer vision applications. In traditional medical image applications both the techniques are applied independently even though the solution to one impacts the solution of the other. Currently medical image segmentation is very complex task due to the lack of sufficient contrast, SNR, and volume averages caused due to the non-uniform magnetic field. The problem is still high with MRI scans rather than other scans due to lack of real boundary. Availability of sophisticated diagnostic methods in the medical domain, demands the fusion of information from different sources for the better analysis. Similarity is enhanced by performing the non-rigid registration, where the local registration highly depends on segmentation of objects. This paper deals with the Atlas-based segmentation technique requires that the given atlas image is to be registered with the target image to find the desired shape segmentation in the target image. This paper discus the joint registration and segmentation process is achieved through highly accurate variational cost effective Distance Regularized Level Set Evolution (DRLSE) method for medical scan images. The key features of this algorithm are, it can accurately converge towards sharp object boundary corners due to forward and backward diffusion and also applied for small and large deformations. It uses less computational cost due to large time steps. Keywords: Medical Image Processing, Image Registration, Segmentation, Joint Registration and Segmentation, Distance Regularized Level Set Evolution, Deformations, Convergence, Computational time.

  • Conference Article
  • Cite Count Icon 2
  • 10.1109/cbms.1995.465430
Multiresolution wavelet decomposition and neuro-fuzzy clustering for segmentation of radiographic images
  • Jun 9, 1995
  • S Pemmaraju + 3 more

Segmentation of medical images is a challenging problem in the field of image analysis. Several diagnostics are based on proper segmentation of the digitized image. Segmentation of medical images is needed for applications involving estimation of the boundary of an object, classification of tissue abnormalities, shape analysis, contour detection and texture segmentation. Despite the existence of several techniques, segmentation of specific medical images still remains a crucial problem due to the complex nature of most medical images. A multiresolution image representation approach is used for better analyzing the information present in an image. We use multiresolution wavelet decomposition to reconstruct the original image such that it contains all the salient features relevant to segmentation and is devoid of the low frequency noise and texture information that can be ignored while segmenting the image. An unsupervised neural network with fuzzy learning rules is then used to segment the reconstructed image. >

  • Book Chapter
  • 10.58532/v3baai6p6ch1
A REVIEW OF SEGMENTATION TECHNIQUES ON MEDICAL IMAGES
  • Mar 5, 2024
  • Sajeev Ram Arumugam + 3 more

A recent development in medical image processing called medical image segmentation has dramatically increased healthcare long-term viability. Medical image segmentation is a critical task in contemporary healthcare. It enables accurate delineation of anatomical features, tumours, and diseased regions, which facilitates precise analysis and diagnosis. Thus, image segmentation is the crucial technique for enabling the discovery, characterization, and visualization of the regions of interest in any medical image. In addition to being complex and prolonged, the clinician manual segmentation of the medical image is also not very precise, mainly in light of the budding scope of medical imaging processes and the irresistible volume of medical images that want to be analyzed. Therefore, it is vital to explore current image segmentation techniques utilizing automated algorithms that are defined and demand the smallest amount of user input, particularly for medical images. Identifying and isolating the anatomical structure during the segmentation process is vital. The significance of image segmentation in extracting decision-making information is projected in this study, and existing medical imaging methods are discussed with numerous research breakthroughs. The segmentation methods used on medical images are thoroughly examined in this paper, which spans a wide range of imaging modalities and approaches. The research technique includes a precise search of the literature, the extraction of pertinent studies, and a thorough analysis of their methodologies and results. The segmentation of studies according to imaging modalities, segmentation goals, and assessment metrics was part of the research approach. The review also highlights how important it is to select evaluation standards that are appropriate for the segmentation task.

  • Research Article
  • 10.36548/jaicn.2023.3.005
Critical Studies on Lesion Segmentation in Medical Images
  • Sep 1, 2023
  • Journal of Artificial Intelligence and Capsule Networks
  • Alok Kumar + 1 more

In medical images, lesion segmentation is used to locate and isolate abnormal structures. It is an essential part of medical image analysis for precise diagnosis and care. However, obstacles exist in medical image lesion segmentation owing to things like image noise, shape and size fluctuation, and poor image quality. Automated lesion segmentation methods include conventional image processing techniques, deep learning (DL) models and machine learning (ML) algorithms to name a few. Thresholding, region growth, and active contour models are examples of conventional methods, while decision trees, random forests, and support vector machines are examples of ML techniques. DL models particularly convolutional neural networks (CNNs), have shown extraordinary performance in lesion segmentation because to their innate potential to autonomously collect high-level characteristics. The objective of the research is to study lesion segmentation in medical images and explore different methods for accurate and precise diagnosis and care.The research focuses on the obstacles faced in lesion segmentation in medical images, such as image noise, shape and size fluctuation, and poor image quality. The research also highlights the need for evaluation metrics, such as sensitivity, specificity, Dice coefficient, and Hausdorff distance, to assess the performance of lesion segmentation algorithms. Additionally, the research emphasizes the importance of annotated datasets for training and evaluating the performance of segmentation algorithms.

  • Conference Article
  • 10.1117/12.912618
Model-based coupled denoising and segmentation of medical images
  • Feb 23, 2012
  • Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE
  • Ahmet Tuysuzoglu + 2 more

We present a new model-based framework for coupled segmentation and de-noising of medical images. The segmentation and de-noising steps are coupled through a discrete formulation of the total variation de-noising problem in a restricted setting such that each pixel in the image has its de-noised intensity level selected from a drastically reduced set of intensities. By creating such a reduced set of intensity levels, in which each intensity level represent the intensity across a region to be segmented, the intensity value for each de-noised pixel will be forced to assume a value in this limited set; by associating all pixels with the same de-noised value as a single region, image segmentation is naturally achieved. We derive two formulations corresponding to two noise models: additive white Gaussian and multiplicative Rayleigh. We furthermore show that the proposed framework enables globally optimal foreground/background segmentation of images with Rayleigh distributed noise.

  • Conference Article
  • Cite Count Icon 24
  • 10.1109/iccrd.2010.155
An Improved Method of Segmentation Using Fuzzy-Neuro Logic
  • Jan 1, 2010
  • S Sathish Kumar + 3 more

Image segmentation is an important process to extract information from complex medical images. Segmentation has wide application in medical field. The main objective of image segmentation is to partition an image into mutually exclusive and exhausted regions such that each region of interest is spatially contiguous and the pixels within the region are homogeneous with respect to a predefined criterion. Widely used homogeneity criteria include values of intensity, texture, color, range, surface normal and surface curvatures. During the past, many researchers in the field of medical imaging and soft computing have made significant survey in the field of image segmentation. This paper aims to develop an improved method of segmentation using Fuzzy- Neuro logic to detect various tissues like white matter, gray matter; cerebral spinal fluid and tumor for a given magnetic resonance image data set. Generally magnetic resonance images always contain a significant amount of noise caused by operator performance, equipment, and the environment, which can lead to serious inaccuracies. So segmentation of such medical images is a challenging problem in the field of image analysis. Several diagnostics are based on proper segmentation of the digitized image. Segmentation of medical images is needed for applications involving estimation of the boundary of an object, classification of tissue abnormalities, shape analysis, contour detection. In particular Fuzzy-Neuro logic segmentation algorithm is used to provide satisfactory results compared to K-means, Fuzzy C-Means, Neural Network and Fuzzy logic.

Save Icon
Up Arrow
Open/Close