Medical Computer Vision Research Articles

An overview of current developments and methods for identifying diabetic foot ulcers: A survey

AbstractDiabetic foot ulcers (DFUs) present a substantial health risk across diverse age groups, creating challenges for healthcare professionals in the accurate classification and grading. DFU plays a crucial role in automated health monitoring and diagnosis systems, where the integration of medical imaging, computer vision, statistical analysis, and gait information is essential for comprehensive understanding and effective management. Diagnosing DFU is imperative, as it plays a major role in the processes of diagnosis, treatment planning, and neuropathy research within automated health monitoring and diagnosis systems. To address this, various machine learning and deep learning‐based methodologies have emerged in the literature to support healthcare practitioners in achieving improved diagnostic analyses for DFU. This survey paper investigates various diagnostic methodologies for DFU, spanning traditional statistical approaches to cutting‐edge deep learning techniques. It systematically reviews key stages involved in diabetic foot ulcer classification (DFUC) methods, including preprocessing, feature extraction, and classification, explaining their benefits and drawbacks. The investigation extends to exploring state‐of‐the‐art convolutional neural network models tailored for DFUC, involving extensive experiments with data augmentation and transfer learning methods. The overview also outlines datasets commonly employed for evaluating DFUC methodologies. Recognizing that neuropathy and reduced blood flow in the lower limbs might be caused by atherosclerotic blood vessels, this paper provides recommendations to researchers and practitioners involved in routine medical therapy to prevent substantial complications. Apart from reviewing prior literature, this survey aims to influence the future of DFU diagnostics by outlining prospective research directions, particularly in the domains of personalized and intelligent healthcare. Finally, this overview is to contribute to the continual evolution of DFU diagnosis in order to provide more effective and customized medical care.This article is categorized under: Application Areas > Health Care Technologies > Machine Learning Technologies > Artificial Intelligence

Deep learning for automated boundary detection and segmentation in organ donation photography

Abstract Objectives Medical photography is ubiquitous and plays an increasingly important role in the fields of medicine and surgery. Any assessment of these photographs by computer vision algorithms requires first that the area of interest can accurately be delineated from the background. We aimed to develop deep learning segmentation models for kidney and liver organ donation photographs where accurate automated segmentation has not yet been described. Methods Two novel deep learning models (Detectron2 and YoloV8) were developed using transfer learning and compared against existing tools for background removal (macBGRemoval, remBGisnet, remBGu2net). Anonymised photograph datasets comprised training/internal validation sets (821 kidney and 400 liver images) and external validation sets (203 kidney and 208 liver images). Each image had two segmentation labels: whole organ and clear view (parenchyma only). Intersection over Union (IoU) was the primary outcome, as the recommended metric for assessing segmentation performance. Results In whole kidney segmentation, Detectron2 and YoloV8 outperformed other models with internal validation IoU of 0.93 and 0.94, and external validation IoU of 0.92 and 0.94, respectively. Other methods – macBGRemoval, remBGisnet and remBGu2net – scored lower, with highest internal validation IoU at 0.54 and external validation at 0.59. Similar results were observed in liver segmentation, where Detectron2 and YoloV8 both showed internal validation IoU of 0.97 and external validation of 0.92 and 0.91, respectively. The other models showed a maximum internal validation and external validation IoU of 0.89 and 0.59 respectively. All image segmentation tasks with Detectron2 and YoloV8 completed within 0.13–1.5 s per image. Conclusions Accurate, rapid and automated image segmentation in the context of surgical photography is possible with open-source deep-learning software. These outperform existing methods and could impact the field of surgery, enabling similar advancements seen in other areas of medical computer vision.

Narrative review of patient-specific 3D visualization and reality technologies in skull base neurosurgery: enhancements in surgical training, planning, and navigation.

Recent advances in medical imaging, computer vision, 3-dimensional (3D) modeling, and artificial intelligence (AI) integrated technologies paved the way for generating patient-specific, realistic 3D visualization of pathological anatomy in neurosurgical conditions. Immersive surgical simulations through augmented reality (AR), virtual reality (VR), mixed reality (MxR), extended reality (XR), and 3D printing applications further increased their utilization in current surgical practice and training. This narrative review investigates state-of-the-art studies, the limitations of these technologies, and future directions for them in the field of skull base surgery. We begin with a methodology summary to create accurate 3D models customized for each patient by combining several imaging modalities. Then, we explore how these models are employed in surgical planning simulations and real-time navigation systems in surgical procedures involving the anterior, middle, and posterior cranial skull bases, including endoscopic and open microsurgical operations. We also evaluate their influence on surgical decision-making, performance, and education. Accumulating evidence demonstrates that these technologies can enhance the visibility of the neuroanatomical structures situated at the cranial base and assist surgeons in preoperative planning and intraoperative navigation, thus showing great potential to improve surgical results and reduce complications. Maximum effectiveness can be achieved in approach selection, patient positioning, craniotomy placement, anti-target avoidance, and comprehension of spatial interrelationships of neurovascular structures. Finally, we present the obstacles and possible future paths for the broader implementation of these groundbreaking methods in neurosurgery, highlighting the importance of ongoing technological advancements and interdisciplinary collaboration to improve the accuracy and usefulness of 3D visualization and reality technologies in skull base surgeries.

A New Class of Edge Filter Based on a Cross-correlation-like Equation Derived from Fractional Calculus Principles

In this paper, we propose a new sliding window edge-oriented filter that computes the output pixels using a cross-correlation-like equation derived from the principles of fractional calculus (FC); thus, we call it the “fractional cross-correlation filter” (FCCF). We assessed the performance of this filter utilizing exclusively edge-preservation-oriented metrics such as the gradient conduction mean square error (GCMSE), the edge-based structural similarity (EBSSIM), and the multi-scale structural similarity (MS-SSIM); we conducted a statistical assessment of the performance of the filter based on those metrics by using the Berkeley segmentation dataset benchmark as a test corpus. Experimental data reveal that our approach achieves higher performance compared to conventional edge filters for all the metrics considered in this study. This is supported by the statistical analysis we carried out; specifically, the FCCF demonstrates a consistent enhancement in edge detection. We also conducted additional experiments for determining the main filter parameters, which we found to be optimal for a broad spectrum of images. The results underscore the FCCF’s potential to make significant contributions to the advancement of image processing techniques since many practical applications such as medical imaging, image enhancement, and computer vision rely heavily on edge detection filters.

Fine structural human phantom in dentistry and instance tooth segmentation

In this study, we present the development of a fine structural human phantom designed specifically for applications in dentistry. This research focused on assessing the viability of applying medical computer vision techniques to the task of segmenting individual teeth within a phantom. Using a virtual cone-beam computed tomography (CBCT) system, we generated over 170,000 training datasets. These datasets were produced by varying the elemental densities and tooth sizes within the human phantom, as well as varying the X-ray spectrum, noise intensity, and projection cutoff intensity in the virtual CBCT system. The deep-learning (DL) based tooth segmentation model was trained using the generated datasets. The results demonstrate an agreement with manual contouring when applied to clinical CBCT data. Specifically, the Dice similarity coefficient exceeded 0.87, indicating the robust performance of the developed segmentation model even when virtual imaging was used. The present results show the practical utility of virtual imaging techniques in dentistry and highlight the potential of medical computer vision for enhancing precision and efficiency in dental imaging processes.

Image Processing using Python

Image processing using Python has become increasingly prevalent due to the availability of powerful libraries such as OpenCV, Pillow (PIL), and scikit-image. This paper provides an overview of digital image processing techniques implemented in Python, focusing on common tasks and operations such as image enhancement, restoration, segmentation, feature extraction, and pattern recognition. We discuss the advantages of using Python for image processing, including its simplicity, versatility, and extensive library support. Through code examples and demonstrations, we illustrate how Python can be used to perform various image processing tasks efficiently and effectively. Additionally, we highlight the applications of digital image processing in different domains, including medical imaging, remote sensing, computer vision, and multimedia. Overall, this paper serves as a comprehensive guide to digital image processing using Python, catering to researchers, practitioners, and enthusiasts interested in leveraging Python for image analysis and manipulation.

EndoViT: pretraining vision transformers on a large collection of endoscopic images

PurposeAutomated endoscopy video analysis is essential for assisting surgeons during medical procedures, but it faces challenges due to complex surgical scenes and limited annotated data. Large-scale pretraining has shown great success in natural language processing and computer vision communities in recent years. These approaches reduce the need for annotated data, which is of great interest in the medical domain. In this work, we investigate endoscopy domain-specific self-supervised pretraining on large collections of data.MethodsTo this end, we first collect Endo700k, the largest publicly available corpus of endoscopic images, extracted from nine public Minimally Invasive Surgery (MIS) datasets. Endo700k comprises more than 700,000 images. Next, we introduce EndoViT, an endoscopy-pretrained Vision Transformer (ViT), and evaluate it on a diverse set of surgical downstream tasks.ResultsOur findings indicate that domain-specific pretraining with EndoViT yields notable advantages in complex downstream tasks. In the case of action triplet recognition, our approach outperforms ImageNet pretraining. In semantic segmentation, we surpass the state-of-the-art (SOTA) performance. These results demonstrate the effectiveness of our domain-specific pretraining approach in addressing the challenges of automated endoscopy video analysis.ConclusionOur study contributes to the field of medical computer vision by showcasing the benefits of domain-specific large-scale self-supervised pretraining for vision transformers. We release both our code and pretrained models to facilitate further research in this direction: https://github.com/DominikBatic/EndoViT.

Deep learning approaches for seizure video analysis: A review

Seizure events can manifest as transient disruptions in the control of movements which may be organized in distinct behavioral sequences, accompanied or not by other observable features such as altered facial expressions. The analysis of these clinical signs, referred to as semiology, is subject to observer variations when specialists evaluate video-recorded events in the clinical setting. To enhance the accuracy and consistency of evaluations, computer-aided video analysis of seizures has emerged as a natural avenue. In the field of medical applications, deep learning and computer vision approaches have driven substantial advancements. Historically, these approaches have been used for disease detection, classification, and prediction using diagnostic data; however, there has been limited exploration of their application in evaluating video-based motion detection in the clinical epileptology setting. While vision-based technologies do not aim to replace clinical expertise, they can significantly contribute to medical decision-making and patient care by providing quantitative evidence and decision support. Behavior monitoring tools offer several advantages such as providing objective information, detecting challenging-to-observe events, reducing documentation efforts, and extending assessment capabilities to areas with limited expertise. The main applications of these could be (1) improved seizure detection methods; (2) refined semiology analysis for predicting seizure type and cerebral localization. In this paper, we detail the foundation technologies used in vision-based systems in the analysis of seizure videos, highlighting their success in semiology detection and analysis, focusing on work published in the last 7 years. We systematically present these methods and indicate how the adoption of deep learning for the analysis of video recordings of seizures could be approached. Additionally, we illustrate how existing technologies can be interconnected through an integrated system for video-based semiology analysis. Each module can be customized and improved by adapting more accurate and robust deep learning approaches as these evolve. Finally, we discuss challenges and research directions for future studies.

Image and video analysis using graph neural network for Internet of Medical Things and computer vision applications

AbstractGraph neural networks (GNNs) have revolutionised the processing of information by facilitating the transmission of messages between graph nodes. Graph neural networks operate on graph‐structured data, which makes them suitable for a wide variety of computer vision problems, such as link prediction, node classification, and graph classification. The authors explore in depth the applications of GNNs in computer vision, including their design considerations, architectural challenges, applications, and implementation concerns. While conventional convolutional neural networks (CNNs) excel at object recognition in images and videos, GNN architectures offer a novel method for addressing various image and video comprehension challenges. A novel deep neural network‐based model for image and video analysis is proposed, which combines a neural network with fully connected layers on a graph. The proposed architecture extracts highly discriminative information from images and videos by leveraging the graph structure. Also, the investigation focuses on the enhancement of underlying connection network estimation using cutting‐edge graph learning algorithms. Experimental results on real‐world datasets demonstrate that the proposed GNN model is preferable to existing state‐of‐the‐art methods. It obtains a remarkable 96.63% accuracy on the ImageNet dataset, outperforming heuristic approaches, artificial neural networks, and conventional CNN techniques. From the results, we can see that GNNs are a potent instrument for graph data analysis and pave the way for machines to achieve human‐level visual intuition.

Uncovering the effects of model initialization on deep model generalization: A study with adult and pediatric chest X-ray images.

Model initialization techniques are vital for improving the performance and reliability of deep learning models in medical computer vision applications. While much literature exists on non-medical images, the impacts on medical images, particularly chest X-rays (CXRs) are less understood. Addressing this gap, our study explores three deep model initialization techniques: Cold-start, Warm-start, and Shrink and Perturb start, focusing on adult and pediatric populations. We specifically focus on scenarios with periodically arriving data for training, thereby embracing the real-world scenarios of ongoing data influx and the need for model updates. We evaluate these models for generalizability against external adult and pediatric CXR datasets. We also propose novel ensemble methods: F-score-weighted Sequential Least-Squares Quadratic Programming (F-SLSQP) and Attention-Guided Ensembles with Learnable Fuzzy Softmax to aggregate weight parameters from multiple models to capitalize on their collective knowledge and complementary representations. We perform statistical significance tests with 95% confidence intervals and p-values to analyze model performance. Our evaluations indicate models initialized with ImageNet-pretrained weights demonstrate superior generalizability over randomly initialized counterparts, contradicting some findings for non-medical images. Notably, ImageNet-pretrained models exhibit consistent performance during internal and external testing across different training scenarios. Weight-level ensembles of these models show significantly higher recall (p<0.05) during testing compared to individual models. Thus, our study accentuates the benefits of ImageNet-pretrained weight initialization, especially when used with weight-level ensembles, for creating robust and generalizable deep learning solutions.

Image Encryption Algorithm based on Pseudo-random Sequence

The applications of digital images is widespread in various fields such as medical diagnosis, film production, intelligent transportation, security monitoring, virtual reality and computer vision. The security of image transmission is a widely discussed topic, and ensuring the protection of digital images has become a focal point in Internet security communication. The selection of an algorithm with strong encryption performance holds significant theoretical and practical importance for image transmission. This paper takes the pseudo-random sequence as a starting point and primarily utilizes scrambling operations, the Mersenne rotation algorithm, and operations based on chaotic sequences to explore image encryption algorithms. Firstly, it introduces the encryption principles of these three methods, followed by respective application cases and implementation using MATLAB software. Pseudo-random sequences are employed as tools in all three methods, reo rdering pixel positions according to their order in the pseudo-random sequence during scrambling operation. In the Mersenne rotation algorithm, pseudo-random sequences are used to generate random number seeds. Chaotic sequences are utilized in combination with scrambling operations to achieve image encryption through generating pseudo-random sequences. Finally, after analyzing information entropy, histograms, encryption/decryption times, and adjacent pixel correlations, we select the optimal image encryption algorithm that provides better digital image encryption effects and higher security levels for each application case along with code implementations.

A Rigid Image Registration by Combined Local Features and Genetic Algorithms

Abstract Image registration is an essential pre-processing step required for many image processing applications such as medical imaging and computer vision. The aim is to geometrically align two or more images of the same scene by establishing a mapping that relies on each point from one image to its corresponding point of another image. Scale invariant feature transform (SIFT) and speeded up robust features (SURF) are well-liked local features descriptors that have been extensively utilised for feature-based image registration due to their inherent properties such as invariance, changes in illumination, and noise. Moreover, the task of registration can be viewed as an optimization problem that can be solved by applying genetic algorithms (GAs). This paper presents an efficient feature image registration method based on combined local features and GAs. Firstly, the procedure consists of extracting the local features from the images by combining SIFT and SURF algorithms and matching them to refine the feature set data. Therefore, an adaptive GA based on fitness sharing and elitism techniques is employed to find the optimal rigid transformation parameters that best align the feature points by minimizing a distance metric. The suggested method is applied for registering medical images and the obtained results are significant compared to other feature-based approaches with reasonable computation time.

MWG-UNet: Hybrid Deep Learning Framework for Lung Fields and Heart Segmentation in Chest X-ray Images.

Deep learning technology has achieved breakthrough research results in the fields of medical computer vision and image processing. Generative adversarial networks (GANs) have demonstrated a capacity for image generation and expression ability. This paper proposes a new method called MWG-UNet (multiple tasking Wasserstein generative adversarial network U-shape network) as a lung field and heart segmentation model, which takes advantages of the attention mechanism to enhance the segmentation accuracy of the generator so as to improve the performance. In particular, the Dice similarity, precision, and F1 score of the proposed method outperform other models, reaching 95.28%, 96.41%, and 95.90%, respectively, and the specificity surpasses the sub-optimal models by 0.28%, 0.90%, 0.24%, and 0.90%. However, the value of the IoU is inferior to the optimal model by 0.69%. The results show the proposed method has considerable ability in lung field segmentation. Our multi-organ segmentation results for the heart achieve Dice similarity and IoU values of 71.16% and 74.56%. The segmentation results on lung fields achieve Dice similarity and IoU values of 85.18% and 81.36%.

A novel hybrid heuristic adopted ensemble of deep learning models for COVID-19 detection framework using CT and X-Ray images

ABSTRACT In 2019, Corona Virus Disease (COVID)-19 has created an important impact on people’s health and economy because of its rapid spreading. Therefore, the earlier detection and diagnosis for handling the further spread is regarded as a significant factor. Computed Tomography (CT) images of the lungs are utilised for the detection process of COVID-19 infection. Chest X-Ray (CXR) and CT serves as the major source for the identification of COVID-19. Some of the classifiers have shown promising outcomes as well as better performance over various medical imaging applications and computer vision processes. Due COVID pandemic, researchers have utilised standard techniques for accurately identifying the coronavirus infection in lung images. Here, a new COVID-19 detection approach has been implemented with the adoption of ensemble classifiers to recognise the occurrence of COVID-19 infection at the starting phase. The required CT and the CXR images are garnered from publicly presented online sources. Meanwhile, the acquired images undergo the process of pre-processing using filtering methods and histogram equalisation techniques in order to ignore the unnecessary artifacts are presented in the images. Then, the detection of COVID-19 is made possible through Ensemble Classifiers (EC) such as Multi-scale Convolutional Neural Networks (CNN), Residual Attention Networks (RAN), Visual Geometry Group (VGG), and Long Short Term Memory (LSTM), where the variables within the machine learning strategies are tuned by Modified Mexican Axolotl Shuffled Frog-Leaping Optimization (MMASFLO). The efficacy of the newly developed ensemble classifier-based COVID-19 detection approaches will be tested through various benchmark datasets, and is tested with several existing approaches and conventional COVID-19 detection approaches.

A UNIFIED PARADIGM OF CLASSIFYING GI TRACT DISEASES IN ENDOSCOPY IMAGES USING MULTIPLE FEATURES FUSION

The automatic identification of Gastrointestinal (GI) tract diseases in endoscopy images has been associated with the domain of medical imaging and computer vision. Its classification has various challenges, including color, low contrast, lesion shape, and complex background. A Deep features-based method for the classification of gastrointestinal disease is implemented in this article. The method suggested involves four significant steps: preprocessing, extraction of handcrafted, and deep Convolutional neural network features (Deep CNN), selection of solid features, fusion, and classification. 3D-Median filtering in the preprocessing stage increases the lesion contrast. The second stage extracts the features centered on the shape. The extracted features are of three types: HOG features, ResNet50, and Xception. Principal Component Analysis (PCA) is chosen to select extracted features, combined by concatenating them in a single array. A support vector system eventually categorizes fused features into multiple classes. The Kvasir dataset is used for the proposed model. The SVM has outstanding efficiency reached 96.6 percent, showing the proposed system's robustness

Melanoma Skin Cancer Detection using Image Processing

Abstract: Early detection of melanoma skin cancer is crucial for effective treatment. Melanoma is considered the most dangerous form of skin cancer due to its higher likelihood of spreading to other parts of the body if not diagnosed and treated promptly. To address this challenge, non-invasive medical computer vision and medical image processing techniques have become increasingly important in clinical diagnosis of various diseases, including melanoma. These techniques offer automated image analysis tools that enable accurate and rapid evaluation of skin lesions.To assess the severity of the lesions, a total dermo copy score was calculated based on the extracted features. Finally, classification was carried out using a convolutional neural network (CNN) model. The results of the study demonstrated a high classification accuracy of 96.5%, indicating the effectiveness of the proposed approach in accurately identifying melanoma skin cancer.

Skin Cancer Detection using Image Processing

Abstract: Early detection of melanoma, the most dangerous form of skin cancer, is crucial for effective treatment. Melanoma has a higher propensity to metastasize to other parts of the body if not identified and treated promptly. In recent years, non-invasive medical computer vision and medical image processing techniques have emerged as valuable tools in clinical diagnosis for various diseases, including melanoma. These techniques enable automatic image analysis, facilitating accurate and efficient evaluation of skin lesions. The study follows a systematic approach, involving the collection of a dermo image database and preprocessing steps such as segmentation using thresholding. Statistical feature extraction methods, including gray level cooccurrence matrix (GLCM), glam asymmetry, border analysis, color assessment, and diameter calculation, are utilized. Principal component analysis (PCA) is employed for feature selection to reduce dimensionality. A total dermo copy score is calculated, followed by classification using a convolutional neural network (CNN). The results indicate an impressive classification accuracy of 92.1%.

Skin Cancer Detection using Image Processing

Abstract: Early detection of melanoma skin cancer is crucial for effective treatment. Among various types of skin cancer, melanoma is considered the most dangerous due to its high likelihood of spreading to other parts of the body if not diagnosed and treated promptly. In recent years, non-invasive medical computer vision and medical image processing techniques have gained significant importance in clinical diagnosis. These techniques offer automated image analysis tools that enable accurate and rapid evaluation of skin lesions. The study involves several steps, including the collection of a dermo image database, preprocessing, segmentation using thresholding, and extraction of statistical features such as gray level co-occurrence matrix (GLCM), asymmetry, border color, and diameter. Feature selection is performed using principal component analysis (PCA), followed by the calculation of a total dermo copy score. The final step involves classification using convolutional neural networks (CNN). The results of the study indicate an impressive classification accuracy of 96.5%.

Skin Melanoma Cancer Detection and Classification using Machine Learning

Early identification of melanoma skin cancer is essential for effective therapy. Melanoma is currently recognized as the most lethal type of skin cancer among all others due to its high propensity to metastasize to distant organs if undiagnosed and untreated. In the clinical diagnosis of many disorders, non-invasive medical computer vision or medical image processing is becoming increasingly important. These methods offer an automatic image analysis tool for a quick and accurate lesion evaluation. The steps in this study's methodology include gathering a database of dermoscopy images, pre-processing, segmentation using thresholding, statistical feature extraction using Gray Level Co-occurrence Matrix (GLCM), Asymmetry, Border, Color, Diameter, etc., and feature selection using Principal Component Analysis

Architecture of Deep Learning and Its Applications

Recently, Deep Learning (DL) has accomplished enormous prosperity in various areas, like natural language processing (NLP), image processing, different medical issues and computer vision. Both Machine Learning (ML) and DL as compared to traditional methods, can learn and make better and enhanced use of datasets for feature extraction. This paper is divided into three parts. The first part introduces a detailed information about different characteristics and learning types in terms of learning problems, hybrid learning problems, statistical inference and learning techniques; besides to an exhausted historical background about feature learning and DL. The second part is about the major architectures of DL with mathematical equations and clarified examples. These architectures include Autoencoders (AEs), Generative Adversarial Networks (GANs), Deep Belief Networks (DBNs), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs) and Recursive Neural Networks. The third part of this work represents an overview with detailed explanation about different applications and use-cases. Finally, the fourth part is about hardware/ software tools used with DL. Index Terms— Deep learning, Machine Learning, Neural Network, Network architecture.