Deep Learning In Computer Vision Research Articles

Computer Vision-Driven Movement Annotations to Advance fNIRS Pre-Processing Algorithms.

Functional near-infrared spectroscopy (fNIRS) is beneficial for studying brain activity in naturalistic settings due to its tolerance for movement. However, residual motion artifacts still compromise fNIRS data quality and might lead to spurious results. Although some motion artifact correction algorithms have been proposed in the literature, their development and accurate evaluation have been challenged by the lack of ground truth information. This is because ground truth information is time- and labor-intensive to manually annotate. This work investigates the feasibility and reliability of a deep learning computer vision (CV) approach for automated detection and annotation of head movements from video recordings. Fifteen participants performed controlled head movements across three main rotational axes (head up/down, head left/right, bend left/right) at two speeds (fast and slow), and in different ways (half, complete, repeated movement). Sessions were video recorded and head movement information was obtained using a CV approach. A 1-dimensional UNet model (1D-UNet) that detects head movements from head orientation signals extracted via a pre-trained model (SynergyNet) was implemented. Movements were manually annotated as a ground truth for model evaluation. The model's performance was evaluated using the Jaccard index. The model showed comparable performance between the training and test sets (J train = 0.954; J test = 0.865). Moreover, it demonstrated good and consistent performance at annotating movement across movement axes and speeds. However, performance varied by movement type, with the best results being obtained for repeated (J test = 0.941), followed by complete (J test = 0.872), and then half movements (J test = 0.826). This study suggests that the proposed CV approach provides accurate ground truth movement information. Future research can rely on this CV approach to evaluate and improve fNIRS motion artifact correction algorithms.

Research on the Knowledge Structure and Sustainable Development Pathways of Artificial Intelligence from the Perspective of Technological Science

Achieving significant breakthroughs in both the fundamental theories and technological applications of artificial intelligence is essential for fostering its long-term development. Under the guidance of Professor Qian Xuesen’s theory of technological science, exploring the internal mechanisms of knowledge evolution in artificial intelligence holds profound theoretical and practical significance for promoting sustainable technological advancement. This study draws on literature from the Web of Science (WOS) database and employs methods such as knowledge mapping, natural language processing, clustering analysis, and citation analysis to outline the knowledge structure of the field, clarify the trajectory of sustainable development, and trace the technological genealogy of VR/AR technologies.This study divides the knowledge structure within the field of technological science into “basic theoretical knowledge—applied basic knowledge—applied knowledge”, enriching Qian’s theory of technological science from within and providing strong intellectual support and technological pathways for sustainable technological development in practice. Artificial intelligence encompasses 10 distinct knowledge domains, among which machine learning and deep learning constitute the basic theoretical knowledge, data intelligence, computer vision, and swarm intelligence are the applied basic knowledge, and image processing and human-computer intelligence are the applied knowledge. The development of VR/AR technology has formed two main sustainable development paths: “machine learning—data intelligence—intelligent systems—human computer intelligence”, and “deep learning—computer vision—image processing”.

Development, deployment and scaling of operating room-ready artificial intelligence for real-time surgical decision support

Deep learning for computer vision can be leveraged for interpreting surgical scenes and providing surgeons with real-time guidance to avoid complications. However, neither generalizability nor scalability of computer-vision-based surgical guidance systems have been demonstrated, especially to geographic locations that lack hardware and infrastructure necessary for real-time inference. We propose a new equipment-agnostic framework for real-time use in operating suites. Using laparoscopic cholecystectomy and semantic segmentation models for predicting safe/dangerous (“Go”/”No-Go”) zones of dissection as an example use case, this study aimed to develop and test the performance of a novel data pipeline linked to a web-platform that enables real-time deployment from any edge device. To test this infrastructure and demonstrate its scalability and generalizability, lightweight U-Net and SegFormer models were trained on annotated frames from a large and diverse multicenter dataset from 136 institutions, and then tested on a separate prospectively collected dataset. A web-platform was created to enable real-time inference on any surgical video stream, and performance was tested on and optimized for a range of network speeds. The U-Net and SegFormer models respectively achieved mean Dice scores of 57% and 60%, precision 45% and 53%, and recall 82% and 75% for predicting the Go zone, and mean Dice scores of 76% and 76%, precision 68% and 68%, and recall 92% and 92% for predicting the No-Go zone. After optimization of the client-server interaction over the network, we deliver a prediction stream of at least 60 fps and with a maximum round-trip delay of 70 ms for speeds above 8 Mbps. Clinical deployment of machine learning models for surgical guidance is feasible and cost-effective using a generalizable, scalable and equipment-agnostic framework that lacks dependency on hardware with high computing performance or ultra-fast internet connection speed.

Clinical Application of Preliminary Breast Cancer Screening for Dense Breasts Using Real-Time AI-Powered Ultrasound with Deep-Learning Computer Vision

Clinical Application of Preliminary Breast Cancer Screening for Dense Breasts Using Real-Time AI-Powered Ultrasound with Deep-Learning Computer Vision

Deep learning for computer vision based activity recognition and fall detection of the elderly: a systematic review

As the proportion of elderly individuals in developed countries continues to rise globally, addressing their healthcare needs, particularly in preserving their autonomy, is of paramount concern. A growing body of research focuses on Ambient Assisted Living (AAL) systems, aimed at alleviating concerns related to the independent living of the elderly. This systematic review examines the literature pertaining to fall detection and Human Activity Recognition (HAR) for the elderly, two critical tasks for ensuring their safety when living alone. Specifically, this review emphasizes the utilization of Deep Learning (DL) approaches on computer vision data, reflecting current trends in the field. A comprehensive search yielded 2,616 works from five distinct sources, spanning the years 2019 to 2023 (inclusive). From this pool, 151 relevant works were selected for detailed analysis. The review scrutinizes the employed DL models, datasets, and hardware configurations, with particular emphasis on aspects such as privacy preservation and real-world deployment. The main contribution of this study lies in the synthesis of recent advancements in DL-based fall detection and HAR for the elderly, providing insights into the state-of-the-art techniques and identifying areas for further improvement. Given the increasing importance of AAL systems in enhancing the quality of life for the elderly, this review serves as a valuable resource for researchers, practitioners, and policymakers involved in developing and implementing such technologies.Graphical abstract

Hybrid Deep Transfer Learning and Feature Fusion Architecture for Diabetic Retinopathy Classification and Severity Grading

Diabetic Retinopathy (DR) is the leading cause of blindness among individuals with diabetes. Automating the diagnosis of DR has the potential to greatly benefit patients by enabling early detection and intervention, thus reducing the risk of blindness. The primary objective of this research is to develop a robust approach for the classification of DR and to analyze its severity grading. By achieving this, we aim to provide an effective tool for accurate diagnosis and assessment of DR, contributing to improved patient care and outcomes. The current literature review analysis reported the importance of deep learning in computer vision based applications. Moreover, plenty of pre-trained models are also present which can be used for classification tasks. Therefore, we present a hybrid DL classification approach by combining Inception V3, VGG-19 and ResNet 50. The proposed architecture uses transfer learning, and feature fusion model to produce the weighted feature vector which is used for classification analysis. The proposed approach is experimented on publicly available datasets APTOS-2019 and Messidor. The performance is measured in terms of accuracy, precision, recall and F1-score.

Automated cooling tower detection through deep learning for Legionnaires’ disease outbreak investigations: a model development and validation study

Cooling towers containing Legionella spp are a high-risk source of Legionnaires' disease outbreaks. Manually locating cooling towers from aerial imagery during outbreak investigations requires expertise, is labour intensive, and can be prone to errors. We aimed to train a deep learning computer vision model to automatically detect cooling towers that are aerially visible. Between Jan 1 and 31, 2021, we extracted satellite view images of Philadelphia (PN, USA) and New York state (NY, USA) from Google Maps and annotated cooling towers to create training datasets. We augmented training data with synthetic data and model-assisted labelling of additional cities. Using 2051 images containing 7292 cooling towers, we trained a two-stage model using YOLOv5, a model that detects objects in images, and EfficientNet-b5, a model that classifies images. We assessed the primary outcomes of sensitivity and positive predictive value (PPV) of the model against manual labelling on test datasets of 548 images, including from two cities not seen in training (Boston [MA, USA] and Athens [GA, USA]). We compared the search speed of the model with that of manual searching by four epidemiologists. The model identified visible cooling towers with 95·1% sensitivity (95% CI 94·0-96·1) and a PPV of 90·1% (95% CI 90·0-90·2) in New York City and Philadelphia. In Boston, sensitivity was 91·6% (89·2-93·7) and PPV was 80·8% (80·5-81·2). In Athens, sensitivity was 86·9% (75·8-94·2) and PPV was 85·5% (84·2-86·7). For an area of New York City encompassing 45 blocks (0·26 square miles), the model searched more than 600 times faster (7·6 s; 351 potential cooling towers identified) than did human investigators (mean 83·75 min [SD 29·5]; mean 310·8 cooling towers [42·2]). The model could be used to accelerate investigation and source control during outbreaks of Legionnaires' disease through the identification of cooling towers from aerial imagery, potentially preventing additional disease spread. The model has already been used by public health teams for outbreak investigations and to initialise cooling tower registries, which are considered best practice for preventing and responding to outbreaks of Legionnaires' disease. None.

Sign Language Recognition (SLR)

Abstract— Sign Language is mainly used by deaf (hard hearing) and dumb people to exchange information between their own community and with other people. It is a language where people use their hand gestures to communicate as they can’t speak or hear. Sign Language Recognition (SLR) deals with recognizing the hand gestures acquisition and continues till text or speech is generated for corresponding hand gestures. Here hand gestures for sign language can be classified as static and dynamic. However, static hand gesture recognition is simpler than dynamic hand gesture recognition, but both recognition is important to the human community. We can use Deep Learning Computer Vision to recognize the hand gestures by building Deep Neural Network architectures (Convolution Neural Network Architectures. Index Terms – HTML, CSS, Python, Pip, Mediapip, opencv, NumPy

A Systematic Literature Review on Leaf Disease Recognition Using Computer Vision and Deep Learning Approach

Background: Plant diseases affect agricultural output, quality and profitability, making them serious obstacles for agriculture. It is essential to detect diseases early in order to reduce losses while retaining sustainable practices. Plant disease detection has benefited greatly from the use of computer vision and deep learning in recent years because of their outstanding precision and computing capability. Objective: In this paper, we intend to investigate the role of deep learning in computer vision for plant disease detection while looking into how these techniques address complex disease identification problems. A variety of deep learning architectures were reviewed, and the contribution of frameworks such as Tensorflow, Keras, Caffe and PyTorch to the researchers' model construction was studied as well. Additionally, the usage of open repositories such as PlantVillage and Kaggle along with the customized datasets were discussed. Methods: We gathered the most recent developments in deep learning techniques for leaf disease detection through a systematic literature review of research papers published over the past decade, using reputable academic databases like Scopus and Web of Science, following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) method for guidance. Results: This study finds that researchers consistently enhance existing deep learning architectures to improve prediction accuracy in plant disease detection, often by introducing novel architectures and employing transfer learning methods. Frameworks like TensorFlow, Keras, Caffe, and PyTorch are widely favored for their efficiency in development. Additionally, most studies opt for public datasets such as PlantVillage, Kaggle, and ImageNet, which offer an abundance of labelled data for training and testing deep learning models. Conclusion: While no singular ‘best' model emerges, the adaptability of deep learning and computer vision demonstrates the dynamic nature of plant disease recognition area, and this paper provides a comprehensive overview of deep learning's transformative impact on plant disease recognition by bringing together information from different studies. Keywords: Deep learning, Computer vision, Plant disease, Systematic literature review

Opportunistic Screening of Chronic Liver Disease with Deep Learning Enhanced Echocardiography.

Chronic liver disease affects more than 1.5 billion adults worldwide, however the majority of cases are asymptomatic and undiagnosed. Echocardiography is broadly performed and visualizes the liver; but this information is not leveraged. To develop and evaluate a deep learning algorithm on echocardiography videos to enable opportunistic screening for chronic liver disease. Retrospective observational cohorts. Two large urban academic medical centers. Adult patients who received echocardiography and abdominal imaging (either abdominal ultrasound or abdominal magnetic resonance imaging) with ≤30 days between tests, between July 4, 2012, to June 4, 2022. Deep learning model predictions from a deep-learning computer vision pipeline that identifies subcostal view echocardiogram videos and detects the presence of cirrhosis or steatotic liver disease (SLD). Clinical diagnosis by paired abdominal ultrasound or magnetic resonance imaging (MRI). A total of 1,596,640 echocardiogram videos (66,922 studies from 24,276 patients) from Cedars-Sinai Medical Center (CSMC) were used to develop EchoNet-Liver, an automated pipeline that identifies high quality subcostal images from echocardiogram studies and detects the presence of cirrhosis or SLD. In the held-out CSMC test cohort, EchoNet-Liver was able to detect the presence of cirrhosis with an AUC of 0.837 (0.789 - 0.880) and SLD with an AUC of 0.799 (0.758 - 0.837). In a separate test cohort with paired abdominal MRIs, cirrhosis was detected with an AUC of 0.704 (0.689-0.718) and SLD was detected with an AUC of 0.726 (0.659-0.790). In an external test cohort of 106 patients (n = 5,280 videos), the model detected cirrhosis with an AUC of 0.830 (0.738 - 0.909) and SLD with an AUC of 0.768 (0.652 - 0.875). Deep learning assessment of clinical echocardiography enables opportunistic screening of SLD and cirrhosis. Application of this algorithm may identify patients who may benefit from further diagnostic testing and treatment for chronic liver disease.

Photography at a Standstill

What has replaced the still photograph are dynamic and distributed image-assemblages that unsettle received notions of space-time — no longer limited to traditional representation and not necessarily even visual. When it comes to computer vision for example, the descriptor photography seems largely redundant (despite deep learning computer vision systems being trained on large datasets of photographic images), and so too the tired metaphor of the eye that once supported its theories and practices. What is at stake here, as ever, is a kind of ‘seeing’ (if we continue to call it that) that makes clear what is visible, sensible and knowable, and crucially also what is not. We might call this seeing algorithmically, or seeing like a dataset, or perhaps even seeing like an infrastructure, comprised of fake images that render fake history. The logic of this invokes the complex notion of ‘image is dialectics at a standstill’, encapsulating a constellation of possible outcomes. To what extent is the radical potential that Benjamin once foresaw in montage applicable to image-based AI given that it seems less an instrument to imagine a qualitatively different future but simply more of the same. What can be seen is not so much representational nor photographic but latent traces of material relations and infrastructures that render historical experience in compromised form.

A framework for training larger networks for deep Reinforcement learning

The success of deep learning in computer vision and natural language processing communities can be attributed to the training of very deep neural networks with millions or billions of parameters, which can then be trained with massive amounts of data. However, a similar trend has largely eluded the training of deep reinforcement learning (RL) algorithms where larger networks do not lead to performance improvement. Previous work has shown that this is mostly due to instability during the training of deep RL agents when using larger networks. In this paper, we make an attempt to understand and address the training of larger networks for deep RL. We first show that naively increasing network capacity does not improve performance. Then, we propose a novel method that consists of (1) wider networks with DenseNet connection, (2) decoupling representation learning from the training of RL, and (3) a distributed training method to mitigate overfitting problems. Using this three-fold technique, we show that we can train very large networks that result in significant performance gains. We present several ablation studies to demonstrate the efficacy of the proposed method and some intuitive understanding of the reasons for performance gain. We show that our proposed method outperforms other baseline algorithms on several challenging locomotion tasks.

Do deep learning models accurately measure visual destination image? A comparison of a fine-tuned model to past work

The measurement of destination image from visual media such as online photography is of growing significance to destination managers and marketers who want to make better decisions and attract more visitors to their destination. However, there is no single approach with proven accuracy for doing this. We present a new approach where we fine-tune a deep learning model for a predetermined set of cognitive attributes of destination image. We then train state of the art neural networks using labelled tourist photography and test accuracy by comparing results with a ground truth dataset built for the same set of visual classes. Comparing our fine-tuned model against results which follow past approaches, we demonstrate that the pre-trained models without fine-tuning are not as accurate in capturing all of the destination image’s cognitive attributes. This is, to the best of our knowledge, the first deep learning computer vision model trained specifically to measure the cognitive component of destination image from photography and can act as a benchmark for future systems.

A video object detector with Spatio-Temporal Attention Module for micro UAV detection

Many deep neural network-based methods have recently been proposed for object detection due to the significant success of deep learning in computer vision. However, existing object detection methods typically extract the appearance features of objects from single image so that they usually suffer from poor performance in detecting micro Unmanned Aerial Vehicle (UAV), because micro UAV lacks of rich color, shape and texture information. To address this issue, we introduce the temporal information of objects from videos and develop a Spatio-Temporal Attention Module (STAM) to efficiently enhance feature map extraction for detecting micro UAV, and then integrate STAM into YOLOX to develop a video object detector for micro UAV. Meanwhile, we propose a lightweight Spatial Pyramid Pooling (SPP) module termed Group Simplified Spatial Pyramid Pooling-Fast with Cross Stage Partial (Group SimSPPFCSP) for the backbone’s final stage layer to efficiently and lightly extract more semantic information, and we propose a neck with rich propagation pathways (NRPP) to facilitate the effective propagation of spatial and temporal information across different levels. Furthermore, we propose two data augmentation operations including SeqMosaic and SeqMixUp, to augment video data for video object detection. Experimental results show that our model can achieve competitive precision (with 5.0 mAP and 8.1 mAPSmall improvement) while maintaining real-time inference speed (35.3 fps).

Sign Language Detection using CNN Model

Abstract: Sign Language is mainly used by deaf (hard hearing) and dumb people to exchange information between their own community and with other people. It is a language where people use their hand gestures to communicate as they can’t speak or hear. Sign Language Recognition (SLR) deals with recognizing the hand gestures acquisition and continues till text or speech is generated for corresponding hand gestures. Here hand gestures for sign language can be classified as static and dynamic. Deep Learning Computer Vision is used to recognize the hand gestures by building Deep Neural Network architectures (Convolution Neural Network Architectures) where the model will learn to recognize the hand gestures images over an epoch.

SAMCF: Adaptive global style alignment and multi-color spaces fusion for joint optic cup and disc segmentation

The optic cup (OC) and optic disc (OD) are two critical structures in retinal fundus images, and their relative positions and sizes are essential for effectively diagnosing eye diseases. With the success of deep learning in computer vision, deep learning-based segmentation models have been widely used for joint optic cup and disc segmentation. However, there are three prominent issues that impact the segmentation performance. First, significant differences among datasets collecting from various institutions, protocols, and devices lead to performance degradation of models. Second, we find that images with only RGB information struggle to counteract the interference caused by brightness variations, affecting color representation capability. Finally, existing methods typically ignored the edge perception, facing the challenges in obtaining clear and smooth edge segmentation results. To address these drawbacks, we propose a novel framework based on Style Alignment and Multi-Color Fusion (SAMCF) for joint OC and OD segmentation. Initially, we introduce a domain generalization method to generate uniformly styled images without damaged image content for mitigating domain shift issues. Next, based on multiple color spaces, we propose a feature extraction and fusion network aiming to handle brightness variation interference and improve color representation capability. Lastly, an edge aware loss is designed to generate fine edge segmentation results. Our experiments conducted on three public datasets, DGS, RIM, and REFUGE, demonstrate that our proposed SAMCF achieves superior performance to existing state-of-the-art methods. Moreover, SAMCF exhibits remarkable generalization ability across multiple retinal fundus image datasets, showcasing its outstanding generality.

Synthetic Data for Deep Learning in Computer Vision & Medical Imaging: A Means to Reduce Data Bias

Deep-learning (DL) performs well in computer-vision and medical-imaging automated decision-making applications. A bottleneck of DL stems from the large amount of labelled data required to train accurate models that generalise well. Data scarcity and imbalance are common problems in imaging applications that can lead DL models towards biased decision making. A solution to this problem is synthetic data. Synthetic data is an inexpensive substitute to real data for improved accuracy and generalisability of DL models. This survey reviews the recent methods published in relation to the creation and use of synthetic data for computer-vision and medical-imaging DL applications. The focus will be on applications that utilised synthetic data to improve DL models by either incorporating an increased diversity of data that is difficult to obtain in real life, or by reducing a bias caused by class imbalance. Computer-graphics software and generative networks are the most popular data generation techniques encountered in the literature. We highlight their suitability for typical computer-vision and medical-imaging applications, and present promising avenues for research to overcome their computational and theoretical limitations.

Deep Learning in Computer Vision: A Critical Review

Today, a machine learning tool has gained a lot of consideration as a data analysis and image processing tool, with encouraging results. Massive amounts of data are being generated in every field or any domain at a distressing rate. This review paper provides a brief overview of current technologies and a conceptual explanation of the development of computer vision with image processing and the usage of specific regions in their respective fields. Computer vision with deep learning allows researchers to investigate pictures and videos to attain essential facts, recognize facts on occasions, or provide explanations and exquisite patterns. It utilized the approach of diverse-variety utility areas with large-scale data mining. We employed a review to extract and synthesize the techniques and attributes that have been used in different applications of computer vision, like image classification, image localization, object detection, and many more, with their features. So according to computer vision tasks, there are Deep Belief Networks, Convolutional Neural Networks (CNN), Deep Boltzmann Machines (DBN), Recurrent Neural Networks (RNN), Stacked Denoising Auto Encoders, and Long-Short Term Memory (LSTM) algorithms that are some of the most important advanced machine learning (deep learning) algorithms. Lastly, a quick evaluation of future approaches to constructing advanced machine learning algorithms is provided for challenges and issues related to computer vision with improved performance.

Augmenting the Softmax with Additional Confidence Scores for Improved Selective Classification with Out-of-Distribution Data

Detecting out-of-distribution (OOD) data is a task that is receiving an increasing amount of research attention in the domain of deep learning for computer vision. However, the performance of detection methods is generally evaluated on the task in isolation, rather than also considering potential downstream tasks in tandem. In this work, we examine selective classification in the presence of OOD data (SCOD). That is to say, the motivation for detecting OOD samples is to reject them so their impact on the quality of predictions is reduced. We show under this task specification, that existing post-hoc methods perform quite differently compared to when evaluated only on OOD detection. This is because it is no longer an issue to conflate in-distribution (ID) data with OOD data if the ID data is going to be misclassified. However, the conflation within ID data of correct and incorrect predictions becomes undesirable. We also propose a novel method for SCOD, Softmax Information Retaining Combination (SIRC), that augments a softmax-based confidence score with a secondary class-agnostic feature-based score. Thus, the ability to identify OOD samples is improved without sacrificing separation between correct and incorrect ID predictions. Experiments on a wide variety of ImageNet-scale datasets and convolutional neural network architectures show that SIRC is able to consistently match or outperform the baseline for SCOD, whilst existing OOD detection methods fail to do so. Interestingly, we find that the secondary scores investigated for SIRC do not consistently improve performance on all tested OOD datasets. To address this issue, we further extend SIRC to incorporate multiple secondary scores (SIRC+). This further improves SCOD performance, both generally, and in terms of consistency over diverse distribution shifts. Code is available at https://github.com/Guoxoug/SIRC.

CHD-CXR: a de-identified publicly available dataset of chest x-ray for congenital heart disease.

Congenital heart disease is a prevalent birth defect, accounting for approximately one-third of major birth defects. The challenge lies in early detection, especially in underdeveloped medical regions where a shortage of specialized physicians often leads to oversight. While standardized chest x-rays can assist in diagnosis and treatment, their effectiveness is limited by subtle cardiac manifestations. However, the emergence of deep learning in computer vision has paved the way for detecting subtle changes in chest x-rays, such as lung vessel density, enabling the detection of congenital heart disease in children. This highlights the need for further investigation. The lack of expert-annotated, high-quality medical image datasets hinders the progress of medical image artificial intelligence. In response, we have released a dataset containing 828 DICOM chest x-ray files from children with diagnosed congenital heart disease, alongside corresponding cardiac ultrasound reports. This dataset emphasizes complex structural characteristics, facilitating the transition from machine learning to machine teaching in deep learning. To ascertain the dataset's applicability, we trained a preliminary model and achieved an area under the receiver operating characteristic curve (ROC 0.85). We provide detailed introductions and publicly available datasets at: https://www.kaggle.com/competitions/congenital-heart-disease.