Computer Deep Learning Research Articles

A resource-aware workload scheduling method for unbalanced GEMMs on GPUs

Abstract GEMM (General Matrix Multiplication) serves as a fundamental operator for deep learning computations. Especially in attention-based deep learning models, such as Bert, GPT, and SAM, the sizes of matrices involved in GEMMs exhibit an unbalanced distribution due to the variable input, resulting in the low utilization of hardware resources. To address the issue, this paper proposes inserting a novel GEMM processing layer into the deep learning inference stack and using an adaptive load balancing method to partition and schedule GEMM computation tasks. The method is implemented with hardware runtime resource information, such as the occupancy of computing units, etc. Experiment results show the remarkable performance of our method in unbalanced input GEMM scenarios, achieving an average performance improvement of 2.3x. The method also performs well in attention-based models (GPT-2 and SAM), achieving an average inference speed improvement of 1.1x. These findings highlight the effectiveness of resource-aware algorithm optimization, especially for computation task scheduling.

Computer Vision-Driven Movement Annotations to Advance fNIRS Pre-Processing Algorithms.

Functional near-infrared spectroscopy (fNIRS) is beneficial for studying brain activity in naturalistic settings due to its tolerance for movement. However, residual motion artifacts still compromise fNIRS data quality and might lead to spurious results. Although some motion artifact correction algorithms have been proposed in the literature, their development and accurate evaluation have been challenged by the lack of ground truth information. This is because ground truth information is time- and labor-intensive to manually annotate. This work investigates the feasibility and reliability of a deep learning computer vision (CV) approach for automated detection and annotation of head movements from video recordings. Fifteen participants performed controlled head movements across three main rotational axes (head up/down, head left/right, bend left/right) at two speeds (fast and slow), and in different ways (half, complete, repeated movement). Sessions were video recorded and head movement information was obtained using a CV approach. A 1-dimensional UNet model (1D-UNet) that detects head movements from head orientation signals extracted via a pre-trained model (SynergyNet) was implemented. Movements were manually annotated as a ground truth for model evaluation. The model's performance was evaluated using the Jaccard index. The model showed comparable performance between the training and test sets (J train = 0.954; J test = 0.865). Moreover, it demonstrated good and consistent performance at annotating movement across movement axes and speeds. However, performance varied by movement type, with the best results being obtained for repeated (J test = 0.941), followed by complete (J test = 0.872), and then half movements (J test = 0.826). This study suggests that the proposed CV approach provides accurate ground truth movement information. Future research can rely on this CV approach to evaluate and improve fNIRS motion artifact correction algorithms.

Research on the Knowledge Structure and Sustainable Development Pathways of Artificial Intelligence from the Perspective of Technological Science

Achieving significant breakthroughs in both the fundamental theories and technological applications of artificial intelligence is essential for fostering its long-term development. Under the guidance of Professor Qian Xuesen’s theory of technological science, exploring the internal mechanisms of knowledge evolution in artificial intelligence holds profound theoretical and practical significance for promoting sustainable technological advancement. This study draws on literature from the Web of Science (WOS) database and employs methods such as knowledge mapping, natural language processing, clustering analysis, and citation analysis to outline the knowledge structure of the field, clarify the trajectory of sustainable development, and trace the technological genealogy of VR/AR technologies.This study divides the knowledge structure within the field of technological science into “basic theoretical knowledge—applied basic knowledge—applied knowledge”, enriching Qian’s theory of technological science from within and providing strong intellectual support and technological pathways for sustainable technological development in practice. Artificial intelligence encompasses 10 distinct knowledge domains, among which machine learning and deep learning constitute the basic theoretical knowledge, data intelligence, computer vision, and swarm intelligence are the applied basic knowledge, and image processing and human-computer intelligence are the applied knowledge. The development of VR/AR technology has formed two main sustainable development paths: “machine learning—data intelligence—intelligent systems—human computer intelligence”, and “deep learning—computer vision—image processing”.

Edge-device collaborative computing for multi-view classification

Motivated by the proliferation of Internet-of-Thing (IoT) devices and the rapid advances in the field of deep learning, there is a growing interest in pushing deep learning computations, conventionally handled by the cloud, to the edge of the network to deliver faster responses to end users, reduce bandwidth consumption to the cloud, and address privacy concerns. However, to fully realize deep learning at the edge, two main challenges still need to be addressed: (i) how to meet the high resource requirements of deep learning on resource-constrained devices, and (ii) how to leverage the availability of multiple streams of spatially correlated data, to increase the effectiveness of deep learning and improve application-level performance. To address the above challenges, we explore collaborative inference at the edge, in which edge nodes and end devices share correlated data and the inference computational burden by leveraging different ways to split computation and fuse data. Besides traditional centralized and distributed schemes for edge-end device collaborative inference, we introduce selective schemes that decrease bandwidth resource consumption by effectively reducing data redundancy. As a reference scenario, we focus on multi-view classification in a networked system in which sensing nodes can capture overlapping fields of view. The proposed schemes are compared in terms of accuracy, computational expenditure at the nodes, communication overhead, inference latency, robustness, and noise sensitivity. Experimental results highlight that selective collaborative schemes can achieve different trade-offs between the above performance metrics, with some of them bringing substantial communication savings (from 18% to 74% of the transmitted data with respect to centralized inference) while still keeping the inference accuracy well above 90%.

Clinical Application of Preliminary Breast Cancer Screening for Dense Breasts Using Real-Time AI-Powered Ultrasound with Deep-Learning Computer Vision

Clinical Application of Preliminary Breast Cancer Screening for Dense Breasts Using Real-Time AI-Powered Ultrasound with Deep-Learning Computer Vision

Automated cooling tower detection through deep learning for Legionnaires’ disease outbreak investigations: a model development and validation study

Cooling towers containing Legionella spp are a high-risk source of Legionnaires' disease outbreaks. Manually locating cooling towers from aerial imagery during outbreak investigations requires expertise, is labour intensive, and can be prone to errors. We aimed to train a deep learning computer vision model to automatically detect cooling towers that are aerially visible. Between Jan 1 and 31, 2021, we extracted satellite view images of Philadelphia (PN, USA) and New York state (NY, USA) from Google Maps and annotated cooling towers to create training datasets. We augmented training data with synthetic data and model-assisted labelling of additional cities. Using 2051 images containing 7292 cooling towers, we trained a two-stage model using YOLOv5, a model that detects objects in images, and EfficientNet-b5, a model that classifies images. We assessed the primary outcomes of sensitivity and positive predictive value (PPV) of the model against manual labelling on test datasets of 548 images, including from two cities not seen in training (Boston [MA, USA] and Athens [GA, USA]). We compared the search speed of the model with that of manual searching by four epidemiologists. The model identified visible cooling towers with 95·1% sensitivity (95% CI 94·0-96·1) and a PPV of 90·1% (95% CI 90·0-90·2) in New York City and Philadelphia. In Boston, sensitivity was 91·6% (89·2-93·7) and PPV was 80·8% (80·5-81·2). In Athens, sensitivity was 86·9% (75·8-94·2) and PPV was 85·5% (84·2-86·7). For an area of New York City encompassing 45 blocks (0·26 square miles), the model searched more than 600 times faster (7·6 s; 351 potential cooling towers identified) than did human investigators (mean 83·75 min [SD 29·5]; mean 310·8 cooling towers [42·2]). The model could be used to accelerate investigation and source control during outbreaks of Legionnaires' disease through the identification of cooling towers from aerial imagery, potentially preventing additional disease spread. The model has already been used by public health teams for outbreak investigations and to initialise cooling tower registries, which are considered best practice for preventing and responding to outbreaks of Legionnaires' disease. None.

Sign Language Recognition (SLR)

Abstract— Sign Language is mainly used by deaf (hard hearing) and dumb people to exchange information between their own community and with other people. It is a language where people use their hand gestures to communicate as they can’t speak or hear. Sign Language Recognition (SLR) deals with recognizing the hand gestures acquisition and continues till text or speech is generated for corresponding hand gestures. Here hand gestures for sign language can be classified as static and dynamic. However, static hand gesture recognition is simpler than dynamic hand gesture recognition, but both recognition is important to the human community. We can use Deep Learning Computer Vision to recognize the hand gestures by building Deep Neural Network architectures (Convolution Neural Network Architectures. Index Terms – HTML, CSS, Python, Pip, Mediapip, opencv, NumPy

Vehicle Classification and Counting for Traffic Analysis based on Single-stage YOLOv8 Model

Intelligent traffic systems are emerging and becoming part of smart city infrastructure that requires computer vision applications to provide traffic information like vehicle classification and counting in real-time. Vehicle counting helps detect heavy traffic on roads, and vehicle classification helps enforce further processes like speed estimation to enforce speed limit laws based on the vehicle class. Deep learning computer vision-based systems provide automatic feature extractions that are robust to changes in lighting, shadows, and occlusions. This paper proposes a software solution for a real-time traffic monitoring system based on a cutting-edge single-stage deep learning model through the state-of-the-art YOLOv8 algorithm. YOLOv8 is the most recent model of the YOLO family, which provides object detection and classification through its CNN architecture. The proposed work detects vehicles and counts them based on their class. The four common vehicle classes are sedan cars, buses, trucks, and motorcycles, and a counter for each class is displayed on the system’s output screen in real-time and recorded in a log file. The results of the proposed system running on the Nvidia GTX 1070 GPU show an average accuracy of 96.58% with an average error of 3.42% for vehicle detection and an average accuracy of 97.54% with a 2.46% average error for vehicle counting. For vehicle classification, the results for the four vehicle classes (car, bus, truck, and motorcycle) show an accuracy of (94.7%, 94.7%, 96.2%, 99.7%), precision (95%, 100%, 81.4%, 100%), recall (97.9%, 36.3%, 100%, 66.6%), and the f1-score (96.3%, 53.2%, 89.7%, 79.9%), respectively. Index Terms— Computer Vision, Deep Learning, Vehicle Detection, Traffic Analysis, YOLO.

Opportunistic Screening of Chronic Liver Disease with Deep Learning Enhanced Echocardiography.

Chronic liver disease affects more than 1.5 billion adults worldwide, however the majority of cases are asymptomatic and undiagnosed. Echocardiography is broadly performed and visualizes the liver; but this information is not leveraged. To develop and evaluate a deep learning algorithm on echocardiography videos to enable opportunistic screening for chronic liver disease. Retrospective observational cohorts. Two large urban academic medical centers. Adult patients who received echocardiography and abdominal imaging (either abdominal ultrasound or abdominal magnetic resonance imaging) with ≤30 days between tests, between July 4, 2012, to June 4, 2022. Deep learning model predictions from a deep-learning computer vision pipeline that identifies subcostal view echocardiogram videos and detects the presence of cirrhosis or steatotic liver disease (SLD). Clinical diagnosis by paired abdominal ultrasound or magnetic resonance imaging (MRI). A total of 1,596,640 echocardiogram videos (66,922 studies from 24,276 patients) from Cedars-Sinai Medical Center (CSMC) were used to develop EchoNet-Liver, an automated pipeline that identifies high quality subcostal images from echocardiogram studies and detects the presence of cirrhosis or SLD. In the held-out CSMC test cohort, EchoNet-Liver was able to detect the presence of cirrhosis with an AUC of 0.837 (0.789 - 0.880) and SLD with an AUC of 0.799 (0.758 - 0.837). In a separate test cohort with paired abdominal MRIs, cirrhosis was detected with an AUC of 0.704 (0.689-0.718) and SLD was detected with an AUC of 0.726 (0.659-0.790). In an external test cohort of 106 patients (n = 5,280 videos), the model detected cirrhosis with an AUC of 0.830 (0.738 - 0.909) and SLD with an AUC of 0.768 (0.652 - 0.875). Deep learning assessment of clinical echocardiography enables opportunistic screening of SLD and cirrhosis. Application of this algorithm may identify patients who may benefit from further diagnostic testing and treatment for chronic liver disease.

Photography at a Standstill

What has replaced the still photograph are dynamic and distributed image-assemblages that unsettle received notions of space-time — no longer limited to traditional representation and not necessarily even visual. When it comes to computer vision for example, the descriptor photography seems largely redundant (despite deep learning computer vision systems being trained on large datasets of photographic images), and so too the tired metaphor of the eye that once supported its theories and practices. What is at stake here, as ever, is a kind of ‘seeing’ (if we continue to call it that) that makes clear what is visible, sensible and knowable, and crucially also what is not. We might call this seeing algorithmically, or seeing like a dataset, or perhaps even seeing like an infrastructure, comprised of fake images that render fake history. The logic of this invokes the complex notion of ‘image is dialectics at a standstill’, encapsulating a constellation of possible outcomes. To what extent is the radical potential that Benjamin once foresaw in montage applicable to image-based AI given that it seems less an instrument to imagine a qualitatively different future but simply more of the same. What can be seen is not so much representational nor photographic but latent traces of material relations and infrastructures that render historical experience in compromised form.

Do deep learning models accurately measure visual destination image? A comparison of a fine-tuned model to past work

The measurement of destination image from visual media such as online photography is of growing significance to destination managers and marketers who want to make better decisions and attract more visitors to their destination. However, there is no single approach with proven accuracy for doing this. We present a new approach where we fine-tune a deep learning model for a predetermined set of cognitive attributes of destination image. We then train state of the art neural networks using labelled tourist photography and test accuracy by comparing results with a ground truth dataset built for the same set of visual classes. Comparing our fine-tuned model against results which follow past approaches, we demonstrate that the pre-trained models without fine-tuning are not as accurate in capturing all of the destination image’s cognitive attributes. This is, to the best of our knowledge, the first deep learning computer vision model trained specifically to measure the cognitive component of destination image from photography and can act as a benchmark for future systems.

Sign Language Detection using CNN Model

Abstract: Sign Language is mainly used by deaf (hard hearing) and dumb people to exchange information between their own community and with other people. It is a language where people use their hand gestures to communicate as they can’t speak or hear. Sign Language Recognition (SLR) deals with recognizing the hand gestures acquisition and continues till text or speech is generated for corresponding hand gestures. Here hand gestures for sign language can be classified as static and dynamic. Deep Learning Computer Vision is used to recognize the hand gestures by building Deep Neural Network architectures (Convolution Neural Network Architectures) where the model will learn to recognize the hand gestures images over an epoch.

A Model for Estimating the Economic Costs of Computer Vision Systems That Use Deep Learning

Deep learning, the most important subfield of machine learning and artificial intelligence (AI) over the last decade, is considered one of the fundamental technologies underpinning the Fourth Industrial Revolution. But despite its record-breaking history, deep learning’s enormous appetite for compute and data means that sometimes it can be too costly to practically use. In this paper, we connect technical insights from deep learning scaling laws and transfer learning with the economics of IT to propose a framework for estimating the cost of deep learning computer vision systems to achieve a desired level of accuracy. Our tool can be of practical use to AI practitioners in industry or academia to guide investment decisions.

Bearing Fault Diagnosis under Varying Work Conditions Based on Synchrosqueezing Transform, Random Projection, and Convolutional Neural Networks

Bearings are critical components in rotating machinery, and their failure can lead to costly repairs and downtime. To prevent such failures, it is important to detect and diagnose bearing faults early. In recent years, deep-learning techniques have shown promise for detecting and diagnosing bearing faults automatically. While these algorithms can all achieve diagnostic accuracy of over 90%, their generalizability and robustness in complex, extreme variable loading conditions have not been thoroughly validated. In this paper, a feature extraction method based on Synchro-squeezing Wavelet Transform (SSWT), Random projection (RP), and deep learning (DL) is presented. To fulfill the data requirements of neural networks, data augmentation is initially utilized to augment the size of the original data. Subsequently, the SSWT technique is employed to convert the signals from the Time domain to the Time-Frequency domain, resulting in the conversion of the 1-D signal to a 2-D feature image. To decrease the complexity of deep learning computation, data preprocessing involves utilizing Random projection to reduce feature dimensionality. The final step involves constructing a Convolutional Neural Network (CNN) model that can identify fault features from the obtained Time-Frequency images and perform accurate fault classification. By utilizing the CWRU and IMS datasets to evaluate the method, the study demonstrates that the suggested approach outperforms advanced techniques in terms of both diagnostic accuracy and robustness.

UC-stack: a deep learning computer automatic detection system for diabetic retinopathy classification.

Object. The existing diagnostic paradigm for diabetic retinopathy (DR) greatly relies on subjective assessments by medical practitioners utilizing optical imaging, introducing susceptibility to individual interpretation. This work presents a novel system for the early detection and grading of DR, providing an automated alternative to the manual examination.Approach. First, we use advanced image preprocessing techniques, specifically contrast-limited adaptive histogram equalization and Gaussian filtering, with the goal of enhancing image quality and module learning capabilities. Second, a deep learning-based automatic detection system is developed. The system consists of a feature segmentation module, a deep learning feature extraction module, and an ensemble classification module. The feature segmentation module accomplishes vascular segmentation, the deep learning feature extraction module realizes the global feature and local feature extraction of retinopathy images, and the ensemble module performs the diagnosis and classification of DR for the extracted features. Lastly, nine performance evaluation metrics are applied to assess the quality of the model's performance.Main results. Extensive experiments are conducted on four retinal image databases (APTOS 2019, Messidor, DDR, and EyePACS). The proposed method demonstrates promising performance in the binary and multi-classification tasks for DR, evaluated through nine indicators, including AUC and quadratic weighted Kappa score. The system shows the best performance in the comparison of three segmentation methods, two convolutional neural network architecture models, four Swin Transformer structures, and the latest literature methods.Significance. In contrast to existing methods, our system demonstrates superior performance across multiple indicators, enabling accurate screening of DR and providing valuable support to clinicians in the diagnostic process. Our automated approach minimizes the reliance on subjective assessments, contributing to more consistent and reliable DR evaluations.

Evaluation of deep learning computer vision for water level measurements in rivers

Image-based gauging stations offer the potential for substantial enhancement in the monitoring networks of river water levels. Nonetheless, the majority of camera gauges fall short in delivering reliable and precise measurements because of the fluctuating appearance of water in the rivers over the course of the year. In this study, we introduce a method for measuring water levels in rivers using both the traditional continuous image subtraction (CIS) approach and a SegNet neural network based on deep learning computer vision. The historical images collected from on-site investigations were employed to train three neural networks (SegNet, U-Net, and FCN) in order to evaluate their effectiveness, overall performance, and reliability. The research findings demonstrated that the SegNet neural network outperformed the CIS method in accurately measuring water levels. The root mean square error (RMSE) between the water level measurements obtained by the SegNet neural network and the gauge station's readings ranged from 0.013 m to 0.066 m, with a high correlation coefficient of 0.998. Furthermore, the study revealed that the performance of the SegNet neural network in analyzing water levels in rivers improved with the inclusion of a larger number of images, diverse image categories, and higher image resolutions in the training dataset. These promising results emphasize the potential of deep learning computer vision technology, particularly the SegNet neural network, to enhance water level measurement in rivers. Notably, the quality and diversity of the training dataset play a crucial role in optimizing the network's performance. Overall, the application of this advanced technology holds great promise for advancing water level monitoring and management in river systems.

Detection of Bowel and Mesenteric Injuries Using Deep Learning Computer Vision Models

Background/Objective:While only seen in 1-5% of patients who undergo a CT (computed tomography) scan, blunt bowel and mesenteric injuries (BMI) are associated with significantly increased morbidity and mortality. A significant cause of the increased morbidity of BMI is due to the difficulty of diagnosis from clinical and imaging information which leads to delay in diagnosis. Accurate and timely diagnosis is vital to reduce the morbidity of BMI.
 Methods:For this project, our primary objective is to create a binary prediction model that determines if a patient has BMI based on their abdominal CT scans. Due to the importance of the early and definitive diagnosis of BMI in trauma patients, an extension of this project will seek to introduce explainability into the model to highlight which features on the CT scan caused the model to make its prediction. The patients with BMI were sourced from a trauma registry that recorded trauma cases from IU Health with relevant diagnosis codes. The images from our search will be reduced to the relevant slices for diagnosis of BMI and then used to train an ML model to makea yes/no prediction from the image. Once the model is trained, testing data will be evaluated on the model and the gradient vectors from the model during inference will be used to create a heatmap with GRAD-CAM that illustrates what portions of the image were relevant for the decision made by the algorithm.
 Future Directions:Using the collected abdominal CTs, we can train our machine learning pipeline to detect BMI. Based on the performance of the model, we will determine if we need to collect more data. Then, we can evaluate the explainability of the model using GRAD-CAM and compareperformance of the ML model to the performance of expert and trainee radiologists.

An AI-assisted integrated, scalable, single-cell phenomic-transcriptomic platform to elucidate intratumor heterogeneity against immune response.

We present a novel framework combining single-cell phenotypic data with single-cell transcriptomic analysis to identify factors underpinning heterogeneity in antitumor immune response. We developed a pairwise, tumor-immune discretized interaction assay between natural killer (NK-92MI) cells and patient-derived head and neck squamous cell carcinoma (HNSCC) cell lines on a microfluidic cell-trapping platform. Furthermore we generated a deep-learning computer vision algorithm that is capable of automating the acquisition and analysis of a large, live-cell imaging data set (>1 million) of paired tumor-immune interactions spanning a time course of 24 h across multiple HNSCC lines (n = 10). Finally, we combined the response data measured by Kaplan-Meier survival analysis against NK-mediated killing with downstream single-cell transcriptomic analysis to interrogate molecular signatures associated with NK-effector response. As proof-of-concept for the proposed framework, we efficiently identified MHC class I-driven cytotoxic resistance as a key mechanism for immune evasion in nonresponders, while enhanced expression of cell adhesion molecules was found to be correlated with sensitivity against NK-mediated cytotoxicity. We conclude that this integrated, data-driven phenotypic approach holds tremendous promise in advancing the rapid identification of new mechanisms and therapeutic targets related to immune evasion and response.

A Survey Study of the Deep Learning for Convolutional Neural Network Architecture

The deep learning (DL) computer paradigm has been the industry standard for machine learning (ML) during the past few years. It has gradually become the most widely used computational technique in machine learning. One of the benefits of DL is its ability to learn massive amounts of data. Deep learning has seen tremendous growth in the last several years and has been successfully used for many traditional applications. More importantly, DL has outperformed popular machine learning algorithms in several domains, such as cybersecurity, bioinformatics, robotics, etc. The field remains mostly uneducated although it has been contributed to several works reviewing the State-ofthe- Art on DL, each of which only covered a specific aspect of the field. We thus propose a more holistic approach to this contribution, providing a more suitable basis upon which to construct a thorough understanding of DL. Concerning the most significant DL features, including the most recent advancements in the field, this evaluation specifically aims to provide a more thorough survey. This study specifically describes the kinds of DL networks and techniques, as well as their significance. The most common type of DL network, convolutional neural networks (CNNs), is then presented, and the evolution of it.

Sparse Non-Local CRF With Applications.

CRFs model spatial coherence in classical and deep learning computer vision. The most common CRF is called pairwise, as it connects pixel pairs. There are two types of pairwise CRF: sparse and dense. A sparse CRF connects the nearby pixels, leading to a linear number of connections in the image size. A dense CRF connects all pixel pairs, leading to a quadratic number of connections. While dense CRF is a more general model, it is much less efficient than sparse CRF. In fact, only Gaussian edge dense CRF is used in practice, and even then with approximations. We propose a new pairwise CRF, which we call sparse non-local CRF. Like dense CRF, it has non-local connections, and, therefore, it is more general than sparse CRF. Like sparse CRF, the number of connections is linear, and, therefore, our model is efficient. Besides efficiency, another advantage is that our edge weights are unrestricted. We show that our sparse non-local CRF models properties similar to that of Gaussian dense CRF. We also discuss connections to other CRF models. We demonstrate the usefulness of our model on classical and deep learning applications, for two and multiple labels.