Deep Learning Tasks Research Articles

A Survey on AI-Driven Mouse Behavior Analysis Applications and Solutions

The physiological similarities between mice and humans make them vital animal models in biological and medical research. This paper explores the application of artificial intelligence (AI) in analyzing mice behavior, emphasizing AI’s potential to identify and classify these behaviors. Traditional methods struggle to capture subtle behavioral features, whereas AI can automatically extract quantitative features from large datasets. Consequently, this study aims to leverage AI to enhance the efficiency and accuracy of mice behavior analysis. The paper reviews various applications of mice behavior analysis, categorizes deep learning tasks based on an AI pyramid, and summarizes AI methods for addressing these tasks. The findings indicate that AI technologies are increasingly applied in mice behavior analysis, including disease detection, assessment of external stimuli effects, social behavior analysis, and neurobehavioral assessment. The selection of AI methods is crucial and must align with specific applications. Despite AI’s promising potential in mice behavior analysis, challenges such as insufficient datasets and benchmarks remain. Furthermore, there is a need for a more integrated AI platform, along with standardized datasets and benchmarks, to support these analyses and further advance AI-driven mice behavior analysis.

Image-Based Generative Artificial Intelligence in Radiology: Comprehensive Updates.

Generative artificial intelligence (AI) has been applied to images for image quality enhancement, domain transfer, and augmentation of training data for AI modeling in various medical fields. Image-generative AI can produce large amounts of unannotated imaging data, which facilitates multiple downstream deep-learning tasks. However, their evaluation methods and clinical utility have not been thoroughly reviewed. This article summarizes commonly used generative adversarial networks and diffusion models. In addition, it summarizes their utility in clinical tasks in the field of radiology, such as direct image utilization, lesion detection, segmentation, and diagnosis. This article aims to guide readers regarding radiology practice and research using image-generative AI by 1) reviewing basic theories of image-generative AI, 2) discussing the methods used to evaluate the generated images, 3) outlining the clinical and research utility of generated images, and 4) discussing the issue of hallucinations.

Diffraction‐Driven Parallel Convolution Processing with Integrated Photonics

AbstractTraditional electronic processors often struggle with bandwidth limitations and high power consumption when executing extensive linear operations for deep learning tasks. Optical computing has emerged as a promising alternative, offering parallel and energy‐efficient computation capabilities. Yet, the development of high‐density optical computing architectures on integrated photonic platforms remains limited, hindered by constraints in neuron scalability and control engineering complexities. Addressing these challenges, this work presents a diffraction‐driven multi‐kernel optical convolution unit (MOCU) that enables on‐chip parallel convolution processing. By utilizing cascaded silica 1D metalines as pre‐trained large‐scale weights and employing spatial multiplexing at the output, MOCU allows simultaneous passive computation of diverse convolutions within a single unit. This architecture facilitates the construction of optical convolutional neural networks (OCNNs), enabling efficient machine vision processing with a streamlined design. To mitigate errors in MOCU‐embedded OCNNs, a lightweight electronic neural network operates concurrently to calibrate systematic deviations via a low‐rank adaptation (LoRA) algorithm, with minimal overhead. The fabricated MOCU chip demonstrates the highest independent 8‐kernel convolutions in parallel, each with a kernel size and occupying just 0.06 . This architecture effectively merges photonic and electronic technologies, offering a scalable design pathway for energy‐efficient, high‐density deep learning hardware.

Appearance-Based Gaze Estimation as a Benchmark for Eye Image Data Generation Methods

Data augmentation is commonly utilized to increase the size and diversity of training sets for deep learning tasks. In this study, we propose a novel application of an existing image generation approach in the domain of realistic eye images that leverages data collected from 40 subjects. This hybrid method combines the benefits of precise control over the image content provided by 3D rendering, while introducing the previously lacking photorealism and diversity into synthetic images through neural style transfer. We prove its general efficacy as a data augmentation tool for appearance-based gaze estimation when generated data are mixed with a sparse train set of real images. It improved the results for 39 out of 40 subjects, with an 11.22% mean and a 19.75% maximum decrease in gaze estimation error, achieving similar metrics for train and held-out subjects. We release our data repository of eye images with gaze labels used in this work for public access.

Feature Selection as Deep Sequential Generative Learning

Feature selection aims to identify the most pattern-discriminative feature subset. In prior literature, filter (e.g., backward elimination) and embedded (e.g., LASSO) methods have hyperparameters (e.g., top- k , score thresholding) and tie to specific models, thus, hard to generalize; wrapper methods search a feature subset in a huge discrete space and is computationally costly. To transform the way of feature selection, we regard a selected feature subset as a selection decision token sequence and reformulate feature selection as a deep sequential generative learning task that distills feature knowledge and generates decision sequences. Our method includes three steps: (1) We develop a deep variational transformer model over a joint of sequential reconstruction, variational, and performance evaluator losses. Our model can distill feature selection knowledge and learn a continuous embedding space to map feature selection decision sequences into embedding vectors associated with utility scores. (2) We leverage the trained feature subset utility evaluator as a gradient provider to guide the identification of the optimal feature subset embedding; (3) We decode the optimal feature subset embedding to autoregressively generate the best feature selection decision sequence with autostop. Extensive experimental results show this generative perspective is effective and generic, without large discrete search space and expert-specific hyperparameters. The code is available at http://tinyurl.com/FSDSGL .

A vision transformer-based deep transfer learning nomogram for predicting lymph node metastasis in lung adenocarcinoma.

Lymph node metastasis (LNM) plays a crucial role in the management of lung cancer; however, the ability of chest computed tomography (CT) imaging to detect LNM status is limited. This study aimed to develop and validate a vision transformer-based deep transfer learning nomogram for predicting LNM in lung adenocarcinoma patients using preoperative unenhanced chest CT imaging. This study included 528 patients with lung adenocarcinoma who were randomly divided into training and validation cohorts at a 7:3 ratio. The pretrained vision transformer (ViT) was utilized to extract deep transfer learning (DTL) feature, and logistic regression was employed to construct a ViT-based DTL model. Subsequently, the model was compared with six classical convolutional neural network (CNN) models. Finally, the ViT-based DTL signature was combined with independent clinical predictors to construct a ViT-based deep transfer learning nomogram (DTLN). The ViT-based DTL model showed good performance, with an area under the curve (AUC) of 0.821 (95% CI, 0.775-0.867) in the training cohort and 0.825 (95% CI, 0.758-0.891) in the validation cohort. The ViT-based DTL model demonstrated comparable performance to classical CNN models in predicting LNM, and the ViT-based DTL signature was then used to construct ViT-based DTLN with independent clinical predictors such as tumor maximum diameter, location, and density. The DTLN achieved the best predictive performance, with AUCs of 0.865 (95% CI, 0.827-0.903) and 0.894 (95% CI, 0845-0942), respectively, surpassing both the clinical factor model and the ViT-based DTL model (p<0.001). This study developed a new DTL model based on ViT to predict LNM status in lung adenocarcinoma patients and revealed that the performance of the ViT-based DTL model was comparable to that of classical CNN models, confirming that ViT was viable for deep learning tasks involving medical images. The ViT-based DTLN performed exceptionally well and can assist clinicians and radiologists in making accurate judgments and formulating appropriate treatment plans.

A comparative evaluation of deep learning approaches for ophthalmology

There is a growing number of publicly available ophthalmic imaging datasets and open-source code for Machine Learning algorithms. This allows ophthalmic researchers and practitioners to independently perform various deep-learning tasks. With the advancement in artificial intelligence (AI) and in the field of imaging, the choice of the most appropriate AI architecture for different tasks will vary greatly. The best-performing AI-dataset combination will depend on the specific problem that needs to be solved and the type of data available. The article discusses different machine learning models and deep learning architectures currently used for various ophthalmic imaging modalities and for different machine learning tasks. It also proposes the most appropriate models based on accuracy and other important factors such as training time, the ability to deploy the model on clinical devices/smartphones, heatmaps that enhance the self-explanatory nature of classification decisions, and the ability to train/adapt on small image datasets to determine if further data collection is worthwhile. The article extensively reviews the existing state-of-the-art AI methods focused on useful machine-learning applications for ophthalmology. It estimates their performance and viability through training and evaluating architectures with different public and private image datasets of different modalities, such as full-color retinal images, OCT images, and 3D OCT scans. The article is expected to benefit the readers by enriching their knowledge of artificial intelligence applied to ophthalmology.

Permutation Equivariant Graph Framelets for Heterophilous Graph Learning.

The nature of heterophilous graphs is significantly different from that of homophilous graphs, which causes difficulties in early graph neural network (GNN) models and suggests aggregations beyond the one-hop neighborhood. In this article, we develop a new way to implement multiscale extraction via constructing Haar-type graph framelets with desired properties of permutation equivariance, efficiency, and sparsity, for deep learning tasks on graphs. We further design a graph framelet neural network model permutation equivariant graph framelet augmented network (PEGFAN) based on our constructed graph framelets. The experiments are conducted on a synthetic dataset and nine benchmark datasets to compare the performance with other state-of-the-art models. The result shows that our model can achieve the best performance on certain datasets of heterophilous graphs (including the majority of heterophilous datasets with relatively larger sizes and denser connections) and competitive performance on the remaining.

Randomness Regularization With Simple Consistency Training for Neural Networks.

Randomness is widely introduced in neural network training to simplify model optimization or avoid the over-fitting problem. Among them, dropout and its variations in different aspects (e.g., data, model structure) are prevalent in regularizing the training of deep neural networks. Though effective and performing well, the randomness introduced by these dropout-based methods causes nonnegligible inconsistency between training and inference. In this paper, we introduce a simple consistency training strategy to regularize such randomness, namely R-Drop, which forces two output distributions sampled by each type of randomness to be consistent. Specifically, R-Drop minimizes the bidirectional KL-divergence between two output distributions produced by dropout-based randomness for each training sample. Theoretical analysis reveals that R-Drop can reduce the above inconsistency by reducing the inconsistency among the sampled sub structures and bridging the gap between the loss calculated by the full model and sub structures. Experiments on 7 widely-used deep learning tasks ( 23 datasets in total) demonstrate that R-Drop is universally effective for different types of neural networks (i.e., feed-forward, recurrent, and graph neural networks) and different learning paradigms (supervised, parameter-efficient, and semi-supervised). In particular, it achieves state-of-the-art performances with the vanilla Transformer model on WMT14 English → German translation ( 30.91 BLEU) and WMT14 English → French translation ( 43.95 BLEU), even surpassing models trained with extra large-scale data and expert-designed advanced variants of Transformer models.

A novel artificial intelligence model for diagnosing Acanthamoeba keratitis through confocal microscopy

PurposeTo develop an artificial intelligence (AI) model to diagnose Acanthamoeba keratitis (AK) based on in vivo confocal microscopy (IVCM) images extracted from the Heidelberg Retinal Tomograph 3 (HRT 3). MethodsThis retrospective cohort study utilized HRT 3 IVCM images from patients who had received a culture-confirmed diagnosis of AK between 2013 and 2021 at Massachusetts Eye and Ear. Two cornea specialists independently labeled the images as AK or nonspecific finding (NSF) in a blind manner. Deep learning tasks were then conducted through Python and TensorFlow. Distinguishing between AK and NSF was designed as the task and completed through a devised convolutional neural network. ResultsA dataset of 3312 confocal images from 17 patients with a culture-confirmed diagnosis of AK was used in this study. The inter-rater agreement for identifying the presence or absence of AK in IVCM images was 84 %, corresponding to a total of 2782 images on which both observers agreed and which were included in the model. 1242 and 1265 images of AK and NSF, respectively, were utilized in the training and validation sets, and 173 and 102 images of AK and NSF, respectively, were utilized in the evaluation set. Our model had an accuracy, sensitivity, and specificity of 76 % each, and a precision of 78 %. ConclusionsWe developed an HRT-based IVCM AI model for AK diagnosis utilizing culture-confirmed cases of AK. We achieved good accuracy in diagnosing AK and our model holds significant promise in the clinical application of AI in improving early AK diagnosis.

A Hyper-Transformer model for Controllable Pareto Front Learning with Split Feasibility Constraints

Controllable Pareto front learning (CPFL) approximates the Pareto optimal solution set and then locates a non-dominated point with respect to a given reference vector. However, decision-maker objectives were limited to a constraint region in practice, so instead of training on the entire decision space, we only trained on the constraint region. Controllable Pareto front learning with Split Feasibility Constraints (SFC) is a way to find the best Pareto solutions to a split multi-objective optimization problem that meets certain constraints. In the previous study, CPFL used a Hypernetwork model comprising multi-layer perceptron (Hyper-MLP) blocks. Transformer can be more effective than previous architectures on numerous modern deep learning tasks in certain situations due to their distinctive advantages. Therefore, we have developed a hyper-transformer (Hyper-Trans) model for CPFL with SFC. We use the theory of universal approximation for the sequence-to-sequence function to show that the Hyper-Trans model makes MED errors smaller in computational experiments than the Hyper-MLP model.

SimMix: Local similarity-aware data augmentation for time series

We find that local similarity is an essential factor for data augmentation in deep learning tasks concerning time series data, the applications of which are prevalent in various domains such as smart healthcare, intelligent transportation, smart finance, etc. With empirical and theoretical analysis, we find deep learning models achieve excellent performance only when the data augmentation method performs with appropriate intensity of local similarity—during the data augmentation process, too large/small intra-class local similarity will decrease the performance of deep learning models. With this discovery, we propose a time series augmentation method based on intra-class Similarity Mixing (SimMix), which accurately controls the intensity by quantifying and adjusting the similarity between augmented samples and original samples. With a PAC (i.e., Probably Approximately Correct) theoretical foundation, we design a cutmix strategy for non-equal length segments to eliminate semantic information loss and noise introduction defects in traditional methods. Through extensive validation on 10 real-world datasets, we demonstrate that the proposed method can outperform the state-of-the-art by a large margin.

Hyperspectral in-memory computing with optical frequency combs and programmable optical memories

The rapid rise of machine learning drives demand for extensive matrix-vector multiplication operations, thereby challenging the capacities of traditional von Neumann computing systems. Researchers explore alternatives, such as in-memory computing architecture, to find energy-efficient solutions. In particular, there is renewed interest in optical computing systems, which could potentially handle matrix-vector multiplication in a more energy-efficient way. Despite promising initial results, developing high-throughput optical computing systems to rival electronic hardware remains a challenge. Here, we propose and demonstrate a hyperspectral in-memory computing architecture, which simultaneously utilizes space and frequency multiplexing, using optical frequency combs and programmable optical memories. Our carefully designed three-dimensional opto-electronic computing system offers remarkable parallelism, programmability, and scalability, overcoming typical limitations of optical computing. We have experimentally demonstrated highly parallel, single-shot multiply-accumulate operations with precision exceeding 4 bits in both matrix-vector and matrix-matrix multiplications, suggesting the system’s potential for a wide variety of deep learning and optimization tasks. Our approach presents a realistic pathway to scale beyond peta operations per second, a major stride towards high-throughput, energy-efficient optical computing.

A novel approach for handling missing data to enhance network intrusion detection system

Managing missing data is a critical challenge in Intrusion Detection System (IDS) datasets, significantly affecting the performance of deep learning models. To address this issue, we introduce DeepLearning_Based_MissingData_Imputation (DMDI), a novel method designed to enhance the quality of input data by efficiently handling missing values. Our approach employs the Random Missing Value (RMV) algorithm to simulate missing data, enabling thorough testing and comparison of various imputation techniques. The DMDI method integrates a stacked denoising autoencoder with Gradient Boosting to improve imputation accuracy. We evaluated the effectiveness of our approach through three experimental phases: generating missing data, imputing missing values, and assessing imputation models. Using the NSL-KDD and UNSW-NB15 datasets, our results demonstrate significant improvements in the performance of five different classifiers (SVM, KNN, Logistic Regression, Decision Tree, and Random Forest) after imputation. On average, our method achieved accuracy improvements ranging from 0.95 to 0.97 across these classifiers compared to baseline imputation methods. Detailed analysis using Python 3 validates our findings, demonstrating enhanced model performance and robustness. This study underscores the necessity of precise missing data imputation for enhancing deep learning tasks, particularly in anomaly detection systems. It provides a reliable solution for managing missing data in IDS datasets.

Data augmentation for Gram-stain images based on Vector Quantized Variational AutoEncoder

Availability of large-scale datasets plays a significant role in segmentation and classification tasks using deep learning. However, domains such as healthcare inherently suffer from unavailability and inaccessibility of data. This leads to challenges in deploying CNN-based models for computer-aided diagnosis. This challenge extends to Gram-stain image analysis for detecting bacterial infections, which is a crucial task. The lack of datasets containing Gram-stained direct and culture smear images exacerbates the significant challenges in deep learning tasks. In this regard, we investigate a novel application of the Variational AutoEncoder. Specifically, the Vector Quantized Variational AutoEncoder model is trained to generate the Gram-stain images. Incorporating a novel loss function, where the quality loss (Lqu) is derived by integrating the LossSSIM, L1, and L2 losses with the VQ-VAE loss (Lossvq) for proposed approach for Gram-stained direct and culture smear images. This modification facilitates the creation of images closely resembling the original input, leading to notable SSIM scores of 0.92 for Gram-stained culture images and 0.88 for Gram-stained direct smear images. The current study compares the proposed method with state-of-the-art machine learning based and CNN based transformations. This work also demonstrates the classification process with and without image augmentation. It shows that the area under the curve in the case of augmentation is higher by an average of 20%.

Multi-Queue-Based Offloading Strategy for Deep Reinforcement Learning Tasks

Multi-Queue-Based Offloading Strategy for Deep Reinforcement Learning Tasks

DGPCD: a benchmark for typical official-style Dougong in ancient Chinese wooden architecture

Dougong, a distinctive component of ancient Chinese wooden architecture, holds significant importance for the preservation and restoration of such structures. In particular, the northern official-style buildings represent the pinnacle of ancient Chinese construction techniques. In the realm of cultural heritage preservation, the application of deep learning has gradually expanded, demonstrating remarkable effectiveness. Point cloud serving as a crucial source for Dougong, encapsulates various information, enabling support for tasks like Dougong point cloud classification and completion. The quality of Dougong datasets directly impacts the outcomes of DNNs (deep neural networks), as they serve as the foundational data support for these models. The typical official-style Dougong, with its standardized and repetitive structural patterns, is highly suitable for training DNNs to accurately recognize and analyze these complex architectural elements. However, due to the inherent characteristics of Dougong, such as coplanarity and occlusion, acquiring point cloud data is challenging, resulting in poor data quality and organizational difficulties. To address this issue, our study adopts a multi-source data fusion approach to tackle the challenges of insufficient data quantity and poor data quality. Further, through data augmentation, we enhance the dataset’s volume and generalize its characteristics. This effort culminates in the creation of the typical official-style Dougong Point Cloud Dataset (DG Dataset), poised to support deep learning tasks related to Dougong scenarios.

Improving job matching with deep learning-based hyper-personalization

This study introduces a novel approach to streamline the recruitment process, benefiting both employers and job seekers. It leverages real-time personality-based classification to match candidates with the most suitable roles in a scalable and precise manner. This is achieved through machine learning-driven hyper-personalization, employing deep learning models to create a predictive language model. The study encompasses two key tasks: binary classification, distinguishing sentences containing soft skills (1) from those that do not (0), and multi-class classification, categorizing positive sentences into five classes based on Big Five personality traits. The research involved a series of experiments. Initially, multiple machine learning algorithms were employed to establish baseline models. Subsequently, the study investigated the impact of deep learning versus these baseline models. The results demonstrated an accuracy of 0.79% and 0.68% for binary classification tasks, and 0.79% and 0.60% for multi-class classification tasks, using Support Vector Machines in the machine learning task, and Bidirectional Long Short-Term Memory in the deep learning task, respectively. This approach showcases promise in revolutionizing the job matching process, offering a more efficient and accurate means of connecting individuals with their ideal employment opportunities based on their unique soft skills and personality traits.

Comparing hybrid models for recognising objects in thermal images at nighttime

This research aims to revolutionize urban object recognition by developing cloud-based Python programs using intelligent algorithms. Unlike current models that focus on colour enhancement in nighttime thermal images, this work addresses the critical challenge of accurate object detection in urban landscapes. The proposed method incorporates a binary generative adversarial network (GAN) generator that can switch bidirectionally between daytime colour (DC) and nighttime infrared (NTIR) images. memory-based visual image memory (MVAM), system extracts important descriptive information from urban landscape images, reducing problems related to small sample sizes. This discussion presents a comprehensive improvement and evaluation of a deep learning image classification pipeline using Google Colab, demonstrating advanced image processing. Using TensorFlow, Keres and scikit image libraries combined with advanced algorithms such as DenseNet121 and MobileNetV2 presents a clear approach. We created a Bidirectional GAN + MVAM for object recognition in this work. Our method performed well, with an accuracy of 81.43%, precision of 51.16, recall of 50.11, and F-score of 46.37. The systematic presentation of the code presents a careful strategy to ensure optimal performance, stability, and efficiency of deep learning and image processing tasks.

Classification of Vascular Dementia on magnetic resonance imaging using deep learning architectures

Vascular Dementia is a severe disease that results from dead nerve cells’ accumulation in blood vessels. This affects the blood flow and impairs memory and decision-making abilities. Machine learning and deep learning have been used in detecting this disease. Nevertheless, their accuracy has been inconsistent, explaining why their utilization in diagnosing patients has led to poor performance. We developed several transfer learning architectures that improve classification accuracy and diagnosis performance in assessing vascular dementia. The process first entails the preprocessing of the dataset where a random selection ensures data representation is balanced. We used a dataset containing resting-state fMRI scans to split training, testing, and validation into 80%, 10%, and 10%. We employ different Convolutional Neural Network architectures such as VGG16, VGG19, DenseNet121, and InceptionResNetV2 to enhance classification. To, enhance these, we incorporated Rectified Linear Unit and leaky activation functions for the training phase to counteract problems associated with vanishing gradient common in deep learning tasks. Our methodology ensures effective information flow throughout different layers, which is essential for a divergent information hierarchy in medical information. As such, the specifics show that our approach achieved 84.67% in accuracy in the multi-classification, which is better than the current state-of-the-art research in the same field. Therefore, the result shows that transfer learning-based approaches are suitable when combined with strategic pre-processing and activation functions in improving the diagnosis of vascular dementia using MRI images.