Deep Learning-based Framework Research Articles

End-to-end deep learning-based framework for path planning and collision checking: bin-picking application – ERRATUM

End-to-end deep learning-based framework for path planning and collision checking: bin-picking application – ERRATUM

DeepArUco++: Improved detection of square fiducial markers in challenging lighting conditions

Fiducial markers are a computer vision tool used for object pose estimation and detection. These markers are highly useful in fields such as industry, medicine and logistics. However, optimal lighting conditions are not always available, and other factors such as blur or sensor noise can affect image quality. Classical computer vision techniques that precisely locate and decode fiducial markers often fail under difficult illumination conditions (e.g. extreme variations of lighting within the same frame). Hence, we propose DeepArUco++, a deep learning-based framework that leverages the robustness of Convolutional Neural Networks to perform marker detection and decoding in challenging lighting conditions. The framework is based on a pipeline using different Neural Network models at each step, namely marker detection, corner refinement and marker decoding. Additionally, we propose a simple method for generating synthetic data for training the different models that compose the proposed pipeline, and we present a second, real-life dataset of ArUco markers in challenging lighting conditions used to evaluate our system. The developed method outperforms other state-of-the-art methods in such tasks and remains competitive even when testing on the datasets used to develop those methods. Code available in GitHub: https://github.com/AVAuco/deeparuco/.

Grad-CAM Enabled Breast Cancer Classification with a 3D Inception-ResNet V2: Empowering Radiologists with Explainable Insights.

Breast cancer (BCa) poses a severe threat to women's health worldwide as it is the most frequently diagnosed type of cancer and the primary cause of death for female patients. The biopsy procedure remains the gold standard for accurate and effective diagnosis of BCa. However, its adverse effects, such as invasiveness, bleeding, infection, and reporting time, keep this procedure as a last resort for diagnosis. A mammogram is considered the routine noninvasive imaging-based procedure for diagnosing BCa, mitigating the need for biopsies; however, it might be prone to subjectivity depending on the radiologist's experience. Therefore, we propose a novel, mammogram image-based BCa explainable AI (BCaXAI) model with a deep learning-based framework for precise, noninvasive, objective, and timely manner diagnosis of BCa. The proposed BCaXAI leverages the Inception-ResNet V2 architecture, where the integration of explainable AI components, such as Grad-CAM, provides radiologists with valuable visual insights into the model's decision-making process, fostering trust and confidence in the AI-based system. Based on using the DDSM and CBIS-DDSM mammogram datasets, BCaXAI achieved exceptional performance, surpassing traditional models such as ResNet50 and VGG16. The model demonstrated superior accuracy (98.53%), recall (98.53%), precision (98.40%), F1-score (98.43%), and AUROC (0.9933), highlighting its effectiveness in distinguishing between benign and malignant cases. These promising results could alleviate the diagnostic subjectivity that might arise as a result of the experience-variability between different radiologists, as well as minimize the need for repetitive biopsy procedures.

Seizure Sources Can Be Imaged from Scalp EEG by Means of Biophysically Constrained Deep Neural Networks.

Seizure localization is important for managing drug-resistant focal epilepsy. Here, the capability of a novel deep learning-based source imaging framework (DeepSIF) for imaging seizure activities from electroencephalogram (EEG) recordings in drug-resistant focal epilepsy patients is investigated. The neural mass model of ictal oscillations is adopted to generate synthetic training data with spatio-temporal-spectra features similar to ictal dynamics. The trained DeepSIF model is rigorously validated using computer simulations and in a cohort of 33 drug-resistant focal epilepsy patients with high-density (76-channel) EEG seizure recordings, by comparing DeepSIF estimates with surgical resection outcome and seizure onset zone (SOZ) . These findings show that the trained DeepSIF model outperforms other methods in estimating the spatial and temporal information of origins of ictal activities. It achieves a high spatial specificity of 96% and a low spatial dispersion of 3.80±5.74mm when compared to the resection region. The source imaging results also demonstrate good coverage of SOZ, with an average distance of 10.89±10.14mm (from the SOZ to the reconstruction). These promising results suggest that DeepSIF has significant potential for advancing noninvasive imaging of the origins of ictal activities in patients with focal epilepsy, aiding management of intractable epilepsy.

A robust deep learning approach for identification of RNA 5-methyluridine sites

RNA 5-methyluridine (m5U) sites play a significant role in understanding RNA modifications, which influence numerous biological processes such as gene expression and cellular functioning. Consequently, the identification of m5U sites can play a vital role in the integrity, structure, and function of RNA molecules. Therefore, this study introduces GRUpred-m5U, a novel deep learning-based framework based on a gated recurrent unit in mature RNA and full transcript RNA datasets. We used three descriptor groups: nucleic acid composition, pseudo nucleic acid composition, and physicochemical properties, which include five feature extraction methods ENAC, Kmer, DPCP, DPCP type 2, and PseDNC. Initially, we aggregated all the feature extraction methods and created a new merged set. Three hybrid models were developed employing deep-learning methods and evaluated through 10-fold cross-validation with seven evaluation metrics. After a comprehensive evaluation, the GRUpred-m5U model outperformed the other applied models, obtaining 98.41% and 96.70% accuracy on the two datasets, respectively. To our knowledge, the proposed model outperformed all the existing state-of-the-art technology. The proposed supervised machine learning model was evaluated using unsupervised machine learning techniques such as principal component analysis (PCA), and it was observed that the proposed method provided a valid performance for identifying m5U. Considering its multi-layered construction, the GRUpred-m5U model has tremendous potential for future applications in the biological industry. The model, which consisted of neurons processing complicated input, excelled at pattern recognition and produced reliable results. Despite its greater size, the model obtained accurate results, essential in detecting m5U.

A Deep Learning-Based Car Accident Detection Framework Using Edge and Cloud Computing

The evolving technological landscape has seen a pivotal shift with the advent of edge computing, transforming various sectors, particularly accident detection. Edge computing enhances road safety by enabling realtime data processing from onboard sensors, cameras, and connected devices, addressing limitations in traditional cloudbased systems. This paper introduces a deep learning-based accident detection framework within an edge-cloud setup. Utilizing a CNN model, accidents are detected at the edge node near the data source, ensuring low latency, reduced network usage, and faster execution times. The model achieves a remarkable 95.91% accuracy.

Efficient video summarization through MobileNetSSD: a robust deep learning-based framework for efficient video summarization focused on objects of interest

Efficient video summarization through MobileNetSSD: a robust deep learning-based framework for efficient video summarization focused on objects of interest

Research on Software Vulnerability Detection Methods Based on Deep Learning

This paper aims to investigate software vulnerability detection methods based on deep learning to address the ever-growing challenges in software security. With the rapid development of information technology, software vulnerabilities have become the primary targets of cyberattacks, posing severe threats to economic, military, and social security. By analyzing the limitations of existing software vulnerability detection methods, this paper explores the potential applications of deep learning technology in this field. Through a literature review, relevant theories of software vulnerabilities are introduced, including the concept, types, and impacts of vulnerabilities. Subsequently, detailed descriptions of deep learning-based vulnerability detection methods are provided, encompassing Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs) and their variant LSTM, as well as the application of program slicing techniques in vulnerability detection. By evaluating and improving existing deep learning models, a novel deep learning-based vulnerability detection framework is proposed, with its workflow and technical principles elaborated. The proposed deep learning-based vulnerability detection method can effectively enhance the accuracy and efficiency of software vulnerability detection, reduce reliance on expert knowledge, and lower human resource costs. Experimental results demonstrate that this method performs exceptionally well on multiple datasets, indicating broad application prospects and significant research value. The deep learning-based software vulnerability detection method offers new insights and tools for addressing current software security challenges. In the future, with the continuous development and improvement of deep learning technology, this method will play an even more crucial role in the field of software vulnerability detection.

A novel interpretable deep learning-based computational framework designed synthetic enhancers with broad cross-species activity.

Enhancers play a critical role in dynamically regulating spatial-temporal gene expression and establishing cell identity, underscoring the significance of designing them with specific properties for applications in biosynthetic engineering and gene therapy. Despite numerous high-throughput methods facilitating genome-wide enhancer identification, deciphering the sequence determinants of their activity remains challenging. Here, we present the DREAM (DNA cis-Regulatory Elements with controllable Activity design platforM) framework, a novel deep learning-based approach for synthetic enhancer design. Proficient in uncovering subtle and intricate patterns within extensive enhancer screening data, DREAM achieves cutting-edge sequence-based enhancer activity prediction and highlights critical sequence features implicating strong enhancer activity. Leveraging DREAM, we have engineered enhancers that surpass the potency of the strongest enhancer within the Drosophila genome by approximately 3.6-fold. Remarkably, these synthetic enhancers exhibited conserved functionality across species that have diverged more than billion years, indicating that DREAM was able to learn highly conserved enhancer regulatory grammar. Additionally, we designed silencers and cell line-specific enhancers using DREAM, demonstrating its versatility. Overall, our study not only introduces an interpretable approach for enhancer design but also lays out a general framework applicable to the design of other types of cis-regulatory elements.

Deep Learning-Assisted Compound Bioactivity Estimation Framework

Drug Discovery is a highly complicated process. On average, it takes six to twelve years to manufacture a new drug and have the product released in the market. It is of utmost importance to find methods that would accelerate the manufacturing process. This significant challenge in drug development can be addressed using deep learning techniques. The aim of this paper is to propose a deep learning-based framework that can help chemists examine compound biological activity in a more accurate manner. The proposed framework employs autoencoder for data representation of the compounds data, which is then classified using deep neural network followed by building a customized deep regression model to estimate an accurate value of the compound bioactivity. The proposed framework achieved an accuracy of 89% in autoencoder reconstruction error, 79.01% in classification, and MAE of 2.4 while predicting compound bioactivity using deep regression model.

Multi-scale asynchronous correlation and 2D convolutional autoencoder for adolescent health risk prediction with limited fMRI data.

Adolescence is a fundamental period of transformation, encompassing extensive physical, psychological, and behavioral changes. Effective health risk assessment during this stage is crucial for timely intervention, yet traditional methodologies often fail to accurately predict mental and behavioral health risks due to the intricacy of neural dynamics and the scarcity of quality-annotated fMRI datasets. This study introduces an innovative deep learning-based framework for health risk assessment in adolescents by employing a combination of a two-dimensional convolutional autoencoder (2DCNN-AE) with multi-sequence learning and multi-scale asynchronous correlation information extraction techniques. This approach facilitates the intricate analysis of spatial and temporal features within fMRI data, aiming to enhance the accuracy of the risk assessment process. Upon examination using the Adolescent Risk Behavior (AHRB) dataset, which includes fMRI scans from 174 individuals aged 17-22, the proposed methodology exhibited a significant improvement over conventional models. It attained a precision of 83.116%, a recall of 84.784%, and an F1-score of 83.942%, surpassing standard benchmarks in most pertinent evaluative measures. The results underscore the superior performance of the deep learning-based approach in understanding and predicting health-related risks in adolescents. It underscores the value of this methodology in advancing the precision of health risk assessments, offering an enhanced tool for early detection and potential intervention strategies in this sensitive developmental stage.

ResNet Modeling for 12 nm FinFET Devices to Enhance DTCO Efficiency

In this paper, a deep learning-based device modeling framework for design-technology co-optimization (DTCO) is proposed. A ResNet surrogate model is utilized as an alternative to traditional compact models, demonstrating high accuracy in both single-task (I–V or C–V) and multi-task (I–V and C–V) device modeling. Moreover, transfer learning is applied to the ResNet model, using the BSIM-CMG compact model for a 12 nm FinFET SPICE model as the pre-trained source. Through this approach, superior modeling accuracy and faster training speed are achieved compared to a ResNet surrogate model initialized with random weights, thereby meeting the rapid and efficient demands of the DTCO process. The effectiveness of the ResNet surrogate model in circuit simulation for 12 nm FinFET devices is demonstrated.

Deep learning-based proactive fault detection method for enhanced quadrotor safety

The early detection of faults in advanced technological systems is imperative for ensuring operational reliability and safety. While there is a growing interest in using artificial intelligence for fault detection, current methodologies often exhibit limitations in utilizing comprehensive system information and sensor data. Hidden faults within collected data further highlight the need for advanced analysis techniques. This study introduces a novel deep learning-based framework designed to predict faults and extract insights from complex system datasets. The model, consisting of LSTM-autoencoder and BiLSTM classification components, effectively reduces feature dimensions, thereby enhancing fault detection accuracy. The autoencoder’s latent layer identifies prominent features across various dimensions, while BiLSTM classification conducts bidirectional analysis using these features from both healthy and faulty states, facilitating early fault detection. Experimental results demonstrate the model’s efficacy, achieving an accuracy of 79.48% in predicting incipient faults 30 seconds before a serious malfunction occurs. This underscores the significant potential of the proposed framework in enhancing operational safety and reliability in complex systems. Moreover, the study emphasizes the importance of leveraging comprehensive data and advanced analysis techniques for early fault detection.

Crispr-SGRU: Prediction of CRISPR/Cas9 Off-Target Activities with Mismatches and Indels Using Stacked BiGRU

CRISPR/Cas9 is a popular genome editing technology, yet its clinical application is hindered by off-target effects. Many deep learning-based methods are available for off-target prediction. However, few can predict off-target activities with insertions or deletions (indels) between single guide RNA and DNA sequence pairs. Additionally, the analysis of off-target data is challenged due to a data imbalance issue. Moreover, the prediction accuracy and interpretability remain to be improved. Here, we introduce a deep learning-based framework, named Crispr-SGRU, to predict off-target activities with mismatches and indels. This model is based on Inception and stacked BiGRU. It adopts a dice loss function to solve the inherent imbalance issue. Experimental results show our model outperforms existing methods for off-target prediction in terms of accuracy and robustness. Finally, we study the interpretability of this model through Deep SHAP and teacher–student-based knowledge distillation, and find it can provide meaningful explanations for sequence patterns regarding off-target activity.

Corrections to “a Sim-to-Real Deep Learning-Based Framework for Autonomous Nano-Drone Racing”

Corrections to “a Sim-to-Real Deep Learning-Based Framework for Autonomous Nano-Drone Racing”

G-Protein Signaling in Alzheimer's Disease: Spatial Expression Validation of Semi-supervised Deep Learning-Based Computational Framework.

Systemic study of pathogenic pathways and interrelationships underlying genes associated with Alzheimer's disease (AD) facilitates the identification of new targets for effective treatments. Recently available large-scale multiomics datasets provide opportunities to use computational approaches for such studies. Here, we devised a novel disease gene identification (digID) computational framework that consists of a semi-supervised deep learning classifier to predict AD-associated genes and a protein-protein interaction (PPI) network-based analysis to prioritize the importance of these predicted genes in AD. digID predicted 1,529 AD-associated genes and revealed potentially new AD molecular mechanisms and therapeutic targets including GNAI1 and GNB1, two G-protein subunits that regulate cell signaling, and KNG1, an upstream modulator of CDC42 small G-protein signaling and mediator of inflammation and candidate coregulator of amyloid precursor protein (APP). Analysis of mRNA expression validated their dysregulation in AD brains but further revealed the significant spatial patterns in different brain regions as well as among different subregions of the frontal cortex and hippocampi. Super-resolution STochastic Optical Reconstruction Microscopy (STORM) further demonstrated their subcellular colocalization and molecular interactions with APP in a transgenic mouse model of both sexes with AD-like mutations. These studies support the predictions made by digID while highlighting the importance of concurrent biological validation of computationally identified gene clusters as potential new AD therapeutic targets.

Based on improved joint detection and tracking of UAV for multi-target detection of livestock

In agriculture, specifically livestock monitoring, drones' ability to track multiple targets is essential for advancing the field. However, limited computing resources and unpredictable drone movements often cause issues like ambiguous video frames, object obstructions, and size deviations. These inconsistencies reduce tracking accuracy, making traditional algorithms inadequate for handling drone footage. This study introduces an enhanced deep learning-based multi-target drone tracker framework that enables real-time processing. The proposed method combines object detection and tracking by leveraging consecutive frame pairs to extract and share features, enhancing computational efficiency. It employs diverse loss functions to address class and sample distribution imbalances and includes a composite deblurring module to enhance detection accuracy. Object association utilizes a dual regress bounding box technique, aiding in object identification verification and predictive motion. Live tracking is achieved by predicting object locations in subsequent frames, enabling real-time tracking. Evaluation against leading benchmarks shows that the system improves precision and speed, achieving a 4.3 % increase in Multi-Object Tracking Accuracy (MOTA) and a 7.7 % boost in F1 score.

Graph domain adaptation-based framework for gene expression enhancement and cell type identification in large-scale spatially resolved transcriptomics.

Spatially resolved transcriptomics (SRT) technologies facilitate gene expression profiling with spatial resolution in a naïve state. Nevertheless, current SRT technologies exhibit limitations, manifesting as either low transcript detection sensitivity or restricted gene throughput. These constraints result in diminished precision and coverage in gene measurement. In response, we introduce SpaGDA, a sophisticated deep learning-based graph domain adaptation framework for both scenarios of gene expression imputation and cell type identification in spatially resolved transcriptomics data by impartially transferring knowledge from reference scRNA-seq data. Systematic benchmarking analyses across several SRT datasets generated from different technologies have demonstrated SpaGDA's superior effectiveness compared to state-of-the-art methods in both scenarios. Further applied to three SRT datasets of different biological contexts, SpaGDA not only better recovers the well-established knowledge sourced from public atlases and existing scientific literature but also yields a more informative spatial expression pattern of genes. Together, these results demonstrate that SpaGDA can be used to overcome the challenges of current SRT data and provide more accurate insights into biological processes or disease development. The SpaGDA is available in https://github.com/shenrb/SpaGDA.

Enhanced classification performance using deep learning based segmentation for pulmonary embolism detection in CT angiography

PurposeTo develop a deep learning-based algorithm that automatically and accurately classifies patients as either having pulmonary emboli or not in CT pulmonary angiography (CTPA) examinations. Materials and methodsFor model development, 700 CTPA examinations from 652 patients performed at a single institution were used, of which 149 examinations contained 1497 PE traced by radiologists. The nnU-Net deep learning-based segmentation framework was trained using 5-fold cross-validation. To enhance classification, we applied logical rules based on PE volume and probability thresholds. External model evaluation was performed in 770 and 34 CTPAs from two independent datasets. ResultsA total of 1483 CTPA examinations were evaluated. In internal cross-validation and test set, the trained model correctly classified 123 of 128 examinations as positive for PE (sensitivity 96.1 %; 95 % C.I. 91–98 %; P < .05) and 521 of 551 as negative (specificity 94.6 %; 95 % C.I. 92–96 %; P < .05), achieving an area under the receiver operating characteristic (AUROC) of 96.4 % (95 % C.I. 79–99 %; P < .05). In the first external test dataset, the trained model correctly classified 31 of 32 examinations as positive (sensitivity 96.9 %; 95 % C.I. 84–99 %; P < .05) and 2 of 2 as negative (specificity 100 %; 95 % C.I. 34–100 %; P < .05), achieving an AUROC of 98.6 % (95 % C.I. 83–100 %; P < .05). In the second external test dataset, the trained model correctly classified 379 of 385 examinations as positive (sensitivity 98.4 %; 95 % C.I. 97–99 %; P < .05) and 346 of 385 as negative (specificity 89.9 %; 95 % C.I. 86–93 %; P < .05), achieving an AUROC of 98.5 % (95 % C.I. 83–100 %; P < .05). ConclusionOur automatic pipeline achieved beyond state-of-the-art diagnostic performance of PE in CTPA using nnU-Net for segmentation and volume- and probability-based post-processing for classification.

Deep learning-based prediction of the dose-volume histograms for volumetric modulated arc therapy of left-sided breast cancer.

The advancements in artificial intelligence and computational power have made deep learning an attractive tool for radiotherapy treatment planning. Deep learning has the potential to significantly simplify the trial-and-error process involved in inverse planning required by modern treatment techniques such as volumetric modulated arc therapy (VMAT). In this study, we explore the ability of deep learning to predict organ-at-risk (OAR) dose-volume histograms (DVHs) of left-sided breast cancer patients undergoing VMAT treatment based solely on their anatomical characteristics. The predicted DVHs could be used to derive patient-specific dose constraints and dose objectives, streamlining the treatment planning process, standardizing the quality of the plans, and personalizing the treatment planning. This study aimed to develop a deep learning-based framework for the prediction of organ-specific dose-volume histograms (DVH) based on structures delineated for left-sided breast cancer treatment. We used a dataset of 249 left-sided breast cancer patients treated with tangential VMAT fields. We extracted delineated structures and dose distributions for each patient and derived slice-by-slice DVHs for planning target volume (PTV) and organs-at-risk. The patients were divided into training (70%, n=174), validation (10%, n=24), and test (20%, n=51) sets. Collected data were used to train a deep learning model for the prediction of the DVHs based on the delineated structures. The developed deep learning model comprised a modified DenseNet architecture followed by a recurrent neural network. In the independent test set (n=51), the point-wise differences in the slice-by-slice DVHs between the clinical and predicted DVHs were small; the mean squared errors were 3.53, 1.58, 2.28, 3.37, and 1.44 [×10-4] for PTV, heart, ipsilateral lung, contralateral lung, and contralateral breast, respectively. With the derived cumulative DVHs, the mean absolute difference±standard deviation of mean doses between the clinical and the predicted DVH were 0.08± 0.04Gy, 0.24± 0.22Gy, 0.73± 0.46Gy, 0.07± 0.06Gy, and 0.14± 0.14Gy for PTV, heart, ipsilateral lung, contralateral lung, and contralateral breast, respectively. The deep learning-based approach enabled automatic and reliable prediction of the DVH based on delineated structures. The predicted DVHs could potentially serve as patient-specific clinical goals used to aid treatment planning and avoid suboptimal plans or to derive optimization objectives and constraints for automated treatment planning.