Deep Learning Implementation Research Articles

Uncertainty Qualification for Deep Learning-Based Elementary Reaction Property Prediction.

The prediction of the thermodynamic and kinetic properties of elementary reactions has shown rapid improvement due to the implementation of deep learning (DL) methods. While various studies have reported the success in predicting reaction properties, the quantification of prediction uncertainty has seldom been investigated, thus compromising the confidence in using these predicted properties in practical applications. Here, we integrated graph convolutional neural networks (GCNN) with three uncertainty prediction techniques, including deep ensemble, Monte Carlo (MC)-dropout, and evidential learning, to provide insights into the uncertainty quantification and utility. The deep ensemble model outperforms others in accuracy and shows the highest reliability in estimating prediction uncertainty across all elementary reaction property data sets. We also verified that the deep ensemble model showed a satisfactory capability in recognizing epistemic and aleatoric uncertainties. Additionally, we adopted a Monte Carlo Tree Search method for extracting the explainable reaction substructures, providing a chemical explanation for DL predicted properties and corresponding uncertainties. Finally, to demonstrate the utility of uncertainty qualification in practical applications, we performed an uncertainty-guided calibration of the DL-constructed kinetic model, which achieved a 25% higher hit ratio in identifying dominant reaction pathways compared to that of the calibration without uncertainty guidance.

Comparison of Deep Learning and Machine Learning Model for Phishing Email Classification

Email is a medium in business communication that people use everyday and not only holds sensitive information but also identity for the user and organization. The uniqueness of information contained make phishing detection and remedy cannot be applied generally. Existing method applied in common on security software such as blacklisting keyword or email address is not enough to outpace evolving phishing method. Nowadays the implementation of machine learning and deep learning grow rapidly fast and the method used by machine learning and deep learning that can learn pattern of inputs and natural language processing making classification to detect email phishing promising, this study present proposed method of phishing mail classification by comparing the use of machine learning and deep learning model, The algorithm used in this paper are recurrent convolutional neural network and random forest with tf-idf and fasttext word embedding. The Dataset contain phishing and legitimate of an email gathered from a trading company in Indonesia. The dataset will be split into 70% for training and 30% for testing. As a result of this comparison, the random forest model with tf-idf word embedding achieve highest accuracy of 100% for the dataset used and the highest accuracy for recurrent convolutional neural network model with fasttext is 98.21%.

FCSSL: fusion enhanced contrastive self-supervised learning method for parallel MRI reconstruction

Objective. The implementation of deep learning in magnetic resonance imaging (MRI) has significantly advanced the reduction of data acquisition times. However, these techniques face substantial limitations in scenarios where acquiring fully sampled datasets is unfeasible or costly.Approach. To tackle this problem, we propose a fusion enhanced contrastive self-supervised learning (FCSSL) method for parallel MRI reconstruction, eliminating the need for fully sampledk-space training dataset and coil sensitivity maps. First, we introduce a strategy based on two pairs of re-undersampling masks within a contrastive learning framework, aimed at enhancing the representational capacity to achieve higher quality reconstruction. Subsequently, a novel adaptive fusion network, trained in a self-supervised learning manner, is designed to integrate the reconstruction results of the framework.Results. Experimental results on knee datasets under different sampling masks demonstrate that the proposed FCSSL achieves superior reconstruction performance compared to other self-supervised learning methods. Moreover,the performance of FCSSL approaches that of the supervised methods, especially under the 2DRU and RADU masks, but no need for fully sample data. The proposed FCSSL, trained under the 3× 1DRU and 2DRU masks, can effectively generalize to unseen 1D and 2D undersampling masks, respectively. For target domain data that exhibit significant differences from source domain data, the proposed model, fine-tuned with just a few dozen instances of undersampled data in the target domain, achieves reconstruction performance comparable to that achieved by the model trained with the entire set of undersampled data.Significance. The novel FCSSL model offers a viable solution for reconstructing high-quality MR images without needing fully sampled datasets, thereby overcoming a major hurdle in scenarios where acquiring fully sampled MR data is difficult.

The collapse of modern exposures in the recording of X-RAY images in medicine

This article reviewed the methods and algorithms for processing and diagnosing X-ray images that are widely used in medicine today. In order to improve the implementation of deep learning in modern medicine and further improve the widespread use of HMLs, the method, algorithm and model of X-ray image processing using the Raspberry Pi device were analyzed.

Modified Alexnet Architecture for Classification of Cassava Based on Leaf Images

The objective of this study is to address the drawbacks of conventional classification approaches through the implementation of deep learning, specifically a modified AlexNet. The primary aim of this study is to precisely categorize the four distinct varieties of cassava, namely Manggu, Gajah, Beracun, and Kapok. The cassava dataset was obtained from farmers in Lamongan, Indonesia, and was used as a source of information. Data collection on cassava leaves was carried out with agricultural research specialists. A total of 1,400 images are included in the dataset, with 350 images corresponding to each variety of cassava produced. The central focus of this research lies in a comprehensive evaluation of the modified AlexNet architecture's performance compared to the original AlexNet architecture for cassava classification. Multiple scenarios were examined, involving diverse combinations of learning rates and epochs, to thoroughly assess the robustness and adaptability of the proposed approach. Among the evaluation criteria that were rigorously examined were accuracy, recall, F1 score, and precision. These metrics were used to determine the predictive capabilities of the model as well as its potential utilization in the actual world. The results show that the modified AlexNet design has better performance than the original AlexNet for recall, accuracy, precision, and F-1 score, all achieving a rate of 87%. In situations where a learning rate of 0.0001 and an epoch count of 150 are utilized, the performance of the approach stands out significantly, displaying an excellent level of competency. Nevertheless, it is crucial to recognize that distinct fluctuations in performance were noted within particular contexts and with diverse learning rates.

Zero-shot autonomous robot manipulation via natural language

Smart manufacturing revolutionizes advanced automation by leveraging deep learning and robotics technology. Nevertheless, deep learning implementation in manufacturing faces several challenges in preparing datasets, training a model, and adapting to various applications. Furthermore, programming languages are required to develop deep learning models and communicate with robots, which is not straightforward to quickly and precisely reflect human intentions in manufacturing processes. To alleviate the issue, this study proposes an autonomous zero-shot robot manipulation framework via natural language. Specifically, the Segmentation Anything Model (SAM), a promptable image segmentation model, first segments objects in images captured by an RGB-D camera, which are passed to Generative Pre-trained Transformers (GPTs), a Large Language Model (LLM), to identify and localize a designated object based on human commands given in natural language. Afterwards, the Robot Operating System (ROS) manipulates a robot to a localized position. With no additional programming or specialized skills in object localization and robot manipulation, the framework demonstrates an autonomous robot automation approach through intuitive communication between humans and machines through natural language.

Deep learning for image-based detection of weeds from emergence to maturity in wheat fields

Effective weed control in wheat (Triticum aestivum L.) fields is crucial for optimizing production and ensuring food security in semi-arid regions. The implementation of deep learning for weed detection could enable precise weed management, leading to enhanced wheat yield, increased income for growers through targeted herbicide use, and a reduced environmental footprint associated with herbicide application. Charlock mustard (Sinapis arvensis L.) [CM], creeping thistle (Cirsium arvense (L.) Scop) [CT], and forking larkspur (Consolida regalis Gray) [FL]) are among the most invasive weed species in wheat fields. Effective management of these weeds is essential to ensure optimal wheat productivity in the semi-arid regions. To detect weeds, an unmanned arial vehicle was used to collect images and videos from 185 wheat fields in Tokat, Turkey. The images were collected from five phenological periods of each species including seedling, rosette, vegetative, flowering, and fruiting (in total 15 classification process). These stages are either important for timely weed control or essential for understanding weed pressure threshold, the potential for increasing weed seedbank increase, and identifying herbicide resistant weeds. A total of 145,792 objects were labeled in the images to create the dataset, divided into 80 % for training + validation and 20 % for testing. The dataset was used with the YOLOv5 (You Only Look Once) deep learning architecture. The YOLOv5 was the newest YOLO model during this study carried out. All neural networks provided by YOLOv5 (nano, small, medium, large, and xlarge) were trained and the Precision, Recall, F-1 Score, and AUC (Area under the Curve) performances of the neural networks were evaluated. The YOLOv5s (small) was the most precise neural network in detecting weeds regardless of the species. In contrast, nano was the least precise neural network in detecting each weed species. For CT, when small neural network was used, precision of detection ranged from 0.87 (fruit stage) to 0.96 (rosette and vegetative stages). For, forking larkspur, the best neural network was achieved with YOLOv5 small and the precision ranged from 0.45 (fruit stage) to 0.93 (vegetative stage). Using YOLOv5 small in Charlock mustard, the most precise detection occurred at seedling, rosette, and vegetative stages (with precision of 0.96) and the least precise detection was recorded at the fruit stage (0.81). Other performance evaluations (Recall, F-1 Score, and AUC) were in agreement with those of Precision. Our results indicated that by using YOLOv5 small neural network, all three weed species could be detected at all growth stages with the precision of 0.86 with the exception of fruit stage. Future research should (1) focus on assessing YOLOv5 small neural network with the newest versions of YOLO and also (2) evaluate this neural network for detecting weed species in wheat fields to allow for improved weed control and future recommendations.

A Proof-of-Concept Study of Stability Monitoring of Implant Structure by Deep Learning of Local Vibrational Characteristics

This study develops a structural stability monitoring method for an implant structure (i.e., a single-tooth dental implant) through deep learning of local vibrational modes. Firstly, the local vibrations of the implant structure are identified from the conductance spectrum, achieved by driving the structure using a piezoelectric transducer within a pre-defined high-frequency band. Secondly, deep learning models based on a convolutional neural network (CNN) are designed to process the obtained conductance data of local vibrational modes. Thirdly, the CNN models are trained to autonomously extract optimal vibration features for structural stability assessment of the implant structure. We employ a validated predictive 3D numerical modeling approach to demonstrate the feasibility of the proposed approach. The proposed method achieved promising results for predicting material loss surrounding the implant, with the best CNN model demonstrating training and testing errors of 3.7% and 4.0%, respectively. The implementation of deep learning allows optimal feature extraction in a lower frequency band, facilitating the use of low-cost active sensing devices. This research introduces a novel approach for assessing the implant’s stability, offering promise for developing future radiation-free stability assessment tools.

A Comprehensive Study on Implementation of Deep Learning on Autonomous Vehicle for Steering Angle Prediction and Stability

The development of autonomous vehicles has recently enhanced the transportation industry and opened up a variety of opportunities and problems that can be solved with the aid of current methods and technology. In this study, three separate algorithms were used to predict the steering angle with a track image: Artificial Neural Network (ANN), Convolution Neural Network (CNN), and a combination of CNN and Long-Short Term Memory (CNN-LSTM). The PID controller was employed for benchmarking, which takes Cross-Track Error (CTE) provided by the simulation to steer the vehicle. To achieve improved performance, the standard NVIDIA CNN self-driving model was slightly altered by feeding it with sequential frames. The comparison analysis was conducted using the OpenAI Gym Donkey Simulator.

Gluconeogenesis unraveled: A proteomic Odyssey with machine learning

The metabolic pathway known as gluconeogenesis, which produces glucose from non-carbohydrate substrates, is essential for maintaining balanced blood sugar levels while fasting. It’s extremely important to anticipate gluconeogenesis rates accurately to recognize metabolic disorders and create efficient treatment strategies. The implementation of deep learning and machine learning methods to forecast complex biological processes has been gaining popularity in recent years. The recognition of both the regulation of the pathway and possible therapeutic applications of proteins depends on accurate identification associated with their gluconeogenesis patterns. This article analyzes the uses of machine learning and deep learning models, to predict gluconeogenesis efficiency. The study also discusses the challenges that come with restricted data availability and model interpretability, as well as possible applications in personalized healthcare, metabolic disease treatment, and the discovery of drugs. The predictor utilizes statistics moments on the structures of gluconeogenesis and their enzymes, while Random Forest is utilized as a classifier to ensure the accuracy of this model in identifying the best outcomes. The method was validated utilizing the independent test, self-consistency, 10 k fold cross-validations, and jackknife test which achieved 92.33 %, 91.87 %, 87.88 %, and 87.02 %. An accurate prediction of gluconeogenesis has significant implications for understanding metabolic disorders and developing targeted therapies. This study contributes to the rising field of predictive biology by mixing algorithms for deep learning, and machine learning, with metabolic pathways.

AI-Assisted Sensor System for the Acetone and Ethanol Detection Using Commercial Metal Oxide-Based Sensor Arrays and Convolutional Neural Network

In recent decades, the traditional landscape of volatile organic compound (VOC) sensing has adopted a new perspective in enhancing the detection of useful VOCs using data intelligence to extract constructive insights of the sensor behaviour towards multiple gases. In the domain of gas sensing, VOCs such as acetone and ethanol have been widely used in sensor testing due to their closely related chemical properties, which poses a challenge in discrimination. Therefore, this study aims to discriminate acetone from ethanol with the use of readily available commercial metal oxide (MOx) sensors through the implementation of Deep Learning (DL) techniques. The data set obtained after exposing a sensing array comprising various MOx sensors to acetone and ethanol was converted to a time-frequency representation known as a scalogram to train and test a multi-input convolutional neural network (CNN). The results show that training the CNN model on the sensor array data set yields better results than with an individual sensor data set. The findings of this research substantiated the ability of DL models to better capture the dynamic interaction of the sensors with acetone and ethanol, leading to the implication of the DL classifier having the capacity to reject sensor inconsistencies and variations in the responses. This research holds promise for advancing health monitoring and disease detection, as the combination of MOx sensors and DL techniques is expected to make significant future contributions in these areas.

Deep learning implementation using convolutional neural network in inorganic packaging waste sorting

Municipal solid waste is a significant issue that causes environmental contamination. One of the most prevalent wastes that is difficult to decompose is waste from inorganic packaging. Inorganic packaging waste management can be done by sorting waste as the first step before going through subsequent processing. However, waste sorting is currently still difficult to do by human power in waste management facilities, so it is necessary to design a system that can assist the waste sorting process. This research aims to develop a model that can classify inorganic packaging at waste processing sites. To develop the model, we used five pre-trained Convolutional Neural Network (CNN) architectures, namely Xception, Inception V3, ResNet-50, Resnet-50 V2, and DenseNet-201. Then, the best architecture based on some metric performances will be tuned. The result displayed that the CNN model with Densenet 201 architecture, accompanied by tuning, achieved the best performance to classify the waste. The accuracy for the validation dataset is 95.31 %, the accuracy for the testing dataset is 95.6 %, precision is 0.96, recall is 0.96, and the F1-score is 0.96. The results of those performance metrics show that the model can predict the image of inorganic packaging waste well for further application to an automated waste sorting system.

Deep Learning-Driven Compiler Enhancements for Efficient Matrix Multiplication

Matrix multiplication is a fundamental operation in many computational fields, requiring optimization to handle increasing data sizes efficiently. In this paper, the implementation of Deep Learning in Matrix multiplication is reviewed, which is considered important nowadays due to the growing complexity of matrix multiplication for gaming and complex programs. The current standard matrix multiplication and the time taken by it on different matrix sizes are described. The Tiled Matrix multiplication, which trims the matrix into various pieces and calculates the product for each piece, and thereafter combines the result, is also described. The times taken by both methods for different matrix sizes were compared. The main idea was to use Deep Neural Networks (DNN) to compare and rank code variants that are obtained in pieces and determine their relative performance. A tournament-based ranking system is used for assigning ranks to the code versions. The effectiveness of these techniques was evaluated on various matrix multiplication operations commonly found in deep learning workloads. Up to 8.844x speedup over the naive implementation for a matrix size of 1024 is achieved by this approach. The results demonstrate the effectiveness of combining compiler optimization techniques and deep learning models in optimizing matrix multiplication.

Exploring deep learning: Preventing HIV through social media data

This paper explores the potential of deep learning in analyzing social media data to identify and support populations at high risk for HIV. With the widespread use of social media, there is an opportunity to leverage this data source for public health research and intervention. Deep learning, a subset of machine learning that utilizes artificial neural networks to analyze complex data, offers a promising approach to extract meaningful insights from large volumes of social media data. The paper begins with an overview of the HIV epidemic and the importance of early detection and prevention efforts. It then introduces deep learning and its applications in public health, highlighting its ability to analyze unstructured data such as text, images, and videos. A literature review is conducted to examine previous studies on using social media data for health surveillance and the applications of deep learning in this context. The review discusses the challenges and limitations of using social media data for public health research, including issues related to privacy, data bias, and algorithm transparency. The methodology section outlines the data collection process, including the sources of social media data such as Twitter and Facebook, and the preprocessing steps to clean and prepare the data for analysis. The section also describes the deep learning models selected for analyzing social media content and the evaluation metrics used to assess their performance. Case studies are presented to illustrate the application of deep learning in identifying HIV-related discussions on social media platforms, analyzing images for signs of risky behavior, and tracking the spread of HIV-related rumors and misinformation. Ethical considerations related to privacy, data bias, and algorithm transparency are discussed, along with recommendations for future research and application. The paper concludes with a summary of key findings and a call to action for further exploration and implementation of deep learning in public health.

Financial Fraud Detection Based on Machine and Deep Learning: A Review

Financial fraud detection is crucial for protecting the integrity of financial markets and institutions globally. Recent advancements in machine learning (ML) and deep learning (DL) have dramatically enhanced the ability to detect and prevent fraudulent activities across various sectors. This review paper examines the implementation of ML and DL in fraud detection, highlighting the evolution from traditional methods to sophisticated models like neural networks, including Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). We explore different ML techniques such as supervised, unsupervised, and hybrid approaches, their effectiveness in handling large, imbalanced datasets, and their application in real-world scenarios. Special attention is given to the integration of technologies like blockchain and IoT with AI to innovate fraud detection frameworks. Despite the promising advancements, challenges remain, such as the need for large volumes of labeled data, potential model bias, and the black-box nature of many deep learning models. Future directions focus on enhancing model transparency, addressing privacy concerns, and expanding the use of federated learning. This review aims to demonstrate the effectiveness of current technologies and encourage their adoption in enhancing global financial security

Deep learning implementation of image segmentation in agricultural applications: a comprehensive review

Image segmentation is a crucial task in computer vision, which divides a digital image into multiple segments and objects. In agriculture, image segmentation is extensively used for crop and soil monitoring, predicting the best times to sow, fertilize, and harvest, estimating crop yield, and detecting plant diseases. However, image segmentation faces difficulties in agriculture, such as the challenges of disease staging recognition, labeling inconsistency, and changes in plant morphology with the environment. Consequently, we have conducted a comprehensive review of image segmentation techniques based on deep learning, exploring the development and prospects of image segmentation in agriculture. Deep learning-based image segmentation solutions widely used in agriculture are categorized into eight main groups: encoder-decoder structures, multi-scale and pyramid-based methods, dilated convolutional networks, visual attention models, generative adversarial networks, graph neural networks, instance segmentation networks, and transformer-based models. In addition, the applications of image segmentation methods in agriculture are presented, such as plant disease detection, weed identification, crop growth monitoring, crop yield estimation, and counting. Furthermore, a collection of publicly available plant image segmentation datasets has been reviewed, and the evaluation and comparison of performance for image segmentation algorithms have been conducted on benchmark datasets. Finally, there is a discussion of the challenges and future prospects of image segmentation in agriculture.

An Improved Lightweight Deep Learning Model and Implementation for Track Fastener Defect Detection with Unmanned Aerial Vehicles

Track fastener defect detection is an essential component in ensuring railway safety operations. Traditional manual inspection methods no longer meet the requirements of modern railways. The use of deep learning image processing techniques for classifying and recognizing abnormal fasteners is faster, more accurate, and more intelligent. With the widespread use of unmanned aerial vehicles (UAVs), conducting railway inspections using lightweight, low-power devices carried by UAVs has become a future trend. In this paper, we address the characteristics of track fastener detection tasks by improving the YOLOv4-tiny object detection model. We improved the model to output single-scale features and used the K-means++ algorithm to cluster the dataset, obtaining anchor boxes that were better suited to the dataset. Finally, we developed the FPGA platform and deployed the transformed model on this platform. The experimental results demonstrated that the improved model achieved an mAP of 95.1% and a speed of 295.9 FPS on the FPGA, surpassing the performance of existing object detection models. Moreover, the lightweight and low-powered FPGA platform meets the requirements for UAV deployment.

Machine and Deep Learning Implementations for Heritage Building Information Modelling: A Critical Review of Theoretical and Applied Research

Research domain and Problem: HBIM modelling from point cloud data has become a crucial research topic in the last decade since it is potentially considered as the central data model paving the way for the digital heritage practice beyond digitization. Reality Capture technologies such as terrestrial laser scanning, drone-mounted LiDAR sensors and photogrammetry enable the reality capture with a sub-millimetre accurate point cloud file that can be used as a reference file for Heritage Building Information Modelling (HBIM). However, HBIM modelling from the point cloud data of heritage buildings is mainly manual, error-prone, and time-consuming. Furthermore, image processing techniques are insufficient for classification and segmentation of point cloud data to speed up and enhance the current workflow for HBIM modelling. Due to the challenges and bottlenecks in the scan-to-HBIM process, which is commonly criticized as complex with its bespoke requirements, semantic segmentation of point clouds is gaining popularity in the literature. Research Aim and Methodology: Therefore, this paper aims to provide a thorough critical review of Machine Learning and Deep Learning methods for point cloud segmentation, classification, and BIM geometry automation for cultural heritage case study applications. Research findings: This paper files the challenges of HBIM practice and the opportunities for semantic point cloud segmentation found across academic literature in the last decade. Beyond definitions and basic occurrence statistics, this paper discusses the success rates and implementation challenges of machine and deep learning classification methods. Research value and contribution: This paper provides a holistic review of point cloud segmentation and its potential for further development and application in the Cultural Heritage sector. The critical analysis provides insight into the current state-of-the-art methods and advises on their suitability for HBIM projects. The review has identified highly original threads of research, which hold the potential to significantly influence practice and further applied research.

One-shot skill assessment in high-stakes domains with limited data via meta learning

Deep Learning (DL) has achieved robust competency assessment in various high-stakes fields. However, the applicability of DL models is often hampered by their substantial data requirements and confinement to specific training domains. This prevents them from transitioning to new tasks where data is scarce. Therefore, domain adaptation emerges as a critical element for the practical implementation of DL in real-world scenarios. Herein, we introduce A-VBANet, a novel meta-learning model capable of delivering domain-agnostic skill assessment via one-shot learning. Our methodology has been tested by assessing surgical skills on five laparoscopic and robotic simulators and real-life laparoscopic cholecystectomy. Our model successfully adapted with accuracies up to 99.5 % in one-shot and 99.9 % in few-shot settings for simulated tasks and 89.7 % for laparoscopic cholecystectomy. This study marks the first instance of a domain-agnostic methodology for skill assessment in critical fields setting a precedent for the broad application of DL across diverse real-life domains with limited data.

Evaluation of resampling techniques for deep learning based identification of promising genotypes in sugarcane varietal trials

Deep learning is a class of machine learning algorithms that extract high-level features from the raw input for making intelligent decisions. Identification of promising genotypes in varietal trials is one of many agriculture domain applications requiring implementation of deep learning to perform intelligent decision using varietal trial data. However, it has been found that varietal trial data to be used for identification is highly imbalanced one providing great challenges for classification tasks in deep learning. For example, only 33 genotypes were identified as promising in zonal varietal trials of All India Coordinated Research Project (AICRP) on Sugarcane during 2016-21, while those of non-promising class are 148. Balancing an imbalanced class is crucial as the classification model, which is trained using the imbalanced class dataset will tend to exhibit the prediction accuracy according to the highest class of the dataset. One way to address this issue is to use resampling, which adjusts the ratio between the different classes, making the data more balanced. Study was conducted to implement and evaluate four resampling techniques viz. random undersampling, random oversampling, ensemble, SMOTE to balance varietal trial dataset in order to build deep learning model to identify promising genotypes in sugarcane. Paper describes the methodology used in our approach for building deep learning model using resampling techniques and then presented comparative performance of these approaches in identifying promising genotypes. Results indicate that SMOTE and random oversampling performed well for balancing imbalanced dataset for developing deep learning model in comparison to no-resampling of imbalanced dataset. SMOTE outperformed all resampling techniques by achieving high values of precision, recall and F1 score for both positive and negative classes. However, ensemble and random undersampling methods did not showed good results in comparison to SMOTE and random oversampling technique. Studies conducted will be useful in developing artificial intelligence based tools for automatic identification of promising genotypes in varietals trials of sugarcane in particular, as well as other crops in general.