Application Of Deep Learning Research Articles

A deep learning-aided multi-objective optimization of a downstream process for production of monoclonal antibody products

Monoclonal antibodies (mAbs) have emerged as the dominant class of therapeutic moieties due to their superior therapeutic potential and safety profile. Downstream processing plays a pivotal role in ensuring the purity, yield, and quality of these therapeutic products. The biopharmaceutical industry generates a significant amount of data, making it an attractive target for creating machine learning (ML) and deep learning (DL) applications. This paper reports a DL-based 2D-convolutional neural network (2D-CNN) to predict: (i) Protein A mAb elute concentration, (ii) Cation exchange (CEX) mAb elute concentration, (iii) % Acidic variant, (iv) Aggregate (%), and (v) % Basic variant from process data that is routinely collected during biopharma manufacturing, including Protein A elution UV, CEX elution UV, CEX elution conductivity, and post viral inactivation pH profiles. Further, interpretation of predicted outputs using the SHAPLEY technique and optimization through multi-objective Bayesian optimization methods have been attempted so as to match the biosimilar to the reference molecule with respect to the critical quality attributes. It is observed that the proposed model outperforms other published approaches in prediction. The optimum results were experimentally validated by performing three consecutive runs, and a mean percentage deviation of less than 3 % was reported. The proposed approach demonstrates the feasibility of real-time prediction and optimization in biopharmaceutical product manufacturing using CNN-aided modeling and multi-objective weighted sum optimization.

Automated identification of aquatic insects: A case study using deep learning and computer vision techniques

Deep learning techniques have recently found application in biodiversity research. Mayflies (Ephemeroptera), stoneflies (Plecoptera) and caddisflies (Trichoptera), often abbreviated as EPT, are frequently used for freshwater biomonitoring due to their large numbers and sensitivity to environmental changes. However, the morphological identification of EPT species is a challenging but fundamental task. Morphological identification of these freshwater insects is therefore not only extremely time-consuming and costly, but also often leads to misjudgments or generates datasets with low taxonomic resolution. Here, we investigated the application of deep learning to increase the efficiency and taxonomic resolution of biomonitoring programs. Our database contains 90 EPT taxa (genus or species level), with the number of images per category ranging from 21 to 300 (16,650 in total). Upon completion of training, a CNN (Convolutional Neural Network) model was created, capable of automatically classifying these taxa into their appropriate taxonomic categories with an accuracy of 98.7 %. Our model achieved a perfect classification rate of 100 % for 68 of the taxa in our dataset. We achieved noteworthy classification accuracy with morphologically closely related taxa within the training data (e.g., species of the genus Baetis, Hydropsyche, Perla). Gradient-weighted Class Activation Mapping (Grad-CAM) visualized the morphological features responsible for the classification of the treated species in the CNN models. Within Ephemeroptera, the head was the most important feature, while the thorax and abdomen were equally important for the classification of Plecoptera taxa. For the order Trichoptera, the head and thorax were almost equally important. Our database is recognized as the most extensive aquatic insect database, notably distinguished by its wealth of included categories (taxa). Our approach can help solve long-standing challenges in biodiversity research and address pressing issues in monitoring programs by saving time in sample identification.

OPTIMIZATION OF CNN + MOBILENETV3 FOR INSECT IDENTIFICATION: TOWARD HIGH ACCURACY

Developments in the field of artificial intelligence and deep learning, particularly Convolutional Neural Networks (CNN) techniques, have expanded the research potential and applications in ecology, including efficient and accurate insect classification. However, there are challenges in achieving high levels of accuracy with similar computational efficiency. In response, the efficient MobileNetV3 architecture was investigated to improve the insect pest classification process. Through an analytical descriptive quantitative approach and insect datasets from Kaggle, this study tested the effectiveness of CNN models optimized with MobileNetV3. The results indicated that the optimized model achieved classification accuracy of up to 90%, with consistent performance between training and validation data and significant loss reduction. With high precision and processing efficiency, this discovery makes a substantial contribution to deep learning applications in the field of intelligent agriculture, promising methodological improvements for other classification problems. Despite offering a promising solution, this study recognizes the limitation in dataset diversity and suggests further exploration with more varied datasets to strengthen the model's application in actual agricultural practices.

The application of deep learning in the innovation of intelligent English teaching mode

With the rapid development of deep learning technology, its application in various fields is increasingly extensive. Especially in the field of education, the application of deep learning technology has brought great challenges and changes to the traditional teaching mode. This research is aimed at the application of deep learning in intelligent English teaching mode. Firstly, the theory of deep learning is studied in depth, and the application cases of deep learning in other fields are discussed. Secondly, the research designs and implements an intelligent English teaching model based on deep learning, and carries out a lot of experiments and tests. The experimental results show that this new teaching mode can effectively improve the efficiency and effect of students’ English learning. However, it is also found that the model has some problems, such as model training needs a lot of computing resources, has certain requirements for hardware equipment, and some students have poor adaptability to the new learning mode. To solve these problems, a series of solutions are proposed. In general, although there are still some challenges in the application of deep learning in intelligent English teaching mode, its potential is huge and it has a profound impact on improving the quality of teaching.

Consumer Price Index Forecasting in Turkey: A Comparison of Deep Learning and Machine Learning Approaches

Accordingly, different deep learning and machine learning models such as long- and short-term memory, temporal recurrent units, random forests, artificial neural networks, and K-nearest neighbors are used for CPI forecasting. The prediction performances of the models on the test data were evaluated with RMSE, MSE, MAE, MAPE, and R^2 error statistics. The results show that the Gateway Recurrent Unit model outperforms the Long and Short Term Memory, Random Forest, Neural Network, and K-Nearest Neighbors models. Compared to the other four models, the RMSE, MSE, MAE, MAPE, and R^2 values performed better in the recurrent unit model. In addition, it has been observed that deep learning and machine learning models can be used effectively in the field of inflation in consumer price index forecasting. These results provide an effective method of CPI forecasting, which is an important component of economic forecasting and inflation management. From an academic perspective, this study demonstrates the applicability of deep learning and machine learning models in economics and finance. In practice, it provides a valuable tool for economic and financial decision-makers and illuminates the way for future similar studies.

Segmentation and Tracking Based on Equalized Memory Matching Network and Its Application in Electric Substation Inspection

With the wide application of deep learning, power inspection technology has made great progress. However, substation inspection videos often present challenges such as complex backgrounds, uneven lighting distribution, variations in the appearance of power equipment targets, and occlusions, which increase the difficulty of object segmentation and tracking, thereby adversely affecting the accuracy and reliability of power equipment condition monitoring. In this paper, a pixel-level equalized memory matching network (PEMMN) for power intelligent inspection segmentation and tracking is proposed. Firstly, an equalized memory matching network is designed to collect historical information about the target using a memory bank, in which a pixel-level equalized matching method is used to ensure that the reference frame information can be transferred to the current frame reliably, guiding the segmentation tracker to focus on the most informative region in the current frame. Then, to prevent memory explosion and the accumulation of segmentation template errors, a mask quality evaluation module is introduced to obtain the confidence level of the current segmentation result so as to selectively store the frames with high segmentation quality to ensure the reliability of the memory update. Finally, the synthetic feature map generated by the PEMMN and the mask quality assessment strategy are unified into the segmentation tracking framework to achieve accurate segmentation and robust tracking. Experimental results show that the method performs excellently on real substation inspection scenarios and three generalized datasets and has high practical value.

Hype or hope? Ketamine for the treatment of depression: results from the application of deep learning to Twitter posts from 2010 to 2023.

To investigate societal perceptions of ketamine's use in depression therapy by analysing Twitter posts from January 1, 2010 to April 1, 2023. Using Twitter as the social media platform of choice, and employing search terms based on (depression OR depressed OR depressive) AND (ketamine OR esketamine OR Spravato), we collected English-language tweets from January 1, 2010, to April 1, 2023. Using unsupervised machine learning and natural language processing (NLP) techniques, including Bidirectional Encoder Representations from Transformers (BERT) and BERTopic, the study identified prevalent topics surrounding public chatter around the use of ketamine in depression treatment. Manual thematic analyses further refined these topics into themes. Out of an initial dataset of 99,405 tweets, after removing duplicate tweets, re-tweets and tweets posted by organizations over Twitter, 18,899 unique tweets from presumably individual users were analysed. Analysis of temporal trends revealed a shift in public attitudes, particularly after the United States Food and Drug Administration (FDA)'s 2019 approval of ketamine for depression. Three major themes emerged: a changing regulatory landscape, cautious optimism, and personal experiences with the drug. There was an initial spike in discussions post-FDA approval in 2019. Thereafter, cautious optimism (Theme 2) decreased among the general public, with more personal accounts (Theme 3) highlighting the potential benefits for some treatment-resistant patients. Limitations of the study include Twitter's inherent biases towards younger, English-speaking demographics. In summary, the public's multifaceted perception leans towards a hopeful stance on ketamine's therapeutic potential for depression.

Application of deep learning in isolated tooth identification

BackgroundTeeth identification has a pivotal role in the dental curriculum and provides one of the important foundations of clinical practice. Accurately identifying teeth is a vital aspect of dental education and clinical practice, but can be challenging due to the anatomical similarities between categories. In this study, we aim to explore the possibility of using a deep learning model to classify isolated tooth by a set of photographs.MethodsA collection of 5,100 photographs from 850 isolated human tooth specimens were assembled to serve as the dataset for this study. Each tooth was carefully labeled during the data collection phase through direct observation. We developed a deep learning model that incorporates the state-of-the-art feature extractor and attention mechanism to classify each tooth based on a set of 6 photographs captured from multiple angles. To increase the validity of model evaluation, a voting-based strategy was applied to refine the test set to generate a more reliable label, and the model was evaluated under different types of classification granularities.ResultsThis deep learning model achieved top-3 accuracies of over 90% in all classification types, with an average AUC of 0.95. The Cohen’s Kappa demonstrated good agreement between model prediction and the test set.ConclusionsThis deep learning model can achieve performance comparable to that of human experts and has the potential to become a valuable tool for dental education and various applications in accurately identifying isolated tooth.

A Systematic Review of Using Deep Learning in Aphasia: Challenges and Future Directions

In this systematic literature review, the intersection of deep learning applications within the aphasia domain is meticulously explored, acknowledging the condition’s complex nature and the nuanced challenges it presents for language comprehension and expression. By harnessing data from primary databases and employing advanced query methodologies, this study synthesizes findings from 28 relevant documents, unveiling a landscape marked by significant advancements and persistent challenges. Through a methodological lens grounded in the PRISMA framework (Version 2020) and Machine Learning-driven tools like VosViewer (Version 1.6.20) and Litmaps (Free Version), the research delineates the high variability in speech patterns, the intricacies of speech recognition, and the hurdles posed by limited and diverse datasets as core obstacles. Innovative solutions such as specialized deep learning models, data augmentation strategies, and the pivotal role of interdisciplinary collaboration in dataset annotation emerge as vital contributions to this field. The analysis culminates in identifying theoretical and practical pathways for surmounting these barriers, highlighting the potential of deep learning technologies to revolutionize aphasia assessment and treatment. This review not only consolidates current knowledge but also charts a course for future research, emphasizing the need for comprehensive datasets, model optimization, and integration into clinical workflows to enhance patient care. Ultimately, this work underscores the transformative power of deep learning in advancing aphasia diagnosis, treatment, and support, heralding a new era of innovation and interdisciplinary collaboration in addressing this challenging disorder.

Synthetic Data for Deep Learning in Computer Vision & Medical Imaging: A Means to Reduce Data Bias

Deep-learning (DL) performs well in computer-vision and medical-imaging automated decision-making applications. A bottleneck of DL stems from the large amount of labelled data required to train accurate models that generalise well. Data scarcity and imbalance are common problems in imaging applications that can lead DL models towards biased decision making. A solution to this problem is synthetic data. Synthetic data is an inexpensive substitute to real data for improved accuracy and generalisability of DL models. This survey reviews the recent methods published in relation to the creation and use of synthetic data for computer-vision and medical-imaging DL applications. The focus will be on applications that utilised synthetic data to improve DL models by either incorporating an increased diversity of data that is difficult to obtain in real life, or by reducing a bias caused by class imbalance. Computer-graphics software and generative networks are the most popular data generation techniques encountered in the literature. We highlight their suitability for typical computer-vision and medical-imaging applications, and present promising avenues for research to overcome their computational and theoretical limitations.

Application of deep reinforcement learning in electricity demand response market: Demand response decision-making of load aggregator

With the large-scale integration of renewable energy generation, developing the potential of demand response is becoming more and more significant. However, the number of consumers is huge, the electricity consumption behaviors are various and the information of consumers is incomplete, thus there are great difficulties in modeling and processing demand response by the conventional mathematical methods. Therefore, how to accurately predict the response behaviors of consumers and select the appropriate consumers for demand response is worthy of in-depth discussion. In this paper, the decision-making method of the load aggregators who participant in demand response market on behalf of the small and medium-sized consumers based on Deep Q-network algorithm is proposed, supporting the aggregators to properly guide the potential demand response consumers. At the same time, a dynamic consumer classification method based on self-organizing maps algorithm is also proposed, supporting the aggregators to accurately predict the response behaviors of the consumers. Simulation results show that the proposed method can effectively realize the classification of the consumers and result in a more beneficial demand response.•The proposed method does not require complete information, such as demand response behavior parameters of the consumers.•Self-organizing maps can realize a dynamic classification of demand response consumers as the case may be.•DQN algorithm can effectively realize the demand response decision-making of aggregators under the condition of incomplete information.

Application of deep learning for the analysis of stomata: A review of current methods and future directions.

Plant physiology and metabolism relies on the function of stomata, structures on the surface of above ground organs, which facilitate the exchange of gases with the atmosphere. The morphology of the guard cells and corresponding pore which make up the stomata, as well as the density (number per unit area) are critical in determining overall gas exchange capacity. These characteristics can be quantified visually from images captured using microscopes, traditionally relying on time-consuming manual analysis. However, deep learning (DL) models provide a promising route to increase the throughput and accuracy of plant phenotyping tasks, including stomatal analysis. Here we review the published literature on the application of DL for stomatal analysis. We discuss the variation in pipelines used; from data acquisition, pre-processing, DL architecture and output evaluation to post processing. We introduce the most common network structures, the plant species that have been studied, and the measurements that have been performed. Through this review, we hope to promote the use of DL methods for plant phenotyping tasks and highlight future requirements to optimise uptake; predominantly focusing on the sharing of datasets and generalisation of models as well as the caveats associated with utilising image data to infer physiological function.

MCAD: Multi-classification anomaly detection with relational knowledge distillation

With the wide application of deep learning in anomaly detection (AD), industrial vision AD has achieved remarkable success. However, current AD usually focuses on anomaly localization and rarely investigates anomaly classification. Furthermore, anomaly classification is currently requested for quality management and anomaly reason analysis. Therefore, it is essential to classify anomalies while improving the accuracy of AD. This paper designs a novel multi-classification AD (MCAD) framework to achieve high-accuracy AD with an anomaly classification function. In detail, the proposal model based on relational knowledge distillation consists of two components. The first one employs a teacher–student AD model, utilizing a relational knowledge distillation approach to transfer the interrelationships of images. The teacher–student critical layer feature activation values are used in the knowledge transfer process to achieve anomaly detection. The second component realizes anomaly multi-classification using the lightweight convolutional neural network. Our proposal has achieved 98.95, 96.04, and 92.94% AUROC AD results on MNIST, FashionMNIST, and CIFAR10 datasets. Meanwhile, we earn 97.58 and 98.10% AUROC for AD and localization in the MVTecAD dataset. The average classification accuracy of anomaly classification has reached 76.37% in fifteen categories of the MVTec-AD dataset. In particular, the classification accuracy of the leather category has gained 95.24%. The results on the MVTec-AD dataset show that MCAD achieves excellent detection, localization, and classification results.

Transformative Impact of Deep Learning in Stock Market Decision-Making: A Comparative Study of Convolutional Neural Networks

This research delves into the transformative impact of deep learning, specifically Convolutional Neural Networks (CNNs) such as VGG16, ResNet50, and InceptionV3, on organizational management and business intelligence. The study follows a comprehensive methodology, emphasizing the importance of high-quality datasets in leveraging deep learning for enhanced decision-making. Results demonstrate the superior performance of CNN models over traditional algorithms, with CNN (VGG19) achieving an accuracy rate of 89.45%. The findings underscore the potential of deep learning in extracting meaningful insights from complex data, offering a paradigm shift in optimizing various organizational processes. The article concludes by emphasizing the significance of investing in infrastructure and expertise for successful CNN integration, ensuring ethical considerations, and addressing data privacy concerns. This research contributes to the growing discourse on the application of deep learning in organizational management, providing a valuable resource for businesses navigating the dynamic landscape of the global market.

GraphEGFR: Multi-task and transfer learning based on molecular graph attention mechanism and fingerprints improving inhibitor bioactivity prediction for EGFR family proteins on data scarcity.

The proteins within the human epidermal growth factor receptor (EGFR) family, members of the tyrosine kinase receptor family, play a pivotal role in the molecular mechanisms driving the development of various tumors. Tyrosine kinase inhibitors, key compounds in targeted therapy, encounter challenges in cancer treatment due to emerging drug resistance mutations. Consequently, machine learning has undergone significant evolution to address the challenges of cancer drug discovery related to EGFR family proteins. However, the application of deep learning in this area is hindered by inherent difficulties associated with small-scale data, particularly the risk of overfitting. Moreover, the design of a model architecture that facilitates learning through multi-task and transfer learning, coupled with appropriate molecular representation, poses substantial challenges. In this study, we introduce GraphEGFR, a deep learning regression model designed to enhance molecular representation and model architecture for predicting the bioactivity of inhibitors against both wild-type and mutant EGFR family proteins. GraphEGFR integrates a graph attention mechanism for molecular graphs with deep and convolutional neural networks for molecular fingerprints. We observed that GraphEGFR models employing multi-task and transfer learning strategies generally achieve predictive performance comparable to existing competitive methods. The integration of molecular graphs and fingerprints adeptly captures relationships between atoms and enables both global and local pattern recognition. We further validated potential multi-targeted inhibitors for wild-type and mutant HER1 kinases, exploring key amino acid residues through molecular dynamics simulations to understand molecular interactions. This predictive model offers a robust strategy that could significantly contribute to overcoming the challenges of developing deep learning models for drug discovery with limited data and exploring new frontiers in multi-targeted kinase drug discovery for EGFR family proteins.

EARTHQUAKE PREDICTION USING ARTIFICIAL INTELLIGENCE IN THE FERGHANA DEPRESSION (UZBEKISTAN)

Solving the problem of predicting earthquakes faces difficulties of both theoretical and practical nature. The reason is that the occurrence of earthquakes depends on many factors, which give rise to various anomalies that are used as precursors. However, because of the complexity of the earthquake process and the unavailability of much information about the detailed structure of the Earth's crust, a small number of them can accurately indicate future seismic events. The results of the application of machine learning and deep learning give hope for the possibility of obtaining more accurate information about future strong earthquakes if disparate factors are combined. To determine the most important signs of an earthquake and determine the spatial location of strong earthquakes in a specific seismically active territory of Uzbekistan, namely, in the Fergana depression, the Cora 3, Cora 4, random forest algorithms of machine learning and LSTM, ANN architectures of deep learning were implemented.

A deep learning approach for Maize Lethal Necrosis and Maize Streak Virus disease detection

Maize is an important crop cultivated in Sub-Saharan Africa, essential for food security. However, its cultivation faces significant challenges due to debilitating diseases such as Maize Lethal Necrosis (MLN) and Maize Streak Virus (MSV), which can lead to severe yield losses. Traditional plant disease diagnosis methods are often time-consuming and prone to errors, necessitating more efficient approaches. This study explores the application of deep learning, specifically Convolutional Neural Networks (CNNs), in the automatic detection and classification of maize diseases. We investigate six architectures: Basic CNN, EfficientNet V2 B0 and B1, LeNet-5, VGG-16, and ResNet50, using a dataset of 15344 images comprising MSV, MLN, and healthy maize leaves. Additionally, We performed hyperparameter tuning to improve the performance of the models and Gradient-weighted Class Activation Mapping (Grad-CAM) for model interpretability. Our results show that the EfficientNet V2 B0 model demonstrated an accuracy of 99.99% in distinguishing between healthy and disease-infected plants. The results of this study contribute to the advancement of AI applications in agriculture, particularly in diagnosing maize diseases within Sub-Saharan Africa.

The application of deep learning in abdominal trauma diagnosis by CT imaging

BackgroundAbdominal computed tomography (CT) scan is a crucial imaging modality for creating cross-sectional images of the abdominal area, particularly in cases of abdominal trauma, which is commonly encountered in traumatic injuries. However, interpreting CT images is a challenge, especially in emergency. Therefore, we developed a novel deep learning algorithm-based detection method for the initial screening of abdominal internal organ injuries.MethodsWe utilized a dataset provided by the Kaggle competition, comprising 3,147 patients, of which 855 were diagnosed with abdominal trauma, accounting for 27.16% of the total patient population. Following image data pre-processing, we employed a 2D semantic segmentation model to segment the images and constructed a 2.5D classification model to assess the probability of injury for each organ. Subsequently, we evaluated the algorithm’s performance using 5k-fold cross-validation.ResultsWith particularly noteworthy performance in detecting renal injury on abdominal CT scans, we achieved an acceptable accuracy of 0.932 (with a positive predictive value (PPV) of 0.888, negative predictive value (NPV) of 0.943, sensitivity of 0.887, and specificity of 0.944). Furthermore, the accuracy for liver injury detection was 0.873 (with PPV of 0.789, NPV of 0.895, sensitivity of 0.789, and specificity of 0.895), while for spleen injury, it was 0.771 (with PPV of 0.630, NPV of 0.814, sensitivity of 0.626, and specificity of 0.816).ConclusionsThe deep learning model demonstrated the capability to identify multiple organ injuries simultaneously on CT scans and holds potential for application in preliminary screening and adjunctive diagnosis of trauma cases beyond abdominal injuries.

Application of deep learning and inline holography to estimate the droplet size distribution

We examine five machine learning-based architectures to estimate the droplet size distributions obtained using digital inline holography. The architectures, namely, U-Net, R2 U-Net, Attention U-Net, V-Net, and Residual U-Net, are trained using synthetic holographic images. Our assessment focuses on evaluating the training, validation, and prediction performance of these architectures. Our results indicate that U-Net and R2 U-Net are the most proficient architectures, consistently demonstrating superior performance trends and achieving the highest Intersection Over Union (IOU) scores among the five architectures studied. We employ additional training using experimental holographic images for the two top-performing architectures to validate their efficacy further. Subsequently, they are employed to segment an experimental dataset illustrating the bag breakup phenomenon, facilitating the extraction of size distribution. The extracted size distribution from U-Net and R2 U-Net segmentation is then compared with the analytical model proposed by Jackiw and Ashgriz (2022) by employing the gamma and log-normal distributions. Our findings indicate that the gamma distribution provides a more accurate prediction of the multi-modal size distribution than the log-normal distribution owing to its long exponential tail. The present study offers valuable insights into the effectiveness of machine learning architectures in estimating particle/droplet sizes, highlighting their practical application in real-world experimental scenarios.

Ensemble learning for retinal disease recognition under limited resources.

Retinal optical coherence tomography (OCT) images provide crucial insights into the health of the posterior ocular segment. Therefore, the advancement of automated image analysis methods is imperative to equip clinicians and researchers with quantitative data, thereby facilitating informed decision-making. The application of deep learning (DL)-based approaches has gained extensive traction for executing these analysis tasks, demonstrating remarkable performance compared to labor-intensive manual analyses. However, the acquisition of retinal OCT images often presents challenges stemming from privacy concerns and the resource-intensive labeling procedures, which contradicts the prevailing notion that DL models necessitate substantial data volumes for achieving superior performance. Moreover, limitations in available computational resources constrain the progress of high-performance medical artificial intelligence, particularly in less developed regions and countries. This paper introduces a novel ensemble learning mechanism designed for recognizing retinal diseases under limited resources (e.g., data, computation). The mechanism leverages insights from multiple pre-trained models, facilitating the transfer and adaptation of their knowledge to retinal OCT images. This approach establishes a robust model even when confronted with limited labeled data, eliminating the need for an extensive array of parameters, as required in learning from scratch. Comprehensive experimentation on real-world datasets demonstrates that the ensemble models constructed by the proposed ensemble method show superior performance over the baseline models under sparse labeled data, especially the triple ensemble model, which achieves the accuracy of 92.06%, which is 8.27%, 7.99%, and 11.14% better than the three baseline models, respectively. In addition, compared with the three baseline models learned from scratch, the triple ensemble model has fewer trainable parameters, only 3.677M, which is lower than the three baseline models of 8.013M, 4.302M, and 20.158M, respectively.