Advanced Deep Learning Techniques Research Articles

Adaptive convolutional-LSTM neural network with NADAM optimization for intrusion detection in underwater IoT wireless sensor networks

Abstract Underwater IoT networks rely on sensor nodes to continuously monitor and collect real-time data from marine environments. The nodes in the underwater networks face security challenges in terms of intrusions. Intrusions are common in conventional wireless sensor networks as well as in underwater sensor networks. Intrusions like data tampering, node insertion, unauthorized access compromises the security and privacy of the network. The existing Intrusion Detection Systems (IDS) faces limitations while detecting intrusions in Underwater IoT Wireless Sensor Networks (UIoTWSN) due to the dynamic condition of underwater environment. To overcome this, a novel IDS for UIoTWSN is proposed by combining the features of advanced deep learning techniques in an optimized manner. The proposed hybrid model comprises convolutional LSTM network with NADAM optimizer to analyze the spatial and temporal features to detect the intrusions. To handle the dynamic nature of underwater sensor network and to improve the convergence speed of the proposed IDS, NADAM optimizer is integrated in this research work. Experimentations of the proposed model validates the higher accuracy of 96.7%, precision of 94.5% and recall of 95.2% which is better than the conventional techniques.

Innovative load identification with Res-UNet: Integrating phase space reconstruction and physics-informed deep learning

This study developed a specialized load identification model for a specific ocean platform that integrates advanced deep learning techniques with the unique physical characteristics of marine engineering. Key achievements include high-dimensional mapping of data features using phase space reconstruction (PSR) techniques to enhance the predictive capacity of the model through chaos system theory; optimization of ResNet and UNet deep learning networks with attention mechanisms to boost performance; calculation of stiffness, mass, and damping matrices using finite element software; establishment of corresponding state-space equations; and integration of structural dynamics equations with the deep learning network loss function to significantly improve load identification accuracy. In addition, proprietary structural equations for ocean platform structures were developed, underscoring the uniqueness and applicability of the algorithm. The proposed physics-informed Res-UNet model exhibited excellent performance in identifying both periodic and random loads. These improvements were validated through training with finite element simulated data and subsequent experimental applications, demonstrating the effectiveness of the proposed innovative methods.

Improve the Performance of SAR Ship Detectors by Small Object Detection Strategies

Although advanced deep learning techniques have significantly improved SAR ship detection, accurately detecting small ships remains challenging due to their limited size and the few appearance and geometric clues available. In order to solve this problem, we propose several small object detection strategies. The backbone network uses space-to-depth convolution to replace stride and pooling. It reduces information loss during down-sampling. The neck integrates multiple layers of features globally and injects global and local information into different levels. It avoids the inherent information loss of traditional feature pyramid networks and strengthens the information fusion ability without significantly increasing latency. The proposed intersection-of-union considers the center distance and scale of small ships specifically. It reduces the sensitivity of intersection-of-union to positional deviations of small ships, which is helpful for training toward small ships. During training, the smaller the localization loss of small ships, the greater their localization loss gains are. By this, the supervision of small ships is strengthened in the loss function, which can make the losses more biased toward small ships. A series of experiments are conducted on two commonly used datasets, SSDD and SAR-Ship-Dataset. The experimental results show that the proposed method can detect small ships successfully and thus improve the overall performance of detectors.

A Novel Transformer-CNN Approach for Predicting Soil Properties from LUCAS Vis-NIR Spectral Data

Soil, a non-renewable resource, requires continuous monitoring to prevent degradation and support sustainable agriculture. Visible-near-infrared (Vis-NIR) spectroscopy is a rapid and cost-effective method for predicting soil properties. While traditional machine learning methods are commonly used for modeling Vis-NIR spectral data, large datasets may benefit more from advanced deep learning techniques. In this study, based on the large soil spectral library LUCAS, we aimed to enhance regression model performance in soil property estimation by combining Transformer and convolutional neural network (CNN) techniques to predict 11 soil properties (clay, silt, pH in CaCl2, pH in H2O, CEC, OC, CaCO3, N, P, and K). The Transformer-CNN model accurately predicted most soil properties, outperforming other methods (partial least squares regression (PLSR), random forest regression (RFR), support vector machine regression (SVR), Long Short-Term Memory (LSTM), ResNet18) with a 10–24 percentage point improvement in the coefficient of determination (R2). The Transformer-CNN model excelled in predicting pH in CaCl2, pH in H2O, OC, CaCO3, and N (R2 = 0.94–0.96, RPD > 3) and performed well for clay, sand, CEC, P, and K (R2 = 0.77–0.85, 2 < RPD < 3). This study demonstrates the potential of Transformer-CNN in enhancing soil property prediction, although future work should aim to optimize computational efficiency and explore a wider range of applications to ensure its utility in different agricultural settings.

Enhancing Drug-Target Binding Affinity Prediction through Deep Learning and Protein Secondary Structure Integration

Background: Conventional approaches to drug discovery are often characterized by lengthy and costly processes. To expedite the discovery of new drugs, the integration of artificial intelligence (AI) in predicting drug-target binding affinity (DTA) has emerged as a crucial approach. Despite the proliferation of deep learning methods for DTA prediction, many of these methods primarily concentrate on the amino acid sequence of proteins. Yet, the interactions between drug compounds and targets occur within distinct segments within the protein structures, whereas the primary sequence primarily captures global protein features. Consequently, it falls short of fully elucidating the intricate relationship between drugs and their respective targets. Objective: This study aims to employ advanced deep-learning techniques to forecast DTA while incorporating information about the secondary structure of proteins. Methods: In our research, both the primary sequence of protein and the secondary structure of protein were leveraged for protein representation. While the primary sequence played the role of the overarching feature, the secondary structure was employed as the localized feature. Convolutional neural networks and graph neural networks were utilized to independently model the intricate features of target proteins and drug compounds. This approach enhanced our ability to capture drugtarget interactions more effectively Results: We have introduced a novel method for predicting DTA. In comparison to DeepDTA, our approach demonstrates significant enhancements, achieving a 3.9% increase in the Concordance Index (CI) and a remarkable 34% reduction in Mean Squared Error (MSE) when evaluated on the KIBA dataset. Conclusion: In conclusion, our results unequivocally demonstrate that augmenting DTA prediction with the inclusion of the protein's secondary structure as a localized feature yields significantly improved accuracy compared to relying solely on the primary structure.

Makeup-Invariant Faces Recognition Using a Pre-Trained Neural Network, Grasshopper Optimization Algorithm, and Random Forest

Makeup-Invariant Face Recognition is a critical area of research that aims to identify and classify manipulated or forged face images in the field of face recognition systems. Advances in deep learning techniques, such as pre-trained neural networks, have shown promising results in this domain. In this paper, a novel approach for detecting manipulated faces is proposed, combining the strengths of pre-trained neural networks, the Grasshopper Optimization (GOA) Algorithm, and Random Forest classifier. The proposed method utilizes the powerful feature extraction capabilities of pre-trained deep neural networks to capture intricate details and patterns in face images. Subsequently, the GOA algorithm is employed to select optimal features. Additionally, the Random Forest classifier is utilized for effective classification of face images and identifying different individuals based on the selected features. By integrating these three algorithms, the proposed approach demonstrates significant improvements in detecting manipulated faces, achieving higher detection rates and lower false positive rates compared to existing state of the art methods. Simulation results on data from 25 different individuals with manipulated face images show an average accuracy of 97.23%, which represents an enhancement over the compared methods.

Enhancing Paddy Leaf Disease Classification using CNN and MobileNetV2

Paddy farming, a cornerstone of global agriculture, faces significant threats from various diseases that affect the crop yield. This research presents a novel approach for detecting paddy leaf diseases using advanced deep learning techniques, specifically transfer learning with the MobileNetV2 architecture. The methodology involves the utilization of a comprehensive dataset consisting of paddy leaf images across multiple disease classes. Data augmentation was extensively employed to address the limitations posed by the dataset size. Both basic and advanced models were trained, with the advanced model achieving a remarkable validation accuracy of 97%. Additionally, Time-Test Augmentation (TTA) was applied to further enhance the model's performance. This research demonstrates the efficacy of deep learning techniques in agricultural disease detection and highlights potential improvements for future applications.

Research on tool wear condition monitoring based on deep transfer learning and residual network

To address the issues of sample scarcity and insufficient recognition accuracy in existing deep learning models for tool wear monitoring, this study developed a milling tool wear monitoring model that combines transfer learning (TL) and deep residual networks (ResNet). The model uses continuous wavelet transform to convert vibration signals into time-frequency maps, which are then fed into the network model for analysis. ResNet50 was selected as the base feature extraction model, and transfer learning techniques were employed to update the classification layer’s weights, enabling tool wear detection. The model based on ResNet-TL achieved a detection accuracy of 94%, significantly exceeding the threshold for intelligent tool wear recognition. This accomplishment markedly improves the precision and stability of tool wear state monitoring, providing more reliable technical support for tool management in manufacturing processes. Additionally, the method demonstrated superiority in addressing the small sample problem, paving the way for future research in tool wear monitoring. By integrating advanced deep learning techniques with transfer learning, the model not only enhances detection capabilities but also improves adaptability and robustness in practical industrial applications.

Text-guided image-to-sketch diffusion models

Recently, with the continuous advancement of deep learning techniques, research on sketch synthesis has been progressing. However, existing methods still face challenges in generating human-like freehand sketches from real-world natural images at both object and scene levels. To address this, we propose SketchDiffusion, a text-guided freehand sketch synthesis method based on conditional stable diffusion. In SketchDiffusion, we design a novel image enhancing module to efficiently extract high-quality image features. Moreover, we utilize additional guidance from global and local features extracted by a U-shaped diffusion guidance network to control the noise addition and denoising process of the diffusion model, thereby significantly improving controllability and performance in freehand sketch synthesis. Beyond the model architecture, we leverage the designed BLIP-based text generation method to create 70,280 text prompts for foreground, background, and panorama sketch synthesis in the extensive SketchyCOCO dataset, thereby improving the overall effectiveness of model training. Compared to the state-of-the-art methods, our proposed SketchDiffusion has shown an average improvement of over 16.4%, 16.75%, and 12.8% on three quantitative metrics (sketch recognition, sketch-based retrieval, and user perceptual study), respectively. Furthermore, our approach not only excels in synthesizing freehand sketches containing multiple abstract objects but also has multiple applications in supporting human–computer interaction.

Portfolio Prediction using Deep Learning

This research investigates the use of Long Short-Term Memory (LSTM) networks for predicting stock prices and optimizing investment portfolios. Utilizing a dataset comprising the top 30 U.S. companies by market capitalization from 2009-12-31 to 2021-12-31, excluding AbbVie, Meta, and Tesla due to their later market listings, we demonstrate that LSTM models outperform conventional Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN) in generating accuracy and portfolio returns. By efficiently capturing long-term dependencies within stock price data, LSTMs offer more reliable predictions, which are crucial for optimizing investment portfolios. Our methodology involves data collection and preprocessing, model building using LSTM architecture, and evaluating performance using metrics such as mean absolute error (MAE), mean squared error (MSE), root mean squared error (RMSE), mean absolute percentage error (MAPE), and mean percentage error (MPE). The results indicate that LSTM models not only enhance prediction accuracy but also improve portfolio returns compared to equally weighted and market capitalization weighted portfolios. This research provides significant insights into the benefits of using advanced deep learning techniques like LSTMs for financial market predictions and portfolio management.

Quantum Chemistry Dataset with Ground- and Excited-state Properties of 450 Kilo Molecules

Due to rapid advancements in deep learning techniques, the demand for large-volume high-quality datasets grows significantly in chemical research. We developed a quantum-chemistry database that includes 443,106 small organic molecules with sizes up to 10 heavy atoms including C, N, O, and F. Ground-state geometry optimizations and frequency calculations of all compounds were performed at the B3LYP/6-31G* level with the BJD3 dispersion correction, while the excited-state single-point calculations were conducted at the ωB97X-D/6-31G* level. Totally twenty-seven molecular properties, such as geometric, thermodynamic, electronic and energetic properties, were gathered from these calculations. Meanwhile, we also established a comprehensive protocol for the construction of a high-volume quantum-chemistry dataset. Our QCDGE (Quantum Chemistry Dataset with Ground- and Excited-State Properties) dataset contains a substantial volume of data, exhibits high chemical diversity, and most importantly includes excited-state information. This dataset, along with its construction protocol, is expected to have a significant impact on the broad applications of machine learning studies across different fields of chemistry, especially in the area of excited-state research.

Yield estimation from SAR data using patch-based deep learning and machine learning techniques

This study demonstrates how the availability of frequent Synthetic Aperture Radar (SAR) observations has transformed crop yield prediction, a critical component of food security and agricultural practices. SAR observations along with the climatic variables are integrated into an advanced deep learning technique for predicting crop yield. Capitalizing on the unique advantages of high-resolution SAR images, including consistent acquisition schedule, and not being affected by cloud cover and variations between day and night, this research explores new potential in agriculture. Deep learning due to its ability to discern both spatial and temporal relationships within SAR data, captures the salient features from SAR observations to predict the yield of Michigan’s non-irrigated Corn, Soybean, and Winter Wheat.We employed advanced deep learning and established machine learning techniques including patch-based 3D Convolutional Neural Networks (3D-CNNs), Random Forest, Support Vector Machine, and XGBoost to significantly improve the accuracy of yield estimation.Spanning eight years from 2016 to 2023, our analysis underscores the exceptional potential of VH channel of Sentinel-1 SAR data for near accurate yield prediction. Among the methods tested, XGBoost consistently surpassed others in crop yield estimating accuracy, particularly in scenarios with limited reference data. Patch-based 3D-CNNs also demonstrated a remarkable ability to approximate XGBoost’s performance, albeit with a streamlined set of input features. Our study further illuminates the delicate balance required in selecting SAR data resolution, demonstrating the need for careful compromise between reducing noise and preserving crucial data intricacies. Notably, our predictive models showcased formidable precision, predicting yields with a mere 7.5% margin of error a full month prior to harvest. These compelling findings signal the need for continued innovation and integration of deep learning technologies, calling for the enrichment of yield datasets to realize more comprehensive and pinpoint-accurate yield predictions.

Hybrid Diffusion Model for Stable, Affinity-Driven, Receptor-Aware Peptide Generation.

The convergence of biotechnology and artificial intelligence has the potential to transform drug development, especially in the field of therapeutic peptide design. Peptides are short chains of amino acids with diverse therapeutic applications that offer several advantages over small molecular drugs, such as targeted therapy and minimal side effects. However, limited oral bioavailability and enzymatic degradation have limited their effectiveness. With advances in deep learning techniques, innovative approaches to peptide design have become possible. In this work, we demonstrate HYDRA, a hybrid deep learning approach that leverages the distribution modeling capabilities of a diffusion model and combines it with a binding affinity maximization algorithm that can be used for de novo design of peptide binders for various target receptors. As an application, we have used our approach to design therapeutic peptides targeting proteins expressed by Plasmodium falciparum erythrocyte membrane protein 1 (PfEMP1) genes. The ability of HYDRA to generate peptides conditioned on the target receptor's binding sites makes it a promising approach for developing effective therapies for malaria and other diseases.

Enhancing autonomous pavement crack detection: Optimizing YOLOv5s algorithm with advanced deep learning techniques

To enhance the safety and comfort of vehicle travel, detecting pavement cracks is a critical task in road management. This article introduces an advanced single-stage target detection method utilizing the YOLOv5s algorithm to enhance real-time performance and accuracy. Initially, Squeeze-and-Excitation Networks are integrated into the model to facilitate self-learning for improved crack characterization. Subsequently, anchors computed through the K-means clustering algorithm are closely aligned with the fracture dataset, achieving an adaptation rate of 99.9 % and enhancing the recall rate of the model. Furthermore, the inclusion of the SimSPPF module from YOLOv6 diminishes memory usage and expedites detection speed. By replacing the original nearest up-sampling method with transposed convolution, optimization of up-sampling for crack datasets is achieved. Performance assessments reveal that the refined YOLOv5s algorithm attains an F1 score of 91 %, a mean Average Precision (mAP) of 93.6 %, and a 1.54 % increase in frames per second (fps) for pavement crack detection. This enhancement in detection technology signifies a substantial advancement in the maintenance and longevity of road infrastructure.

Rapeseed Flower Counting Method Based on GhP2-YOLO and StrongSORT Algorithm.

Accurately quantifying flora and their respective anatomical structures within natural ecosystems is paramount for both botanical breeders and agricultural cultivators. For breeders, precise plant enumeration during the flowering phase is instrumental in discriminating genotypes exhibiting heightened flowering frequencies, while for growers, such data inform potential crop rotation strategies. Moreover, the quantification of specific plant components, such as flowers, can offer prognostic insights into the potential yield variances among different genotypes, thereby facilitating informed decisions pertaining to production levels. The overarching aim of the present investigation is to explore the capabilities of a neural network termed GhP2-YOLO, predicated on advanced deep learning techniques and multi-target tracking algorithms, specifically tailored for the enumeration of rapeseed flower buds and blossoms from recorded video frames. Building upon the foundation of the renowned object detection model YOLO v8, this network integrates a specialized P2 detection head and the Ghost module to augment the model's capacity for detecting diminutive targets with lower resolutions. This modification not only renders the model more adept at target identification but also renders it more lightweight and less computationally intensive. The optimal iteration of GhP2-YOLOm demonstrated exceptional accuracy in quantifying rapeseed flower samples, showcasing an impressive mean average precision at 50% intersection over union metric surpassing 95%. Leveraging the virtues of StrongSORT, the subsequent tracking of rapeseed flower buds and blossom patterns within the video dataset was adeptly realized. By selecting 20 video segments for comparative analysis between manual and automated counts of rapeseed flowers, buds, and the overall target count, a robust correlation was evidenced, with R-squared coefficients measuring 0.9719, 0.986, and 0.9753, respectively. Conclusively, a user-friendly "Rapeseed flower detection" system was developed utilizing a GUI and PyQt5 interface, facilitating the visualization of rapeseed flowers and buds. This system holds promising utility in field surveillance apparatus, enabling agriculturalists to monitor the developmental progress of rapeseed flowers in real time. This innovative study introduces automated tracking and tallying methodologies within video footage, positioning deep convolutional neural networks and multi-target tracking protocols as invaluable assets in the realms of botanical research and agricultural administration.

COVID-19 severity detection using chest X-ray segmentation and deep learning

COVID-19 has resulted in a significant global impact on health, the economy, education, and daily life. The disease can range from mild to severe, with individuals over 65 or those with underlying medical conditions being more susceptible to severe illness. Early testing and isolation are vital due to the virus’s variable incubation period. Chest radiographs (CXR) have gained importance as a diagnostic tool due to their efficiency and reduced radiation exposure compared to CT scans. However, the sensitivity of CXR in detecting COVID-19 may be lower. This paper introduces a deep learning framework for accurate COVID-19 classification and severity prediction using CXR images. U-Net is used for lung segmentation, achieving a precision of 0.9924. Classification is performed using a Convulation-capsule network, with high true positive rates of 86% for COVID-19, 93% for pneumonia, and 85% for normal cases. Severity assessment employs ResNet50, VGG-16, and DenseNet201, with DenseNet201 showing superior accuracy. Empirical results, validated with 95% confidence intervals, confirm the framework’s reliability and robustness. This integration of advanced deep learning techniques with radiological imaging enhances early detection and severity assessment, improving patient management and resource allocation in clinical settings.

Advancing Fake News Detection with Graph Neural Network and Deep Learning

Abstract In the modern era of digital technology, the rapid distribution of news via social media platforms substantially contributes to the propagation of false information, presenting challenges in upholding the accuracy and reliability of information. This study presents an updated approach that utilizes Graph Neural Networks (GNNs) alongside with advanced deep learning techniques to improve the identification of false information. In contrast to traditional approaches that primarily rely on analyzing text and assessing the credibility of sources, our methodology utilizes the structural information of news propagation networks. This allows for a detailed comprehension of the interconnections and patterns that are indicative of misinformation. By analyzing the intricate, graph-based connections between news items, our approach not only overcomes the constraints of conventional fake news detection methods but also demonstrates significant enhancements in detection accuracy. This paper emphasizes the revolutionary nature of utilizing Graph Neural Networks (GNNs) in the field of fake news detection. It also examines the potential consequences of our research in reducing the propagation of false information. Our model achieved an impressive accuracy rate of 97\%, demonstrating a significant improvement in its ability to identify and classify fake news. The findings highlight the substantial improvement in the ability to detect fake news provided by GNNs in comparison to traditional methods, demonstrating promising growth in the struggle against false information.

Advanced Deep Learning Techniques for Indoor-Outdoor Scene Recognition Integrating CNN and Edge Detection for Enhanced Classification Accuracy in Dynamic Environments

Recognizing indoor-outdoor scenes is an vital portion of computer vision that has an impact on numerous zones, counting driverless driving, virtual reality, and natural following. This article talks approximately a more progressed profound learning framework that employments Convolutional Neural Systems (CNNs) and edge acknowledgment strategies to create classification more precise in settings that alter over time. Distinctive lighting, surfaces, and structure highlights can make it difficult for conventional scene acknowledgment strategies to tell the contrast between complex scenes. We recommend a way to solve these issues that employments the leading parts of both CNNs and edge acknowledgment to urge exact pictures of geometric shapes. The system begins by altering the photographs it gets by utilizing edge acknowledgment strategies, like Canny and Sobel, to discover critical structure edges and lower the clamor. The edge-enhanced pictures are at that point sent to a CNN engineering that was made to recognize scenes. The CNN show is learned on a huge dataset that incorporates a wide run of indoor and open air settings. This makes beyond any doubt that it works well in all sorts of circumstances. A modern layer integration strategy is utilized to connect the CNN's learned highlights with the edge-based highlights. This makes a difference demonstrate tell the contrast between complex scene points of interest superior. It comes about of our tests appear that our combined strategy makes classification much more exact than conventional CNN models. The edge acknowledgment portion truly makes strides the model's capacity to choose up on little changes in picture structures, which leads to more exact forecasts. Moreover, the framework's capacity to adjust to changing situations has been demonstrated by a parcel of testing in a part of distinctive circumstances. Combining profound learning with edge discovery might make scene acknowledgment way better, and the proposed strategy shows how this may work. This may lead to more advanced and viable computer vision frameworks in real-world circumstances that are complicated.

Adaptive Sensing Data Augmentation for Drones Using Attention-Based GAN.

Drones have become essential tools across various industries due to their ability to provide real-time data and perform automated tasks. However, integrating multiple sensors on a single drone poses challenges such as payload limitations and data management issues. This paper proposes a comprehensive system that leverages advanced deep learning techniques, specifically an attention-based generative adversarial network (GAN), to address data scarcity in drone-collected time-series sensor data. By adjusting sensing frequency based on operational conditions while maintaining data resolution, our system ensures consistent and high-quality data collection. The spatiotemporal The attention mechanism within the GAN enhances the generation of synthetic data, filling gaps caused by reduced sensing frequency with realistic data. This approach improves the efficiency and performance of various applications, such as precision agriculture, environmental monitoring, and surveillance. The experimental results demonstrated the effectiveness of our methodology in extending the operational range and duration of drones and providing reliable augmented data utilizing a variety of evaluation metrics. Furthermore, the superior performance of the proposed system was verified by comparing it with various comparative GAN models.

Enhancing Pulmonary Embolism Detection in COVID-19 Patients Through Advanced Deep Learning Techniques.

The intersection of COVID-19 and pulmonary embolism (PE) has posed unprecedented challenges in medical diagnostics. The critical nature of PE and its increased incidence during the pandemic underline the need for improved detection methods. This study evaluates the effectiveness of advanced deep learning techniques in enhancing PE detection in post-COVID-19 patients through Computed Tomography Pulmonary Angiography (CTPA) scans. Using a dataset of 746 anonymized CTPA images from 25 patients, we fine-tuned the state-of-the-art Ultralytics YOLOv8 object detection model, which was trained on 676 images with 1,517 annotated bounding boxes and validated on 70 images with 108 bounding boxes. After 200 epochs of training, which lasted approximately 1.021 hours, the YOLOv8 model demonstrated significant diagnostic proficiency, achieving a mean Average Precision (mAP) of 0.683 at an IoU threshold of 0.50 and a mAP of 0.246 at the IoU range of 0.50:0.95 in the validation dataset. Notably, the model reached a maximum precision of 0.85949 and a maximum recall of 0.81481, though these metrics were observed in separate epochs. These findings emphasize the model's potential for high diagnostic accuracy and offer a promising direction for deploying AI tools in clinical settings, significantly contributing to healthcare innovation and patient care post-pandemic.