Two-stage Deep Learning Research Articles

Two-stage deep reinforcement learning method for agile optical satellite scheduling problem

Two-stage deep reinforcement learning method for agile optical satellite scheduling problem

A two-stage deep-learning model for determination of the contact of mandibular third molars with the mandibular canal on panoramic radiographs.

This study aimed to assess the accuracy of a two-stage deep learning (DL) model for (1)detecting mandibular third molars (MTMs) and the mandibular canal (MC), and (2)classifying the anatomical relationship between these structures (contact/no contact) on panoramic radiographs. MTMs and MCs were labeled on panoramic radiographs by a calibrated examiner using bounding boxes. Each bounding box contained MTM and MC on one side. The relationship of MTMs with the MC was assessed on CBCT scans by two independent examiners without the knowledge of the condition of MTM and MC on the corresponding panoramic image, and dichotomized as contact/no contact. Data were split into training, validation, and testing sets with a ratio of 80:10:10. Faster R-CNN was used for detecting MTMs and MCs and ResNeXt for classifying their relationship. AP50 and AP75 were used as outcomes for detecting MTMs and MCs, and accuracy, precision, recall, F1-score, and the area-under-the-receiver-operating-characteristics curve (AUROC) were used to assess classification performance. The training and validation of the models were conducted using the Python programming language with the PyTorch framework. Three hundred eighty-seven panoramic radiographs were evaluated. MTMs were present bilaterally on 232 and unilaterally on 155 radiographs. In total, 619 images were collected which included MTMs and MCs. AP50 and AP75 indicating accuracy for detecting MTMs and MCs were 0.99 and 0.90 respectively. Classification accuracy, recall, specificity, F1-score, precision, and AUROC values were 0.85, 0.85, 0.93, 0.84, 0.86, and 0.91, respectively. DL can detect MTMs and MCs and accurately assess their anatomical relationship on panoramic radiographs.

Enhancing Alaskan wildfire prediction and carbon flux estimation: a two-stage deep learning approach within a process-based model

Abstract Wildfires in boreal forests release substantial amounts of carbon into the atmosphere. However, current land-surface models are limited in their representation of fire processes, including their ignition and spread. This study thus developed FireDL, a novel data-driven machine-learning model for the prediction of natural wildfires, and combined it with a land-surface model to better understand the impact of fire on carbon fluxes. FireDL has a two-stage deep learning structure that sequentially combines a long short-term memory (LSTM) algorithm and an artificial neural network (ANN). Preliminary random forest analysis identified fire duration as an important factor in predicting the burned area. Thus, in FireDL, the LSTM algorithm was employed to predict fire occurrence and duration, utilizing lightning, vegetation, and climate datasets. Subsequently, the ANN predicted the total burned area using the LTSM-derived fire duration predictions and climate datasets as input. FireDL produced a robust performance in predicting large fires (>10 000 ha), achieving a correlation coefficient of 0.72. The daily-scaled burned area predictions derived from FireDL were integrated into the Community Land Model version 5—Biogeochemistry (CLM5-BGC) to produce CLM5-BGC-FireDL. This integration considerably improved carbon emission estimations. Notably, the total net ecosystem exchange (NEE) estimated using CLM5-BGC-FireDL in 2019, the year with the highest recorded burned area during our study, was twice that estimated using the standard CLM5-BGC. Discrepancies in the NEE can significantly influence atmospheric CO2 levels, highlighting the importance of our fire prediction model in forecasting the burned area and carbon emissions. The use of FireDL with future climate scenarios is thus anticipated to yield valuable insights into ecosystem management and climate change mitigation strategies.

UCapsNet: A Two-Stage Deep Learning Model Using U-Net and Capsule Network for Breast Cancer Segmentation and Classification in Ultrasound Imaging

Background/Objectives: Breast cancer remains one of the biggest health challenges for women worldwide, and early detection can be truly lifesaving. Although ultrasound imaging is commonly used to detect tumors, the images are not always of sufficient quality, and, thus, traditional U-Net models often miss the finer details needed for accurate detection. This outcome can result in lower accuracy, making early and precise diagnosis more difficult. Methods: This study presents an enhanced U-Net model integrated with a Capsule Network (called UCapsNet) to overcome the limitations of conventional techniques. Our approach improves segmentation by leveraging higher filter counts and skip connections, while the capsule network enhances classification by preserving spatial hierarchies through dynamic routing. The proposed UCapsNet model operates in two stages: first, it segments tumor regions using an enhanced U-Net, followed by a classification of the segmented images with the capsule network. Results: We have tested our model against well-known pre-trained models, including VGG-19, DenseNet, MobileNet, ResNet-50, and Xception. By properly addressing the limitations found in previous studies and using a capsule network trained on the Breast Ultrasound Image (BUSI) dataset, our model resulted in top-achieving impressive precision, recall, and accuracy rates of 98.12%, 99.52%, and 99.22%, respectively. Conclusions: By combining the U-Net’s powerful segmentation capabilities with the capsule network’s high classification accuracy, UCapsNet boosts diagnostic precision and addresses key weaknesses in existing methods. The findings demonstrate that the proposed model is not only more effective in detecting tumors but also more reliable for practical applications in clinical settings.

A two-stage deep learning architecture for detection global coastal and offshore submesoscale ocean eddy using SDGSAT-1 multispectral imagery

Submesoscale ocean eddies are essential oceanic phenomenon that control and influence the ocean energy cascade. Most existing eddy detection methods rely on low-resolution satellite altimeter data, which fail to capture submesoscale ocean features and oceanographic phenomena in shallow water. Introducing high-resolution multispectral data can alleviate these problems, yet it has been largely overlooked. A generalized and efficient deep learning architecture that combines developments in deep learning with Sustainable Development Goals Science Satellite 1 (SDGSAT-1) multispectral data from earth observations offers a potential pathway for more fine detection of ocean eddies. Considering that oceanic eddy exhibits spatially sparse characteristics on high-resolution remote sensing scenes, the oceanic eddy detection (OED) model suitable for global coastal and offshore regions is divided into two stages: eddy information judgment and eddy position determination. Correspondingly, SDGSAT-1 multispectral data from November 2021 to December 2022 were carried out to construct two submesoscale eddy datasets for training and testing each stage model. The union validation of multiple metrics demonstrates that the proposed OED model and its stage models achieve state-of-the-art (SOTA) performance, especially in optically complex coastal and offshore waters. We applied the model to real-world scenes captured by SDGSAT-1 in 2023, and found that the detected results were mainly located at the water depth below 200 m. The authenticity of the recognition results is validated using sea surface chlorophyll concentration, temperature, and topography data, indicating that the OED model has achieved remarkable effectiveness under various sea conditions. In addition, the temporal distributions and statistical characteristics of detected submesoscale eddies are analyzed over an extended period (November 2021 to November 2023). Finally, HISEA-2, Landsat-9, and Sentinel-2 served as testing grounds to validate the generalization of the proposed methodology, with experimental results demonstrating that the OED model possesses significant developmental potential for multi-source remote sensing data. This paper presents a comprehensive deep learning framework for the global-scale detection of submesoscale eddies and underscores the pivotal role of high-resolution multispectral imagery as an innovative data source for global coastal and offshore eddy identification.

Two-stage deep learning framework for occlusal crown depth image generation

The generation of depth images of occlusal dental crowns is complicated by the need for customization in each case. To decrease the workload of skilled dental technicians, various computer vision models have been used to generate realistic occlusal crown depth images with definite crown surface structures that can ultimately be reconstructed to three-dimensional crowns and directly used in patient treatment. However, it has remained difficult to generate images of the structure of dental crowns in a fluid position using computer vision models. In this paper, we propose a two-stage model for generating depth images of occlusal crowns in diverse positions. The model is divided into two parts: segmentation and inpainting to obtain both shape and surface structure accuracy. The segmentation network focuses on the position and size of the crowns, which allows the model to adapt to diverse targets. The inpainting network based on a GAN generates curved structures of the crown surfaces based on the target jaw image and a binary mask made by the segmentation network. The performance of the model is evaluated via quantitative metrics for the area detection and pixel-value metrics. Compared to the baseline model, the proposed method reduced the MSE score from 0.007001 to 0.002618 and increased DICE score from 0.9333 to 0.9648. It indicates that the model showed better performance in terms of the binary mask from the addition of the segmentation network and the internal structure through the use of inpainting networks. Also, the results demonstrated an improved ability of the proposed model to restore realistic details compared to other models.

Towards the development of a ‘land-river-lake’ two-stage deep learning model for water quality prediction and its application in a large plateau lake

Accurate prediction of water quality that is essential for effective lake management requires a comprehensive understanding of influential factors in the ’land-river-lake’ continuum. We develop a novel deep learning model employing a Long Short-Term Memory (LSTM) model for predicting water quality, and apply it for a large plateau lake, Lake Dianchi. The model comprises two stages: the first establishes a correlation between land and rivers by predicting nutrient loads, incorporating socioeconomic and meteorological features. The second establishes a correlation between feeding rivers and the lake by predicting nutrient concentrations, taking water quality, meteorological conditions, riverine import, and agricultural Non-Point Source (NPS) loads into consideration. Compared to single-stage deep learning models developed using classical methods, the two-stage model reduces Mean Absolute Percentage Error (MAPE), Mean Absolute Error (MAE), and Root Mean Square Error (RMSE), indicating enhanced prediction accuracy. Shapley Additive Explanation (SHAP) analysis is further employed to identify key contributing features. Precipitation emerges as the primary feature in the first stage, while in the second stage, disparities in the most contributing features for two zones of the lake are revealed due to dam isolation and water diversion projects. In the main zone, the primary predicting features are temperature and agricultural NPS features, while in the dam-isolated zone, Secchi depth is the key predictor, highlighting spatial–temporal variations in driving features. Our model contributes a novel deep learning framework and valuable insights for water quality management in large lakes, emphasizing the importance of considering the dynamic interplay of features or factors in the ’land-river-lake’ system.

Research on Rail Surface Defect Detection Based on Improved CenterNet

Rail surface defect detection is vital for railway safety. Traditional methods falter with varying defect sizes and complex backgrounds, while two-stage deep learning models, though accurate, lack real-time capabilities. To overcome these challenges, we propose an enhanced one-stage detection model based on CenterNet. We replace ResNet with ResNeXt and implement a multi-branch structure for better low-level feature extraction. Additionally, we integrate SKNet attention mechanism with the C2f structure from YOLOv8, improving the model’s focus on critical image regions and enhancing the detection of minor defects. We also introduce an elliptical Gaussian kernel for size regression loss, better representing the aspect ratio of rail defects. This approach enhances detection accuracy and speeds up training. Our model achieves a mean accuracy (mAP) of 0.952 on the rail defects dataset, outperforming other models with a 6.6% improvement over the original and a 35.5% increase in training speed. These results demonstrate the efficiency and reliability of our method for rail defect detection.

LEVERAGING DEEP LEARNING FOR IMPROVED CYCLONE FORCASTING

Tropical cyclones (TCs) are frequently rarest but one of the foremost dangerous weather phenomena to arise over tropical oceans, usually creating huge issues with nearly ninety storms globally annually. Early detection and monitoring of TCs are necessary for the early warning of areas in danger. Since these storms originate over the open ocean and frequently out at sea many miles of continental regions, they are detected using remote sensing. In this paper, we propose an automatic method to identify the formation of TCs through satellite images combining a deep learning architecture. The methodology is based on a two-stage deep learning framework: Mask CNN as detector stage—second wind speed filter and lastly Convolutional Neural Network (CNN) for classification. The best possible performance is obtained by hyperparameter optimization using Bayesian Optimization across the pipeline. Results show that the proposed method achieves a precision, specificity, and accuracy of 97.10%, 97.59%, and 86.55% in test images respectively KEYWORDS— Convolutional Neural Networks (CNN), Deep Learning, Cyclone Detection and Tracking, Tropical cyclone Intensity.

Drones in Action: A Comprehensive Analysis of Drone-Based Monitoring Technologies

Unmanned aerial vehicles (UAVs), commonly referred to as drones, are extensively employed in various real-time applications, including remote sensing, disaster management and recovery, logistics, military operations, search and rescue, law enforcement, and crowd monitoring and control, owing to their affordability, rapid processing capabilities, and high-resolution imagery. Additionally, drones mitigate risks associated with terrorism, disease spread, temperature fluctuations, crop pests, and criminal activities. Consequently, this paper thoroughly analyzes UAV-based surveillance systems, exploring the opportunities, challenges, techniques, and future trends of drone technology. It covers common image preprocessing methods for drones and highlights notable one- and two-stage deep learning algorithms used for object detection in drone-captured images. The paper also offers a valuable compilation of online datasets containing drone-acquired photographs for researchers. Furthermore, it compares recent UAV-based imaging applications, detailing their purposes, descriptions, findings, and limitations. Lastly, the paper addresses potential future research directions and challenges related to drone usage

Spatiotemporal analysis using deep learning and fuzzy inference for evaluating broiler activities

Observing poultry activity is crucial for assessing their health status; however, the inspection process is often time-consuming and labor-intensive, particularly in cases involving large numbers of chickens. Inexperienced breeders may also misjudge their activity levels, potentially missing opportunities for prevention and treatment. This study integrates traditional video surveillance with an advanced monitoring system to identify various broiler behaviors in a breeding environment. A two-stage deep learning approach is employed: in the first stage, the broilers are detected, and in the second stage, five key body points (head, abdomen, two legs, and tail) are identified. A skeleton-based model is then developed centered around the abdomen, with six angles calculated using trigonometric methods. These angles are analyzed by a long short-term memory network to estimate behaviors such as “Standing”, “Walking”, “Resting”, “Eating”, “Preening”, and “Flapping”, selecting the behavior with the highest probability. Dual-layer fuzzy logic inference systems were used to evaluate the proportion of time broilers spent in static versus dynamic states, providing a robust determination of their activity levels. Validated in a mixed-sex breeding environment, the proposed system achieved accuracies of at least 85.2% for identifying broiler type, 79.2% for identifying body parts, and 50.8% for identifying behaviors. The activity level evaluation results were consistent with those conducted by experienced poultry experts.

Unveiling consumer preferences: A two-stage deep learning approach to enhance accuracy in multi-channel retail sales forecasting

In the dynamic and turbulent business environment, sales forecasting for multi-channel retailers has become increasingly intricate, particularly with the shift from traditional brick-and-mortar channels to a diverse range of distribution channels. This transition not only complicates forecasting challenges but also highlights the crucial role of utilizing online traceable consumer purchase data to discern consumer preferences for stores and products and enhance sales forecasting accuracy. This paper proposes a two-stage deep learning approach based on the Online Channel Consumer Preference Heterogram and Multi-Head Attention mechanism (OCCPH-MHA). In the first stage, the model identifies potential consumer group preferences based on individual purchasing behavior. In the second stage, it seamlessly integrates this identified feature with time-series demand data using a global–local attention mechanism, thereby facilitating multi-step forecasting. This study’s robust validation involves testing the model on the dataset from a multi-channel retail restaurant company, showcasing its prowess in significantly improving the precision of sales forecasting. This not only substantiates the model’s effectiveness but also underscores the importance of consumer group preferences, as it contributes to a comprehensive framework. This framework, focused on tracking the preferences of potential consumer groups, emerges as a valuable tool that collectively refines and optimizes the sales forecasting process for both industry practitioners and researchers.

OCR-Diff: A Two-Stage Deep Learning Framework for Optical Character Recognition Using Diffusion Model in Industrial Internet of Things

OCR-Diff: A Two-Stage Deep Learning Framework for Optical Character Recognition Using Diffusion Model in Industrial Internet of Things

Enhanced deep learning models for automatic fish species identification in underwater imagery

Underwater cameras are crucial in marine ecology, but their data management needs automatic species identification. This study proposes a two-stage deep learning approach. First, the Unsharp Mask Filter (UMF) preprocesses images. Then, an enhanced region-based fully convolutional network (R–FCN) detects fish using two-order integrals for position-sensitive score maps and precise region of interest (PS-Pr-RoI) pooling for accuracy. The second stage integrates ShuffleNetV2 with the Squeeze and Excitation (SE) module, forming the Improved ShuffleNetV2 model, enhancing classification focus. Hyperparameters are optimized with the Enhanced Northern Goshawk Optimization Algorithm (ENGO). The improved R–FCN model achieves 99.94 % accuracy, 99.58 % precision and recall, and a 99.27 % F-measure on the Fish4knowledge dataset. Similarly, the ENGO-based ShuffleNetV2 model, evaluated on the same dataset, shows 99.93 % accuracy, 99.19 % precision, 98.29 % recall, and a 98.71 % F-measure, highlighting its superior classification accuracy.

Deep learning for intracranial aneurysm segmentation using CT angiography

Objective. This study aimed to employ a two-stage deep learning method to accurately detect small aneurysms (4–10 mm in size) in computed tomography angiography images. Approach. This study included 956 patients from 6 hospitals and a public dataset obtained with 6 CT scanners from different manufacturers. The proposed method consists of two components: a lightweight and fast head region selection (HRS) algorithm and an adaptive 3D nnU-Net network, which is used as the main architecture for segmenting aneurysms. Segments generated by the deep neural network were compared with expert-generated manual segmentation results and assessed using Dice scores. Main Results. The area under the curve (AUC) exceeded 79% across all datasets. In particular, the precision and AUC reached 85.2% and 87.6%, respectively, on certain datasets. The experimental results demonstrated the promising performance of this approach, which reduced the inference time by more than 50% compared to direct inference without HRS. Significance. Compared with a model without HRS, the deep learning approach we developed can accurately segment aneurysms by automatically localizing brain regions and can accelerate aneurysm inference by more than 50%.

Thoracic computed tomography (CT) image-based identification and severity classification of COVID-19 cases using vision transformer (ViT)

In this research, we developed a two-stage deep learning (DL) model using Vision Transformer (ViT) to detect COVID-19 and assess its severity from thoracic CT images. In the first stage, we utilized a pre-trained ViT model (ViT_B/32) and a custom CNN model to classify CT images as COVID-19 or non-COVID-19. The ViT model achieved superior performance with a fivefold cross-validated accuracy of 99.7%, compared to the custom CNN’s 98%. In the second stage, we employed a ViT-based U-Net model (Vision Transformer for Biomedical Image Segmentation, VITBIS) to segment lung and infection regions in COVID-19 positive CT images, determining the infection severity. This model uses transformers with attention mechanisms in both the encoder and decoder. The lung segmentation network achieved an Intersection Over Union (IOU) of 95.8% and a sensitivity of 99.67%, while the lesion segmentation network attained an IOU of 94% and a sensitivity of 98.3%.

Unraveling Brain Synchronisation Dynamics by Explainable Neural Networks using EEG Signals: Application to Dyslexia Diagnosis

The electrical activity of the neural processes involved in cognitive functions is captured in EEG signals, allowing the exploration of the integration and coordination of neuronal oscillations across multiple spatiotemporal scales. We have proposed a novel approach that combines the transformation of EEG signal into image sequences, considering cross-frequency phase synchronisation (CFS) dynamics involved in low-level auditory processing, with the development of a two-stage deep learning model for the detection of developmental dyslexia (DD). This deep learning model exploits spatial and temporal information preserved in the image sequences to find discriminative patterns of phase synchronisation over time achieving a balanced accuracy of up to 83%. This result supports the existence of differential brain synchronisation dynamics between typical and dyslexic seven-year-old readers. Furthermore, we have obtained interpretable representations using a novel feature mask to link the most relevant regions during classification with the cognitive processes attributed to normal reading and those corresponding to compensatory mechanisms found in dyslexia.Graphical

Research on green supply chain finance risk identification based on two-stage deep learning

As a resonance product between financial services and the upgrading of the green industry, green supply chain finance has garnered extensive attention in the process of ecological civilization construction. Effectively promoting the green transformation of small and medium-sized enterprises and achieving the "dual carbon" goals necessitate the avoidance of corporate green risks. However, the complex interdependence and information asymmetry among green supply chain finance enterprises result in data characteristics such as multi-source small samples and high-dimensional imbalance. To address these issues, this paper proposes a risk assessment model based on two-stage deep learning. In the first stage, we employ Generative Adversarial Network (GAN) to generate minority class default samples, and utilize Stacked Auto-Encoder (SAE) to extract data features with closed-form parameter calculation capability. In the second stage, the obtained features are input into a Deep Neural Network (DNN), and parameter learning and model optimization are conducted through joint training. Finally, to model low-order feature interactions, we integrate the Support Vector Machine (SVM) algorithm. The paper is grounded in the green innovation production of enterprises, collecting financial data of 176 upstream and downstream enterprises and corresponding core enterprise green indicators from 2013 to 2022. Experimental results demonstrate that GAN oversampling technique not only enhances the model's AUC metric but also significantly improves the F1 score. Compared with traditional deep learning methods, the proposed two-stage deep integration model effectively reduces training loss and exhibits superiority in identifying green supply chain finance risks.

Deep reinforcement learning-based scheduling for integrated energy system utilizing retired electric vehicle battery energy storage

Retired electric vehicle batteries (REVBs) retain substantial energy storage capacity, holding great potential for utilization in integrated energy systems. However, the dynamics of supply and demand, alongside battery safety constraints, present challenges to the optimal dispatch of energy. This paper proposes a hybrid system including thermal and electric energy employing REVB as the energy storage component. This system relies on photovoltaics (PV) technology to harness solar energy and convert it into thermal and electrical energy for supplying users. The REVB, as the core component of the system, functions to regulate the allocation of energy by managing its charging and discharging behavior. To explore the optimal energy scheduling strategy, a two-stage deep reinforcement learning (DRL) optimization method is presented. In the first phase, a deep deterministic policy gradient (DDPG) method is introduced to determine the REVB's behavior, and in the second phase, a competitive bidding mechanism is presented to optimally distribute energy. Simulation results, validated with real scenarios and data, achieve a reduction of 35.2 % in energy dissipation and 4.6 % in users' expenditure for purchasing energy compared to standard strategy. Simultaneously, experiments have shown that REVB is nearly as efficient as new battery as energy storage component, with energy wastage only about 1.1 % higher and users' expenditure less than 1 % higher.

A Neural-Network-Based Cost-Effective Method for Initial Weld Point Extraction from 2D Images

This paper presents a novel approach for extracting 3D weld point information using a two-stage deep learning pipeline based on readily available 2D RGB cameras. Our method utilizes YOLOv8s for object detection, specifically targeting vertices, followed by semantic segmentation for precise pixel localization. This pipeline addresses the challenges posed by low-contrast images and complex geometries, significantly reducing costs compared with traditional 3D-based solutions. We demonstrated the effectiveness of our approach through a comparison with a 3D-point-cloud-based method, showcasing the potential for improved speed and efficiency. This research advances the field of automated welding by providing a cost-effective and versatile solution for extracting key information from 2D images.