Deep Learning Scheme Research Articles

ChemAP: predicting drug approval with chemical structures before clinical trial phase by leveraging multi-modal embedding space and knowledge distillation

Recent studies showed that the likelihood of drug approval can be predicted with clinical data and structure information of drug using computational approaches. Predicting the likelihood of drug approval can be innovative and of high impact. However, models that leverage clinical data are applicable only in clinical stages, which is not very practical. Prioritizing drug candidates and early-stage decision-making in the de novo drug development process is crucial in pharmaceutical research to optimize resource allocation. For early-stage decision-making, we need a computational model that uses only chemical structures. This seemingly impossible task may utilize the predictive power with multi-modal features including clinical data. In this work, we introduce ChemAP (Chemical structure-based drug Approval Predictor), a novel deep learning scheme for drug approval prediction in the early-stage drug discovery phase. ChemAP aims to enhance the possibility of early-stage decision-making by enriching semantic knowledge to fill in the gap between multi-modal and single-modal chemical spaces through knowledge distillation techniques. This approach facilitates the effective construction of chemical space solely from chemical structure data, guided by multi-modal knowledge related to efficacy, such as clinical trials and patents of drugs. In this study, ChemAP achieved state-of-the-art performance, outperforming both traditional machine learning and deep learning models in drug approval prediction, with AUROC and AUPRC scores of 0.782 and 0.842 respectively on the drug approval benchmark dataset. Additionally, we demonstrated its generalizability by outperforming baseline models on a recent external dataset, which included drugs from the 2023 FDA-approved list and the 2024 clinical trial failure drug list, achieving AUROC and AUPRC scores of 0.694 and 0.851. These results demonstrate that ChemAP is an effective method in predicting drug approval only with chemical structure information of drug so that decision-making can be done at the early stages of drug development process. To the best of our knowledge, our work is the first of its kind to show that prediction of drug approval is possible only with structure information of drug by defining the chemical space of approved and unapproved drugs using deep learning technology.

Sensitivity Matrix Update Method for Electrical Resistance Tomography Based on Error-Constrained Cross Fusion Residual Attention Network

Abstract Electrical resistance tomography (ERT) is an imaging technique of conductivity distribution. The image reconstruction algorithm based on the empty field sensitivity matrix is widely used because it provides an approximate solution to the complex ERT inverse problems. However, due to the soft field effect, the sensitivity matrix changes with the media distribution, leading to significant errors in image reconstruction. To address this issue, an error-constrained deep learning scheme for bubble flow, namely cross fusion residual attention network (CFRA-Net) is designed to update the sensitivity matrix during gas-liquid-solid fluidized bed operation. CFRA-Net employs a multi-head self-attention mechanism to enhance bubble features in boundary measurements, a multilevel cross fusion module to fuse empty field strength and boundary measurement information, and a novel serial-channel residual spatial attention block to boost the feature extraction capability of the model. A weighted loss function is designed to give more weight to the gas phase distribution region. Additionally, this paper establishes a dataset for gas-liquid-solid three-phase flow, considering background field conductivity variations. Validation and reliability of the proposed method are assessed through ablation, comparison, and noise experiments. The experimental results demonstrate that CFRA-Net achieves rapid updates of the sensitivity matrix, high imaging accuracy, and robust noise immunity.

SExpHGS-based DBN: Serial exponential hunger games search algorithm-based deep belief network for heartbeat classification using ECG signal

Electrocardiogram (ECG) signal plays an important role in monitoring and diagnosing patients who suffer from several cardiovascular diseases. Numerous conventional techniques designed for cardiovascular disease classification face challenges regarding classification accuracy and also, find difficulty in automatic monitoring and classification techniques. Therefore, this work aspires to design a robust approach, which can precisely classify the ECG even in the presence of noise. Following that, this research introduces the heartbeat classification scheme by utilizing the optimization-based deep learning scheme. Here, the optimization algorithm, called the Serial Exponential Hunger Games Search Algorithm (SExpHGS) is newly designed by integrating the serial exponential weighted moving average concept in the Hunger Games Search (HGS) approach to train deep learning scheme. Initially, the pre-processing is performed by utilizing a median filter and subsequently, wave components are detected by utilizing the resolution wavelet-based scheme. Ultimately, SExpHGS-based Deep Belief Network (SExpHGS-based DBN) recognizes the ECG conditions of individuals. Here, the techniques are analyzed by utilizing the ECG Lead 2 Dataset PhysioNet dataset and analysis is carried out based on performance parameters, namely accuracy, specificity, and sensitivity. The attained values of the aforementioned metrics are 0.954, 0.965, and 0.938, correspondingly.

Synergizing AI and Physical Expertise to Close the Water Budget from Satellite Data

Abstract A multitude of Earth observation (EO) products are available for monitoring the terrestrial water cycle. These EO datasets have resulted in a multiplicity of datasets for the same geophysical variable. Furthermore, inconsistencies between the water components prevent the water budget closure. A maximum a posteriori (MAP) estimator has been used in the past to optimally combine EO datasets. This framework has many advantages, but it can only be utilized when all four water components are available (precipitation P, evapotranspiration E, total water storage change dS, and river discharge R) and solely at the basin scale. By combining physical expertise with the statistical inference of neural networks (NNs), we designed a custom deep learning scheme to optimize EO data. This hybrid approach benefits from the optimization capabilities of NNs to estimate the parameters of interconnected physical modules. The NN is trained using basin-scale data (from MAP results) over 38 basins to obtain optimized EOs globally. The NN integration offers several enhancements compared to MAP: Independent calibration/mixing models are obtained with imbalance reduction and optimization at the pixel level, and environmental variables can be used to extrapolate results to unmonitored regions. The NN integration enables combining EO estimates of individual water components (P, E, dS, and R) in a hydrologically coherent manner, resulting in a significant decrease in the water budget imbalance at the global scale. Mean imbalance errors can be significant on raw EOs, but they become negligible when EOs are integrated. The standard deviation (STD) of the imbalance is around 26 mm month−1 for raw EOs, and they decrease to 21 when combined and 19 when mixed.

Option pricing for Barndorff–Nielsen and Shephard model by supervised deep learning

This paper aims to develop a supervised deep learning scheme to compute call option prices for the Barndorff-Nielsen and Shephard model with a non-martingale asset price process having infinite active jumps. In our deep learning scheme, teaching data are generated through the Monte Carlo method developed by Arai and Imai [(2024). Monte Carlo simulation for Barndorff-Nielsen and Shephard model under change of measure, Mathematics and Computers in Simulation, 218, 223–234]. Moreover, the BNS model includes many parameters, which makes the deep learning accuracy worse. Therefore, we will create another input parameter using the Black–Scholes formula. As a result, the accuracy is improved dramatically.

Deep Learning of a 200-Member Ensemble with a Limited Historical Training to Improve the Prediction of Extreme Precipitation Events

Abstract This study introduces a deep learning (DL) scheme to generate reliable and skillful probabilistic quantitative precipitation forecasts (PQPFs) in a postprocessing framework. Enhanced machine learning model architecture and training mechanisms are proposed to improve the reliability and skill of PQPFs while permitting computationally efficient model fitting using a short training dataset. The methodology is applied to postprocessing of 24-h accumulated PQPFs from an ensemble forecast system recently introduced by the Center for Western Weather and Water Extremes (CW3E) and for lead times from 1 to 6 days. The ensemble system was designed based on a high-resolution version of the Weather Research and Forecasting (WRF) Model, named West-WRF, to produce a 200-member ensemble in near–real time (NRT) over the western United States during the boreal cool seasons to support Forecast-Informdayed Reservoir Operations (FIRO) and studies of prediction of heavy-to-extreme events. Postprocessed PQPFs are compared with those from the raw West-WRF ensemble, the operational Global Ensemble Forecast System version 12 (GEFSv12), and the ensemble from the European Centre for Medium-Range Weather Forecasts (ECMWF). As an additional baseline, we provide PQPF verification metrics from a recently developed neural network postprocessing scheme. The results demonstrate that the skill of postprocessed forecasts significantly outperforms PQPFs and deterministic forecasts from raw ensembles and the recently developed algorithm. The resulting PQPFs broadly improve upon the reliability and skill of baselines in predicting heavy-to-extreme precipitation (e.g., >75 mm) across all lead times while maintaining the spatial structure of the high-resolution raw ensemble.

Hierarchical deep reinforcement learning for self-adaptive economic dispatch

It is challenging to accurately model the overall uncertainty of the power system when it is connected to large-scale intermittent generation sources such as wind and photovoltaic generation due to the inherent volatility, uncertainty, and indivisibility of renewable energy. Deep reinforcement learning (DRL) algorithms are introduced as a solution to avoid modeling the complex uncertainties and to adapt the fluctuation of uncertainty by interacting with the environment and using feedback to continuously improve their strategies. However, the large-scale nature and uncertainty of the system lead to the sparse reward problem and high-dimensional space issue in DRL. A hierarchical deep reinforcement learning (HDRL) scheme is designed to decompose the process of solving this problem into two stages, using the reinforcement learning (RL) agent in the global stage and the heuristic algorithm in the local stage to find optimal dispatching decisions for power systems under uncertainty. Simulation studies have shown that the proposed HDRL scheme is efficient in solving power system economic dispatch problems under both deterministic and uncertain scenarios thanks to its adaptation system uncertainty, and coping with the volatility of uncertain factors while significantly improving the speed of online decision-making.

Inversion of Sea Surface Currents From Satellite‐Derived SST‐SSH Synergies With 4DVarNets

AbstractSatellite altimetry offers a unique approach for direct sea surface current observation, but it is limited to measuring the surface‐constrained geostrophic component. Ageostrophic dynamics, prevalent at horizontal scales below 100 km and time scales below 10 days, are often underestimated by ocean reanalyzes employing data assimilation schemes. To address this limitation, we introduce a novel deep learning scheme, rooted in a variational data assimilation formulation with trainable observations and a priori terms, that harnesses the synergies between satellite‐derived sea surface observations, namely sea surface height (SSH) and sea surface temperature (SST), to enhance sea surface current reconstruction. Numerical experiments, conducted using realistic simulations, in a case study area of the Gulf Stream, demonstrate the potential of the proposed scheme to capture ageostrophic dynamics at time scales of 2.5–3.0 days and horizontal scales of 0.5°–0.7°. The analysis of diverse observation configurations, encompassing nadir along‐track altimetry, wide‐swath SWOT (Surface Water and Ocean Topography) altimetry, and SST data, highlights the pivotal role of SST features in retrieving a significant portion of the ageostrophic dynamics (approximately 47%). These findings underscore the potential of deep learning and 4DVarNet schemes in improving ocean reanalyzes and enhancing our understanding of ocean dynamics.

Learning-based data-driven optimal deployment control of tethered space robot

To avoid complex constraints of the traditional nonlinear method for tethered space robot (TSR) deployment, a data-driven optimal control framework with an improve deep learning based Koopman operator is proposed in this work. In consideration of nonlinearity of tethered space robot dynamics, its finite dimensional global linear representation called lifted linear system is derived with the Koopman operator theory. A deep learning scheme is adopted to find the embedding functions associate with Koopman operator. And an auxiliary neural network is developed to encode the nonlinear control term of finite dimensional lifted system. Then a controllability constraint is considered for learning a controllable lifted linear system. Besides two loss functions that relate to reconstruction and prediction ability of lifted linear system are designed for training the deep neural network. With the learned lifted linear dynamics, Linear Quadratic Regulator (LQR) is applied to derive the optimal control policy for the tethered space robot deployment. Finally, simulation results verify the effectiveness of proposed framework and show that it could deploy tethered space robot more quickly with less swing of in-plane angle.

A hybrid deep learning scheme for MRI-based preliminary multiclassification diagnosis of primary brain tumors.

The diagnosis and treatment of brain tumors have greatly benefited from extensive research in traditional radiomics, leading to improved efficiency for clinicians. With the rapid development of cutting-edge technologies, especially deep learning, further improvements in accuracy and automation are expected. In this study, we explored a hybrid deep learning scheme that integrates several advanced techniques to achieve reliable diagnosis of primary brain tumors with enhanced classification performance and interpretability. This study retrospectively included 230 patients with primary brain tumors, including 97 meningiomas, 66 gliomas and 67 pituitary tumors, from the First Affiliated Hospital of Yangtze University. The effectiveness of the proposed scheme was validated by the included data and a commonly used data. Based on super-resolution reconstruction and dynamic learning rate annealing strategies, we compared the classification results of several deep learning models. The multi-classification performance was further improved by combining feature transfer and machine learning. Classification performance metrics included accuracy (ACC), area under the curve (AUC), sensitivity (SEN), and specificity (SPE). In the deep learning tests conducted on two datasets, the DenseNet121 model achieved the highest classification performance, with five-test accuracies of 0.989 ± 0.006 and 0.967 ± 0.013, and AUCs of 0.999 ± 0.001 and 0.994 ± 0.005, respectively. In the hybrid deep learning tests, LightGBM, a promising classifier, achieved accuracies of 0.989 and 0.984, which were improved from the original deep learning scheme of 0.987 and 0.965. Sensitivities for both datasets were 0.985, specificities were 0.988 and 0.984, respectively, and relatively desirable receiver operating characteristic (ROC) curves were obtained. In addition, model visualization studies further verified the reliability and interpretability of the results. These results illustrated that deep learning models combining several advanced technologies can reliably improve the performance, automation, and interpretability of primary brain tumor diagnosis, which is crucial for further brain tumor diagnostic research and individualized treatment.

A novel approach for credit card fraud transaction detection using deep reinforcement learning scheme.

Online transactions are still the backbone of the financial industry worldwide today. Millions of consumers use credit cards for their daily transactions, which has led to an exponential rise in credit card fraud. Over time, many variations and schemes of fraudulent transactions have been reported. Nevertheless, it remains a difficult task to detect credit card fraud in real-time. It can be assumed that each person has a unique transaction pattern that may change over time. The work in this article aims to (1) understand how deep reinforcement learning can play an important role in detecting credit card fraud with changing human patterns, and (2) develop a solution architecture for real-time fraud detection. Our proposed model utilizes the Deep Q network for real-time detection. The Kaggle dataset available online was used to train and test the model. As a result, a validation performance of 97.10% was achieved with the proposed deep learning component. In addition, the reinforcement learning component has a learning rate of 80%. The proposed model was able to learn patterns autonomously based on previous events. It adapts to the pattern changes over time and can take them into account without further manual training.

Dynamic-Deep-Ensemble-Learning Scheme for Probabilistic Voltage Stability Margin Estimation to Enhance Resilient Power Grid Monitoring

Modern power grids are characterized by significant penetration of renewable energy sources (RES), variable power demand, and aging transmission infrastructure, all of which contribute to a high degree of operational uncertainty. Such uncertainty complicates the assessment of the static voltage stability in power grids. In response to this challenge, this paper proposes a novel deep ensemble learning-based approach to assess the probabilistic voltage stability margin (PVSM) for strengthening the resilience of power grid monitoring. First, the estimation of the PVSM is formulated as a quantile regression problem. Then, an improved deep quantile regression (iDQR) is utilized to generate a set of quantiles under specific nominal proportions. Next, a dynamic deep ensemble learning (D2EL) scheme based on diverse iDQR models and an improved Choquet fuzzy integral (iCFI) algorithm is proposed to enhance the overall performance of predictive quantiles for the PVSM. The proposed D2EL-based PVSM estimation approach is capable of accommodating system changes in a timely manner, thus providing higher estimation accuracy and stronger adaptability than conventional approaches. A comprehensive numerical study of several test systems is carried out, taking into account uncertain RES and loads, as well as topology changes. The results reveal the impressive performance of the proposed approach in the PVSM assessment.

Hybrid channel and spatial attention-UNet for skin lesion segmentation

<span>Melanoma is a type of skin cancer which has affected many lives globally. The American Cancer Society research has suggested that it a serious type of skin cancer and lead to mortality but it is almost 100% curable if it is detected and treated in its early stages. Currently automated computer vision-based schemes are widely adopted but these systems suffer from poor segmentation accuracy. To overcome these issue, deep learning (DL) has become the promising solution which performs extensive training for pattern learning and provide better classification accuracy. However, skin lesion segmentation is affected due to skin hair, unclear boundaries, pigmentation, and mole. To overcome this issue, we adopt UNet based deep learning scheme and incorporated attention mechanism which considers low level statistics and high-level statistics combined with feedback and skip connection module. This helps to obtain the robust features without neglecting the channel information. Further, we use channel attention, spatial attention modulation to achieve the final segmentation. The proposed DL based scheme is instigated on publically available dataset and experimental investigation shows that the proposed Hybrid Attention UNet approach achieves average performance as 0.9715, 0.9962, 0.9710.</span>

Online Joint Optimization of Virtual Network Function Deployment and Trajectory Planning for Virtualized Service Provision in Multiple-Unmanned-Aerial-Vehicle Mobile-Edge Networks

The multiple-unmanned-aerial-vehicle (multi-UAV) mobile edge network is a promising networking paradigm that uses multiple resource-limited and trajectory-planned unmanned aerial vehicles (UAVs) as edge servers, upon which on-demand virtual network functions (VNFs) are deployed to provide low-delay virtualized network services for the requests of ground users (GUs), who often move randomly and have difficulty accessing the Internet. However, VNF deployment and UAV trajectory planning are both typical NP-complete problems, and the two operations have a strong coupling effect: they affect each other. Achieving optimal virtualized service provision (i.e., maximizing the number of accepted GU requests under a given period T while minimizing the energy consumption and the cost of accepting the requests in all UAVs) is a challenging issue. In this paper, we propose an improved online deep reinforcement learning (DRL) scheme to tackle this issue. First, we formulate the joint optimization of the two operations as a nonconvex mixed-integer nonlinear programming problem, which can be viewed as a sequence of one-frame joint VNF deployment and UAV-trajectory-planning optimization subproblems. Second, we propose an online DRL based on jointly optimizing discrete (VNF deployment) and continuous (UAV trajectory planning) actions to solve each subproblem, whose key idea is establishing and achieving the coupled influence of discrete and continuous actions. Finally, we evaluate the proposed scheme through extensive simulations, and the results demonstrate its effectiveness.

An effective deep learning scheme for android malware detection leveraging performance metrics and computational resources

With the rise in the use of Android smartphones, there has been a proportional surge in the proliferation of malicious applications (apps). As mobile phone users are at a heightened risk of data theft, detecting malware on Android devices has emerged as a pressing concern within the realm of cybersecurity. Conventional techniques, such as signature-based routines, are no longer sufficient to safeguard users from the continually evolving sophistication and swift behavioral modifications of novel varieties of Android malware. Hence, there has been a significant drive in recent times towards leveraging machine learning (ML) models and methodologies to identify and generalize malicious behavioral patterns of mobile apps for detecting malware. This paper proposes Deep learning (DL) based on new and highly reliable classifier, deep neural decision forest (DNDF) for detecting Android malware. Two datasets were used: Drebin and 2014 for comparison with previous studies, and TUANDROMD collected in 2021 for detecting the latest threats with advanced obfuscation and morphing techniques. We have also calculated the time-consuming and computational resources taken by our classifier. After conducting a thorough performance evaluation, our proposed approach attained impressive results on two datasets. The empirical findings reveal that the proposed DBN and DNDF models demonstrated exceptional performance, achieving an accuracy of 99%, a sensitivity of 1, and an AUC value of 0.98%. The metrics we obtained are comparable to those of state-of-the-art ML-based Android malware detection techniques and several commercial antivirus engines.

Deep learning framework for three-dimensional surface reconstruction of object of interest in photoacoustic tomography.

Photoacoustic tomography (PAT) is a non-ionizing hybrid imaging technology of clinical importance that combines the high contrast of optical imaging with the high penetration of ultrasonic imaging. Two-dimensional (2D) tomographic images can only provide the cross-sectional structure of the imaging target rather than its overall spatial morphology. This work proposes a deep learning framework for reconstructing three-dimensional (3D) surface of an object of interest from a series of 2D images. It achieves end-to-end mapping from a series of 2D images to a 3D image, visually displaying the overall morphology of the object. The framework consists of four modules: segmentation module, point cloud generation module, point cloud completion module, and mesh conversion module, which respectively implement the tasks of segmenting a region of interest, generating a sparse point cloud, completing sparse point cloud and reconstructing 3D surface. The network model is trained on simulation data sets and verified on simulation, phantom, and in vivo data sets. The results showed superior 3D reconstruction performance both visually and on the basis of quantitative evaluation metrics compared to the state-of-the-art non-learning and learning approaches. This method potentially enables high-precision 3D surface reconstruction from the tomographic images output by the preclinical PAT system without changing the imaging system. It provides a general deep learning scheme for 3D reconstruction from tomographic scanning data.

A Blockchain-Based Scheme for Secure Data Offloading in Healthcare With Deep Reinforcement Learning

With the widespread popularity of the Internet of Things and various intelligent medical devices, the amount of medical data is rising sharply, and thus medical data processing has become increasingly challenging. Mobile edge computing technology allows computing power to be allocated at the edge closer to users, which enables efficient data offloading for healthcare systems. However, existing studies on medical data offloading seldom guarantee effective data privacy and security. Moreover, the research equipping data offloading architectures with Blockchain neglect the delay and energy consumption costs incurred in using Blockchain technology for medical data offloading. Therefore, in this paper, we propose a data offloading scheme for healthcare based on Blockchain technology, which achieves optimal medical resource allocation and simultaneously minimizes the cost of offloading tasks. Specifically, we design a smart contract to ensure secure data offloading. And, we formulate the cost problem as a Markov Decision Process, solved by a policy search-based deep reinforcement learning (Asynchronous Advantage Actor-Critic) scheme, where we jointly consider offloading decisions, allocation of computing resources and radio transmission bandwidth, and Blockchain data security audits. The security of our smart-contract-based mechanism is theoretically and empirically proved, while extensive experimental results also show that our solution can obtain superior performance gains with lower cost than other baselines.

A Physics-Induced Deep Learning Scheme for Electromagnetic Inverse Scattering

A Physics-Induced Deep Learning Scheme for Electromagnetic Inverse Scattering

Classification and characterization of encoded traffic in SCADA network using hybrid deep learning scheme

Classification and characterization of encoded traffic in SCADA network using hybrid deep learning scheme

Medley deep reinforcement learning-based workload offloading and cache placement decision in UAV-enabled MEC networks

Internet of Things devices generate a large number of heterogeneous workloads in real-time that require specific application to tackle, and the inability to communicate between devices and communication base stations due to complex scenarios is a thorny issue. Service caching play a key role in managing specific-request workload from devices, and unmanned aerial vehicles with computation and communication functions can effectively solve communication barrier between devices and ground base stations. In addition, the joint optimization of workload offloading and service cache placement is a key issue. Accordingly, we design an unmanned aerial vehicle-enabled mobile edge computing system with multiple devices, unmanned aerial vehicles and edge servers. The proposed framework takes into account the randomness of workload arrival, the time-varying nature of channel states, the limitations of the hosting service caching, and wireless communication blocking. Furthermore, we designed workload offloading and service caching hosting decision-making optimization problems to minimize the long-term weighted average latency and energy consumption costs. To tackle this joint optimization problem, we propose a request-specific workload offloading and service caching decision-making scheme based on the medley deep reinforcement learning scheme. To this end, the proposed scheme is decomposed into two-stage optimization subproblems: the workload offloading decision-making problem and the service caching hosting selection problem. In terms of the first subproblem, we model each device as a learning agent and propose the workloads offloading decision-making scheme based on multi-agent deep deterministic policy gradient. For the second subproblem, we present the decentralized double deep Q-learning scheme to tackle the service caching hosting policy. According to the comprehensive experimental results, the proposed scheme is able to converge rapidly on various parameter configurations and whose performance surpasses the other four baseline learning algorithms.