Deep Learning Scheme Research Articles

Accurate and robust sparse-view angle CT image reconstruction using deep learning and prior image constrained compressed sensing (DL-PICCS).

Sparse-view CT image reconstruction problems encountered in dynamic CT acquisitions are technically challenging. Recently, many deep learning strategies have been proposed to reconstruct CT images from sparse-view angle acquisitions showing promising results. However, two fundamental problems with these deep learning reconstruction methods remain to be addressed: (1) limited reconstruction accuracy for individual patients and (2) limited generalizability for patient statisticalcohorts. The purpose of this work is to address the previously mentioned challenges in current deep learningmethods. A method that combines a deep learning strategy with prior image constrained compressed sensing (PICCS) was developed to address these two problems. In this method, the sparse-view CT data were reconstructed by the conventional filtered backprojection (FBP) method first, and then processed by the trained deep neural network to eliminate streaking artifacts. The outputs of the deep learning architecture were then used as the needed prior image in PICCS to reconstruct the image. If the noise level from the PICCS reconstruction is not satisfactory, another light duty deep neural network can then be used to reduce noise level. Both extensive numerical simulation data and human subject data have been used to quantitatively and qualitatively assess the performance of the proposed DL-PICCS method in terms of reconstruction accuracy andgeneralizability. Extensive evaluation studies have demonstrated that: (1) quantitative reconstruction accuracy of DL-PICCS for individual patient is improved when it is compared with the deep learning methods and CS-based methods; (2) the false-positive lesion-like structures and false negative missing anatomical structures in the deep learning approaches can be effectively eliminated in the DL-PICCS reconstructed images; and (3) DL-PICCS enables a deep learning scheme to relax its working conditions to enhance itsgeneralizability. DL-PICCS offers a promising opportunity to achieve personalized reconstruction with improved reconstruction accuracy and enhanced generalizability.

Multi-Agent Deep Reinforcement Learning for Computation Offloading and Interference Coordination in Small Cell Networks

Integrating mobile edge computing (MEC) with small cell networks has been conceived as a promising solution to provide pervasive computing services. However, the interactions among small cells due to inter-cell interference, the diverse application-specific requirements, as well as the highly dynamic wireless environment make it challenging to design an optimal computation offloading scheme. In this paper, we focus on the joint design of computation offloading and interference coordination for edge intelligence empowered small cell networks. To this end, we propose a distributed multi-agent deep reinforcement learning (DRL) scheme with the objective of minimizing the overall energy consumption while ensuring the latency requirements. Specifically, we exploit the collaboration among small cell base station (SBS) agents to adaptively adjust their strategies, considering computation offloading, channel allocation, power control, and computation resource allocation. Further, to decrease the computation complexity and signaling overhead of the training process, we design a federated DRL scheme which only requires SBS agents to share their model parameters instead of local training data. Numerical results demonstrate that our proposed schemes can significantly reduce the energy consumption and effectively guarantee the latency requirements compared with the benchmark schemes.

Integrated Deep Auto-Encoder and Q-Learning-Based Scheme to Detect Anomalies and Supporting Forensics in Cloud Computing Environments

The cloud computing environment when deployed correctly is responsible for delivering scalability, cost efficiency, reliability, security and interoperability to the end users. Log analysis is considered to be an indispensable component of security regulations and framework, since these computer-generated records help the organizations, businesses and networks to respond to different kinds of risks that are possible to cloud environment in a reactive and proactive manner. In this paper, an Integrated Deep Auto-Encoder and Q-learning-based Deep Learning (IDEA-QLDL) Scheme is proposed for attaining maximum prediction accuracy during the process of exploring log data and classifying them into genuine and anomalous. It initiates the process of acceptance or denial based on the continuous investigation of behavioral patterns that are highly applicable for classification. The results of the proposed IDEA-QLDL Scheme confirmed its predominance in improving the classification accuracy, precision, recall and detection time compared to the benchmarked schemes considered for investigation.

A novel privacy-preserving deep learning scheme without a cryptography component

Recently, deep learning using Convolutional Neural Networks has played an essential role in many fields. Traditional cryptography, such as the technologies of the garbled circuit and the homomorphic encryption, may provide both parties with a private and secure computation in the neural networks as well as a secure inference scheme. However, it suffers heavy computation in practical designs especially for the training of a CNN model. Hence, the scalability of the model is restricted by these components. In this paper, we propose a novel deep learning model and a secure inferencing scheme in an application of a neural network. We utilize the inherent properties of a convolutional neural network to design a secure mechanism without using any complicated cryptography component. The security analysis shows our proposed scheme is secure, and the experimental results demonstrate that our method is very efficient and suitable for practical applications.

A hierarchical deep reinforcement learning framework for intelligent automatic treatment planning of prostate cancer intensity modulated radiation therapy

Purpose. We have previously proposed an intelligent automatic treatment planning (IATP) framework that builds a virtual treatment planner network (VTPN) to operate a treatment planning system (TPS) to generate high-quality radiation therapy (RT) treatment plans. While the potential of IATP in automating RT treatment planning has been demonstrated, its poor scalability caused by an almost linear growth of network size with the number of treatment planning parameters (TPPs) is a bottleneck, preventing its application in complicate, but clinically relevant treatment planning problems. The decision-making behavior of the trained network is hard to understand. Motivated by the decision-making process of a human planner, this study proposes a hierarchical IATP framework. Methods and materials. The hierarchical VTPN (HieVTPN) consists of three networks, i.e. Structure-Net, Parameter-Net, and Action-Net. When interacting with a TPS, the networks are employed in a sequential order in each step to decide the structure to adjust, the TPP to adjust for the selected structure, and the specific adjustment manner for the parameter, respectively. We developed an end-to-end hierarchical deep reinforcement learning scheme to simultaneously train the three networks. We then evaluated the effectiveness of the proposed framework in the treatment planning problems for prostate cancer intensity modulated RT (IMRT) and stereotactic body RT (SBRT). We benchmarked the performance of our approach by comparing plans made by VTPN of a parallel architecture, and the human plans submitted for competition in the 2016 American Association of Medical Dosimetrist (AAMD)/Radiosurgery Society (RSS) Plan Study. We analyzed scalability of the network size with respect to the number of TPPs. Numerical experiments were also performed to understand the rationale of the decision-making behaviors of the trained HieVTPN. Results. Both HieVTPNs for prostate IMRT and SBRT were trained successfully using 10 training patient cases and 5 validation cases. For IMRT, HieVTPN was able to generate high-quality plans for 59 testing patient cases that were not included in training process, achieving an average plan score of 8.62 (±0.83), with 9 being the maximal score. The score was comparable to that of the VTPN, 8.45 (±0.48). For SBRT planning, HieVTPN achieved an average plan score of 139.07 on five testing patient cases compared to the score of 132.21 averaged over the human plans summited for competition in AAMD/RSS plan study. Different from VTPN with network size linearly scaling with the number of TPPs, the network size of HieVTPN is almost independent of the number of TPPs. It was also observed that the decision-making behaviors of HieVTPN were understandable and generally agreed with the human experience. Conclusions. With the scalability and explainability, the hierarchical IATP framework is more favorable than the previous framework in terms of handling treatment planning problems involving a large number of TPPs.

Fine-Grained User Location Prediction using Meta-Path Context with Attention Mechanism

The prevalence of Location-Based Social Networks (LBSNs) significantly improves the location-aware capability of services by providing Geo-tagged information. Relied on a great number of user check-in data in the location-based social networks, their essential mobility modes are able to be comprehensively studied, which is basic for forecasting the next venue where a specific user is going to visit considering his relevant historical check-in data. Since there exist different kinds of nodes and interactions between nodes, these information could be look upon as a network that is made up of heterogeneous information. In this network a few of different semantic meta paths could be obtained. Enlightened from the competitive advantage of embedding method relied upon meta-path contexts in the heterogeneous information network, we study a joint deep learning scheme exploring different meta-path context information to forecast fine-grained location. In order to capture different semantics in a user-location interaction, we adopt a simple but high-efficient attention method to learn the meta-path importance or weights. In the terms of model optimization, considering we have only positive sample data and there exists intrinsically latent feedback in check-in information, herein a pairwise learning method is utilized for maximizing the margin between visited and invisible venues. Experiment in different data-sets validate the competitive performance of the suggested approach under different assessment criterion.

Detection of Constellation-Modulated Wireless Covert Channel Based on Adjusted CNN Model

With the development of wireless communication technology, more and more information leakage is realized through a wireless covert channel, which brings great challenges to the security of wireless communication. Compared with the wireless covert channel on the upper layer, the wireless covert channel based on the physical layer (WCC-P) has better concealment and greater capacity. As the most widely used scheme of WCC-P, the wireless covert channel with the modulation of the constellation point (WCC-MC) has attracted more and more attention. In this paper, a deep learning scheme based on amplitude-phase characteristics is proposed to detect and classify the WCC-MC scheme. We first extract the amplitude and phase characteristic of error vector magnitude (EVM) and constellation points and then map the amplitude and phase characteristic to the grayscale image, respectively. Finally, the generated feature images are trained, detected, and classified with the adjusted convolution neural network. The experimental results show that the detection accuracy of our proposed scheme can reach 98.5%, and the classification accuracy can reach 81.7%.

2-D Convolutional Deep Neural Network for the Multivariate Prediction of Photovoltaic Time Series

Here, we propose a new deep learning scheme to solve the energy time series prediction problem. The model implementation is based on the use of Long Short-Term Memory networks and Convolutional Neural Networks. These techniques are combined in such a fashion that inter-dependencies among several different time series can be exploited and used for forecasting purposes by filtering and joining their samples. The resulting learning scheme can be summarized as a superposition of network layers, resulting in a stacked deep neural architecture. We proved the accuracy and robustness of the proposed approach by testing it on real-world energy problems.

Adversarial nets for baseline correction in spectra processing

Almost all kinds of spectra data, such as Raman spectroscopy, X-Ray Diffraction (XRD), mass spectroscopy, and infrared spectroscopy, etc., are interrupted by baseline drifts. This large-scale background fluctuation seriously affects the identification of signals. Traditional baseline recognition methods require manual parameters to achieve better performance. In this article, a deep learning scheme is proposed that provides a strategy for generating sufficient training data and a baseline recognition model using adversarial nets. The new scheme is named as Baseline Recognition Networks. It is an intelligent system that has substantial advantages in automation and offers better performance both in terms of qualitative and quantitative studies.

Physics-driven deep-learning inversion with application to transient electromagnetics

Machine learning, and specifically deep-learning (DL) techniques applied to geophysical inverse problems, is an attractive subject, which has promising potential and, at the same time, presents some challenges in practical implementation. Some obstacles relate to scarce knowledge of the searched geologic structures, a problem that can limit the interpretability and generalizability of the trained DL networks when applied to independent scenarios in real applications. Commonly used (physics-driven) least-squares optimization methods are very efficient local optimization techniques but require good starting models close to the correct solution to avoid local minima. We have developed a hybrid workflow that combines both approaches in a coupled physics-driven/DL inversion scheme. We exploit the benefits and characteristics of both inversion techniques to converge to solutions that typically outperform individual inversion results and bring the solution closer to the global minimum of a nonconvex inverse problem. The completely data-driven and self-feeding procedure relies on a coupling mechanism between the two inversion schemes taking the form of penalty functions applied to the model term. Predictions from the DL network are used to constrain the least-squares inversion, whereas the feedback loop from inversion to the DL scheme consists of the network retraining with partial results obtained from inversion. The self-feeding process tends to converge to a common agreeable solution, which is the result of two independent schemes with different mathematical formalisms and different objective functions on the data and model misfit. We determine that the hybrid procedure is converging to robust and high-resolution resistivity models when applied to the inversion of the synthetic and field transient electromagnetic data. Finally, we speculate that the procedure may be adopted to recast the way we solve inverse problems in several different disciplines.

Developing normalization schemes for data isolated distributed deep learning

Distributed deep learning is an important and indispensable direction in the field of deep learning research. Earlier research has proposed many algorithms or techniques on accelerating distributed neural network training. This study discusses a new distributed training scenario, namely data isolated distributed deep learning. Specifically, each node has its own local data and cannot be shared for some reasons. However, in order to ensure the generalization of the model, the goal is to train a global model that required learning all the data, not just based on data from a local node. At this time, distributed training with data isolation is needed. An obvious challenge for distributed deep learning in this scenario is that the distribution of training data used by each node could be highly imbalanced because of data isolation. This brings difficulty to the normalization process in neural network training, because the traditional batch normalization (BN) method will fail under this kind of data imbalanced scenario. At this time, distributed training with data isolation is needed. Aiming at such data isolation scenarios, this study proposes a comprehensive data isolation deep learning scheme. Specifically, synchronous stochastic gradient descent algorithm is used for data exchange during training, and provides several normalization approaches to the problem of BN failure caused by data imbalance. Experimental results show the efficiency and accuracy of the proposed data isolated distributed deep learning scheme.

End-to-End Video Saliency Detection via a Deep Contextual Spatiotemporal Network.

As an interesting and important problem in computer vision, learning-based video saliency detection aims to discover the visually interesting regions in a video sequence. Capturing the information within frame and between frame at different aspects (such as spatial contexts, motion information, temporal consistency across frames, and multiscale representation) is important for this task. A key issue is how to jointly model all these factors within a unified data-driven scheme in an end-to-end fashion. In this article, we propose an end-to-end spatiotemporal deep video saliency detection approach, which captures the information on spatial contexts and motion characteristics. Furthermore, it encodes the temporal consistency information across the consecutive frames by implementing a convolutional long short-term memory (Conv-LSTM) model. In addition, the multiscale saliency properties for each frame are adaptively integrated for final saliency prediction in a collaborative feature-pyramid way. Finally, the proposed deep learning approach unifies all the aforementioned parts into an end-to-end joint deep learning scheme. Experimental results demonstrate the effectiveness of our approach in comparison with the state-of-the-art approaches.

Periodic-Aware Intelligent Prediction Model for Information Diffusion in Social Networks

Due to the rapid development of information and communication technologies with several emerging computing paradigms, such as ubiquitous computing, social computing, and mobile computing, modeling of information diffusion becomes an increasingly significant issue in the big data era. In this study, we focus on a periodic-aware intelligent prediction method based on a comprehensive modeling of user and contagion features, which can be applied to support information diffusion across social networks in accordance with users' adoption behaviors. In particular, the Dynamically Socialized User Networking (DSUN) model and sentiment-Latent Dirichlet Allocation (LDA) topic model, which consider a series of social factors, including user interests and social roles, semantic topics and sentiment polarities, are constructed and integrated together to facilitate the information diffusion process. A periodic-aware preception mechanism usingreinforcement learning with a newly designed reward rule based on topic distribution is then designed to detect and classify different periods into the so-called routine period and emergency period. Finally, a deep learning scheme based on multi-factor analysis is developed for adoption behavior prediction within the identified different periods. Experiments using the real-world data demonstrate the effectiveness and usefulness of our proposed model and method in heterogenous social network environments.

Baseline monitoring of volcanic regions with little recent activity: application of Sentinel-1 InSAR to Turkish volcanoes

Volcanoes have dormancy periods that may last decades to centuries meaning that eruptions at volcanoes with no historical records of eruptions are common. Baseline monitoring to detect the early stages of reawakening is therefore important even in regions with little recent volcanic activity. Satellite techniques, such as InSAR, are ideally suited for routinely surveying large and inaccessible regions, but the large datasets typically require expert interpretation. Here we focus on Turkey where there are 10 Holocene volcanic systems, but no eruptions since 1855 and consequently little ground-based monitoring. We analyse data from the first five years of the European Space Agency Sentinel-1 mission which collects data over Turkey every 6 days on both ascending and descending passes. The high relief edifices of Turkey’s volcanoes cause two challenges: 1) snow cover during the winter months causes a loss of coherence and 2) topographically-correlated atmospheric artefacts could be misinterpreted as deformation. We propose mitigation strategies for both. The raw time series at Hasan Dag volcano shows uplift of ~ 10 cm between September 2017 and July 2018, but atmospheric corrections based on global weather models demonstrate that this is an artefact and reduce the scatter in the data to < 1 cm. We develop two image classification schemes for dealing with the large datasets: one is an easy to follow flowchart designed for non-specialist monitoring staff, and the other is an automated flagging system using a deep learning approach. We apply the deep learning scheme to a dataset of ~ 5000 images over the 10 Turkish volcanoes and find 4 possible signals, all of which are false positives. We conclude that there has been no cm-scale volcano deformation in Turkey in 2015–2020, but further analysis would be required to rule out slower rates of deformation (< 1 cm/yr). This study has demonstrated that InSAR techniques can be used for baseline monitoring in regions with few historical eruptions or little reported deformation.

Monte Carlo simulation fused with target distribution modeling via deep reinforcement learning for automatic high-efficiency photon distribution estimation

Particle distribution estimation is an important issue in medical diagnosis. In particular, photon scattering in some medical devices extremely degrades image quality and causes measurement inaccuracy. The Monte Carlo (MC) algorithm is regarded as the most accurate particle estimation approach but is still time-consuming, even with graphic processing unit (GPU) acceleration. The goal of this work is to develop an automatic scatter estimation framework for high-efficiency photon distribution estimation. Specifically, a GPU-based MC simulation initially yields a raw scatter signal with a low photon number to hasten scatter generation. In the proposed method, assume that the scatter signal follows Poisson distribution, where an optimization objective function fused with sparse feature penalty is modeled. Then, an over-relaxation algorithm is deduced mathematically to solve this objective function. For optimizing the parameters in the over-relaxation algorithm, the deep Q -network in the deep reinforcement learning scheme is built to intelligently interact with the over-relaxation algorithm to accurately and rapidly estimate a scatter signal with the large range of photon numbers. Experimental results demonstrated that our proposed framework can achieve superior performance with structural similarity &gt; 0.94 , peak signal-to-noise ratio &gt; 26.55 dB , and relative absolute error &lt; 5.62 % , and the lowest computation time for one scatter image generation can be within 2 s.

Applying artificial intelligence to longitudinal imaging analysis of vestibular schwannoma following radiosurgery

Artificial intelligence (AI) has been applied with considerable success in the fields of radiology, pathology, and neurosurgery. It is expected that AI will soon be used to optimize strategies for the clinical management of patients based on intensive imaging follow-up. Our objective in this study was to establish an algorithm by which to automate the volumetric measurement of vestibular schwannoma (VS) using a series of parametric MR images following radiosurgery. Based on a sample of 861 consecutive patients who underwent Gamma Knife radiosurgery (GKRS) between 1993 and 2008, the proposed end-to-end deep-learning scheme with automated pre-processing pipeline was applied to a series of 1290 MR examinations (T1W+C, and T2W parametric MR images). All of which were performed under consistent imaging acquisition protocols. The relative volume difference (RVD) between AI-based volumetric measurements and clinical measurements performed by expert radiologists were + 1.74%, − 0.31%, − 0.44%, − 0.19%, − 0.01%, and + 0.26% at each follow-up time point, regardless of the state of the tumor (progressed, pseudo-progressed, or regressed). This study outlines an approach to the evaluation of treatment responses via novel volumetric measurement algorithm, and can be used longitudinally following GKRS for VS. The proposed deep learning AI scheme is applicable to longitudinal follow-up assessments following a variety of therapeutic interventions.

OPTICAL+: a frequency-based deep learning scheme for recognizing brain wave signals

A human–computer interaction (HCI) system can be used to detect different categories of the brain wave signals that can be beneficial for neurorehabilitation, seizure detection and sleep stage classification. Research on developing HCI systems using brain wave signals has progressed a lot over the years. However, real-time implementation, computational complexity and accuracy are still a concern. In this work, we address the problem of selecting the appropriate filtering frequency band while also achieving a good system performance by proposing a frequency-based approach using long short-term memory network (LSTM) for recognizing different brain wave signals. Adaptive filtering using genetic algorithm is incorporated for a hybrid system utilizing common spatial pattern and LSTM network. The proposed method (OPTICAL+) achieved an overall average classification error rate of 30.41% and a kappa coefficient value of 0.398, outperforming the state-of-the-art methods. The proposed OPTICAL+ predictor can be used to develop improved HCI systems that will aid in neurorehabilitation and may also be beneficial for sleep stage classification and seizure detection.

Combined Spiral Transformation and Model-Driven Multi-Modal Deep Learning Scheme for Automatic Prediction of TP53 Mutation in Pancreatic Cancer.

Pancreatic cancer is a malignant form of cancer with one of the worst prognoses. The poor prognosis and resistance to therapeutic modalities have been linked to TP53 mutation. Pathological examinations, such as biopsies, cannot be frequently performed in clinical practice; therefore, noninvasive and reproducible methods are desired. However, automatic prediction methods based on imaging have drawbacks such as poor 3D information utilization, small sample size, and ineffectiveness multi-modal fusion. In this study, we proposed a model-driven multi-modal deep learning scheme to overcome these challenges. A spiral transformation algorithm was developed to obtain 2D images from 3D data, with the transformed image inheriting and retaining the spatial correlation of the original texture and edge information. The spiral transformation could be used to effectively apply the 3D information with less computational resources and conveniently augment the data size with high quality. Moreover, model-driven items were designed to introduce prior knowledge in the deep learning framework for multi-modal fusion. The model-driven strategy and spiral transformation-based data augmentation can improve the performance of the small sample size. A bilinear pooling module was introduced to improve the performance of fine-grained prediction. The experimental results show that the proposed model gives the desired performance in predicting TP53 mutation in pancreatic cancer, providing a new approach for noninvasive gene prediction. The proposed methodologies of spiral transformation and model-driven deep learning can also be used for the artificial intelligence community dealing with oncological applications. Our source codes with a demon will be released at https://github.com/SJTUBME-QianLab/SpiralTransform.

Urban Perception: Sensing Cities via a Deep Interactive Multi-task Learning Framework

Social scientists have shown evidence that visual perceptions of urban attributes, such as safe, wealthy, and beautiful perspectives of the given cities, are highly correlated to the residents’ behaviors and quality of life. Despite their significance, measuring visual perceptions of urban attributes is challenging due to the following facts: (1) Visual perceptions are subjectively contradistinctive rather than absolute. (2) Perception comparisons between image pairs are usually conducted region by region, and highly related to the specific urban attributes. And (3) the urban attributes have both the shared and specific information. To address these problems, in this article, we present a Deep inteRActive Multi-task leArning scheme, DRAMA for short. DRAMA comparatively quantifies the perceptions of urban attributes by jointly integrating the pairwise comparisons, regional interactions, and urban attribute correlations within a unified deep scheme. In DRAMA, each urban attribute is treated as a task, whereby the task-sharing and the task-specific information is fully explored. By conducting extensive experiments over a public large-scale benchmark dataset, it is demonstrated that our proposed DRAMA scheme outperforms several state-of-the-art baselines. Meanwhile, we applied the pairwise comparisons of our DRAMA model to further quantify the urban attributes and hence rank cities with respect to the given urban attributes. As a byproduct, we have released the codes and parameter settings to facilitate other researches.

Auto-Annotated Deep Segmentation for Surface Defect Detection

This article presents a deep learning scheme for automatic defect detection in material surfaces. The success of deep learning model training is generally determined by the number of representative training samples and the quality of the annotation. It is extremely tedious and tiresome to annotate defects pixel-by-pixel in an image to train a semantic network model for defect segmentation. In this study, we propose a two-stage deep learning scheme to tackle the pixel-wise defect detection in textured surfaces without manual annotation. The first stage of the deep learning scheme uses two cycle-consistent adversarial network (CycleGAN) models to automatically synthesize and annotate defect pixels in an image. The synthesized defect images and their corresponding annotated results from the CycleGAN models are then used as the input-output pairs for training the U-Net semantic network. The proposed scheme requires only a few real defect samples for the training and completely requires no manual annotation work. It is practical and computationally very efficient for the implementation in manufacturing. Experimental results show that the proposed deep learning scheme can be applied for defect detection in a variety of textured and patterned surfaces, and results in high detection accuracy.