Application Of Deep Neural Networks Research Articles

Deep neural networks in the cloud: Review, applications, challenges and research directions

Deep neural networks (DNNs) are currently being deployed as machine learning technology in a wide range of important real-world applications. DNNs consist of a huge number of parameters that require millions of floating-point operations (FLOPs) to be executed both in learning and prediction modes. A more effective method is to implement DNNs in a cloud computing system equipped with centralized servers and data storage sub-systems with high-speed and high-performance computing capabilities. This paper presents an up-to-date survey on current state-of-the-art deployed DNNs for cloud computing. Various DNN complexities associated with different architectures are presented and discussed alongside the necessities of using cloud computing. We also present an extensive overview of different cloud computing platforms for the deployment of DNNs and discuss them in detail. Moreover, DNN applications already deployed in cloud computing systems are reviewed to demonstrate the advantages of using cloud computing for DNNs. The paper emphasizes the challenges of deploying DNNs in cloud computing systems and provides guidance on enhancing current and new deployments.

Application of Deep Neural Network for Gas Source Localization in an Indoor Environment

Nowadays, the quality of air in the environment has been impacted by the industry. It is important to make sure our ambient air especially in an indoor environment is clean from contaminating particles or harmful gases. Therefore, the air quality inside the indoor environment should be monitored regularly. One of the major problems, when a particular environment has been contaminated by harmful gases, is finding the source of the emission. If the indoor environment has been contaminated by a harmful source it should be instantly localized and eliminated to prevent severe casualties. In this paper, we propose the utilization of synthetic data generated by the Computational Fluid Dynamic (CFD) approach to train the Deep Neural Network (DNN) model called CFD-DNN to perform gas source localization in an indoor environment. The model is capable to locate the contaminated source within a small area of an indoor environment. A total of 361 datasets with different locations of contaminated source release have been obtained using the CFD approach. The obtained dataset was divided into training and testing datasets. The training dataset was used for the model training process while the testing dataset is fed into the model to test model reliability to predict the gas source location. The Euclidian distance equation was used to measure the distance error between the actual and predicted location of the source. The result shows that the model is capable to locate the gas source within a minimum and maximum error of 0.03m to 0.46m respectively.

Towards Query-Efficient Black-Box Attacks: A Universal Dual Transferability-Based Framework

Adversarial attacks have threatened the application of deep neural networks in security-sensitive scenarios. Most existing black-box attacks fool the target model by interacting with it many times and producing global perturbations. However, all pixels are not equally crucial to the target model; thus, indiscriminately treating all pixels will increase query overhead inevitably. In addition, existing black-box attacks take clean samples as start points, which also limits query efficiency. In this article, we propose a novel black-box attack framework, constructed on a strategy of dual transferability (DT), to perturb the discriminative areas of clean examples within limited queries. The first kind of transferability is the transferability of model interpretations. Based on this property, we identify the discriminative areas of clean samples for generating local perturbations. The second is the transferability of adversarial examples, which helps us to produce local pre-perturbations for further improving query efficiency. We achieve the two kinds of transferability through an independent auxiliary model and do not incur extra query overhead. After identifying discriminative areas and generating pre-perturbations, we use the pre-perturbed samples as better start points and further perturb them locally in a black-box manner to search the corresponding adversarial examples. The DT strategy is general; thus, the proposed framework can be applied to different types of black-box attacks. We conduct extensive experiments to show that, under various system settings, our framework can significantly improve the query efficiency of existing black-box attacks and attack success rates.

Sleep Apnea Detection by Tracheal Motion and Sound, and Oximetry via Application of Deep Neural Networks.

Sleep apnea (SA) is highly prevalent, but under diagnosed due to inaccessibility of sleep testing. To address this issue, portable devices for home sleep testing have been developed to provide convenience with reasonable accuracy in diagnosing SA. The objective of this study was to test the validity a novel portable sleep apnea testing device, BresoDX1, in SA diagnosis, via recording of trachea-sternal motion, tracheal sound and oximetry. Adults with a suspected sleep disorder were recruited to undergo in-laboratory polysomnography (PSG) and a simultaneous BresoDX1 recording. Data from BresoDX1 were collected and features related to breathing sounds, body motions and oximetry were extracted. A deep neural network (DNN) model was trained with 61-second epochs of the extracted features to detect apneas and hypopneas from which an apnea-hypopnea index (AHI) was calculated. The AHI estimated by BresoDX1 (AHIbreso) was compared to the AHI determined from PSG (AHIPSG) and the sensitivity and specificity of SA diagnosis were assessed at an AHIPSG ≥ 15. Two-hundred thirty-three participants (mean ± SD) 50 ± 16 years of age, with BMI of 29.8 ± 6.6 and AHI of 19.5 ± 22.7, were included. There was a strong relationship between AHIbreso and AHIPSG (r = 0.91, p < 0.001). SA detection for an AHIPSG ≥ 15 was highly sensitive (90.0%) and specific (85.9%). We conclude that the DNN model we developed via recording and analyses of trachea-sternal motion and sound along with oximetry provides an accurate estimate of the AHIPSG with high sensitivity and specificity. Therefore, BresoDX1 is a simple, convenient and accurate portable SA monitoring device that could be employed for home SA testing in the future.

BayesianPUFNet: Training Sample Efficient Modeling Attack for Physically Unclonable Functions

This paper proposes a deep neural network named BayesianPUFNet that can achieve high prediction accuracy even with few challenge-response pairs (CRPs) available for training. Generally, modeling attacks are a vulnerability that could compromise the authenticity of physically unclonable functions (PUFs); thus, various machine learning methods including deep neural networks have been proposed to assess the vulnerability of PUFs. However, conventional modeling attacks have not considered the cost of CRP collection and analyzed attacks based on the assumption that sufficient CRPs were available for training; therefore, previous studies may have underestimated the vulnerability of PUFs. Herein, we show that the application of Bayesian deep neural networks that incorporate Bayesian statistics can provide accurate response prediction even in situations where sufficient CRPs are not available for learning. Numerical experiments show that the proposed model uses only half the CRP to achieve the same response prediction as that of the conventional methods. Our code is openly available on https://github.com/bayesian-puf-net/bayesian-puf-net.git

DNN Migration in IoTs: Emerging Technologies, Current Challenges, and Open Research Directions

With the rapid development of the Internet of Things (IoT) and communication technology, Deep Neural Network (DNN) applications have been widely used in IoT devices. However, due to resource constraints on these devices, IoT devices cannot support complicated DNN operation effectively and thus fail to fulfill the requirements of Quality of Service (QoS) of mobile users. One promising approach is to migrate the DNN model to a remote cloud server to reduce the computing burden on IoT devices. Unfortunately, it still suffers from high delay and low bandwidth when communicating with cloud servers. Although the transmission delay of the edge server is low, its computing capacity lacks scalability and elasticity. In this paper, we describe a DNN model migration framework to overcome the above challenges, which consists of three parts: DNN model preprocessing, partition-offloading plan, and partition-uploading plan. Accordingly, we introduce the operation of the DNN migration and the available methods for each part. In addition, we improve the DNN partition-uploading plan in a multi-user edge-cloud collaborative computing environment. Finally, we highlight the important challenges of achieving more efficient DNN migration and point out the unresolved issues of DNN migration, which may shed light on future research directions.

Enhanced Defensive Model Using CNN against Adversarial Attacks for Medical Education through Human Computer Interaction

The design of human-machine interfaces is more precise and demanding in the medical and healthcare industries. Medical monitoring equipment demands more consistent and effective interpretation, as well as fast and straightforward operation due to its monitoring and reference functions. Consequently, it is crucial to consider how people interact with computers when designing the interface for medical monitoring devices. Nowadays people are giving more importance to health than anything in the world. Therefore, as it is related to peoples’ safety, the architecture of human-computer communication must be carefully considered in the studies and development of high-end medical equipment. The price of training physicians and other medical professionals is rising dramatically. Most of the countries have stepped forward from the traditional medical teaching system to a more human computer interactive teaching and learning environment with innovative technologies. This article focuses on the related researches, existing HCI applications for healthcare and the application of deep neural network for disease classification. The proposed work is to develop a healthcare learning platform to offer healthcare education to both medical practitioners and also for common people. This can be implemented as Mobile apps using human-computer interface technology and also as a website with Artificial Intelligence and Machine Learning Techniques. This proposal’s primary goal is to provide anytime, everywhere access to healthcare education for physiological and medical teaching courses, consequently advancing national health care.

An Overview of Open Source Deep Learning-Based Libraries for Neuroscience

In recent years, deep learning has revolutionized machine learning and its applications, producing results comparable to human experts in several domains, including neuroscience. Each year, hundreds of scientific publications present applications of deep neural networks for biomedical data analysis. Due to the fast growth of the domain, it could be a complicated and extremely time-consuming task for worldwide researchers to have a clear perspective of the most recent and advanced software libraries. This work contributes to clarifying the current situation in the domain, outlining the most useful libraries that implement and facilitate deep learning applications for neuroscience, allowing scientists to identify the most suitable options for their research or clinical projects. This paper summarizes the main developments in deep learning and their relevance to neuroscience; it then reviews neuroinformatic toolboxes and libraries collected from the literature and from specific hubs of software projects oriented to neuroscience research. The selected tools are presented in tables detailing key features grouped by the domain of application (e.g., data type, neuroscience area, task), model engineering (e.g., programming language, model customization), and technological aspect (e.g., interface, code source). The results show that, among a high number of available software tools, several libraries stand out in terms of functionalities for neuroscience applications. The aggregation and discussion of this information can help the neuroscience community to develop their research projects more efficiently and quickly, both by means of readily available tools and by knowing which modules may be improved, connected, or added.

Anticipatory analysis of AGV trajectory in a 5G network using machine learning

A new generation of Automatic Guided Vehicles (AGV) virtualises their Programmable Logic Controller (PLC) in the cloud deploying 5G-based communication infrastructures to provide ultra-fast and reliable links between the AGV and its PLC. Stopping an AGV can result in a loss of tens of thousands of euros per minute and therefore, the use of machine learning techniques to anticipate AGV behavior seems to be appropriate. This work proposes the application of advanced deep neural networks to forecast AGV trajectory errors even if disturbances appear in the 5G network by capturing the packets of the PLC-AGV connection and not using any sensor in the user equipment (AGV or PLC), which facilitates the real-time deployment of the solution. To demonstrate the proposed solution, an industrial AGV and a virtualised PLC were deployed in a real 5G network. Furthermore, a set of advanced deep learning architectures was selected, and an extensive collection of experiments was designed to analyse the forecasting performance of each architecture. Additionally, we discuss the real-time issues that appeared during the execution of the best models in a 5G open laboratory, that provided a realistic deployment in a controlled scenario.

Retinal Disease Detection Using Deep Learning Techniques: A Comprehensive Review.

Millions of people are affected by retinal abnormalities worldwide. Early detection and treatment of these abnormalities could arrest further progression, saving multitudes from avoidable blindness. Manual disease detection is time-consuming, tedious and lacks repeatability. There have been efforts to automate ocular disease detection, riding on the successes of the application of Deep Convolutional Neural Networks (DCNNs) and vision transformers (ViTs) for Computer-Aided Diagnosis (CAD). These models have performed well, however, there remain challenges owing to the complex nature of retinal lesions. This work reviews the most common retinal pathologies, provides an overview of prevalent imaging modalities and presents a critical evaluation of current deep-learning research for the detection and grading of glaucoma, diabetic retinopathy, Age-Related Macular Degeneration and multiple retinal diseases. The work concluded that CAD, through deep learning, will increasingly be vital as an assistive technology. As future work, there is a need to explore the potential impact of using ensemble CNN architectures in multiclass, multilabel tasks. Efforts should also be expended on the improvement of model explainability to win the trust of clinicians and patients.

A modified random forest incremental interpretation method for explaining artificial and deep neural networks in cycle time prediction

Methods based on artificial neural network (ANN) or deep neural network (DNN) applications have been proposed to predict job cycle time effectively. However, the predicting mechanism of ANNs (or DNNs) is often difficult to understand and communicate. This problem has hindered their acceptability (and practicability). Furthermore, existing explainable artificial intelligence (XAI) techniques use simpler decision rules to approximate the prediction process and/or outcomes of ANNs (or DNNs). However, these decision rules may not conform to domain knowledge, which limits their usefulness. This study proposes a modified random forest incremental interpretation (RFII) approach to address this issue. In the proposed methodology, the input–output relationship of an ANN (or DNN) is fitted by decision rules using a random forest approach. Then, domain knowledge is extracted, and decision rules are checked against this knowledge. Decision rules with poor conformance to domain knowledge are excluded. Subsequently, the remaining decision rules are aggregated in a way that is convenient for user interpretation. The modified RFII method has been applied to a real case in the literature to demonstrate its applicability.

Predictive deep learning for pitting corrosion modeling in buried transmission pipelines

Despite significant efforts and investments in the renewable energy sector, fossil fuels continue to provide the majority of the world's energy supply. Transmission pipelines, which are extensively used in the oil and gas industry, are vulnerable to various failure mechanisms, such as corrosion. Among these, pitting corrosion in offshore pipelines is the most prevalent type of external corrosion. This study explores the potential of deep learning models (Generalization and Generalization-Memorization models) to predict the maximum depth of pitting corrosion in oil and gas pipelines. The models are trained considering various characteristics of the soil where the pipe is buried and different types of the protective coating of the pipes. The application of deep neural networks resulted in a mean squared error of prediction of 0.0055 in training data and 0.0037 in test data. These results demonstrate that deep learning models outperform all empirical and hybrid models applied in previous studies on the same dataset. The proposed model in this study has the potential to predict failure rates of the pipelines due to external corrosion and enhance the safety and reliability of these facilities.

A tree-level analysis of baboon damage in commercial forest stands using deep learning techniques

Commercial forest plantations in South Africa are homogeneous monocultures of highly bred exotic species grown to deliver timber products of the best potential quality. As such, these stands are susceptible to adverse effects of biotic and abiotic factors, and therefore require intense management to mitigate these risks. A sustainable forest monitoring system that can detect real-time changes in the physiological state of these plantations is needed for timeous management intervention to reduce losses. The use of machine learning algorithms has recently become popular, with acceptable levels of success. This study explores the application of deep learning neural networks for early detection of damage caused by baboons in evergreen plantations of Pinus species. Using PlanetScope imagery (spectral band 590–860 nm), which is captured by a constellation of Dove nanosatellites, with a high temporal resolution available daily at 3 m spatial resolution, the study achieved an overall accuracy of 81.54%, with a kappa value of 0.69, using a deep neural network. In comparison, using a random-forest classifier produced 74.04% accuracy and a kappa value of 0.62. The study successfully mapped different levels of baboon damage within commercial pine forests. We provide a repeatable method for daily monitoring initiatives, and attest to the utility of higher-resolution imagery such as PlanetScope for mapping health and damage severity at the tree level.

Research of Convolutional Neural Network on Image Classification

With the progress of artificial intelligence, technology based on deep learning is becoming more and more mature, and the application of deep convolutional neural network for image classification has become a popular topic for researchers. The number of the structure of deep convolutional neural network for image classification is keep increasing, and its performance is consistently improving, gradually replace that of traditional methods. According to the process of model development and model optimization, this paper divides the convolutional neural network into two models: classical deep convolutional neural network model and attention mechanism deep convolutional neural network model. The construction methods and characteristics of various kinds of deep convolutional neural network models are comprehensively reviewed, and the performance of various classification models is compared and analyzed. Finally, the problems of deep convolutional neural network model are presented.

Precise geopressure predictions in active foreland basins: An application of deep feedforward neural networks

Precise geopressure predictions in active foreland basins: An application of deep feedforward neural networks

Model-driven Cluster Resource Management for AI Workloads in Edge Clouds

Since emerging edge applications such as Internet of Things (IoT) analytics and augmented reality have tight latency constraints, hardware AI accelerators have been recently proposed to speed up deep neural network (DNN) inference run by these applications. Resource-constrained edge servers and accelerators tend to be multiplexed across multiple IoT applications, introducing the potential for performance interference between latency-sensitive workloads. In this article, we design analytic models to capture the performance of DNN inference workloads on shared edge accelerators, such as GPU and edgeTPU, under different multiplexing and concurrency behaviors. After validating our models using extensive experiments, we use them to design various cluster resource management algorithms to intelligently manage multiple applications on edge accelerators while respecting their latency constraints. We implement a prototype of our system in Kubernetes and show that our system can host 2.3× more DNN applications in heterogeneous multi-tenant edge clusters with no latency violations when compared to traditional knapsack hosting algorithms.

Task parallelism-based architectures on FPGA to optimize the energy efficiency of AI at the edge

In the world of artificial intelligence (AI) at the edge, we need to focus primarily on the energy efficiency with which we approach deep neural network (DNN) applications. In many applications, the speed of obtaining an inference can be critical; but many applications easily meet their time requirements, and the energy needed to calculate the huge numbers of multiplication and addition operations of DNNs becomes the essential element. We have provided systolic architectural solutions written in C++ and OpenCL that are highly flexible and easily tunable to take full advantage of the resources of programmable devices and achieve superior energy efficiencies. We focused on low-cost solutions with soft macro microprocessors (Nios2) and hard macro microprocessors (ARM cortex A9).

Bio-medical imaging (X-ray, CT, ultrasound, ECG), genome sequences applications of deep neural network and machine learning in diagnosis, detection, classification, and segmentation of COVID-19: a Meta-analysis & systematic review.

This review investigates how Deep Machine Learning (DML) has dealt with the Covid-19 epidemic and provides recommendations for future Covid-19 research. Despite the fact that vaccines for this epidemic have been developed, DL methods have proven to be a valuable asset in radiologists' arsenals for the automated assessment of Covid-19. This detailed review debates the techniques and applications developed for Covid-19 findings using DL systems. It also provides insights into notable datasets used to train neural networks, data partitioning, and various performance measurement metrics. The PRISMA taxonomy has been formed based on pretrained(45 systems) and hybrid/custom(17 systems) models with radiography modalities. A total of 62 systems with respect to X-ray(32), CT(19), ultrasound(7), ECG(2), and genome sequence(2) based modalities as taxonomy are selected from the studied articles. We originate by valuing the present phase of DL and conclude with significant limitations. The restrictions contain incomprehensibility, simplification measures, learning from incomplete labeled data, and data secrecy. Moreover, DML can be utilized to detect and classify Covid-19 from other COPD illnesses. The proposed literature review has found many DL-based systems to fight against Covid19. We expect this article will assist in speeding up the procedure of DL for Covid-19 researchers, including medical, radiology technicians, and data engineers.

A privacy protection approach in edge-computing based on maximized dnn partition strategy with energy saving

With the development of deep neural network (DNN) techniques, applications of DNNs show state-of-art performance. In the cloud edge collaborative mode, edge devices upload the raw data, such as texts, images, and videos, to the cloud for processing. Then, the cloud returns prediction or classification results. Although edge devices take advantage of the powerful performance of DNN, there are also colossal privacy protection risks. DNN partition strategy can effectively solve the privacy problems by offload part of the DNN model to the edge, in which the encoded features are transmitted rather than original data. We explore the relationship between privacy and the intermedia result of the DNN. The more parts offloaded to the edge, the more abstract features we can have, indicating more conducive to privacy protection. We propose a privacy protection approach based on a maximum DNN partition strategy. Besides, a mix-precision quantization approach is adopted to reduce the energy use of edge devices. The experiments show that our method manages to increase at most 20% model privacy in various DNN architecture. Through the energy-aware mixed-precision quantization approach, the model’s energy consumption is reduced by at most 5x comparing to the typical edge-cloud solution.

SLRProp: A Back-Propagation Variant of Sparse Low Rank Method for DNNs Reduction

Application of deep neural networks (DNN) in edge computing has emerged as a consequence of the need of real time and distributed response of different devices in a large number of scenarios. To this end, shredding these original structures is urgent due to the high number of parameters needed to represent them. As a consequence, the most representative components of different layers are kept in order to maintain the network's accuracy as close as possible to the entire network's ones. To do so, two different approaches have been developed in this work. First, the Sparse Low Rank Method (SLR) has been applied to two different Fully Connected (FC) layers to watch their effect on the final response, and the method has been applied to the latest of these layers as a duplicate. On the contrary, SLRProp has been proposed as a variant case, where the relevances of the previous FC layer's components were weighed as the sum of the products of each of these neurons' absolute values and the relevances of the neurons from the last FC layer that are connected with the neurons from the previous FC layer. Thus, the relationship of relevances across layer was considered. Experiments have been carried out in well-known architectures to conclude whether the relevances throughout layers have less effect on the final response of the network than the independent relevances intra-layer.