Accuracy Of Deep Neural Network Research Articles

Impact of the Radar Image Resolution of Military Objects on the Accuracy of their Classification by a Deep Convolutional Neural Network

Introduction. Deep convolutional neural networks are considered as one of the most promising tools for classifying small-sized objects on radar images. However, no systemic study has been reported so far on the dependence between the classification accuracy achieved by convolutional neural networks and such an important image characteristic as resolution.Aim. Determination of a dependence between of the accuracy of classifying military objects by a deep convolutional neural network and the resolution of their radar images.Materials and methods. An eight-layer convolutional neural network was designed, trained and tested using the Keras library and Tensorflow 2.0 framework. For training and testing, the open part of the standard MSTAR dataset comprising ten classes of military objects radar images was used. The initial weight values of the MobileNetV1 and Xception networks used for a comparative assessment of the achieved classification accuracy were obtained from the training results on the Imagenet.Results. The accuracy of classifying military objects decreases rapidly along with a deterioration in resolution, amounting to 97.91, 90.22, 79.13, 52.2 and 23.68 % at a resolution of 0.3, 0.6, 0.9, 1.5 and 3 m, respectively. It is shown that the use of pretrained MobileNetV1 and Xception networks does not lead to an improvement in the classification accuracy compared to a simple VGG-type network.Conclusion. Effective recognition of military objects at a resolution worse than one meter is practically impossible. The classification accuracy of deep neural networks depends significantly on the difference in the image resolution of the training and test sets. A significant increase in the resistance of the classification accuracy to changes in the resolution can be achieved by training on a set of images with different resolutions.

Deep Learning Neural Modelling as a Precise Method in the Assessment of the Chronological Age of Children and Adolescents Using Tooth and Bone Parameters.

Dental age is one of the most reliable methods for determining a patient’s age. The timing of teething, the period of tooth replacement, or the degree of tooth attrition is an important diagnostic factor in the assessment of an individual’s developmental age. It is used in orthodontics, pediatric dentistry, endocrinology, forensic medicine, and pathomorphology, but also in scenarios regarding international adoptions and illegal immigrants. The methods used to date are time-consuming and not very precise. For this reason, artificial intelligence methods are increasingly used to estimate the age of a patient. The present work is a continuation of the work of Zaborowicz et al. In the presented research, a set of 21 original indicators was used to create deep neural network models. The aim of this study was to verify the ability to generate a more accurate deep neural network model compared to models produced previously. The quality parameters of the produced models were as follows. The MAE error of the produced models, depending on the learning set used, was between 2.34 and 4.61 months, while the RMSE error was between 5.58 and 7.49 months. The correlation coefficient R2 ranged from 0.92 to 0.96.

Black-box Safety Analysis and Retraining of DNNs based on Feature Extraction and Clustering

Deep neural networks (DNNs) have demonstrated superior performance over classical machine learning to support many features in safety-critical systems. Although DNNs are now widely used in such systems (e.g., self driving cars), there is limited progress regarding automated support for functional safety analysis in DNN-based systems. For example, the identification of root causes of errors, to enable both risk analysis and DNN retraining, remains an open problem. In this paper, we propose SAFE, a black-box approach to automatically characterize the root causes of DNN errors. SAFE relies on a transfer learning model pre-trained on ImageNet to extract the features from error-inducing images. It then applies a density-based clustering algorithm to detect arbitrary shaped clusters of images modeling plausible causes of error. Last, clusters are used to effectively retrain and improve the DNN. The black-box nature of SAFE is motivated by our objective not to require changes or even access to the DNN internals to facilitate adoption. Experimental results show the superior ability of SAFE in identifying different root causes of DNN errors based on case studies in the automotive domain. It also yields significant improvements in DNN accuracy after retraining, while saving significant execution time and memory when compared to alternatives.

CMD: controllable matrix decomposition with global optimization for deep neural network compression

The compression and acceleration of Deep neural networks (DNNs) are necessary steps to deploy sophisticated networks into resource-constrained hardware systems. Due to the weight matrix tends to be low-rank and sparse, several low-rank and sparse compression schemes are leveraged to reduce the overwhelmed weight parameters of DNNs. In these previous schemes, how to make the most of the low-rank and sparse components of weight matrices and how to globally decompose the weight matrix of different layers for efficient compression need to be further investigated. In this paper, in order to effectively utilize the low-rank and sparse characteristics of the weight matrix, we first introduce a sparse coefficient to dynamically control the allocation between the low-rank and sparse components, and an efficient reconstructed network is designed to reduce the inference time. Secondly, since the results of low-rank decomposition can affect the compression ratio and accuracy of DNNs, we establish an optimization problem to automatically select the optimal hyperparameters of the compressed network and achieve global compression for all the layers of network synchronously. Finally, to solve the optimization problem, we present a decomposition-searching algorithm to search the optimal solution. The algorithm can dynamically keep the balance between the compression ratio and accuracy. Extensive experiments of AlexNet, VGG-16 and ResNet-18 on CIFAR-10 and ImageNet are employed to evaluate the effectiveness of the proposed approach. After slight fine-tuning, compressed networks have gained \(1.2\times\) to \(11.3\times\) speedup and our method reduces the size of different networks by \(1.4\times\) to \(14.6\times\).

TaxoNERD: Deep neural models for the recognition of taxonomic entities in the ecological and evolutionary literature

Abstract Easy access to multi‐taxa information (e.g. distribution, traits, diet) in the scientific literature is essential to understand, map and predict all‐inclusive biodiversity. Tools are needed to automatically extract useful information from the ever‐growing corpus of ecological texts and feed this information to open data repositories. A prerequisite is the ability to recognise mentions of taxa in text, a special case of named entity recognition (NER). In recent years, deep learning‐based NER systems have become ubiquitous, yielding state‐of‐the‐art results in the general and biomedical domains. However, no such tool is available to ecologists wishing to extract information from the biodiversity literature. We propose a new tool called TaxoNERD that provides deep neural network (DNN) models to recognise taxon mentions in ecological documents. To achieve high performance, these models usually need to be trained on a large corpus of manually annotated text. Creating such a corpus is a laborious and costly process, with the result that manually annotated corpora in the ecological domain tend to be too small to learn an accurate DNN model from scratch. To address this issue, we leverage existing models pretrained on large biomedical corpora using transfer learning. The performance of our models is evaluated on four corpora and compared to the most popular taxonomic entity recognition tools. Our experiments suggest that existing taxonomic NER tools are not suited to the extraction of ecological information from text as they performed poorly on ecologically oriented corpora, either because they do not take account of the variability of taxon naming practices or because they do not generalise well to the ecological domain. Conversely, a domain‐specific DNN‐based tool like TaxoNERD outperformed the other approaches on an ecological information extraction task. Efforts are needed to raise ecological information extraction to the same level of performance as its biomedical counterpart. One promising direction is to leverage the huge corpus of unlabelled ecological texts to learn a language representation model that could benefit downstream tasks. These efforts could be highly beneficial to ecologists on the long term.

A cascade model-aware generative adversarial example detection method

Deep Neural Networks (DNNs) are demonstrated to be vulnerable to adversarial examples, which are elaborately crafted to fool learning models. Since the accuracy and robustness of DNNs are at odds for the adversarial training method, the adversarial example detection algorithms check whether the specific example is adversarial, which is promising to solve the issue of the adversarial example. However, among the existing methods, model-aware detection methods do not generalize well, while the detection accuracies of the generative-based methods are lower compared to the model-aware methods. In this paper, we propose a cascade model-aware generative adversarial example detection method, namely CMAG. CMAG consists of two first-order reconstructors and a second-order reconstructor, which can illustrate what the model sees to the human by reconstructing the logit and feature maps of the last convolution layer. Experimental results demonstrate that our method is effective and is more interpretable compared to some state-of-the-art methods.

Deep Neural Network for Speaker Identification Using Static and Dynamic Prosodic Feature for Spontaneous and Dictated Data

We can recognize a person by his voice alone. In principle, the sound has a tone (pitch) that is different for each person. This study aims to measure a Deep Neural Network (DNN) performance with static and dynamic prosodic features. Prosodic is information about sound related to tone, intonation, pressure, duration, and rhythm of a person's pronunciation. The data used is dictated and spontaneous voice data that taken from YouTube. The data used consists of three male voices and one female voice. The data is segmented into various duration, 3 seconds, 5 seconds, and 10 seconds. After the data has been segmented, the static prosodic features with 103 dimensions will be extracted and the dynamic prosodic features with 13 dimensions will be extracted too. Each feature and feature combination will be trained and tested using DNN with a ratio of 90:10. The result shows that the 10 seconds segmented data has higher accuracy than the others. Accuracy of static prosodic features is better than dynamic prosodic features. The average accuracy of DNN for static prosodic features is 87.02%. The average accuracy of DNN for dynamic prosodic features is 72.97%. The average accuracy of DNN for combined static and dynamic prosodic features is 87.72%.

Deep Neural Network for Speaker Identification Using Static and Dynamic Prosodic Feature for Spontaneous and Dictated Data

We can recognize a person by his voice alone. In principle, the sound has a tone (pitch) that is different for each person. This study aims to measure a Deep Neural Network (DNN) performance with static and dynamic prosodic features. Prosodic is information about sound related to tone, intonation, pressure, duration, and rhythm of a person's pronunciation. The data used is dictated and spontaneous voice data that taken from YouTube. The data used consists of three male voices and one female voice. The data is segmented into various duration, 3 seconds, 5 seconds, and 10 seconds. After the data has been segmented, the static prosodic features with 103 dimensions will be extracted and the dynamic prosodic features with 13 dimensions will be extracted too. Each feature and feature combination will be trained and tested using DNN with a ratio of 90:10. The result shows that the 10 seconds segmented data has higher accuracy than the others. Accuracy of static prosodic features is better than dynamic prosodic features. The average accuracy of DNN for static prosodic features is 87.02%. The average accuracy of DNN for dynamic prosodic features is 72.97%. The average accuracy of DNN for combined static and dynamic prosodic features is 87.72%.

A Smoothed LASSO-Based DNN Sparsification Technique

Deep Neural Networks (DNNs) are increasingly being used in a variety of applications. However, DNNs have huge computational and memory requirements. One way to reduce these requirements is to sparsify DNNs by using smoothed LASSO (Least Absolute Shrinkage and Selection Operator) functions. In this paper, we show that irrespective of error profile, the sparsity values obtained using various smoothed LASSO functions are similar, provided the maximum error of these functions with respect to the LASSO function is the same. We also propose a layer-wise DNN pruning algorithm, where the layers are pruned based on their individual allocated accuracy loss budget, determined by estimates of the reduction in number of multiply-accumulate operations (in convolutional layers) and weights (in fully connected layers). Further, the structured LASSO variants in both convolutional and fully connected layers are explored within the smoothed LASSO framework and the tradeoffs involved are discussed. The efficacy of proposed algorithm in enhancing the sparsity within the allowed degradation in DNN accuracy and results obtained on structured LASSO variants are shown on MNIST, SVHN, CIFAR-10, and Imagenette datasets and on larger networks such as ResNet-50 and Mobilenet.

Implementation of Lung Cancer Diagnosis based on DNN in Healthcare System

Cancer represents a kind of disease that is widespread throughout the world. Actually, there are several kinds of cancer. However, lung cancer represents the most prevalent cancer form and can lead to death with late healthcare. Therefore, it is essential to initialize therapy via diagnosing lung cancer for decreasing the death chance. Classification is one of the fundamental issues in the knowledge discovery fields and scientific decisions. There are many types of techniques used for constructing classifiers and cancer diagnosis. Recently, deep learning becomes a powerful and popular classification technique for many areas of medical data diagnosis in the healthcare systems. In this paper, an effective and accurate deep neural network (DNN) based lung cancer diagnosis implemented in the healthcare system has been proposed which includes three main phases; pre-processing, generating strong rules, and classification. The input data are pre-processed in the first phase. Because these data are entered from databases, so there are missing data that should be replaced with zero values. Then, data are normalized for speeding up the learning phase. After that, the class association rule is used to enhance the classification performance by generating frequent patterns inducible from the dataset which include features that are significant to the class attribute. Finally, DNN is utilized in the process of classification for obtaining a sample diagnosis estimate. DNN based diagnosis system was tested and evaluated on the lung cancer dataset which has 25 attributes and 1000 instances. The obtained results demonstrated that the proposed system achieved a high performance compared to other existing lung cancer diagnosis systems with 95% accuracy, 97% specificity, and 95% sensitivity.

FLASH: F ast Neura l A rchitecture S earch with H ardware Optimization

Neural architecture search (NAS) is a promising technique to design efficient and high-performance deep neural networks (DNNs). As the performance requirements of ML applications grow continuously, the hardware accelerators start playing a central role in DNN design. This trend makes NAS even more complicated and time-consuming for most real applications. This paper proposes FLASH, a very fast NAS methodology that co-optimizes the DNN accuracy and performance on a real hardware platform. As the main theoretical contribution, we first propose the NN-Degree, an analytical metric to quantify the topological characteristics of DNNs with skip connections (e.g., DenseNets, ResNets, Wide-ResNets, and MobileNets). The newly proposed NN-Degree allows us to do training-free NAS within one second and build an accuracy predictor by training as few as 25 samples out of a vast search space with more than 63 billion configurations. Second, by performing inference on the target hardware, we fine-tune and validate our analytical models to estimate the latency, area, and energy consumption of various DNN architectures while executing standard ML datasets. Third, we construct a hierarchical algorithm based on simplicial homology global optimization (SHGO) to optimize the model-architecture co-design process, while considering the area, latency, and energy consumption of the target hardware. We demonstrate that, compared to the state-of-the-art NAS approaches, our proposed hierarchical SHGO-based algorithm enables more than four orders of magnitude speedup (specifically, the execution time of the proposed algorithm is about 0.1 seconds). Finally, our experimental evaluations show that FLASH is easily transferable to different hardware architectures, thus enabling us to do NAS on a Raspberry Pi-3B processor in less than 3 seconds.

Evolutionary trajectory hybridization for improving deep learning accuracy in medical data prediction system

Medical prediction systems recognize as a significant AI field of study. A huge number of methodologies have been produced in the last decades to solve different kinds of prediction system problems in medical fields, in order to assist physicians in their diagnoses. Deep neural network (DNN) considers an extraordinary method in the medical prediction area. The purpose of this study is to enhance DNN accuracy using a meta-heuristic approach by propose a hybrid model formulated of two meta-heuristic algorithms evolutionary as population-based solution and trajectory as a single-based solution to improve (DNN) to facilitate exploration and exploitation balance and consequently enhance medical dataset prediction. The complex structure of DNN to achieve a best classification accuracy would be useful for meta-heuristic algorithm efficiency for searching the global optimum. To verify the proposal method, the study enhanced DNN with four meta-heuristic algorithms to test the selected medical benchmarks separately. The meta-heuristic algorithms included two population-based algorithms, namely, genetic algorithms GA and differential evolution DE, and two single-based algorithms, namely, simulated annealing SA and Tabu search TS. The proposed methodology registered a significant enhancement in anemia classification comparing with four other meta-heuristic algorithms; the anemia data was composed of a real dataset gathered from Iraqi blood laboratories to detect anemia diseases. Meanwhile, the proposal applied two benchmarks from the University of California repository (UCI), namely, Pima Indian diabetes data and liver disorder diseases. The proposed method attained remarkable results.

(AD)[formula omitted]: Adversarial domain adaptation to defense with adversarial perturbation removal

Deep Neural Networks (DNNs) are demonstrated to be vulnerable to adversarial examples, which are crafted by adding adversarial perturbations to the legitimate examples. To address this issue, some defense methods have been proposed. Among them, the adversarial training (AT) is a popular method to improve the robustness of DNNs. However, theory analysis has shown that in the adversarial training framework, the improvement of the robustness will lead to a decline of standard accuracy. In this paper, we propose a modularized defense framework, namely Adversarial Domain Adaptation to Defense ((AD)2). Different from all adversarial training methods, (AD)2 detects adversarial example using a generative algorithm and applies the adversarial domain adaptation method to remove adversarial perturbation. Experimental results show that (AD)2 is effective to remove the adversarial perturbation and mitigate the odds between the robustness and standard accuracy for DNNs.

Epistatic Net allows the sparse spectral regularization of deep neural networks for inferring fitness functions

Despite recent advances in high-throughput combinatorial mutagenesis assays, the number of labeled sequences available to predict molecular functions has remained small for the vastness of the sequence space combined with the ruggedness of many fitness functions. While deep neural networks (DNNs) can capture high-order epistatic interactions among the mutational sites, they tend to overfit to the small number of labeled sequences available for training. Here, we developed Epistatic Net (EN), a method for spectral regularization of DNNs that exploits evidence that epistatic interactions in many fitness functions are sparse. We built a scalable extension of EN, usable for larger sequences, which enables spectral regularization using fast sparse recovery algorithms informed by coding theory. Results on several biological landscapes show that EN consistently improves the prediction accuracy of DNNs and enables them to outperform competing models which assume other priors. EN estimates the higher-order epistatic interactions of DNNs trained on massive sequence spaces-a computational problem that otherwise takes years to solve.

Analog Nanoscale Electro-Optical Synapses for Neuromorphic Computing Applications.

The typically nonlinear and asymmetric response of synaptic memristors to positive and negative electrical pulses makes the realization of accurate deep neural networks very challenging. Here, we integrate a two-terminal valence change memory (VCM) into a photonic/plasmonic circuit and show that the switching properties of this memristor become more gradual and symmetric under light irradiation. The added optical input acts on the VCM as a third, independent modulation channel. It locally heats the active area of the device, which enhances the generation of oxygen vacancies and broadens the resulting nanoscale conductive filaments. The measured conductance modulation of the VCM is then inserted into a neural network simulator. Using the MNIST data set of handwritten digits as an application, a light-enhanced recognition accuracy of 93.53% is demonstrated, similar to ideally performing memristors (94.86%) and much higher than those without light (67.37%). Notably, the optical signal does not increase the overall energy consumption by more than 3.2%. Finally, an approach to scale up our electro-optical technology is proposed, which could allow high-density, energy-efficient neuromorphic computing chips.

Exploring domains, clinical implications and environmental associations of a deep learning marker of biological ageing.

Deep Neural Networks (DNN) have been recently developed for the estimation of Biological Age (BA), the hypothetical underlying age of an organism, which can differ from its chronological age (CA). Although promising, these population-specific algorithmswarrant further characterization and validation, since their biological, clinical and environmental correlates remain largely unexplored. Here, an accurate DNN was trained to compute BA based on 36 circulating biomarkers in an Italian population (N = 23,858; age ≥ 35years; 51.7% women). This estimate was heavily influenced by markers of metabolic, heart, kidney and liver function. The resulting Δage (BA-CA) significantly predicted mortality and hospitalization risk for all and specific causes. Slowed biological aging (Δage < 0) was associated with higher physical and mental wellbeing, healthy lifestyles (e.g. adherence to Mediterranean diet) and higher socioeconomic status (educational attainment, household income and occupational status), while accelerated aging (Δage > 0) was associated with smoking and obesity. Together, lifestyles and socioeconomic variables explained ~48% of the total variance in Δage, potentially suggesting the existence of a genetic basis. These findings validate blood-based biological aging as a marker of public health in adult Italians and provide a robust body of knowledge on its biological architecture, clinical implications and potential environmental influences.

Accuracy of Deep Neural Network in Triaging Common Skin Diseases of Primary Care Attention.

Access to dermatological care can be challenging in certain regions of the world. The triage process is usually conducted by primary care physicians; however, they may not be able to diagnose and assign the correct referral and level of priority for different dermatosis. The present research aimed to test different deep neural networks to obtain the highest level of accuracy for the following: (1) diagnosing groups of dermatoses; (2) correct referrals; and (3) the level of priority given to the referral compared to dermatologists. Using 140,446 images from a teledermatology project, previously labeled with the clinical diagnosis, and their respective referrals, namely biopsy, in-person dermatologist visits or monitoring the case via teledermatology along with the general physician, 27 different scenarios of neural networks were derived, and the algorithm accuracies in classifying different dermatosis, according to the group of the diagnosis they belong to, were calculated. The most accurate algorithm was then tested for accuracy in diagnosis, referral, and level of priority given to 6,945 cases. The GoogLeNet architecture, trained with 24,000 images and 1,000 epochs, using weight random initialization and learning rates of 10−3 was found to be the most accurate network, showing an accuracy of 89.72% for diagnosis, 96.03% for referrals and 92.54% for priority level in 6,975 image testing. Our study population, however, was confined to individuals with chronic skin conditions and, therefore, it has limited value as a triage tool because it has not been tested for acute conditions. Deep neural networks are accurate in triaging, correct referral and prioritizing common chronic skin diseases related to primary care attention. They can also help health-care systems optimize patients' access to dermatologists.

Do not get fooled: Defense against the one-pixel attack to protect IoT-enabled Deep Learning systems

With the rapid evolution and proliferation of Artificial Intelligence (AI) throughout the past decade, industry-wide Deep Learning (DL) applications within IIoT ecosystems are expected to rise exponentially within this coming decade. However, recent studies have highlighted the extreme sensitivity and vulnerability of modern Deep Neural Networks (DNNs) to adversarial examples. While being imperceptible and benign to the naked human eye, those adversarial perturbations crafted from one-pixel attacks can drastically impact the accuracy of DNNs. The consequences of those adversarial attacks in critical applications such as healthcare and autonomous vehicles can prove disastrous. Therefore, the urge to ensure the robustness and resilience of DL systems against one-pixel attacks has recently started attracting stupendous attention from both academicians and industry professionals. Throughout this paper, we first develop a one-pixel threat model within an IIoT ecosystem and propose a novel image recovery defense mechanism to detect and mitigate one-pixel attacks based on Accelerated Proximal Gradient approach. Experimental results proved that our proposed solution is highly effective and efficient in detecting and mitigating one-pixel attacks in state-of-the-art neural networks, more specifically LeNet and ResNet, using the CIFAR10 and MNIST datasets.

Influence of transportation network on transmission heterogeneity of COVID-19 in China

In this paper, we propose a novel approach to model spatial heterogeneity for epidemic spreading, which combines the relevance of transport proximity in human movement and the excellent estimation accuracy of deep neural network. We apply this model to investigate the effects of various transportation networks on the heterogeneous propagation of COVID-19 in China. We further apply it to predict the development of COVID-19 in China in two scenarios, i.e., i) assuming that different types of traffic restriction policies are conducted and ii) assuming that the epicenter of the COVID-19 outbreak is in Beijing, so as to illustrate the potential usage of the model in generating various policy insights to help the containment of the further spread of COVID-19. We find that the most effective way to prevent the coronavirus from spreading quickly and extensively is to control the routes linked to the epicenter at the beginning of the pandemic. But if the virus has been widely spread, setting restrictions on hub cities would be much more efficient than imposing the same travel ban across the whole country. We also show that a comprehensive consideration of the epicenter location is necessary for disease control.

Supporting Deep Neural Network Safety Analysis and Retraining Through Heatmap-Based Unsupervised Learning

Deep neural networks (DNNs) are increasingly important in safety-critical systems, for example in their perception layer to analyze images. Unfortunately, there is a lack of methods to ensure the functional safety of DNN-based components. We observe three major challenges with existing practices regarding DNNs in safety-critical systems: (1) scenarios that are underrepresented in the test set may lead to serious safety violation risks, but may, however, remain unnoticed; (2) characterizing such high-risk scenarios is critical for safety analysis; (3) retraining DNNs to address these risks is poorly supported when causes of violations are difficult to determine. To address these problems in the context of DNNs analyzing images, we propose HUDD, an approach that automatically supports the identification of root causes for DNN errors. HUDD identifies root causes by applying a clustering algorithm to heatmaps capturing the relevance of every DNN neuron on the DNN outcome. Also, HUDD retrains DNNs with images that are automatically selected based on their relatedness to the identified image clusters. We evaluated HUDD with DNNs from the automotive domain. HUDD was able to identify all the distinct root causes of DNN errors, thus supporting safety analysis. Also, our retraining approach has shown to be more effective at improving DNN accuracy than existing approaches.