Deep Neuronal Networks Research Articles

Recurrent Neural Networks and Deep Neural Networks Based on Intrusion Detection System

The computer security has become a major challenge. Tools and mechanisms have been developed to ensure a level of compliance. These include the Intrusion Detection Systems (IDS). The principle of conventional IDS is to detect attempts to attack a network and to identify abnormal activities and behaviors. The reasons, including the uncertainty in searching for types of attacks and the increasing complexity of advanced cyber-attacks, IDS calls for the need for integration of methods such as Deep Neuron Networks (DNN) and Recurring Neuron Networks (RNN) more precisely long-term memory (LSTM). In this submission, DNN and LSTM were used to predict attacks against the Network Intrusion Detection System (NIDS). In this memory, we used four hidden layers for all deep learning algorithms, forty-one layers of inputs and two layers of outputs and with 100 iterations. In fact, learning is kept constant at 0.01 while the other parameters are optimized. After that for DNN, the number of neurons of the first hidden layer was further increased to 1280 but did not give any appreciable increase in accuracy. Therefore, the number of neurons has been set to 1024 and the LSTM we set the number of neurons of all hidden layers to 32. The results were compared and concluded that a three-layer LSTM performs better than all other conventional machine learning and deep learning algorithms.

How Deep Should be the Depth of Convolutional Neural Networks: a Backyard Dog Case Study

The work concerns the problem of reducing a pre-trained deep neuronal network to a smaller network, with just few layers, whilst retaining the network’s functionality on a given task. In this particular case study, we are focusing on the networks developed for the purposes of face recognition. The proposed approach is motivated by the observation that the aim to deliver the highest accuracy possible in the broadest range of operational conditions, which many deep neural networks models strive to achieve, may not necessarily be always needed, desired or even achievable due to the lack of data or technical constraints. In relation to the face recognition problem, we formulated an example of such a use case, the ‘backyard dog’ problem. The ‘backyard dog’, implemented by a lean network, should correctly identify members from a limited group of individuals, a ‘family’, and should distinguish between them. At the same time, the network must produce an alarm to an image of an individual who is not in a member of the family, i.e. a ‘stranger’. To produce such a lean network, we propose a network shallowing algorithm. The algorithm takes an existing deep learning model on its input and outputs a shallowed version of the model. The algorithm is non-iterative and is based on the advanced supervised principal component analysis. Performance of the algorithm is assessed in exhaustive numerical experiments. Our experiments revealed that in the above use case, the ‘backyard dog’ problem, the method is capable of drastically reducing the depth of deep learning neural networks, albeit at the cost of mild performance deterioration. In this work, we proposed a simple non-iterative method for shallowing down pre-trained deep convolutional networks. The method is generic in the sense that it applies to a broad class of feed-forward networks, and is based on the advanced supervise principal component analysis. The method enables generation of families of smaller-size shallower specialized networks tuned for specific operational conditions and tasks from a single larger and more universal legacy network.

Abstract 693: Deep learning analysis of circulating tumor DNA identified a pan-tumor molecular subtype with enhanced response to durvalumab (anti-PDL1)

Abstract Tumor mutation burden (TMB) has emerged as a promising predictive biomarker for anti-PD1/L1; however, whether somatic mutations in specific genes can sensitize to immunotherapy is poorly understood. We used deep learning to identify molecular subtypes from mutations detectable in circulating tumor DNA (ctDNA) and associated these with outcomes with anti-PDL1 (durvalumab) and durvalumab + anti-CTLA4 (tremelimumab) therapy. Patient blood samples from 4 nonrandomized phase 2 trials evaluating durvalumab (NCT01693562, 1L+ solid tumors; NCT02087423, 3L+ NSCLC) or durvalumab+ tremelimumab (NCT02000947, 2L+ NSCLC; NCT02261220, 2L+ solid tumors) were sequenced with the Guardant 360 assay of 73 genes. CtDNA mutations were examined pre-treatment in a discovery cohort of 818 patients composed of >15 tumor types and validated in a cohort of 94 patients composed of >5 tumor types. Deep learning neuron network was computed using Tensorflow to identify 4 unique molecular subgroups. Kaplan-Meier analysis was calculated and objective response rates (ORR) were assessed with RECISIST v1.1. The subgroups consisted of: DS1 (N = 215/818, 26%) enriched in RB1 mutations (17%) and low TMB, DS2 (N = 237/818, 29%) enriched in ARID1A (33%), NFE2L2 (8%), and cKIT (13%) mutations, DS3 (N = 251/818, 31%) enriched in EGFR, ERBB2, BRAF and STK11 mutations, and DS4 (N = 115/818, 14%) enriched in TERT, FGFR3, and PIK3CA mutations. In the discovery cohort, the DS2 subgroup had most improved outcomes (mOS=24.6 months, mPFS=3.7 months, ORR=27%) compared to the other 3 subgroups. DS1 subgroup had the worst outcomes (mOS=6.3 months, mPFS=1.6 months, ORR=10%), while the biomarker evaluable population (BEP, n=818) had ORR=17%. In the validation cohort, DS2 also had the most improved outcomes (mOS=not reached, mPFS=13.2 months, and ORR=64%), with the BEP (n=94) having an ORR=37%. PD-L1 was not correlated with any specific molecular subgroup, while an IFNg mRNA signature trended with correlation with subgroup DS2. Deep learning analysis of ctDNA identified a molecular subgroup with enhanced response to anti-PDL1 treatment in multiple tumor types. Citation Format: Song Wu, Sriram Sridhar, Han Si, Michael Kuziora, Judson Englert, Shaad E. Abdullah, Philip Dennis, Feng Xiao, Guozhi Gao, Koustubh Ranade, Brandon W. Higgs. Deep learning analysis of circulating tumor DNA identified a pan-tumor molecular subtype with enhanced response to durvalumab (anti-PDL1) [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 693.

Künstliche Intelligenz in der Endoskopie: Neuronale Netze und maschinelles Sehen – Techniken und Perspektiven

Artificial neural networks, as a specific approach towards artificial intelligence (AI), can open up a variety of new perspectives for endoscopy, such as automated lesion detection and the precise prediction of a lesion's histology by its endoscopic appearance. Whilst early experiments do suggest an enormous potential for these methods, public expectations on their application in various fields of medicine sometimes appear to be grounded on general fascination rather than detailed understanding of their inner workings. Based on a selective review of the literature, this article shall convey an intuitive understanding of the underlying methods in order to help close the gap between functioning and fascination and allow for a realistic discussion of their perspectives and limitations in endoscopy.After decades of research, the success of deep neuronal networks in image classification has provoked rising interest for AI during recent years. We quickly touch upon the developments surrounding this breakthrough and the reasons for their impact on various disciplines much beyond computer science. Through a comparison with the functioning of the human vision system, we aim to understand the mechanisms of these techniques and their success in computer vision tasks in detail. Based on these considerations, we analyse the functioning of some important AI applications in endoscopy, deduce specific limitations and perspectives, discuss the current state of their evaluation in practical endoscopy and make a plea for the need for additional and realistic tests. Moreover, we seek to give an impression of some further specific applications that can currently be foreseen and how these can shape the role that AI might finally acquire in the routine clinical practice of GI endoscopy.

Combining high-throughput imaging flow cytometry and deep learning for efficient species and life-cycle stage identification of phytoplankton

BackgroundPhytoplankton species identification and counting is a crucial step of water quality assessment. Especially drinking water reservoirs, bathing and ballast water need to be regularly monitored for harmful species. In times of multiple environmental threats like eutrophication, climate warming and introduction of invasive species more intensive monitoring would be helpful to develop adequate measures. However, traditional methods such as microscopic counting by experts or high throughput flow cytometry based on scattering and fluorescence signals are either too time-consuming or inaccurate for species identification tasks. The combination of high qualitative microscopy with high throughput and latest development in machine learning techniques can overcome this hurdle.ResultsIn this study, image based cytometry was used to collect ~ 47,000 images for brightfield and Chl a fluorescence at 60× magnification for nine common freshwater species of nano- and micro-phytoplankton. A deep neuronal network trained on these images was applied to identify the species and the corresponding life cycle stage during the batch cultivation. The results show the high potential of this approach, where species identity and their respective life cycle stage could be predicted with a high accuracy of 97%.ConclusionsThese findings could pave the way for reliable and fast phytoplankton species determination of indicator species as a crucial step in water quality assessment.

Net-Net Auto Machine Learning (AutoML) Prediction of Complex Ecosystems

Biological Ecosystem Networks (BENs) are webs of biological species (nodes) establishing trophic relationships (links). Experimental confirmation of all possible links is difficult and generates a huge volume of information. Consequently, computational prediction becomes an important goal. Artificial Neural Networks (ANNs) are Machine Learning (ML) algorithms that may be used to predict BENs, using as input Shannon entropy information measures (Shk) of known ecosystems to train them. However, it is difficult to select a priori which ANN topology will have a higher accuracy. Interestingly, Auto Machine Learning (AutoML) methods focus on the automatic selection of the more efficient ML algorithms for specific problems. In this work, a preliminary study of a new approach to AutoML selection of ANNs is proposed for the prediction of BENs. We call it the Net-Net AutoML approach, because it uses for the first time Shk values of both networks involving BENs (networks to be predicted) and ANN topologies (networks to be tested). Twelve types of classifiers have been tested for the Net-Net model including linear, Bayesian, trees-based methods, multilayer perceptrons and deep neuronal networks. The best Net-Net AutoML model for 338,050 outputs of 10 ANN topologies for links of 69 BENs was obtained with a deep fully connected neuronal network, characterized by a test accuracy of 0.866 and a test AUROC of 0.935. This work paves the way for the application of Net-Net AutoML to other systems or ML algorithms.

Colony analysis and deep learning uncover 5-hydroxyindole as an inhibitor of gliding motility and iridescence in Cellulophaga lytica.

Iridescence is an original type of colouration that is relatively widespread in nature but has been either incompletely described or entirely neglected in prokaryotes. Recently, we reported a brilliant 'pointillistic' iridescence in agar-grown colony biofilms of Cellulophaga lytica and some other marine Flavobacteria that exhibit gliding motility. Bacterial iridescence is created by a unique self-organization of sub-communities of cells, but the mechanisms underlying such living photonic crystals are unknown. In this study, we used Petri dish assays to screen a large panel of potential activators or inhibitors of C. lytica's iridescence. Derivatives potentially interfering with quorum-sensing and other communication or biofilm formation processes were tested, as well as metabolic poisons or algal exoproducts. We identified an indole derivative, 5-hydroxyindole (5HI, 250 µM) which inhibited both gliding and iridescence at the colonial level. 5HI did not affect growth or cell respiration. At the microscopic level, phase-contrast imaging confirmed that 5HI inhibits the gliding motility of cells. Moreover, the lack of iridescence correlated with a perturbation of self-organization of the cell sub-communities in both the WT and a gliding-negative mutant. This effect was proved using recent advances in machine learning (deep neuronal networks). In addition to its effect on colony biofilms, 5HI was found to stimulate biofilm formation in microplates. Our data are compatible with possible roles of 5HI or marine analogues in the eco-biology of iridescent bacteria.

Scale Genetic Programming for large Data Sets: Case of Higgs Bosons Classification

Extract knowledge and significant information from very large data sets is a main topic in Data Science, bringing the interest of researchers in machine learning field. Several machine learning techniques have proven effective to deal with massive data like Deep Neuronal Networks. Evolutionary algorithms are considered not well suitable for such problems because of their relatively high computational cost. This work is an attempt to prove that, with some extensions, evolutionary algorithms could be an interesting solution to learn from very large data sets. We propose the use of the Cartesian Genetic Programming (CGP) as meta-heuristic approach to learn from the Higgs big data set. CGP is extended with an active sampling technique in order to help the algorithm to deal with the mass of the provided data. The proposed method is able to take up the challenge of dealing with the complete benchmark data set of 11 million events and produces satisfactory preliminary results.

Survey of Business Perception Based on Sentiment Analysis through Deep Neuronal Networks for Natural Language Processing

Survey of Business Perception Based on Sentiment Analysis through Deep Neuronal Networks for Natural Language Processing