Number Of Convolution Filters Research Articles

Intrusion detection system for cloud environment based on convolutional neural networks and PSO algorithm

Authentication of clients and their applications to cloud services is a major concern. Network security and the identification of hostile activities are greatly aided by intrusion detection systems (IDS). In general, optimisation strategies can be applied to improve IDS model performance. Convolutional neural networks (CNN) and other deep learning (DL) algorithms is utilised to enhance IDS’s capability to identify and categories intrusions. IDSs can identify prior attacks, adapt to changing threats, and minimise false positives by utilising these strategies. In this work, a lightweight CNN is proposed for intrusion detection in cloud environment. The main contribution of this research is to use particle swarm optimization (PSO), ametaheuristic algorithm to find the CNNs optimal parameters that comprise the number of convolutional layers, the size of the filter utilized in the convolutional procedure, the number of convolutional filters, and the batch size. Heuristic based searches are useful for solving these kinds of problems. The experimental outcomes demonstrate that the proposed method reaches 91.70% of accuracy, 91.82% of precision, 91.99% of recall and 91.90% of F1-score. Cloud providers can gain from improved security measures by incorporating the proposed IDS paradigm into cloud settings, thereby minimizing unauthorized access and any data breaches.

Revealing the Boundaries of Selected Gastro-Intestinal (GI) Organs by Implementing CNNs in Endoscopic Capsule Images.

The detection of where an organ starts and where it ends is achievable and, since this information can be delivered in real time, it could be quite important for several reasons. For one, by having the practical knowledge of the Wireless Endoscopic Capsule (WEC) transition through an organ's domain, we are able to align and control the endoscopic operation with any other possible protocol, i.e., delivering some form of treatment on the spot. Another is having greater anatomical topography information per session, therefore treating the individual in detail (not "in general"). Even the fact that by gathering more accurate information for a patient by merely implementing clever software procedures is a task worth exploiting, since the problems we have to overcome in real-time processing of the capsule findings (i.e., wireless transfer of images to another unit that will apply the necessary real time computations) are still challenging. This study proposes a computer-aided detection (CAD) tool, a CNN algorithm deployed to run on field programmable gate array (FPGA), able to automatically track the capsule transitions through the entrance (gate) of esophagus, stomach, small intestine and colon, in real time. The input data are the wireless transmitted image shots of the capsule's camera (while the endoscopy capsule is operating). We developed and evaluated three distinct multiclass classification CNNs, trained on the same dataset of total 5520 images extracted by 99 capsule videos (total 1380 frames from each organ of interest). The proposed CNNs differ in size and number of convolution filters. The confusion matrix is obtained by training each classifier and evaluating the trained model on an independent test dataset comprising 496 images extracted by 39 capsule videos, 124 from each GI organ. The test dataset was further evaluated by one endoscopist, and his findings were compared with CNN-based results. The statistically significant of predictions between the four classes of each model and the comparison between the three distinct models is evaluated by calculating the p-values and chi-square test for multi class. The comparison between the three models is carried out by calculating the macro average F1 score and Mattheus correlation coefficient (MCC). The quality of the best CNN model is estimated by calculations of sensitivity and specificity. Our experimental results of independent validation demonstrate that the best of our developed models addressed this topological problem by exhibiting an overall sensitivity (96.55%) and specificity of (94.73%) in the esophagus, (81.08% sensitivity and 96.55% specificity) in the stomach, (89.65% sensitivity and 97.89% specificity) in the small intestine and (100% sensitivity and 98.94% specificity) in the colon. The average macro accuracy is 95.56%, the average macro sensitivity is 91.82%.

Convolution-layer parameters optimization in Convolutional Neural Networks

Convolutional Neural Networks (CNNs) are the most important deep learning algorithms to classify images based on their visual features. CNNs architectures are made of convolution, pooling and fully-connected (FC) layers and their corresponding parameters, which affect classification performance significantly. Convolution parameters optimization for CNNs, referred as CPOCNN, is proposed in this paper. To the best of our knowledge, this is the first optimization model which assigns adaptive upper-bounds of convolution parameters depends on data dimension in current layer and number of remained layers to reach the output layer. For this task, a comprehensive mathematical model is presented for constant structures of CNNs and it is proven that more optimization space is explored in comparison with all state-of-the-art methods. In optimization process, number of convolution filters and type of pooling filters are selected randomly; dimension of pooling filters, zero-padding and stride are considered constant. CPOCNN has been evaluated on 7 publicly available datasets and compared with 53 competitive CNN models with constant and optimized structures. Interested results showed that CPOCNN not only outperforms state-of-the-art CNN methods but also enriches weak CNN models by improving their accuracies more than 35%. Source Code of this paper is available online at Github.11https://github.com/kohzadi/CHOCNN.git.

Structural MRI-Based Schizophrenia Classification Using Autoencoders and 3D Convolutional Neural Networks in Combination with Various Pre-Processing Techniques.

Schizophrenia is a severe neuropsychiatric disease whose diagnosis, unfortunately, lacks an objective diagnostic tool supporting a thorough psychiatric examination of the patient. We took advantage of today’s computational abilities, structural magnetic resonance imaging, and modern machine learning methods, such as stacked autoencoders (SAE) and 3D convolutional neural networks (3D CNN), to teach them to classify 52 patients with schizophrenia and 52 healthy controls. The main aim of this study was to explore whether complex feature extraction methods can help improve the accuracy of deep learning-based classifiers compared to minimally preprocessed data. Our experiments employed three commonly used preprocessing steps to extract three different feature types. They included voxel-based morphometry, deformation-based morphometry, and simple spatial normalization of brain tissue. In addition to classifier models, features and their combination, other model parameters such as network depth, number of neurons, number of convolutional filters, and input data size were also investigated. Autoencoders were trained on feature pools of 1000 and 5000 voxels selected by Mann-Whitney tests, and 3D CNNs were trained on whole images. The most successful model architecture (autoencoders) achieved the highest average accuracy of 69.62% (sensitivity 68.85%, specificity 70.38%). The results of all experiments were statistically compared (the Mann-Whitney test). In conclusion, SAE outperformed 3D CNN, while preprocessing using VBM helped SAE improve the results.

Optimization of Convolutional Neural Networks Architectures Using PSO for Sign Language Recognition

This paper presents an approach to design convolutional neural network architectures, using the particle swarm optimization algorithm. The adjustment of the hyper-parameters and finding the optimal network architecture of convolutional neural networks represents an important challenge. Network performance and achieving efficient learning models for a particular problem depends on setting hyper-parameter values and this implies exploring a huge and complex search space. The use of heuristic-based searches supports these types of problems; therefore, the main contribution of this research work is to apply the PSO algorithm to find the optimal parameters of the convolutional neural networks which include the number of convolutional layers, the filter size used in the convolutional process, the number of convolutional filters, and the batch size. This work describes two optimization approaches; the first, the parameters obtained by PSO are kept under the same conditions in each convolutional layer, and the objective function evaluated by PSO is given by the classification rate; in the second, the PSO generates different parameters per layer, and the objective function is composed of the recognition rate in conjunction with the Akaike information criterion, the latter helps to find the best network performance but with the minimum parameters. The optimized architectures are implemented in three study cases of sign language databases, in which are included the Mexican Sign Language alphabet, the American Sign Language MNIST, and the American Sign Language alphabet. According to the results, the proposed methodologies achieved favorable results with a recognition rate higher than 99%, showing competitive results compared to other state-of-the-art approaches.

LBTS-Net: A fast and accurate CNN model for brain tumour segmentation.

An accurate tumour segmentation in brain images is a complicated task due to the complext structure and irregular shape of the tumour. In this letter, our contribution is twofold: (1) a lightweight brain tumour segmentation network (LBTS‐Net) is proposed for a fast yet accurate brain tumour segmentation; (2) transfer learning is integrated within the LBTS‐Net to fine‐tune the network and achieve a robust tumour segmentation. To the best of knowledge, this work is amongst the first in the literature which proposes a lightweight and tailored convolution neural network for brain tumour segmentation. The proposed model is based on the VGG architecture in which the number of convolution filters is cut to half in the first layer and the depth‐wise convolution is employed to lighten the VGG‐16 and VGG‐19 networks. Also, the original pixel‐labels in the LBTS‐Net are replaced by the new tumour labels in order to form the classification layer. Experimental results on the BRATS2015 database and comparisons with the state‐of‐the‐art methods confirmed the robustness of the proposed method achieving a global accuracy and a Dice score of 98.11% and 91%, respectively, while being much more computationally efficient due to containing almost half the number of parameters as in the standard VGG network.

Exploring Deep Learning for Complex Trait Genomic Prediction in Polyploid Outcrossing Species.

Genomic prediction (GP) is the procedure whereby the genetic merits of untested candidates are predicted using genome wide marker information. Although numerous examples of GP exist in plants and animals, applications to polyploid organisms are still scarce, partly due to limited genome resources and the complexity of this system. Deep learning (DL) techniques comprise a heterogeneous collection of machine learning algorithms that have excelled at many prediction tasks. A potential advantage of DL for GP over standard linear model methods is that DL can potentially take into account all genetic interactions, including dominance and epistasis, which are expected to be of special relevance in most polyploids. In this study, we evaluated the predictive accuracy of linear and DL techniques in two important small fruits or berries: strawberry and blueberry. The two datasets contained a total of 1,358 allopolyploid strawberry (2n=8x=112) and 1,802 autopolyploid blueberry (2n=4x=48) individuals, genotyped for 9,908 and 73,045 single nucleotide polymorphism (SNP) markers, respectively, and phenotyped for five agronomic traits each. DL depends on numerous parameters that influence performance and optimizing hyperparameter values can be a critical step. Here we show that interactions between hyperparameter combinations should be expected and that the number of convolutional filters and regularization in the first layers can have an important effect on model performance. In terms of genomic prediction, we did not find an advantage of DL over linear model methods, except when the epistasis component was important. Linear Bayesian models were better than convolutional neural networks for the full additive architecture, whereas the opposite was observed under strong epistasis. However, by using a parameterization capable of taking into account these non-linear effects, Bayesian linear models can match or exceed the predictive accuracy of DL. A semiautomatic implementation of the DL pipeline is available at https://github.com/lauzingaretti/deepGP/.

Maxout Networks for Visual Recognition

This study investigates the effectiveness of multiple maxout activation variants on image classification, facial identification and verification tasks using convolutional neural networks. A network with maxout activation has a higher number of trainable parameters compared to networks with traditional activation functions. However, it is not clear if the activation function itself or the increase in the number of trainable parameters is responsible for yielding the best performance on different entity recognition tasks. This article investigates if an increase in the number of convolutional filters on the rectified linear unit activation performs equal-to or better-than maxout networks. Our experiments compare rectified linear unit, leaky rectified linear unit, scaled exponential linear unit, and hyperbolic tangent to four maxout variants. Throughout the experiments, we found that on average, across all datasets, the rectified linear unit networks perform better than any maxout activation when the number of convolutional filters is increased six times.

Feature visualization of Raman spectrum analysis with deep convolutional neural network

We demonstrate a recognition and feature visualization method that uses a deep convolutional neural network for Raman spectrum analysis. The visualization is achieved by calculating important regions in the spectra from weights in pooling and fully-connected layers. The method is first examined for simple Lorentzian spectra, then applied to the spectra of pharmaceutical compounds and numerically mixed amino acids. We investigate the effects of the size and number of convolution filters on the extracted regions for Raman-peak signals using the Lorentzian spectra. It is confirmed that the Raman peak contributes to the recognition by visualizing the extracted features. A near-zero weight value is obtained at the background level region, which appears to be used for baseline correction. Common component extraction is confirmed by an evaluation of numerically mixed amino acid spectra. High weight values at the common peaks and negative values at the distinctive peaks appear, even though the model is given one-hot vectors as the training labels (without a mix ratio). This proposed method is potentially suitable for applications such as the validation of trained models, ensuring the reliability of common component extraction from compound samples for spectral analysis.

Evaluation of maxout activations in deep learning across several big data domains

This study investigates the effectiveness of multiple maxout activation function variants on 18 datasets using Convolutional Neural Networks. A network with maxout activation has a higher number of trainable parameters compared to networks with traditional activation functions. However, it is not clear if the activation function itself or the increase in the number of trainable parameters is responsible in yielding the best performance for different entity recognition tasks. This paper investigates if an increase in the number of convolutional filters on traditional activation functions performs equal-to or better-than maxout networks. Our experiments compare the Rectified Linear Unit, Leaky Rectified Linear Unit, Scaled Exponential Linear Unit, and Hyperbolic Tangent activations to four maxout function variants. We observe that maxout networks train relatively slower than networks with traditional activation functions, e.g. Rectified Linear Unit. In addition, we found that on average, across all datasets, the Rectified Linear Unit activation function performs better than any maxout activation when the number of convolutional filters is increased. Furthermore, adding more filters enhances the classification accuracy of the Rectified Linear Unit networks, without adversely affecting their advantage over maxout activations with respect to network-training speed.

Narrow Convolutional Neural Network for Arabic Dialects Polarity Classification

The complexities and tangles of Arabic dialect in orthography and morphology typically make the sentimental analysis quite challenging. Moreover, most of the classification approaches have addressed this problem based on hand-crafted features. Since the Arabic language has multi-dialects and the language has no word-based order, the extraction process and the classification tasks are more difficult and time consuming. Deep neural network approaches applied to the Arabic language colloquial are very limited. These deep learning approaches typically comprise a structure that is very complex for small quantities of data. The structures are based on wide convolutional networks that are not capable of capturing the entire semantic and sentiment features for Arabic dialects. In this paper, a narrow structure of the convolutional neural network (CNN) has been proposed in order to obtain the tweets representations and classify the Arabic tweets into five, three and two polarities. Sensitivity analysis has been conducted to evaluate the impact of various combination structural properties, such as the number of convolutional filters, pooling size, and filter size on the classification performances. The proposed Arabic narrow convolutional neural network (NCNN) has captured the entire semantic and sentiment information contained in the tweet by maximizing the features of the detector’s range. The NCNN performances were estimated to be at its optimum when structured by three convolutional layers, each one followed by the max pooling layer. The model has been developed without using lexicon resources and lexical features or augmented the dataset with extra training data. The narrow model is the first baseline model for Arabic dialects sentiment classifications for a sentence level as it is the first narrow CNN model addressing the Arabic Dialect tweets. NCNN model achieved the lowest macro average mean absolute error (MAEM) for five polarity and higher Macro average recall (P) for three and two polarities on the SemEval-2017 Arabic dialect Twitter datasets when compared to the other state-of-the-art approaches.