Convolutional Neural Network Hyperparameters Research Articles

Novel ensemble of optimized CNN and dynamic selection techniques for accurate Covid-19 screening using chest CT images

The world is significantly affected by infectious coronavirus disease (covid-19). Timely prognosis and treatment are important to control the spread of this infection. Unreliable screening systems and limited number of clinical facilities are the major hurdles in controlling the spread of covid-19. Nowadays, many automated detection systems based on deep learning techniques using computed tomography (CT) images have been proposed to detect covid-19. However, these systems have the following drawbacks: (i) limited data problem poses a major hindrance to train the deep neural network model to provide accurate diagnosis, (ii) random choice of hyperparameters of Convolutional Neural Network (CNN) significantly affects the classification performance, since the hyperparameters have to be application dependent and, (iii) the generalization ability using CNN classification is usually not validated. To address the aforementioned issues, we propose two models: (i) based on a transfer learning approach, and (ii) using novel strategy to optimize the CNN hyperparameters using Whale optimization-based BAT algorithm + AdaBoost classifier built using dynamic ensemble selection techniques. According to our second method depending on the characteristics of test sample, the classifier is chosen, thereby reducing the risk of overfitting and simultaneously produced promising results. Our proposed methodologies are developed using 746 CT images. Our method obtained a sensitivity, specificity, accuracy, F-1 score, and precision of 0.98, 0.97, 0.98, 0.98, and 0.98, respectively with five-fold cross-validation strategy. Our developed prototype is ready to be tested with huge chest CT images database before its real-world application.

MODE-CNN: A fast converging multi-objective optimization algorithm for CNN-based models

Convolutional neural networks (CNNs) have been used to solve many problems in computer science with a high level of success, and have been applied in many fields in recent years. However, most of the designs of these models are still tuned manually; obtaining the highest performing CNN model is therefore very time-consuming, and is sometimes not achievable. Recently, researchers have started using optimization algorithms for the automatic adjustment of the hyper-parameters of CNNs. In particular, single-objective optimization algorithms have been used to achieve the highest network accuracy for the design of a CNN. When these studies are examined, it can be seen that the most significant problem in the optimization of the parameters of a CNN is that a great deal of time is required for tuning. Hence, optimization algorithms with high convergence rates are needed for the parameter optimization of deep networks. In this study, we first develop an algorithm called MODE-CNN, based on the multi-objective differential evolution (MODE) algorithm for parameter optimization of CNN or CNN-based methods. MODE-CNN is then compared with four different multi-objective optimization algorithms. This comparison is carried out using 16 benchmark functions and four different metrics, with 100 independent runs. It is observed that the algorithm is robust and competitive compared to alternative approaches, in terms of its accuracy and convergence. Secondly, the MODE-CNN algorithm is used in the parameter optimization of a CNN-based method, developed previously by the authors, for the segmentation and classification of medical images. In this method, there are three parameters that influence the test time and accuracy: the general stride (GS), neighbour distance (ND), and patch accuracy (PA). These parameters need to be optimized to give the highest possible accuracy and lowest possible test time. With the MODE-CNN algorithm, the most appropriate GS, ND, and PA values are obtained for the test time and accuracy. As a result, it is observed that the MODE-CNN algorithm is successful, both in comparison with multi-objective algorithms and in the parameter optimization of a CNN-based method.

Unified Deep Learning approach for Efficient Intrusion Detection System using Integrated Spatial–Temporal Features

Intrusion detection systems (IDS) differentiate the malicious entries from the legitimate entries in network traffic data and helps in securing the networks. Deep learning algorithms have been greatly employed in the network security field for large scale data in modern cyberspace networks because of their ability to learn the deeply integrated features. However, learning both space and time aspects of system information are very challenging for any individual deep knowledge model. While Convolutional Neural Networks (CNN) effectively acquires the spatial aspects, the Long Short-Term Memory (LSTM) neural networks perform better for temporal features. Integrating the benefits of these models has the potential for improving the large scale IDS. In this paper, a high accurate IDS model is proposed by using a unified model of Optimized CNN (OCNN) and Hierarchical Multi-scale LSTM (HMLSTM) for effective extraction and learning of spatial–temporal features. The proposed IDS model performs the pre-processing, feature extraction through network training and network testing and final classification. In the OCNN–HMLSTM model, the Lion Swarm Optimization (LSO) is used to tune the hyper-parameters of CNN for the optimal configuration of learning spatial features. The HMLSTM learns the hierarchical relationships between the different features and extracts the time features. Lastly, the unified IDS approach utilizes the extracted spatial–temporal features for categorizing the network data. Tests are performed over public IDS datasets namely NSL-KDD, ISCX-IDS and UNSWNB15. Assessing the performance of OCNN–HMLSTM​ against the contemporary IDS methods, the proposed model performs better intrusion detection with high accuracy of above 90% with less false values and better classification coefficients.

Performance evaluation of Convolutional Neural Network for web security

Due to the daily use of web applications in several critical domains such as banking and online shopping, cybersecurity has become a challenge. Recently, deep learning techniques have achieved promising results and attracted cybersecurity researchers. In this paper, we explore and evaluate deep learning techniques used for the security of web applications. We analyze through experiments the different factors influencing the performance of the Convolutional Neural Network (CNN) technique for web attacks detection. The experiments done in this paper focus on CNN and have three goals. First, we evaluate the performance of different CNN models using two different methods of data input presentation and data input splitting. Second, we study the impact of the different CNN hyper-parameters on the attack detection rate. Third, we select the best deep learning toolbox that will be used in our future proposed detection technique. Through the experiments conducted in this paper, we reveal that an adequate tuning of hyper-parameters and the way of pre-processing data input have a significant impact on the attack detection rate.

An EPSO_CNN Approach to Predict the Soil Texture Properties in Smart Farming

Prediction of soil is one the important factors in smart farming. The use of near infrared spectroscopy that has exclusive benefit in the forecast of soil moisture content is an appropriate and operative method. Convolutional Neural Network (CNN) is a kind of deep learning model with extraordinary performance. Effective attributes can be mined from the complex spectral data and the internal organisation of attributes can be learned with the help of CNN. When a comparison has been made with conventional models, CNN has influential modelling capability since it has objective of attaining better outcomes in soil prediction for hyperspectral data. CNN is proficient to comprehend the arrangement of hyperspectral data by spatial interpolation. The concept of CNN was utilized to forecast the soil humidity content by near infrared spectroscopy. LUCAS dataset have been adopted in this which have nearly 84 attributes. In order to enhance the results of CNN, EPSO-CNN is proposed. The projected PSO variant is used to optimize the learning hyper-parameters of CNNs to overcome performance barriers. The empirical results have showed that the proposed EPSO CNN achieves better results in terms of precision, recall and accuracy and that too in minimum time.

CovidXrayNet: Optimizing data augmentation and CNN hyperparameters for improved COVID-19 detection from CXR

To mitigate the spread of the current coronavirus disease 2019 (COVID-19) pandemic, it is crucial to have an effective screening of infected patients to be isolated and treated. Chest X-Ray (CXR) radiological imaging coupled with Artificial Intelligence (AI) applications, in particular Convolutional Neural Network (CNN), can speed the COVID-19 diagnostic process. In this paper, we optimize the data augmentation and the CNN hyperparameters for detecting COVID-19 from CXRs in terms of validation accuracy. This optimization increases the accuracy of the popular CNN architectures such as the Visual Geometry Group network (VGG-19) and the Residual Neural Network (ResNet-50), by 11.93% and 4.97%, respectively. We then proposed CovidXrayNet model that is based on EfficientNet-B0 and our optimization results. We evaluated CovidXrayNet on two datasets, including our generated balanced COVIDcxr dataset (960 CXRs) and the benchmark COVIDx dataset (15,496 CXRs). With only 30 epochs of training, CovidXrayNet achieves state-of-the-art accuracy of 95.82% on the COVIDx dataset in the three-class classification task (COVID-19, normal or pneumonia). The CovidXRayNet model, the COVIDcxr dataset, and several optimization experiments are publicly available at https://github.com/MaramMonshi/CovidXrayNet.

A Comparative Study of Window Size and Channel Arrangement on EEG-Emotion Recognition Using Deep CNN.

Emotion recognition based on electroencephalograms has become an active research area. Yet, identifying emotions using only brainwaves is still very challenging, especially the subject-independent task. Numerous studies have tried to propose methods to recognize emotions, including machine learning techniques like convolutional neural network (CNN). Since CNN has shown its potential in generalization to unseen subjects, manipulating CNN hyperparameters like the window size and electrode order might be beneficial. To our knowledge, this is the first work that extensively observed the parameter selection effect on the CNN. The temporal information in distinct window sizes was found to significantly affect the recognition performance, and CNN was found to be more responsive to changing window sizes than the support vector machine. Classifying the arousal achieved the best performance with a window size of ten seconds, obtaining 56.85% accuracy and a Matthews correlation coefficient (MCC) of 0.1369. Valence recognition had the best performance with a window length of eight seconds at 73.34% accuracy and an MCC value of 0.4669. Spatial information from varying the electrode orders had a small effect on the classification. Overall, valence results had a much more superior performance than arousal results, which were, perhaps, influenced by features related to brain activity asymmetry between the left and right hemispheres.

A Convolutional Neural Network Architecture to Enhance Oximetry Ability to Diagnose Pediatric Obstructive Sleep Apnea.

This study aims at assessing the usefulness of deep learning to enhance the diagnostic ability of oximetry in the context of automated detection of pediatric obstructive sleep apnea (OSA). A total of 3196 blood oxygen saturation (SpO2) signals from children were used for this purpose. A convolutional neural network (CNN) architecture was trained using 20-min SpO2 segments from the training set (859 subjects) to estimate the number of apneic events. CNN hyperparameters were tuned using Bayesian optimization in the validation set (1402 subjects). This model was applied to three test sets composed of 312, 392, and 231 subjects from three independent databases, in which the apnea-hypopnea index (AHI) estimated for each subject (AHICNN) was obtained by aggregating the output of the CNN for each 20-min SpO2 segment. AHICNN outperformed the 3% oxygen desaturation index (ODI3), a clinical approach, as well as the AHI estimated by a conventional feature-engineering approach based on multi-layer perceptron (AHIMLP). Specifically, AHICNN reached higher four-class Cohen's kappa in the three test databases than ODI3 (0.515 vs 0.417, 0.422 vs 0.372, and 0.423 vs 0.369) and AHIMLP (0.515 vs 0.377, 0.422 vs 0.381, and 0.423 vs 0.306). In addition, our proposal outperformed state-of-the-art studies, particularly for the AHI severity cutoffs of 5 e/h and 10 e/h. This suggests that the information automatically learned from the SpO2 signal by deep-learning techniques helps to enhance the diagnostic ability of oximetry in the context of pediatric OSA.

Transient Simulations of High-Speed Channels Using CNN-LSTM With an Adaptive Successive Halving Algorithm for Automated Hyperparameter Optimizations

Transient simulations of high-speed channels can be very time intensive. Recurrent neural network (RNN) based methods can be used to speed up the process by training a RNN model on a relatively short bit sequence, and then using a multi-steps rolling forecast method to predict subsequent bits. However, the performance of the RNN model is highly affected by its hyperparameters. We propose an algorithm named adaptive successive halving automated hyperparameter optimization (ASH-HPO) which combines successive halving, Bayesian optimization (BO), and progressive sampling to tune the hyperparameters of the RNN models. Modifications are proposed to the successive halving and progressive sampling algorithms for better efficiency on time series data. The ASH-HPO algorithm trains on smaller dataset subsets initially, then expands the training dataset progressively and adaptively adds or removes models along the process. In this paper, we use the ASH-HPO algorithm to optimize the hyperparameters of convolutional neural networks (CNNs), long short-term memory (LSTM) networks, and CNN-LSTM networks. We demonstrate the effectiveness of the ASH-HPO algorithm using a PCIe Gen 2 channel, a PCIe Gen 5 channel, and a PAM4 differential channel. We also investigate the effects of several settings and tunable variables of the ASH-HPO algorithm on its convergence speed. As a benchmark, we compared the ASH-HPO algorithm to three state-of-the-art HPO methods: BO, successive halving, and hyperband. The results show that the ASH-HPO algorithm converges faster than the other HPO methods on transient simulation problems.

An Efficient Heartbeats Classifier Based on Optimizing Convolutional Neural Network Model

Recently, deep learning models have emerged as promising methods for the diagnosis of different diseases. Cardiac disease is among the leading life-threatening diseases on a global scale. The aim of this paper is to propose an optimized Convolutional Neural Network (CNN) model for the classification of electrocardiogram (ECG) heartbeat data. The proposed ECG classification approach is designed with an optimal CNN configuration to classify cardiac arrhythmias quickly and effectively. Finding an optimal configuration for the CNN hyperparameters is time-consuming and needs extensive experimentation. To overcome this challenge, we present an optimization step for the proposed CNN model using a customized genetic algorithm. It provides an automatic suggestion for the best hyperparameter settings of the proposed CNN. The challenge in utilizing the genetic algorithm is that its operators need to be customized to handle our problem domain. Our approach accepts raw ECG signals without any preprocessing steps, which has benefit in saving the computation time. Our approach also provides a resampling step to ensure generalization, to better handle imbalanced ECG classes. Experiments show promising results of our proposed approach against other approaches whose CNN hyperparameters setting depended on numerous trials, requiring extensive ECG feature extraction steps, and do not consider imbalanced classes. The performance of our proposed approach is better than other existing methods both in terms of higher classification accuracy (98.45%), and lower computational complexity.

Visualization Analysis on CNN Based CTR Prediction

클릭률(click through rate, CTR) 예측은 사용자에게 노출된 광고가 클릭될 확률을 추정하는 것으로 온라인 광고 수익 극대화를 위한 광고 노출 전략 결정에 중요한 역할을 담당한다. CTR 예측의 핵심은 데이터 필드(특징) 간 상호작용을 효과적으로 모델링하는 것인데 최근 시·공간적 연계성을 갖는 데이터에 있어 탁월한 성능을 발휘하고 있는 심층신경망(deep neural network)을 모델링에 활용한 연구가 활발히 진행되고 있다. 본 연구는 CTR 예측을 위해 이미지·음성 인식 분야에서 뛰어난 성능을 보여주고 있는 신경망 구조인 Convolutional Neural Network(CNN)의 활용가능성에 관한 기초적인 연구 결과를 제시한다. 먼저 이미지·음성 데이터와 구별되는 CTR 데이터의 특성과 이로 인한 CNN 적용 시 발생할 수 있는 성능상의 한계점을 도출한다. CNN의 하이퍼 파라미터인 임베딩과 풀링에 관한 비교 설정 실험 결과를 시각적으로 해석하여 CTR 데이터에 대한 하이퍼 파라미터의 영향력을 분석한다. 실험 결과에 따르면, CTR 데이터에 있어 임베딩과 풀링은 유기적으로 성능에 영향을 미치므로 데이터 특성에 맞게 둘 간의 최적 조합을 선택하는 것이 CNN 성능 향상에 매우 중요한 요소가 됨을 확인하였다.The click through rate(CTR) prediction estimates the probability that an advertisement exposed to a user is clicked, and plays an important role in determining an advertisement exposure strategy for maximizing online advertisement revenue. The key to predicting CTR is to effectively model the interactions between data fields(features). Recently, researches using deep neural network(DNN), which exhibits excellent performance in data with temporal and spatial correlation, are actively conducted for modeling. This study presents basic research results on the possibility of using a convolutional neural network(CNN), a neural network structure that shows the best performance in the image and speech recognition field, for CTR prediction. First, we derive the characteristics of CTR data that are distinguished from image and voice data and the performance limitations that can occur when regular CNN is applied. Then, we visually analyze the experiment results of the comparison setup for CNN hyperparameters(embedding and pooling) to understand the imapct of hyperparameters on CTR data. According to the experimental results, since embedding and pooling organically affect the performance of CTR data, it was confirmed that selecting the optimal combination between the two based on the data characteristics is a very important factor in improving CNN performance.

A Fast Evolutionary Learning to Optimize CNN

Deep neural networks (DNNs) show great performance in lots of applications. Convolutional neural network (CNN) is one of the classic DNNs, and various modified CNNs have been brought up, such as DenseNet, GoogleNet, ResNet, etc. For diverse tasks, a unique structure of CNN may show its advantage. However, how to design an effective CNN model for a practical task is a puzzle. In this paper, we model the architecture optimization of CNN as an optimization problem and design a Genetic network programming based Fast evolutionary learning (GNP-FEL) to optimize CNN. GNP-FEL contains three main ideas: First GNP is adopted to optimize CNN architecture and hyperparameters, which can build diverse network structures and make network parameters selfevolve; Second multi-objective optimization is designed by balancing both CNN model efficiency and structure compactness; Last a novel incremental training method is proposed to train offspring CNN models in GNP, which is capable of reducing time complexity sharply. Experiments have validated that GNP-FEL can quickly evolve a CNN classifier with a sufficiently compact architecture. And the classifier has a comparable classification effect to state-ofthe-art CNN model.

Deep Neural Network-Based Screening Model for COVID-19-Infected Patients Using Chest X-Ray Images

There are limited coronavirus disease 2019 (COVID-19) testing kits, therefore, development of other diagnosis approaches is desirable. The doctors generally utilize chest X-rays and Computed Tomography (CT) scans to diagnose pneumonia, lung inflammation, abscesses, and/or enlarged lymph nodes. Since COVID-19 attacks the epithelial cells that line our respiratory tract, therefore, X-ray images are utilized in this paper, to classify the patients with infected (COVID-19 [Formula: see text]ve) and uninfected (COVID-19 [Formula: see text]ve) lungs. Almost all hospitals have X-ray imaging machines, therefore, the chest X-ray images can be used to test for COVID-19 without utilizing any kind of dedicated test kits. However, the chest X-ray-based COVID-19 classification requires a radiology expert and significant time, which is precious when COVID-19 infection is increasing at a rapid rate. Therefore, the development of an automated analysis approach is desirable to save the medical professionals’ valuable time. In this paper, a deep convolutional neural network (CNN) approach is designed and implemented. Besides, the hyper-parameters of CNN are tuned using Multi-objective Adaptive Differential Evolution (MADE). Extensive experiments are performed by considering the benchmark COVID-19 dataset. Comparative analysis reveals that the proposed technique outperforms the competitive machine learning models in terms of various performance metrics.

Recycling waste classification using optimized convolutional neural network

An automatic classification robot based on effective image recognition could help reduce huge labors of recycling tasks. Convolutional neural network (CNN) model, such as DenseNet121, improved the traditional image recognition technology and was the currently dominant approach to image recognition. A famous benchmark dataset, i.e., TrashNet, comprised of a total of 2527 images with six different waste categories was used to evaluate the CNNs’ performance. To enhance the accuracy of waste classification driven by CNNs, the data augmentation method could be adopted to do so, but fine-tuning optimally hyper-parameters of CNN's fully-connected-layer was never used. Therefore, besides data augmentation, this study aims to utilize a genetic algorithm (GA) to optimize the fully-connected-layer of DenseNet121 for improving the classification accuracy of DenseNet121 on TrashNet and proposes the optimized DenseNet121.The results show that the optimized DenseNet121 achieved the highest accuracy of 99.6%, when compared with other studies’ CNNs. The data augmentation could perform higher efficiency on accuracy improvement of image classification than optimizing fully-connected-layer of DenseNet121 for TrashNet. To replace the function of the original classifier of DenseNet121 with fully-connected-layer can improve DenseNet121’s performance. The optimized DenseNet121 further improved the accuracy and demonstrated the efficiency of using GA to optimize the neuron number and the dropout rate of fully-connected-layer. Gradient-weighted class activation mapping helped highlight the coarse features of the waste image and provide additional insight into the explainability of optimized DenseNet121.

Performance Improvement Of Pre-trained Convolutional Neural Networks For Action Recognition

Abstract Action recognition is a challenging task. Deep learning models have been investigated to solve this problem. Setting up a new neural network model is a crucial and time-consuming process. Alternatively, pre-trained convolutional neural network (CNN) models offer rapid modeling. The selection of the hyperparameters of CNNs is a challenging issue that heavily depends on user experience. The parameters of CNNs should be carefully selected to get effective results. For this purpose, the artificial bee colony (ABC) algorithm is used for tuning the parameters to get optimum results. The proposed method includes three main stages: the image preprocessing stage involves automatic cropping of the meaningful area within the images in the data set, the transfer learning stage includes experiments with six different pre-trained CNN models and the hyperparameter tuning stage using the ABC algorithm. Performance comparison of the pre-trained CNN models involving the use and nonuse of the ABC algorithm for the Stanford 40 data set is presented. The experiments show that the pre-trained CNN models with ABC are more successful than pre-trained CNN models without ABC. Additionally, to the best of our knowledge, the improved NASNet-Large CNN model with the ABC algorithm gives the best accuracy of 87.78% for the overall success rate-based performance metric.

Fusion of deep-learned and hand-crafted features for cancelable recognition systems

The recent years have witnessed a dramatic shift in the way of biometric identification, authentication, and security processes. Among the essential challenges that face these processes are the online verification and authentication. These challenges lie in the complexity of such processes, the necessity of the personal real-time identifiable information, and the methodology to capture temporal information. In this paper, we present an integrated biometric recognition method to jointly recognize face, iris, palm print, fingerprint and ear biometrics. The proposed method is based on the integration of the extracted deep-learned features together with the hand-crafted ones by using a fusion network. Also, we propose a novel convolutional neural network (CNN)-based model for deep feature extraction. In addition, several techniques are exploited to extract the hand-crafted features such as histogram of oriented gradients (HOG), oriented rotated brief (ORB), local binary patterns (LBPs), scale-invariant feature transform (SIFT), and speeded-up robust features (SURF). Furthermore, for dimensional consistency between the combined features, the dimensions of the hand-crafted features are reduced using independent component analysis (ICA) or principal component analysis (PCA). The core of this paper is the template protection via a cancelable biometric scheme without significantly affecting the recognition performance. Specifically, we have used the bio-convolving approach to enhance the user’s privacy and ensure the robustness against spoof attacks. Additionally, various CNN hyper-parameters with their impact on the proposed model performance are studied. Our experiments on various datasets revealed that the proposed method achieves 96.69%, 95.59%, 97.34%, 96.11% and 99.22% recognition accuracies for face, iris, fingerprint, palm print and ear recognition, respectively.

Effectively Diagnosing Malaria by Optimizing the Hyperparameters of CNN using Genetic Algorithm on the Multi core GPU

Image classification is an important task in computer vision involving a large area of applications such as object detection, localization and image segmentation. When it comes to image classification, the most adopted methods are based on deep neural network and especially convolutional Neural Networks(CNN). Selection of hyperparameters plays a crucial role in performance of model and it comes by experience. So, in this paper, we will use the genetic algorithm(GA) to automate and build the CNN model for higher accuracy on GPU which is provided by Google Collaboratory cloud. The best architecture of CNN after several generations of the genetic algorithm is then compared to the state-of-the-art CNN. We have used the malaria cell images dataset to find out whether the person is normal or if they are suffering from malaria. We trained two types of malaria cells, which are uninfected and parasitized on Tesla P100 multi core GPU. We got a high training accuracy of 97% and got a testing accuracy of about 95% on the multicore GPU that boosted the speed of execution of training time period and testing time period.

Convolutional Neural Network—Optimized Moth Flame Algorithm for Shallow Landslide Susceptible Analysis

Convolutional neural network (CNN) is a widely used method in solving classification and regression applications in industries, engineering, and science. This study investigates the optimizing capability of a swarm intelligence algorithm named moth flame optimizer (MFO) for the optimal search of a CNN hyper-parameters (values of filters) and weights of fully connected layers. The proposed model was run with a 3-dimensional dataset (7 width × 7 height ×12 depth), which was constructed through including seven neighbor pixels (vertically and horizontally) from landslide location and 12 predictor variables. Muong Te district, Lai Chau province, Vietnam was selected as the case study, as it had recently undergone severe impacts of landslides and flash floods. The performance of this proposed model was compared with conventional classifiers, i.e., Random forest, Random subspace, and CNN-optimized Adaptive gradient descend, by using standard metrics. The results showed that the CNN-optimized MFO (Root mean square error = 0.3685, Mean absolute error = 0.2888, Area under Receiver characteristic curve = 0.889 and Overall accuracy = 80.1056%) outperformed the benchmarked methods in all comparing indicators. Besides, the statistical test of difference was also carried out by using the Wilcoxon signed ranked test for non-parametric variables. With these statistical measurements, the proposed model could be used as an alternative solution for landslide susceptibility mapping to support local disaster preparedness plans.

CPSO-CNN: An efficient PSO-based algorithm for fine-tuning hyper-parameters of convolutional neural networks

Swarm intelligence algorithms have been widely adopted in solving many highly nonlinear, multimodal problems and have achieved tremendous successes. However, their application on deep neural networks is largely unexplored. On the other hand, deep neural networks, especially convolutional neural network (CNN), have recently achieved breakthroughs in tackling many intractable problems; nevertheless their performance depends heavily on the chosen values of their hyper-parameters, whose fine-tuning is both labor-intensive and time-consuming. In this paper, we propose a novel particle swarm optimization (PSO) variant cPSO-CNN for optimizing the hyper-parameter configuration of architecture-determined CNNs. cPSO-CNN utilizes a confidence function defined by a compound normal distribution to model experts' knowledge on CNN hyper-parameter fine-tunings so as to enhance the canonical PSO's exploration capability. cPSO-CNN also redefines the scalar acceleration coefficients of PSO as vectors to better adapt for the variant ranges of CNN hyper-parameters. Besides, a linear prediction model is adopted for fast ranking the PSO particles to reduce the cost of fitness function evaluation. The experimental results demonstrate that cPSO-CNN performs competitively when compared with several reported algorithms in terms of both CNN hyper-parameter superiority and overall computation cost.

Evolving Ensemble Models for Image Segmentation Using Enhanced Particle Swarm Optimization

In this paper, we propose particle swarm optimization (PSO)-enhanced ensemble deep neural networks and hybrid clustering models for skin lesion segmentation. A PSO variant is proposed, which embeds diverse search actions including simulated annealing, levy flight, helix behavior, modified PSO, and differential evolution operations with spiral search coefficients. These search actions work in a cascade manner to not only equip each individual with different search operations throughout the search process but also assign distinctive search actions to different particles simultaneously in every single iteration. The proposed PSO variant is used to optimize the learning hyper-parameters of convolutional neural networks (CNNs) and the cluster centroids of classical Fuzzy C-Means clustering respectively to overcome performance barriers. Ensemble deep networks and hybrid clustering models are subsequently constructed based on the optimized CNN and hybrid clustering segmenters for lesion segmentation. We evaluate the proposed ensemble models using three skin lesion databases, i.e., PH2, ISIC 2017, and Dermofit Image Library, and a blood cancer data set, i.e., ALL-IDB2. The empirical results indicate that our models outperform other hybrid ensemble clustering models combined with advanced PSO variants, as well as state-of-the-art deep networks in the literature for diverse challenging image segmentation tasks.