Optimal Convolutional Neural Network Research Articles

Deep convolutional neural networks for eigenvalue problems in mechanics

SummaryWe show that deep convolutional neural networks (CNNs) can massively outperform traditional densely connected neural networks (NNs) (both deep or shallow) in predicting eigenvalue problems in mechanics. In this sense, we strike out in a new direction in mechanics computations with strongly predictive NNs whose success depends not only on architectures being deep but also being fundamentally different from the widely used to date. We consider a model problem: predicting the eigenvalues of one‐dimensional (1D) and two‐dimensional (2D) phononic crystals. For the 1D case, the optimal CNN architecture reaches 98% accuracy level on unseen data when trained with just 20 000 samples, compared to 85% accuracy even with 100 000 samples for the typical network of choice in mechanics research. We show that, with relatively high data efficiency, CNNs have the capability to generalize well and automatically learn deep symmetry operations, easily extending to higher dimensions and our 2D case. Most importantly, we show how CNNs can naturally represent mechanical material tensors, with its convolution kernels serving as local receptive fields, which is a natural representation of mechanical response. Strategies proposed are applicable to other mechanics' problems and may, in the future, be used to sidestep cumbersome algorithms with purely data‐driven approaches based upon modern deep architectures.

Solar cell surface defect inspection based on multispectral convolutional neural network

Similar and indeterminate defect detection of solar cell surface with heterogeneous texture and complex background is a challenge of solar cell manufacturing. The traditional manufacturing process relies on human eye detection which requires a large number of workers without a stable and good detection effect. In order to solve the problem, a visual defect detection method based on multi-spectral deep convolutional neural network (CNN) is designed in this paper. Firstly, a selected CNN model is established. By adjusting the depth and width of the model, the influence of model depth and kernel size on the recognition result is evaluated. The optimal CNN model structure is selected. Secondly, the light spectrum features of solar cell color image are analyzed. It is found that a variety of defects exhibited different distinguishable characteristics in different spectral bands. Thus, a multi-spectral CNN model is constructed to enhance the discrimination ability of the model to distinguish between complex texture background features and defect features. Finally, some experimental results and K-fold cross validation show that the multi-spectral deep CNN model can effectively detect the solar cell surface defects with higher accuracy and greater adaptability. The accuracy of defect recognition reaches 94.30%. Applying such an algorithm can increase the efficiency of solar cell manufacturing and make the manufacturing process smarter.

Particle bee optimized convolution neural network for managing security using cross-layer design in cognitive radio network

Cognitive radio (CR) is one of the promising technologies for better utilization of radio spectrum. The parameters like operating frequency, transmission power, type of modulation etc., extend a major contribution in CR. These parameters helps to transmit the information in communication environment. The CR monitors their own performance and subsequently reads the radio outputs in radio frequency environment in an effective manner. During the process of this cognitive communication, security is one of the major issues owing to the factors such as anomalous spectrum usage, hindrance in the identification of primary and secondary spectrum users. This paper introduces Cross-Layer Design Based Particle Bee optimized Convolution Neural Network (CLPBCNN) for managing the security issues present in the cognitive radio networks. Initially, user’s spectrum has been sensed based on the energy level with cyclostationary process which helps to analyze the primary user related radio features. Along with the extracted user features, authentication is handled by symmetric triple data encryption algorithm and the features are trained by proposed network which examines the intermediate attacks and malicious activities. Furthermore, an efficient multi-path energy routing protocol is proposed to monitor the neighboring nodes and spectrum usage for detecting the attacks in CR. This process is repeated continuously for creating the effective attack free cognitive radio based wireless communication system. Then the excellence of the system is evaluated and validated with the help of NS2 simulation tool and the efficiency is analyzed in terms of miss detection probability, route dis-connectivity ratio, and detection delay. The effective utilization of this cyclostationary and routing process leads to overall improved performance.

Packing Convolutional Neural Networks in the Frequency Domain.

Deep convolutional neural networks (CNNs) are successfully used in a number of applications. However, their storage and computational requirements have largely prevented their widespread use on mobile devices. Here we present a series of approaches for compressing and speeding up CNNs in the frequency domain, which focuses not only on smaller weights but on all the weights and their underlying connections. By treating convolution filters as images, we decompose their representations in the frequency domain as common parts (i.e., cluster centers) shared by other similar filters and their individual private parts (i.e., individual residuals). A large number of low-energy frequency coefficients in both parts can be discarded to produce high compression without significantly compression romising accuracy. Furthermore, we explore a data-driven method for removing redundancies in both spatial and frequency domains, which allows us to discard more useless weights by keeping similar accuracies. After obtaining the optimal sparse CNN in the frequency domain, we relax the computational burden of convolution operations in CNNs by linearly combining the convolution responses of discrete cosine transform (DCT) bases. The compression and speed-up ratios of the proposed algorithm are thoroughly analyzed and evaluated on benchmark image datasets to demonstrate its superiority over state-of-the-art methods.

Multiclass classification of obstructive sleep apnea/hypopnea based on a convolutional neural network from a single-lead electrocardiogram

Objective: In this paper, we propose a convolutional neural network (CNN)-based deep learning architecture for multiclass classification of obstructive sleep apnea and hypopnea (OSAH) using single-lead electrocardiogram (ECG) recordings. OSAH is the most common sleep-related breathing disorder. Many subjects who suffer from OSAH remain undiagnosed; thus, early detection of OSAH is important. Approach: In this study, automatic classification of three classes—normal, hypopnea, and apnea—based on a CNN is performed. An optimal six-layer CNN model is trained on a training dataset (45 096 events) and evaluated on a test dataset (11 274 events). The training set (69 subjects) and test set (17 subjects) were collected from 86 subjects with length of approximately 6 h and segmented into 10 s durations. Main results: The proposed CNN model reaches a mean -score of 93.0 for the training dataset and 87.0 for the test dataset. Significance: Thus, proposed deep learning architecture achieved a high performance for multiclass classification of OSAH using single-lead ECG recordings. The proposed method can be employed in screening of patients suspected of having OSAH.

Convolutional neural network-based PSO for lung nodule false positive reduction on CT images

Background and objectiveDetection of lung nodules is critical in CAD systems; this is because of their similar contrast with other structures and low density, which result in the generation of numerous false positives (FPs). Therefore, this study proposes a methodology to reduce the FP number using a deep learning technique in conjunction with an evolutionary technique. MethodThe particle swarm optimization (PSO) algorithm was used to optimize the network hyperparameters in the convolutional neural network (CNN) in order to enhance the network performance and eliminate the requirement of manual search. ResultsThe methodology was tested on computed tomography (CT) scans from the Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) with the highest accuracy of 97.62%, sensitivity of 92.20%, specificity of 98.64%, and area under the receiver operating characteristic (ROC) curve of 0.955. ConclusionThe results demonstrate the high performance-potential of the PSO algorithm in the identification of optimal CNN hyperparameters for lung nodule candidate classification into nodules and non-nodules, increasing the sensitivity rates in the FP reduction step of CAD systems.

Optimization of CNN through Novel Training Strategy for Visual Classification Problems.

The convolution neural network (CNN) has achieved state-of-the-art performance in many computer vision applications e.g., classification, recognition, detection, etc. However, the global optimization of CNN training is still a problem. Fast classification and training play a key role in the development of the CNN. We hypothesize that the smoother and optimized the training of a CNN goes, the more efficient the end result becomes. Therefore, in this paper, we implement a modified resilient backpropagation (MRPROP) algorithm to improve the convergence and efficiency of CNN training. Particularly, a tolerant band is introduced to avoid network overtraining, which is incorporated with the global best concept for weight updating criteria to allow the training algorithm of the CNN to optimize its weights more swiftly and precisely. For comparison, we present and analyze four different training algorithms for CNN along with MRPROP, i.e., resilient backpropagation (RPROP), Levenberg–Marquardt (LM), conjugate gradient (CG), and gradient descent with momentum (GDM). Experimental results showcase the merit of the proposed approach on a public face and skin dataset.

Convolutional Neural Networks for the Detection of Diseased Hearts Using CT Images and Left Atrium Patches.

Cardiovascular disease is a leading cause of death in the United States. The identification of cardiac diseases on conventional three-dimensional (3D) CT can have many clinical applications. An automated method that can distinguish between healthy and diseased hearts could improve diagnostic speed and accuracy when the only modality available is conventional 3D CT. In this work, we proposed and implemented convolutional neural networks (CNNs) to identify diseased hears on CT images. Six patients with healthy hearts and six with previous cardiovascular disease events received chest CT. After the left atrium for each heart was segmented, 2D and 3D patches were created. A subset of the patches were then used to train separate convolutional neural networks using leave-one-out cross-validation of patient pairs. The results of the two neural networks were compared, with 3D patches producing the higher testing accuracy. The full list of 3D patches from the left atrium was then classified using the optimal 3D CNN model, and the receiver operating curves (ROCs) were produced. The final average area under the curve (AUC) from the ROC curves was 0.840 ± 0.065 and the average accuracy was 78.9% ± 5.9%. This demonstrates that the CNN-based method is capable of distinguishing healthy hearts from those with previous cardiovascular disease.

Convolutional Neural Network Hyper-Parameters Optimization based on Genetic Algorithms

In machine learning for computer vision based applications, Convolutional Neural Network (CNN) is the most widely used technique for image classification. Despite these deep neural networks efficiency, choosing their optimal architecture for a given task remains an open problem. In fact, CNNs performance depends on many hyper-parameters namely CNN depth, convolutional layer number, filters number and their respective sizes. Many CNN structures have been manually designed by researchers and then evaluated to verify their efficiency. In this paper, our contribution is to propose an innovative approach, labeled Enhanced Elite CNN Model Propagation (Enhanced E-CNN-MP), to automatically learn the optimal structure of a CNN. To traverse the large search space of candidate solutions our approach is based on Genetic Algorithms (GA). These meta-heuristic algorithms are well-known for non-deterministic problem resolution. Simulations demonstrate the ability of the designed approach to compute optimal CNN hyper-parameters in a given classification task. Classification accuracy of the designed CNN based on Enhanced E-CNN-MP method, exceed that of public CNN even with the use of the Transfer Learning technique. Our contribution advances the current state by offering to scientists, regardless of their field of research, the ability of designing optimal CNNs for any particular classification problem.

Diagnosis and prediction of periodontally compromised teeth using a deep learning-based convolutional neural network algorithm.

PurposeThe aim of the current study was to develop a computer-assisted detection system based on a deep convolutional neural network (CNN) algorithm and to evaluate the potential usefulness and accuracy of this system for the diagnosis and prediction of periodontally compromised teeth (PCT).MethodsCombining pretrained deep CNN architecture and a self-trained network, periapical radiographic images were used to determine the optimal CNN algorithm and weights. The diagnostic and predictive accuracy, sensitivity, specificity, positive predictive value, negative predictive value, receiver operating characteristic (ROC) curve, area under the ROC curve, confusion matrix, and 95% confidence intervals (CIs) were calculated using our deep CNN algorithm, based on a Keras framework in Python.ResultsThe periapical radiographic dataset was split into training (n=1,044), validation (n=348), and test (n=348) datasets. With the deep learning algorithm, the diagnostic accuracy for PCT was 81.0% for premolars and 76.7% for molars. Using 64 premolars and 64 molars that were clinically diagnosed as severe PCT, the accuracy of predicting extraction was 82.8% (95% CI, 70.1%–91.2%) for premolars and 73.4% (95% CI, 59.9%–84.0%) for molars.ConclusionsWe demonstrated that the deep CNN algorithm was useful for assessing the diagnosis and predictability of PCT. Therefore, with further optimization of the PCT dataset and improvements in the algorithm, a computer-aided detection system can be expected to become an effective and efficient method of diagnosing and predicting PCT.

Effective training of convolutional neural networks for face-based gender and age prediction

Convolutional Neural Networks (CNNs) have been proven very effective for human demographics estimation by a number of recent studies. However, the proposed solutions significantly vary in different aspects leaving many open questions on how to choose an optimal CNN architecture and which training strategy to use. In this work, we shed light on some of these questions improving the existing CNN-based approaches for gender and age prediction and providing practical hints for future studies. In particular, we analyse four important factors of the CNN training for gender recognition and age estimation: (1) the target age encoding and loss function, (2) the CNN depth, (3) the need for pretraining, and (4) the training strategy: mono-task or multi-task. As a result, we design the state-of-the-art gender recognition and age estimation models according to three popular benchmarks: LFW, MORPH-II and FG-NET. Moreover, our best model won the ChaLearn Apparent Age Estimation Challenge 2016 significantly outperforming the solutions of other participants.

基于优化卷积神经网络的图像超分辨率重建

基于优化卷积神经网络的图像超分辨率重建

Hybrid Algorithm for the Optimization of Training Convolutional Neural Network

The training optimization processes and efficient fast classification are vital elements in the development of a convolution neural network (CNN). Although stochastic gradient descend (SGD) is a Prevalence algorithm used by many researchers for the optimization of training CNNs, it has vast limitations. In this paper, it is endeavor to diminish and tackle drawbacks inherited from SGD by proposing an alternate algorithm for CNN training optimization. A hybrid of genetic algorithm (GA) and particle swarm optimization (PSO) is deployed in this work. In addition to SGD, PSO and genetic algorithm (PSO-GA) are also incorporated as a combined and efficient mechanism in achieving non trivial solutions. The proposed unified method achieves state-of-the-art classification results on the different challenge benchmark datasets such as MNIST, CIFAR-10, and SVHN. Experimental results showed that the results outperform and achieve superior results to most contemporary approaches.