Deep Pyramidal Residual Networks for Spectral–Spatial Hyperspectral Image Classification

Convolutional Neural Networks (CNN) are useful methods for identification of previously unknown embedded patterns in images. Several object and facial recognition along with image segmentation tasks have benefited from the non-linear abstraction of hybrid features using CNN. This work presents a novel CNN model parametrization work-flow developed on the cloud-computing platform of Microsoft Azure Machine Learning Studio (MAMLS) that is capable of learning from the feature maps and classifying multi-modal images with different variabilities using one common flow. This two-step work-flow trains CNN models using 70/30 data split. First, the CNN layers are fixed and the optimal kernel and normalization parameters are identified that maximize classification accuracy on the test data. Next, using the optimal kernel and normalization parameters, the best CNN architecture that maximizes classification accuracy is detected. Finally, the activated feature maps (AFMs) from the optimally parameterized CNN model are analyzed to learn new features that can enhance image-based classification accuracies. The proposed flow achieves classification accuracies in the range of 92.5–99.2% that can be further enhanced by doubling the samples based on the features learned from the AFMs. The proposed non-deep CNN models in the MAMLS platform are capable of processing image data sets with 400–4 million samples using a common flow without exponential increase in the computation time. Thus, optimally parametrized non-deep CNN models are capable of identifying novel features that may enhance image-based classification accuracies.

Convolutional neural networks for hyperspectral image classification

BackgroundHuman Down syndrome (DS) is usually caused by genomic micro-duplications and dosage imbalances of human chromosome 21. It is associated with many genomic and phenotype abnormalities. Even though human DS occurs about 1 per 1,000 births worldwide, which is a very high rate, researchers haven’t found any effective method to cure DS. Currently, the most efficient ways of human DS prevention are screening and early detection.MethodsIn this study, we used deep learning techniques and analyzed a set of Illumina genotyping array data. We built a bi-stream convolutional neural networks model to screen/predict the occurrence of DS. Firstly, we built image input data by converting the intensities of each SNP site into chromosome SNP maps. Next, we proposed a bi-stream convolutional neural network (CNN) architecture with nine layers and two branch models. We further merged two CNN branch models into one model in the fourth convolutional layer, and output the prediction in the last layer.ResultsOur bi-stream CNN model achieved 99.3% average accuracies, and very low false-positive and false-negative rates, which was necessary for further applications in disease prediction and medical practice. We further visualized the feature maps and learned filters from intermediate convolutional layers, which showed the genomic patterns and correlated SNPs variations in human DS genomes. We also compared our methods with other CNN and traditional machine learning models. We further analyzed and discussed the characteristics and strengths of our bi-stream CNN model.ConclusionsOur bi-stream model used two branch CNN models to learn the local genome features and regional patterns among adjacent genes and SNP sites from two chromosomes simultaneously. It achieved the best performance in all evaluating metrics when compared with two single-stream CNN models and three traditional machine-learning algorithms. The visualized feature maps also provided opportunities to study the genomic markers and pathway components associated with Human DS, which provided insights for gene therapy and genomic medicine developments.

The convolutional neural network (CNN) method has been widely used in the classification of hyperspectral images (HSIs). However, the efficiency and accuracy of the HSI classification are inevitably degraded when small samples are available. This study proposes a multidimensional CNN model named MDAN, which is constructed with an attention mechanism, to achieve an ideal classification performance of CNN within the framework of few-shot learning. In this model, a three-dimensional (3D) convolutional layer is carried out for obtaining spatial–spectral features from the 3D volumetric data of HSI. Subsequently, the two-dimensional (2D) and one-dimensional (1D) convolutional layers further learn spatial and spectral features efficiently at an abstract level. Based on the most widely used convolutional block attention module (CBAM), this study investigates a convolutional block self-attention module (CBSM) to improve accuracy by changing the connection ways of attention blocks. The CBSM model is used with the 2D convolutional layer for better performance of HSI classification purposes. The MDAN model is applied for classification applications using HSI, and its performance is evaluated by comparing the results with the support vector machine (SVM), 2D CNN, 3D CNN, 3D–2D–1D CNN, and CBAM. The findings of this study indicate that classification results from the MADN model show overall classification accuracies of 97.34%, 96.43%, and 92.23% for Salinas, WHU-Hi-HanChuan, and Pavia University datasets, respectively, when only 1% HSI data were used for training. The training and testing times of the MDAN model are close to those of the 3D–2D–1D CNN, which has the highest efficiency among all comparative CNN models. The attention model CBSM is introduced into MDAN, which achieves an overall accuracy of about 1% higher than that of the CBAM model. The performance of the two proposed methods is superior to the other models in terms of both efficiency and accuracy. The results show that the combination of multidimensional CNNs and attention mechanisms has the best ability for small-sample problems in HSI classification.

Hyperspectral image (HSI) classification has been one of the most important tasks in the remote sensing community over the last few decades. Due to the presence of highly correlated bands and limited training samples in HSI, discriminative feature extraction was challenging for traditional machine learning methods. Recently, deep learning based methods have been recognized as powerful feature extraction tool and have drawn a significant amount of attention in HSI classification. Among various deep learning models, convolutional neural networks (CNNs) have shown huge success and offered great potential to yield high performance in HSI classification. Motivated by this successful performance, this paper presents a systematic review of different CNN architectures for HSI classification and provides some future guidelines. To accomplish this, our study has taken a few important steps. First, we have focused on different CNN architectures, which are able to extract spectral, spatial, and joint spectral-spatial features. Then, many publications related to CNN based HSI classifications have been reviewed systematically. Further, a detailed comparative performance analysis has been presented between four CNN models namely 1D CNN, 2D CNN, 3D CNN, and feature fusion based CNN (FFCNN). Four benchmark HSI datasets have been used in our experiment for evaluating the performance. Finally, we concluded the paper with challenges on CNN based HSI classification and future guidelines that may help the researchers to work on HSI classification using CNN.

Of late, convolutional neural networks (CNNs) find great attention in hyperspectral image (HSI) classification since deep CNNs exhibit commendable performance for computer vision-related areas. CNNs have already proved to be very effective feature extractors, especially for the classification of large data sets composed of 2-D images. However, due to the existence of noisy or correlated spectral bands in the spectral domain and nonuniform pixels in the spatial neighborhood, HSI classification results are often degraded and unacceptable. However, the elementary CNN models often find intrinsic representation of pattern directly when employed to explore the HSI in the spectral-spatial domain. In this article, we design an end-to-end spectral-spatial squeeze-and-excitation (SE) residual bag-of-feature (S3EResBoF) learning framework for HSI classification that takes as input raw 3-D image cubes without engineering and builds a codebook representation of transform feature by motivating the feature maps facilitating classification by suppressing useless feature maps based on patterns present in the feature maps. To boost the classification performance and learn the joint spatial-spectral features, every residual block is connected to every other 3-D convolutional layer through an identity mapping followed by an SE block, thereby facilitating the rich gradients through backpropagation. Additionally, we introduce batch normalization on every convolutional layer (ConvBN) to regularize the convergence of the network and scale invariant BoF quantization for the measure of classification. The experiments conducted using three well-known HSI data sets and compared with the state-of-the-art classification methods reveal that S3EResBoF provides competitive performance in terms of both classification and computation time.

Convolutional Neural Networks (CNN) have emerged as a viable solution for text classification, including aspect extraction. Yet the existing state-of-the-art CNN architectures used in aspect extraction do not incorporate improvements (e.g. non-static CNN, multi-kernel convolution layers, and dropout regularization) that have been identified as beneficial for general text classification tasks. In this research, we present an improved CNN architecture with these enhancements for aspect extraction. Our modifications to the basic CNN model improve feature extraction of the convolutional layer, while allowing the dense layers to represent complex non-linear relationships between selected features and the output layer. Furthermore, we show that using non-static CNN instead of static CNN further fine-tunes the word embedding features for the specific domain of customer reviews in the absence of domain-specific corpora to train word2Vec models. We show that using Skip-gram trained word2vec models opposed to Continuous bag of word (CBOW) architecture improves the quality of word2vec embedding features for aspect extraction and text classification tasks. This new CNN model and baseline CNN are evaluated using SemEval-2016 Task 5 Restaurant and Laptop domain datasets. Our CNN model shows significant improvements over the baseline CNN model for aspect extraction from customer reviews for both restaurant and laptop domains. It also outperforms the currently available state-of-the-art systems for restaurant domain.

Hyperspectral image (HSI) classification is an active and important research task driven by many practical applications. To leverage deep learning models especially convolutional neural networks (CNNs) for HSI classification, this paper proposes a simple yet effective method to extract hierarchical deep spatial feature for HSI classification by exploring the power of off-the-shelf CNN models, without any additional retraining or fine-tuning on the target data set. To obtain better classification accuracy, we further propose a unified metric learning-based framework to alternately learn discriminative spectral–spatial features, which have better representation capability and train support vector machine (SVM) classifiers. To this end, we design a new objective function that explicitly embeds a metric learning regularization term into SVM training. The metric learning regularization term is used to learn a powerful spectral–spatial feature representation by fusing spectral feature and deep spatial feature, which has small intraclass scatter but big between class separation. By transforming HSI data into new spectral–spatial feature space through CNN and metric learning, we can pull the pixels from the same class closer, while pushing the different class pixels farther away. In the experiments, we comprehensively evaluate the proposed method on three commonly used HSI benchmark data sets. State-of-the-art results are achieved when compared with the existing HSI classification methods.

The Part 1 prequel to this review evaluated the evolution of modelling techniques used in evaluation of fruit quality over the past three decades and noted a progression towards the use of artificial neural networks (ANNs) and convolutional neural networks (CNNs). In this review, Part 2, the use of CNNs for NIR fruit quality evaluation is explored, given the success of CNNs in various other fields, such as image, video, speech, and audio processing, and the availability of large (open source) datasets of fruit spectra and reference quality attribute, which is required for the training of CNN models. The review provides an overview of deep learning and the CNN architectures and techniques used in NIR spectroscopy for regression modelling, with advantages and disadvantages identified. Studies using CNN for NIR based fruit quality evaluation are then critically examined. Eight publications have presented on models using the same open-source mango dry matter calibration and test set, enabling inter-method comparisons. CNN models have been demonstrated to be accurate, precise and robust. Techniques of transfer learning for CNN models offer an alternative solution to model updating and calibration transfer methods applied in traditional chemometrics. The review has highlighted crucial areas that require resolution and exploration in this application through future research, including, (i) data requirements for training a CNN (ii) optimal spectral pre-processing for CNN (iii) CNN architecture and hyper-parameter selection and tuning for fruit quality evaluation (iv) CNN model interpretability and explainability. Future studies must conduct clearer comparison to partial least squares (PLS) regression and shallow ANNs to better assess the prospective benefit of using CNN, a more complex model. The potential for visualisation of spectra relevance to the CNN model using techniques such as GradCam, currently employed in visualising 2D-CNN models, remains to be explored.

One of the diseases that attacks the lungs is pneumonia. Pneumonia is inflammation and fluid in the lungs making it difficult to breathe. This disease is diagnosed using X-Ray. Against the darker background of the lungs, infected tissue shows denser areas, which causes them to appear as white spots called infiltrates. In the image processing approach, pneumonia-infected X-rays can be detected using machine learning as well as deep learning. The convolutional neural network model is able to recognize images well and focus on points that are invisible to the human eye. Previous research using a convolutional neural network model with 10 convolution layers and 6 convolution layers has not achieved optimal accuracy. The aim of this research is to develop a convolutional neural network with a simpler architecture, namely two convolution layers and three convolution layers to solve the same problem, as well as examining the combination of various hyperparameter sizes and regularization techniques. We need to know which convolutional neural network architecture is better. As a result, the convolutional neural network classification model can recognize chest x-rays infected with pneumonia very well. The best classification model obtained an average accuracy of 89.743% with a three-layer convolution architecture, batch size 32, L2 regularization 0.0001, and dropout 0.2. The precision reached 94.091%, recall 86.456%, f1-score 89.601%, specificity 85.491, and error rate 10.257%. Based on the results obtained, convolutional neural network models have the potential to diagnose pneumonia and other diseases.

Facial expression recognition is a challenging problem in image classification. Recently, the use of deep learning is gaining importance in image classification. This has led to increased efforts in solving the problem of facial expression recognition using convolutional neural networks (CNNs). A significant challenge in deep learning is to design a network architecture that is simple and effective. A simple architecture is fast to train and easy to implement. An effective architecture achieves good accuracy on the test data. CNN architectures are black boxes to us. VGGNet, AlexNet and Inception are well-known CNN architectures. These architectures have strongly influenced CNN model designs for new datasets. Almost all CNN models known to achieve high accuracy on facial expression recognition problem are influenced by these architectures. This work tries to overcome this limitation by using FER-2013 dataset as starting point to design new CNN models. In this work, the effect of CNN parameters namely kernel size and number of filters on the classification accuracy is investigated using FER-2013 dataset. Our major contribution is a thorough evaluation of different kernel sizes and number of filters to propose two novel CNN architectures which achieve a human-like accuracy of 65% (Goodfellow et al. in: Neural information processing, Springer, Berlin, pp 117–124, 2013) on FER-2013 dataset. These architectures can serve as a basis for standardization of the base model for the much inquired FER-2013 dataset.

BackgroundLung segmentation constitutes a critical procedure for any clinical-decision supporting system aimed to improve the early diagnosis and treatment of lung diseases. Abnormal lungs mainly include lung parenchyma with commonalities on CT images across subjects, diseases and CT scanners, and lung lesions presenting various appearances. Segmentation of lung parenchyma can help locate and analyze the neighboring lesions, but is not well studied in the framework of machine learning.MethodsWe proposed to segment lung parenchyma using a convolutional neural network (CNN) model. To reduce the workload of manually preparing the dataset for training the CNN, one clustering algorithm based method is proposed firstly. Specifically, after splitting CT slices into image patches, the k-means clustering algorithm with two categories is performed twice using the mean and minimum intensity of image patch, respectively. A cross-shaped verification, a volume intersection, a connected component analysis and a patch expansion are followed to generate final dataset. Secondly, we design a CNN architecture consisting of only one convolutional layer with six kernels, followed by one maximum pooling layer and two fully connected layers. Using the generated dataset, a variety of CNN models are trained and optimized, and their performances are evaluated by eightfold cross-validation. A separate validation experiment is further conducted using a dataset of 201 subjects (4.62 billion patches) with lung cancer or chronic obstructive pulmonary disease, scanned by CT or PET/CT. The segmentation results by our method are compared with those yielded by manual segmentation and some available methods.ResultsA total of 121,728 patches are generated to train and validate the CNN models. After the parameter optimization, our CNN model achieves an average F-score of 0.9917 and an area of curve up to 0.9991 for classification of lung parenchyma and non-lung-parenchyma. The obtain model can segment the lung parenchyma accurately for 201 subjects with heterogeneous lung diseases and CT scanners. The overlap ratio between the manual segmentation and the one by our method reaches 0.96.ConclusionsThe results demonstrated that the proposed clustering algorithm based method can generate the training dataset for CNN models. The obtained CNN model can segment lung parenchyma with very satisfactory performance and have the potential to locate and analyze lung lesions.

This study builds a convolutional neural network (CNN) architecture specifically to identify Alzheimer’s disease. The background of this study is that at present there is no proper testing tool for AD and there is a pressing need for the diagnosis and treatment of this disorder; which can greatly impact patient outcomes. The goal of this study is to propose an AD classification system based on CNN architecture that classifies four categories in real-time by MRI images. The dataset contained 6400 MRI images and the CNN model was trained for an epoch using a batch size of $\mathbf{3 2}$ running at the default learning rate over 20 epochs. The CNN architecture was formed through powerful iterative rounds of training to make sure it can perform accurately when divided into Alzheimer’s disease subtypes. The results show that this CNN model achieved a stunning 99% accuracy rate in diagnosing Alzheimer’s disease based on MRI images. A standard method for AD estimation, the results achieved for age-specific access are highly instructive for determining the effectiveness of CNN-based methodologies in medical image analysis and differentiating diseases. The results of this study are significant because it can be diagnosed and classified into a system, as a matter of course Alzheimer’s will be detected earlier the outcome for patients made better treatment outcomes superior. The CNN model proposed in this study creates an opportunity for future testing systems in Alzheimer’s disease classification that promise significantly improved performance. This in turn will contribute not only to the digital image technology of medical imaging technology but also to patient care.

In this paper, the Convolutional Neural Network (CNN) architecture, which is one of the deep learning architectures, is used to classify the basic circuit components drawn by hand. During the training and testing stages of the model, a new dataset containing images of 863 circuit components manually drawn by different people is created. The data set contains images of four different classes of circuit components such as resistor, inductor, capacitor and voltage source. All images have been fixed to the same size and converted to grayscale to increase recognition performance and reduce process complexity. In the study, training for four classes is performed with CNN architecture. Based on the CNN architecture, four new CNN models are employed with different the number of layers. The training and validation results of these models are compared separately, the model with the highest training and validation performance is observed with four layer CNN model (CNN-4). This model obtained 84.41% accuracy rate at classification task.

INTRODUCTION: When Diabetic Retinopathy (DR) is not identified promptly; it frequently results in sight impairment. To properly diagnose and treat DR, preprocessing of picture methods and precise prediction models are essential. With the help of numerous well-liked filters and a Deep CNN (Convolutional Neural Network) model, the comprehensive method for DR image preparation and prognosis presented in this research is described. Using the filters that focus boundaries and contours in the ocular pictures is the first step in the initial processing stage. This procedure tries to find anomalies linked to DR. By the usage of filters, the excellence of pictures can be developed and minimize disturbances, preserving critical information. The Deep CNN algorithm has been trained to generate forecasts on the cleaned retinal pictures following the phase of preprocessing. The filters efficiently eliminate interference without sacrificing vital data. Convolutional type layers, pooling type layers, and fully associated layers are used in the CNN framework, which was created especially for image categorization tasks, to acquire data and understand the relationships associated with DR. OBJECTIVES: Using image preprocessing techniques such as the Sobel, Wiener, Gaussian, and non-local mean filters is a promising approach for DR analysis. Then, predicting using a CNN completes the approach. These preprocessing filters enhance the images and prepare them for further examination. The pre-processed images are fed into a CNN model. The model extracts significant information from the images by identifying complex patterns. DR or classification may be predicted by the CNN model through training on a labeled dataset. METHODS: The Method Preprocessing is employed for enhancing the clarity and difference of retina fundus picture by removing noise and fluctuation. The preprocessing stage is utilized for the normalization of the pictures and non-uniform brightness adjustment in addition to contrast augmentation and noise mitigation to remove noises and improve the rate of precision of the subsequent processing stages. RESULTS: To improve image quality and reduce noise, preprocessing techniques including Sobel, Wiener, Gaussian, and non-local mean filters are frequently employed in image processing jobs. For a particular task, the non-local mean filter produces superior results; for enhanced performance, it may be advantageous to combine it with a CNN. Before supplying the processed images to the CNN for prediction, the non-local mean filter can assist reduce noise and improve image details. CONCLUSION: A promising method for DR analysis entails the use of image preprocessing methods such as the Sobel, Wiener, Gaussian, and non-local mean filters, followed by prediction using a CNN. These preprocessing filters improve the photos and get them ready for analysis. After being pre-processed, the photos are sent into a CNN model, which uses its capacity to discover intricate patterns to draw out important elements from the images. The CNN model may predict DR or classification by training it on a labeled dataset. The development of computer-aided diagnosis systems for DR is facilitated by the integration of CNN prediction with image preprocessing filters. This strategy may increase the effectiveness of healthcare workers, boost patient outcomes, and lessen the burden of DR.