Deep Learning Based Image Classification Research Articles

A novel CapsNet neural network based on MobileNetV2 structure for robot image classification.

Image classification indicates that it classifies the images into a certain category according to the information in the image. Therefore, extracting image feature information is an important research content in image classification. Traditional image classification mainly uses machine learning methods to extract features. With the continuous development of deep learning, various deep learning algorithms are gradually applied to image classification. However, traditional deep learning-based image classification methods have low classification efficiency and long convergence time. The training networks are prone to over-fitting. In this paper, we present a novel CapsNet neural network based on the MobileNetV2 structure for robot image classification. Aiming at the problem that the lightweight network will sacrifice classification accuracy, the MobileNetV2 is taken as the base network architecture. CapsNet is improved by optimizing the dynamic routing algorithm to generate the feature graph. The attention module is introduced to increase the weight of the saliency feature graph learned by the convolutional layer to improve its classification accuracy. The parallel input of spatial information and channel information reduces the computation and complexity of network. Finally, the experiments are carried out in CIFAR-100 dataset. The results show that the proposed model is superior to other robot image classification models in terms of classification accuracy and robustness.

Random augmentation pipe for hyperspectral image classification with few samples

ABSTRACT In recent years, deep learning has developed rapidly thanks to advances in computing capabilities. Researchers in many fields, including remote sensing, have used this technology to achieve remarkable breakthroughs. For deep learning-based image classification tasks, including hyperspectral image classification, a large amount of labelled data is needed. However, manually labelling hyperspectral images is expensive and difficult. We aim to design a new approach to considerably improve the classification performance when the labelled data are extremely few without adding too much computational overhead. As a simple and viable solution, applying data augmentation at the pre-processing stage to improve the classifier’s performance is becoming a trend. Apart from traditional data augmentation, like flipping, rotation, and adding noise, several new augmentation methods are proposed, including Cut-out, CutMix, sample pairing, and utilizing unlabelled hyperspectral data. Furthermore, existing studies have proven their effectiveness under a sparse data regime. However, when extremely few labelled samples are available, the classification accuracy of current methods is relatively low. Therefore, we propose Random Augmentation Pipe by sequentially employing a series of augmentations at the training stage. To prevent excessive computational cost, one affine transformation is introduced as a substitute for all the geometric augmentations. Moreover, the Pipe is extended by introducing five other augmentations. Among the augmentations, Square Rotation provides a new distortion without changing the original data, that effectively enhances performance. Furthermore, a principal component analysis and extended morphological profile are applied at the pre-processing stage. The Pipe is then embedded at the beginning of a carefully designed deep learning network built on convolutional neural networks and deep residual networks. Tests were performed over five commonly used datasets, including Pavia University, Pavia Centre, Botswana, Indian Pines, and Kennedy Space Centre. Through a set of comprehensive experiments, it was shown that the proposed Extended-RAP outperformed the 16 other comparison methods.

Automatic Detection of Local Obstacles in a Long Length RE-123 Coated Conductor by Deep Learning Based Image Classification in Reel-to-Reel Magnetic Microscopy

We have succeeded in detecting local obstacles automatically in a 200-m-long RE-123 coated conductor (CC) by introducing deep-learning based image recognition in reel-to-reel scanning Hall probe microscopy (RTR-SHPM). Longitudinal critical current ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I_c</i> ) homogeneity in CCs is one of the most important requirements for practical applications. Usually, such properties as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I_c</i> variation as a function of longitudinal coordinate is characterized by magnetization measurements adopting Hall probe array as a de facto standard characterization method for ensuring uniformity in long CCs. In this measurement, local <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I_c</i> drop indicates the existence of current blocking obstacles. The group of authors also developed a magnetic microscopy applicable to reel-to-reel continuous measurements, RTR-SHPM, which makes it possible to visualize two-dimensional critical current density ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">J_c</i> ) in the tape plane because of its high-resolution imaging along the tape width. As a result, more elaborate defect detection has been enabled. However, in the conventional technique, the observation depends on the human eye in order for characterizing detailed features of the obstacles such as shape and size especially in case with a small <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</sub> drop, therefore, there was a limit to analyze the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">J_c</i> mapping with such detailed information extending to thousands of images in the long tape of several 100 of meters. In this study, the image classification based on the deep learning method was introduced in our magnetic microscopy. The analytical model classifies the input image into the defect position and the normal position, respectively, together with a heat map and a score of confidence in the recognition. As a result, we have succeeded in detecting obstacles automatically from more than 4,000 of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">J_c</i> images with a high accuracy of 98.5%. Furthermore, we revealed the existence of the obstacles which were not distinguishable by the local <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I_c</i> criterion. This method allows us to clarify the origin of the instability of long CC wire and will have a strong impact as an evaluation technique for dramatically improving the reliability of the CCs.

Transferability analysis of adversarial attacks on gender classification to face recognition: Fixed and variable attack perturbation

Most deep learning-based image classification models are vulnerable to adversarial attacks that introduce imperceptible changes to the input images for the purpose of model misclassification. It has been demonstrated that these attacks, targeting a specific model, are transferable among models performing the same task. However, models performing different tasks but sharing the same input space and model architecture were never considered in the transferability scenarios presented in the literature. In this paper, this phenomenon was analysed in the context of VGG16-based and ResNet50-based biometric classifiers. The authors investigate the impact of two white-box attacks on a gender classifier and contrast a defence method as a countermeasure. Then, using adversarial images generated by the attacks, a pre-trained face recognition classifier is attacked in a black-box fashion. Two verification comparison settings are employed, in which images perturbed with the same and different magnitude of the perturbation are compared. The authors’ results indicate transferability in the fixed perturbation setting for a Fast Gradient Sign Method attack and non-transferability in a pixel-guided denoiser attack setting. The interpretation of this non-transferability can support the use of fast and train-free adversarial attacks targeting soft biometric classifiers as means to achieve soft biometric privacy protection while maintaining facial identity as utility.

Automatic Damage Detection on Traditional Wooden Structures with Deep Learning-Based Image Classification Method

Wood has a long history of being used as a valuable resource when it comes to building materials. Due to various external factors, in particular the weather, wood is liable to progressive damage over time, which negatively impacts the endurance of wooden structures. Damage assessment is key in understanding, as well as in effectively mitigating, problems that wooden structures are likely to face. The use of a classification system, via deep learning, can potentially reduce the probability of damage in engineering projects reliant on wood. The present study employed a transfer learning technique, to achieve greater accuracy, and instead of training a model from scratch, to determine the likelihood of risks to wooden structures prior to project commencement. Pretrained MobileNet_V2, Inception_V3, and ResNet_V2_50 models were used to customize and initialize weights. A separate set of images, not shown to the trained model, was used to examine the robustness of the models. The three models were compared in their abilities to assess the possibilities and types of damage. Results revealed that all three models achieve performance rates of similar reliability. However, when considering the loss ratios in regard to efficiency, it became apparent that the multi-layered MobileNet_V2 classifier stood out as the most effective of the pre-trained deep convolutional neural network (CNN) models.

Deep learning-enabled image content-adaptive field sequential color LCDs with mini-LED backlight.

The mini-LED as the backlight of field sequential color LCD (FSC-LCD) enables high contrast, thin volume, and theoretically tripled light efficiency and resolution. However, color breakup (CBU) induced by a relative speed between an observer and the display severely limits the application of FSC-LCDs. Several driving algorithms have been proposed for CBU suppression, but their performance depends on image content. Moreover, their performance plateaus with increasing image segment number, preventing taking advantage of the massive segments introduced by mini-LEDs. Therefore, this study proposes an image content-adaptive driving algorithm for mini-LED FSC-LCDs. Deep learning-based image classification accurately determines the best FSC algorithm with the lowest CBU. In addition, the algorithm is heterogeneous that the image classification is independently performed in each segment, guaranteeing minimized CBU in all segments. We perform objective and subjective validation. Compared with the currently best algorithm, the proposed algorithm improves the performance in suppressing CBU by more than 20% using two evaluation metrics, supported by experiment-based subjective evaluation. Mini-LED FSC-LCDs driven by the proposed algorithm with outstanding CBU suppression can be ideal for display systems requiring high brightness and high resolution, such as head-up displays, virtual reality, and augmented reality displays.

Learning of Time-Frequency Attention Mechanism for Automatic Modulation Recognition

Recently, deep learning-based image classification and speech recognition approaches have made extensive use of attention mechanisms to achieve state-of-the-art recognition, which demonstrates the effectiveness of attention mechanisms. Motivated by the fact that the frequency and time information of modulated radio signals are crucial for modulation recognition, this letter proposes a time-frequency attention mechanism for convolutional neural network (CNN)-based automatic modulation recognition. The proposed time-frequency attention mechanism is designed to learn which channel, frequency and time information is more meaningful in CNN for modulation recognition. We analyze the effectiveness of the proposed attention mechanism and evaluate the performance of the proposed models. Experiment results show that the proposed methods outperform existing learning-based methods and attention mechanisms.

Training data reduction for deep learning-based image classifications using random sample consensus

Training data for deep learning algorithms can have many redundancies, which should be resolved to achieve faster training speed and efficient storage usage. We proposed a random sample consensus (RANSAC)-based training data selection technique to reduce the training data size for deep learning-based image classification tasks. First, we formulate the data reduction problem as a least square problem and reformulate the equation as maximizing the accuracy of the total training set. Based on the reformulated equation, we applied an RANSAC algorithm to solve the optimization problem. We obtain superior or comparable accuracies to other data selection approaches, such as random, greedy k-means-based, and least square-based approaches. Notably, our algorithm was not degraded in small data selection, unlike other state-of-the-art algorithms.

Fault Detection in Nuclear Power Plants using Deep Leaning based Image Classification with Imaged Time-series Data

Fault detection is critical to ensure the safely routine operations in nuclear power plants (NPPs), requiring very high accuracy and efficiency. Meanwhile, the rapid development of modern information technologies have profoundly changed and promoted various sectors including nuclear industry. Inspired by the great progress and promising performance of deep learning based image classification recent years, a two-stage fault detection methodology in NPPs has been proposed in this paper. First the time-series data describing the operating status of NPPs have been transformed into two-dimensional images by four methods, preserving the time-series information in images and converting the fault detection problem into a supervised image classification task. Then four specific image classifying models based on three primary deep learning architectures have been separately experimented on the imaged time-series data, achieving excellent accuracies. Further the performances of different combinations of transforming means and classifying models have been compared and discussed with extensive experiments and detailed analysis of throughput for four transforming methods. This methodology proposed has obtained remarkable results by reshaping data format and structure, making image classifying models applicable, which not only efficiently detect and warn possible faults in NPPs but also enhances the capability for safety management in nuclear power systems.

A Survey on Hyperspectral Image Classification and Object Detection Techniques

Machine Learning is vast field which finds its application in almost every field. The image classification is one of the important application of Supervised Machine learning algorithms. Image classification is basically concerned with identifying the objects in the images. The complexity of this task is dependent on the image features and type of images. For the research work here, the hyperspectral images are considered for deep learning based image classification. The object detection in the Hyperspectral images have applications in various areas including defense, precision agriculture, atmospheric analysis, environmental analysis, anomaly detection, fraud detection , etc. The work presented here is divided into broad survey of image classification methods using machine learning and deep learning methods. Continuing with this work, the further work presents object detection methods in ML and DL. The later work presents the deep review of the research articles over Hyperspectral image classification using Machine Learning and Deep Learning Algorithms. A lot of challenges are present to solve the object detection problems in Hyperspectral images. The later section of this work describes the object detection based on Hyperspectral images survey in detail highlighting the major developments.

Comparison of Deep Learning Models for Corn Disease Region Location, Identification of Disease Type, and Severity Estimation Using Images Acquired From UAS-Mounted and Handheld Sensors

Highlights An approach using deep learning was proposed for identifying diseased regions in UAS imagery of corn fields with 97.23% testing accuracy using the VGG16 model. Disease types were identified within the diseased regions with a testing accuracy of 98.85% using the VGG16 model. On the diseased leaves, severity was estimated with a testing accuracy of 94.20% using the VGG16 model. Deep Learning models have the potential to bring efficiency and accuracy to field scouting. Abstract. Accurately locating diseased regions, identifying disease types, and estimating disease severity in corn fields are all connected steps for developing an effective disease management system. Traditional disease management that relied on a manual scouting approach was inefficient. Therefore, the research community is working on developing advanced disease management systems using deep learning. However, most of the past studies used public datasets consisting of images with uniform backgrounds acquired under lab conditions to train deep learning models, thus, limiting their use under field conditions. In addition, limited studies have been conducted for in-field corn disease analysis using Unmanned Aerial System (UAS) imagery. Therefore, UAS and handheld imagery sensors were used in this study to acquire corn disease images from fields located at Purdue University’s Agronomy Center for Research and Education (ACRE) in the summer of 2020. A total of 55 UAS flights were conducted over three different corn fields from June 20 through September 29, resulting in a collection of approximately 59,000 images. A novel three-stage approach was proposed by independently training a total of nine image classification models using three neural network architectures, namely: VGG16, ResNet50, and InceptionV3, for locating diseased regions, identifying disease types, and estimating disease severity under field conditions. Diseased regions were first identified accurately in UAS-acquired corn field imagery by a sliding window and deep learning-based image classification, with testing accuracies of up to 97.23%. Diseased region identification was followed by accurately identifying three common corn diseases, namely Northern Leaf Blight (NLB), Gray Leaf Spot (GLS), and Northern Leaf Spot (NLS), within the diseased regions with testing accuracies of up to 98.85%. Finally, the severity of the NLS disease on leaves was estimated with a testing accuracy of up to 94.20%. The VGG16 model achieved the highest testing accuracies for identifying diseased regions in corn fields, identifying corn disease types, and estimating NLS's severity. This study presents promising results for three main elements of a disease management system and could advance traditional scouting by integrating deep learning with UAS imagery. Keywords: Corn Diseases, Datasets, Deep Learning, Disease Identification, Disease Region Location, Image Classification, Severity Estimation, UAS Imagery.

Less is More: Lighter and Faster Deep Neural Architecture for Tomato Leaf Disease Classification

To ensure global food security and the overall profit of stakeholders, the importance of correctly detecting and classifying plant diseases is paramount. In this connection, the emergence of deep learning-based image classification has introduced a substantial number of solutions. However, the applicability of these solutions in low-end devices requires fast, accurate, and computationally inexpensive systems. This work proposes a lightweight transfer learning-based approach for detecting diseases from tomato leaves. It utilizes an effective preprocessing method to enhance the leaf images with illumination correction for improved classification. Our system extracts features using a combined model consisting of a pretrained MobileNetV2 architecture and a classifier network for effective prediction. Traditional augmentation approaches are replaced by runtime augmentation to avoid data leakage and address the class imbalance issue. Evaluation on tomato leaf images from the PlantVillage dataset shows that the proposed architecture achieves 99.30% accuracy with a model size of 9.60MB and 4.87M floating-point operations, making it a suitable choice for low-end devices. Our codes and models will be made available upon publication.

Haptic and Biometric Sensor Technologies, Deep Learning-based Image Classification Algorithms, and Movement and Behavior Tracking Tools in the Metaverse Economy

Haptic and Biometric Sensor Technologies, Deep Learning-based Image Classification Algorithms, and Movement and Behavior Tracking Tools in the Metaverse Economy

Deep learning-based image classification for online multi-coal and multi-class sorting

Deep learning is an effective way to improve the classification accuracy of coal images for the machine vision-based coal sorting. However, the related research on deep learning-based mineral image classification has not systematically considered the models for multi-coal and multi-class sorting. Additionally, the universal CNNs model for multi-coal image classification has not been proposed. Given the above problems, combined with deep learning and transfer learning and based on VGG Net, Inception Net, and Res Net, this study builds four CNNs models with different depth and structure for multi-coal and multi-class image classification. Finally, we take anthracite, gas coal, coking coal as the research objects and propose a universal CNNs model suitable for multi-coal and multi-class sorting. Moreover, with the Channel Visualization map, Heatmap, Gard-CAM map, and Guided Backpropagation map, the operational processes of CNNs model in coal image recognition and classification are revealed, and the features that affect the classification weights are analyzed.

A Comparative Study of Deep Learning based Image Classification Models for Material Classification

A Comparative Study of Deep Learning based Image Classification Models for Material Classification

Deep learning-based classification models for beehive monitoring

Honey bees are not only the fundamental producers of honey but also the leading pollinators in nature. While honey bees play such a vital role in the ecosystem, they also face a variety of threats, including parasites, ants, hive beetles, and hive robberies, some of which could even lead to the collapse of colonies. Therefore, early and accurate detection of abnormalities at a beehive is crucial to take appropriate countermeasures promptly. In this paper, deep learning-based image classification models are proposed for beehive monitoring. The proposed models particularly classify honey bee images captured at beehives and recognize different conditions, such as healthy bees, pollen-bearing bees, and certain abnormalities, such as Varroa parasites, ant problems, hive robberies, and small hive beetles. The models utilize transfer learning with pre-trained deep neural networks (DNNs) and also a support vector machine classifier with deep features, shallow features, and both deep and shallow features extracted from these DNNs. Three benchmark datasets, consisting of a total of 19,393 honey bee images for different conditions, were used to train and evaluate the models. The results of the extensive experimental work revealed that the proposed models can recognize different conditions as well as abnormalities with an accuracy of up to 99.07% and stand out as good candidates for smart beekeeping and beehive monitoring.

Deep Learning-Based Image Classification in Differentiating Tufted Astrocytes, Astrocytic Plaques, and Neuritic Plaques.

This study aimed to develop a deep learning-based image classification model that can differentiate tufted astrocytes (TA), astrocytic plaques (AP), and neuritic plaques (NP) based on images of tissue sections stained with phospho-tau immunohistochemistry. Phospho-tau-immunostained slides from the motor cortex were scanned at 20× magnification. An automated deep learning platform, Google AutoML, was used to create a model for distinguishing TA in progressive supranuclear palsy (PSP) from AP in corticobasal degeneration (CBD) and NP in Alzheimer disease (AD). A total of 1500 images of representative tau lesions were captured from 35 PSP, 27 CBD, and 33 AD patients. Of those, 1332 images were used for training, and 168 images for cross-validation. We tested the model using 100 additional test images taken from 20 patients of each disease. In cross-validation, precision and recall for each individual lesion type were 100% and 98.0% for TA, 98.5% and 98.5% for AP, and 98.0% and 100% for NP, respectively. In a test set, all images of TA and NP were correctly predicted. Only eleven images of AP were predicted to be TA or NP. Our data indicate the potential usefulness of deep learning-based image classification methods to assist in differential diagnosis of tauopathies.

Performance of deep learning models for classifying and detecting common weeds in corn and soybean production systems

Knowing precise location and having accurate information about weed species is a prerequisite for developing an effective site-specific weed management (SSWM) system. Due to the effectiveness of deep learning techniques for vision-based tasks such as image classification and object detection, its use for discriminating between weeds and crops is gaining acceptance among the agricultural research community. However, limited studies have used deep learning for identifying multiple weeds in a single image and most of the studies have not compared the effectiveness of deep learning based image classification and object detection by using a common, annotated imagery dataset of early season weeds under field conditions. This study addresses the research gap by evaluating comparative performance of three different pre-trained image classification models for classifying weed species and also assesses the performance of an object detection model for locating and identifying weed species. The image classification models were trained on two commonly used deep learning frameworks i.e., Keras and PyTorch, to assess any performance differential due to the choice of framework. An annotated dataset comprising of RGB images of four, early season weeds, found in corn and soybean production system in the Midwest US, namely, cocklebur (Xanthium strumarium), foxtail (Setaria viridis), redroot pigweed (Amaranthus retroflexus), and giant ragweed (Ambrosia trifida) was used in this study. VGG16, ResNet50, and InceptionV3 pre-trained models were used for image classification. The object detection model, based on the You Only Look Once (YOLOv3) library, was trained to locate and identify different weed species within an image. The performance of image classification models was assessed using testing accuracy and F1-score metrics. Average precision (AP) and mean average precision (mAP) were used to assess the performance of the object detection model. The best performing image classification model was VGG16 with an accuracy of 98.90% and an F1-score of 99%. Faster training times and higher accuracies were observed with PyTorch. The detection model helped locate and identify multiple weeds within an image with AP scores of 43.28%, 26.30%, 89.89%, and 57.80% for cocklebur, foxtail, redroot pigweed, and giant ragweed respectively and an overall mAP score of 54.3%. The results suggest that under field conditions, use of pre-trained models for image classification and YOLOv3 for object detection are promising for identifying single and multiple weeds, respectively, given that sufficient data is available. Additionally, unlike image classification, the localization capabilities of object detection are desirable for developing a system for SSWM.

Deep learning-based image classification of gas coal

Machine vision-based sorting technology is a potential mineral separation method with the merits of high cost performance and security. However, the classification accuracy of common mineral pattern recognition methods is not satisfactory. Therefore, this paper proposes a deep learning-based classification method for the multi-class prediction of gas coal in the machine vision sorting system. The single coal images are obtained by the image segmentation algorithm and taken as the inputs of deep learning model based on the Inception_v3 network with the aid of transfer learning. The cross-entropy loss function and test accuracy are applied as the evaluation indexes of the classification effect. The result derived using the proposed model indicates that the classification accuracies of four-products, three-products and two-products are up to 82.3%, 87.8% and 99.4%, respectively. Additionally, a grad-cam convolution thermal map is used to visualise the differences in the deep learning classification process. Finally, the proposed deep learning model proves its superiority over the traditional models such as Back Propagation Neural Network (BP), Random Forest (RF) and Support Vector Machine (SVM).

Deep learning-based image classification of gas coal

Machine vision-based sorting technology is a potential mineral separation method with the merits of high cost performance and security. However, the classification accuracy of common mineral pattern recognition methods is not satisfactory. Therefore, this paper proposes a deep learning-based classification method for the multi-class prediction of gas coal in the machine vision sorting system. The single coal images are obtained by the image segmentation algorithm and taken as the inputs of deep learning model based on the Inception_v3 network with the aid of transfer learning. The cross-entropy loss function and test accuracy are applied as the evaluation indexes of the classification effect. The result derived using the proposed model indicates that the classification accuracies of four-products, three-products and two-products are up to 82.3%, 87.8% and 99.4%, respectively. Additionally, a grad-cam convolution thermal map is used to visualise the differences in the deep learning classification process. Finally, the proposed deep learning model proves its superiority over the traditional models such as Back Propagation Neural Network (BP), Random Forest (RF) and Support Vector Machine (SVM).