CNN-based Image Classification Research Articles

From simple labels to semantic image segmentation: leveraging citizen science plant photographs for tree species mapping in drone imagery

Abstract. Knowledge of plant species distributions is essential for various application fields, such as nature conservation, agriculture, and forestry. Remote sensing data, especially high-resolution orthoimages from unoccupied aerial vehicles (UAVs), paired with novel pattern-recognition methods, such as convolutional neural networks (CNNs), enable accurate mapping (segmentation) of plant species. Training transferable pattern-recognition models for species segmentation across diverse landscapes and data characteristics typically requires extensive training data. Training data are usually derived from labor-intensive field surveys or visual interpretation of remote sensing images. Alternatively, pattern-recognition models could be trained more efficiently with plant photos and labels from citizen science platforms, which include millions of crowd-sourced smartphone photos and the corresponding species labels. However, these pairs of citizen-science-based photographs and simple species labels (one label for the entire image) cannot be used directly for training state-of-the-art segmentation models used for UAV image analysis, which require per-pixel labels for training (also called masks). Here, we overcome the limitation of simple labels of citizen science plant observations with a two-step approach. In the first step, we train CNN-based image classification models using the simple labels and apply them in a moving-window approach over UAV orthoimagery to create segmentation masks. In the second phase, these segmentation masks are used to train state-of-the-art CNN-based image segmentation models with an encoder–decoder structure. We tested the approach on UAV orthoimages acquired in summer and autumn at a test site comprising 10 temperate deciduous tree species in varying mixtures. Several tree species could be mapped with surprising accuracy (mean F1 score =0.47). In homogenous species assemblages, the accuracy increased considerably (mean F1 score =0.55). The results indicate that several tree species can be mapped without generating new training data and by only using preexisting knowledge from citizen science. Moreover, our analysis revealed that the variability in citizen science photographs, with respect to acquisition data and context, facilitates the generation of models that are transferable through the vegetation season. Thus, citizen science data may greatly advance our capacity to monitor hundreds of plant species and, thus, Earth's biodiversity across space and time.

A cyber defense system against phishing attacks with deep learning game theory and LSTM-CNN with African vulture optimization algorithm (AVOA)

Phishing attacks pose a significant threat to online security, utilizing fake websites to steal sensitive user information. Deep learning techniques, particularly convolutional neural networks (CNNs), have emerged as promising tools for detecting phishing attacks. However, traditional CNN-based image classification methods face limitations in effectively identifying fake pages. To address this challenge, we propose an image-based coding approach for detecting phishing attacks using a CNN-LSTM hybrid model. This approach combines SMOTE, an enhanced GAN based on the Autoencoder network, and swarm intelligence algorithms to balance the dataset, select informative features, and generate grayscale images. Experiments on three benchmark datasets demonstrate that the proposed method achieves superior accuracy, precision, and sensitivity compared to other techniques, effectively identifying phishing attacks and enhancing online security.

Enhancing Malware Detection Efficiency through CNN-Based Image Classification in a User-Friendly Web Portal

The proposed CNN-based malware detection web portal classifies images on a unique, self-made dataset to identify malware files as input. There are many different types of malware out there, but no method can detect them all. An anti-virus programme could be created that enforces malware image classification for the aforementioned scenarios as opposed to the traditional signature-based methods used by the majority of anti-virus programmes currently available in the market, which are time-consuming and ineffective because they rely just on signatures of previous malware attacks and need to be updated regularly. The fact that some malware is encrypted and requires a significant amount of computing power to decrypt makes this strategy ineffective for identifying all malware that accesses the network. As a result, fresh malware cannot be detected because this method simulates the behavior of malware samples and matches it to new programs. An online portal with a candid user interface will be used to deploy the proposed Deep-learning based malware detection algorithm. The file to be tested or classified will be uploaded onto the website

Performance Comparison of Vision Transformer- and CNN-Based Image Classification Using Cross Entropy: A Preliminary Application to Lung Cancer Discrimination from CT Images

Performance Comparison of Vision Transformer- and CNN-Based Image Classification Using Cross Entropy: A Preliminary Application to Lung Cancer Discrimination from CT Images

A Degradation Type Adaptive and Deep CNN-Based Image Classification Model for Degraded Images

A Degradation Type Adaptive and Deep CNN-Based Image Classification Model for Degraded Images

The Application of CNN-Based Image Classification To Wildfire Early Detection

Wildfires have become increasingly common and devastating in recent years, yet many regions still rely on traditional methods of using human spotters atop lookout towers to detect fires. These methods are slow and inefficient, and as fires become more frequent demand for new detection methods has grown along with it. This study aims to utilize artificial intelligence along with stationary camera systems to perform early detection of wildfires. An open-source dataset of 1900 wildfire pictures was obtained and processed, and a convolutional neural network model was trained on the dataset. The final model achieved an accuracy of 95.59% on the validation dataset after being trained for 37 epochs.

An Efficient Comparative Analysis of CNN-based Image Classification in the Jupyter Tool Using Multi-Stage Techniques

The main process of this image classification with a convolution neural network using deep learning model was performed in the programming language Python code in the Jupyter tool, mainly using the data set of IRS P-6 LISS IV from an Indian remote sensing satellite with a high resolution multi-spectral camera with around 5.8m from an 817 km altitude Delhi image. To classify the areas within the cropped image required to apply enhancement techniques, the image size was 1000 mb. To view this image file required high-end software for opening. For that, initially, ERDAS imaging software viewer was used for cropping into correct resolution pixels. based on that cropped image used for image classification with preprocessing for applying filters for enhancement. And with the convolution neural network model, required to train the sample images of the same pixels, was collected from the group of objects that were cropped. Then we needed to use image sample areas to train the model with learning rate and epoch rate to improve object detection accuracy using the Jupyter notebook tool with tensorflow and machine learning model produce the accuracy rate of 90.78%.

Progress in the Application of CNN-Based Image Classification and Recognition in Whole Crop Growth Cycles

The categorization and identification of agricultural imagery constitute the fundamental requisites of contemporary farming practices. Among the various methods employed for image classification and recognition, the convolutional neural network (CNN) stands out as the most extensively utilized and swiftly advancing machine learning technique. Its immense potential for advancing precision agriculture cannot be understated. By comprehensively reviewing the progress made in CNN applications throughout the entire crop growth cycle, this study aims to provide an updated account of these endeavors spanning the years 2020 to 2023. During the seed stage, classification networks are employed to effectively categorize and screen seeds. In the vegetative stage, image classification and recognition play a prominent role, with a diverse range of CNN models being applied, each with its own specific focus. In the reproductive stage, CNN’s application primarily centers around target detection for mechanized harvesting purposes. As for the post-harvest stage, CNN assumes a pivotal role in the screening and grading of harvested products. Ultimately, through a comprehensive analysis of the prevailing research landscape, this study presents the characteristics and trends of current investigations, while outlining the future developmental trajectory of CNN in crop identification and classification.

Is My Pruned Model Trustworthy? PE-Score: A New CAM-Based Evaluation Metric

One of the strategies adopted to compress CNN models for image classification tasks is pruning, where some elements, channels or filters of the network are discarded. Typically, pruning methods present results in terms of model performance before and after pruning (assessed by accuracy or a related parameter such as the F1-score), assuming that if the difference is less than a certain value (e.g., 2%), the pruned model is trustworthy. However, state-of-the-art models are not concerned with measuring the actual impact of pruning on the network by evaluating the pixels used by the model to make the decision, or the confidence of the class itself. Consequently, this paper presents a new metric, called the Pruning Efficiency score (PE-score), which allows us to identify whether a pruned model preserves the behavior (i.e., the extracted patterns) of the unpruned model, through visualization and interpretation with CAM-based methods. With the proposed metric, it will be possible to better compare pruning methods for CNN-based image classification models, as well as to verify whether the pruned model is efficient by focusing on the same patterns (pixels) as those of the original model, even if it has reduced the number of parameters and FLOPs.

Image Classification Using Convolutional Neural Networks

Abstract: Convolutional Neural Networks (CNNs) have emerged as powerful tools for image classification tasks due to their ability to automatically learn hierarchical representations from raw pixel data. This paper provides a comprehensive review of CNN-based image classification methods, covering various aspects such as network architectures, training techniques, and evaluation metrics. Additionally, we discuss recent advancements, challenges, and future directions in CNN-based image classification research. The aim of this paper is to provide researchers and practitioners with a thorough understanding of CNNbased image classification and to guide future research in this field.

Abstract 2176: A cell mass-based platform for characterizing immune cell function and response to checkpoint blockade in primary blood and malignant fluid samples

Abstract Existing predictive biomarkers for immune checkpoint blockade (ICB) therapies that rely on characterization of tumor genomics or immunohistological assessment of the tumor microenvironment have demonstrated mixed success, with no single biomarker proving to be universally predictive of ICB treatment outcome. To complement these existing approaches, we present a workflow that relies on high-resolution single-cell mass measurements curated with CNN-based image classification to characterize T cell activation and functional response to treatment with ICB therapies. As a highly integrative measure of cellular phenotype, cell mass can be used as a label-free biomarker for characterizing immune cell function. For instance, upon activation with anti-CD3/CD28, T cells isolated from human PBMC showed a rapid and significant increase in cell mass. However, in T cells that had been repeatedly stimulated in vitro to generate a PD-1+ exhausted-like phenotype, this mass response to activation was blunted. We hypothesized that this variability in activation-induced mass response was the result of an altered balance of stimulatory and inhibitory “signal two” cues seen by the T cells. Consistent with this hypothesis, purified T cells stimulated with anti-CD3 alone (no co-stimulation) demonstrated a significantly smaller increase in cell mass when compared with those stimulated with both anti-CD3 and anti-CD28. Furthermore, the T cell response measured when unpurified PBMC were activated with anti-CD3 demonstrated an intermediate mass increase, suggesting contributions of both stimulatory and inhibitory signaling from other cells in the milieu. To explore how ICB treatment disrupts this co-stimulatory/inhibitory signaling balance, we measured T cell mass responses for samples treated with anti-CD3 alone or in combination with pembrolizumab, atezolizumab, nivolumab, or dostarlimab. These samples included PBMC isolated from blood and tumor/immune cell milieus isolated from malignant fluids (pleural effusions or abdominal ascites) collected from patients with advanced cancer. We found that T cells isolated from samples treated with both anti-CD3 and an ICB drug often showed a significantly greater mass increase than the same samples treated with anti-CD3 alone. This mass response showed significant heterogeneity across patients, with many samples showing no significant treatment effect. Despite all four of these therapies targeting ostensibly the same PD-1/PD-L1 signaling axis, we also found significant response heterogeneity to these different treatments within individual samples.These results suggest that mass response measurements capture biological heterogeneity associated with ICB response and may serve as a valuable complementary readout to existing biomarkers. Citation Format: Robert Kimmerling, Mark Stevens, Selim Olcum, Anthony Minnah, Madeleine Vacha, Rachel LaBella, Katie Katsis, Audrey Guo, Juanita Fujii, Zayna Shaheen, Srividya Sundaresan, Anobel Tamrazi, Cliff Reid. A cell mass-based platform for characterizing immune cell function and response to checkpoint blockade in primary blood and malignant fluid samples [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 2176.

Application of CNN based image classification technique for oil spill detection

Application of CNN based image classification technique for oil spill detection

CNN Based Automatic Speed and Steering Control System Using Matlab

The current technology revolution in automobile industry is the autonomous vehicle wherein the vehicle without any intervention from the human driver responds to all kind of external conditions and performs all necessary functions that a human driver would perform while driving. Traffic sign identification is a crucial task in an autonomous driving system and even a minute misinterpretation could turn into a fatal one. This paper proposes a CNN based image classification which specifically uses pre-trained feature generated neural networks present in GoogleNet. GoogleNet is a CNN which recognizes the traffic signboards present by the roadside and sends the control signal to a microcontroller (Arduino Board) which controls the speed and the steering of the vehicle. The implemented CNN based automatic speed and steering control has been verified with a greater accuracy. The work presented in this paper leads towards the development of smart and sustainable cities under the Smart Cities Mission of India.

A Collaborative Region Detection and Grading Framework for Forest Fire Smoke Using Weakly Supervised Fine Segmentation and Lightweight Faster-RCNN

Forest fires are serious disasters that affect countries all over the world. With the progress of image processing, numerous image-based surveillance systems for fires have been installed in forests. The rapid and accurate detection and grading of fire smoke can provide useful information, which helps humans to quickly control and reduce forest losses. Currently, convolutional neural networks (CNN) have yielded excellent performance in image recognition. Previous studies mostly paid attention to CNN-based image classification for fire detection. However, the research of CNN-based region detection and grading of fire is extremely scarce due to a challenging task which locates and segments fire regions using image-level annotations instead of inaccessible pixel-level labels. This paper presents a novel collaborative region detection and grading framework for fire smoke using a weakly supervised fine segmentation and a lightweight Faster R-CNN. The multi-task framework can simultaneously implement the early-stage alarm, region detection, classification, and grading of fire smoke. To provide an accurate segmentation on image-level, we propose the weakly supervised fine segmentation method, which consists of a segmentation network and a decision network. We aggregate image-level information, instead of expensive pixel-level labels, from all training images into the segmentation network, which simultaneously locates and segments fire smoke regions. To train the segmentation network using only image-level annotations, we propose a two-stage weakly supervised learning strategy, in which a novel weakly supervised loss is proposed to roughly detect the region of fire smoke, and a new region-refining segmentation algorithm is further used to accurately identify this region. The decision network incorporating a residual spatial attention module is utilized to predict the category of forest fire smoke. To reduce the complexity of the Faster R-CNN, we first introduced a knowledge distillation technique to compress the structure of this model. To grade forest fire smoke, we used a 3-input/1-output fuzzy system to evaluate the severity level. We evaluated the proposed approach using a developed fire smoke dataset, which included five different scenes varying by the fire smoke level. The proposed method exhibited competitive performance compared to state-of-the-art methods.

Spatial-Channel Attention-Based Class Activation Mapping for Interpreting CNN-Based Image Classification Models

Convolutional neural network (CNN) has been applied widely in various fields. However, it is always hindered by the unexplainable characteristics. Users cannot know why a CNN-based model produces certain recognition results, which is a vulnerability of CNN from the security perspective. To alleviate this problem, in this study, the three existing feature visualization methods of CNN are analyzed in detail firstly, and a unified visualization framework for interpreting the recognition results of CNN is presented. Here, class activation weight (CAW) is considered as the most important factor in the framework. Then, the different types of CAWs are further analyzed, and it is concluded that a linear correlation exists between them. Finally, on this basis, a spatial-channel attention-based class activation mapping (SCA-CAM) method is proposed. This method uses different types of CAWs as attention weights and combines spatial and channel attentions to generate class activation maps, which is capable of using richer features for interpreting the results of CNN. Experiments on four different networks are conducted. The results verify the linear correlation between different CAWs. In addition, compared with the existing methods, the proposed method SCA-CAM can effectively improve the visualization effect of the class activation map with higher flexibility on network structure.

Convolutional Neural Network Based Advertisement Classification Models for Online English Newspapers

Image processing for knowledge management and effective information extraction is the key element for steering towards society 5.0. There has been a substantial research and progress in the area of image recognition and classification in the recent years but at the same time, there is a lack of significant work in the field of advertisement image classification from online English newspapers. This research paper analyses and compares various popular image classification techniques to find out the most suitable technique for advertisement image classification problem. Automatic feature extraction without any prior knowledge of features makes Convolutional Neural Networks (CNN) the most suitable technique for advertisement image classification. This paper further explores and implements three different CNN-based image classification models that can classify advertisement images from online English newspapers into four pre-defined categories including Admission-notices, Job-advertisements, Sales and Promotional advertisements and Tenders. These models are trained and tested on an advertisement image dataset collected from four different online English newspapers over a time frame of 15 months. Fine-tuned ResNet50 Model using ‘Transfer-learning’ is found to be the most suitable model for this advertisement image classification task with results exhibiting around 74% accuracy. This CNN-model based automated classification of advertisement images will help newspaper readers in performing exhaustive advertisement search in a category of their own interest, saving the time and efforts of sequential manual search across a range of multiple newspapers. Also, the proposed research will help in performing advertisement analysis and studies.

Deep CNN-based hyperspectral image classification using discriminative multiple spatial-spectral feature fusion

ABSTRACT Convolutional Neural Networks (CNNs) are widely used in various fields, and have shown good performance in hyperspectral image (HSI) classification. Recently, utilizing deep networks to learn spatial-spectral features has become of great interest. However, excessively increasing the depth of network may result in overfitting. Moreover, in HSI classification, the existing network models ignore the strong complementary yet correlated spatial-spectral information among different hierarchical layers. In order to address these two problems, a novel CNN-based method for HSI classification is proposed. Firstly, it considers fusing the outputs of recurrent two layers in each large convolutional block and thereby using the fusion result as the input of next layer, which facilitates the extraction of discriminative features. Next, the spectral-spatial features are extracted by cascading spectral features to four-scale spatial features from shallow to deep layers. Finally, a 11 convolution layer is used to interact and integrate information across channels. Without increasing the number of training samples and the size of pixel patches at the training stage, the proposed approach achieves the state-of-the-art results in the experiment on three well-known hyperspectral images.

Psc2code

Programming screencasts have become a pervasive resource on the Internet, which help developers learn new programming technologies or skills. The source code in programming screencasts is an important and valuable information for developers. But the streaming nature of programming screencasts (i.e., a sequence of screen-captured images) limits the ways that developers can interact with the source code in the screencasts. Many studies use the Optical Character Recognition (OCR) technique to convert screen images (also referred to as video frames) into textual content, which can then be indexed and searched easily. However, noisy screen images significantly affect the quality of source code extracted by OCR, for example, no-code frames (e.g., PowerPoint slides, web pages of API specification), non-code regions (e.g., Package Explorer view, Console view), and noisy code regions with code in completion suggestion popups. Furthermore, due to the code characteristics (e.g., long compound identifiers like ItemListener), even professional OCR tools cannot extract source code without errors from screen images. The noisy OCRed source code will negatively affect the downstream applications, such as the effective search and navigation of the source code content in programming screencasts. In this article, we propose an approach named psc2code to denoise the process of extracting source code from programming screencasts. First, psc2code leverages the Convolutional Neural Network (CNN) based image classification to remove non-code and noisy-code frames. Then, psc2code performs edge detection and clustering-based image segmentation to detect sub-windows in a code frame, and based on the detected sub-windows, it identifies and crops the screen region that is most likely to be a code editor. Finally, psc2code calls the API of a professional OCR tool to extract source code from the cropped code regions and leverages the OCRed cross-frame information in the programming screencast and the statistical language model of a large corpus of source code to correct errors in the OCRed source code. We conduct an experiment on 1,142 programming screencasts from YouTube. We find that our CNN-based image classification technique can effectively remove the non-code and noisy-code frames, which achieves an F1-score of 0.95 on the valid code frames. We also find that psc2code can significantly improve the quality of the OCRed source code by truly correcting about half of incorrectly OCRed words. Based on the source code denoised by psc2code , we implement two applications: (1) a programming screencast search engine; (2) an interaction-enhanced programming screencast watching tool. Based on the source code extracted from the 1,142 collected programming screencasts, our experiments show that our programming screencast search engine achieves the precision@5, 10, and 20 of 0.93, 0.81, and 0.63, respectively. We also conduct a user study of our interaction-enhanced programming screencast watching tool with 10 participants. This user study shows that our interaction-enhanced watching tool can help participants learn the knowledge in the programming video more efficiently and effectively.

A smart surface inspection system using faster R-CNN in cloud-edge computing environment

Automated surface inspection has become a hot topic with the rapid development of machine vision technologies. Traditional machine vision methods need experts to carefully craft image features for defect detection. This limits their applications to wider areas. The emerging convolutional neural networks (CNN) can automatically extract features and yield good results in many cases. However, the CNN-based image classification methods are more suitable for flat surface texture inspection. It is difficult to accurately locate small defects in geometrically complex products. Furthermore, the computational power required in CNN algorithms is usually high and it is not efficient to be implemented on embedded hardware. To solve these problems, a smart surface inspection system is proposed using faster R-CNN algorithm in the cloud-edge computing environment. The faster R-CNN as a CNN-based object detection method can efficiently identify defects in complex product images and the cloud-edge computing framework can provide fast computation speed and evolving algorithm models. A real industrial case study is presented to illustrate the effectiveness of the proposed method. The results show that the proposed method can provide high detection accuracy within a short time.

Enhanced Visual Attention-Guided Deep Neural Networks for Image Classification

A fully connected layer is essential for a CNN, i.e., convolutional neural network, which has been shown to be successful in classifying images in several related applications. A CNN begins with convolution and pooling operations for decomposing an input image into features. The result of this process is then fed into a fully connected neural network, driving the final classification decision for the input image. However, it has been found that the learned feature maps in a CNN are sometimes not good enough for being fed into the fully connected layers to get good classification results. In this article, a visual attention learning module is proposed to enhance the classification capability of the fully connected layers in a CNN. By learning better feature maps to emphasize salient regions and weaken meaningless regions, better classification performance can be obtained with integrating the proposed module into the fully connected layers. The proposed visual attention learning module can be imposed on any existed CNN-based image classification models to achieve incremental improvements with negligible overhead. Based on our experiments, the proposed method achieves the top-1 accuracies of 95.32%, 92.73%, and 66.50% on average, respectively, obtained on our collected Underwater Fish dataset, the public Animals-10 dataset, and the public Stanford Cars dataset.