Scene Classification Accuracy Research Articles

Optimizing Fire Scene Analysis: Hybrid Convolutional Neural Network Model Leveraging Multiscale Feature and Attention Mechanisms

The rapid and accurate detection of fire scenes in various environments is crucial for effective disaster management and mitigation. Fire scene classification is a critical aspect of modern fire detection systems that directly affects public safety and property preservation. This research introduced a novel hybrid deep learning model designed to enhance the accuracy and efficiency of fire scene classification across diverse environments. The proposed model integrates advanced convolutional neural networks with multiscale feature extraction, attention mechanisms, and ensemble learning to achieve superior performance in real-time fire detection. By leveraging the strengths of pre-trained networks such as ResNet50, VGG16, and EfficientNet-B3, the model captures detailed features at multiple scales, ensuring robust detection capabilities. Including spatial and channel attention mechanisms further refines the focus on critical areas within the input images, reducing false positives and improving detection precision. Extensive experiments on a comprehensive dataset encompassing wildfires, building fires, vehicle fires, and non-fire scenes demonstrate that the proposed framework outperforms existing cutting-edge techniques. The model also exhibited reduced computational complexity and enhanced inference speed, making it suitable for deployment in real-time applications on various hardware platforms. This study sets a new benchmark for fire detection and offers a powerful tool for early warning systems and emergency response initiatives.

A Scene Classification Model Based on Global-Local Features and Attention in Lie Group Space

The efficient fusion of global and local multi-scale features is quite important for remote sensing scene classification (RSSC). The scenes in high-resolution remote sensing images (HRRSI) contain many complex backgrounds, intra-class diversity, and inter-class similarities. Many studies have shown that global features and local features are helpful for RSSC. The receptive field of a traditional convolution kernel is small and fixed, and it is difficult to capture global features in the scene. The self-attention mechanism proposed in transformer effectively alleviates the above shortcomings. However, such models lack local inductive bias, and the calculation is complicated due to the large number of parameters. To address these problems, in this study, we propose a classification model of global-local features and attention based on Lie Group space. The model is mainly composed of three independent branches, which can effectively extract multi-scale features of the scene and fuse the above features through a fusion module. Channel attention and spatial attention are designed in the fusion module, which can effectively enhance the crucial features in the crucial regions, to improve the accuracy of scene classification. The advantage of our model is that it extracts richer features, and the global-local features of the scene can be effectively extracted at different scales. Our proposed model has been verified on publicly available and challenging datasets, taking the AID as an example, the classification accuracy reached 97.31%, and the number of parameters is 12.216 M. Compared with other state-of-the-art models, it has certain advantages in terms of classification accuracy and number of parameters.

Competent scene classification using feature fusion of pre-trained convolutional neural networks

In view of the fact that the development of convolutional neural networks (CNN) and other deep learning techniques, scientists have become more interested in the scene categorization of remotely acquired images as well as other algorithms and datasets. The spatial geometric detail information may be lost as the convolution layer thickness increases, which would have a significant impact on the classification accuracy. Fusion-based techniques, which are regarded to be a viable way to express scene features, have recently attracted a lot of interest as a solution to this issue. Here, we suggested a convolutional feature fusion network that makes use of canonical correlation, which is the linear correlation between two feature maps. Then, to improve scene classification accuracy, the deep features extracted from various pre-trained convolutional neural networks are efficiently fused. We thoroughly evaluated three different fused CNN designs to achieve the best results. Finally, we used the support vector machine for categorization (SVM). In the analysis, two real-world datasets UC Merced and SIRI-WHU were employed, and the competitiveness of the investigated technique was evaluated. The improved categorization accuracy demonstrates that the fusion technique under consideration has produced affirmative results when compared to individual networks.

Improving remote sensing scene classification using quality-based data augmentation

ABSTRACT This study presents an improved data augmentation technique for the classification of remote sensing (RS) scenes. The method is called Quality-based Sample Selection (QSS) data augmentation technique. It is based on generating a large number of samples using geometric transformations and then selecting the best ones based on a quality criterion. Sample images are generated online, i.e. during training of the convolutional neural network (CNN) model. For each training sample image, a few images are generated randomly using a set of geometric transformations. Then, the generated images are passed to the CNN model being trained, and the predicted probabilities are used to evaluate the quality of the images. The motivation behind this research is that we aim to augment the training set with new images that will benefit the learning of the CNN model. Therefore, an objective method to evaluate the quality of the new images is the CNN model itself. The image with the best criteria score is used to augment the training set. Several quality criteria based on the prediction probabilities of the CNN model are considered including entropy, breaking-ties and our own proposed criteria called True Label Probability (TLP). QSS is tested on five common RS scene classification datasets: UCMerced, Optimal31, RSSCN7, AID and NWPU-RS45. It has outperformed all previous methods except for one case involving NWPU-RS45 dataset. For the 10%–90% train–test split, QSS achieved 94.51, 81.72, 89.89, 94.09 and 93.98, respectively, for the mentioned datasets. While for the 20%–80% train–test split, it achieved 97.09, 88.37, 94.01, 95.71 and 94.71, respectively, for the listed datasets. Thus, this work presents a novel data augmentation method that uses data with higher quality instead of randomly selected. The paper proves experimentally that the proposed method improves the classification accuracy of RS scene classification.

A multimodal hyper-fusion transformer for remote sensing image classification

The multispectral (MS) and the panchromatic (PAN) images represent complementary and synergistic spatial spectral information, how to make optimal use of the advantages of them has become a hot research topic. This paper proposes a selectable Transformer and Gist CNN network (STGC-Net). It designs a subspace similar recombination module (SSR-Module) based on non-negative matrix factorization (NMF) and the self-attention mechanism for feature decomposition. This can alleviate the redundant information of multi-modal data and extract their own singular and common features. Considering that the MS and the PAN images exhibit different advantageous properties, a selectable self-attention spectral feature extraction module (S3FE-Module) and a multi-stream Gist spatial feature extraction module (MGSFE-Module) are proposed for the different singular features. The former can refine the Transformer’s input and simultaneously characterize the sequence information between channels for the MS image. The latter introduces the positional relationship between local features while extracting spatial features for the PAN image, thereby improving the accuracy of scene classification. Experimental results indicate that the proposed method performs better than the other methods. The relevant code of this paper is provided at: https://github.com/ru-willow/ST-GC-Net.

Geographic Information Visualization and Sustainable Development of Low-Carbon Rural Slow Tourism under Artificial Intelligence

This study conducts in-depth research on geographic information visualization and the sustainable development of low-carbon rural slow tourism under artificial intelligence (AI) to analyze and discuss the visualization of geographic information and the sustainable development of low-carbon slow tourism in rural areas. First, the development options related to low-carbon tourism in rural areas are discussed. Then, a low-carbon rural slow tourism recommendation method based on AI and a low-carbon rural tourism scene recognition method based on Cross-Media Retrieval (CMR) data are proposed. Finally, the proposed scheme is tested. The test results show that the carbon dioxide emissions of one-day tourism projects account for less than 10% of the total tourism industry. From the proportion, it is found that air transport accounts for the largest proportion, more than 40%. With the development of time, the number of rural slow tourists in Guizhou has increased the most, while the number of rural slow tourists in Yunnan has increased to a lesser extent. In the K-means clustering model, the accuracy of scenario classification based on the semantic features of scene attributes is 5.26% higher than that of attribute likelihood vectors. On the Support Vector Machine classifier, the scene classification accuracy based on the semantic features of scene attributes is 19.2% higher than that of the scene classification based on attribute likelihood vector features. CMR techniques have also played a satisfying role in identifying rural tourism scenarios. They enable passengers to quickly identify tourist attractions to save preparation time and provide more flexible time for the tour process. The research results have made certain contributions to the sustainable development of low-carbon rural slow tourism.

Adversarial Remote Sensing Scene Classification Based on Lie Group Feature Learning

Convolutional Neural Networks have been widely used in remote sensing scene classification. Since this kind of model needs a large number of training samples containing data category information, a Generative Adversarial Network (GAN) is usually used to address the problem of lack of samples. However, GAN mainly generates scene data samples that do not contain category information. To address this problem, a novel supervised adversarial Lie Group feature learning network is proposed. In the case of limited data samples, the model can effectively generate data samples with category information. There are two main differences between our method and the traditional GAN. First, our model takes category information and data samples as the input of the model and optimizes the constraint of category information in the loss function, so that data samples containing category information can be generated. Secondly, the object scale sample generation strategy is introduced, which can generate data samples of different scales and ensure that the generated data samples contain richer feature information. After large-scale experiments on two publicly available and challenging datasets, it is found that our method can achieve better scene classification accuracy even with limited data samples.

Resolution-Agnostic Remote Sensing Scene Classification With Implicit Neural Representations

Remote sensing scene classification is an important yet challenging task. In recent years, the excellent feature representation ability of convolutional neural networks (CNNs) has led to substantial improvements in scene classification accuracy. However, handling resolution variations of remote sensing images is still challenging because CNNs are not inherently capable of modeling multiresolution input images. In this letter, we propose a novel scene classification method with scale and resolution adaptation ability by leveraging the recent advances in implicit neural representations (INRs). Unlike previous CNN-based methods that make predictions based on rasterized image inputs, the proposed method converts the images as continuous functions with INRs optimization and then performs classification within the function space. When the image is represented as a function, the image resolution can be decoupled from the pixel values so that the resolution does not have much impact on the classification performance. Our method also shows great potential for multiresolution remote sensing scene classification. Using only a simple multilayer perceptron (MLP) classifier in the proposed function space, our method achieves classification accuracy comparable to deep CNNs but exhibits better adaptability to image scale and resolution changes.

Ensemble Learning for Multispectral Scene Classification

In the recent decades, various techniques based on deep convolutional neural networks (DCNNs) have been applied to scene classification. Most of the techniques are established upon single-spectral images such that environmental conditions may greatly affect the quality of images in the visible (RGB) spectrum. One remedy for this downside is to merge the infrared (IR) with the visible spectrum for gaining the complementary information in comparison with the unimodal analysis. This paper incorporates the RGB, IR and near-infrared (NIR) images into a multispectral analysis for scene classification. For this purpose, two strategies are adopted. In the first strategy, each RGB, IR and NIR image is separately applied to DCNNs and then classified according to the output score of each network. In addition, an optimal decision threshold is obtained based on the same output score of each network. In the second strategy, three image components are extracted from each type of image using wavelet transform decomposition. Independent DCNNs are then trained on the image components of all the scene classes. Eventually, the final classification of the scene is accomplished through an appropriate ensemble architecture. The use of this architecture alongside a transfer learning approach and simple classifiers leads to lesser computational costs in small datasets. These experiments reveal the superiority of the proposed method over the state-of-the-art architectures in terms of the accuracy of scene classification.

Remote sensing image scene classification by transfer learning to augment the accuracy

Efficiently executing image categorization with high spatial quality imagery from remote sensing can bring great benefits to scene classification. Effective feature representation is critical in the development of high-performance scene categorization techniques because sensor data processing is tough. Remote sensing as well as deep learning abilities have made it easier to extract spatiotemporal information for classification. Furthermore, other scientific disciplines, together with remote sensing, have made significant advances in image categorization by convolutional neural networks (CNNs), and transfer learning is being combined. Image categorization in this article was performed to enrich the accuracy of Scene classification using Transfer learning utilizing pre-trained Alex Net, and Visual Geometry Group (VGG) networks and compared with feature extraction methods. First, features were retrieved from the pre-trained network's second fully-connected layer and employed in SVM classification. Second, substituting the last layers of pre-trained networks with the notion of transfer learning was used to categorize new datasets. It is executed on the UCM Dataset as well as the SIRI-WHU Dataset. The proposed methodologies produced improved accuracy of 95% for UCM, 93% for SIRI-WHU Datasets.

CAW: A Remote-Sensing Scene Classification Network Aided by Local Window Attention.

Remote-sensing image scene data contain a large number of scene images with different scales. Traditional scene classification algorithms based on convolutional neural networks are difficult to extract complex spatial distribution and texture information in images, resulting in poor classification results. In response to the above problems, we introduce the vision transformer network structure with strong global modeling ability into the remote-sensing image scene classification task. In this paper, the parallel network structure of the local-window self-attention mechanism and the equivalent large convolution kernel is used to realize the spatial-channel modeling of the network so that the network has better local and global feature extraction performance. Experiments on the RSSCN7 dataset and the WHU-RS19 dataset show that the proposed network can improve the accuracy of scene classification. At the same time, the effectiveness of the network structure in remote-sensing image classification tasks is verified through ablation experiments, confusion matrix, and heat map results comparison.

Dance Art Scene Classification Based on Convolutional Neural Networks

Digital multimedia resources have become an important part of people’s daily cultural life. Automatic scene classification of a large number of dance art videos is the basis for scene semantic based video content retrieval. In order to improve the accuracy of scene classification, the videos are identified using a deep convolutional neural network based on differential evolution for dance art videos. First, the Canny operator is used in YCbCr colour space to detect the human silhouette in the key frames of the video. Then, the AdaBoost algorithm based on cascade structure is used to implement human target tracking and labelling, and the construction and updating of weak classifiers are analysed. Next, a differential evolution algorithm is used to optimise the structural parameters of the convolutional neural network, and an adaptive strategy is adopted for the scaling factor of the differential evolution algorithm to improve the optimisation solution accuracy. Finally, the improved deep convolutional neural network is used to train the classification of the labelled videos in order to obtain stable scene classification results. The experimental results show that by reasonably setting the crossover rate of differential evolution and the convolutional kernel size of the convolutional neural network, high scene classification performance can be obtained. The high accuracy and low root-mean-square error validate the applicability of the proposed method in dance art scene classification.

GenericConv: A Generic Model for Image Scene Classification Using Few-Shot Learning

Scene classification is one of the most complex tasks in computer-vision. The accuracy of scene classification is dependent on other subtasks such as object detection and object classification. Accurate results may be accomplished by employing object detection in scene classification since prior information about objects in the image will lead to an easier interpretation of the image content. Machine and transfer learning are widely employed in scene classification achieving optimal performance. Despite the promising performance of existing models in scene classification, there are still major issues. First, the training phase for the models necessitates a large amount of data, which is a difficult and time-consuming task. Furthermore, most models are reliant on data previously seen in the training set, resulting in ineffective models that can only identify samples that are similar to the training set. As a result, few-shot learning has been introduced. Although few attempts have been reported applying few-shot learning to scene classification, they resulted in perfect accuracy. Motivated by these findings, in this paper we implement a novel few-shot learning model—GenericConv—for scene classification that has been evaluated using benchmarked datasets: MiniSun, MiniPlaces, and MIT-Indoor 67 datasets. The experimental results show that the proposed model GenericConv outperforms the other benchmark models on the three datasets, achieving accuracies of 52.16 ± 0.015, 35.86 ± 0.014, and 37.26 ± 0.014 for five-shots on MiniSun, MiniPlaces, and MIT-Indoor 67 datasets, respectively.

A Lightweight Self-Supervised Representation Learning Algorithm for Scene Classification in Spaceborne SAR and Optical Images

Despite the increasing amount of spaceborne synthetic aperture radar (SAR) images and optical images, only a few annotated data can be used directly for scene classification tasks based on convolution neural networks (CNNs). For this situation, self-supervised learning methods can improve scene classification accuracy through learning representations from extensive unlabeled data. However, existing self-supervised scene classification algorithms are hard to deploy on satellites, due to the high computation consumption. To address this challenge, we propose a simple, yet effective, self-supervised representation learning (Lite-SRL) algorithm for the scene classification task. First, we design a lightweight contrastive learning structure for Lite-SRL, we apply a stochastic augmentation strategy to obtain augmented views from unlabeled spaceborne images, and Lite-SRL maximizes the similarity of augmented views to learn valuable representations. Then, we adopt the stop-gradient operation to make Lite-SRL’s training process not rely on large queues or negative samples, which can reduce the computation consumption. Furthermore, in order to deploy Lite-SRL on low-power on-board computing platforms, we propose a distributed hybrid parallelism (DHP) framework and a computation workload balancing (CWB) module for Lite-SRL. Experiments on representative datasets including OpenSARUrban, WHU-SAR6, NWPU-Resisc45, and AID dataset demonstrate that Lite-SRL can improve the scene classification accuracy under limited annotated data, and it is generalizable to both SAR and optical images. Meanwhile, compared with six state-of-the-art self-supervised algorithms, Lite-SRL has clear advantages in overall accuracy, number of parameters, memory consumption, and training latency. Eventually, to evaluate the proposed work’s on-board operational capability, we transplant Lite-SRL to the low-power computing platform NVIDIA Jetson TX2.

Land scene classification for remote sensing images with an improved capsule network

Recognizing land type using remote sensing images is vital for land resource management and monitoring. To improve the accuracy of land scene recognition from remote sensing images captured by sensors, an innovative classification method based on an improved capsule network was proposed. First, a circular local binary pattern algorithm extracted texture features from remote sensing images. The textured images were then fused with the original remote sensing images, and a 1-by-1 convolutional layer combined the fused feature graphs linearly. Finally, an improved capsule network performed remote sensing image scene classification. The experimental results show that the average accuracy rate of the proposed method improved by more than 6% compared with that of the typical convolutional neural network method, and the proposed approach outperformed other comparable methods in terms of accuracy. The proposed method not only provides a fast convergence speed but also improves the scene classification accuracy for remote sensing images, especially for images with rich texture information. This is helpful for land remote sensing image scene classification in various applications.

Application of a graph convolutional network with visual and semantic features to classify urban scenes

Urban scenes consist of visual and semantic features and exhibit spatial relationships among land-use types (e.g. industrial areas are far away from the residential zones). This study applied a graph convolutional network with neighborhood information (henceforth, named the neighbour supporting graph convolutional neural network), to learn spatial relationships for urban scene classification. Furthermore, a co-occurrence analysis with visual and semantic features proceeded to improve the accuracy of urban scene classification. We tested the proposed method with the fifth ring road of Beijing with an overall classification accuracy of 0.827 and a Kappa coefficient of 0.769. In comparison with other methods, such as support vector machine, random forest, and general graph convolutional network, the case study showed that the proposed method improved about 10% in urban scene classification.

A Deep Neural Network-Based Target Recognition Algorithm for Robot Scenes

Intelligent robots are a key vehicle for artificial intelligence and are widely employed in all aspects of everyday life and work, not just in the industry. One of the talents required for intelligent robots to complete their jobs is the capacity to identify their environment, which is a crucial obstacle to be overcome. Deep learning-based target identification algorithms currently do not fully leverage the link between high-level semantic and low-level detail information in the prediction step and hence are less successful in recognizing tiny target objects. Target recognition via vision sensors has also improved in accuracy and efficiency because of the development of deep learning. However, due to the insufficient usage of semantic information and precise texture information of underlying characteristics, tiny target recognition remains a difficulty. To address the aforementioned issues, we propose a target detection method based on a jump-connected pyramid model to improve the target detection performance of robots in complex scenarios. In order to verify the effectiveness of the algorithm, we designed and implemented a software system for target detection of intelligent robots and performed software integration of the proposed algorithm model with excellent experimental results. These experiments reveal that, when compared to other algorithms, our suggested algorithm’s characteristics have higher flexibility and robustness and can deliver a higher scene classification accuracy rate.

GSCCTL: a general semi-supervised scene classification method for remote sensing images based on clustering and transfer learning

ABSTRACT Recently, much research has shown that deep learning methods are superior in scene classification for remote sensing images (HSIs). However, the lack of labelled samples and computational resources present serious obstacles for HSI scene classification. To address these problems, we consider the pragmatic deep learning problem with only a few or no labelled samples. A general semi-supervised scene classification method based on clustering and transfer learning named GSCCTL is proposed to transfer outstanding supervised learning models to semi-supervised scene classification. Specifically, the GSCCTL model iterates between (1) scene clustering and (2) transfer learning to improve scene classification performance for the HSI data set. Since the clustering results may contain noise, we propose a new method to calculate the distance between the samples and the clustering center and add a reliable sample selection operation to the data after clustering. Initially, the model’s scene classification performance is weak, and transfer learning is carried out on a few samples close to the clustering center. In subsequent iterations, the model is constantly optimized, more and more samples are selected as reliable samples for training, and the model’s scene classification performance is strengthened. Through iterative clustering and transfer learning, the model is progressively optimized and gradually converged until the end condition is reached. Extensive experiments on the UCMerced, AID, and NWPU-RESISC45 remote sensing data sets demonstrate that the GSCCTL model based on clustering and transfer learning improved scene classification accuracy with only a few labelled samples and is suitable for semi-supervised scene classification.

A Supervised Progressive Growing Generative Adversarial Network for Remote Sensing Image Scene Classification

Remote sensing image scene classification is a challenging task. With the development of deep learning, methods based on convolutional neural networks (CNNs) have made great achievements in remote sensing image scene classification. Since the training of a CNN requires a large number of labeled samples, a generative adversarial network (GAN) for sample generation represents a new opportunity to solve the problem of the limited samples. However, most of the existing GAN-based sample generation methods can only generate unlabeled samples, instead of samples labeled with the corresponding scene category. In this article, to solve the problem, a supervised progressive growing generative adversarial network (SPG-GAN) is proposed for remote sensing image scene classification. The proposed method can generate labeled samples for the remote sensing image scene classification, significantly improving the classification accuracy in the case of limited samples. The SPG-GAN method has two main improvements. First, a conditional generative framework for labeled samples is proposed, in which the label information is added in the channel dimension as the input. By considering the constraints of the label information in the loss function, the network can be trained in the direction of a specific category. As a result, the network can generate remote sensing image scene classification samples with label categories. Second, a progressive growing sample generation method is introduced. In order to ensure that the generated samples have more spatial details, they are generated by progressively adding modules to the generator and discriminator, thereby ensuring that the generated sample is of better quality. After testing on two benchmark datasets and carrying out a large-scale experiment in the central area of the city of Wuhan in China, it was found that the proposed method can obtain a superior scene classification accuracy in the case of limited samples.

A Cross-Layer Nonlocal Network for Remote Sensing Scene Classification

Remote sensing scene classification (RSSC) is a fundamental yet challenging task in the domain of remote sensing (RS). Currently, the methods based on deep features from convolutional neural networks (CNNs) have significantly improved the scene classification accuracy (ACC). However, the standard convolution operations have limited capacity to model the long-range correlations and cannot effectively obtain global contextual understanding ability. In this letter, we propose a novel scene classification framework, termed cross-layer nonlocal network (CL-NL-Net), consisting of a backbone network, a cross-layer nonlocal (CL-NL) module, and a classifier. Among them, the backbone network is used to obtain multilayer convolutional features. The CL-NL module is the core of the proposed method, which captures the long-range correlations between different layers, so as to achieve a better global scene understanding ability. To verify the effectiveness of the proposed CL-NL-Net, we conduct experiments on four benchmark datasets, and the results demonstrate that the proposed method achieves competitive classification ACC and outperforms some state-of-the-art methods.