Scene Classification Research Articles

CartoMark: a benchmark dataset for map pattern recognition and map content retrieval with machine intelligence

Maps are fundamental medium to visualize and represent the real word in a simple and philosophical way. The emergence of the big data tide has made a proportion of maps generated from multiple sources, significantly enriching the dimensions and perspectives for understanding the characteristics of the real world. However, a majority of these map datasets remain undiscovered, unacquired and ineffectively used, which arises from the lack of numerous well-labelled benchmark datasets, which are of significance to implement the deep learning techniques into identifying complicated map content. To address this issue, we develop a large-scale benchmark dataset involving well-labelled datasets to employ the state-of-the-art machine intelligence technologies for map text annotation recognition, map scene classification, map super-resolution reconstruction, and map style transferring. Furthermore, these well-labelled datasets would facilitate map feature detection, map pattern recognition and map content retrieval. We hope our efforts would provide well-labelled data resources for advancing the ability to recognize and discover valuable map content.

Retraction Note: Improved transfer learning of CNN through fine-tuning and classifier ensemble for scene classification

Retraction Note: Improved transfer learning of CNN through fine-tuning and classifier ensemble for scene classification

Hierarchical-Concatenate Fusion TDNN for sound event classification.

Semantic feature combination/parsing issue is one of the key problems in sound event classification for acoustic scene analysis, environmental sound monitoring, and urban soundscape analysis. The input audio signal in the acoustic scene classification is composed of multiple acoustic events, which usually leads to low recognition rate in complex environments. To address this issue, this paper proposes the Hierarchical-Concatenate Fusion(HCF)-TDNN model by adding HCF Module to ECAPA-TDNN model for sound event classification. In the HCF module, firstly, the audio signal is converted into two-dimensional time-frequency features for segmentation. Then, the segmented features are convolved one by one for improving the small receptive field in perceiving details. Finally, after the convolution is completed, the two adjacent parts are combined before proceeding with the next convolution for enlarging the receptive field in capturing large targets. Therefore, the improved model further enhances the scalability by emphasizing channel attention and efficient propagation and aggregation of feature information. The proposed model is trained and validated on the Urbansound8K dataset. The experimental results show that the proposed model can achieve the best classification accuracy of 95.83%, which is an approximate improvement of 5% (relatively) over the ECAPA-TDNN model.

Artificial Intelligence Scene Creation with Virtual Technology for the DigitalFilm

Artificial Intelligence-based scene creation leverages advanced algorithms to generate realistic and immersive environments for various applications, including gaming, virtual reality, and simulations. By utilizing techniques such as procedural generation, deep learning, and computer vision, these systems can automatically create complex landscapes, detailed textures, and dynamic elements that respond to user interactions. This not only enhances the user experience but also significantly reduces the time and effort required for manual scene design, enabling developers to focus on creativity and innovation. In the realm of digital media production, the quest for lifelike scenes and efficient evaluation methodologies has led to the exploration of novel techniques. This paper investigates the synergy between point estimation and artificial intelligence (AI) in advancing digital scene creation and evaluation. Through comprehensive simulations and analyses, we assess the effectiveness of point estimation in quantifying scene attributes such as visual realism, dynamic interactivity, physical accuracy, and artistic expression. Additionally, we delve into the utilization of AI algorithms for automating scene classification, thereby streamlining decision-making processes and optimizing resource allocation in production workflows. Through comprehensive simulations and analyses, we assess the effectiveness of point estimation in quantifying scene attributes such as visual realism (mean score: 0.85), dynamic interactivity (mean score: 0.72), physical accuracy (mean score: 0.93), and artistic expression (mean score: 0.65).

Multi-Scale and Multi-Network Deep Feature Fusion for Discriminative Scene Classification of High-Resolution Remote Sensing Images

The advancement in satellite image sensors has enabled the acquisition of high-resolution remote sensing (HRRS) images. However, interpreting these images accurately and obtaining the computational power needed to do so is challenging due to the complexity involved. This manuscript proposed a multi-stream convolutional neural network (CNN) fusion framework that involves multi-scale and multi-CNN integration for HRRS image recognition. The pre-trained CNNs were used to learn and extract semantic features from multi-scale HRRS images. Feature extraction using pre-trained CNNs is more efficient than training a CNN from scratch or fine-tuning a CNN. Discriminative canonical correlation analysis (DCCA) was used to fuse deep features extracted across CNNs and image scales. DCCA reduced the dimension of the features extracted from CNNs while providing a discriminative representation by maximizing the within-class correlation and minimizing the between-class correlation. The proposed model has been evaluated on NWPU-RESISC45 and UC Merced datasets. The accuracy associated with DCCA was 10% and 6% higher than discriminant correlation analysis (DCA) in the NWPU-RESISC45 and UC Merced datasets. The advantage of DCCA was better demonstrated in the NWPU-RESISC45 dataset due to the incorporation of richer within-class variability in this dataset. While both DCA and DCCA minimize between-class correlation, only DCCA maximizes the within-class correlation and, therefore, attains better accuracy. The proposed framework achieved higher accuracy than all state-of-the-art frameworks involving unsupervised learning and pre-trained CNNs and 2–3% higher than the majority of fine-tuned CNNs. The proposed framework offers computational time advantages, requiring only 13 s for training in NWPU-RESISC45, compared to a day for fine-tuning the existing CNNs. Thus, the proposed framework achieves a favourable balance between efficiency and accuracy in HRRS image recognition.

Acoustic Scene Classification using Dynamic Time Warping Technique based on Short Time Fourier Transform and Discrete Wavelet Transforms

Acoustic Scene Classification using Dynamic Time Warping Technique based on Short Time Fourier Transform and Discrete Wavelet Transforms

Scene Classification of Remote Sensing Image Based on Multi-Path Reconfigurable Neural Network

Land image recognition and classification and land environment detection are important research fields in remote sensing applications. Because of the diversity and complexity of different tasks of land environment recognition and classification, it is difficult for researchers to use a single model to achieve the best performance in scene classification of multiple remote sensing land images. Therefore, to determine which model is the best for the current recognition classification tasks, it is often necessary to select and experiment with many different models. However, finding the optimal model is accompanied by an increase in trial-and-error costs and is a waste of researchers’ time, and it is often impossible to find the right model quickly. To address the issue of existing models being too large for easy selection, this paper proposes a multi-path reconfigurable network structure and takes the multi-path reconfigurable residual network (MR-ResNet) model as an example. The reconfigurable neural network model allows researchers to selectively choose the required modules and reassemble them to generate customized models by splitting the trained models and connecting them through modules with different properties. At the same time, by introducing the concept of a multi-path input network, the optimal path is selected by inputting different modules, which shortens the training time of the model and allows researchers to easily find the network model suitable for the current application scenario. A lot of training data, computational resources, and model parameter experience are saved. Three public datasets, NWPU-RESISC45, RSSCN7, and SIRI-WHU datasets, were used for the experiments. The experimental results demonstrate that the proposed model surpasses the classic residual network (ResNet) in terms of both parameters and performance.

Instance-Level Scaling and Dynamic Margin-Alignment Knowledge Distillation for Remote Sensing Image Scene Classification

Instance-Level Scaling and Dynamic Margin-Alignment Knowledge Distillation for Remote Sensing Image Scene Classification

Effects of automatic auditory scene classification on speech perception in noise and real-world functional communication in children using cochlear implants

Objectives To investigate the effects of automatic scene classification (SCAN) on speech perception in noise and real-world functional performance in children using cochlear implants (CIs). Methods We used a within-subjects repeated measures design in two studies. The first study assessed speech perception in noise with or without SCAN enabled in 25 school-aged children. The second study evaluated functional auditory performance in real life. Parents of 18 children provided ratings using the Parents’ Evaluation of Aural/oral Performance of Children (PEACH) questionnaire; and children provided ratings using the Self Evaluation of Listening Function (SELF) questionnaire. Analyses of variance with repeated measures were used to examine the effect of SCAN. Results On average, speech perception in noise was significantly better with SCAN enabled (mean SRT: −4.1 dB; SD: 4.0), compared to SCAN disabled (mean SRT: 0.5 dB; SD: 3.5). Children’s functional performance in real life was similar between the two device settings. Conclusion Automatic auditory scene classification provides significant benefits for speech perception in noise (4.6 dB improvement). On average, there were no perceived detrimental or beneficial effects in real life. These findings support the use of SCAN in CIs for young children.

Identification of Residential and Commercial Area using Convolutional Neural Network

Abstract— Image classification of land use using aerial scene classification has become increasingly common around the world. Utilizing the power of Convolutional Neural Networks (CNNs), identification of various city township areas using satellite imagery has become more efficient compared to the previous manual labeling. The objective of this research is to build a convolutional neural network model for residential and commercial area identification. In the research, we also adopted Inception V3 and VGG16 to develop two transfer learning models for the identification system. The Inception V3-based model achieved the highest overall accuracy value of 100%, showing its effectiveness in accurate residential and commercial area identification. The proposed CNN model achieved an accuracy of 99%, while the VGG-16 model with all configurations being frozen achieved 99% accuracy.

Semi-Supervised Subcategory Centroid Alignment-Based Scene Classification for High-Resolution Remote Sensing Images

It is usually hard to obtain adequate annotated data for delivering satisfactory scene classification results. Semi-supervised scene classification approaches can transfer the knowledge learned from previously annotated data to remote sensing images with scarce samples for satisfactory classification results. However, due to the differences between sensors, environments, seasons, and geographical locations, cross-domain remote sensing images exhibit feature distribution deviations. Therefore, semi-supervised scene classification methods may not achieve satisfactory classification accuracy. To address this problem, a novel semi-supervised subcategory centroid alignment (SSCA)-based scene classification approach is proposed. The SSCA framework is made up of two components, namely the rotation-robust convolutional feature extractor (RCFE) and the neighbor-based subcategory centroid alignment (NSCA). The RCFE aims to suppress the impact of rotation changes on remote sensing image representation, while the NSCA aims to decrease the impact of intra-category variety across domains on cross-domain scene classification. The SSCA algorithm and several competitive approaches are validated on two datasets to demonstrate its effectiveness. The results prove that the proposed SSCA approach performs better than most competitive approaches by no less than 2% overall accuracy.

Low-complexity short-time acoustic scene classification based on causal convolutions

Acoustic scene classification is vital for building smart cities. Most acoustic scene classification studies are performed by training convolutional neural networks. The networks constructed based on the general convolutional approach need to consider the temporal-frequency connections in the audio spectrogram. This work proposes a bottleneck structure neural network model for acoustic scene classification based on causal convolution and Sub-spectral normalization to overcome the abovementioned problem. To obtain more sample data, a new mix-spec augmentation data enhancement method is proposed by drawing on the existing classical mixup and spec augmentation methods, and the three are combined for sample augmentation. It was experimentally found that the proposed bottleneck structure helps improve the recognition accuracy and robustness of the model. In addition, the ablation experiment showed that the data augmentation training model combined with the mixup, SpecAugment, and the newly proposed mix-SpecAugment method had the highest classification accuracy. Finally, the near-square audio spectrogram has better training results by employing different shapes of audio spectrograms for the experiments. Compared with the baseline results, it is found that our proposed model significantly improved the accuracy of the baseline and reduced loss.

Remote sensing scene classification based on contextual attention mechanism of lie group space

ABSTRACT High-resolution remote sensing (HRRS) scene classification is based on the semantic information in the image and its contextual spatial correlation, labelling different semantic categories for different HRRS. Convolutional neural networks (CNN) have been widely studied and applied in remote sensing scene classification in recent years. However, most of the existing CNN models focus on the global and high-level semantic features, ignoring the shallower feature information. In addition, complex background information and variable scales lead to a series of problems with large intra-class differences and high inter-class similarities, which also brings challenges to scene classification. To address the above problems and challenges, a scenario classification model based on the contextual spatial attention and channel attention mechanism of Lie Group manifold space learning is proposed in this study. In this model, we fully explore the multi-scale features of the scene (shallower-level and high-level) and propose a novel contextual spatial attention mechanism and channel attention mechanism. Extensive experimentation was carried out on the Union Remote Sensing Image Data Set (URSIS), which improved by 7.13% compared with the classical model. The experimental results showed that compared with other state-of-the-art remote sensing scene classification models, our proposed method has achieved significant improvement in classification accuracy and performance.

Teacher-Student Prototype Enhancement Network for a Few-Shot Remote Sensing Scene Classification

Few-shot remote sensing scene classification identifies new classes from limited labeled samples where the great challenges are intraclass diversity, interclass similarity, and limited supervision. To alleviate these problems, a teacher-student prototype enhancement network is proposed for a few-shot remote sensing scene classification. Instead of introducing an attentional mechanism in mainstream studies, a prototype enhancement module is recommended to adaptively select high-confidence query samples, which can enhance the support prototype representations to emphasize intraclass and interclass relationships. The construction of a few-shot teacher model generates more discriminative predictive representations with inputs from many labeled samples, thus providing a strong supervisory signal to the student model and encouraging the network to achieve accurate classification with a limited number of labeled samples. Extensive experiments of four public datasets, including NWPU-remote sens ing image scene classification (NWPU-RESISC45), aerial image dataset (AID), UC Merced, and WHU-RS19, demonstrate that this method achieves superior competitive performance than the state-of-the-art methods on five-way, one-shot, and five-shot classifications.

Multi-reference source aircraft classification network based on unknown class prototype estimation

Abstract Gathering complete aircraft types for close-set tasks is challenging and costly in fine-grained aircraft classification, resulting in encountering unknown class aircraft images in real-world models. To address this problem, we propose a network called a multi-reference source fine-grained aircraft classification network (MRSN) to explicitly and implicitly distinguish known class boundaries and embed unknown classes into the sample space. Specifically, we propose an embedding module (UCPE) to synthesize an unknown class prototype to facilitate the modeling of unknown class data distribution. Besides, we introduce a concatenated multi-reference source module (CMS) that combines distance measurement and probability estimation to classify whether testing images are of unknown class or not. Our proposed MRSN innovatively extends the open-set learning paradigm to the fine-grained classification of geospatial objects rather than simple scene classification. Besides, it shows excellent performance in our constructed dataset, which demonstrates the effectiveness and progressiveness of our method.

Low-complexity multi task learning for joint acoustic scenes classification and sound events detection

Low-complexity multi task learning for joint acoustic scenes classification and sound events detection

Enhancing land cover classification via deep ensemble network

The rapid adoption of drones has transformed industries such as agriculture, environmental monitoring, surveillance, and disaster management by enabling more efficient data collection and analysis. However, existing UAV-based image scene classification techniques face limitations, particularly in handling dynamic scenes, varying environmental conditions, and accurately identifying small or partially obscured objects. These challenges necessitate more advanced and robust methods for land cover classification. In response, this study explores ensemble learning (EL) as a powerful alternative to traditional machine learning approaches. By integrating predictions from multiple models, EL enhances accuracy, precision, and robustness in UAV-based land use and land cover classification. This research introduces a two-phase approach combining data preprocessing with feature extraction using three advanced ensemble models DenseNet201, EfficientNetV2S, and Xception employing transfer learning. These models were selected based on their higher performance during preliminary evaluations. Furthermore, a soft attention mechanism is incorporated into the ensembled network to optimize feature selection, resulting in improved classification outcomes. The proposed model achieved an accuracy of 97 %, precision of 96 %, recall of 96 %, and an F1-score of 97 % on UAV image datasets. Comparative analysis reveals a 4.2 % accuracy improvement with the ensembled models and a 1 % boost with the advanced hybrid models. This work significantly advances UAV image scene classification, offering a practical solution to enhance decision-making precision in various applications. The ensemble system demonstrates its effectiveness in remote sensing applications, especially in land cover analysis across diverse geographical and environmental settings.

A multi-scale dense residual correlation network for remote sensing scene classification

Most existing scene classification methods based on remote sensing images tend to ignore important interactive information at different levels in the image. We propose an effective remote sensing scene classification method named multi-scale dense residual correlation network. The method is divided into three parts. First, the multi-stream feature extraction module is introduced which effectively utilizes features at different scales to extract different levels of information. Secondly, the dense residual connected feature fusion technology is proposed, which allows for a wide range of feature fusion. The Correlation Attention Module learn feature representations at multiple levels. This improves classification performance. The method outperforms existing algorithms in terms of effectiveness and accuracy, achieving state-of-the-art results on widely used remote sensing scene classification benchmarks.

End-to-end training of acoustic scene classification using distributed sound-to-light conversion devices: verification through simulation experiments

We propose a framework for classifying acoustic scenes utilizing distributed sound sensor devices capable of sound-to-light conversion, which we term as Blinkies. These Blinkies can convert acoustic signals into varying intensities of light via an inbuilt light-emitting diode. By using Blinkies, we can aggregate the spatial acoustic information across a wide region by recording the fluctuating light intensities of numerous Blinkies distributed throughout the region. Nonetheless, the signal communicated is subject to the bandwidth limitation imposed by the frame rate of the video camera, typically capped at 30 frames per second. Our objective is to refine the process of transforming sound into light for the purpose of acoustic scene classification within these bandwidth confines. While traversing through the air, a light signal is affected by inherent physical limitations such as the attenuation of light and interference from noise. To account for these factors, we have integrated these physical constraints into differentiable physical layers. This approach enables us to jointly train a pair of deep neural networks for the conversion of sound to light and for the classification of acoustic scenes. Our simulation studies, which employed the SINS database for acoustic scene classification, demonstrated that our proposed framework outperforms the previous one that utilized Blinkies. These findings emphasize the effectiveness of Blinkies in the field of acoustic scene classification.

Multi-label remote sensing scene classification using two-level double channel spatial attention residual blocks

Multi-label remote sensing scene classification using two-level double channel spatial attention residual blocks