Scene Classification Research Articles

Remote sensing image scene classification based on residual dense network

Remote sensing image scene classification based on residual dense network

Gender, Ethnicity, and Age on TikTok: Content Analysis of Ecuadorian Tiktokers

The social network TikTok has seen significant growth since its launch in 2016. It currently has more than one billion users worldwide. In the case of Ecuador, the platform experienced an increase of more than 30% in users from September 2022 to the first quarter of 2024. Given this revealing panorama, this study aims to know what type of content is being disseminated and what differences in relation to gender, age and ethnicity, are found in the videos uploaded by Ecuadorian tiktokers. Using the content analysis methodology, 1,865 videos from 19 Ecuadorian TikTokers, between 18 and 36 years old, with more than 800,000 subscribers, were studied. Likewise, the Pearson correlation coefficient was applied with the Chi Square test to analyze whether there is a correlation between the three variables and the types of content. The results show that the most popular content corresponds to the categories of music, humor and scenes of everyday life. There is a presence of Tiktokers over 30 years old, which means that TikTok is ceasing to be a platform exclusively for teenagers, evolving towards more adult generations. The results revealed that female TikTokers have a greater following in relation to the number of subscribers and content. The data in relation to ethnicity reflects a statistical correlation that allows us to confirm that there are differences in the productions between indigenous and mestizo tiktokers.

Feature pyramid attention network for audio‐visual scene classification

AbstractAudio‐visual scene classification (AVSC) poses a formidable challenge owing to the intricate spatial‐temporal relationships exhibited by audio‐visual signals, coupled with the complex spatial patterns of objects and textures found in visual images. The focus of recent studies has predominantly revolved around extracting features from diverse neural network structures, inadvertently neglecting the acquisition of semantically meaningful regions and crucial components within audio‐visual data. The authors present a feature pyramid attention network (FPANet) for audio‐visual scene understanding, which extracts semantically significant characteristics from audio‐visual data. The authors’ approach builds multi‐scale hierarchical features of sound spectrograms and visual images using a feature pyramid representation and localises the semantically relevant regions with a feature pyramid attention module (FPAM). A dimension alignment (DA) strategy is employed to align feature maps from multiple layers, a pyramid spatial attention (PSA) to spatially locate essential regions, and a pyramid channel attention (PCA) to pinpoint significant temporal frames. Experiments on visual scene classification (VSC), audio scene classification (ASC), and AVSC tasks demonstrate that FPANet achieves performance on par with state‐of‐the‐art (SOTA) approaches, with a 95.9 F1‐score on the ADVANCE dataset and a relative improvement of 28.8%. Visualisation results show that FPANet can prioritise semantically meaningful areas in audio‐visual signals.

Empirical Evidence Regarding Few-Shot Learning for Scene Classification in Remote Sensing Images

Empirical Evidence Regarding Few-Shot Learning for Scene Classification in Remote Sensing Images

Optimizing Fire Scene Analysis: Hybrid Convolutional Neural Network Model Leveraging Multiscale Feature and Attention Mechanisms

The rapid and accurate detection of fire scenes in various environments is crucial for effective disaster management and mitigation. Fire scene classification is a critical aspect of modern fire detection systems that directly affects public safety and property preservation. This research introduced a novel hybrid deep learning model designed to enhance the accuracy and efficiency of fire scene classification across diverse environments. The proposed model integrates advanced convolutional neural networks with multiscale feature extraction, attention mechanisms, and ensemble learning to achieve superior performance in real-time fire detection. By leveraging the strengths of pre-trained networks such as ResNet50, VGG16, and EfficientNet-B3, the model captures detailed features at multiple scales, ensuring robust detection capabilities. Including spatial and channel attention mechanisms further refines the focus on critical areas within the input images, reducing false positives and improving detection precision. Extensive experiments on a comprehensive dataset encompassing wildfires, building fires, vehicle fires, and non-fire scenes demonstrate that the proposed framework outperforms existing cutting-edge techniques. The model also exhibited reduced computational complexity and enhanced inference speed, making it suitable for deployment in real-time applications on various hardware platforms. This study sets a new benchmark for fire detection and offers a powerful tool for early warning systems and emergency response initiatives.

CartoMark: a benchmark dataset for map pattern recognition and map content retrieval with machine intelligence

Maps are fundamental medium to visualize and represent the real word in a simple and philosophical way. The emergence of the big data tide has made a proportion of maps generated from multiple sources, significantly enriching the dimensions and perspectives for understanding the characteristics of the real world. However, a majority of these map datasets remain undiscovered, unacquired and ineffectively used, which arises from the lack of numerous well-labelled benchmark datasets, which are of significance to implement the deep learning techniques into identifying complicated map content. To address this issue, we develop a large-scale benchmark dataset involving well-labelled datasets to employ the state-of-the-art machine intelligence technologies for map text annotation recognition, map scene classification, map super-resolution reconstruction, and map style transferring. Furthermore, these well-labelled datasets would facilitate map feature detection, map pattern recognition and map content retrieval. We hope our efforts would provide well-labelled data resources for advancing the ability to recognize and discover valuable map content.

Retraction Note: Improved transfer learning of CNN through fine-tuning and classifier ensemble for scene classification

Retraction Note: Improved transfer learning of CNN through fine-tuning and classifier ensemble for scene classification

Multidimensional knowledge distillation for multimodal scene classification of remote sensing images

The advancement of deep learning technology has significantly improved the performance of remote sensing image (RSI) scene classification. However, it is important to note that most RSI scene classification models heavily depend on complex structures, resulting in high computational requirements and substantial costs. This study addresses this issue by utilizing a state-of-the-art model compression technique known as knowledge distillation (KD). The objective of KD is to transfer extensive knowledge from an excellent teacher model to a lightweight student model. While existing models focus on guiding the student network to learn specific stage or scale features from the teacher network, they lack comprehensiveness. To enhance the model's feature representation capability in complex scenarios, this study proposes a multidimensional KD approach (MKD). MKD enables the student network (MKD-S) to learn the feature representation capability of the teacher network (MKD-T) at each stage through a hybrid KD method. Specifically, the encoder incorporates a local-global KD mechanism to capture both low-level local information and high-level global information based on feature differences. Moreover, the fusion stage introduces inter-layer relationship KD and intra-layer feature KD to account for the dependencies between intermediate features within the MKD-S and MKD-T models. Additionally, the discrete wavelet transform, known for its ability to capture frequency domain and time domain features, is applied in the decoding stage of the MKD-T. This integration of decoding features across layers enables the completion of the knowledge response in the MKD-S. Experimental results demonstrate the effectiveness of our MKD on two benchmark datasets: Vaihingen and Potsdam.

Feature selection through adaptive sparse learning for scene recognition

Scene recognition is an important and challenging task in the field of computer vision. Current research typically focuses on local features in scene images by utilizing pretrained convolutional neural networks (CNN) to obtain a feature dictionary. However, these local features often contain similar features to other scene categories, leading to feature confusion and subsequently low accuracy in scene recognition. Therefore, focusing on local features for scene recognition is controversial. In contrast to existing works, we consider extracting common features within the same category from scene images and using them as discriminative features between different categories. We propose a scene recognition method based on adaptive sparse learning feature selection. This method leverages sparse learning to distinguish the contribution of each feature in the deep features to the classification task, aiming to construct a salient feature dictionary that describes the importance of features in scene images. Subsequently, an adaptive weight optimization method is employed through alternating updates to automatically adjust feature weights, enabling the selection of common features within the same scene category from the compact dictionary. Experimental results on multiple benchmark scene recognition datasets demonstrate that our proposed method is superior to state-of-the-art methods.

Dynamic Loss Balancing and Sequential Enhancement for Road-Safety Assessment and Traffic Scene Classification

Dynamic Loss Balancing and Sequential Enhancement for Road-Safety Assessment and Traffic Scene Classification

Hierarchical-Concatenate Fusion TDNN for sound event classification.

Semantic feature combination/parsing issue is one of the key problems in sound event classification for acoustic scene analysis, environmental sound monitoring, and urban soundscape analysis. The input audio signal in the acoustic scene classification is composed of multiple acoustic events, which usually leads to low recognition rate in complex environments. To address this issue, this paper proposes the Hierarchical-Concatenate Fusion(HCF)-TDNN model by adding HCF Module to ECAPA-TDNN model for sound event classification. In the HCF module, firstly, the audio signal is converted into two-dimensional time-frequency features for segmentation. Then, the segmented features are convolved one by one for improving the small receptive field in perceiving details. Finally, after the convolution is completed, the two adjacent parts are combined before proceeding with the next convolution for enlarging the receptive field in capturing large targets. Therefore, the improved model further enhances the scalability by emphasizing channel attention and efficient propagation and aggregation of feature information. The proposed model is trained and validated on the Urbansound8K dataset. The experimental results show that the proposed model can achieve the best classification accuracy of 95.83%, which is an approximate improvement of 5% (relatively) over the ECAPA-TDNN model.

Artificial Intelligence Scene Creation with Virtual Technology for the DigitalFilm

Artificial Intelligence-based scene creation leverages advanced algorithms to generate realistic and immersive environments for various applications, including gaming, virtual reality, and simulations. By utilizing techniques such as procedural generation, deep learning, and computer vision, these systems can automatically create complex landscapes, detailed textures, and dynamic elements that respond to user interactions. This not only enhances the user experience but also significantly reduces the time and effort required for manual scene design, enabling developers to focus on creativity and innovation. In the realm of digital media production, the quest for lifelike scenes and efficient evaluation methodologies has led to the exploration of novel techniques. This paper investigates the synergy between point estimation and artificial intelligence (AI) in advancing digital scene creation and evaluation. Through comprehensive simulations and analyses, we assess the effectiveness of point estimation in quantifying scene attributes such as visual realism, dynamic interactivity, physical accuracy, and artistic expression. Additionally, we delve into the utilization of AI algorithms for automating scene classification, thereby streamlining decision-making processes and optimizing resource allocation in production workflows. Through comprehensive simulations and analyses, we assess the effectiveness of point estimation in quantifying scene attributes such as visual realism (mean score: 0.85), dynamic interactivity (mean score: 0.72), physical accuracy (mean score: 0.93), and artistic expression (mean score: 0.65).

Multi-Scale and Multi-Network Deep Feature Fusion for Discriminative Scene Classification of High-Resolution Remote Sensing Images

The advancement in satellite image sensors has enabled the acquisition of high-resolution remote sensing (HRRS) images. However, interpreting these images accurately and obtaining the computational power needed to do so is challenging due to the complexity involved. This manuscript proposed a multi-stream convolutional neural network (CNN) fusion framework that involves multi-scale and multi-CNN integration for HRRS image recognition. The pre-trained CNNs were used to learn and extract semantic features from multi-scale HRRS images. Feature extraction using pre-trained CNNs is more efficient than training a CNN from scratch or fine-tuning a CNN. Discriminative canonical correlation analysis (DCCA) was used to fuse deep features extracted across CNNs and image scales. DCCA reduced the dimension of the features extracted from CNNs while providing a discriminative representation by maximizing the within-class correlation and minimizing the between-class correlation. The proposed model has been evaluated on NWPU-RESISC45 and UC Merced datasets. The accuracy associated with DCCA was 10% and 6% higher than discriminant correlation analysis (DCA) in the NWPU-RESISC45 and UC Merced datasets. The advantage of DCCA was better demonstrated in the NWPU-RESISC45 dataset due to the incorporation of richer within-class variability in this dataset. While both DCA and DCCA minimize between-class correlation, only DCCA maximizes the within-class correlation and, therefore, attains better accuracy. The proposed framework achieved higher accuracy than all state-of-the-art frameworks involving unsupervised learning and pre-trained CNNs and 2–3% higher than the majority of fine-tuned CNNs. The proposed framework offers computational time advantages, requiring only 13 s for training in NWPU-RESISC45, compared to a day for fine-tuning the existing CNNs. Thus, the proposed framework achieves a favourable balance between efficiency and accuracy in HRRS image recognition.

Acoustic Scene Classification using Dynamic Time Warping Technique based on Short Time Fourier Transform and Discrete Wavelet Transforms

Acoustic Scene Classification using Dynamic Time Warping Technique based on Short Time Fourier Transform and Discrete Wavelet Transforms

Scene Classification of Remote Sensing Image Based on Multi-Path Reconfigurable Neural Network

Land image recognition and classification and land environment detection are important research fields in remote sensing applications. Because of the diversity and complexity of different tasks of land environment recognition and classification, it is difficult for researchers to use a single model to achieve the best performance in scene classification of multiple remote sensing land images. Therefore, to determine which model is the best for the current recognition classification tasks, it is often necessary to select and experiment with many different models. However, finding the optimal model is accompanied by an increase in trial-and-error costs and is a waste of researchers’ time, and it is often impossible to find the right model quickly. To address the issue of existing models being too large for easy selection, this paper proposes a multi-path reconfigurable network structure and takes the multi-path reconfigurable residual network (MR-ResNet) model as an example. The reconfigurable neural network model allows researchers to selectively choose the required modules and reassemble them to generate customized models by splitting the trained models and connecting them through modules with different properties. At the same time, by introducing the concept of a multi-path input network, the optimal path is selected by inputting different modules, which shortens the training time of the model and allows researchers to easily find the network model suitable for the current application scenario. A lot of training data, computational resources, and model parameter experience are saved. Three public datasets, NWPU-RESISC45, RSSCN7, and SIRI-WHU datasets, were used for the experiments. The experimental results demonstrate that the proposed model surpasses the classic residual network (ResNet) in terms of both parameters and performance.

Instance-Level Scaling and Dynamic Margin-Alignment Knowledge Distillation for Remote Sensing Image Scene Classification

Instance-Level Scaling and Dynamic Margin-Alignment Knowledge Distillation for Remote Sensing Image Scene Classification

Effects of automatic auditory scene classification on speech perception in noise and real-world functional communication in children using cochlear implants

Objectives To investigate the effects of automatic scene classification (SCAN) on speech perception in noise and real-world functional performance in children using cochlear implants (CIs). Methods We used a within-subjects repeated measures design in two studies. The first study assessed speech perception in noise with or without SCAN enabled in 25 school-aged children. The second study evaluated functional auditory performance in real life. Parents of 18 children provided ratings using the Parents’ Evaluation of Aural/oral Performance of Children (PEACH) questionnaire; and children provided ratings using the Self Evaluation of Listening Function (SELF) questionnaire. Analyses of variance with repeated measures were used to examine the effect of SCAN. Results On average, speech perception in noise was significantly better with SCAN enabled (mean SRT: −4.1 dB; SD: 4.0), compared to SCAN disabled (mean SRT: 0.5 dB; SD: 3.5). Children’s functional performance in real life was similar between the two device settings. Conclusion Automatic auditory scene classification provides significant benefits for speech perception in noise (4.6 dB improvement). On average, there were no perceived detrimental or beneficial effects in real life. These findings support the use of SCAN in CIs for young children.

Enhancing land cover classification via deep ensemble network

The rapid adoption of drones has transformed industries such as agriculture, environmental monitoring, surveillance, and disaster management by enabling more efficient data collection and analysis. However, existing UAV-based image scene classification techniques face limitations, particularly in handling dynamic scenes, varying environmental conditions, and accurately identifying small or partially obscured objects. These challenges necessitate more advanced and robust methods for land cover classification. In response, this study explores ensemble learning (EL) as a powerful alternative to traditional machine learning approaches. By integrating predictions from multiple models, EL enhances accuracy, precision, and robustness in UAV-based land use and land cover classification. This research introduces a two-phase approach combining data preprocessing with feature extraction using three advanced ensemble models DenseNet201, EfficientNetV2S, and Xception employing transfer learning. These models were selected based on their higher performance during preliminary evaluations. Furthermore, a soft attention mechanism is incorporated into the ensembled network to optimize feature selection, resulting in improved classification outcomes. The proposed model achieved an accuracy of 97 %, precision of 96 %, recall of 96 %, and an F1-score of 97 % on UAV image datasets. Comparative analysis reveals a 4.2 % accuracy improvement with the ensembled models and a 1 % boost with the advanced hybrid models. This work significantly advances UAV image scene classification, offering a practical solution to enhance decision-making precision in various applications. The ensemble system demonstrates its effectiveness in remote sensing applications, especially in land cover analysis across diverse geographical and environmental settings.

Application of logistic function in a new PMV modification model for elderly people: Combining age and TSV

The accurate prediction of thermal perception paves the way to create a thermal comfort environment. This study proposes a modified predicted mean vote (PMV) model for elderly people. Data from three age groups in two pensioners’ buildings are collected and divided: 80 % for calculating key parameters (B and BT) via logistic function, establishing the mPMV model, and 20 % for validation data. The model is further validated by a subset of the ASHRAE Global Thermal Comfort Database II. The mPMV model demonstrated improved prediction quality over the original PMV model, reducing mean mean absolute error (MAE), standard deviation (SD), and coefficient of variation of root mean square error (CVRMSE) by 0.67, 0.36, and 1.26, respectively, in the experimental data. However, due to various scene classifications in the ASHRAE data, the prediction quality of the mPMV model remains basically unchanged compared with the PMV model, with minor increases in MAE and CVRMSE and a decrease in SD. With the elimination of the climate differences, the mean MAE, SD and CVRMSE of the mPMV model reduces by 0.19, 0.16 and 0.29 compared with the PMV model. In addition, the mPMV model outperforms the other three modified models across all three age groups of the experimental data, but this advantage is only observed in the younger elderly group of the ASHRAE data. This study can offer methodological assistance in predicting the thermal sensation for elderly people and contribute to creating a thermal environment accordant to their thermal preferences.

Identification of Residential and Commercial Area using Convolutional Neural Network

Abstract— Image classification of land use using aerial scene classification has become increasingly common around the world. Utilizing the power of Convolutional Neural Networks (CNNs), identification of various city township areas using satellite imagery has become more efficient compared to the previous manual labeling. The objective of this research is to build a convolutional neural network model for residential and commercial area identification. In the research, we also adopted Inception V3 and VGG16 to develop two transfer learning models for the identification system. The Inception V3-based model achieved the highest overall accuracy value of 100%, showing its effectiveness in accurate residential and commercial area identification. The proposed CNN model achieved an accuracy of 99%, while the VGG-16 model with all configurations being frozen achieved 99% accuracy.