Scene Classification Techniques Research Articles

White shark optimizer with optimal deep learning based effective unmanned aerial vehicles communication and scene classification

Unmanned aerial vehicles (UAVs) become a promising enabler for the next generation of wireless networks with the tremendous growth in electronics and communications. The application of UAV communications comprises messages relying on coverage extension for transmission networks after disasters, Internet of Things (IoT) devices, and dispatching distress messages from the device positioned within the coverage hole to the emergency centre. But there are some problems in enhancing UAV clustering and scene classification using deep learning approaches for enhancing performance. This article presents a new White Shark Optimizer with Optimal Deep Learning based Effective Unmanned Aerial Vehicles Communication and Scene Classification (WSOODL-UAVCSC) technique. UAV clustering and scene categorization present many deep learning challenges in disaster management: scene understanding complexity, data variability and abundance, visual data feature extraction, nonlinear and high-dimensional data, adaptability and generalization, real-time decision making, UAV clustering optimization, sparse and incomplete data. the need to handle complex, high-dimensional data, adapt to changing environments, and make quick, correct decisions in critical situations drives deep learning in UAV clustering and scene categorization. The purpose of the WSOODL-UAVCSC technique is to cluster the UAVs for effective communication and scene classification. The WSO algorithm is utilized for the optimization of the UAV clustering process and enables to accomplish effective communication and interaction in the network. With dynamic adjustment of the clustering, the WSO algorithm improves the performance and robustness of the UAV system. For the scene classification process, the WSOODL-UAVCSC technique involves capsule network (CapsNet) feature extraction, marine predators algorithm (MPA) based hyperparameter tuning, and echo state network (ESN) classification. A wide-ranging simulation analysis was conducted to validate the enriched performance of the WSOODL-UAVCSC approach. Extensive result analysis pointed out the enhanced performance of the WSOODL-UAVCSC method over other existing techniques. The WSOODL-UAVCSC method achieved an accuracy of 99.12%, precision of 97.45%, recall of 98.90%, and F1-score of 98.10% when compared to other existing techniques.

A Survey Paper on AutoMate - A Smart Assistant

Abstract: In the subject of intelligent transportation systems, autonomous vehicles are a prominent study issue because they have the potential to significantly reduce traffic and increase travel efficiency. One of the essential technologies for self-driving automobiles is scene classification, which serves as the foundation for these vehicles' decision-making. Deep learning-based approaches have shown promise in solving the scene classification problem in recent years. Nonetheless, there are a few issues with scene classification techniques that need more research, like how to handle similarities and contrasts within the same category. This paper proposes an enhanced deep network-based scene classification technique to address these issues. . In the subject of intelligent transportation systems, autonomous vehicles are a prominent study issue because they have the potential to significantly reduce traffic and increase travel efficiency. One of the essential technologies for self-driving automobiles is scene classification, which serves as the foundation for these vehicles' decision-making. Deep learning-based approaches have shown promise in solving the scene classification problem in recent years. Nonetheless, there are a few issues with scene classification techniques that need more research, like how to handle similarities and contrasts within the same category. This paper proposes an enhanced deep network-based scene classification technique to address these issues. . The experimental results show that the accuracy of the proposed method can reach which is higher than the state-of-the art methods

Ebola optimization with modified DarkNet‐53 model for scene classification and security on Internet of Things in smart cities

Remote sensing (RS) has developed significantly with the progress of the Internet of Things (IoT) which is allowed the cheap and fast acquisition of data in millions and billions of interrelated devices utilized throughout the whole world. RS scene classifier that purposes for classifying scene types for RS images has wide applications in several domains like urban planning, national defence security, environmental monitoring, and natural hazard detection. State-of-the-art deep learning (DL) successes are performed in a novel wave of RS scene classification applications, but it is the absence of explainability and trustworthiness. An intrusion detection system (IDS) plays a vital role to ensure security in the RS-based IoT environment. In this aspect, this study presents an ebola optimization algorithm with deep learning-based scene classification and intrusion detection (EOADL-SCID) technique on IoT-enabled remote sensing images. The aim of the EOADL-SCID system lies in the effectual scene classification of remote sensing images and intrusion detection. It involves a two-stage procedure. In the initial stage, the EOADL-SCID algorithm involves a modified DarkNet-53 feature extractor, EOA-based hyperparameter tuning, and graph convolution network (GCN) based classification. Next, in the second stage, the intrusion detection process takes place via two subprocesses namely variational autoencoder (VAE) based intrusion detection and skill optimization algorithm (SOA) based parameter tuning. The simulation outcomes of the EOADL-SCID approach are tested utilizing two benchmark databases and the experimental outcomes highlighted the improved performance of the EOADL-SCID algorithm on scene classification and intrusion classification processes.

Efficient key frame extraction and hybrid wavelet convolutional manta ray foraging for sports video classification

ABSTRACT Sports video classification (SVC) is now considered a challenging topic, therefore, developing an automatic sports scene classification technique has received tremendous interest. This research develops an efficient key frame extraction method and hybrid Wavelet Convolution Neural Network (WCNN) framework with optimization scheme to classify sports videos. Initially, input videos are converted into number of frames, and keyframes are extracted using Enhanced threshold with Discrete Wavelet Transform (ETDWT) method. Then, Cross Guided Bilateral Filter (CGBF) method eliminates the noise from the keyframe. After that, segmentation process is performed by the Fuzzy Equilibrium Optimizer (FEO) algorithm, and then motions are detected using the Farneback optical flow (OF) method. Finally, classification process is performed using Hybrid Wavelet Convolutional Manta Ray Foraging Optimization (HWCMRFO) algorithm to categorize different sports videos. The overall work is implemented using Python language. Simulation results proved that the proposed work achieved the highest accuracy (93.17%) compared to existing approaches.

IoT-Cloud Empowered Aerial Scene Classification for Unmanned Aerial Vehicles

Recent trends in communication technologies and unmanned aerial vehicles (UAVs) find its application in several areas such as healthcare, surveillance, transportation, etc. Besides, the integration of Internet of things (IoT) with cloud computing environment offers several benefits for the UAV communication. At the same time, aerial scene classification is one of the major research areas in UAV-enabled MEC systems. In UAV aerial imagery, efficient image representation is crucial for the purpose of scene classification. The existing scene classification techniques generate mid-level image features with limited representation capabilities that often end up in producing average results. Therefore, the current research work introduces a new DL-enabled aerial scene classification model for UAV-enabled MEC systems. The presented model enables the UAVs to capture aerial images which are then transmitted to MEC for further processing. Next, Capsule Network (CapsNet)-based feature extraction technique is applied to derive a set of useful feature vectors from the aerial image. It is important to have an appropriate hyperparameter tuning strategy, since manual parameter tuning of DL model tend to produce several configuration errors. In order to achieve this and to determine the hyperparameters of CapsNet model, Shuffled Shepherd Optimization (SSO) algorithm is implemented. Finally, Backpropagation Neural Network (BPNN) classification model is applied to determine the appropriate class labels of aerial images. The performance of SSO-CapsNet model was validated against two openly-accessible datasets namely, UC Merced (UCM) Land Use dataset and WHU-RS dataset. The proposed SSO-CapsNet model outperformed the existing state-of-the-art methods and achieved maximum accuracy of 0.983, precision of 0.985, recall of 0.982, and F-score of 0.983.

Robust Regressive Feature Extraction Based Relevance Vector Margin Boosting For Aerial Images Scene Classification

Aerial image scene classification is a key problem to be resolved in image processing. Many research works have been designed for carried outing scene classification. But, accuracy of existing scene classification was lower. In order to overcome such limitation, a Robust Regressive Feature Extraction Based Relevance Vector Margin Boosting Scene Classification (RRFERVMBSC) Technique is proposed. The RRFE-RVMBSC technique is designed for improving the classification performance of aerial images with minimal time. The RRFERVMBSC technique comprises two main processes namely feature extraction and classification. Initially, RRFE-RVMBSC technique gets number of aerial images as input. After taking input, Robust Regressive Independent Component Analysis Based Feature Extraction process is performed in order to extract the features i.e. shape, color, texture and size from aerial image. After completing feature extraction process, RRFERVMBSC technique carried outs Ensembled Relevance Vector Margin Boosting Classification (ERVMBC) where all the input aerial images are classified into multiple classes with higher accuracy. The RRFE-RVMBSC technique constructs a strong classifier by reducing the training error of weak RVM classifier for effectual aerial images scene categorization. The RRFERVMBSC technique accomplishes simulation work using parameters such as feature extraction time classification accuracy and false positive rate with respect to number of aerial images.

Assessing Neural Network Scene Classification from Degraded Images

Scene recognition is an essential component of both machine and biological vision. Recent advances in computer vision using deep convolutional neural networks (CNNs) have demonstrated impressive sophistication in scene recognition, through training on large datasets of labeled scene images (Zhou et al. 2018, 2014). One criticism of CNN-based approaches is that performance may not generalize well beyond the training image set (Torralba and Efros 2011), and may be hampered by minor image modifications, which in some cases are barely perceptible to the human eye (Goodfellow et al. 2015; Szegedy et al. 2013). While these “adversarial examples” may be unlikely in natural contexts, during many real-world visual tasks scene information can be degraded or limited due to defocus blur, camera motion, sensor noise, or occluding objects. Here, we quantify the impact of several image degradations (some common, and some more exotic) on indoor/outdoor scene classification using CNNs. For comparison, we use human observers as a benchmark, and also evaluate performance against classifiers using limited, manually selected descriptors. While the CNNs outperformed the other classifiers and rivaled human accuracy for intact images, our results show that their classification accuracy is more affected by image degradations than human observers. On a practical level, however, accuracy of the CNNs remained well above chance for a wide range of image manipulations that disrupted both local and global image statistics. We also examine the level of image-by-image agreement with human observers, and find that the CNNs’ agreement with observers varied as a function of the nature of image manipulation. In many cases, this agreement was not substantially different from the level one would expect to observe for two independent classifiers. Together, these results suggest that CNN-based scene classification techniques are relatively robust to several image degradations. However, the pattern of classifications obtained for ambiguous images does not appear to closely reflect the strategies employed by human observers.

Semantic and Spatial Co-Occurrence Analysis on Object Pairs for Urban Scene Classification

Urban scene classifications with very-high-resolution (VHR) satellite images play an important role in urban functional-zone investigation and landscape analysis. Most scene classification techniques utilize visual features which however are weak in identifying scenes, as scenes are usually composed of diverse objects with variant visual cues. To resolve this issue, we use spatial object relations to characterize urban scenes and recognize their categories. Accordingly, two questions are considered in this study: how to measure spatial object relations and how to use these relations to classify urban scenes? First, we propose a novel scene feature, semantic and spatial co-occurrence probability (SSCP), to measure spatial relations between objects with considering their directions, distances, and semantics, which can hopefully resolve the three key issues in measuring spatial object relations, i.e., anisotropy, scale- and semantic-dependencies. Then, a semilatent Dirichlet allocation is employed to classify scenes based on the proposed SSCP. In experiments, our method is first verified based on the UC Merced data set, and then used to generate an urban functional-zone map for Beijing, China. Both experiments indicate high accuracies and state-of-the-art performances of our method. In addition, we give a deep insight into SSCP's characteristics including convergence, high efficiency, and invariance to affine transformation, which can make our method more applicable.

Pre-Trained AlexNet Architecture with Pyramid Pooling and Supervision for High Spatial Resolution Remote Sensing Image Scene Classification

The rapid development of high spatial resolution (HSR) remote sensing imagery techniques not only provide a considerable amount of datasets for scene classification tasks but also request an appropriate scene classification choice when facing with finite labeled samples. AlexNet, as a relatively simple convolutional neural network (CNN) architecture, has obtained great success in scene classification tasks and has been proven to be an excellent foundational hierarchical and automatic scene classification technique. However, current HSR remote sensing imagery scene classification datasets always have the characteristics of small quantities and simple categories, where the limited annotated labeling samples easily cause non-convergence. For HSR remote sensing imagery, multi-scale information of the same scenes can represent the scene semantics to a certain extent but lacks an efficient fusion expression manner. Meanwhile, the current pre-trained AlexNet architecture lacks a kind of appropriate supervision for enhancing the performance of this model, which easily causes overfitting. In this paper, an improved pre-trained AlexNet architecture named pre-trained AlexNet-SPP-SS has been proposed, which incorporates the scale pooling—spatial pyramid pooling (SPP) and side supervision (SS) to improve the above two situations. Extensive experimental results conducted on the UC Merced dataset and the Google Image dataset of SIRI-WHU have demonstrated that the proposed pre-trained AlexNet-SPP-SS model is superior to the original AlexNet architecture as well as the traditional scene classification methods.

AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification

Aerial scene classification, which aims to automatically label an aerial image with a specific semantic category, is a fundamental problem for understanding high-resolution remote sensing imagery. In recent years, it has become an active task in remote sensing area and numerous algorithms have been proposed for this task, including many machine learning and data-driven approaches. However, the existing datasets for aerial scene classification like UC-Merced dataset and WHU-RS19 are with relatively small sizes, and the results on them are already saturated. This largely limits the development of scene classification algorithms. This paper describes the Aerial Image Dataset (AID): a large-scale dataset for aerial scene classification. The goal of AID is to advance the state-of-the-arts in scene classification of remote sensing images. For creating AID, we collect and annotate more than ten thousands aerial scene images. In addition, a comprehensive review of the existing aerial scene classification techniques as well as recent widely-used deep learning methods is given. Finally, we provide a performance analysis of typical aerial scene classification and deep learning approaches on AID, which can be served as the baseline results on this benchmark.

A Linear Dirichlet Mixture Model for decomposing scenes: Application to analyzing urban functional zonings

Urban scenes are fundamental to assessing urban landscapes and analyzing the spatial arrangements of functional zonings. Thus, it is important to obtain the information on urban scenes automatically from high-resolution remote sensing images. Previous studies have focused solely on obtaining qualitative labels of scenes (i.e., scene classification) as opposed to quantitative measurements on scenes (i.e., scene decomposition). However, scene classification alone cannot satisfy the demands of practical applications, as most scenes are mixed and composed of many categories. This study aims at 1) developing a decomposition model for quantifying mixed semantics of urban scenes, and 2) applying the proposed model to analyze urban functional zonings. For the first target, a Linear Dirichlet Mixture Model (LDMM) is proposed to quantify the relationships between pure scenes (including one single category) and mixed ones (including multiple categories). Since both mixed and pure scenes are characterized by frequency features (e.g., gray histograms of shapes), the variations in the quantization levels of frequency features could lead to different decomposition results. Accordingly, the stationary Markov process is further introduced to estimate the convergent decomposition results. For the second target, the LDMM was applied to obtain the qualitative categories and their quantitative proportions of urban scenes in Beijing and Zhuhai and to assess their spatial arrangements and temporal changes. Our methods were experimentally verified to be reliable and more effective in performance than the Linear Mixture Model and scene classification techniques. The case studies also suggested that Beijing and Zhuhai significantly differ in urban functional zonings and the functional zonings in Beijing have changed much over time, indicating an important transition in the city's economic structure.

Human-inspired features for natural scene classification

Scene classification has been the target of much research. Most psychological studies have agreed that humans perceive a scene first globally recognizing its category and then they localize and recognize objects. In previous work the same feature set were used in classifying both natural scenes and manmade scenes simultaneously. We suggest the use of different features for each. In this paper the proposed features for natural scenes classification are presented. The new proposed features are inspired from the way humans perceive and recognize scenes at a glance. Outdoor scenes global features such as openness, roughness, and dominant directions have been investigated and translated into a new feature set, focusing on characteristics that efficiently differentiate between natural scene sub-classes. The effectiveness of the proposed features is tested using two datasets consists of 4 natural scenes (coast, mountain, forest, and open country) and 6 natural scenes (the previous 4 scenes plus desert and waterfall scenes), the first dataset is a benchmark data set used for testing scene classification techniques. Results showed that a classification accuracy of up to 95% could be achieved using the proposed feature set.

Estimating Scene-Oriented Pseudo Depth With Pictorial Depth Cues

Estimating depth information from a single image has recently attracted great attention in 3D-TV applications, such as 2D-to-3D conversion owing to an insufficient supply of 3-D contents. In this paper, we present a new framework for estimating depth from a single image via scene classification techniques. Our goal is to produce perceptually reasonable depth for human viewers; we refer to this as pesudo depth estimation. Since the human visual system highly relies on structural information and salient objects in understanding scenes, we propose a framework that combines two depth maps: initial pseudo depth map (PDM) and focus depth map. We use machine learning based scene classification to classify the image into one of two classes, namely, object-view and non-object-view. The initial PDM is estimated by segmenting salient objects (in the case of object-view) and by analyzing scene structures (in the case of non-object-view). The focus blur is locally measured to improve the initial PDM. Two depth maps are combined, and a simple filtering method is employed to generate the final PDM. Simulation results show that the proposed method outperforms other state-of-the-art approaches for depth estimation in 2D-to-3D conversion, both quantitatively and qualitatively. Furthermore, we discuss how the proposed method can effectively be extended to image sequences by employing depth propagation techniques.

Wearable Mobility Aid for Low Vision Using Scene Classification in a Markov Random Field Model Framework

This article describes work on a novel approach to vision enhancement for people with severe visual impairments. This approach utilizes computer vision techniques to classify scene content so that visual enhancement of the scene can identify semantically important concepts. The mediated view of a scene presented to the user is in the form of a highly-saturated color image in which distinct colors represent important object types in the scene. The effectiveness of this scheme was demonstrated in a pilot study participated in by people with a range of visual impairments. The scene classification technique uses an artificial neural network classifier within the framework of a Markov random field model, and the accuracy and robustness of this technique using low quality video images from a hand-held camera is demonstrated.

Head-Mounted Mobility Aid for Low Vision Using Scene Classification Techniques

This paper describes a new mobility aid for people with severe visual impairments which combines technology from the field of virtual reality with advanced computer vision techniques. A neural-network classifier is used to identify objects in images from a head mounted camera so that scene content specifically important for mobility may be made more visible. Enhanced images are displayed to the user on a head mounted display using a high saturation colour scheme where each type of object has a different colour, resulting in images which are highly visible and easy to interpret. The object classifier achieves a level of accuracy over 90%. Results from a pilot study conducted using people with a range of visual impairments are presented in which performance on a difficult mobility-related task was improved by over 100% using the system.

A Normalization Procedure for METEOSAT Visible Channel Data

Abstract An efficient normalization scheme for satellite visible channel data is presented. The method is based on the percentile hypothesis. It is applicable to all weather scenes without any need for scene classification and detection techniques. Performance assessment with METEOSAT-2 visible data from March to October 1987, subsequent to parameter adjustment with May to August 1987 data, shows that the procedure, which accounts for the solar zenith angle, produces reasonable results for solar zenith distances up to almost 90°. The normalization reference angle may be chosen in order to optimize the brightness range of the normalized picture. Simple theoretical considerations allow insight into the structure of the scheme.