Segmentation Of Aerial Images Research Articles

Regulating Modality Utilization within Multimodal Fusion Networks

Multimodal fusion networks play a pivotal role in leveraging diverse sources of information for enhanced machine learning applications in aerial imagery. However, current approaches often suffer from a bias towards certain modalities, diminishing the potential benefits of multimodal data. This paper addresses this issue by proposing a novel modality utilization-based training method for multimodal fusion networks. The method aims to guide the network’s utilization on its input modalities, ensuring a balanced integration of complementary information streams, effectively mitigating the overutilization of dominant modalities. The method is validated on multimodal aerial imagery classification and image segmentation tasks, effectively maintaining modality utilization within ±10% of the user-defined target utilization and demonstrating the versatility and efficacy of the proposed method across various applications. Furthermore, the study explores the robustness of the fusion networks against noise in input modalities, a crucial aspect in real-world scenarios. The method showcases better noise robustness by maintaining performance amidst environmental changes affecting different aerial imagery sensing modalities. The network trained with 75.0% EO utilization achieves significantly better accuracy (81.4%) in noisy conditions (noise variance = 0.12) compared to traditional training methods with 99.59% EO utilization (73.7%). Additionally, it maintains an average accuracy of 85.0% across different noise levels, outperforming the traditional method’s average accuracy of 81.9%. Overall, the proposed approach presents a significant step towards harnessing the full potential of multimodal data fusion in diverse machine learning applications such as robotics, healthcare, satellite imagery, and defense applications.

3D Modeling of rural environments from multiscale aerial imagery

Given the increasing attention to environmental preservation and sustainable development, the digitization of rural landscapes stands out as a pivotal strategy for effective environmental management and sustainability, land use planning, and preservation of cultural heritage. This work proposes a novel methodology for generating 3D models of rural landscapes by integrating multiscale data sources. Although Unmanned Aerial Vehicles (UAV) simplify the acquisition of multi-source data, their coverage is typically restricted to small landscapes due to their limited range and flight time. On the other hand, although the use of aerial images provides a broader view of the terrain, it is important to note that the low resolution of these images interferes with the task of accurate 3D modeling. Given these challenges, we propose a methodology that combines UAV data and high-resolution aerial imagery provided by the Spanish National Orthophoto Program (PNOA). This multi-source data integration is crucial to generating detailed and accurate 3D models of rural environments. The proposed methodology involves three steps: (1) semantic segmentation of aerial images identifying features such as vegetation, ground, and human-made structures, (2) estimation of the Digital Elevation Model (DEM), and (3) 3D modeling of rural environments using the point clouds generated from UAV images. The conducted experiments demonstrate the effectiveness of our approach identifying and representing previously mentioned features. Thus, this work presents advances in 3D representation techniques for real scenarios, contributing to the coordination of land utilization and environmental sustainability in rural landscapes.

Cross-Correlation Driven Aerial Image Segmentation: Leveraging Multi-Scale Features and Edge Information

Abstract: Semantic segmentation of remote sensing images is crucial for interpreting these large, rich in information scenes and our study introduces a new method for segmenting remote sensing imagery (RSI) when faced with limited training data and imbalanced classes. Our approach utilizes a unique potential function that merges information from both super pixel segmentation and edge detection. This combination allows the model to effectively analyze features at various scales and reduce the influence of potential errors in super pixel segmentation. Furthermore, the inclusion of edge details extracted via the Sketch token algorithm refines object boundaries, yielding more accurate segmentation results. This work offers a promising solution for achieving reliable interpretation of RSIs in scenarios with limited training data.

Deep Learning-Based Detection and Segmentation of Damage in Solar Panels

Renewable energy can lead to a sustainable future and solar energy is one the primary sources of renewable energy. Solar energy is harvested mainly by photovoltaic plants. Though there are a large number of solar panels, the economic efficiency of solar panels is not that high in comparison to energy production from coal or nuclear matter. The main risk involved in solar plants is the high maintenance cost involved in maintaining the plants. To help reduce this issue, automated solutions using Unmanned Aerial Vehicles (UAVs) and satellite imagery are proposed. In this research work, we propose a novel deep learning architecture for the segmentation of solar plant aerial images, which not only helps in automated solar plant maintenance, but can also be used for the area estimation and extraction of solar panels from an image. Along with this, we also propose a transfer learning-based model for the efficient classification of solar panel damage. Solar panel damage classification has a lot of applications. It can be integrated into monitoring systems, raising alerts when there is severe damage or damage of a certain type. The adaptive UNet model with Atrous Spatial Pyramid Pooling (ASPP) module that performed the dilated convolutions that we proposed achieved an overall accuracy of 98% with a Mean Intersection-Over-Union (IoU) Score of 95% and took under a second to process an image. Our classification model using Visual Geometry Group 19 (VGG19) as the backbone for feature extraction has achieved a classification accuracy of 98% with an F1 score of 99%, thus detecting the five classes of damage, including undamaged solar panels, in an efficient manner.

Hybridizing Deep Neural Networks and Machine Learning Models for Aerial Satellite Forest Image Segmentation.

Forests play a pivotal role in mitigating climate change as well as contributing to the socio-economic activities of many countries. Therefore, it is of paramount importance to monitor forest cover. Traditional machine learning classifiers for segmenting images lack the ability to extract features such as the spatial relationship between pixels and texture, resulting in subpar segmentation results when used alone. To address this limitation, this study proposed a novel hybrid approach that combines deep neural networks and machine learning algorithms to segment an aerial satellite image into forest and non-forest regions. Aerial satellite forest image features were first extracted by two deep neural network models, namely, VGG16 and ResNet50. The resulting features are subsequently used by five machine learning classifiers including Random Forest (RF), Linear Support Vector Machines (LSVM), k-nearest neighbor (kNN), Linear Discriminant Analysis (LDA), and Gaussian Naive Bayes (GNB) to perform the final segmentation. The aerial satellite forest images were obtained from a deep globe challenge dataset. The performance of the proposed model was evaluated using metrics such as Accuracy, Jaccard score index, and Root Mean Square Error (RMSE). The experimental results revealed that the RF model achieved the best segmentation results with accuracy, Jaccard score, and RMSE of 94%, 0.913 and 0.245, respectively; followed by LSVM with accuracy, Jaccard score and RMSE of 89%, 0.876, 0.332, respectively. The LDA took the third position with accuracy, Jaccard score, and RMSE of 88%, 0.834, and 0.351, respectively, followed by GNB with accuracy, Jaccard score, and RMSE of 88%, 0.837, and 0.353, respectively. The kNN occupied the last position with accuracy, Jaccard score, and RMSE of 83%, 0.790, and 0.408, respectively. The experimental results also revealed that the proposed model has significantly improved the performance of the RF, LSVM, LDA, GNB and kNN models, compared to their performance when used to segment the images alone. Furthermore, the results showed that the proposed model outperformed other models from related studies, thereby, attesting its superior segmentation capability.

Fine-Grained Permeable Surface Mapping through Parallel U-Net.

Permeable surface mapping, which mainly is the identification of surface materials that will percolate, is essential for various environmental and civil engineering applications, such as urban planning, stormwater management, and groundwater modeling. Traditionally, this task involves labor-intensive manual classification, but deep learning offers an efficient alternative. Although several studies have tackled aerial image segmentation, the challenges in permeable surface mapping arid environments remain largely unexplored because of the difficulties in distinguishing pixel values of the input data and due to the unbalanced distribution of its classes. To address these issues, this research introduces a novel approach using a parallel U-Net model for the fine-grained semantic segmentation of permeable surfaces. The process involves binary classification to distinguish between entirely and partially permeable surfaces, followed by fine-grained classification into four distinct permeability levels. Results show that this novel method enhances accuracy, particularly when working with small, unbalanced datasets dominated by a single category. Furthermore, the proposed model is capable of generalizing across different geographical domains. Domain adaptation is explored to transfer knowledge from one location to another, addressing the challenges posed by varying environmental characteristics. Experiments demonstrate that the parallel U-Net model outperforms the baseline methods when applied across domains. To support this research and inspire future research, a novel permeable surface dataset is introduced, with pixel-wise fine-grained labeling for five distinct permeable surface classes. In summary, in this work, we offer a novel solution to permeable surface mapping, extend the boundaries of arid environment mapping, introduce a large-scale permeable surface dataset, and explore cross-area applications of the proposed model. The three contributions are enhancing the efficiency and accuracy of permeable surface mapping while progressing in this field.

Proposed Framework for Semantic Segmentation of Aerial Hyperspectral Images Using Deep Learning and SVM Approach

The combination of deep neural networks and assistance vector machines for hyperspectral image recognition is presented in this work. A key issue in the real-world hyperspectral imaging system is hyperspectral picture recognition. Although deep learning can replicate highly dimensional feature vectors from source data, it comes at a high cost in terms of time and the Hugh phenomenon. The selection of the kernel feature and limit has a significant impact on the presentation of a kernel-based learning system. We introduce Support Vector Machine (SVM), a kernel learning method that is used to feature vectors obtained from deep learning on hyperspectral images. By modifying the data structure's parameters and kernel functions, the learning system's ability to solve challenging problems is enhanced. The suggested approaches' viability is confirmed by the outcomes of the experiments. At a particular rate, accuracy of testing for classification is around 90%. Moreover, to significantly make framework robust, validation is done using 5-flod verification.

LSRFormer: Efficient Transformer Supply Convolutional Neural Networks With Global Information for Aerial Image Segmentation

LSRFormer: Efficient Transformer Supply Convolutional Neural Networks With Global Information for Aerial Image Segmentation

CNN with New Spatial Pyramid Pooling and Advanced Filter-Based Techniques: Revolutionizing Traffic Monitoring via Aerial Images

The escalation in vehicular traffic, in conjunction with global population growth, has precipitated heightened road congestion, augmented air pollution, and a rise in vehicular accidents. Over the past decade, the global vehicular count has witnessed a substantial surge. In this context, traffic monitoring emerges as a paramount challenge, especially within developing nations. This research introduces an innovative system for vehicle detection and categorization aimed at intelligent traffic monitoring. The system utilizes a convolutional neural network-based U-Net model for the segmentation of aerial images. After segmentation, the outputs are further analyzed for vehicle identification. This vehicle detection utilizes an advanced spatial pyramid pooling (ASPP) mechanism which refines the spatial partitions of the image and captures intricate details, enhancing the accuracy and precision of the detection process. Detected vehicles are then categorized into distinct subcategories. For the effective management and control of high-density traffic flow, the extended Kalman filter (EKF) technique is employed, thereby reducing the reliance on human oversight. In experimental evaluations, our proposed model exhibits exemplary vehicle detection capabilities across the German Aerospace Center (DLR3K) and the Vehicle Detection in Aerial Imagery (VEDAI) datasets. Potential applications of the system encompass vehicle identification in traffic streams, traffic congestion assessment, intersection traffic density analysis, differentiation of vehicle types, and pedestrian pathway determination.

Aerial image semantic segmentation based on 3D fits a small dataset of 1D

<p>Time restrictions and lack of precision demand that the initial technique be abandoned. Even though the remaining datasets had fewer identified classes than initially planned for the study, the labels were more accurate. Because of the need for additional data, a single network cannot categorize all the essential elements in a picture, including bodies of water, roads, trees, buildings, and crops. However, the final network gains some invariance in detecting these classes with environmental changes due to the different geographic positions of roads and buildings discovered in the final datasets, which could be valuable in future navigation research. At the moment, binary classifications of a single class are the only datasets that can be used for the semantic segmentation of aerial images. Even though some pictures have more than one classification, images of roads and buildings were only found in a significant number of samples. Then, the building datasets were pooled to produce a larger dataset and for the constructed models to gain some invariance on image location. Because of the massive disparity in sample size, road datasets needed to be integrated.</p>

HilbertSCNet: Self-attention networks for small target segmentation of aerial drone images

In recent years, many advanced semantic segmentation algorithms have made great progress in common datasets. However, in the drones aerial photography dataset, since some of the segmentation targets are very small in such high-resolution images, these current algorithms are not designed specifically for them resulting in their mediocre performance in the drones aerial photography dataset. To address the problem of difficult segmentation of these small targets, this paper proposes a new semantic segmentation network model-HilbertSCNet: (1) Combine the image dimensionality reduction algorithm of Hilbert curve traversal and the idea of dual pathway to design a new spatial computation module to solve the problem of small target information is easily lost in the downsampling; (2) Add a windowing algorithm in the image dimensionality reduction algorithm of Hilbert curve traversal to solve the problem that the computational complexity of the module is too high, which makes the module cannot be applied to high-resolution feature maps. Experiments show that the proposed network is very effective for segmentation of small targets under high-resolution maps such as drone aerial photography, with certain superiority and generalization. The overall segmentation performance is improved compared to the current new network, including 1.76% better than OCNet in MIoU and 4.55% better than the dedicated drone algorithm RCCT-ASPPNet.

An improved beluga whale optimizer-Derived Adaptive multi-channel DeepLabv3+ for semantic segmentation of aerial images.

Semantic segmentation process over Remote Sensing images has been regarded as hot research work. Even though the Remote Sensing images provide many essential features, the sampled images are inconsistent in size. Even if a similar network can segment Remote Sensing images to some extents, segmentation accuracy needs to be improved. General neural networks are used to improve categorization accuracy, but they also caused significant losses to target scale and spatial features, and the traditional common features fusion techniques can only resolve some of the issues. A segmentation network has been designed to resolve the above-mentioned issues as well. With the motive of addressing the difficulties in the existing semantic segmentation techniques for aerial images, the adoption of deep learning techniques is utilized. This model has adopted a new Adaptive Multichannel Deeplabv3+ (AMC-Deeplabv3+) with the help of a new meta-heuristic algorithm called Improved Beluga whale optimization (IBWO). Here, the hyperparameters of Multichannel deeplabv3+ are optimized by the IBWO algorithm. The proposed model significantly enhances the performance of the overall system by measuring the accuracy and dice coefficient. The proposed model attains improved accuracies of 98.65% & 98.72% for dataset 1 and 2 respectively and also achieves the dice coefficient of 98.73% & 98.85% respectively with a computation time of 113.0123 seconds. The evolutional outcomes of the proposed model show significantly better than the state of the art techniques like CNN, MUnet and DFCNN models.

An adaptive multichannel DeepLabv3 + for semantic segmentation of aerial images using improved Beluga Whale Optimization Algorithm

An adaptive multichannel DeepLabv3 + for semantic segmentation of aerial images using improved Beluga Whale Optimization Algorithm

Design of an Urban Domestic Waste Landfill Based on Aerial Image Segmentation and Ecological Restoration Theory

Properly managed urban household waste landfills play a crucial role in achieving sustainable development and ecological civilization. This paper proposes an innovative design method that aims to overcome the limitations and extensive challenges of traditional urban domestic waste landfill design by combining aerial image segmentation with ecological restoration theory. We suggest an enhanced SEVnet, a combination of the basic vnet network and the sequence-and-excitation module, for precise and efficient garbage dump identification. Through this module, the network can independently analyze the significance of each feature channel and provide weights, leading to enhanced image detail recovery and more precise segmentation. We implemented the proposed SEVnet model based on an aerial photography database of urban garbage dumps. Following precise urban garbage disposal site division, we adopt ecological restoration theory to achieve sustainable design. This article presents the Saihanba area in China as a case study, collects pertinent data, and conducts image segmentation and GIS-assisted analysis. This paper examines the landscape application and design of the landfill site based on an analysis and planning of land use in its vicinity. The intervention of landscape ecology imbues the landfill site with new functions. We propose comprehensive planning and landscape restoration design measures from four perspectives, encompassing disadvantages, threats, opportunities, and strengths, using SWOT analysis to better integrate the Hebei Saihanba landfill site with the surrounding environment. This paper introduces a groundbreaking design scheme for a landfill that aligns with the principles of current green and sustainable development.

MSINet: Mining scale information from digital surface models for semantic segmentation of aerial images

Compared with other kinds of images, aerial images have more obvious object scale distinction and larger resolution, which results in that the whole scale information of aerial images can hardly be explored. To address this difficulty, we develop a novel network based on the digital surface models (DSMs) of aerial images in this paper. The proposed network termed MSINet can efficiently mine scale information through the DSMs from two aspects. Firstly, we propose an interpolation pyramid algorithm to encode the scale information from the DSMs and hence provide a scale prior information to the normal segmentation network. The interpolation pyramid algorithm implements interpolation operations with different scales on the DSMs and detects the pixel value change after the interpolation operations. Objects with different scales will express diverse changes, which provides useful information to encode their scale information. Secondly, aiming to address the problem that the DSMs contain a large amount of noise in the boundary part, a spatial information enhancement module and a mutual-guidance module are developed in this paper. These two modules can fix the misleading guidance information caused by the noise in the boundary part of the DSMs and hence achieve more accurate scale information inserting. The extensive experimental results prove that our methods can outperform other competitors in terms of qualitative and quantitative performance.

Disentangling the Benefits of Self-Supervised Learning to Deployment-Driven Downstream Tasks of Satellite Images (Student Abstract)

In this paper, we investigate the benefits of self-supervised learning (SSL) to downstream tasks of satellite images. Unlike common student academic projects, this work focuses on the advantages of the SSL for deployment-driven tasks which have specific scenarios with low or high-spatial resolution images. Our preliminary experiments demonstrate the robust benefits of the SSL trained by medium-resolution (10m) images to both low-resolution (100m) scene classification case (4.25%↑) and very high-resolution (5cm) aerial image segmentation case (1.96%↑), respectively.

An improved golden jackal optimization for multilevel thresholding image segmentation.

Aerial photography is a long-range, non-contact method of target detection technology that enables qualitative or quantitative analysis of the target. However, aerial photography images generally have certain chromatic aberration and color distortion. Therefore, effective segmentation of aerial images can further enhance the feature information and reduce the computational difficulty for subsequent image processing. In this paper, we propose an improved version of Golden Jackal Optimization, which is dubbed Helper Mechanism Based Golden Jackal Optimization (HGJO), to apply multilevel threshold segmentation to aerial images. The proposed method uses opposition-based learning to boost population diversity. And a new approach to calculate the prey escape energy is proposed to improve the convergence speed of the algorithm. In addition, the Cauchy distribution is introduced to adjust the original update scheme to enhance the exploration capability of the algorithm. Finally, a novel "helper mechanism" is designed to improve the performance for escape the local optima. To demonstrate the effectiveness of the proposed algorithm, we use the CEC2022 benchmark function test suite to perform comparison experiments. the HGJO is compared with the original GJO and five classical meta-heuristics. The experimental results show that HGJO is able to achieve competitive results in the benchmark test set. Finally, all of the algorithms are applied to the experiments of variable threshold segmentation of aerial images, and the results show that the aerial photography images segmented by HGJO beat the others. Noteworthy, the source code of HGJO is publicly available at https://github.com/Vang-z/HGJO.

An optimization based framework for region wise optimal clusters in MR images using hybrid objective

Swarm Intelligence based methods are amongst the highly efficient approaches for optimization in image clustering. Optimal clustering has been studied in many real-world applications, such as medical and aerial image segmentation. Region wise clustering is a class of challenges in image region segmentation. Uncertain convergence and high computational load are critical issues in the region-wise image clustering due to local optimum and the NP-hard cluster computation. Meta-heuristics approaches are efficient to achieve global optimum by including better search space exploration techniques. This paper develops a framework for cluster optimization by selecting the seeds in pathological medical resonance (MR) images using a variant of firefly optimization. The heuristics based method uses Gaussian random walk for convergence that occasionally results in local optima; therefore, we have investigated the firefly method with more search space exploration techniques and improved region-wise objective. Our framework applies the levy flights for exploration and compared with other search spaces like Cauchy, and Gaussian random walk. The intra-cluster and inter-cluster-based hybrid objective is converged swiftly. The framework has been compared with two of its variants and three other meta-heuristic-based methods, namely simulated annealing, PSO, and Cuckoo search. The MSE, PSNR, structural similarity(SSIM), and feature similarity(FSIM) based evaluation indices are measured for normal and abnormal MR images and listed in table-5, 6. Reported indices values for our frame work are better than existing methods. Figure-9, 10, 11 compares the stochastic search spaces among Levy flights, Cauchy random walk, Gaussian random walk and observed Levy flights as better search space. In section-3.5, The convergence for proposed framework is shown for multi-objective function against the single objective in two normal and abnormal images. Single objective converged from 188 to 119 and multi-objective converged from 196 to 117 for first image. For second image, the single objective converged from 168 to 94 and multi-objective converged from 183 to 93. Finally, we have illustrated the convergence criteria and computation complexity on publicly available MR data sets.

Unsupervised Cross-Scene Aerial Image Segmentation via Spectral Space Transferring and Pseudo-Label Revising

Unsupervised domain adaptation (UDA) is essential since manually labeling pixel-level annotations is consuming and expensive. Since the domain discrepancies have not been well solved, existing UDA approaches yield poor performance compared with supervised learning approaches. In this paper, we propose a novel sequential learning network (SLNet) for unsupervised cross-scene aerial image segmentation. The whole system is decoupled into two sequential parts—the image translation model and segmentation adaptation model. Specifically, we introduce the spectral space transferring (SST) approach to narrow the visual discrepancy. The high-frequency components between the source images and the translated images can be transferred in the Fourier spectral space for better preserving the important identity and fine-grained details. To further alleviate the distribution discrepancy, an efficient pseudo-label revising (PLR) approach was developed to guide pseudo-label learning via entropy minimization. Without additional parameters, the entropy map works as the adaptive threshold, constantly revising the pseudo labels for the target domain. Furthermore, numerous experiments for single-category and multi-category UDA segmentation demonstrate that our SLNet is the state-of-the-art.

Research on semantic segmentation of unmanned aerial vehicle visual image based on deep learning—take the outdoor environment of Anhui University of Finance & Economics as an example

In recent years, with the rapid development of unmanned aerial vehicle (UAV) technology, UAV processing visual information, especially image semantic segmentation technology, has developed rapidly. This paper proposes a semantic segmentation model, which has achieved high accuracy on CityScapes data set, and has been verified on the newly collected data set, and the verification results are in line with the actual situation.