Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Earth observation (EO) data are critical for monitoring the state of planet Earth and can be helpful for various real-world applications [1]. Although numerous benchmark datasets have been released, there is no unified platform for developing and fairly comparing deep learning models on EO data [2]. For deep learning methods, the backbone networks, hyper-parameters, and training details are influential factors while comparing the performances.. However, existing works usually neglect these details and even evaluate the performance with different training/validation/test dataset splits. This makes it difficult to fairly and reliably compare different algorithms. In this study, we introduce the EarthNets platform, an open deep-learning platform for remote sensing and Earth observation. The platform is based on PyTorch [3] and TorchData. There are about ten different libraries, covering different tasks in remote sensing. Among them, Dataset4EO is designed as a standard and easy-to-use data-loading library, which can be used alone or together with other high-level libraries like RSI-Classification (for image classification), RSI-Detection (for object detection), RSI-Segmentation (for semantic segmentation), and so on. Two factors are considered for the design of the EarthNets platform: the first one is the decoupling between dataset loading and high-level EO tasks. As there are more than 400 RS datasets with different data modalities, research domains, and download links, efficient preparation of analysis-ready data can largely accelerate the research for the whole community. The other factor is to bring advances in machine learning to EO by providing new deep-learning models. The EarthNets platform provides a fair and consistent evaluation of deep learning methods on remote sensing and Earth observation data [4]. It also helps bring together the remote sensing and a larger machine-learning community. The platform, dataset collections are publicly available at https://earthnets.github.io.[1] Zhu, Xiao Xiang, et al. "Deep learning in remote sensing: A comprehensive review and list of resources." IEEE Geoscience and Remote Sensing Magazine 5.4 (2017): 8-36.[2]Long, Yang, et al. "On creating benchmark dataset for aerial image interpretation: Reviews, guidances, and million-aid." IEEE Journal of selected topics in applied earth observations and remote sensing 14 (2021): 4205-4230.[3] Paszke, Adam, et al. "Pytorch: An imperative style, high-performance deep learning library." Advances in neural information processing systems 32 (2019).[4] Xiong, Zhitong, et al. "EarthNets: Empowering AI in Earth observation." arXiv preprint arXiv:2210.04936 (2022).

Comprehensive Study for Breast Cancer Using Deep Learning and Traditional Machine Learning

Deep learning is a deep field of neural networks, and its application in remote sensing image classification and recognition processing has attracted attention and discussion from all walks of life. This paper first briefly introduces the traditional remote sensing image processing methods and the limitations of these algorithms and emphasizes the limitations of these techniques. Then, the research status of target recognition and change detection in remote sensing images based on deep learning is discussed, and how to select and design appropriate deep learning models. And then, the datasets of two different provinces were selected for comparative experiments, and the implementation process of target recognition and change detection in remote sensing images was described in detail. Finally, based on the experimental results, the future trend of deep learning application in remote sensing identification and classification is prospected.

Machine Learning and Deep Learning in Remote Sensing Data Analysis

Central to the looming paradigm shift toward data-intensive science, machine-learning techniques are becoming increasingly important. In particular, deep learning has proven to be both a major breakthrough and an extremely powerful tool in many fields. Shall we embrace deep learning as the key to everything? Or should we resist a black-box solution? These are controversial issues within the remote-sensing community. In this article, we analyze the challenges of using deep learning for remote-sensing data analysis, review recent advances, and provide resources we hope will make deep learning in remote sensing seem ridiculously simple. More importantly, we encourage remote-sensing scientists to bring their expertise into deep learning and use it as an implicit general model to tackle unprecedented, large-scale, influential challenges, such as climate change and urbanization.

A survey of machine learning and deep learning in remote sensing of geological environment: Challenges, advances, and opportunities

Detecting changes in land use and land cover (LULC) from space has long been the main goal of satellite remote sensing (RS), yet the existing and available algorithms for cloud classification are not reliable enough to attain this goal in an automated fashion. Clouds are very strong optical signals that dominate the results of change detection if they are not removed completely from imagery. As various architectures of deep learning (DL) have been proposed and advanced quickly, their potential in perceptual tasks has been widely accepted and successfully applied to many fields. A comprehensive survey of DL in RS has been reviewed, and the RS community has been suggested to be leading researchers in DL. Based on deep residual learning, semantic image segmentation, and the concept of atrous convolution, we propose a new DL architecture, named CloudNet, with an enhanced capability of feature extraction for classifying cloud and haze from Sentinel-2 imagery, with the intention of supporting automatic change detection in LULC. To ensure the quality of the training dataset, scene classification maps of Taiwan processed by Sen2cor were visually examined and edited, resulting in a total of 12,769 sub-images with a standard size of 224 × 224 pixels, cut from the Sen2cor-corrected images and compiled in a trainset. The data augmentation technique enabled CloudNet to have stable cirrus identification capability without extensive training data. Compared to the traditional method and other DL methods, CloudNet had higher accuracy in cloud and haze classification, as well as better performance in cirrus cloud recognition. CloudNet will be incorporated into the Open Access Satellite Image Service to facilitate change detection by using Sentinel-2 imagery on a regular and automatic basis.

This study is conducted in accordance with a systematic literature review (SLR) protocol. SLR is tasked with finding publications, publishers, deep learning types, enhanced and adapted deep learning techniques, impacts, proactive approaches, key parameters, and applications in the field of remote sensing. It is also expected to identify current research directions, gaps, and unsolved issues in order to provide understanding and recommendations for future studies. The data is collected from important research papers published in recognized journals between the years 2015 and 2021, however, conference/seminar proceedings and other online resources are excluded to minimize unnecessary complications. Based on previously established exclusion, inclusion, and quality parameter criteria, a total of 122 primary studies are considered. The literature review overcomes a number of significant problems, including key variables taken into account by researchers in the remote sensing (RS) domain, various deep learning (DL) solutions proposed for RS analysis, various proactive strategies recommended in the literature to reduce risks linked to the RS domain, and various DL applications reported in the remote sensing domain. The results show that there is still a lack of structured information that enables DL to be employed for crucial applications in the field of remote sensing, despite substantial research and development of numerous DL algorithms. Furthermore, it is evident that DL approaches in the remote sensing domain have not been thoroughly exploited, thus demanding further research. The findings suggest that deep learning techniques need further investigations and the development of an authentic mechanism is essential for accurate results retrieved from remote sensing data. The proposed study would let scientists examine previous investigations into deep learning methods, which can then be utilized as support for further investigations.

A comprehensive review on detection of plant disease using machine learning and deep learning approaches

Cardiovascular diseases like arrhythmia are a significant health concern worldwide, affecting both elderly and young population due to lifestlye changes. Early diagnosis of cardiac arrhythmia using Electrocardiogram (ECG) by trained cardiologists is vital to prevent heart ailments and save lives. With the growth of wearable and standard ECG monitoring devices and a dearth of qualified cardiologists required to analyse the vast amounts of data collected, automated arrhythmia detection by Machine Learning (ML) and Deep Learning (DL) techniques have become very popular in recent years. In this study, we have reviewed the literature and described standard ML and DL studies in ECG arrhythmia classification. While ML techniques do demonstrate very good metrics, ML classifiers like SVM, knearest-neighbours, Decision Trees, etc. need preprocessing and hand-crafted feature extraction. DL methods which use networks like Convolutional Neural Networks (CNN), Long-Short-Term-Memory (LSTM) do not need any feature extraction as they automatically learn the features by themselves. Recent studies in DL have demonstrated very high performance metrics without the need for feature extraction. While some DL techniques do need noise filtering and determination of other features like the QRS complex, many of them can work with raw ECG signals and hence are ideally suited over their ML counterparts for real time ECG classification. DL networks can also be used as feature extractors and combined with ML classifiers. We thus conclude that state-of-the-art DL methods offer inherent advantages and flexibility over ML methods for automated arrhythmia classification. This review aggregates the niche features of leading ML and DL studies in this field which interested researchers can benefit from.

Deep learning (DL) has increasingly witnessed a lot of applications and advancements in remote sensing (RS). However, it remains unclear whether it can accurately detect historical buildings in RS imagery. Here we proposed a new deep transfer learning approach based on aerial photographs to automatically detect Hakka Weilong Houses (HWHs), a famous type of historical residence and an important cultural symbol of Hakka, a Han Chinese subgroup across the world. An RS image dataset, namely Hakka Weilong House Image Dataset (HWHID), was created by using aerial photographs of the urban and suburban Meizhou, which is called the world Hakka capital. The dataset was randomly shuffled into training and testing ones with a ratio of 8:2. Our approach used ResNet50 as the backbone transfer network and YOLO v2 as a training framework. Experimental results showed that the average precision was $0.9599\pm 0.0150$ , the loss rate was 0.0250, the Root Mean Square Error (RMSE) for training was 0.1580, and the average detecting time per image clip was $0.0383\pm 0.0150$ second, suggesting that our model has a high accuracy and an excellent performance for the HWH detection task. Our findings provide concrete evidences that aerial-imagery-based deep transfer learning can be used as a new archaeological RS method to detect historical buildings accurately and rapidly in aerial photographs.

Deep learning-based land cover and land use (LCLU) classification systems are a significant aspiration for remote sensing communities. In nature, remote sensing images have various properties that need to be analyzed. Analyzing and interpreting image properties is difficult due to the nature of the image, the sensor technology’s capability, and other determinant variables such as seasons and weather conditions. The problem is essential for environmental monitoring, agricultural decision-making, and urban planning if it can be supported by deep learning systems. Therefore, deep learning approaches are proposed to quickly analyze and interpret the remote sensing image to classify the LCLU. The deep learning methods could be designed starting from scratch or using pre-trained networks. However, there are few comparisons of deep learning methods developed from scratch and trained on pre-trained networks. Thus, we proposed evaluating and comparing the deep learning models convolutional neural network feature extractor (CNN-FE) by developing it from scratch, transfer learning, and fine-tuning it for the LCLU classification system using remote sensed images. Using CNN-FE, TL, and fine-tuning deep learning models as examples, this paper compares and analyzes deep learning algorithms for remote sensed image classification. After developing and training each deep learning model on the UCM dataset, we evaluated and compared their performances using the performance measurement metrics accuracy, precision, recall, f1-score, and confusion matrix. The proposed deep learning algorithms can adapt and learn the features of the remote sensing images, and the TL and fine-tuning classification performances are significantly improved. As a result of the efficient time used for training the models, this paper discovered that the fine-tuned deep learning model achieved profound accuracy performance results in the UCM dataset.

Remote sensing is used in an increasingly wide range of applications. Models and methodologies based on artificial intelligence (AI) are commonly used to increase the performance of remote sensing technologies. Deep learning (DL) models are the most widely researched AI-based models because of their effectiveness and high performance. Therefore, we organized a Special Issue on remote sensing titled “Artificial Intelligence and Machine Learning Applications in Remote Sensing.” In this paper, we review nine articles included in this Special Issue, most of which report studies based on satellite data and DL, reflecting the most prevalent trends in remote sensing research, as well as how DL architecture and the functioning of DL models can be analyzed and explained is a hot topic in AI research. DL methods can outperform conventional machine learning methods in remote sensing; however, DL remains a black box and understanding the details of the mechanisms through which DL models make decisions is difficult. Therefore, researchers must continue to investigate how explainable DL methods for use in the field of remote sensing can be developed.

In recent years, deep learning has witnessed astonishing success in the field of remote sensing in images. Generally, deep learning requires a large amount of labeled training data. Nevertheless, in remote sensing, sufficient labeled data are scarce because labeled data are often difficult, expensive, or time-consuming to obtain. To address these problems, we propose a deep curriculum learning semi-supervised framework (DCLSSF) for remote sensing image scene classification. This framework employs a multimodal deep curriculum learning method which can realize the classification of images on a range of easy–difficult. Specifically, by utilizing multiple pretrained networks to extract multiple deep features of images as their multimodal feature representations, it can comprehensively mine the information from labeled and unlabeled images from diverse perspectives. Subsequently, a feature fusion method is used on deep features of different modalities to obtain deep fusion features with a strong discrimination ability and low dimensionality. Finally, the multimodal deep features are fed into multimodal curriculum learning methods for classification. Multimodal curriculum learning can integrate the easy curricula recommended by each modal according to the order of the samples of each modal and then learn step by step. Experiments on three publicly available datasets (UC Merced, AID, and NWPU-RESISC45) show that the semi-supervised classification framework achieves high accuracy rates (99.14%, 97.95%, and 93.01%), even surpassing those of the most supervised classification methods. The DCLSSF method can not only fully exploit the rich features extracted by the multimodal deep learning network but can also perform the semi-supervised classification of unlabeled samples in a range of easy–difficult.

AN ASSESSMENT OF SPATIAL VARIATION OF LAND SURFACE CHARACTERISTICS OF MINNA, NIGER STATE NIGERIA FOR SUSTAINABLE URBANIZATION USING GEOSPATIAL TECHNIQUES