Convolutional Autoencoder Neural Network Research Articles

Performance comparison analysis of classification methodologies for effective detection of intrusions

Intrusion detection systems (IDS) are critical in many applications, including cloud environments. The intrusion poses a security threat and extracts privacy data and information from the cloud. Additionally, intrusions can cause damage to system hardware, resulting in significant financial losses and exposing critical IT infrastructure to risk. To overcome these issues, this study employs the performance comparison analysis for IDS, which has been performed with different models like Autoencoder Convolutional neural network (AE+CNN), Random forest K-means clustering assisted deep neural network (RF+K-means+DNN), Autoencoder K-means clustering assisted long short term memory (AE+K-means+LSTM), Alexnet+Bi-GRU, AE+Alexnet+Bi-GRU and Wild horse AlexNet assisted Bi-directional Gated Recurrent Unit (WABi-GRU) models to choose the best methodology for effective detection of intrusions. The data needed for the analysis is collected from CICIDS2018, UNSW-NB15, NSL-KDD and ToN-IoT datasets. The collected data are pre-processed using data normalization and data cleaning. Finally, the best model has been chosen for effective intrusion detection, which is used for further processes. Various performances, such as accuracy, precision, recall, and f1-score, are analyzed for various existing and proposed models. From this performance comparison of six models such as AE+CNN, RF+K-means+DNN, AE+K-means+LSTM, Alexnet+Bi-GRU, AE+Alexnet+Bi-GRU and WABi-GRU. WABi-GRU can attain an accuracy of 99.890 % for multi-class classification in the CICIDS 2018 dataset, 99.7 % in the NSL-KDD dataset, 99.53 % in the UNSW-NB 15 dataset and 99.988 % for the ToN-IoT dataset. In this analysis, the models containing AlexNet Bi-GRU-based models can obtain better performances than other existing models. The WABi-GRU model can obtain better results than other models.

Multitask-Guided Deep Clustering With Boundary Adaptation.

Multitask learning uses external knowledge to improve internal clustering and single-task learning. Existing multitask learning algorithms mostly use shallow-level correlation to aid judgment, and the boundary factors on high-dimensional datasets often lead algorithms to poor performance. The initial parameters of these algorithms cause the border samples to fall into a local optimal solution. In this study, a multitask-guided deep clustering (DC) with boundary adaptation (MTDC-BA) based on a convolutional neural network autoencoder (CNN-AE) is proposed. In the first stage, dubbed multitask pretraining (M-train), we construct an autoencoder (AE) named CNN-AE using the DenseNet-like structure, which performs deep feature extraction and stores captured multitask knowledge into model parameters. In the second phase, the parameters of the M-train are shared for CNN-AE, and clustering results are obtained by deep features, which is termed as single-task fitting (S-fit). To eliminate the boundary effect, we use data augmentation and improved self-paced learning to construct the boundary adaptation. We integrate boundary adaptors into the M-train and S-fit stages appropriately. The interpretability of MTDC-BA is accomplished by data transformation. The model relies on the principle that features become important as the reconfiguration loss decreases. Experiments on a series of typical datasets confirm the performance of the proposed MTDC-BA. Compared with other traditional clustering methods, including single-task DC algorithms and the latest multitask clustering algorithms, our MTDC-BA achieves better clustering performance with higher computational efficiency. Deep features clustering results demonstrate the stability of MTDC-BA by visualization and convergence verification. Through the visualization experiment, we explain and analyze the whole model data input and the middle characteristic layer. Further understanding of the principle of MTDC-BA. Through additional experiments, we know that the proposed MTDC-BA is efficient in the use of multitask knowledge. Finally, we carry out sensitivity experiments on the hyper-parameters to verify their optimal performance.

A component diagnostic and prognostic framework for pump bearings based on deep learning with data augmentation

To support the mission of providing safe electricity generation with a high capacity factor, a Predictive Maintenance (PdM) framework using Machine Learning Models (MLM) to optimize component maintenance operations is developed. Using sensor measurements to better predict the true component’s Remaining Useful Life (RUL), the PdM framework has the potential to optimize maintenance costs by performing maintenance only when necessary. The PdM framework to pump bearings, the framework consists of a Convolutional Neural Network Autoencoder (CNN-AE) to detect component deviations from normality, a CNN to characterize component fault modes, and a Bayesian Neural Network (BNN) to estimate the component RUL with uncertainty. To increase the number of training samples, a synthetic data generation procedure was developed and includes procedures to recreate the fault-specific characteristic frequencies for diagnostics and the degradation trends for prognostics. The MLMs trained on the synthetic data are tested on the Center for Intelligent Maintenance Systems (IMS) dataset to showcase how well the synthetic data replicates measurement data. Utilizing this framework, the PdM was found to delay maintenance on average by a total of 8.92 years over 40 years and decrease the unexpected component failure rate from 10% to 0% when compared to traditional maintenance philosophies.

Fast fluid–structure interaction simulation method based on deep learning flow field modeling

The rapid acquisition of high-fidelity flow field information is of great significance for engineering applications such as multi-field coupling. Current research in flow field modeling predominantly focuses on low Reynolds numbers and periodic flows, exhibiting weak generalization capabilities and notable issues with temporal inferring error accumulation. Therefore, we establish a reduced order model (ROM) based on Convolutional Auto-Encoder (CAE) and Long Short-Term Memory (LSTM) neural network and propose an unsteady flow field modeling method for the airfoil with a high Reynolds number and strong nonlinear characteristics. The attention mechanism and weak physical constraints are integrated into the model architecture to improve the modeling accuracy. A broadband excitation training strategy is proposed to overcome the error accumulation problem of long-term inferring. With only a small amount of latent codes, the relative error of the flow field reconstructed by CAE can be less than 5‰. By training LSTM with broadband excitation signals, stable dynamic evolution can be achieved in the time dimension. CAE-LSTM can accurately predict the forced response and complex limit cycle behavior of the airfoil in a wide range of amplitude and frequency under subsonic/transonic conditions. The relative errors of predicted variables and integral force are less than 1%. The fluid–structure interaction framework is built by coupling the ROM and motion equations of the structure. CAE-LSTM predicts the time series response of pitch displacement and moment coefficient at different reduced frequencies, which is in good agreement with computational fluid dynamics, and the simulation time savings exceed one order of magnitude.

An intelligent fault detection and diagnosis monitoring system for reactor operational resilience: Unknown fault detection

Multiple advanced reactor designs envision deployment scenarios that feature reactor operations with significantly reduced operating staff compared to present reactors, or even feature fully autonomous operations. Since many of these new reactor designs, like small modular reactors (SMRs) and microreactors, also include extended fuel cycles which limit inspection intervals, load-following capabilities, and reduced access to critical components, safe and reliable semi- or fully autonomous operations must be enabled through an on-line monitoring (OLM) system that effectively detects and diagnoses any faults in the reactor. This paper presents the development of the Fault Detection Module of the previously proposed intelligent OLM system: the Fault Detection and Diagnosis Monitoring System (FDDMS). The Fault Detection Module monitors various plant parameters from 86 sensors throughout the nuclear power plant and accurately defines operations as nominal or abnormal. The methodology leverages the advantages of unsupervised learning and semi-supervised optimization data-driven techniques to successfully detect any fault in the system in real-time without fault information a priori. This allows the Fault Detection Module to identify any unknown fault that may affect the system. When integrated with the rest of the FDDMS architecture, the Fault Detection Module provides power transient dependent fault detection by using a separate model for steady state, ramping up in power, and ramping down in power operations. This makes the FDDMS especially applicable for load-following reactor systems. The Fault Detection Module architecture features a dimensionality reduction algorithm followed by an anomaly detection method. Three dimensionality reduction techniques: 1) Principal Component Analysis (PCA), 2) Deep Neural Network Autoencoder (DNNAE), and 3) Convolutional Neural Network Autoencoder (CNNAE); and three anomaly detection methods: 1) One-Class Support Vector Machine (OC-SVM), 2) Clustering (DBSCAN), and 3) Reconstruction Error (RecErr) thresholding were compared to find the best model for each power transient dataset. Precision rates, recall rates, F1-scores, and accuracies were used to compare model performance. The best models produced total accuracies of 94.50%, 93.63%, and 91.76% for the steady state, ramp up, and ramp down datasets, respectively, or 99.19%, 98.97%, and 97.09% when omitting 1%-severe fault cases. Specifically, the best performing models for the steady state, ramp up, and ramp down cases were the CNNAE-RecErr, DNNAE-DBSCAN, and CNNAE-RecErr, respectively. Recall rates were calculated for various fault types at multiple severity levels. The results show that the Fault Detection Module can identify any type of fault in the system with a delay of a few seconds on average. Additionally, robustness against noisy sensor data was tested, with models maintaining 100% accuracy at various levels of noise, and an illustrative real-time application of the methodology is provided.

Application of machine learning in determining and resolving state estimation anomalies in power systems

The state estimation (SE) process is one of the most important and efficient tools in achieving this goal. However, the occurrence of anomalies in the power grid, such as false data injection (FDI), can significantly impact the accuracy of the SE results. FDI results in reduced accuracy of SE results, potentially putting the power system in a critical situation. This paper addresses the limitations of simple residual-based algorithms in detecting FDI and proposes a scenario that utilizes machine learning (ML) models such as auto-encoder (AE), long short-term memory auto-encoder (LSTM AE), and 1-D convolutional neural network auto-encoder (1-D CNN AE) algorithms. The proposed method aims to offer a more effective FDI detection approach in the power grid, exhibiting higher detection accuracy. Also, the paper introduces the LSTM variational auto-encoder (LSTM VAE) algorithm for reconstructing anomalous data. By utilizing the LSTM VAE, anomalous data can be transformed to closely resemble the original data with an acceptable level of accuracy. Moreover, in critical situations involving FDI, the power system can maintain normal operation by employing the proposed method to reconstruct anomalous data. Finally, the performance of the presented methods is evaluated on the IEEE 14-bus, 30-bus, and 118-bus test networks. The results are then presented and discussed to demonstrate the effectiveness of the proposed method.

Design and implementation of an Automatic Deep Stacked Sparsely Connected Convolutional Autoencoder (ADSSCCA) neural network for remote sensing lithological mapping using calculated dropout

Design and implementation of an Automatic Deep Stacked Sparsely Connected Convolutional Autoencoder (ADSSCCA) neural network for remote sensing lithological mapping using calculated dropout

Direct Exoplanet Detection using Convolutional Image Reconstruction (ConStruct): A New Algorithm for Post-processing High-contrast Images

We present a novel machine-learning approach for detecting faint point sources in high-contrast adaptive optics (AO) imaging data sets. The most widely used algorithms for primary subtraction aim to decouple bright stellar speckle noise from planetary signatures by subtracting an approximation of the temporally evolving stellar noise from each frame in an imaging sequence. Our approach aims to improve the stellar noise approximation and increase the planet detection sensitivity by leveraging deep learning in a novel direct imaging post-processing algorithm. We show that a convolutional autoencoder neural network, trained on an extensive reference library of real imaging sequences, accurately reconstructs the stellar speckle noise at the location of a potential planet signal. This tool is used in a post-processing algorithm we call Direct Exoplanet Detection with Convolutional Image Reconstruction, or ConStruct. The reliability and sensitivity of ConStruct are assessed using real Keck/NIRC2 angular differential imaging data sets. Of the 30 unique point sources we examine, ConStruct yields a higher signal-to-noise ratio than traditional principal component analysis-based processing for 67% of the cases and improves the relative contrast by up to a factor of 2.6. This work demonstrates the value and potential of deep learning to take advantage of a diverse reference library of point-spread function realizations to improve direct imaging post-processing. ConStruct and its future improvements may be particularly useful as tools for post-processing high-contrast images from JWST and extreme AO instruments, both for the current generation and those being designed for the upcoming 30 m class telescopes.

A 3d convolutional neural network autoencoder for predicting solvent configuration changes in condensed phase biomass reactions

Self-supervised 3d CNN autoencoder combined with a distance-based classifier is trained on first principles simulations to predict the extent of solvent configuration changes during reactive chemical transformations.

SERS combined with the SAE-CNN model for estimating apple rootstocks under heavy metal copper stress

Copper-containing chemicals are commonly used in orchard management, and their use over a long time causes excessive copper content in orchard soil. Surface-enhanced Raman spectroscopy (SERS) technique combined with deep learning (DL) methods were used to characterize the response of apple rootstocks under heavy metal Cu stress. The hydroponic method prepared Apple rootstocks under five concentration gradients of heavy metal Cu stress as the experimental samples. The Raman spectral data were collected and subjected to spectral preprocessing. A stacked auto-encoder convolutional neural network (SAE-CNN) model was used to develop a rapid discriminative model for root, stem, and leaf stress levels of heavy metal Cu on apple rootstocks. The SAE-CNN models all outperformed the traditional models, with performance metrics above 99%. The results showed that the proposed SAE-CNN model could rapidly discriminate heavy metal Cu in apple rootstocks.

Performance Comparison Analysis of Classification Methodologies for Effective Detection of Intrusions

Intrusion detection systems (IDS) are critical in many applications, including cloud environments. The intrusion poses a security threat and extracts privacy data and information from the cloud. The user has an Internet function that allows him to store personal information in the cloud environment. The cloud can be affected by various issues such as data loss, data breaches, lower security and lack of privacy due to some intruders. A single intrusion incident can result in data within computer and network systems being quickly stolen or deleted. Additionally, intrusions can cause damage to system hardware, resulting in significant financial losses and exposing critical IT infrastructure to risk. To overcome these issues, the study employs the performance comparison analysis of Autoencoder Convolutional neural network (AE+CNN), Random K-means clustering assisted deep neural network (RF+K-means+DNN), Autoencoder K-means clustering assisted long short term memory (AE+K-means+LSTM), Alexnet+Bi-GRU, AE+Alexnet+Bi-GRU and Wild horse AlexNet assisted Bi-directional Gated Recurrent Unit (WABi-GRU) models to choose the best methodology for effective detection of intrusions. The data needed for the analysis is collected from CICIDS2018, UNSW-NB15 and NSL-KDD datasets. The collected data are pre-processed using data normalization and data cleaning. Finally, through this research, the best model suitable for effective intrusion detection can be identified and used for further processes. The proposed models, such as RF+K-means+DNN, AE+K-Means+LSTM, AlexNet Bi-GRU, AE+Alexnet+Bi-GRU and WABi-GRU can obtain an accuracy of 99.278%, 99.33%, 99.45%, 99.50%, 99.65% for the CICIDS dataset 2018 for binary classification. In multi-class classification, the AlexNet Bi-GRU, AE+Alexnet+Bi-GRU and WABi-GRU can attain accuracy of 99.819%, 99.852% and 99.890%. In NSL-KDD, the AlexNet Bi-GRU, AE+Alexnet+Bi-GRU and WABi-GRU achieve accuracy of 99.34%, 99.546% and 99.7%. In UNSW-NB 15 dataset, AlexNet Bi-GRU, AE+Alexnet+Bi-GRU and WABi-GRU achieve accuracy of 99.313%, 99.399% and 99.53%. AlexNet Bi-GRU-based models can obtain better performances than other existing models.

Denoising convolutional autoencoder for improving the classification performance based on noisy galaxy images

Images of galaxy objects are of great importance to the work of astronomers. Nowadays, the task of galaxy classification has been aided by Neural-Network-based classification models, who are powerful yet vulnerable to the attack of noisy images. In this research, RGB noises and bright spots simulating stars were generated, and a Convolutional-Neural-Network (CNN) based lightweight denoising image autoencoder was proposed. Firstly, a benchmark CNN classifier using DenseNet structure was trained on the Galaxy 10 DECals dataset, which consists of over 17, 000 RGB color galaxy images. Then, noisy images were generated by adding bright spots of different size and color simulating stars and applying gaussian RGB noises over the original images. The CNN autoencoder that consists of Convolutional layer in its encoder and Convolution Transpose layers in decoder was trained on the raw and noisy training data to learn effective galaxy image denoising. Finally, the effect of the autoencoder was evaluated by contrasting the performance of the CNN classifier over the noisy and denoised images. In contrast with being evaluated on the raw testing set, the CNN classifiers accuracy dropped by 0.41 when tested on the generated noisy testing images, indicating the effectiveness of the attack of image noises. While after denoising with the proposed autoencoder, the classifiers accuracy increased significantly by 0.37. Output denoised images also suggest that the autoencoder can effectively remove the applied bright spot and gaussian RGB noise, recreating the original shape of the galaxy.

Make It Less Complex: Autoencoder for Speckle Noise Removal-Application to Breast and Lung Ultrasound.

Ultrasound (US) imaging is used in the diagnosis and monitoring of COVID-19 and breast cancer. The presence of Speckle Noise (SN) is a downside to its usage since it decreases lesion conspicuity. Filters can be used to remove SN, but they involve time-consuming computation and parameter tuning. Several researchers have been developing complex Deep Learning (DL) models (150,000-500,000 parameters) for the removal of simulated added SN, without focusing on the real-world application of removing naturally occurring SN from original US images. Here, a simpler (<30,000 parameters) Convolutional Neural Network Autoencoder (CNN-AE) to remove SN from US images of the breast and lung is proposed. In order to do so, simulated SN was added to such US images, considering four different noise levels (σ = 0.05, 0.1, 0.2, 0.5). The original US images (N = 1227, breast + lung) were given as targets, while the noised US images served as the input. The Structural Similarity Index Measure (SSIM) and Peak Signal-to-Noise Ratio (PSNR) were used to compare the output of the CNN-AE and of the Median and Lee filters with the original US images. The CNN-AE outperformed the use of these classic filters for every noise level. To see how well the model removed naturally occurring SN from the original US images and to test its real-world applicability, a CNN model that differentiates malignant from benign breast lesions was developed. Several inputs were used to train the model (original, CNN-AE denoised, filter denoised, and noised US images). The use of the original US images resulted in the highest Matthews Correlation Coefficient (MCC) and accuracy values, while for sensitivity and negative predicted values, the CNN-AE-denoised US images (for higher σ values) achieved the best results. Our results demonstrate that the application of a simpler DL model for SN removal results in fewer misclassifications of malignant breast lesions in comparison to the use of original US images and the application of the Median filter. This shows that the use of a less-complex model and the focus on clinical practice applicability are relevant and should be considered in future studies.

Deep reference autoencoder convolutional neural network for damage identification in parallel steel wire cables

This paper addresses the challenge of overcoming the limited availability of training samples for the damage state of parallel steel wire cables in practical engineering and the difficulty of detecting minor damages. To tackle this issue, we propose an unsupervised deep learning damage identification technique called Deep Reference Autoencoder Convolutional Neural Network (DRACNN) for analyzing the damage state of parallel steel wire cables in bridge engineering. The DRACNN method utilizes multi-dimensional cross-correlation function (CCF) derived from acceleration signals at various health stages as input to train the network structure and obtain optimal parameters. Subsequently, we analyze the layer decomposition to identify neurons in the lowest hidden layer indicating damage. The neuronal change information is then extracted using an Exponentially Weighted Moving Average (EWMA) Control Chart to determine the damage state of the structure. Finally, we present a comprehensive numerical analysis describing the method's flow and network architecture and demonstrate the feasibility of this approach through experiments.

Pavement crack detection through a deep-learned asymmetric encoder-decoder convolutional neural network

ABSTRACT Crack detection on roads’ surfaces is an important issue in pavement management, as it provides an indication of the quality of the road and its deterioration over time. Pavement cracks are one of the most common types of damage observed on roads, and they can be seen visually. Despite the fact that it does not provide immediate resolution to the issue, understanding the extent of crack damage is essential for the upkeep of roads. This paper presents a novel approach to automatically detecting pavement cracks using the orthoimage generated by a consumer-grade photogrammetric Unmanned Aerial Vehicle (UAV) and a deep learning algorithm. We used an autoencoder Convolutional Neural Network (CNN) to train a dataset full of challenging factors such as road lines and marks, oil and colour spots, and water stains. The model was tested on a dataset of RGB patches of different patterns of cracks and achieved an overall accuracy (OA) and F1 score of about 0.98. The results demonstrate the effectiveness of the proposed method in accurately detecting pavement cracks in challenging real-world conditions. This approach provides an efficient and cost-effective solution for pavement crack detection, that can be used for measuring the road's quality and monitoring it.

A data-driven machine learning approach for yaw control applications of wind farms

This study proposes a cost-effective machine-learning based model for predicting velocity and turbulence kinetic energy fields in the wake of wind turbines for yaw control applications. The model consists of an auto-encoder convolutional neural network (ACNN) trained to extract the features of turbine wakes using instantaneous data from large-eddy simulation (LES). The proposed framework is demonstrated by applying it to the Sandia National Laboratory Scaled Wind Farm Technology facility consisting of three 225 kW turbines. LES of this site is performed for different wind speeds and yaw angles to generate datasets for training and validating the proposed ACNN. It is shown that the ACNN accurately predicts turbine wake characteristics for cases with turbine yaw angle and wind speed that were not part of the training process. Specifically, the ACNN is shown to reproduce the wake redirection of the upstream turbine and the secondary wake steering of the downstream turbine accurately. Compared to the brute-force LES, the ACNN developed herein is shown to reduce the overall computational cost required to obtain the steady state first and second-order statistics of the wind farm by about 85%.

Nonlinear mode decomposition via physics-assimilated convolutional autoencoder for unsteady flows over an airfoil

I have developed a physics-assimilated convolutional autoencoder (CAE) neural network, namely, PhyAENet, to carry out nonlinear mode decomposition of the unsteady flow field around a National Advisory Committee for Aeronautics 4412 airfoil near stall conditions. The flow field snapshots are mapped into latent modes by the encoder part of the well-trained CAE, which are used for dynamic mode decomposition (DMD) analysis. The computed DMD modes are split into modes covering different frequency ranges. These high and low-frequency DMD modes are used to form reconstructed encoded sequences, which are then mapped back to generate the nonlinear decomposed spatiotemporal modes by the decoder of the CAE. As such, physics is assimilated into the neural network by incorporating the frequencies of the DMD modes into the latent modes in the latent space. The proposed PhyAENet is capable of extracting the dominant features of the flow fields, accounting for the nonlinearity of the underlying dynamics. Furthermore, the extracted nonlinear modes are evolving with time and physically interpretable. It is revealed that the nonlinear modes can be well represented when using more DMD modes for reconstruction of the encoded sequences. The energy spectrum of the nonlinear modes are obtained by ranking the Frobenius norm of the mode vector.

A graphics-accelerated deep neural network approach for turbomachinery flows based on large eddy simulation

In turbomachinery, strongly unsteady rotor–stator interaction triggers complex three-dimensional turbulent flow phenomena such as flow separation and vortex dynamics. Large eddy simulation (LES) is an advanced numerical method that has recently been used to resolve large-scale turbulent motions and model subgrid-scale turbulence in turbomachinery. To largely reduce the computing cost of LES for turbomachinery flow, a graphics processing unit (GPU)-accelerated deep neural network-based flow field prediction approach is explored, which combines convolutional neural network autoencoder (CNN-AE) with long short-term memory (LSTM). CNN-AE extracts spatial features of turbomachinery flow by mapping high-dimensional flow fields into low-dimensional space, while LSTM is used to predict the temporal evolution of fluid dynamics. Automatic mixed precision (AMP) is employed to achieve rapid neural network training using Nvidia GTX 1080 Ti GPU, which shows a significant speedup compared with that without AMP. We evaluated the proposed CNN-AE-LSTM (CAL) method against gated recurrent units (GRU) and simple recurrent network (SRN) on two types of turbomachinery, i.e., centrifugal and axial flow pumps. The results show that the proposed CAL shows better capability of capturing the vortex structure details of turbomachinery. When predicting the temporal vorticity field, the mean square error of CAL results is 0.105%–0.124% for centrifugal pumps and 0.071%–0.072% for axial flow pumps. Meanwhile, the structural similarity index measure of the CAL results is 92.51%–92.77% for centrifugal pumps and 93.81%–94.61% for axial flow pumps. The proposed CAL is noticeably better than GRU and SRN in terms of both mean square error and structural similarity index measure.

Denoising gravitational-wave signals from binary black holes with a dilated convolutional autoencoder

Abstract The broadband frequency output of gravitational-wave (GW) detectors is a non-stationary and non-Gaussian time series data stream dominated by noise populated by local disturbances and transient artifacts, which evolve on the same timescale as the GW signals and may corrupt the astrophysical information. We study a denoising algorithm dedicated to expose the astrophysical signals by employing a convolutional neural network in the encoder-decoder configuration, i.e. apply the denoising procedure of coalescing binary black hole signals to the publicly available LIGO O1 time series strain data. The denoising convolutional autoencoder neural network is trained on a dataset of simulated astrophysical signals injected into the real detector’s noise and a dataset of detector noise artifacts (‘glitches’), and its fidelity is tested on real GW events from O1 and O2 LIGO-Virgo observing runs.

Spatiotemporal dynamics of population density in China using nighttime light and geographic weighted regression method*

ABSTRACT The distribution and dynamic changes of regional or national population data with long time series are very important for regional planning, resource allocation, government decision-making, disaster assessment, ecological protection, and other sustainability research. However, the existing population datasets such as LandScan and WorldPop all provide data from 2000 with limited time series, while GHS-POP only utilizes land use data with limited accuracy. In view of the limited remote sensing images of long time series, it is necessary to combine existing multi-source remote sensing data for population spatialization research. In this research, we developed a nighttime light desaturation index (NTLDI). Through the cross-sensor calibration model based on an autoencoder convolutional neural network, the NTLDI was calibrated with the same period Visible Infrared Imaging Radiometer Suite Day/Night Band (VIIRS-DNB) data. Then, the geographically weighted regression method is used to determine the population density of China from 1990 to 2020 based on the long time series NTL. Furthermore, the change characteristics and the driving factors of China’s population spatial distribution are analyzed. The large-scale, long-term population spatialization results obtained in this study are of great significance in government planning and decision-making, disaster assessment, resource allocation, and other aspects.