Parallel Feature Extraction Research Articles

Hybrid deep models for parallel feature extraction and enhanced emotion state classification

Emotions play a vital role in recognizing a person’s thoughts and vary significantly with stress levels. Emotion and stress classification have gained considerable attention in robotics and artificial intelligence applications. While numerous methods based on machine learning techniques provide average classification performance, recent deep learning approaches offer enhanced results. This research presents a hybrid deep learning model that extracts features using AlexNet and DenseNet models, followed by feature fusion and dimensionality reduction via Principal Component Analysis (PCA). The reduced features are then classified using a multi-class Support Vector Machine (SVM) to categorize different types of emotions. The proposed model was evaluated using the DEAP and EEG Brainwave datasets, both well-suited for emotion analysis due to their comprehensive EEG signal recordings and diverse emotional stimuli. The DEAP dataset includes EEG signals from 32 participants who watched 40 one-minute music videos, while the EEG Brainwave dataset categorizes emotions into positive, negative, and neutral based on EEG recordings from participants exposed to six different film clips. The proposed model achieved an accuracy of 95.54% and 97.26% for valence and arousal categories in the DEAP dataset, respectively, and 98.42% for the EEG Brainwave dataset. These results significantly outperform existing methods, demonstrating the model’s superior performance in terms of precision, recall, F1-score, specificity, and Mathew correlation coefficient. The integration of AlexNet and DenseNet, combined with PCA and multi-class SVM, makes this approach particularly effective for capturing the intricate patterns in EEG data, highlighting its potential for applications in human-computer interaction and mental health monitoring, marking a significant advancement over traditional methods.

Multimodal Spatio-Temporal Framework for Real-World Affect Recognition

Deep learning models show great potential in applications involving video-based affect recognition, including human-computer interaction, robotic interfaces, stress and depression assessment, and Alzheimer's disease detection. The low complex Multimodal Diverse Spatio-Temporal Network (MDSTN) has been analysed to effectively capture spatio-temporal information from audio-visual modalities for affect recognition using the Acted Facial Expressions in the Wild (AFEW) dataset. The scarcity of data is handled by data augmented parallel feature extraction for visual network. Visual features extracted by carefully reviewing and customizing Convolutional 3D architecture over different ranges are combined to train a neural network for classification. Multi-resolution Cochleagram (MRCG) features from speech, along with spectral and prosodic audio features, are processed by a supervised classifier. The late fusion technique is explored to integrate audio and video modalities, considering their processing over different temporal spans. The MDSTN approach significantly boosts the accuracy of basic emotion recognition to 71.54% on the AFEW dataset. It demonstrates exceptional proficiency in identifying emotions such as disgust and surprise, thus exceeding current benchmarks in real-world affect recognition.

Spatio-temporal fusion with motion masks for the moving small target detection from remote-sensing videos

Moving small target detection is an eye-catching research topic, full of great challenges in object detection field. Its primary difficulties exist in (i) the very small size proportion of the target region in background images, (ii) the quite weak contrast to background and (iii) the perception of slight target motion. Currently-existing detection methods mainly utilize the image features from spatial domain. This kind of features is quite weak to small targets. More new feature kinds need to be concerned. In view of these, besides traditional image clues, motion features are currently attracting more and more attention in small target detection. In order to promote detection performance, this paper proposes a Spatio-Temporal Fusion Network (STFNet) with two parallel feature extraction branches for detecting moving small targets. In this network, one branch is designed to capture the traditional semantic features from spatial domain, and the other is devised to perceive the slight target motion features hidden in temporal domain. Meanwhile, to optimize the extraction of spatio-temporal features, a group of motion masks is specially designed to guide feature extractors to pay more precise attention to the positions where small targets appear. Moreover, we design a new bridging module to enhance the cross-domain and cross-scale fusion of spatio-temporal features. Through extensive comparison and ablation experiments on seven sub-datasets, it is proved that our new STFNet is effective and it is obviously superior to the compared ones in detecting moving small targets from satellite sequence images. It achieves a Precision of 0.93, an mAP50 of 71.10% and an F1 of 0.84, evidently higher than existing state-of-the-art methods. Our codes are available at https://github.com/UESTC-nnLab/STF.

Human Muscle sEMG Signal and Gesture Recognition Technology Based on Multi-Stream Feature Fusion Network

Surface electromyography signals have significant value in gesture recognition due to their ability to reflect muscle activity in real time. However, existing gesture recognition technologies have not fully utilized surface electromyography signals, resulting in unsatisfactory recognition results. To this end, firstly, a Butterworth filter was adopted to remove high-frequency noise from the signal. A combined method of moving translation threshold was introduced to extract effective signals. Then, a gesture recognition model based on multi-stream feature fusion network was constructed. Feature extraction and fusion were carried out through multiple parallel feature extraction paths, combined with convolutional neural networks and residual attention mechanisms. Compared to popular methods of the same type, this new recognition method had the highest recognition accuracy of 92.1% and the lowest recognition error of 5%. Its recognition time for a single-gesture image was as short as 4s, with a maximum Kappa coefficient of 0.92. Therefore, this method combining multi-stream feature fusion networks can effectively improve the recognition accuracy and robustness of gestures and has high practical value.

A novel motherboard test item yield prediction model based on parallel feature extraction

Abstract Functional testing of motherboards in Surface Mount Technology (SMT) assembly lines is crucial. Accurate yield prediction for each test item optimizes testing strategies, reduces costs, and ensures test coverage. Manual estimation of test item yields remains common, hindering accurate on-site predictions. Existing research on motherboard yield lacks predictions for individual test items and ignores temporal correlations during placement. This paper introduces a method, a convolutional bidirectional long short-term memory attention network (CBA-Net), which combines a convolutional neural network and a bidirectional long short-term memory network with an attention mechanism for parallel processing. It preprocesses historical test data, leveraging both networks to identify key features and extract temporal correlations. The attention mechanism optimizes yield predictions by assigning weights to information at different time steps. Experimental validation using actual production data demonstrates that the proposed method performs better compared to traditional models.

TMP: Temporal Motion Perception with spatial auxiliary enhancement for moving Infrared dim-small target detection

Infrared dim and small target detection has significant research value and application prospects in both military affairs and civil safety scenes. This topic presents substantial challenges due to the small size of the targets and their low contrast against the background in images. In the past ten years, most detection methodologies have primarily focused on single-frame images, mainly relying on spatial-domain image features to identify dim and small targets, with limited consideration of temporal-domain motion features. However, unlike general objects, the image features of dim-small targets are often easy to be interfered by background, and even seriously lost in capturing stages. These deficiencies can be the primary factors to impact the detection performance of single-frame methods. To promote infrared dim-small target detection, this paper proposes a novel multi-frame pipeline called Temporal Motion Perception with spatial auxiliary enhancement (TMP), which consists of two parallel feature extraction branches. One auxiliary branch is designed to capture traditional image features from spatial domain, and the other is proposed to extract the motion features of targets from temporal domain. In addition, a new Complementary Symmetry Weighting module is specially designed to fulfill the cross-domain fusion of spatio-temporal features, further enhancing feature representation. Experimental results on three public datasets demonstrate the efficiency of our proposed scheme. Compared with existing methods, it could often refresh the state-of-the-art (SOTA) performance on most metrics. Our codes are available at https://github.com/UESTC-nnLab/TMP.

Specific emitter identification by wavelet residual network based on attention mechanism

AbstractSpecific emitter identification technology plays a very important role in spectrum resource management, wireless network security, cognitive radio etc. However, in complex electromagnetic environments, the variability and uncertainty of signals make it very difficult to extract representative feature representations of the signals. At the same time, the feature extraction capability of the recognition model is also a factor that needs to be considered. To address these issues, a wavelet residual neural network model based on attention mechanism is proposed for specific emitter identification. First, multi‐level wavelet decomposition is performed on all received signals to obtain their wavelet detail coefficients at different scales. Next, all the wavelet detail coefficients are used as the feature input for the attention‐based residual network, and perform parallel feature extraction at multi scales. Finally, the feature representation capability of all coefficients are compared, and the model's recognition results based on it are obtained. The recognition rates on the three datasets are 94.7%, 93.21%, and 86.1%, respectively, all of which are superior to recent state‐of‐the‐art algorithms. In addition, through ablation experiment, the validity of each component of the model has been verified.

A common feature-driven prediction model for multivariate time series data

Multivariate time series data contain a variety of common features that are difficult to extract, among which the sudden irregular fluctuation trend, the trend feature of large fluctuation amplitude, and the long-term dependency relationship in the time series have an important impact on the accuracy of the prediction model. To predict non-stationary trends in large-scale data accurately using the common characteristics of multivariate time series. A novel generalised forecasting model NeA3L based on common features of time series is developed. The NeA3L model utilizes multiple independent parallel feature extraction modules to obtain the common features of multivariate time series. It utilizes the three-layer iterative structure to deal with sudden irregular fluctuation patterns. NeA3L optimizes the network structure to realize the heterogeneous codec structure. It applies the attention mechanism between the encoder and the decoder to accomplish the multivariate prediction of the multivariate time series, which has good stability for predicting various types of multivariate data. A comparison of the NeA3L model with nine current time series prediction methods on four publicly available datasets shows that the NeA3L model outperforms the current methods in various evaluation metrics. The RMSE is improved by 2.3 %, 26.5 %, 28.9 %, and 20.84 % on average, respectively. The NeA3L model can be universally applicable to the field of multivariate time series prediction, which is important for optimizing the intelligent management and decision-making.

CAC-YOLOv8: real-time bearing defect detection based on channel attenuation and expanded receptive field strategy

Bearing defect detection plays a crucial role in the intelligent production of chemical transmission equipment, where timely identification and handling of defective bearings are essential. However, in practical large-scale industrial production, product surface defects are often complex, diverse, and exhibit significant variations in appearance, posing severe challenges to the discriminative ability and detection efficiency of bearing defect detection algorithms. This paper proposes a real-time bearing surface defect detection algorithm, CAC-YOLOv8, which designs the Channel Attenuation Network (CAN) and Compound Pooling Pyramid Spatial Pyramid Pooling Fast (CPPSPPF) structure. Specifically, the model introduces the Channel Attenuation Network to achieve parallel feature extraction, deep feature processing, and feature fusion under different channel numbers, capturing critical features related to bearing defects and thereby improving the inference speed. Subsequently, based on the concept of overlapped receptive fields, a CPPSPPF structure is constructed, utilizing multiple iterations of max-pooling operations with smaller pooling kernel sizes to prevent information loss while expanding the receptive field, thereby strengthening the capturing ability of features at different scales. The experimental results indicate that the proposed CAC-YOLOv8 bearing surface defect detection algorithm, compared to the YOLOv8 model, achieved a 0.3% improvement in mAP@0.5, reduced model size by 14.4%, and enhanced model inference speed by 33.3%. This enables the CAC-YOLOv8 model to significantly improve the real-time performance of bearing defect detection while maintaining high-precision detection. The performance in practical industrial detection demonstrates that the proposed approach has achieved outstanding results in both speed and accuracy.

Research on lightweight speech emotion recognition in vehicle noisy environment

In order to reduce the incidence of traffic accidents caused by the emotional state of drivers, this study proposes an emotion recognition algorithm based on vehicle noise environment. This algorithm can effectively identify the emotional state of drivers and provide support for further improving their emotions. To address challenges in existing research on speech emotion recognition, such as excessive model parameters, poor generalization, and suboptimal performance in noisy environments, this paper proposes a lightweight network model suitable for small datasets. The model utilizes Power Normalized Cepstral Coefficients (PNCC) as input features, and employs parallel feature extraction layers at different scales. These features are then fed into a feature learning module for in-depth extraction, with the final determination of the driver’s emotional state made by the output layer. Experimental results show that the model achieves an accuracy of 96.08% on the EMO-DB speech dataset. Even in simulated in-vehicle noise environments, the model exhibits high accuracy and robustness. Moreover, compared to other lightweight models, it has fewer training parameters and faster processing speed, making it suitable for deployment on edge devices in mobile applications.

Surface defect detection and semantic segmentation with a novel lightweight deep neural network

Current approaches to defect detection and segmentation make essential use of machine learning methods. To develop lightweight models is one of key tasks for many defect detection and segmentation applications. In this work, we present a lightweight trilateral parallel feature extraction with multi-feature aggregation network (TriMFANet) for surface defect detection and segmentation. In TriMFANet, the top lateral is the feature-rich extraction used to capture detailed information. The other two laterals, efficient semantic feature extraction (ESFE) and reverse ESFE, leverage Hadamard product attention to jointly extract deep-level global feature information. Additionally, the MFA module employs origin-symmetric sigmoid attention to enhance deep feature information and integrates the triple features. We conducted binary defect segmentation tasks on the SD-saliency-900 and RSDDs datasets, achieving outstanding performance in both S α and E ξ . For multi-class defect detection tasks on the NEU-Seg and MSD datasets, we rank first with mIoU scores of 79.0% and 81.2% respectively. Experimental results demonstrate that our lightweight model with only 90 K parameters exhibits excellent performance.

SRMA: a dual-branch parallel multi-scale attention network for remote sensing images sea-land segmentation

ABSTRACT The use of deep learning-based high resolution remote sensing image sea and land segmentation method has become prevalent in various fields such as environmental monitoring, resource assessment, and urban planning. However, the segmentation process faces challenges due to the complex coastline and the similarity in spectral colour difference between the sea and land in certain areas. The remote sensing images obtained from satellites often exhibit unclear boundaries between the sea and land, as well as intricate coastlines. Consequently, the segmentation of sea and land becomes a difficult task. The existing sea-land segmentation methods have limitations in accurately segmenting long and narrow waters, as well as areas with low differentiation between sea and land. In this paper, we propose a new sea-land segmentation method called SRMA (Swin-Res-Multi-Attention). SRMA performs parallel feature extraction using a dual-branch parallel: Swin-Transformer branch and ResNet branch. The combination of Swin-Transformer’s self-attention and ResNet’s residual design helps reduce the loss of feature information, thereby avoiding incorrect segmentation in regions with low distinctness. To improve overall stability, we introduce the MCA (Multiscale channel attention module) module, which combines feature information from different scales. The multi-scale pyramid design of MCA module ensures stability and improves segmentation accuracy in narrow waters. Experimental results on a public dataset demonstrate that our model outperforms existing methods. Additionally, we evaluate the model’s performance in complex scenarios by creating a new dataset that includes a complex urban water body and cloud interference. The results showcase the robustness and superiority of our model.

An epilepsy detection method based on multi-dimensional feature extraction and dual-branch hypergraph convolutional network.

Epilepsy is a disease caused by abnormal neural discharge, which severely harms the health of patients. Its pathogenesis is complex and variable with various forms of seizures, leading to significant differences in epilepsy manifestations among different patients. The changes of brain network are strongly correlated with related pathologies. Therefore, it is crucial to effectively and deeply explore the intrinsic features of epilepsy signals to reveal the rules of epilepsy occurrence and achieve accurate detection. Existing methods have faced the following issues: 1) single approach for feature extraction, resulting in insufficient classification information due to the lack of rich dimensions in captured features; 2) inability to deeply analyze the essential commonality of epilepsy signal after feature extraction, making the model susceptible to data distribution and noise interference. Thus, we proposed a high-precision and robust model for epileptic seizure detection, which, for the first time, applies hypergraph convolution to the field of epilepsy detection. Through a hypergraph network structure constructed based on relationships between channels in electroencephalogram (EEG) signals, the model explores higher-order characteristics of epilepsy EEG data. Specifically, we use the Conv-LSTM module and Power spectral density (PSD), a two-branch parallel method, to extract channel features from space-time and frequency domains to solve the problem of insufficient feature extraction, and can adequately describe the data structure and distribution from multiple perspectives through double-branch parallel feature extraction. In addition, we construct a hypergraph on the captured features to explore the intrinsic features in the high-dimensional space in an attempt to reveal the essential commonality of epileptic signal feature extraction. Finally, using the ensemble learning concept, we accomplished epilepsy detection on the dual-branch hypergraph convolution. The model underwent leave-one-out cross-validation on the TUH dataset, achieving an average accuracy of 96.9%, F1 score of 97.3%, Pre of 98.2% and Re of 96.7%. In addition, the model was generalized performance tested on CHB-MIT scalp EEG dataset with leave-one-out cross-validation, and the average ACC, F1 score, Pre and Re were 94.4%, 95.1%, 95.8%, and 93.9% respectively. Experimental results indicate that the model outperforms related literature, providing valuable reference for the clinical application of epilepsy detection.

Fair large kernel embedding with relation-specific features extraction for link prediction

Knowledge graph embedding is a crucial technique for addressing the challenge of incomplete knowledge graphs, and convolutional neural networks are widely applied in this domain. However, convolutional neural networks face limitations in effectively capturing long-range dependencies between entities and relations in feature maps. Moreover, existing knowledge graph embedding models often adopt a uniform perspective on all relations within a knowledge graph, thereby overlooking the relation-specific features. To address the two issues, two novel knowledge graph embedding models, Large Kernel Embedding (LKE) and Large Kernel Embedding with Relation-specific Features Extraction (LKER), are proposed. The effectiveness of knowledge graph embedding is improved by integrating fair large kernel attention and introducing a parallel branch for relation-specific feature extraction in these models. Specifically, fair large kernel attention is incorporated into LKE, capturing long-range dependencies within the model. Based on LKE, LKER adds a parallel relationship-specific feature extraction branch to obtain more comprehensive relationship-specific feature information. The proposed models are extensively evaluated on five benchmark datasets, and the results show that the proposed models yield significant performance improvements in link prediction compared to classical and latest knowledge graph embedding models.

Breast Cancer Diagnosis Using YOLO-Based Multiscale Parallel CNN and Flattened Threshold Swish

In the field of biomedical imaging, the use of Convolutional Neural Networks (CNNs) has achieved impressive success. Additionally, the detection and pathological classification of breast masses creates significant challenges. Traditional mammogram screening, conducted by healthcare professionals, is often exhausting, costly, and prone to errors. To address these issues, this research proposes an end-to-end Computer-Aided Diagnosis (CAD) system utilizing the ‘You Only Look Once’ (YOLO) architecture. The proposed framework begins by enhancing digital mammograms using the Contrast Limited Adaptive Histogram Equalization (CLAHE) technique. Then, features are extracted using the proposed CNN, leveraging multiscale parallel feature extraction capabilities while incorporating DenseNet and InceptionNet architectures. To combat the ‘dead neuron’ problem, the CNN architecture utilizes the ‘Flatten Threshold Swish’ (FTS) activation function. Additionally, the YOLO loss function has been enhanced to effectively handle lesion scale variation in mammograms. The proposed framework was thoroughly tested on two publicly available benchmarks: INbreast and CBIS-DDSM. It achieved an accuracy of 98.72% for breast cancer classification on the INbreast dataset and a mean Average Precision (mAP) of 91.15% for breast cancer detection on the CBIS-DDSM. The proposed CNN architecture utilized only 11.33 million parameters for training. These results highlight the proposed framework’s ability to revolutionize vision-based breast cancer diagnosis.

Parallel neural network feature extraction method for predicting buckling load of composite stiffened panels

A novel Parallel Neural Network (PNN) feature extraction method is proposed in this paper to predict the buckling load of composite stiffened panels. The PNN effectively processes both stacking sequences and discrete variables by leveraging the parallel operation of the Recurrent Neural Network (RNN) and the Feedforward Neural Network (FNN). This approach addresses limitations of previous models, such as feature loss due to the Classical Laminate Theory (CLT) and struggled with variable length stacking sequences. A Self-attention-based Bidirectional Long Short-Term Memory network (T-Bi-LSTM) is introduced to handle variable length stacking sequences comprehensively. The T-Bi-LSTM, which incorporates self-attention and other mechanisms, improves the network's ability to capture crucial information. The dataset for training and testing is generated using a finite element model verified by the corresponding experiments, where the PNN with T-Bi-LSTM and other contrasting models are trained. The results suggest that the T-Bi-LSTM demonstrates better capability in extracting comprehensive stacking sequences than Bidirectional Long Short-Term Memory network (Bi-LSTM). Furthermore, the proposed PNN feature extraction method exhibits superior fitting ability and generalization performance than the feature extraction method based on CLT.

Research and implementation of multi-disease diagnosis on chest X-ray based on vision transformer.

Disease diagnosis in chest X-ray images has predominantly relied on convolutional neural networks (CNNs). However, Vision Transformer (ViT) offers several advantages over CNNs, as it excels at capturing long-term dependencies, exploring correlations, and extracting features with richer semantic information. We adapted ViT for chest X-ray image analysis by making the following three key improvements: (I) employing a sliding window approach in the image sequence feature extraction module to divide the input image into blocks to identify small and difficult-to-detect lesion areas; (II) introducing an attention region selection module in the encoder layer of the ViT model to enhance the model's ability to focus on relevant regions; and (III) constructing a parallel patient metadata feature extraction network on top of the image feature extraction network to integrate multi-modal input data, enabling the model to synergistically learn and expand image-semantic information. The experimental results showed the effectiveness of our proposed model, which had an average area under the curve value of 0.831 in diagnosing 14 common chest diseases. The metadata feature network module effectively integrated patient metadata, further enhancing the model's accuracy in diagnosis. Our ViT-based model had a sensitivity of 0.863, a specificity of 0.821, and an accuracy of 0.834 in diagnosing these common chest diseases. Our model has good general applicability and shows promise in chest X-ray image analysis, effectively integrating patient metadata and enhancing diagnostic capabilities.

Local feature expansion Vision Transformer model for bearing fault diagnosis under noise environments

Vision Transformer (ViT) shows potential in bearing fault diagnosis due to its multi-head self-attention mechanism and parallel feature extraction network which are efficient to achieve the robust complete feature representation of the fault. However, its adaption to the noise interference relies on the sufficient huge amount of training samples to prepare the local features of the fault and may suffer performance degradation when only a limited number of samples are available for the model training. To combat this challenge, an improved ViT diagnosis model based on the local feature expansion, i.e., LFE-ViT, is proposed. An auxiliary feature extraction block is introduced using a local feature expansion network and works as a parallel module with the ViT encoder. Through the enlargement of the receptive field, the multi-scale local features on a high dimensional space are available upon the limited samples. Then, through a feature embedding channel, the extracted local features are transmitted to the ViT encoder. Finally, by virtue of the multi-head self-attention mechanism to capture the time sequence global information, a fault diagnosis model comprising comprehensively local and global feature information is derived. Experimental validation on the bearing fault dataset from Case Western Reserve University shows that LFE-ViT has provided a rather satisfactory diagnosis performance under limited samples and noise environment.

MOTC: Abdominal Multi-objective Segmentation Model with Parallel Fusion of Global and Local Information.

Convolutional Neural Networks have been widely applied in medical image segmentation. However, the existence of local inductive bias in convolutional operations restricts the modeling of long-term dependencies. The introduction of Transformer enables the modeling of long-term dependencies and partially eliminates the local inductive bias in convolutional operations, thereby improving the accuracy of tasks such as segmentation and classification. Researchers have proposed various hybrid structures combining Transformer and Convolutional Neural Networks. One strategy is to stack Transformer blocks and convolutional blocks to concentrate on eliminating the accumulated local bias of convolutional operations. Another strategy is to nest convolutional blocks and Transformer blocks to eliminate bias within each nested block. However, due to the granularity of bias elimination operations, these two strategies cannot fully exploit the potential of Transformer. In this paper, a parallel hybrid model is proposed for segmentation, which includes a Transformer branch and a Convolutional Neural Network branch in encoder. After parallel feature extraction, inter-layer information fusion and exchange of complementary information are performed between the two branches, simultaneously extracting local and global features while eliminating the local bias generated by convolutional operations within the current layer. A pure convolutional operation is used in decoder to obtain final segmentation results. To validate the impact of the granularity of bias elimination operations on the effectiveness of local bias elimination, the experiments in this paper were conducted on Flare21 dataset and Amos22 dataset. The average Dice coefficient reached 92.65% on Flare21 dataset, and 91.61% on Amos22 dataset, surpassing comparative methods. The experimental results demonstrate that smaller granularity of bias elimination operations leads to better performance.

Dunhuang murals image restoration method based on generative adversarial network

AbstractMurals are an important part of China’s cultural heritage. After more than a 1000 years of exposure to the sun and wind, most of these ancient murals have become mottled, with damage such as cracking, mold, and even large-scale detachment. It is an urgent work to restore these damaged murals. The technique of digital restoration of mural images refers to the reconstruction of structures and textures to virtually fill in the damaged areas of the image. Existing digital restoration methods have the problems of incomplete restoration and distortion of local details. In this paper, we propose a generative adversarial network model combining a parallel dual convolutional feature extraction depth generator and a ternary heterogeneous joint discriminator. The generator network is designed with the mechanism of parallel extraction of image features by vanilla convolution and dilated convolution, capturing multi-scale features simultaneously, and reasonable parameter settings reduce the loss of image information. A pixel-level discriminator is proposed to identify the pixel-level defects of the captured image, and its joint global discriminator and local discriminator discriminate the generated image at different levels and granularities. In this paper, we create the Dunhuang murals dataset and validate our method on this dataset, and the experimental results show that the method of this paper has an overall improvement in the evaluation metrics of PSNR and SSIM compared with the comparative methods. The restored resultant image is more in line with the subjective vision of human beings, which achieves the effective restoration of mural images.