Global Feature Extraction Research Articles

A Multichannel-Based CNN and GRU Method for Short-Term Wind Power Prediction

Incorporating wind energy on a large scale into power systems presents challenges for the operation and control of the grid. To enhance the safety of power grid operation, accurate short-term forecasting of wind power is necessary, as it minimizes the impact of randomness. Considering the uncertainty and prediction issues associated with wind power, this paper introduces a CNN–GRU ultra-short-term wind power prediction model. This model relies on multichannel signals, including data such as wind speed, wind direction, climate conditions, and historical power outputs collected from wind farms. These data types contribute to the formation of a comprehensive multichannel signal for wind power. Following that, the CNN method extracts both global and partial features from these signals. Concurrently, features are extracted from past power outputs based on their time series. These features are then combined with the ones obtained from the convolution process. Subsequently, these combined features are input into a fully connected network. This step is crucial for blending multichannel information and generating forecast results. To validate the model, it was tested using data from a wind farm located in a specific region of Sichuan Province. According to our experimental results, the model demonstrates a high level of accuracy in computation and robust generalization ability.

AstroYOLO: A hybrid CNN–Transformer deep-learning object-detection model for blue horizontal-branch stars

Abstract Blue horizontal-branch stars (BHBs) are ideal tracers for studying the Milky Way (MW) due to their bright and nearly constant magnitude. However, an incomplete screen of BHBs from a survey would result in bias of estimation of the structure or mass of the MW. With surveys of large sky telescopes like the Sloan Digital Sky Survey (SDSS), it is possible to obtain a complete sample. Thus, detecting BHBs from massive photometric images quickly and effectually is necessary. The current acquisition methods of BHBs are mainly based on manual or semi-automatic modes. Therefore, novel approaches are required to replace manual or traditional machine-learning detection. The mainstream deep-learning-based object-detection methods are often vanilla convolutional neural networks whose ability to extract global features is limited by the receptive field of the convolution operator. Recently, a new Transformer-based method has benefited from the global receptive field advantage brought by the self-attention mechanism, exceeded the vanilla convolution model in many tasks, and achieved excellent results. Therefore, this paper proposes a hybrid convolution and Transformer model called AstroYOLO to take advantage of the convolution in local feature representation and Transformer’s easier discovery of long-distance feature dependences. We conduct a comparative experiment on the 4799 SDSS DR16 photometric image dataset. The experimental results show that our model achieves 99.25% AP@50, 93.79% AP@75, and 64.45% AP@95 on the test dataset, outperforming the YOLOv3 and YOLOv4 object-detection models. In addition, we test on larger cutout images based on the same resolution. Our model can reach 99.02% AP@50, 92.00% AP@75, and 61.96% AP@95 respectively, still better than YOLOv3 and YOLOv4. These results also suggest that an appropriate size for cutout images is necessary for the performance and computation of object detection. Compared with the previous models, our model has achieved satisfactory object-detection results and can effectively improve the accuracy of BHB detection.

Brain tumor image pixel segmentation and detection using an aggregation of GAN models with vision transformer

AbstractA number of applications in the field of medical analysis require the difficult and crucial tasks of brain tumor detection and segmentation from magnetic resonance imaging (MRI). Given that each type of brain imaging provides distinctive information about the specifics of each tumor component, in order to create a flexible and successful brain tumor segmentation system, we first suggest a normalization preprocessing method along with pixel segmentation. Then creating synthetic images is advantageous in many fields thanks to generative adversarial networks (GANs). In contrast, combining different GANs may enable understanding of the distributed features but it can make the model very complex and confusing. Standalone GAN may only retrieve the localized features in the latent version of an image. To achieve global and local feature extraction in a single model, we have used a vision transformer (ViT) along with a standalone GAN which will further improve the similarity of the images and can increase the performance of the model for detection of tumor. By effectively overcoming the constraint of data scarcity, high computational time, and lower discrimination capability, our suggested model can comprehend better accuracy, and lower computational time and also give the understanding of the information variance in various representations of the original images. The proposed model was evaluated on the BraTS 2020 dataset and Masoud2021 dataset, that is, a combination of the three datasets SARTAJ, Figshare, and BR35H. The obtained results demonstrate that the suggested model is capable of producing fine‐quality images with accuracy and sensitivity scores of 0.9765 and 0.977 on the BraTS 2020 dataset as well as 0.9899 and 0.9683 on the Masoud2021 dataset.

YH-Pose: Human pose estimation in complex coal mine scenarios

Human pose estimation in continuous video frames captured from complex coal mine scenes is challenging. The video frames in this scene may suffer from problems such as uneven brightness, blurred image details, and excessive noise. Mainstream pose estimation methods have good detection performance for high-quality static images, but their accuracy and prediction rate can be significantly reduced in coal mine scenes. In this work, a yielding human pose estimation framework is proposed, termed YH-Pose. The framework strives to incorporate additional visual evidences from neighboring frames to facilitate the pose estimation of the current frame. Firstly, a human detector is introduced to locate the person’s position in the video frame and extract global features. This can provide a good initialization for the latter keypoint detection, making the training process converge quickly. Secondly, heatmaps are used to encode the joint locations as Gaussian peaks, and a temporal road module (TRM) is designed, which encodes video frames at intervals. The module efficiently fuses spatio-temporal information through frame-rate groups in a hierarchical manner. Lastly, the spatial road module (SRM) learns from fused keypoint context features and predicts the location of the keypoints in the next frame. In addition, a dataset called Colliery-1 is proposed, which derives from underground surveillance video from chinese coal mines and consists of 3600 video clips. The experimental results on the Colliery-1 dataset indicate that the framework achieved an average accuracy of 82% and 80% on the training and test sets, respectively. Moreover, the framework achieved a 94.2% prediction rate for pose estimation. To further evaluate the effectiveness of the proposed method, some comparisons have been made between it and various mainstream methods using different metrics.

Fusing bi-directional global–local features for single image super-resolution

Image super resolution, which obtains high resolution output from a corresponding low resolution image, has been challenging due to inefficiencies in establishing complex high dimensional mapping for massive raw data. Single image super resolution can dramatically improve performance compared with current algorithms due to the proliferation of deep learning systems. However, convolutional kernels in deep neural networks are locally connected to the input feature maps, whereas features only interact with their local neighbors. Mutual interference between local features without considering global features causes blurring and staircase effects. This paper proposes an end-to-end single image super resolution model by simultaneously separating high and low frequency features and learning adaptive local and global features to effectively reconstruct the high resolution image by minimizing the loss of edges and texture information. The image frequency decomposition module with an attention block emphasizes self-representative low frequency features to separate high and low frequency features. The bidirectional global and local feature exchange module extracts global and local features from the separated network and fuses each feature to improve performance. Quantitative and qualitative analyses for the proposed frequency adaptive network validated that the proposed method is stable and robust against blurring and staircase effects by separating texture and the structure into adaptive and shared networks.

SID-TGAN: A Transformer-Based Generative Adversarial Network for Sonar Image Despeckling

Sonar images are inherently affected by speckle noise, which degrades image quality and hinders image exploitation. Despeckling is an important pre-processing task that aims to remove such noise so as to improve the accuracy of analysis tasks on sonar images. In this paper, we propose a novel transformer-based generative adversarial network named SID-TGAN for sonar image despeckling. In the SID-TGAN framework, transformer and convolutional blocks are used to extract global and local features, which are further integrated into the generator and discriminator networks for feature fusion and enhancement. By leveraging adversarial training, SID-TGAN learns more comprehensive representations of sonar images and shows outstanding performance in speckle denoising. Meanwhile, SID-TGAN introduces a new adversarial loss function that combines image content, local texture style, and global similarity to reduce image distortion and information loss during training. Finally, we compare SID-TGAN with state-of-the-art despeckling methods on one image dataset with synthetic optical noise and four real sonar image datasets. The results show that it achieves significantly better despeckling performance than existing methods on all five datasets.

TransPCGC: Point Cloud Geometry Compression Based on Transformers

Due to the often substantial size of the real-world point cloud data, efficient transmission and storage have become critical concerns. Point cloud compression plays a decisive role in addressing these challenges. Recognizing the importance of capturing global information within point cloud data for effective compression, many existing point cloud compression methods overlook this crucial aspect. To tackle this oversight, we propose an innovative end-to-end point cloud compression method designed to extract both global and local information. Our method includes a novel Transformer module to extract rich features from the point cloud. Utilization of a pooling operation that requires no learnable parameters as a token mixer for computing long-distance dependencies ensures global feature extraction while significantly reducing both computations and parameters. Furthermore, we employ convolutional layers for feature extraction. These layers not only preserve the spatial structure of the point cloud, but also offer the advantage of parameter independence from the input point cloud size, resulting in a substantial reduction in parameters. Our experimental results demonstrate the effectiveness of the proposed TransPCGC network. It achieves average Bjontegaard Delta Rate (BD-Rate) gains of 85.79% and 80.24% compared to Geometry-based Point Cloud Compression (G-PCC). Additionally, in comparison to the Learned-PCGC network, our approach attains an average BD-Rate gain of 18.26% and 13.83%. Moreover, it is accompanied by a 16% reduction in encoding and decoding time, along with a 50% reduction in model size.

A Dual Convolutional Neural Network with Attention Mechanism for Thermal Infrared Image Enhancement

In industrial applications, thermal infrared images, which are commonly used, often suffer from issues such as low contrast and blurred details. Traditional image enhancement algorithms are limited in their effectiveness in improving the visual quality of thermal infrared images due to the specific nature of the application. Therefore, we propose a dual Convolutional Neural Network (CNN) combined with an attention mechanism to address the challenges of enhancing low-quality thermal infrared images and improving their visual quality. Firstly, we employ two parallel sub-networks to extract both global and local features. In one sub-network, we utilize a sparse mechanism incorporating dilated convolutions, while the other sub-network employs Feature Attention (FA) blocks based on channel attention and pixel attention. This architecture significantly enhances the feature extraction capability. The use of attention mechanisms allows the network to filter out irrelevant background information, enabling more flexible feature extraction. Finally, through a simple yet effective fusion block, we thoroughly integrate the extracted features to achieve an optimal fusion strategy, ensuring the highest quality enhancement of the final image. Extensive experiments on benchmark datasets and real images demonstrate that our proposed method outperforms other state-of-the-art models in terms of objective evaluation metrics and subjective assessments. The generated images also exhibit superior visual quality.

Enhancing Microseismic Signal Classification in Metal Mines Using Transformer-Based Deep Learning

As microseismic monitoring technology gains widespread application in mine risk pre-warning, the demand for automatic data processing has become increasingly evident. One crucial requirement that has emerged is the automatic classification of signals. To address this, we propose a Transformer-based method for signal classification, leveraging the global feature extraction capability of the Transformer model. Firstly, the original waveform data were framed, windowed, and feature-extracted to obtain a 16 × 16 feature matrix, serving as the primary input for the subsequent microseismic signal classification models. Then, we verified the classification performance of the Transformer model compared with five microseismic signal classification models, including VGG16, ResNet18, ResNet34, SVM, and KNN. The experimental results demonstrate the effectiveness of the Transformer model, which outperforms previous methods in terms of accuracy, precision, recall, and F1 score. In addition, a comprehensive analysis was performed to investigate the impact of the Transformer model’s parameters and feature importance on outcomes, which provides a valuable reference for further enhancing microseismic signal classification performance.

DP-FishNet: Dual-path Pyramid Vision Transformer-based underwater fish detection network

The detection of underwater fish targets is critical for ecological monitoring and marine biodiversity research. However, underwater fish detection is typically constrained by problems including low image quality and variable underwater surroundings. On behalf of further improving the underwater fish detection accuracy in complex underwater environments, this paper proposes a dual-path (DP) Pyramid Vision Transformer (PVT) feature extraction network named DP-FishNet. The backbone network DP-PVT composed from the PVT Network is made up of two feature extraction paths. The first represents the Vision Transformer path, which extracts global features to enhance the distinction between the foreground and background of underwater images. The second is the convolutional neural network path, which enhances the accuracy of detecting small targets by extracting local features. Additionally, to more effectively utilize the feature information extracted by the network, this paper provides a promising solution to employ the content-aware reassembly of features (Carafe) in the feature pyramid network (FPN). The seesaw loss is utilized as a classification loss to address the problem of unbalanced samples caused by the gap in the number of fish populations. According to the experimental findings, the AP and AP50 of the DP-FishNet are 76.0% and 95.2%, respectively. In comparison to currently available advanced two-stage detection algorithms, the quantity of computation and parameters is reduced by approximately 40%. DP-FishNet strengthens the ability to extract global and local features from underwater images and enhances feature reuse. DP-FishNet can be utilized to detect fish targets in actual and complicated underwater habitats.

Multi-Scale Residual Depthwise Separable Convolution for Metro Passenger Flow Prediction

Accurate prediction of metro passenger flow helps operating departments optimize scheduling plans, alleviate passenger flow pressure, and improve service quality. However, existing passenger flow prediction models tend to only consider the historical passenger flow of a single station while ignoring the spatial relationships between different stations and correlations between passenger flows, resulting in low prediction accuracy. Therefore, a multi-scale residual depthwise separable convolution network (MRDSCNN) is proposed for metro passenger flow prediction, which consists of three pivotal components, including residual depthwise separable convolution (RDSC), multi-scale depthwise separable convolution (MDSC), and attention bidirectional gated recurrent unit (AttBiGRU). The RDSC module is designed to capture local spatial and temporal correlations leveraging the diverse temporal patterns of passenger flows, and then the MDSC module is specialized in obtaining the inter-station correlations between the target station and other heterogeneous stations throughout the metro network. Subsequently, these correlations are fed into AttBiGRU to extract global interaction features and obtain passenger flow prediction results. Finally, the Hangzhou metro passenger inflow and outflow data are employed to assess the model performance, and the results show that the proposed model outperforms other models.

Complete perception self-attention network for weak seismic signal recovery in distributed acoustic sensing vertical seismic profile data

Distributed acoustic sensing (DAS) is a new technology for acquiring seismic data with high spatial resolution at low cost. Furthermore, in real downhole seismic exploration, DAS can receive some weak signals reflected from deep and thin layers. Unfortunately, some real downhole seismic data received by DAS often are characterized by low quality. Specifically, in real DAS records, desired signals with weak energy often are contaminated by some new noise not presented in seismic data received by conventional electronic geophones. Due to the characteristics of seismic data, such as frequency band aliasing, low signal-to-noise ratio, and complex noise wavefield, existing linear or nonlinear denoising methods based on information processing theories cannot effectively eliminate this complex and multitype noise. Recently, the deep-learning method has been regarded as a powerful tool for background noise attenuation in seismic data. Most of the existing deep-learning methods are concerned with local features and ignore the global features that can be used to enhance their performance further. To simultaneously extract global and local features, we design a novel complete perception self-attention network (CP-SANet) based on the transformer framework and apply it to the denoising of downhole DAS records. The network embeds a transformer module into multilevel encoder-decoder framework. Depth-wise convolution is applied to enhance the local perception capability. Given the transformer’s requirement for a large amount of data, we specifically design abundant seismic data samples using formation models with different parameters. The noise in the data sets is obtained from actual field DAS data. The effectiveness and feasibility of CP-SANet are verified on synthetic and field DAS records. All of the experimental results prove its satisfactory performance compared with some classical and network methods.

Multi-scale SE-residual network with transformer encoder for myocardial infarction classification

In recent years, there has been an increase in the number of deaths caused by heart disease. The design of computer-aided systems for the diagnosis of heart diseases, especially myocardial infarction (MI), has become a hot topic. The diagnosis of MI is a classification task. Electrocardiogram (ECG) is one of the most important tools for diagnosing heart disease. ECG signals are classified into corresponding MI categories based on their extracted features. Thus, the diagnosis of MI depends on the feature extraction of the ECG signal. Because signal processing methods, such as wavelet transform, may not effectively extract hidden features, researchers nowadays turn to using convolutional neural networks to solve the problem of hidden feature extraction. However, this approach ignores the role of temporal information. we proposed a new network called Multi-scale SE-Residual Network with Transformer encoder (MRTNet) to extract hidden features and temporal information features better. The data processed are localized to heartbeat units based on the way cardiologists diagnose myocardial infarction. Inspired by multi-scale learning, the sampling module was designed to acquire heartbeat units at different scales. We provided a processing model for multi-scale data, called the feature extraction module. The residual network with SE block and Customized Pooling Component (CPC) was used to extract global and local features of the heartbeat units. A Transformer encoder was used to extract the common features of the previous module’s output. The common features at different scales were fused to provide the basis for the classification task. The experiment was conducted on the public PTB-XL dataset. MRTNet achieved a higher accuracy and F1 score than other models by approximately 2%. Furthermore, MRTNet demonstrated a superior AUC of approximately 0.77. It is a superior classifier when compared to other models. The results demonstrate the effectiveness of the designed multi-scale architecture. And it provides a way into exploring further potential information from ECG signals.

An attention-based dual-encoding network for fire flame detection using optical remote sensing

Automatic extraction of flame area plays an important role in forest fire detection, which can accurately understand the spatial distribution and development trend of forest fire, so as to effectively realize the protection of forest resources. However, due to the instability and spread of fires, and the complexity of the background, accurate early fire detection is extremely challenging. At the same time, the image pixel proportion of the flame area in early stage is much smaller than that in the background, which causes a serious class imbalance problem. With the fast development of deep learning, some achievements have been made in flame extraction, but there are still some deficiencies in the existing networks, such as limited feature representation, poor feature capturing ability on micro objects, insufficiency processing of local features, etc. This paper proposes an attention-based dual-encoding segmentation network, abbreviated as ADE-Net, for pixelwise early fire detection. To realize strong feature representation, a dual-encoding path, consisting of semantic units and spatial units, is introduced to extract richer features, and an attention fusion module (AFM) is introduced to fully integrate spatial and semantic information and achieve effective feature aggregation. In addition, faced with the class imbalance problem, a multi-attention fusion (MAF) module is introduced to obtain more discriminating features to make the segmentation network to focus on the key pixel areas. Furthermore, a feature enhancement module, named attention-guided enhancement (AGE) module, is proposed to enrich the feature representation of local feature maps. Finally, to realize better multi-scale global feature extraction and fusion, a global context fusion (GCF) module is proposed into the bottleneck layer for multi-scale feature enhancement. Experimental results show that the proposed ADE-Net has a good early fire detection ability from remote sensing images, and it has obtained a competitive advantage compared with advanced segmentation models.

Boundary-guided part reasoning network for human parsing

The task of human parsing aims to segment the human body into different semantic regions. Despite advancements in this field, there are still two issues with current works: boundary indistinction and parsing inconsistency. In this paper, we investigate how to utilize structural information and auxiliary information to jointly solve the above two problems. Drawing inspiration from Transformer architecture, a Boundary-guided Part Reasoning Network (BPRNet) is proposed to combine edge information and associated semantics of body parts for human parsing. Specifically, we design a part representation module to represent human body parts as part features. Based on the Transformer decoder, a multi-head self-attention is used to capture the semantic correlation between the human body. Moreover, we propose a boundary-guided module consisting of absolute boundary attention and reinforced boundary attention. They take advantage of edge information and multi-scale image features to jointly constrain cross-attention to extract global features. Experiments and corresponding results on three public datasets show that the proposed method performs favorably against the state-of-the-art methods.

Vision-based aircraft pose estimation with dual attention module for global feature extraction in complex airport scenes

Vision-based aircraft pose estimation with dual attention module for global feature extraction in complex airport scenes

Development and validation of an artificial intelligence assisted prenatal ultrasonography screening system for trainees.

Fetal anomaly screening via ultrasonography, which involves capturing and interpreting standard views, is highly challenging for inexperienced operators. We aimed to develop and validate a prenatal-screening artificial intelligence system (PSAIS) for real-time evaluation of the quality of anatomical images, indicating existing and missing structures. Still ultrasonographic images obtained from fetuses of 18-32 weeks of gestation between 2017 and 2018 were used to develop PSAIS based on YOLOv3 with global (anatomic site) and local (structures) feature extraction that could evaluate the image quality and indicate existing and missing structures in the fetal anatomical images. The performance of the PSAIS in recognizing 19 standard views was evaluated using retrospective real-world fetal scan video validation datasets from four hospitals. We stratified sampled frames (standard, similar-to-standard, and background views at approximately 1:1:1) for experts to blindly verify the results. The PSAIS was trained using 134 696 images and validated using 836 videos with 12 697 images. For internal and external validations, the multiclass macro-average areas under the receiver operating characteristic curve were 0.943 (95% confidence interval [CI], 0.815-1.000) and 0.958 (0.864-1.000); the micro-average areas were 0.974 (0.970-0.979) and 0.973 (0.965-0.981), respectively. For similar-to-standard views, the PSAIS accurately labeled 90.9% (90.0%-91.4%) with key structures and indicated missing structures. An artificial intelligence system developed to assist trainees in fetal anomaly screening demonstrated high agreement with experts in standard view identification.

RTAU-Net: A novel 3D rectal tumor segmentation model based on dual path fusion and attentional guidance

Background and objectiveAccording to the Global Cancer Statistics 2020, colorectal cancer has the third-highest diagnosis rate (10.0 %) and the second-highest mortality rate (9.4 %) among the 36 types. Rectal cancer accounts for a large proportion of colorectal cancer. The size and shape of the rectal tumor can directly affect the diagnosis and treatment by doctors. The existing rectal tumor segmentation methods are based on two-dimensional slices, which cannot analyze a patient's tumor as a whole and lose the correlation between slices of MRI image, so the practical application value is not high. MethodsIn this paper, a three-dimensional rectal tumor segmentation model is proposed. Firstly, image preprocessing is performed to reduce the effect caused by the unbalanced proportion of background region and target region, and improve the quality of the image. Secondly, a dual-path fusion network is designed to extract both global features and local detail features of rectal tumors. The network includes two encoders, a residual encoder for enhancing the spatial detail information and feature representation of the tumor and a transformer encoder for extracting global contour information of the tumor. In the decoding stage, we merge the information extracted from the dual paths and decode them. In addition, for the problem of the complex morphology and different sizes of rectal tumors, a multi-scale fusion channel attention mechanism is designed, which can capture important contextual information of different scales. Finally, visualize the 3D rectal tumor segmentation results. ResultsThe RTAU-Net is evaluated on the data set provided by Shanxi Provincial Cancer Hospital and Xinhua Hospital. The experimental results showed that the Dice of tumor segmentation reached 0.7978 and 0.6792, respectively, which improved by 2.78 % and 7.02 % compared with suboptimal model. ConclusionsAlthough the morphology of rectal tumors varies, RTAU-Net can precisely localize rectal tumors and learn the contour and details of tumors, which can relieve physicians' workload and improve diagnostic accuracy.

Efficient multimodel method based on transformers and CoAtNet for Alzheimer's diagnosis

Convolutional neural networks (CNNs) have been widely used in medical imaging applications, including brain diseases such as Alzheimer's disease (AD) classification based on neuroimaging data. Researchers extract the potential brain regions related to AD disease using CNN from various imaging modalities due to its architectural inductive bias. The major limitation of the current CNN-based model is that it doesn't capture long-range relationships and long-distance correlation within the image features. Vision transformers (ViT) have proven an astounding performance in encoding long-range relationships with strong modeling capacity and global feature extraction due to the self attention mechanism. However, ViT doesn't model the spatial information or the local features within the image and is hard to train. Researchers have demonstrated that combining CNN and a transformer yields outstanding results. In this study, two new methods are proposed for Alzheimer's disease diagnosis. The first method combines the Swin transformer with an enhanced EfficientNet with multi-head attention and a Depthwise Over-Parameterized Convolutional Layer (DO-Conv). The second method consists of modifying the CoAtNet network with ECA-Net and fused inverted residuals blocks. We evaluated the effectiveness of our proposed methods based on the Open Access Series of Imaging Studies (OASIS) and the Alzheimer's Disease Neuroimaging Initiative (ADNI). Further, we evaluated the proposed methods using the Gradient-based Localization (Grad-CAM) method. The first method achieved 93.23% accuracy of classification on the OASIS dataset. The second method achieved 97.33% accuracy of classification on the OASIS dataset. We applied different multimodal image fusion methods (MRI and PET, MRI and CT) using our proposed method. The experimental results demonstrate that the fusion method based on PET and MRI outperforms the fusion method based on MRI and CT achieving 99.42% accuracy. Our methods outperform some traditional CNN models and the recent methods that are based on transformer for AD classification.

Contextual-Assisted Scratched Photo Restoration

Printed photographs can be easily warped, wrinkled, and even deteriorated over time. Existing methods treat the restoration of scratches as a pure inpainting problem that neglects the underlying corrupted contextual knowledge. However, important underlying contents are hidden behind the scratches, which are essential hints for producing a semantically consistent result. Motivated by this insight, we explore how to harmonize the scratch-free features and noisy but essential scratch features to produce a visually consistent restoration. Specifically, in this paper, we propose an automatic retouching approach for scratched photographs with the aid of scratch/background context. We explicitly process scratch and background context in two stages. In the first stage, we mainly extract global scratch features, while the mask is introduced in the second stage to filter out and inpaint the scratches. Both contexts are carefully reciprocated for a faithful restoration. Particularly, we propose a Scratch Contextual Assisted Module (SCAM) to adaptively learn texture within the detected mask. This module utilizes the distance between the scratch mask-out feature and scratch encoder feature for modeling the pixel-wise correspondence, which determines the importance of the encoder feature within the scratch mask. Furthermore, to facilitate the evaluation of scratch restoration methods, we create two new scratched photo datasets which have 238 scratch/scratch-free photo pairs to promote the development in the scratch restoration field, namely Old Scratched Photo Dataset (OSPD) and Modern Scratched Photo Dataset (MSPD). Extensive experimental results on the proposed datasets demonstrate that our model outperforms existing methods. To extend the application, we also perform the proposed method on video samples and obtain visual-pleasing results. The code can be found at https://github.com/cwyyt/ Contextual-assisted-Scratched-Photo-Restoration.