Multi-scale Frequency Research Articles

CTFNet: Long-Sequence Time-Series Forecasting Based on Convolution and Time-Frequency Analysis.

Although current time-series forecasting methods have significantly improved the state-of-the-art (SOTA) results for long-sequence time-series forecasting (LSTF), they still have difficulty in capturing and extracting the features and dependencies of long-term sequences and suffer from information utilization bottlenecks and high-computational complexity. To address these issues, a lightweight single-hidden layer feedforward neural network (SLFN) combining convolution mapping and time-frequency decomposition called CTFNet is proposed with three distinctive characteristics. First, time-domain (TD) feature mining-in this article, a method for extracting the long-term correlation of horizontal TD features based on matrix factorization is proposed, which can effectively capture the interdependence among different sample points of a long time series. Second, multitask frequency-domain (FD) feature mining-this can effectively extract different frequency feature information of time-series data from the FD and minimize the loss of data features. Integrating multiscale dilated convolutions, simultaneously focusing on both global and local context feature dependencies at the sequence level, and mining the long-term dependencies of the multiscale frequency information and the spatial dependencies among the different scale frequency information, break the bottleneck of data utilization, and ensure the integrity of feature extraction. Third, highly efficient-the CTFNet model has a short training time and fast inference speed. Our empirical studies with nine benchmark datasets show that compared with state-of-the-art methods, CTFNet can reduce prediction error by 64.7% and 53.7% for multivariate and univariate time series, respectively.

A fully locally selective large kernel network for traffic video detection

Vision Transformer (ViT) are widely used in the field of traffic video inspection because of their wide sensing field and strong global context-capturing capability. However, the ViT model has inadequate local detail feature extraction and high computational complexity. Convoluted neural networks(CNN) are commonly used as a means of video image detection. Although it is more flexible and less computationally complex in the extraction of local details, the fixed-size kernel limits it. It is difficult to effectively utilize the full local context information, resulting in the difficulty of effectively extracting the tiny targets with uneven scale distribution in the region. It is prone to leakage and misdetection of small targets. In view of this, this study proposes a new fully localized selective large kernel network (FL-SLKNet). The network effectively captures specific targets and background information through a fully localized feature extraction method, thus making small targets in the region more distinctive features. On this basis, the Adaptive Expansion Residual Module (AERM) is proposed, which solves the problem of image resolution degradation due to sensory field expansion by combining point-by-point convolution and expansion convolution and utilizing residual connections to supplement the lost information. In order to effectively identify tiny targets in the local area, this study proposes the Multi-Scale Frequency Domain Encoder (MSFDEncoder); this encoder utilizes a high-frequency branch to capture local details and a low-frequency branch to focus on the global structure, effectively capturing granular features of targets at various scales. Experimental results on the VisDrone2019-DET and BDD-100K datasets indicate that FL-SLKNet outperforms other advanced models in traffic video detection, with an increase of 3.3% and 3.4% in mAP0.5 compared to the RT-DETR model. Meanwhile, the detection performance of FL-SLKNet is excellent compared with the YOLO series on the SZ Actual Traffic Video dataset. In addition, in the rainy and hazy simulation scenarios, FL-SLKNet’s mAP0.5 improves by 5.5% and 2.6% compared to the RT-DETR model, and the extreme weather robustness performance is better.

A Novel Wind Power Prediction Model That Considers Multi-Scale Variable Relationships and Temporal Dependencies

Wind power forecasting is a critical technology for promoting the effective integration of wind energy. To enhance the accuracy of wind power predictions, this paper introduces a novel wind power prediction model that considers the evolving relationships of multi-scale variables and temporal dependencies. In this paper, a multi-scale frequency decomposition module is designed to split the raw data into high-frequency and low-frequency parts. Subsequently, features are extracted from the high-frequency information using a multi-scale temporal graph neural network combined with an adaptive graph learning module and from the low-frequency data using an improved bidirectional temporal network. Finally, the features are integrated through a cross-attention mechanism. To validate the effectiveness of the proposed model, extensive comprehensive experiments were conducted using a wind power dataset provided by the State Grid. The experimental results indicate that the MSE of the model proposed in this paper has decreased by an average of 7.1% compared to the state-of-the-art model and by 48.9% compared to the conventional model. Moreover, the improvement in model performance becomes more pronounced as the prediction horizon increases.

A Method for Sorting High-Quality Fresh Sichuan Pepper Based on a Multi-Domain Multi-Scale Feature Fusion Algorithm.

Post-harvest selection of high-quality Sichuan pepper is a critical step in the production process. To achieve this, a visual system needs to analyze Sichuan pepper with varying postures and maturity levels. To quickly and accurately sort high-quality fresh Sichuan pepper, this study proposes a multi-scale frequency domain feature fusion module (MSF3M) and a multi-scale dual-domain feature fusion module (MS-DFFM) to construct a multi-scale, multi-domain fusion algorithm for feature fusion of Sichuan pepper images. The MultiDomain YOLOv8 Model network is then built to segment and classify the target Sichuan pepper, distinguishing the maturity level of individual Sichuan peppercorns. A selection method based on the average local pixel value difference is proposed for sorting high-quality fresh Sichuan pepper. Experimental results show that the MultiDomain YOLOv8-seg achieves an mAP50 of 88.8% for the segmentation of fresh Sichuan pepper, with a model size of only 5.84 MB. The MultiDomain YOLOv8-cls excels in Sichuan pepper maturity classification, with an accuracy of 98.34%. Compared to the YOLOv8 baseline model, the MultiDomain YOLOv8 model offers higher accuracy and a more lightweight structure, making it highly effective in reducing misjudgments and enhancing post-harvest processing efficiency in agricultural applications, ultimately increasing producer profits.

State of health estimation for lithium-ion batteries based on multi-scale frequency feature and time domain feature fusion method

Abstract Accurately estimating the state of health (SOH) of lithium-ion batteries is important for improving battery safety performance. The single time-domain feature extraction is hard to efficiently extract discriminative features from strongly nonlinear coupled data, leading to difficulties in accurately estimating the battery SOH. To this end, this paper proposes a multi-scale frequency domain feature and time domain feature fusion method for SOH estimation of lithium-ion batteries based on the Transformer model. Firstly, the voltage, current, temperature and time information of the battery are extracted as time domain features; secondly, the battery signal is processed by a multi-scale filter bank based on Mel-frequency cepstral coefficients (MFCC) to obtain the multi-scale frequency domain features; then, a parallel focusing network (PFN) is designed to fuse the time domain features with the frequency domain features, which yields low-coupling complementary discriminative features; finally, constructing the SOH estimation mechanism based on the Transformer deep network model. The algorithm is validated by NASA and Oxford datasets, the mean's absolute error (MAE) and root mean square error (RMSE) are as low as 0.06% and 0.23%, respectively.

Multi-scale frequency domain learning for texture classification

Multi-scale frequency domain learning for texture classification

Measurement of aero-engine feature-hierarchy fusion degradation trend based on parameter-adaptive VMD method and improved transformer model

The accumulation of operational time in aero-engines leads to irreversible mechanical wear and tear, necessitating accurate measurement of the health evolution trend for effective predictive maintenance, thus reducing the risk of accidents and ensuring personnel safety. In this paper, a parameter-adaptive variational mode decomposition (VMD) method and improved transformer model are proposed to forecast the degradation trend of aero-engine feature hierarchy fusion. Firstly, in order to quantitatively evaluate the engine health evolution process, the health state aggregate indicator (HSAI) is innovatively constructed by employing the deep blend auto-encoder and self-organizing map network, which facilitate the feature-hierarchy fusion of multi-source sensory data. Secondly, for the significant characteristics with nonlinearity and stochastic fluctuation of the HSAI sequence, the multiscale frequency features are extracted by the parameter-adaptive VMD method with the improved gray wolf optimizer, which analyzes the inherent degradation law. Finally, considering the problem of parameter sharing in the transformer model, a simplified mixture of experts routing algorithm is introduced to implement the switch transformer model to further measure the future aero-engine health trends. Extensive experiments on the multi-source dataset of aero-engine confirm that the proposed method accomplishes the more superior performance for health evolution measurement compared with other available methods.

Medical image segmentation network based on multi-scale frequency domain filter

With the development of deep learning, medical image segmentation in computer-aided diagnosis has become a research hotspot. Recently, UNet and its variants have become the most powerful medical image segmentation methods. However, these methods suffer from (1) insufficient sensing field and insufficient depth; (2) computational nonlinearity and redundancy of channel features; and (3) ignoring the interrelationships among feature channels. These problems lead to poor network segmentation performance and weak generalization ability. Therefore, first of all, we propose an effective replacement scheme of UNet base block, Double residual depthwise atrous convolution (DRDAC) block, to effectively improve the deficiency of receptive field and depth. Secondly, a new linear module, the Multi-scale frequency domain filter (MFDF), is designed to capture global information from the frequency domain. The high order multi-scale relationship is extracted by combining the depthwise atrous separable convolution with the frequency domain filter. Finally, a channel attention called Axial selection channel attention (ASCA) is redesigned to enhance the network’s ability to model feature channel interrelationships. Further, we design a novel frequency domain medical image segmentation baseline method FDFUNet based on the above modules. We conduct extensive experiments on five publicly available medical image datasets and demonstrate that the present method has stronger segmentation performance as well as generalization ability compared to other state-of-the-art baseline methods.

EISATC-Fusion: Inception Self-Attention Temporal Convolutional Network Fusion for Motor Imagery EEG Decoding.

The motor imagery brain-computer interface (MI-BCI) based on electroencephalography (EEG) is a widely used human-machine interface paradigm. However, due to the non-stationarity and individual differences among subjects in EEG signals, the decoding accuracy is limited, affecting the application of the MI-BCI. In this paper, we propose the EISATC-Fusion model for MI EEG decoding, consisting of inception block, multi-head self-attention (MSA), temporal convolutional network (TCN), and layer fusion. Specifically, we design a DS Inception block to extract multi-scale frequency band information. And design a new cnnCosMSA module based on CNN and cos attention to solve the attention collapse and improve the interpretability of the model. The TCN module is improved by the depthwise separable convolution to reduces the parameters of the model. The layer fusion consists of feature fusion and decision fusion, fully utilizing the features output by the model and enhances the robustness of the model. We improve the two-stage training strategy for model training. Early stopping is used to prevent model overfitting, and the accuracy and loss of the validation set are used as indicators for early stopping. The proposed model achieves within-subject classification accuracies of 84.57% and 87.58% on BCI Competition IV Datasets 2a and 2b, respectively. And the model achieves cross-subject classification accuracies of 67.42% and 71.23% (by transfer learning) when training the model with two sessions and one session of Dataset 2a, respectively. The interpretability of the model is demonstrated through weight visualization method.

ConvGRU-based Multi-scale Frequency Fusion Network for PAN-MS Joint Classification

ConvGRU-based Multi-scale Frequency Fusion Network for PAN-MS Joint Classification

On-machine frequency analysis of diamond turned surfaces with surface intrinsic mode decomposition

Evaluating surface frequency components in the fabrication process is critical for controlling the machined surface quality. The presence of anisotropic ripples on diamond-turned surfaces makes this challenging. A multiscale frequency evaluation method, referred to as Surface Intrinsic Mode Decomposition (SIMD), is proposed for evaluating on-machine surface measurement (OMSM) data. It decomposes continuous surface probing profiles, incorporating both temporal and spatial frequency information. In comparison to the conventional power spectral density (PSD) analysis method, the approach enriches frequency details over a wider range, which contributes to a more comprehensive understanding of surface quality and helps to identify mid-spatial frequency (MSF) errors.

Wavelet-Based Self-Attention GAN With Collaborative Feature Fusion for Image Inpainting

Image inpainting is a significant task in the applications of computer vision, that aims to fill in damaged regions with visually realistic contents. With the development of deep learning, generative adversarial network (GAN)-based image inpainting approaches have achieved remarkable progress. However, these methods only utilize one-sided structure information to assist inpainting, which can not achieve satisfactory results, especially when synthesizing large-area missing complex images. In order to tackle this problem, a wavelet-based self-attention GAN (WSA-GAN) with collaborative feature fusion is proposed, which is embedded with a wavelet-based self-attention (WSA) and a collaborative feature fusion (CFF). The WSA is designed to conduct long-range dependence among multi-scale frequency information to highlight significant structure details for better generating texture boundaries. The CFF is presented to couple channel-guided space and space-affected channel streams to facilitate the interaction of spatial and channel features, which can effectively avoid potential domain conflicts. Besides, a novel wavelet consistency loss and a hierarchical pyramid feature matching (PFM) discriminator are introduced to stabilize the model training. Extensive experiments on three public datasets, including Paris StreetView, CelebA-HQ and Places, demonstrate that the proposed method outperforms the state-of-the-art methods both quantitatively and qualitatively.

Multi-Scale Discrete Cosine Transform Network for Building Change Detection in Very-High-Resolution Remote Sensing Images

With the rapid development and promotion of deep learning technology in the field of remote sensing, building change detection (BCD) has made great progress. Some recent approaches have improved detailed information about buildings by introducing high-frequency information. However, there are currently few methods considering the effect of other frequencies in the frequency domain for enhancing feature representation. To overcome this problem, we propose a multi-scale discrete cosine transform (DCT) network (MDNet) with U-shaped architecture, which is composed of two novel DCT-based modules, i.e., the dual-dimension DCT attention module (D3AM) and multi-scale DCT pyramid (MDP). The D3AM aims to employ the DCT to obtain frequency information from both spatial and channel dimensions for refining building feature representation. Furthermore, the proposed MDP can excavate multi-scale frequency information and construct a feature pyramid through multi-scale DCT, which can elevate multi-scale feature extraction of ground targets with various scales. The proposed MDNet was evaluated with three widely used BCD datasets (WHU-CD, LEVIR-CD, and Google), demonstrating that our approach can achieve more convincing results compared to other comparative methods. Moreover, extensive ablation experiments also present the effectiveness of our proposed D3AM and MDP.

MS-FTSCNN: An EEG emotion recognition method from the combination of multi-domain features

Electroencephalography (EEG), as a physiological cue, is more objective and reliable in identifying emotions than non-physiological cues. Previous methods only consider one or two relationships among frequency, time and spatial domain features of EEG signals, and the designed models may still be relatively large in terms of parameters. Meanwhile, the training process of the previous networks is troublesome during algorithm optimization. To address these challenges, we design a simple and efficient feature preprocessing method to obtain a 3D feature structure that contains EEG signal information in the frequency, time and spatial domains simultaneously. Then, we propose a multiscale frequency–time–spatial convolutional model, MS-FTSCNN, which is able to capture frequency, time and spatial features from the input signals and fuse three features more efficiently. Moreover, the multi-scale one-dimensional convolutional kernel in our method can reduce network parameters, providing possibilities for real-time online applications. Finally, the recognition accuracies of arousal and valence of our proposed model are 93.82%, 94.48% on DEAP dataset and 92.64%, 92.15% on MOHNOB-HCI dataset, which is higher than most existing methods.

MFAGNet: multi-scale frequency attention gating network for land cover classification

ABSTRACT As a classic problem of high-resolution remote sensing image tasks, land cover classification has many challenges, for instance, variable scales and low-edge discrimination. Traditional spatial semantic segmentation methods based on single-scale feature extraction are challenging to model multi-scale features. They cannot effectively fit high-resolution remote sensing images with variable scales and are prone to misclassification issues. On the other hand, spatial multi-scale semantic segmentation methods lack edge constraints, which can easily lead to edge segmentation ambiguity issues. We propose a segmentation network based on a multi-scale frequency-domain attention gating mechanism (MFAGNet) to solve the above problems. Specifically, we obtain multi-scale features through multi-scale input and design the discrete cosine transform channel attention module (DCTCAM). In DCTCAM, we extract global low-frequency features and effectively alleviate misclassification caused by scale changes. The edge features of the image are obtained through the high-frequency feature extraction module (HFEM) by using edge detection operators to enhance the model’s ability to recognize edges. Additionally, no further pooling operations are employed except for a single max pooling layer in the backbone network. In this paper, we design a new gating mechanism to dynamically control the extraction ratio of various frequency features so that it can adapt to input images of different scales. This paper conducts comparative experiments on three high-resolution land cover datasets: GID, Vaihingen, and Potsdam. The new algorithm proposed compares with classic semantic segmentation models such as DANet, PSPNet, SegNet, UNet, DeepLabv3+, FcaNet, BiSeNet, AFN, and FastFCN. The experimental results show that MFAGNet is superior to the comparison models. With no significant increase in computing overhead, MFAGNet obtains MIoU of 69.05%, 73.40%, and 79.42% on the standard data sets GID, Vaihingen, and Potsdam, respectively.

Frequency adaptive fault detection by feature pyramid network with wavelet transform

Fault detection is a key step in seismic structure interpretation. Current research has achieved good results in fault detection by using synthetic training data to train deep-learning models. However, there is an inevitable difference in frequency bandwidth between synthetic training data and real seismic data, which makes it difficult for deep-learning models to obtain ideal fault detection results on real seismic data. To solve this problem, the feature pyramid network (FPN) is introduced to obtain multiscale deep-learning features, which can reduce the impact of seismic data frequency bandwidth differences on fault detection. Then, we apply the multiscale wavelet transform to extract multiscale frequency spectral features of the seismic data and combine them with the multiscale deep-learning features through concatenation operation. Furthermore, the seismic data is decomposed into signals with different frequency bands through the wavelet transform, and we use the energy of these signals as the network weights of multiscale mixed features to further improve the frequency adaptability of our method. Based on these works, we not only improve the fault detection effect in a specific work area but also improve the generalization ability of the deep-learning model in different work areas, thus further promoting the application of deep learning in actual production. Compared with the fault detection results by the traditional deep-learning model U-Net and the traditional FPN on multiple real seismic data and synthetic seismic data, experimental results demonstrate the effectiveness of our method.

Working condition perception for froth flotation based on NSCT multiscale features

Visual features at the froth surface of a flotation cell are closely related to the process condition and performance. Accurate identification of flotation conditions is crucial. However, the multi-scale properties of froth images are not all extracted based on existing perception methods. Considering that multi-scale frequency domain analysis can obtain more informative statistical froth features, a new working condition recognition based on nonsubsampled contourlet transform (NSCT) multiscale features is proposed and successfully validated by industrial froth images taken under different process conditions. NSCT was used to enhance the froth image, including the decomposition and reconstruction, contrast-limited adaptive histogram equalization (CLAHE), and multi-scale features of the enhanced image are extracted and analyzed. The fuzzy binarization method is adopted for extracting the morphological feature. Texture feature parameters are extracted from other high-frequency images. The particle swarm optimization support vector machine is used to recognize the working condition. The results indicate that the proposed algorithms, in particular the multiscale features classification based on NSCT, have the highest classification accuracy (93.33 %), and can accurately and reliably identify the working condition in the actual froth images.

Multi-Scale Frequency Separation Network for Image Deblurring

Image deblurring aims to restore the detailed texture information or structures from the blurry images, which has become an indispensable step in many computer vision tasks. Although various methods have been proposed to deal with the image deblurring problem, most of them treated the blurry image as a whole and neglected the characteristics of different image frequencies. In this paper, we present a new method called multi-scale frequency separation network (MSFS-Net) for image deblurring. MSFS-Net introduces the frequency separation module (FSM) into an encoder-decoder network architecture to capture the low- and high-frequency information of image at multiple scales. Then, a simple cycle-consistency strategy and a sophisticated contrastive learning module (CLM) are respectively designed to retain the low-frequency information and recover the high-frequency information during deblurring. At last, the features of different scales are fused by a cross-scale feature fusion module (CSFFM). Extensive experiments on benchmark datasets show that the proposed network achieves state-of-the-art performance.

Fault Diagnosis for Rolling Bearing of Combine Harvester Based on Composite-Scale-Variable Dispersion Entropy and Self-Optimization Variational Mode Decomposition Algorithm.

Because of the influence of harsh and variable working environments, the vibration signals of rolling bearings for combine harvesters usually show obvious characteristics of strong non-stationarity and nonlinearity. Accomplishing accurate fault diagnosis using these signals for rolling bearings is a challenging subject. In this paper, a novel fault diagnosis method based on composite-scale-variable dispersion entropy (CSvDE) and self-optimization variational mode decomposition (SoVMD) is proposed, systematically combining the nonstationary signal analysis approach and machine learning technology. Firstly, an improved SoVMD algorithm is developed to realize adaptive parameter optimization and to further extract multiscale frequency components from original signals. Subsequently, a CSvDE-based feature learning model is established to generate the multiscale fault feature space (MsFFS) of frequency components for the improvement of fault feature learning ability. Finally, the generated MsFFS can serve as the inputs of the Softmax classifier for fault category identification. Extensive experiments on the vibration datasets collected from rolling bearings of combine harvesters are conducted, and the experimental results demonstrate the more superior and robust fault diagnosis performance of the proposed method compared to other existing approaches.

CWAN: Self-supervised learning for deep grape disease image composition

Grape leaf diseases seriously affect grape yield and directly lead to the loss of the industrial economy. An accurate diagnosis is crucial for effective spraying and control. Presently, the combination of image blending and a deep recognition model is the most effective method for accurately identifying grape leaf diseases. Existing methods of image blending depend primarily on single-channel and multi-scale channel blending, computing and optimizing the Poisson equation, and the iterative optimization of learning-based methods for simple composite results. Although these methods have had some success, they are not able to simultaneously address lesion and leaf gap size, color bleeding, and edge smoothness. In this paper, we propose a novel image blending method named cascade wavelet attention network (CWAN). Specifically, CWAN consists of a cascade wavelet attention (CWA) and a blending autoencoder. First, we build a CWA to decompose the image into a multi-level and multi-scale frequency domain image. Then, the detailed features of the source and target domains are extracted, and the edges are smoothed. Following this, a shared decoder is designed to fuse these features in the corresponding frequency-domain layers, and the result serves as the attention feature. Finally, we design a blending autoencoder with a feature fusion mechanism to seamlessly blend lesions and leaves enhanced by CWA. We also construct a weighted blended loss function to avoid leaf color infiltration into the lesion in the blended image. Comprehensive experiments on two grape leaf disease datasets. Qualitative and quantitative results demonstrate the superiority and effectiveness of our model.