Spatial Frequency Features Research Articles

FFS-Net: Fourier-based segmentation of colon cancer glands using frequency and spatial edge interaction

The morphological features of glands provide a reliable basis for pathologists to diagnose colon cancer correctly. Currently, most methods are limited in their ability to address blurred edges, adhesions, and morphological differences in glands when working in the spatial domain. In this study, a new perspective is proposed to segment glands by utilizing the interaction between Fourier and spatial domain features. Specifically, a Fourier transform is used to analyze the frequency characteristics of pathological images in which the phase component is related to image structural information and the amplitude component contains image intensity information. A Fourier-based colon cancer gland segmentation network (FFS-Net) is proposed, which gradually constructs feature representations in both the spatial and frequency domains through spatial frequency edge interactions. This network comprises spatial subnetworks, frequency subnetworks, and an edge phase fusion module (EPFM). To alleviate the interference of complex background information in feature discrimination, the spatial subnetwork collects high-frequency contour information regarding the glands. Then, the EPFM completes the spatial-frequency feature interaction to add edge-related constraints to the extracted features, enhance edge representation, and obtain global features from the frequency domain, thereby, addressing significant differences in gland morphology. Finally, learnable Gabor filters are introduced to enhance edge texture detail information in the frequency subnetwork, which further improves the segmentation of glands exhibiting blurred edges. The experimental results demonstrate that FFS-Net’s performance on two representative datasets was competitive and highly interpretable.

Depression detection based on the temporal-spatial-frequency feature fusion of EEG

Depression is a prevalent affective psychiatric disorder projected to be the leading contributor to the world’s disease burden by 2030. Due to its high prevalence and low recognition rate, an objective and effective detection method is urgently needed. Deep learning methods based on electroencephalography (EEG) have shown significant potential in depression detection. However, excessive channels can increase redundancy and computational complexity in EEG, while irrelevant channels may reduce accuracy. Additionally, existing models often overlook the complementarity between the temporal-, spatial-, and frequency-domain features of EEG, limiting their detection capabilities. To address these issues, we propose a method that fuses the temporal, spatial, and frequency domain features of EEG to enhance the detection accuracy while eliminating redundant channels. We introduce an EEG channel selection method based on frequency domain weighting that automatically adjusts the channel weights to select the EEG channels that best capture spatial information across the delta, theta, alpha, beta, and gamma bands, thereby optimizing the extraction of spatial-frequency features. In addition, we designed a multiscale spatiotemporal convolutional attention network to extract the spatiotemporal features of EEG. In this network, the multiscale convolutional attention module enhanced the model’s ability to capture spatial features, whereas the temporal trend-aware self-attention module extracted long-term temporal features by analyzing global correlations across different time points. Experimental results on the MODMA dataset show that our method achieved a 97.24% detection accuracy, surpassing current state-of-the-art models. This study offers a novel approach for constructing depression detection models, providing a foundation for future research and application.

Fusing spatial and frequency features for compositional zero-shot image classification

Compositional Zero-Shot Learning (CZSL) is a particular Zero-Shot Learning (ZSL) task that aims to utilize known concepts (e.g., states and objects) to identify novel state-object compositions for Image Classification. Previous works have primarily focused on disentangling concept compositions or exploring the complex interactions between the states and objects while neglecting the critical fact that the inference of many states and compositions is related to different frequency components, which should be analyzed from the global perspective. Therefore, we propose a Spatial-frequency Feature Fusion Network (SFFNet) to introduce a new branch that utilizes a frequency-domain filtering encoder to enhance key frequency components and capture non-local interactions adaptively. Besides, we also find that the widely used backbone in conventional CZSL settings behaves superior in perceiving local features. Thus, we construct a fusion block to combine both strengths to capture the local and non-local information. In addition, the traditional one-hot ground-truth distribution in the training phase does not reflect the accurate relationships between compositions, so we propose a composition-relation based label distribution regularization to encourage the model to actively learn the inner relationships between compositions, and extend this method to construct unseen composition pseudo distribution to further enhance the model’s generalization ability to unseen compositions. Extensive experiments and detailed analysis are conducted on three popular datasets, and the results show that our method can achieve state-of-the-art performance, which reveals its superiority in identifying novel compositions. Code is available at https://github.com/lisuyi/SFFNet_czsl.

Automated diagnosis of schizophrenia based on spatial–temporal residual graph convolutional network

BackgroundSchizophrenia (SZ), a psychiatric disorder for which there is no precise diagnosis, has had a serious impact on the quality of human life and social activities for many years. Therefore, an advanced approach for accurate treatment is required.New methodIn this study, we provide a classification approach for SZ patients based on a spatial–temporal residual graph convolutional neural network (STRGCN). The model primarily collects spatial frequency features and temporal frequency features by spatial graph convolution and single-channel temporal convolution, respectively, and blends them both for the classification learning, in contrast to traditional approaches that only evaluate temporal frequency information in EEG and disregard spatial frequency features across brain regions.ResultsWe conducted extensive experiments on the publicly available dataset Zenodo and our own collected dataset. The classification accuracy of the two datasets on our proposed method reached 96.32% and 85.44%, respectively. In the experiment, the dataset using delta has the best classification performance in the sub-bands.Comparison with existing methodsOther methods mainly rely on deep learning models dominated by convolutional neural networks and long and short time memory networks, lacking exploration of the functional connections between channels. In contrast, the present method can treat the EEG signal as a graph and integrate and analyze the temporal frequency and spatial frequency features in the EEG signal.ConclusionWe provide an approach to not only performs better than other classic machine learning and deep learning algorithms on the dataset we used in diagnosing schizophrenia, but also understand the effects of schizophrenia on brain network features.

Super-Resolution Image Reconstruction Method between Sentinel-2 and Gaofen-2 Based on Cascaded Generative Adversarial Networks

Due to the multi-scale and spectral features of remote sensing images compared to natural images, there are significant challenges in super-resolution reconstruction (SR) tasks. Networks trained on simulated data often exhibit poor reconstruction performance on real low-resolution (LR) images. Additionally, compared to natural images, remote sensing imagery involves fewer high-frequency components in network construction. To address the above issues, we introduce a new high–low-resolution dataset GF_Sen based on GaoFen-2 and Sentinel-2 images and propose a cascaded network CSWGAN combined with spatial–frequency features. Firstly, based on the proposed self-attention GAN (SGAN) and wavelet-based GAN (WGAN) in this study, the CSWGAN combines the strengths of both networks. It not only models long-range dependencies and better utilizes global feature information, but also extracts frequency content differences between different images, enhancing the learning of high-frequency information. Experiments have shown that the networks trained based on the GF_Sen can achieve better performance than those trained on simulated data. The reconstructed images from the CSWGAN demonstrate improvements in the PSNR and SSIM by 4.375 and 4.877, respectively, compared to the relatively optimal performance of the ESRGAN. The CSWGAN can reflect the reconstruction advantages of a high-frequency scene and provides a working foundation for fine-scale applications in remote sensing.

Salient object detection in low-light RGB-T scene via spatial-frequency cues mining

Low-light conditions pose significant challenges to vision tasks, such as salient object detection (SOD), due to insufficient photons. Light-insensitive RGB-T SOD models mitigate the above problems to some extent, but they are limited in performance as they only focus on spatial feature fusion while ignoring the frequency discrepancy. To this end, we propose an RGB-T SOD model by mining spatial-frequency cues, called SFMNet, for low-light scenes. Our SFMNet consists of spatial-frequency feature exploration (SFFE) modules and spatial-frequency feature interaction (SFFI) modules. To be specific, the SFFE module aims to separate spatial-frequency features and adaptively extract high and low-frequency features. Moreover, the SFFI module integrates cross-modality and cross-domain information to capture effective feature representations. By deploying both modules in a top-down pathway, our method generates high-quality saliency predictions. Furthermore, we construct the first low-light RGB-T SOD dataset as a benchmark for evaluating performance. Extensive experiments demonstrate that our SFMNet can achieve higher accuracy than the existing models for low-light scenes.

Surface microtopography evolution of monocrystalline silicon in chemical mechanical polishing

Chemical mechanical polishing (CMP) is a widely utilized technique for achieving ultra-smooth surfaces, yet a comprehensive understanding of the surface microtopography evolution remains elusive due to the intricate chemical and mechanical interactions inherent in chemical mechanical polishing. Accordingly, this study explores the evolution law of different spatial frequency features of surface microtopography of monocrystalline silicon by analyzing the changes in the power spectral density (PSD) curve. More importantly, the evolution mechanism of the chemical mechanical polishing surface micromorphology is revealed through the analysis of the power spectral density under different mechanical and chemical conditions. The results emphasize the noticeable impact of both mechanical and chemical actions on the evolution of different spatial frequency features in surface microtopography. The mechanical action of a large particle, less likely to contact the valley region with a small curvature radius, enhances the smoothing speed of high-spatial frequency roughness. Moreover, the mechanical action of polishing pad demonstrates a greater efficiency in smoothing mid-spatial frequency roughness when utilizing a hard pad. This is attributed to the limited impact of pad hardness on the removal depth of single particles within the reaction layer, rendering it less significant for high-spatial frequency roughness. Upon elevating the chemical reaction rate, a notable acceleration in the reduction of roughness in both mid- and high-spatial frequency regions is observed. Furthermore, insights derived from molecular dynamics (MD) simulations of monocrystalline silicon and quartz glass show that the manner of surface atom removal, whether through single atom removal or cluster removal, exerts a significant effect on the final high-spatial frequency components of the power spectral density. This study contributes valuable insights into the intricate formation mechanism of a smooth surface in chemical mechanical polishing.

CEDFlow: Latent Contour Enhancement for Dark Optical Flow Estimation

Accurately computing optical flow in low-contrast and noisy dark images is challenging, especially when contour information is degraded or difficult to extract. This paper proposes CEDFlow, a latent space contour enhancement for estimating optical flow in dark environments. By leveraging spatial frequency feature decomposition, CEDFlow effectively encodes local and global motion features. Importantly, we introduce the 2nd-order Gaussian difference operation to select salient contour features in the latent space precisely. It is specifically designed for large-scale contour components essential in dark optical flow estimation. Experimental results on the FCDN and VBOF datasets demonstrate that CEDFlow outperforms state-of-the-art methods in terms of the EPE index and produces more accurate and robust flow estimation. Our code is available at: https://github.com/xautstuzfy.

FC-TFS-CGRU: A Temporal-Frequency-Spatial Electroencephalography Emotion Recognition Model Based on Functional Connectivity and a Convolutional Gated Recurrent Unit Hybrid Architecture.

The gated recurrent unit (GRU) network can effectively capture temporal information for 1D signals, such as electroencephalography and event-related brain potential, and it has been widely used in the field of EEG emotion recognition. However, multi-domain features, including the spatial, frequency, and temporal features of EEG signals, contribute to emotion recognition, while GRUs show some limitations in capturing frequency-spatial features. Thus, we proposed a hybrid architecture of convolutional neural networks and GRUs (CGRU) to effectively capture the complementary temporal features and spatial-frequency features hidden in signal channels. In addition, to investigate the interactions among different brain regions during emotional information processing, we considered the functional connectivity relationship of the brain by introducing a phase-locking value to calculate the phase difference between the EEG channels to gain spatial information based on functional connectivity. Then, in the classification module, we incorporated attention constraints to address the issue of the uneven recognition contribution of EEG signal features. Finally, we conducted experiments on the DEAP and DREAMER databases. The results demonstrated that our model outperforms the other models with remarkable recognition accuracy of 99.51%, 99.60%, and 99.59% (58.67%, 65.74%, and 67.05%) on DEAP and 98.63%, 98.7%, and 98.71% (75.65%, 75.89%, and 71.71%) on DREAMER in a subject-dependent experiment (subject-independent experiment) for arousal, valence, and dominance.

Dual-path dehazing network with spatial-frequency feature fusion

With rapid improvement of deep learning, significant progress has been made in image dehazing, leading to favorable outcomes in many methods. However, a common challenge arises as most of these methods struggle to restore intricate details with vibrant colors in complex haze. In response to this challenge, we present a novel dual-path dehazing network with spatial-frequency feature fusion (DDN-SFF) to remove heterogeneous haze. The proposed dual-path network consists of a spatial-domain vanilla path and a frequency-domain frequency-guided path, effectively harnessing spatial-frequency knowledge. To maximize the versatility of the learned features, we introduce a relaxation dense feature fusion (RDFF) module in the vanilla path. This module can skillfully re-exploit features from non-adjacent levels and concurrently generate new features. In the frequency-guided path, we integrate the discrete wavelet transform (DWT) and introduce a frequency attention (FA) mechanism for the flexible handling of specific channels. More precisely, we deploy a channel attention (CA) and a dense feature fusion (DFF) module for low-frequency channels, whereas a pixel attention (PA) and a residual dense block (RDB) module are implemented for high-frequency channels. In summary, the deep dual-path network fuses sub-bands with specific spatial-frequency features, effectively eliminating the haze and restoring intricate details along with rich textures. Extensive experimental results demonstrate the superior performance of the proposed DDN-SFF over state-of-the-art dehazing algorithms.

Deepfake detection based on cross-domain local characteristic analysis with multi-domain transformer

Deepfake videos present a significant challenge in the current media landscape. While current deepfake detection methods demonstrate satisfactory performance, there is still room for improvement in their ability to generalize and detect unseen scenarios, particularly those involving imperceptible cues. This paper introduces a novel multi-modal deepfake detection model named SpectraVisionFusion Transformer (SVFT), which incorporates spatial and frequency domain statistical artifacts to improve generalization performance. The SVFT framework uses two different backbone encoder models to take advantage of both spatial and frequency domain cues in video sequences, along with a decoder and classifier, for common cross-attention and classification, respectively. The spatial domain branch uses a convolutional transformer-based encoder to analyze facial visual features. In contrast, the frequency domain branch employs a language transformer encoder. Additionally, we introduce a weighted feature embedding fusion mechanism that integrates spectral-based statistical feature embeddings and visual cues to achieve a more comprehensive and balanced spatial-frequency feature representation. By coordinately analyzing these modalities, our model exhibits improved detection and generalization capabilities in unseen scenarios. Our proposed SVFT model achieved 92.57% and 80.63% accuracy in extensive cross-manipulation and dataset evaluation, respectively, while surpassing the performance of traditional and single-domain-based approaches.

SFE-Net: Spatial-Frequency Enhancement Network for robust nuclei segmentation in histopathology images

Morphological features of individual nuclei serve as a dependable foundation for pathologists in making accurate diagnoses. Existing methods that rely on spatial information for feature extraction have achieved commendable results in nuclei segmentation tasks. However, these approaches are not sufficient to extract edge information of nuclei with small sizes and blurred outlines. Moreover, the lack of attention to the interior of the nuclei leads to significant internal inconsistencies. To address these challenges, we introduce a novel Spatial-Frequency Enhancement Network (SFE-Net) to incorporate spatial-frequency features and promote intra-nuclei consistency for robust nuclei segmentation. Specifically, SFE-Net incorporates a distinctive Spatial-Frequency Feature Extraction module and a Spatial-Guided Feature Enhancement module, which are designed to preserve spatial-frequency information and enhance feature representation respectively, to achieve comprehensive extraction of edge information. Furthermore, we introduce the Label-Guided Distillation method, which utilizes semantic features to guide the segmentation network in strengthening boundary constraints and learning the intra-nuclei consistency of individual nuclei, to improve the robustness of nuclei segmentation. Extensive experiments on three publicly available histopathology image datasets (MoNuSeg, TNBC and CryoNuSeg) demonstrate the superiority of our proposed method, which achieves 79.23%, 81.96% and 73.26% Aggregated Jaccard Index, respectively. The proposed model is available at https://github.com/jinshachen/SFE-Net.

DONet: Dual-Octave Network for Fast MR Image Reconstruction.

Magnetic resonance (MR) image acquisition is an inherently prolonged process, whose acceleration has long been the subject of research. This is commonly achieved by obtaining multiple undersampled images, simultaneously, through parallel imaging. In this article, we propose the dual-octave network (DONet), which is capable of learning multiscale spatial-frequency features from both the real and imaginary components of MR data, for parallel fast MR image reconstruction. More specifically, our DONet consists of a series of dual-octave convolutions (Dual-OctConvs), which are connected in a dense manner for better reuse of features. In each Dual-OctConv, the input feature maps and convolutional kernels are first split into two components (i.e., real and imaginary) and then divided into four groups according to their spatial frequencies. Then, our Dual-OctConv conducts intragroup information updating and intergroup information exchange to aggregate the contextual information across different groups. Our framework provides three appealing benefits: 1) it encourages information interaction and fusion between the real and imaginary components at various spatial frequencies to achieve richer representational capacity; 2) the dense connections between the real and imaginary groups in each Dual-OctConv make the propagation of features more efficient by feature reuse; and 3) DONet enlarges the receptive field by learning multiple spatial-frequency features of both the real and imaginary components. Extensive experiments on two popular datasets (i.e., clinical knee and fastMRI), under different undersampling patterns and acceleration factors, demonstrate the superiority of our model in accelerated parallel MR image reconstruction.

MSFNet: Multi-scale Spatial-frequency Feature Fusion Network for Remote Sensing Change Detection

MSFNet: Multi-scale Spatial-frequency Feature Fusion Network for Remote Sensing Change Detection

Spatial-Frequency Feature Fusion Network for Lightweight and Arbitrary-Sized JPEG Steganalysis

Spatial-Frequency Feature Fusion Network for Lightweight and Arbitrary-Sized JPEG Steganalysis

CSFwinformer: Cross-Space-Frequency Window Transformer for Mirror Detection.

Mirror detection is a challenging task since mirrors do not possess a consistent visual appearance. Even the Segment Anything Model (SAM), which boasts superior zero-shot performance, cannot accurately detect the position of mirrors. Existing methods determine the position of the mirror under hypothetical conditions, such as the correspondence between objects inside and outside the mirror, and the semantic association between the mirror and surrounding objects. However, these assumptions do not apply to all scenarios. For instance, there may be no corresponding real objects to the reflected objects in the scene, or it may be challenging to extract meaningful semantic associations in complex scenes. On the other hand, humans can easily recognize mirrors through the specular texture caused by materials. To mine mirror features in more general scenes, we propose a Cross-Space-Frequency Window Transformer (CSFwinformer) to extract spatial and frequency features for texture analysis. Specifically, we design a Spatial-Frequency Window Alignment module (SFWA) to calculate spatial-frequency feature affinities and learn the difference between mirror and non-mirror textures. We then propose a Dilated Window Attention (DWA) to extract global features to complement the limitation of window alignment. Besides, we propose a Cross-Modality Context Contrast module (CMCC) to fuse cross-modality features and global features, which enables information flow between different windows to take full advantage of cross-modality information. Extensive experiments show that our method performs favorably against state-of-the-art methods on three mirror detection benchmarks and significantly improved SAM performance on mirror detection. The code is available at https://github.com/wangsen99/CSFwinformer.

Painterly Image Harmonization in Dual Domains

Image harmonization aims to produce visually harmonious composite images by adjusting the foreground appearance to be compatible with the background. When the composite image has photographic foreground and painterly background, the task is called painterly image harmonization. There are only few works on this task, which are either time-consuming or weak in generating well-harmonized results. In this work, we propose a novel painterly harmonization network consisting of a dual-domain generator and a dual-domain discriminator, which harmonizes the composite image in both spatial domain and frequency domain. The dual-domain generator performs harmonization by using AdaIN modules in the spatial domain and our proposed ResFFT modules in the frequency domain. The dual-domain discriminator attempts to distinguish the inharmonious patches based on the spatial feature and frequency feature of each patch, which can enhance the ability of generator in an adversarial manner. Extensive experiments on the benchmark dataset show the effectiveness of our method. Our code and model are available at https://github.com/bcmi/PHDNet-Painterly-Image-Harmonization.

Fine-Grained Visual Categorization: A Spatial–Frequency Feature Fusion Perspective

Fine-grained visual categorization is a challenging issue owing to high intra-class and low inter-class variances. Classical approaches rely on pre-trained models or many fine annotations. In this paper, we observe that spatial and frequency information provides distinct image views, and propose a new spatial–frequency feature fusion (SFFF) perspective to handle this challenging issue. Specifically, we design a heterogeneous feature extraction loss function, construct a global and local fusion SFFF network, and propose an importance-sparsity selection strategy. For feature extraction, we focus on the frequency domain feature learning network, extract fine-grained features, and achieve feature complementarity. For feature selection, we propose importance ranking and sparse regularity to constrain spatial–frequency features. For feature fusion, we design a spatial–frequency loss and an inter-layer switching strategy to achieve local-global collaboration. Comparative experiments were performed on popular fine-grained datasets and classic datasets such as CUB200-2011, Stanford Cars, Stanford Dogs, FGVC-Aircraft, and CIFAR100. The effectiveness and outstanding performance of SFFF are confirmed by comparisons with more than 40 state-of-the-art fine-grained categorization methods. Ablation studies and visualizations are provided to facilitate an understanding of our approach.

Privacy Preserving Contraband Detection Using a Millimeter-Wave Dynamic Antenna Array

We present a privacy preserving technique for contraband detection using a real-time 75 GHz rotational dynamic antenna array that samples only a reduced set of Fourier-domain information to prevent image reconstruction of the screened subject. Complex objects such as handguns have sharp edges that correspond to sharp spatial frequency features in the Fourier domain. These features are dependent on the orientation of the object and manifest at millimeter-wave frequencies through conditions such as being concealed under clothing. The technique combines a dynamically rotated active interferometer and simple arithmetic feature extraction approach supporting a threshold-based classifier to identify when a handgun is concealed under clothing. The active interferometer technique utilizes noise transmitters to illuminate the test subject to produce thermal-like radiation, enabling Fourier-domain signal sampling, while the rotational dynamics support sampling that spans a circle in the 2-D Fourier domain to retrieve sharp-edge responses induced by concealed objects. We experimentally demonstrate the detection of concealed objects under clothing in a laboratory environment using a 75 GHz dynamic antenna array based on simple heuristic features extracted from statistics of the measured responses. The classifier is evaluated based on processing one or more subsequent measurements, showing a classification accuracy of 0.908 and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F$</tex-math> </inline-formula> 1-score of 0.916 using four consecutive measurements.

Bi-Branch Vision Transformer Network for EEG Emotion Recognition

Electroencephalogram (EEG) signals have emerged as an important tool for emotion research due to their objective reflection of real emotional states. Deep learning-based EEG emotion classification algorithms have made preliminary progress, but existing models struggle with capturing long-range dependence and integrating temporal, frequency, and spatial domain features to limit their classification ability. To address these challenges, this study proposes a Bi-branch Vision Transformer-based EEG emotion recognition model, Bi-ViTNet, that integrates spatial-temporal and spatial-frequency feature representations. Specifically, Bi-ViTNet is composed of spatial-frequency feature extraction branch and spatial-temporal feature extraction branch, which can fuse spatial-frequency-temporal features in a unified framework. Each branch is composed of Linear Embedding and Transformer Encoder, which is used to extract spatial-frequency features and spatial-temporal features. Finally, fusion and classification are performed by the Fusion and Classification layer. Experiments on SEED and SEED-IV datasets demonstrate that Bi-ViTNet outperforms state-of-the-art baselines.