Frequency-domain Reconstruction Research Articles

A dual descriptor combined with frequency domain reconstruction learning for face forgery detection in deepfake videos

Conventional face forgery detectors have primarily relied on image artifacts produced by deepfake video generation models. These methods have performed well when the training and test sets were derived from the same deepfake algorithm, but accuracy and generalizability remain a challenge for diverse datasets. In this study, both supervised and unsupervised approaches are proposed for more accurate detection in in-domain and cross-domain experiments. Specifically, two descriptors are introduced to extract rich information in the spatial domain to achieve higher accuracy. A frequency domain reconstruction module is then included to expand the representation space for facial features. A reconstruction method based on an auto-encoder was also applied to obtain a frequency domain coding vector. In this process, reconstruction learning was sufficient for extracting unknown information, while a combination with classification learning provided essential high-frequency pixel differences between real and fake samples, thus facilitating forgery identification. A series of validation experiments with large-scale benchmark datasets demonstrated that the proposed technique was superior to existing methods.

Deep-learning approach to stratified reconstructions of tissue absorption and scattering in time-domain spatial frequency domain imaging.

The conventional optical properties (OPs) reconstruction in spatial frequency domain (SFD) imaging, like the lookup table (LUT) method, causes OPs aliasing and yields only average OPs without depth resolution. Integrating SFD imaging with time-resolved (TR) measurements enhances space-TR information, enabling improved reconstruction of absorption () and reduced scattering () coefficients at various depths. To achieve the stratified reconstruction of OPs and the separation between and , using deep learning workflow based on the temporal and spatial information provided by time-domain SFD imaging technique, while enhancing the reconstruction accuracy. Two data processing methods are employed for the OPs reconstruction with TR-SFD imaging, one is full TR data, and the other is the featured data extracted from the full TR data (, continuous-wave component, , mean time of flight). We compared their performance using a series of simulation and phantom validations. Compared to the LUT approach, utilizing full TR, and datasets yield high-resolution OPs reconstruction results. Among the three datasets employed, full TR demonstrates the optimal accuracy. Utilizing the data obtained from SFD and TR measurement techniques allows for achieving high-resolution separation reconstruction of and at different depths within 5mm.

Spectral feature extraction of rocket exhaust plume using spectral proper orthogonal decomposition

The spectral characterization of flow-field parameters provides a new perspective for understanding the spatiotemporal evolution of unsteady supersonic exhaust plumes and for extracting typical structures. In this study, a large-eddy simulation is performed to calculate the three-dimensional unsteady supersonic plume flow field of rocket engines, and a spectral proper orthogonal decomposition (SPOD) method with a spatiotemporal separation is established. This approach is used to extract the coherent structural features of the unsteady exhaust plume flow field and analyze the mode space structure at different frequencies. The three-dimensional reconstruction and denoising of the exhaust plume flow-field parameters can be achieved via the frequency- and time-domain reconstructions of the SPOD algorithm and oblique projection method, respectively. The ground rocket exhaust plume of ballistic evaluation motor-II is analyzed. The results indicate that the SPOD method can effectively extract the single-frequency mode structure of the reactive supersonic flow field, and that low-order behavior appears in the m = 0 and m = 1 azimuth modes. The potential core exhibits a high-frequency wave-packet structure that is affected by shock waves and shear layers. Time-domain reconstruction based on the oblique projection method facilitates the capture of the dynamic characteristics of the flow field. For the first-order SPOD mode, the frequency- and time-domain reconstruction errors are 3.3% and 1.5%, respectively. The frequency-domain reconstruction method exhibits a 4% improvement in denoising ability compared to low-pass filters. This study provides a novel method for the spectral characterization and spatiotemporal feature extraction of supersonic exhaust plume flow fields.

Infrared colorization with cross-modality zero-shot learning

Existing infrared colorization methods based on image-to-image translation in a GAN framework require paired image data for network learning and are limited by data quality. They are also restricted in their colorization ability to perform spectral translation between infrared bands, specifically, long-wave infrared colorization methods cannot be directly applied to the near-infrared modality. This paper proposes a novel infrared colorization algorithm that leverages similarities in high-frequency features between visible and infrared images. The method achieves infrared colorization through frequency domain feature decoupling and reconstruction. By decoupling high-frequency and low-frequency features in the frequency domain, it retains similar high-frequency features and reconstructs removed low-frequency features. The network is trained using visible images, enabling cross-modality zero-shot learning, eliminating the need for infrared datasets during training. During inference, the low-frequency information of the infrared image’s frequency domain is removed, and the visible low-frequency information is supplemented using the trained network’s reconstruction capability, resulting in the desired coloring effect. The method is equipped with multi-modality cross-spectral colorization capability and performs well on multiple infrared spectral colorization tasks. The experimental results fully demonstrate the excellent cross-modality adaptability and broad spectral colorization capability of the method.

Fault detection and analysis for wheelset bearings via improved explicit shift-invariant dictionary learning

The wheelset bearing is an indispensable part of the high-speed train, and monitoring its service performance is a concern of many researchers. Effective extraction of those impulse signals induced by the defects on the bearing elements is the key to fault detection and behaviour analysis. However, the presence of considerable noise and irrelevant components brings difficulties to extracting the wheelset bearing fault impulse signals from the measured vibration signals. This paper proposes an improved explicit shift-invariant dictionary learning (IE-SIDL) method to address this issue. Based on the shift-invariant characteristics of the wheelset bearing fault impulse signal in the time-domain, the circulant matrix is used to construct a shift-invariant dictionary and explicitly characterize the fault impulses at any time. To improve the efficiency of dictionary learning, a method of three flips is introduced to realize fast dictionary construction, and the frequency-domain reconstruction property of the circulant matrix is employed to quickly update the dictionary. Besides, an indicator-guided subspace pursuit (SP) method based on the sparsity of envelope spectrum (SES) is adopted for the sparse coding to improve sparse solution accuracy and adaptation. The effectiveness of the IE-SIDL method is proved through the simulated and experimental signals. The results demonstrate that the improved dictionary learning method has an excellent capacity in extracting fault impulse signal of the wheelset bearings, and the good time- and frequency-domain characteristics of the processed signals facilitate fault detection and behaviour analysis.

DPNet: Detail-preserving image deraining via learning frequency domain knowledge

The quality of images captured in rainy days is severely degraded, which affects the accuracy of subsequent computer vision tasks. Recently, many deep learning-based methods have demonstrated superior performance for single image deraining. However, there are still many issues left. Since real-world rain images and their corresponding ground truths are difficult to collect, models trained on limited data may lead to overfitting. Meanwhile, although many methods can remove part of the rain streaks, most of them cannot reconstruct precise edges and textures. For the first issue, we use the transfer learning approach. Loading pre-trained parameters trained on the ImageNet enables the network to have robust feature representation, which improves the generalization of the network. For the second issue, we restore clear details by making full use of the frequency domain information of the image. Specifically, we design a novel frequency domain residual block (FRDB) and use an efficient fusion strategy in FRDB to fuse spatial and frequency domain features. Then, we propose a frequency domain reconstruction loss function (FDR loss) to restore details by reducing the differences in high-frequency space. Finally, a simple detail enhancement attention module (DEAM) is used to further enhance the image details. Extensive experimental results demonstrate that our DPNet has superior performance on both synthetic and real data. Furthermore, we verify the effectiveness of our method on downstream computer vision tasks. The source codes will be open at https://github.com/noxsine/DPNet.

DVS-Net: Dual-domain Variable Splitting Network for Accelerated Parallel MRI Data.

Parallel imaging is an important method to accel-erate the acquisition of magnetic resonance imaging data, which can shorten the breath-hold times and reduce motion artifacts. In this paper, we propose a joint frequency domain and image domain (dual-domain) reconstruction method by introducing the full sampling condition for the undersampled multi-coil MR data. The motivation is that the dual domain method can provide more information for accurate image reconstruction. An efficient iterative algorithm is developed based on the variable splitting technique and alternating direction method of multipliers, which is unrolled into an end-to-end trainable deep neural network. We evaluate the proposed network on complex valued multi-coil knee images for both 6-fold and 8-fold acceleration factors, and compare with both variational and deep learning based reconstruction algorithms. The numerical results demonstrate that our method provides better reconstruction accuracy and perceptual quality by making using of the dual domain information. Clinical relevance: This improves the reconstruction quality for accelerated parallel MRI data both visually and quantitatively.

Optimization and assessment of blade tip timing probe layout with concrete autoencoder and reconstruction error

Blade tip timing (BTT) is a promising non-contact measurement method for blade vibration monitoring. The limited number of probes leads to the inherent under-sampled problem of BTT signal. The quality of the analysis result of under-sampled signal depends on the probe layout. However, the application of existing probe layout optimization methods requires domain knowledge about blade vibration. Additionally, the lack of comparisons of probe layouts generated by different probe layout optimization methods makes it difficult to evaluate the performance of different methods. In this paper, we proposed a new probe layout optimization method based on a concrete autoencoder to reduce the reliance on the domain knowledge. The probe layouts were compared using an independently trained reconstructor in the time domain. And the spectral distance was defined to evaluate the performance of different probe layouts in the frequency domain. The validation results showed that the proposed method has superior time domain reconstruction performance and decent frequency domain reconstruction performance compared to other methods. The most informative and representative probe layout can be effectively selected by the proposed method.

3-D ultrasonic image reconstruction in frequency domain using a virtual transducer model

In ultrasonic non-destructive testing, image reconstruction is essential to restore the diffracted ultrasound signals to improve the lateral resolution of images. Some reconstruction methods, like DAS-based synthetic aperture imaging, are inefficient, especially for reconstructing three-dimensional (3-D) images. Other methods do not provide high-resolution results, because they neglect the distortion effect introduced by transducer geometry. To overcome these disadvantages, we propose a 3-D ultrasonic image reconstruction method based on synthetic aperture wavenumber algorithm. It considers wave diffraction and transducer geometry effects, and can refocus the reflectors even in non-focal zone, which suits for large depth range imaging. This method builds a virtual transducer model in frequency domain by treating the focused transducer as a virtual planar transducer on its focal plane. In addition, the method uses non-uniform fast Fourier transform and deconvolution operation to achieve the 3-D image reconstruction, which has remarkably improved the efficiency and accuracy. According to the experimental results, the lateral resolution of an image reconstructed by the proposed method can reach 290.2 μm, exceeding the lateral resolution limitation of the 15 MHz focused transducer (523.24 μm). Furthermore, the proposed method only takes 0.744 s to reconstruct a 3-D image with 1000×100×100 pixels, while the time domain SAFT takes about 1163.8 s. It shows the potential for real-time 3-D imaging under advanced hardware.

In Vivo Three-Dimensional Raster Scan Optoacoustic Mesoscopy Using Frequency Domain Inversion.

Optoacoustic signals are typically reconstructed into images using inversion algorithms applied in the time-domain. However, time-domain reconstructions can be computationally intensive and therefore slow when large amounts of raw data are collected from an optoacoustic scan. Here we considered a fast weighted ω-k (FWOK) algorithm operating in the frequency domain to accelerate the inversion in raster-scan optoacoustic mesoscopy (RSOM), while seamlessly incorporating impulse response correction with minimum computational burden. We investigated the FWOK performance with RSOM measurements from phantoms and mice in vivo and obtained 360-fold speed improvement over inversions based on the back-projection algorithm in the time-domain. This previously unexplored inversion of in vivo optoacoustic data with impulse response correction in frequency domain reconstructions points to a promising strategy of accelerating optoacoustic imaging computations, toward video-rate tomography.

Magnetic resonance imaging reconstruction algorithm under complex convolutional neural network in diagnosis and prognosis of cerebral infarction.

This study was to explore the application value of magnetic resonance imaging (MRI) image reconstruction model based on complex convolutional neural network (CCNN) in the diagnosis and prognosis of cerebral infarction. Two image reconstruction methods, frequency domain reconstruction network (FDRN) and image domain reconstruction network (IDRN), were introduced based on the CCNN algorithm. In addition, they were integrated to form two new MRI image reconstruction models, namely D-FDRN and D-IDRN. The peak signal to noise ratio (PSNR) value and structural similarity index measure (SSIM) value of the image were compared and analyzed before and after the integration. The MRI images of patients with cerebral infarction in the dataset were undertaken as the data source, the average diffusion coefficient (DCavg) and apparent diffusion coefficient (ADC) values of different parts of the MRI image were measured, respectively. The correlation of the vein abnormality grading (VABG) to the infarct size and the degree of stenosis of the responsible vessel was analyzed in this study. The results showed that the PSNR and SSIM values of the MRI reconstructed image of the D-IDRN algorithm based on the CCNN algorithm in this study were higher than those of other algorithms. There was a positive correlation between the VABG and the infarct size (r = 0.48 and P = 0.002), and there was a positive correlation between the VABG the degree of stenosis of the responsible vessel (r = 0.58 and P < 0.0001). The ADC value of the central area of the infarct on the affected side was significantly greatly lower than that of the normal side (P < 0.01), and the DCavg value of the central area of the infarct was much lower in contrast to the normal side (P < 0.05). It indicated that an image reconstruction algorithm constructed in this study could improve the quality of MRI images. The ADC value and DCavg value changed in the infarct central area could be used as the basis for the diagnosis of cerebral infarction. If the vein was abnormal, the patient suffered from severe vascular stenosis, large infarction area, and poorer prognosis.

Rapid Image Reconstruction of Structured Illumination Microscopy Directly in the Spatial Domain.

Super-resolution structured illumination microscopy (SIM) routinely performs image reconstruction in the frequency domain using an approach termed frequency-domain reconstruction (FDR). Due to multiple Fourier transforms between the spatial and frequency domains, SIM suffers from low reconstruction speed, constraining its applications in real-time, dynamic imaging. To overcome this limitation, we developed a new method for SIM image reconstruction, termed spatial domain reconstruction (SDR). SDR is intrinsically simpler than FDR, does not require Fourier transforms and the theory predicts it to be a rapid image reconstruction method. Results show that SDR reconstructs a super-resolution image 7-fold faster than FDR, producing images that are equal to either FDR or the widely-used FairSIM. We provide a proof-of-principle using mobile fluorescent beads to demonstrate the utility of SDR in imaging moving objects. Consequently, replacement of the FDR approach with SDR significantly enhances SIM as the desired method for live-cell, instant super-resolution imaging. This means that SDR-SIM is a “What You See Is What You Get” approach to super-resolution imaging.

Modeling the recoveryof the optoacoustic image of oxygenated erythrocyte

Optoacousticimaging makes it possible to detect the generated ultrasonic waves in the medium under study, while absorbing pulsed laser radiation based on the optoacoustic effect. This method combines the advantages of spectral optical contrast and scalable acoustic resolution at depths from millimeter to centimeter and is gradually becoming a practical tool for many biomedical applications. Optoacoustic imaging is a hybrid, non-invasive technique.In recent years, much attention has been paid to the restoration of optoacoustic images, in particular, various reconstruction methods are being developed, such as rear projection, frequency do-main reconstruction, time reversal and model-based reconstruction. Although these methods are based on various propagation theories, they have relatively simple implementations for im-age reconstruction in homogeneous environments. However, in the case of heterogeneous lay-ered media, the existing propagation models need to be modified to take into account various acoustic effects at the interface of the layers, which complicates the reconstruction process. To eliminate distortions caused by diffraction of acoustic waves, the method of moving a virtu-al detector is used for image reconstruction. The proposed method can be used to obtain imag-es in layered inhomogeneous media.

Compressive imaging based on multi-scale modulation and reconstruction in spatial frequency domain* *Project supported by the National Natural Science Foundation of China (Grant Nos. 61601442, 61605218, and 61575207), the National Key Research and Development Program of China (Grant No. 2018YFB0504302), and the Youth Innovation Promotion Association of the Chinese Academy of Sciences (Grant Nos. 2015124 and 2019154).

Imaging quality is a critical component of compressive imaging in real applications. In this study, we propose a compressive imaging method based on multi-scale modulation and reconstruction in the spatial frequency domain. Theoretical analysis and simulation show the relation between the measurement matrix resolution and compressive sensing (CS) imaging quality. The matrix design is improved to provide multi-scale modulations, followed by individual reconstruction of images of different spatial frequencies. Compared with traditional single-scale CS imaging, the multi-scale method provides high quality imaging in both high and low frequencies, and effectively decreases the overall reconstruction error. Experimental results confirm the feasibility of this technique, especially at low sampling rate. The method may thus be helpful in promoting the implementation of compressive imaging in real applications.

3D steerable pyramid decomposition and its application in seismic data processing and interpretation

The application of 3D multi-scale geometric analysis in seismic data processing and interpretation is still in the preliminary stage. The principle of 3D Steerable Pyramid decomposition and reconstruction in frequency domain is analyzed in detail. Spherical low- pass filters and steerable filters are adopted to realize multi-scale and multi-directional decomposition of 3D Steerable Pyramid, and applied to the actual 3D seismic data for random noise attenuation, detection of faults, channels and sand bodies by selecting weighting function in the process of reconstruction. The application of actual 3D seismic data shows that: 1) When the weights are all equal to 1, perfect reconstruction is achieved, the random noise can be removed by threshold method in the decomposition domain of 3D Steerable Pyramid, the denoising results are clean, and the continuity of the event, the characteristics of the reflected wave group and the breakpoint are clearer; 2) Weighting reconstruction can enhance some features. Compared with amplitude and coherent attribute analysis, the data weighting reconstructed by 3D Steerable Pyramid decomposition not only shows faults and channels completely and meticulously, but also presents an even more impressive picture of sand body’s boundary, thus achieving the optimal application effect.

Pre-migration: A General Extension for Photoacoustic Imaging Reconstruction

In recent years, photoacoustic image reconstruction has received extensive attention. Various reconstruction methods, such as back-projection, frequency domain reconstruction, time reversal, and model-based reconstructions have been developed. Although these methods are based on different propagation theories, they have relatively simple implementations when reconstructing images on homogenous media. However, in cases of heterogeneous layered media, like photoacoustic transcranial imaging, the propagation models have to be modified to accommodate various acoustic effects at layer interface, which complicates the reconstruction process. In this article, we propose an algorithm extension called pre-migration to first convert the reconstruction problems that apply to homogeneous media. From there, the sources can be rebuilt again using classical reconstruction methods. Pre-migration does not require classical reconstructions to be modified. It only preadjusts the sensor position via wave extrapolation. It also simplifies a focused transducer model as a point-like sensor. Based on simulation and experiment results, pre-migration can be integrated into almost all classical reconstruction algorithms and successfully solves reconstruction problems when imaging through heterogeneous media. In our experiments, it takes less than 20% of the total computational time during the entire reconstruction process.

Iterative technique for optimizing two-channel: Quadrature mirror, bi-orthogonal and graph filter bank

An iterative technique is proposed to optimally design the two-channel linear phase (LP) FIR quadrature mirror filter bank (2c-LP-FIR-QMFB). In each iteration, first, the frequency-domain reconstruction error function of the 2c-LP-FIR-QMFB is approximately converted into a convex function, by approximating all its sub-functions using first order Taylor series expansion. Then, the approximated reconstruction error function is minimized as the optimization objective function while the frequency selectivity performance metric function of the sub-filter is taken as the constraint function. Further, the optimal step-size optimization problem for each iteration is solved using a similar iterative strategy. Additionally, the proposed technique can also be utilized to optimize the coefficients of a class of structurally perfect reconstruction two-channel LP FIR bi-orthogonal filter bank, as well as a class of two-channel graph filter bank. Simulation results reveal that the proposed technique outperforms the previously existing techniques in terms of the frequency-domain reconstruction error and/or the frequency selectivity performance metric of the sub-filter. Particularly, for the graph filter bank, the reconstruction error of the proposed technique is only (3.1e-11)% of its previous counterpart. The proposed technique might also be useful to optimize the coefficients of other classes of two-channel traditional signal or graph signal filter banks.

Photoacoustic tomography with direction dependent data: an exact series reconstruction approach

Photoacoustic image reconstruction often assumes that the restriction of the acoustic pressure on the detection surface is given. However, commonly used detectors often have a certain directivity and frequency dependence, in which case the measured data are more accurately described as a linear combination of the acoustic pressure and its normal derivative on the detection surface. In this paper, we consider the inverse source problem for data that are a combination of the acoustic pressure and its normal derivative. For the special case of a spherical detection geometry we derive exact frequency domain reconstruction formulas. We present numerical results showing the robustness and validity of the derived formulas. Moreover, we compare several different combinations of the pressure and its normal derivative showing that used measurement model significantly affects the recovered initial pressure.

Sparsity-based photoacoustic image reconstruction with a linear array transducer and direct measurement of the forward model.

Photoacoustic (PA) imaging is an emerging imaging technique for many clinical applications. One of the challenges posed by clinical translation is that imaging systems often rely on a finite-aperture transducer rather than a full tomography system. This results in imaging artifacts arising from an underdetermined reconstruction of the initial pressure distribution (IPD). Furthermore, clinical applications often require deep imaging, resulting in a low-signal-to-noise ratio for the acquired signal because of strong light attenuation in tissue. Conventional approaches to reconstruct the IPD, such as back projection and time-reversal, do not adequately suppress the artifacts and noise. We propose a sparsity-based optimization approach that improves the reconstruction of IPD in PA imaging with a linear array ultrasound transducer. In simulation studies, the forward model matrix was measured from k-Wave simulations, and the approach was applied to reconstruct simulated point objects and the Shepp-Logan phantom. The results were compared with the conventional back projection, time-reversal approach, frequency-domain reconstruction, and the iterative least-squares approaches. In experimental studies, the forward model of our imaging system is directly measured by scanning a graphite point source through the imaging field of view. Experimental images of graphite inclusions in tissue-mimicking phantoms are reconstructed and compared with the back projection and iterative least-squares approaches. Overall these results show that our proposed optimization approach can leverage the sparsity of the PA images to improve the reconstruction of the IPD and outperform the existing popular reconstruction approaches.

Paley–Wiener Characterization of Kernels for Finite-Rate-of-Innovation Sampling

Exact reconstruction of finite-rate-of-innovation signals can be achieved by employing customized sampling kernels that satisfy certain frequency-domain properties. We impose compact support in time as an additional constraint. Considering frequency-domain reconstruction, we derive conditions for admissible sampling kernels and corresponding sampling rates. Our constructive kernel design methodology is based on the Paley–Wiener theorem for compactly supported functions. The new kernels satisfy generalized Strang-Fix conditions and have specific polynomial-modulated-exponential-reproducing properties. Unlike exponential splines, which have a support that is directly proportional to the number of exponentials they can generate, the proposed kernels have a support that is independent of that number. To analyze noise robustness, we consider a special member of the class that has a sum-of-modulated splines (SMS) form in the time domain and optimize its parameters to minimize the noise variance. The sum-of-sincs (SoS) kernel reported in the literature is an instance of this construction. In noise robustness analysis, SMS kernels show improvement in mean-squared error (MSE) compared with the state-of-the-art alternatives. In continuous-time noise, the improvement in MSE is about 2 dB for low signal-to-noise ratio (SNR) and 7 dB for high SNR. In the case of discrete white Gaussian noise, the MSE is lower by as much as 25 dB by using a higher-order SMS kernel compared with the SoS kernel for SNRs in the range of 10–15 dB.