Bicubic Research Articles

Data-driven Prediction of Underwater Navigation Fitness Zone Classification

This article presents a pioneering approach to enhancing underwater navigation accuracy and stealth through the utilization of gravity anomaly data. Developed by researchers from leading universities in China, the study aims to revolutionize autonomous marine navigation systems in unknown sea areas. By integrating advanced data processing, feature extraction, model construction, and optimization techniques, a cutting-edge navigation system has been formulated, showcasing remarkable improvements in navigation precision.The core innovation lies in leveraging bicubic interpolation to refine data resolution, enabling a meticulous evaluation of regional suitability and segmentation of sea zones. Subsequently, a gradient-based suitability prediction model is constructed, which is validated through migration prediction, underscoring its effectiveness and reliability. This research not only addresses the limitations of traditional terrain matching navigation methods, plagued by data accuracy and resolution constraints, but also paves the way for advancements in oceanic exploration and technological competitiveness.The introduction of gravity anomaly data, coupled with machine learning and visualization tools, marks a significant step forward in the development of adaptive navigation zones. This innovative system holds immense potential to enhance the autonomy, accuracy, and concealment of underwater vehicles, thereby facilitating safer and more efficient oceanic missions in the future.

Integrating hydrological knowledge into deep learning for DEM super-resolution

Deep learning-based super-resolution methods have been successfully applied to digital elevation model (DEM) downscaling studies by designing structures and loss functions of the model. However, little attention has been paid to the design of super-resolution models that can maintain the hydrological characteristics of the DEM, which is important for hydrological studies. This study introduces a super-resolution model that integrates hydrologic knowledge (HKSRCGAN), with the aim to effectively maintain topographic features as well as the hydrologic connectivity of the DEM. The hydrological knowledge derived from surface flow direction and hydrological features are integrated into a deep learning algorithm to guide model training. The 30 m spatial resolution FABDEM is used to demonstrate the utility of the proposed method. Results show that the HKSRCGAN outperforms the bicubic interpolation, SRCNN, SRGAN, SRResNet and TfaSR methods in reducing topographic errors and maintaining hydrologic characteristics. In the test area, the entropy difference analysis shows that the DEM generated by HKSRCGAN is similar to the information contained in the reference DEM. Furthermore, super-resolution models integrating hydrological knowledge are valuable for modeling terrain primarily shaped by gravity and surface water flows. In the future, deep learning-based models integrating hydrologic knowledge are expected to be applied in DEM upscaling to maintain consistent hydrological characteristics.

Increased virtual resolution for sub-pixel displacement algorithm optimization in digital image correlation for AISI 1020 steel

The development of the digital image correlation (DIC) technique has required the use of cameras with increasingly higher resolution, as well as requiring complex sub-pixel tracking algorithms with exponentially increasing computational cost. This article proposes a DIC algorithm based on the correlation coefficient curve-fitting method, minimizing SSD or ZNSSD correlation criteria, with the help of increasing the virtual resolution of the images through bicubic interpolation, for obtain an accurate, robust and reliable algorithm according to the highest computational efficiency. The developed algorithm was virtually validated using synthetic images and experimentally using AISI 1020 steel samples test in tension, performed with strain gauges. The results showed a reduction in the error observed in synthetic images with increased magnification and good agreement with the deformation data obtained using the conventional extensometry technique.

Outcomes of CEM43 in Predicting Thermal Damage Induced by Focal Laser Ablation in Controlled Ex Vivo Experiments: A Comparison to Histology and MRI.

Focal laser ablation (FLA) serves as a targeted therapy for prostate cancer (PCa). Clinical studies have demonstrated significant variations in ablation volumes with consistent fiber configurations. Consequently, a prediction model is needed for the safe application of FLA in treating PCa. This study aimed to evaluate the reproducibility of FLA-induced temperature profiles in controlled ex vivo experiments using clinical laser treatment protocols. Additionally, it sought to examine the effectiveness of the CEM43 model in predicting the zone of irreversible damage (ZID) and to compare these findings with outcomes derived from the Arrhenius model. Freshly excised postmortem human prostate and porcine liver specimens were used for controlled ex vivo ablation. Tissues were secured in a Perspex sample holder for precise placement of the laser fiber and thermocouples. FLA was conducted with a 1064-nm Nd:YAG laser at 3 W in continuous-wave mode for 10 min. Pre- and post-FLA 3D T1-weighted 7 T MRI scans were obtained to assess the treatment area. Whole-mount hematoxylin and eosin histological slides were prepared and digitized. On histology, the ZID was defined as the total of vaporized, carbonized, and coagulated tissue. A 2D thermal development map was created from temperature data, using bi-cubic interpolation. The cumulative equivalent thermal isoeffect dose at 43°C in minutes (CEM43) model was applied to predict the ZID, with 240 equivalent minutes (240-CEM43) used as the damage threshold. Additionally, the Arrhenius thermal model was used for comparison of CEM43 results. Predicted ZIDs were compared to MRI and histology. FLA treatment was performed on ex vivo human prostate samples (n = 2) and porcine liver specimens (n = 5). For human prostate tissue, FLA did not result in an identifiable ZID upon histological macroscopic examination or a lesion on MRI. Ex vivo porcine liver samples showed a clearly demarcated oval-shaped hyperintense lesion surrounding the laser fiber tip on post-FLA MRI. The MRI lesion (range 1.6-2.1 cm2) corresponded with the shape and location of the ZID on histology, but was smaller (median 1.7 vs. 3.2, p = 0.02). Histological examination of porcine liver samples revealed ZIDs ranging from 2.1 to 4.1 cm2, whereas 240-CEM43-predicted ZIDs ranged from 3.3 to 3.8 cm2. Although the median 240-CEM43-predicted ZID was not significantly larger than the histology ZID (3.8 vs. 3.2 cm2, p = 0.22), it tended to overpredict the histological results in most experiments. The median Arrhenius-predicted ZID was similar to the histologicalZID (3.2 vs. 3.2 cm2, p = 0.56), but varied in size when comparing individual experiments (range 2.5-3.2 cm2). FLA on ex vivo human prostate showed no thermal damage on histopathology or MRI. Ex vivo porcine liver FLA resulted in identifiable ZID on histology and lesions on MRI. 240-CEM43 generally overestimated the ZID and had less variability compared to histology. Results from the Arrhenius model were in better agreement with the histology findings, but still did not predict the individual FLA-induced histological thermal damage. Inter-experiment ZID variability underlines the need for developing a more comprehensive predictive dosimetry model for FLA in PCa treatment.

Stereo Bi-Telecentric Phase-Measuring Deflectometry.

Replacing the endocentric lenses in traditional Phase-Measuring Deflectometry (PMD) with bi-telecentric lenses can reduce the number of parameters to be optimized during the calibration process, which can effectively increase both measurement precision and efficiency. Consequently, the low distortion characteristics of bi-telecentric PMD contribute to improved measurement accuracy. However, the calibration of the extrinsic parameters of bi-telecentric lenses requires the help of a micro-positioning stage. Using a micro-positioning stage for the calibration of external parameters can result in an excessively cumbersome and time-consuming calibration process. Thus, this study proposes a holistic and flexible calibration solution for which only one flat mirror in three poses is needed. In order to obtain accurate measurement results, the calibration residuals are utilized to construct the inverse distortion map through bicubic Hermite interpolation in order to obtain accurate anchor positioning result. The calibrated stereo bi-telecentric PMD can achieve 3.5 μm (Peak-to-Valley value) accuracy within 100 mm (Width) × 100 mm (Height) × 200 mm (Depth) domain for various surfaces. This allows the obtaining of reliable measurement results without restricting the placement of the surface under test.

Subpixel water area mapping based on spatial information processing at superpixel scale for multispectral imagery

ABSTRACT Subpixel water area mapping based on spatial information processing at superpixel scale (SISS) for multispectral imagery is proposed in this paper. In SISS, the superpixels with irregular shape are first obtained by segmenting the first principal component of the improved imagery which is improved from the the original coarse multispectral imagery by using bicubic interpolation; The random walk algorithm is then introduced to calculate these superpixels to obtain the spatial information. Finally, according to this spatial information, the label of the water area or the non-water area is assigned to each subpixel by the class allocation method, producing the final mapping result. The experimental results on two datasets show that the proposed method can achieve the best performance in both visual effects and quantitative indicators.

Deep Learning Applied to Imbalanced Malware Datasets Classification

In the current day, the evolution and exponential proliferation of malware involve modifications and camouflage of their structure through techniques like obfuscation, polymorphism, metamorphism, and encryption. With the advancements in deep learning, methods such as convolutional neural networks (CNN) have emerged as potent tools for deciphering intricate patterns within this malicious software. The present research uses the capacity of CNN to learn the global structure of the code converted to an RGB or grayscale image and decipher the patterns present in the malware datasets generated from these images. The study explores fine-tuning techniques, including bicubic interpolation, ReduceLROnPlateau, and class weight estimation, in order to generalize the model and reduce the risk of overfitting for malware that uses evasion techniques against classification. Taking advantage of transfer learning and the MobileNet architecture, we created a MobileNet fine-tuning (FT) model. The application of this new model in four datasets, including Microsoft Big 2015, Malimg, MaleVis, and a new Fusion dataset, achieved 98.71%, 99.08%, 96.04%, and 98.04% accuracy, respectively, which underscores the robustness of the proposed model. The Fusion dataset is a combination of the first three datasets, consisting of a set of 32,601 known malware image files representing a mix of 59 different families. Despite the success, the study reveals performance deterioration with an increase in the number of malware families, highlighting the need for further exploration into the limits of CNNs in malware classification.

Detection of Abnormal Blood Flow Region Based on Near Infrared Correlation Spectroscopy

Blood flow measurement of microvessels in human tissues is of vital importance for the diagnosis and treatment of many diseases. In this paper, the detection method of abnormal blood flow regions based on near-infrared correlation spectroscopy is studied. We used the NL-Bregman-TV imaging algorithm to realize Blood flow imaging. However, due to the limitation of the number and distribution of detectors, the pixels obtained from images are extremely low, which cannot meet the practical requirements of the visual and the abnormal blood flow range measurement. In this paper, the bicubic interpolation method is used to improve the resolution of low-pixel blood flow images. The parameter index of the normalized similarity was proposed to help judge the effect of the interpolation method on the resolution of this kind of image. Aiming at the extraction of abnormal regions, a threshold segmentation algorithm based on the histogram difference method and a morphological processing algorithm is proposed to extract the contour of abnormal blood flow. The method proposed in this paper can be used to accurately locate and extract the clear and smooth contour of abnormal blood flow.

Generative adversarial networks with deep blind degradation powered terahertz ptychography

Ptychography is an imaging technique that uses the redundancy of information generated by the overlapping of adjacent light regions to calculate the relative phase of adjacent regions and reconstruct the image. In the terahertz domain, in order to make the ptychography technology better serve engineering applications, we propose a set of deep learning terahertz ptychography system that is easier to realize in engineering and plays an outstanding role. To address this issue, we propose to use a powerful deep blind degradation model which uses isotropic and anisotropic Gaussian kernels for random blurring, chooses the downsampling modes from nearest interpolation, bilinear interpolation, bicubic interpolation and down-up-sampling method, and introduces Gaussian noise, JPEG compression noise, and processed detector noise. Additionally, a random shuffle strategy is used to further expand the degradation space of the image. Using paired low/high resolution images generated by the deep blind degradation model, we trained a multi-layer residual network with residual scaling parameters and dense connection structure to achieve the neural network super-resolution of terahertz ptychography for the first time. We use two representative neural networks, SwinIR and RealESRGAN, to compare with our model. Experimental result shows that the proposed method achieved better accuracy and visual improvement than other terahertz ptychographic image super-resolution algorithms. Further quantitative calculation proved that our method has significant advantages in terahertz ptychographic image super-resolution, achieving a resolution of 33.09 dB on the peak signal-to-noise ratio (PSNR) index and 3.05 on the naturalness image quality estimator (NIQE) index. This efficient and engineered approach fills the gap in the improvement of terahertz ptychography by using neural networks.

Benchmarking Time-Frequency Representations of Phonocardiogram Signals for Classification of Valvular Heart Diseases Using Deep Features and Machine Learning

Heart sounds and murmur provide crucial diagnosis information for valvular heart diseases (VHD). A phonocardiogram (PCG) combined with modern digital processing techniques provides a complementary tool for clinicians. This article proposes a benchmark different time–frequency representations, which are spectograms, mel-spectograms and cochleagrams for obtaining images, in addition to the use of two interpolation techniques to improve the quality of the images, which are bicubic and Lanczos. Deep features are extracted from a pretrained model called VGG16, and for feature reduction, the Boruta algorithm is applied. To evaluate the models and obtain more precise results, nested cross-validation is used. The best results achieved in this study were for the cochleagram with 99.2% accuracy and mel-spectogram representation with the bicubic interpolation technique, which reached 99.4% accuracy, both having a support vector machine (SVM) as a classifier algorithm. Overall, this study highlights the potential of time–frequency representations of PCG signals combined with modern digital processing techniques and machine learning algorithms for accurate diagnosis of VHD.

Automated high-precision alignment of geologic maps with well orientation

The 3D reflection seismic interpretation entails the creation of structure maps of subsurface horizons across extensive areas, often spanning tens of thousands of square kilometers within sedimentary basins. To be correlated with well measurements, maps made from 3D reflection seismic are routinely assigned the correct depth at well control points. However, integrating orientation information measured from wells remains a challenge due to the absence of upscaling guidelines. We address the scale-dependent nature of geologic structure and well orientation information in subsurface mapping, particularly focusing on integrating kilometer-scale seismic interpretation with centimeter-scale well-measured dip and azimuth data. Our pioneering methodology streamlines this integration process through three essential steps: first, transforming well dip and azimuth into a gradient representation; second, smoothly extending gradient differences between the map and well within a user-defined radius of influence; and third, constructing an updated horizon map by bicubic spline interpolation to ensure automated and high-precision alignment. In addition, a structure-oriented interpolation technique is introduced to preserve faults and local structures during orientation correction. Application of the methodology to a 3D reflection seismic horizon demonstrates its effectiveness in automating the complex task of tying subsurface maps to well orientation information. This not only reduces the need for manual interventions but also introduces a new perspective on subsurface mapping. It provides a robust framework that enhances the accuracy and reliability of geologic interpretation, offering significant advancements beyond previous efforts in the literature.

Evaluation of extra pixel interpolation with mask processing for medical image segmentation with deep learning

Current mask processing operations rely on interpolation algorithms that do not produce extra pixels, such as nearest neighbor (NN) interpolation, as opposed to algorithms that do produce extra pixels, like bicubic (BIC) or bilinear (BIL) interpolation. In our previous study, the author proposed an alternative approach to NN-based mask processing and evaluated its effects on deep learning training outcomes. In this study, the author evaluated the effects of both BIC-based image and mask processing and BIC-and-NN-based image and mask processing versus NN-based image and mask processing. The evaluation revealed that the BIC-BIC model/network was an 8.9578% (with image size 256 × 256) and a 1.0496% (with image size 384 × 384) increase of the NN-NN network compared to the NN-BIC network which was an 8.3127% (with image size 256 × 256) and a 0.2887% (with image size 384 × 384) increase of the NN-NN network.

A Remote Sensing Image Super-Resolution Reconstruction Model Combining Multiple Attention Mechanisms.

Remote sensing images are characterized by high complexity, significant scale variations, and abundant details, which present challenges for existing deep learning-based super-resolution reconstruction methods. These algorithms often exhibit limited convolutional receptive fields and thus struggle to establish global contextual information, which can lead to an inadequate utilization of both global and local details and limited generalization capabilities. To address these issues, this study introduces a novel multi-branch residual hybrid attention block (MBRHAB). This innovative approach is part of a proposed super-resolution reconstruction model for remote sensing data, which incorporates various attention mechanisms to enhance performance. First, the model employs window-based multi-head self-attention to model long-range dependencies in images. A multi-branch convolution module (MBCM) is then constructed to enhance the convolutional receptive field for improved representation of global information. Convolutional attention is subsequently combined across channels and spatial dimensions to strengthen associations between different features and areas containing crucial details, thereby augmenting local semantic information. Finally, the model adopts a parallel design to enhance computational efficiency. Generalization performance was assessed using a cross-dataset approach involving two training datasets (NWPU-RESISC45 and PatternNet) and a third test dataset (UCMerced-LandUse). Experimental results confirmed that the proposed method surpassed the existing super-resolution algorithms, including Bicubic interpolation, SRCNN, ESRGAN, Real-ESRGAN, IRN, and DSSR in the metrics of PSNR and SSIM across various magnifications scales.

Super-resolution reconstruction of background-oriented Schlieren displacement field based on the convolution neural network with the gradient loss function

Abstract To refine the displacement field of the background-oriented Schlieren method, a novel super-resolution method based on deep learning has been proposed and compared with the bicubic interpolation in this study. The gradient loss functions were first introduced into the hybrid downsampled skip-connection/multi-scale model to improve the reconstruction effect. The reconstruction effects of the new loss functions were compared with that of the traditional mean square error (MSE) loss function. The results show that the Laplace operator with average pooling exhibits better performance than the origin loss function in all the indexes including peak signal-to-noise ratio, MSE, MSE of the gradient, and the maximum MSE. In these four indexes, the MSE of the gradient and the maximum MSE performed especially better than the others, where the MSE of the gradient was reduced from 3. 0× 10−05 to 3.30 × 10−05, and the maximum MSE was reduced from 0.392 to 0.360.

Not just a pretty picture: Mapping Leaf Area Index at 10 m resolution using Sentinel-2

Achieving the Global Climate Observing System goal of 10 m resolution leaf area index (LAI) maps is critical for applications related to climate adaptation, sustainable agriculture, and ecosystem monitoring. Five strategies for producing 10 m LAI maps from Sentinel-2 (S2) imagery are evaluated: i. bi-cubic interpolation of 20 m resolution S2 LAI maps from the Simplified Level 2 Prototype Processor Version 1 (SL2PV1) as currently performed by the Sentinel Applications Platform (SNAP), ii. applying SL2PV1 to S2 reflectance bands spatially downscaled to 10 m using bi-cubic interpolation (BICUBIC), iii. Applying SL2PV1 to S2 reflectance bands spatially downscaled to 10 m using Area to Point Regression Kriging (ATPRK), iv. using a recalibrated version of SL2PV1 (SL2PV2) requiring only three S2 10m bands, and iv) a novel use of the previously developed Active Learning Regularization (ALR) approach to locally approximate the SL2PV1 algorithm using only 10 m bands.Algorithms were assessed in terms of per-pixel accuracy and spatial metrics when comparing 10 m LAI maps produced using either actual S2 imagery or S2 imagery synthesized from airborne hyperspectral imagery to reference 10 m LAI maps traceable to in-situ fiducial reference measurements at 10 sites across the continental US. ATPRK and ALR algorithms had the lowest precision error of ∼0.15 LAI, compared to 0.19 LAI for SNAP and BICUBIC and 0.35 LAI for SL2PV2, and ranked highest in terms of local correlation and Structural Similarity Index measure as well as qualitative agreement with reference maps. SL2PV2 LAI showed evidence of saturation over forests related to decreased sensitivity of input visible reflectance. All algorithms had a similar uncertainty of ∼0.55 LAI compared to traceable reference maps, due to the trade-off between bias and precision. However, ATPRK and ALR uncertainty reduced to 0.11 LAI and 0.16 LAI, respectively, when compared to reference maps that ignored canopy clumping. These results suggest that both ATPRK and ALR are suitable for producing 10 m S2 LAI maps assuming bias due to local clumping can be corrected in the underlying SL2PV1 algorithm.

Super-resolution of digital rock images with hybrid attention multi-branch neural network

High-resolution digital rock images are vital in studying microstructure and fluid flow characterization of oil and gas reservoirs. Computed tomography (CT) is currently the primary tool to acquire digital rock images because of its non-destructiveness and three-dimensional imaging capability. However, the restriction of imaging capability of CT sets a compromise between image resolution and field of view (FOV). The Hybrid Attention Multi-branch Super-Resolution Network (HAMSR) proposed in this study aims to remedy this deficiency. HAMSR employs a limited discrete distribution to capture all possible high-order information and generate reliable high-resolution images. The network is built to prioritize the recovery of pores, cracks, and textures in digital rock images by introducing channel and spatial attention mechanisms. The structure together with modules of HAMSR have been designed and optimized to yield superior performance with fewer parameters and less training time. HAMSR and several contemporary advanced models are trained on the DeepRock-SR 2D dataset which contains lots of sandstone and carbonate images. It is evident that HAMSR-generated images have minimal noise, clearer edges, and more micropores. Quantitative calculations show that compared to traditional bicubic interpolation, HAMSR acquires a 2.669 dB and 1.376 dB increase in peak signal-to-noise ratio (PSNR) respectively on the test sets (27%–38% reduction in relative error). Furthermore, the high-resolution (HR) images reconstructed by HAMSR are generally consistent with real HR images concerning statistical characteristics such as porosity, pore size distribution, shape factor distribution, and two-point correlation function. This work provides a new super-resolution model to generate high-quality digital rock images for further analysis.

Image interpolation with spiking neural network based pixel similarity

Image interpolation is an important topic in the field of image processing. It is defined as the process of transforming low-resolution images into high-resolution ones using image processing methods. Recent studies on interpolation have shown that researchers are focusing on successful interpolation techniques that preserve edge information. Therefore, the edge detection phase plays a vital role in interpolation studies. However, these approaches typically rely on gradient-based linear computations for edge detection. On the other hand, non-linear structures that effectively simulate the human visual system have gained attention. In this study, a non-linear method was developed to detect edge information using a pixel similarity approach. Pixel similarity-based edge detection approach offers both lower computational complexity and more successful interpolation results compared to gradient-based approaches. 1D cubic interpolation was applied to the pixels identified as edges based on pixel similarity, while bicubic interpolation was applied to the remaining pixels. The algorithm was tested on 12 commonly used images and compared with various interpolation techniques. The results were evaluated using metrics such as SSIM and PSNR, as well as visual assessment. The experimental findings clearly demonstrated that the proposed method outperformed other approaches. Additionally, the method offers significant advantages, such as not requiring any parameters and having competitive computational cost.

SR-RDFAN-LOG: Arbitrary-scale logging image super-resolution reconstruction based on residual dense feature aggregation

In the process of oil and gas exploration and development, ultrasonic logging imaging technology visually displays the lithology, structure, fractures, pores, and other characteristics of the wellbore. This technology has been widely applied in the exploration and development of oil and gas resources. However, the low circumferential sampling rate negatively affects the imaging quality and resolution during drilling logging. To address the issues of low resolution and blurriness in ultrasonic logging images caused by these factors, as well as the limitation of existing super-resolution reconstruction algorithms that can only reconstruct fixed scales, this paper presents an Image Super-Resolution Reconstruction Network called Residual Dense Feature Aggregation (SR-RDFAN-LOG), which is specifically designed to improve ultrasonic logging image resolution. The proposed network consists of two stages, including encoding and decoding. In the encoding stage, a local feature extraction module is initially designed to integrate contextual features. Subsequently, a residual dense module is introduced for extracting deep features. Finally, a spatial attention module is constructed to focus on high-frequency details. In the decoding stage, a multilayer perceptron is used to decode the feature maps and achieve super-resolution reconstruction at arbitrary scales. The method is tested on an ultrasonic logging images test set with in-distribution scale factors of × 2 and × 4, and out-of-distribution scale factors of × 8 and × 16. When the scaling factors are × 2 and × 4, compared to the best-performing method, the peak signal-to-noise ratio (PSNR) increased by 0.4892 dB and 0.3849 dB, respectively, the structural similarity (SSIM) values increased by 0.0002 and 0.0012, respectively, and the learned perceptual image patch similarity (LPIPS) decreased by 0.0001 and 0.0008, respectively. Compared to the bicubic interpolation method, the LPIPS decreased by 0.0001 and 0.0008 at scale factors of × 8 and × 16, PSNR increased by 1.9614 dB and 2.1002 dB, respectively, and SSIM values increased by 0.0476 and 0.033, respectively. The LPIPS values decreased by 0.2233 and 0.1070, respectively. The experimental results demonstrate that our method outperforms other mainstream methods, achieves super-resolution reconstruction at arbitrary scale, and provides support for subsequent fracture-cave evaluation and oil and gas exploration.

An Implicit Transformer-based Fusion Method for Hyperspectral and Multispectral Remote Sensing Image

There is an effective way to enhance the spatial resolution of hyperspectral remote sensing images by fusing them with multispectral remote sensing images. However, most of the existing deep fusion techniques adopt discretized explicit models to approximate the complex continuous nonlinear mapping in the fusion process, leading to limitations in enhancing the fidelity of spatial details. Additionally, existing algorithms commonly utilize discrete methods such as bilinear or bicubic interpolation during the hyperspectral upsampling process, leading to the loss of crucial spatial-spectral features. To this end, this study proposes a novel Implicit Transformer Fusion Generative Adversarial Network (ITF-GAN), which incorporates the continuity perception mechanism of implicit neural representation with the powerful self-attention mechanism of the Transformer architecture, which uses point-to-point implicit functions aiming to efficiently process information in both spatial and spectral dimensions. Besides, a guided implicit neural sampling module is introduced in the hyperspectral image up-sampling process to enhance the coordinated expression of features in the spatial and spectral domains, which improves the spatial resolution and spectral fidelity of the fused image during the upsampling process. A series of fusion experiments including 4x, 8x, and 16x scale factors have shown that ITF-GAN has significant advantages over current popular fusion algorithms in both objective evaluation indicators and subjective visual evaluation.

Deformation-encoding Deep Learning Transformer for High-Frame-Rate Cardiac Cine MRI.

Purpose To develop a deep learning model for increasing cardiac cine frame rate while maintaining spatial resolution and scan time. Materials and Methods A transformer-based model was trained and tested on a retrospective sample of cine images from 5840 patients (mean age, 55 years ± 19 [SD]; 3527 male patients) referred for clinical cardiac MRI from 2003 to 2021 at nine centers; images were acquired using 1.5- and 3-T scanners from three vendors. Data from three centers were used for training and testing (4:1 ratio). The remaining data were used for external testing. Cines with downsampled frame rates were restored using linear, bicubic, and model-based interpolation. The root mean square error between interpolated and original cine images was modeled using ordinary least squares regression. In a prospective study of 49 participants referred for clinical cardiac MRI (mean age, 56 years ± 13; 25 male participants) and 12 healthy participants (mean age, 51 years ± 16; eight male participants), the model was applied to cines acquired at 25 frames per second (fps), thereby doubling the frame rate, and these interpolated cines were compared with actual 50-fps cines. The preference of two readers based on perceived temporal smoothness and image quality was evaluated using a noninferiority margin of 10%. Results The model generated artifact-free interpolated images. Ordinary least squares regression analysis accounting for vendor and field strength showed lower error (P < .001) with model-based interpolation compared with linear and bicubic interpolation in internal and external test sets. The highest proportion of reader choices was "no preference" (84 of 122) between actual and interpolated 50-fps cines. The 90% CI for the difference between reader proportions favoring collected (15 of 122) and interpolated (23 of 122) high-frame-rate cines was -0.01 to 0.14, indicating noninferiority. Conclusion A transformer-based deep learning model increased cardiac cine frame rates while preserving both spatial resolution and scan time, resulting in images with quality comparable to that of images obtained at actual high frame rates. Keywords: Functional MRI, Heart, Cardiac, Deep Learning, High Frame Rate Supplemental material is available for this article. © RSNA, 2024.