Up-sampling Factor Research Articles

Learning-based and quality preserving super-resolution of noisy images

AbstractPurpose: Several applications require the super-resolution of noisy images and the preservation of geometrical and texture features. State-of-the-art super-resolution methods do not account for noise and generally enhance the output image’s artefacts (e.g., aliasing, blurring). Methods: We propose a learning-based method that accounts for the presence of noise and preserves the properties of the input image, as measured by quantitative metrics, e.g., normalised crossed correlation, normalised mean squared error, peak-signal-to-noise-ration, structural similarity feature-based similarity, universal image quality. We train our network to up-sample a low-resolution noisy image while preserving its properties. We perform our tests on the Cineca Marconi100 cluster, at the 26th position in the “top500” list. Results: The experimental results show that our method outperforms learning-based methods, has comparable results with standard methods, preserves the properties of the input image as contours, brightness, and textures, and reduces the artefacts. As average quantitative metrics, our approach has a PSNR value of 23.81 on the super-resolution of Gaussian noise images with a 2X up-sampling factor. In contrast, previous work has a PSNR value of 23.09 (standard method) and 21.78 (learning-based method). Conclusion: Our learning-based and quality-preserving super-resolution improves the high-resolution prediction of noisy images with respect to state-of-the-art methods with different noise types and up-sampling factors.

Error approximation and bias correction in dynamic problems using a recurrent neural network/finite element hybrid model

This work proposes a hybrid modeling framework based on recurrent neural networks (RNNs) and the finite element (FE) method to approximate model discrepancies in time-dependent, multi-fidelity problems, and use the trained hybrid models to perform bias correction of the low-fidelity models. The hybrid model uses FE basis functions as a spatial basis and RNNs for the approximation of the time dependencies of the FE basis' degrees of freedom. The training data sets consist of sparse, non-uniformly sampled snapshots of the discrepancy function, pre-computed from trajectory data of low- and high-fidelity dynamic FE models. To account for data sparsity and prevent overfitting, data upsampling and local weighting factors are employed, to instigate a trade-off between physically conforming model behavior and neural network regression. The proposed hybrid modeling methodology is showcased in three highly non-trivial engineering test-cases, all featuring transient FE models, namely, heat diffusion out of a heat sink, eddy-currents in a quadrupole magnet, and sound wave propagation in a cavity. The results show that the proposed hybrid model is capable of approximating model discrepancies to a high degree of accuracy and accordingly correct low-fidelity models.

Learn from orientation prior for radiograph super-resolution: Orientation operator transformer

Background and objective: High-resolution radiographic images play a pivotal role in the early diagnosis and treatment of skeletal muscle-related diseases. It is promising to enhance image quality by introducing single-image super-resolution (SISR) model into the radiology image field. However, the conventional image pipeline, which can learn a mixed mapping between SR and denoising from the color space and inter-pixel patterns, poses a particular challenge for radiographic images with limited pattern features. To address this issue, this paper introduces a novel approach: Orientation Operator Transformer - O2former. Methods: We incorporate an orientation operator in the encoder to enhance sensitivity to denoising mapping and to integrate orientation prior. Furthermore, we propose a multi-scale feature fusion strategy to amalgamate features captured by different receptive fields with the directional prior, thereby providing a more effective latent representation for the decoder. Based on these innovative components, we propose a transformer-based SISR model, i.e., O2former, specifically designed for radiographic images. Results: The experimental results demonstrate that our method achieves the best or second-best performance in the objective metrics compared with the competitors at ×4 upsampling factor. For qualitative, more objective details are observed to be recovered. Conclusions: In this study, we propose a novel framework called O2former for radiological image super-resolution tasks, which improves the reconstruction model's performance by introducing an orientation operator and multi-scale feature fusion strategy. Our approach is promising to further promote the radiographic image enhancement field.

Lightweight DB-YOLO Facemask Intelligent Detection and Android Application Based on Bidirectional Weighted Feature Fusion

Conventional facemask detection algorithms face challenges of insufficient accuracy, large model size, and slow computation speed, limiting their deployment in real-world scenarios, especially on edge devices. Aiming at addressing these issues, we proposed a DB-YOLO facemask intelligent detection algorithm, which is a lightweight solution that leverages bidirectional weighted feature fusion. Our method is built on the YOLOv5 algorithm model, replacing the original YOLOv5 backbone network with the lightweight ShuffleNetv2 to reduce parameters and computational requirements. Additionally, we integrated BiFPN as the feature fusion layer, enhancing the model’s detection capability for objects of various scales. Furthermore, we employed a CARAFE lightweight upsampling factor to improve the model’s perception of details and small-sized objects and the EIOU loss function to expedite model convergence. We validated the effectiveness of our proposed method through experiments conducted on the Pascal VOC2007+2012 and Face_Mask datasets. Our experimental results demonstrate that the DB-YOLO model boasts a compact size of approximately 1.92 M. It achieves average precision values of 70.1% and 93.5% on the Pascal VOC2007+2012 and Face_Mask datasets, respectively, showcasing a 2.3% improvement in average precision compared to the original YOLOv5s. Furthermore, the model’s size is reduced by 85.8%. We also successfully deployed the model on Android devices using the NCNN framework, achieving a detection speed of up to 33 frames per second. Compared to lightweight algorithm models like YOLOv5n, YOLOv4-Tiny, and YOLOv3-Tiny, DB-YOLO not only reduces the model’s size but also effectively improves detection accuracy, exhibiting excellent practicality and promotional value on edge devices.

Machine Learning Enhancement of Flow Cytometry Data Accelerates the Identification of Minimal Residual Acute Myeloid Leukemia

Background: Identification of minimal residual disease (MRD) in acute myeloid leukemia (AML) is essential for assessing risk of relapse and guiding therapeutic management. While molecular assays have streamlined AML MRD detection, flow cytometric immunophenotyping (FCI) remains the only available test modality for approximately 50% of patients lacking traceable molecular MRD targets. Unfortunately, due to the high level of expertise required, long manual analysis time and complexity of AML MRD FCI data, only a few reference laboratories offer this as an orderable test. Methods: To facilitate the analysis of AML MRD by FCI, we devised an unsupervised machine learning (ML) pipeline that intakes raw FCI data, eliminates acquisition errors (FlowCut), performs state-of-the-art clustering (PARC) and dimensionality reduction (UMAP), estimates how aberrant or different-from-normal each cluster is compared to a cohort of controls (15 donor and 15 reactive marrow aspirates), downsamples clusters with high event counts without significant loss of low-event clusters, and corrects for downsampling using an upsampling factor. This CCADDAS (Clinically-intended Clustering of All events, Dimensionality reduction, Downsampling and Aberrancy Scaling) pipeline was containerized, deployed in a cloud environment (Google Cloud Vertex AI), and used to process raw FCI files from 50 AML MRD FCI assays (3-tube/10-color panel) with 0.1% to <5% leukemic subsets, and 20 assays with no detectable leukemic subsets by prior conventional analysis. Machine learning-enhanced data files were analyzed using a commercially available FCI analysis software (Kaluza v 2.1, Beckman Coulter). Results: Cluster-informed downsampling reduced the total number of events needed to be analyzed from 1 million cells to 179,531 cells per tube on average (82% data reduction), while preserving low event subsets (clusters smaller than 5000 cells) and allowing for accurate quantitative estimates for any gated population [# events x mean (upsampling factor)]. MRD was detected in all positive cases, and absent on all negative cases when analyzed using the enhanced downsampled data (100% correlation with original data), with accurate quantitative estimates (Spearman r = 0.85). Analysis time was dramatically reduced from 73 minutes to 8 minutes per case on average (89% reduction of manual analysis time). Dimensionality reduction parameters simplified gating of lineage- and maturation-defined subsets. Cluster comparison against control samples facilitated the rapid identification of MRD subsets, based on a calculated aberrancy scale parameter that helped distinguish MRD blasts (mean ± 2SD: 0.85 ± 0.3) from benign blasts (mean ± 2SD: 0.26 ± 0.3) (p<0.0001). Conclusion: We introduce CCADDAS, an unsupervised containerized cloud ML pipeline that enhances raw AML-MRD FCI data with ML annotations, cluster-informed downsampling and comparison to normal controls to create markedly smaller and software-agnostic FCI files. CCADDAS simplifies and accelerates detection of AML-MRD in clinical diagnostics, reducing number of cells analyzed by 82% and manual analysis time by 89%, without impacting test performance. Moreover, CCADDAS can be deployed on any laboratory-developed AML-MRD assay, and its small-sized export is compatible with any clinical FCI software, computing platform and analysis strategy. Adoption of CCADDAS is likely to facilitate the implementation of AML MRD FCI analysis by more clinical laboratories.

Applications of Deep Learning-Based Super-Resolution Networks for AMSR2 Arctic Sea Ice Images

Studies have indicated that the decrease in the extent of Arctic sea ice in recent years has had a significant impact on the Arctic ecosystem and global climate. In order to understand the evolution of sea ice, it is becoming increasingly imperative to have continuous observations of Arctic-wide sea ice with high spatial resolution. Passive microwave sensors have the benefit of being less susceptible to weather, wider coverage, and higher temporal resolution. However, it is challenging to retrieve accurate parameters of sea ice due to the low spatial resolution of passive microwave images. Therefore, improving the spatial resolution of passive microwave images is beneficial for reducing the uncertainty of sea ice parameters. In this paper, four competitive multi-image super-resolution (MISR) networks are selected to explore the applicability of the networks on multi-frequency Advanced Microwave Scanning Radiometer 2 (AMSR2) images of Arctic sea ice. The upsampling factor is set to 4 in the experiment. Firstly, the optimal input lengths of the image sequence for the four MISR networks are found, and then the best network on different frequency band images is further identified. Furthermore, some factors, including seasons, sea ice motion, and polarization mode of images, that may affect the super-resolution (SR) results are analyzed. The experimental results indicate that utilizing images from winter yields superior SR results. Conversely, SR results are the worst during summer across all four MISR networks, exhibiting the largest difference in PSNR of 4.48 dB. Additionally, the SR performance is observed to be better for images with smaller magnitudes of sea ice motion compared to those with larger motions, with the maximum PSNR difference of 2.04 dB. Finally, the SR results for vertically polarized images surpass those for horizontally polarized images, showcasing an average advantage of 4.02 dB in PSNR and 0.0061 in SSIM. In summary, valuable suggestions for selecting MISR models for passive microwave images of Arctic sea ice at different frequency bands are offered in this paper. Additionally, the quantification of the various impact factors on SR performance is also discussed in this paper, which provides insights into optimizing MISR algorithms for passive microwave sea ice imagery.

Depth Map Super-Resolution Reconstruction Based on Multi-Channel Progressive Attention Fusion Network

Depth maps captured by traditional consumer-grade depth cameras are often noisy and low-resolution. Especially when upsampling low-resolution depth maps with large upsampling factors, the resulting depth maps tend to suffer from vague edges. To address these issues, we propose a multi-channel progressive attention fusion network that utilizes a pyramid structure to progressively recover high-resolution depth maps. The inputs of the network are the low-resolution depth image and its corresponding color image. The color image is used as prior information in this network to fill in the missing high-frequency information of the depth image. Then, an attention-based multi-branch feature fusion module is employed to mitigate the texture replication issue caused by incorrect guidance from the color image and inconsistencies between the color image and the depth map. This module restores the HR depth map by effectively integrating the information from both inputs. Extensive experimental results demonstrate that our proposed method outperforms existing methods.

Super-resolution of 2D ultrasound images and videos

This paper proposes a novel deep-learning framework for super-resolution ultrasound images and videos in terms of spatial resolution and line reconstruction. To this end, we up-sample the acquired low-resolution image through a vision-based interpolation method; then, we train a learning-based model to improve the quality of the up-sampling. We qualitatively and quantitatively test our model on different anatomical districts (e.g., cardiac, obstetric) images and with different up-sampling resolutions (i.e., 2X, 4X). Our method improves the PSNR median value with respect to SOTA methods of 1.7% on obstetric 2X raw images, 6.1% on cardiac 2X raw images, and 4.4% on abdominal raw 4X images; it also improves the number of pixels with a low prediction error of 9.0% on obstetric 4X raw images, 5.2% on cardiac 4X raw images, and 6.2% on abdominal 4X raw images. The proposed method is then applied to the spatial super-resolution of 2D videos, by optimising the sampling of lines acquired by the probe in terms of the acquisition frequency. Our method specialises trained networks to predict the high-resolution target through the design of the network architecture and the loss function, taking into account the anatomical district and the up-sampling factor and exploiting a large ultrasound data set. The use of deep learning on large data sets overcomes the limitations of vision-based algorithms that are general and do not encode the characteristics of the data. Furthermore, the data set can be enriched with images selected by medical experts to further specialise the individual networks. Through learning and high-performance computing, the proposed super-resolution is specialised to different anatomical districts by training multiple networks. Furthermore, the computational demand is shifted to centralised hardware resources with a real-time execution of the network’s prediction on local devices.Graphical abstract

RefinePose: Towards More Refined Human Pose Estimation

Human pose estimation is a very important research topic in computer vision and attracts more and more researchers. Recently, ViTPose based on heatmap representation refreshed the state of the art for pose estimation methods. However, we find that ViTPose still has room for improvement in our experiments. On the one hand, the PatchEmbedding module of ViTPose uses a convolutional layer with a stride of 14 × 14 to downsample the input image, resulting in the loss of a significant amount of feature information. On the other hand, the two decoding methods (Classical Decoder and Simple Decoder) used by ViTPose are not refined enough: transpose convolution in the Classical Decoder produces the inherent chessboard effect; the upsampling factor in the Simple Decoder is too large, resulting in the blurry heatmap. To this end, we propose a novel pose estimation method based on ViTPose, termed RefinePose. In RefinePose, we design the GradualEmbedding module and Fusion Decoder, respectively, to solve the above problems. More specifically, the GradualEmbedding module only downsamples the image to 1/2 of the original size in each downsampling stage, and it reduces the input image to a fixed size (16 × 112 in ViTPose) through multiple downsampling stages. At the same time, we fuse the outputs of max pooling layers and convolutional layers in each downsampling stage, which retains more meaningful feature information. In the decoding stage, the Fusion Decoder designed by us combines bilinear interpolation with max unpooling layers, and gradually upsamples the feature maps to restore the predicted heatmap. In addition, we also design the FeatureAggregation module to aggregate features after sampling (upsampling and downsampling). We validate the RefinePose on the COCO dataset. The experiments show that RefinePose has achieved better performance than ViTPose.

Multi-scale deformable transformer for multi-contrast knee MRI super-resolution

Background and objective:Magnetic resonance imaging can present the precise anatomic structure in clinical applications. Nevertheless, due to the limited scanning equipment cost, scanning time and so on, high-resolution knee MR images are difficult to obtain. So the super-resolution technique is developed to improve the image quality. Unfortunately, conventional CNN-based methods cannot explicitly learn the long-range dependencies in images and simply integrate the auxiliary contrast without considering the characteristics of medical images. To tackle this issue, our approach aims to adaptively capture and fuse the significant auxiliary information of the multi-contrast images to improve the knee magnetic resonance image quality. Methods:We propose a multi-scale deformable transformer network (MSDT) for multi-contrast knee magnetic resonance imaging super-resolution. First, we aggregate multi-scale patch embedding from the multi-contrast knee MR images to effectively preserve the local contextual details and global structure information. Then, the deformable transformer architecture is designed to learn the data-dependent sparse attention of the knee MR image, which can adaptively obtain the high-frequency foreground details according to the image content. Results:The proposed method is evaluated on the fastMRI dataset under 2× and 4× enlargements. Our MSDT achieves higher PSNR of 31.98 and SSIM of 0.713 at 2× upsampling factor and PSNR of 30.38 and SSIM of 0.615 at 4× upsampling factor. Moreover, our method can generate clear tissue structures and fine details. Conclusions:The experimental results show superior performance in comparison to the state-of-the-art super-resolution methods. This indicates that the MSDT can effectively reconstruct the high-quality knee MR images.

Multimodal biomicroscopic system for the characterization of cells with high spatial phase sensitivity and sub-pixel accuracy.

Multimodal analysis is highly advantageous for various biomedical applications including cancer and brain studies. Simultaneous measurement of quantitative phase with sub-pixel accuracy and fluorescence image is difficult to achieve in single measurement. Conventionally, off-axis interferograms are analyzed using the Fourier-transform method which limits the accuracy of the phase maps by pixel size, and usually the location of the carrier peak is in sub-pixel. We report a multimodal microscopic system consisting of high-resolution (HR) quantitative phase interferometer to retrieve sub-pixel accuracy in phase imaging and an oblique-illumination-based fluorescence imaging system which decouples the excited light from emitted signal light to avoid saturation of the camera, both integrated into a single unit. Here, highly resolved phase maps are obtained using a two-step process. First, using a speckle-free illumination which offers high spatial phase sensitivity. Second, using a hamming window for accurate estimation of original signal frequency information and HR discrete Fourier transform (DFT) which offers sub-pixel accuracy in phase measurements. HR-DFT has computational load of , where is the size of the interferogram and is the upsampling factor, making system computationally more robust and efficient compared to zero-padded FFT. The experiment is conducted on MG63 osteosarcoma and human mesenchymal stem cells (hMSCs) and their quantitative parameters are extracted with significantly improved accuracy. The average phase for MG63 cells and hMSCs, for nucleus is obtained to be 8.02 rad ± 0.80 rad and 4.29 rad ± 0.43 rad, respectively, and for cytoplasm is obtained to be 2.63 rad ± 0.96 rad and 1.73 rad ± 0.57 rad, respectively.

Improvement of Computational Efficiency for Fast ISAR Image Simulation Through Nonuniform Fast Fourier Transform

The image-domain ray-tube integration formula (IRIF) is an efficient way to build a large database for synthetic-aperture radar automatic target recognition (SAR-ATR) because the IRIF can rapidly generate an ISAR image of a computer-aided design (CAD) model. Additionally, calculation of the IRIF can be accelerated by a nonuniformly sampled impulse train and the nonuniform fast Fourier transform (NUFFT). In a numerical implementation, upsampled ISAR images are required and the IRIF can calculate the upsampled ISAR images directly in image-domain. However, image-domain upsampling increases the computation time because the NUFFT calculates a matrix whose size is increased by the upsampling factor. In this letter, we propose a method to minimize the increase in computation time caused by image-domain upsampling. We derive a formula to approximately predict the scattered far-field under the small-angle condition and accelerate the formula by utilizing the nonuniformly sampled impulse train and the NUFFT. The NUFFT is used to calculate a far-field matrix without upsampling. Then, zero-padding in the field-domain and the FFT are utilized to generate an upsampled ISAR image. To validate the proposed method, ISAR images of a tank and a helicopter CAD models were simulated and were compared with those of the previous methods.

Sub-pixel high-resolution imaging of high-energy x-rays inspired by sub-wavelength optical imaging.

We have developed and demonstrated an image super-resolution method-XR-UNLOC: X-Ray UNsupervised particle LOCalization-for hard x-rays measured with fast-frame-rate detectors that is an adaptation of the principle of photo-activated localization microscopy (PALM) and stochastic optical reconstruction microscopy (STORM), which enabled biological fluorescence imaging at sub-optical-wavelength scales. We demonstrate the approach on experimental coherent Bragg diffraction data measured with 52 keV x-rays from a nanocrystalline sample. From this sample, we resolve the fine fringe detail of a high-energy x-ray Bragg coherent diffraction pattern to an upsampling factor of 16 of the native pixel pitch of 30 μm of a charge-integrating fastCCD detector. This was accomplished by analysis of individual photon locations in a series of "nearly-dark" instances of the diffraction pattern that each contain only a handful of photons. Central to our approach was the adaptation of the UNLOC photon fitting routine for PALM/STORM to the hard x-ray regime to handle much smaller point spread functions, which required a different statistical test for photon detection and for sub-pixel localization. A comparison to a photon-localization strategy used in the x-ray community ("droplet analysis") showed that XR-UNLOC provides significant improvement in super-resolution. We also developed a metric by which to estimate the limit of reliable upsampling with XR-UNLOC under a given set of experimental conditions in terms of the signal-to-noise ratio of a photon detection event and the size of the point spread function for guiding future x-ray experiments in many disciplines where detector pixelation limits must be overcome.

Fast Ewald summation for electrostatic potentials with arbitrary periodicity.

A unified treatment for the fast and spectrally accurate evaluation of electrostatic potentials with periodic boundary conditions in any or none of the three spatial dimensions is presented. Ewald decomposition is used to split the problem into real-space and Fourier-space parts, and the Fast Fourier Transform (FFT)-based Spectral Ewald (SE) method is used to accelerate computation of the latter, yielding the total runtime O(N⁡log(N)) for N sources. A key component is a new FFT-based solution technique for the free-space Poisson problem. The computational cost is further reduced by a new adaptive FFT for the doubly and singly periodic cases, allowing for different local upsampling factors. The SE method is most efficient in the triply periodic case where the cost of computing FFTs is the lowest, whereas the rest of the algorithm is essentially independent of periodicity. We show that removing periodic boundary conditions from one or two directions out of three will only moderately increase the total runtime, and in the free-space case, the runtime is around four times that of the triply periodic case. The Gaussian window function previously used in the SE method is compared with a new piecewise polynomial approximation of the Kaiser-Bessel window, which further reduces the runtime. We present error estimates and a parameter selection scheme for all parameters of the method, including a new estimate for the shape parameter of the Kaiser-Bessel window. Finally, we consider methods for force computation and compare the runtime of the SE method with that of the fast multipole method.

Random-Walk Laplacian for Frequency Analysis in Periodic Graphs.

This paper presents the benefits of using the random-walk normalized Laplacian matrix as a graph-shift operator and defines the frequencies of a graph by the eigenvalues of this matrix. A criterion to order these frequencies is proposed based on the Euclidean distance between a graph signal and its shifted version with the transition matrix as shift operator. Further, the frequencies of a periodic graph built through the repeated concatenation of a basic graph are studied. We show that when a graph is replicated, the graph frequency domain is interpolated by an upsampling factor equal to the number of replicas of the basic graph, similarly to the effect of zero-padding in digital signal processing.

Optical Single Sideband Signal Reconstruction Based on Time-Domain Iteration

Due to its low cost, simple architecture and robustness to fiber dispersion, single sideband (SSB) transmission with direct detection (DD) system is an attractive solution for 80-km inter data center interconnects (DCIs). However, it will suffer performance penalty caused by the signal-to-signal beating interference (SSBI). Kramers-Kronig (KK) receiver has been extensively investigated for SSBI elimination by reconstructing the SSB signal. The nonlinear operations in KK algorithm require up-sampling to cope with spectral broadening, which results in high complexity for practical application. Optical signal phase retrieval method based on the minimum phase signal has also been investigated for SSB signal recovery, in which the SSB and DC-Value properties are iteratively imposed on the amplitude signal in frequency domain. In this paper, we propose a low complexity iterative algorithm for minimum phase signal recovery without up-sampling in time domain. Finite impulse response (FIR) filter is applied to iteratively generate the SSB signal and update the phase component. Based on the proposed scheme, the transmission of 30 GHz SSB 16-QAM discrete multitone (DMT) signal over 80 km single mode fiber (SMF) is successfully demonstrated with the bit error rate (BER) below the hard-decision forward error correction (HD-FEC) threshold of 3.8 × 10−3. The experimental results show that, the BER performance of KK scheme with up-sampling factor of 2, frequency-domain iterative scheme and our proposed scheme is almost the same. However, compared with the KK scheme, the proposed method can save the numbers of adders and multipliers by the factors of 29 and 7, while the factors are 5.5 and 4 comparing to the frequency-domain iteration scheme.

Deep Magnification-Flexible Upsampling Over 3D Point Clouds.

This paper addresses the problem of generating dense point clouds from given sparse point clouds to model the underlying geometric structures of objects/scenes. To tackle this challenging issue, we propose a novel end-to-end learning-based framework. Specifically, by taking advantage of the linear approximation theorem, we first formulate the problem explicitly, which boils down to determining the interpolation weights and high-order approximation errors. Then, we design a lightweight neural network to adaptively learn unified and sorted interpolation weights as well as the high-order refinements, by analyzing the local geometry of the input point cloud. The proposed method can be interpreted by the explicit formulation, and thus is more memory-efficient than existing ones. In sharp contrast to the existing methods that work only for a pre-defined and fixed upsampling factor, the proposed framework only requires a single neural network with one-time training to handle various upsampling factors within a typical range, which is highly desired in real-world applications. In addition, we propose a simple yet effective training strategy to drive such a flexible ability. In addition, our method can handle non-uniformly distributed and noisy data well. Extensive experiments on both synthetic and real-world data demonstrate the superiority of the proposed method over state-of-the-art methods both quantitatively and qualitatively. The code will be publicly available at https://github.com/ninaqy/Flexible-PU.

Efficient subpixel image registration algorithm for high precision visual vibrometry

As one of the most reliable motion estimation algorithms that apply phase correlation methods, the single-step DFT (SSDFT) approach has superior characteristics, including the high accuracy and low complexity. However, this approach is limited by the accuracy of the initial estimation. Therefore, it is an enormous challenge to reduce the dimension of the searching area in the subsequent refinement step. As a result, this algorithm is inefficient with large upsampling factors. In order to overcome this problem, an improved two-step image registration algorithm is proposed in the present study. In the first step, the accuracy of the initial estimation is improved by using a motion amplified cross-correlation function. The improved initial estimation is then amended to ensure that retained error is eliminated. In the refinement step, the dimension of the searching area is effectively reduced in accordance with the improved initial estimation and upsampling factor. Obtained results show that for large upsampling factors, the modified SSDFT achieves the same subpixel accuracy as the original algorithm. Meanwhile, it is found that the modified scheme remarkably reduces the computational expense. Finally, conducted experiments on high-speed video sequences demonstrate that the proposed modifications significantly reduce the required time for high precision target tracking tasks.

MDCN: Multi-Scale Dense Cross Network for Image Super-Resolution

Convolutional neural networks have been proven to be of great benefit for single-image super-resolution (SISR). However, previous works do not make full use of multi-scale features and ignore the inter-scale correlation between different upsampling factors, resulting in sub-optimal performance. Instead of blindly increasing the depth of the network, we are committed to mining image features and learning the inter-scale correlation between different upsampling factors. To achieve this, we propose a Multi-scale Dense Cross Network (MDCN), which achieves great performance with fewer parameters and less execution time. MDCN consists of multi-scale dense cross blocks (MDCBs), hierarchical feature distillation block (HFDB), and dynamic reconstruction block (DRB). Among them, MDCB aims to detect multi-scale features and maximize the use of image features flow at different scales, HFDB focuses on adaptively recalibrate channel-wise feature responses to achieve feature distillation, and DRB attempts to reconstruct SR images with different upsampling factors in a single model. It is worth noting that all these modules can run independently. It means that these modules can be selectively plugged into any CNN model to improve model performance. Extensive experiments show that MDCN achieves competitive results in SISR, especially in the reconstruction task with multiple upsampling factors. The code is provided at https://github.com/MIVRC/MDCN-PyTorch .

Enhanced 3D-CAP modulation with information-inserted time slots for seven-core fiber transmissions.

An enhanced three-dimensional carrier-less amplitude phase (3D-CAP) modulation with information-inserted time slots is proposed in this paper. Compared with the traditional 3D-CAP, the proposed scheme can reduce the value of the up-sampling factor by loading information on the time slots position, thus achieving better bit error rate (BER) performance under the same bit rate. An experiment demonstrating 43.75 Gb/s MIMO-free transmission employing the proposed enhanced 3D-CAP modulation over 2 km weakly coupled 7-core fiber is successfully carried out. The experimental results show that the difference between the best and worst core is about 1.7 dB in receiver sensitivity at a BER of 1 × 10-3 due to the geometrical reason, and the proposed scheme outperforms the traditional 3D-CAP-16 by 2.7 dB in core-1.